Ask Your Question
2

Best Fedora based backup server?

asked 2016-06-27 17:53:23 -0500

centinel20 gravatar image

updated 2016-06-29 14:04:05 -0500

I built a little home server which i installed fedora23 server with a btrfs filesystem and six 1tb hd on a virtual RAID6 array, I have 4tb of useful space but will use most on other things. I have virtualized a few servers and want one of them to be a Backup server. Was looking into amanda, clonezilla and deja-dup. I am inclining to amanda but read here on askfedora that dejadup is more native to fedora. Is that true? are they similar? I want to be able to make remote backups from windows and linux systems, if possible even android phones. Also was thinking of doing the backups on a 3tb external usb 3.0 drive, or should i use the internal server memory?(I am less inclined to use the server internal hard drives even if it is btrfs and raid, I understand i shouldn't backup in the same disks that I have most of my data in right?). Also I'm interested in knowing which application will optimize memory by only backing up parts of the disks that are used and not all of the disk image, or is this unavoidable? also encryption would be nice.

Thanks for the help.

edit retag flag offensive close merge delete

2 Answers

Sort by » oldest newest most voted
4

answered 2016-06-28 11:37:47 -0500

danieljrmay gravatar image

updated 2016-06-29 12:19:00 -0500

I don't know that I can answer all of your questions but I'm going to try and make a start which others can hopefully build on…

Backup can get a bit complicated. The best solution for you depends on your particular circumstances. People have written whole books on the subject, here's one that I found very useful: Backup & Recovery by W. Curtis Preston.

However, the most important thing to remember is that pretty much any kind of backup is better than no backup at all! So don't obsess over planning the perfect backup solution without getting round to actually doing something. Better to do something really quick & simple right away — then look to improve on it.

Here's what I would recommend as a starting point given my understanding of your situation.

  • Yes, use your external USB 3.0 drive for backup. While there is nothing wrong with having a backup on your internal drives as one of your layers of backup, you definitley would not want it to be your only backup. Your internal drives are always connected to your computer and so would be an example of On-line storage. In contrast your external USB drive could be an example of Off-line storage i.e. disconnected from your computer most of the time and only connected when you are performing backups. While less convienient, Off-line storage is less vunerable to things like accidental deletion & malware.
  • Create an off-site, off-line backup: Label your external USB drive something like "My_Backup_A" and buy a second extetnal drive and label it something like "My_Backup_B". Set these drives up the same and keep one of them at home and take the other off-site e.g. friends or parents house, a draw at work, etc. Swap them over regularly, so you always have a reasonably fresh backup off-site.
  • Encryption can be taken care of by creating one or more LUKS encrypted partitions on your external drives. You can do this easily with the gnome-disks graphical utility in Fedora.
  • In order to support the backup of windows machines you could set up your backup server as a iSCSI SAN target or SAMBA server. Each windows machine can then mount a partition or directory on your backup server and use it as the storage location for its own native windows Backup and Restore client. This method would work for Macs too, allowing them to use their native "Time Machine" backup software.
  • If you want to be space efficient then it sounds like you want to use a backup utility which is capable of file-based, incremental backups. Both amanda and deja-dup do this, as well as my personal favourite rdiff-backup.

    I haven't used amanda, but my understanding is that it is very mature and powerful — capable of backing up entire networks. It uses standard tools like tar underneath. It has a native windows client so it could handle your windows machines without needing to resort to the SAN/SAMBA method I mentioned above ...

(more)
edit flag offensive delete link more

Comments

1

wow that was pretty complete. Thank you. I will wait for a while to see if anyone else has anything else to add but that gets me going. What happens next, should I mark this as a correct answer?

centinel20 gravatar imagecentinel20 ( 2016-06-29 00:10:23 -0500 )edit

You're welcome. Judging by the Ask fedora guidelines and [Sticky] Why so many unsolved questions?, you should mark the answer as correct if it was helpful to you.

If and when other answers come in you can up/down vote them according to their level of helpfulness!

danieljrmay gravatar imagedanieljrmay ( 2016-06-29 05:00:26 -0500 )edit
1

Yes a very fulfilling answer. Thanks

sigurdsk gravatar imagesigurdsk ( 2016-10-07 04:31:15 -0500 )edit
1

answered 2016-07-04 19:27:03 -0500

cmurf gravatar image

updated 2016-07-04 20:02:46 -0500

Design the backup so that if the backup storage completely implodes totally inexplicably and without warning and has no chance of recovery, doesn't make you a sad panda. That might mean there is a primary "active" backup and then a secondary backup or archive that uses separate storage devices. The main idea of RAID is to avoid downtime in the face of a drive failure, not as a way to avoid a second backup copy of everything important.

Also I'm not totally clear what you mean by virtual RAID 6. Is this Btrfs native raid6? Or is it Btrfs on an mdadm or LVM based RAID 6? The former, Btrfs raid56, isn't production ready and comes with numerous expert level caveats. Using mdadm or LVM RAID 6 is stable. Btrfs requires quite a bit of familiarity if you run into problems, the normal sequence of recovery is different on Btrfs. So you need to be pretty familiar with this, have backups, and you might even hedge your bets and make either the first or second backup on a different file system, such as XFS which at least by default now checksums metadata.

And yes, if you were to create two backups by splitting the same member devices among them, that means if a drive dies, both backups are now degraded. It's not ideal. In my own case, I keep a totally segregated 3rd backup that's mostly offline/shelved except when it's being updated in which case either the primary or secondary is made offline. That way even in the face of user error (very common vector for data loss), one of the storage stacks should survive even my mistakes.

ADDITION: Also, I suggest reading this thread from linux-raid@ list which mainly applies to md based arrays, but is applicable for LVM, mdadm, and Btrfs arrays. The gist is to make certain that SCT ERC timeout is shorter than the kernel SCSI command timer. On consumer drives, SCT ERC is disabled or unsupported, and that means it's not easy to figure out what the drive timeout is for bad sectors. The estimate for the high end is 180 seconds, so that means for drives not supporting SCT ERC, the kernel command timer needs to be raised to 180 seconds for all block devices used in the array. If SCT ERC is disabled, then it needs to be enabled with a sane timeout value such as 70 deciseconds using 'smartctl -l scterc,70,70 <dev>' and that is per drive. Both of these are not persistent settings: SCT ERC resets if the drive itself is reset or powered off; and the kernel command timer goes back to default of 30 seconds if you reboot.

http://marc.info/?l=linux-raid&m=1336...

edit flag offensive delete link more

Comments

The SCT ERC timeout is very important with software raids. Good point.

I want to also highlight that btrfs raid56 has currently some serious bugs which will eat your data. Do not use it for anything else than testing.

jorti gravatar imagejorti ( 2016-07-05 02:44:58 -0500 )edit

Question Tools

3 followers

Stats

Asked: 2016-06-27 17:53:23 -0500

Seen: 1,373 times

Last updated: Jul 04 '16