Re: [BLUG] RAID-like redundancy for a linux?

Linux has had software RAID for years and years. There are some idiosyncrasies, but overall it works real well. (A Google search of "software RAID Linux" should cover all the needed info.)

Due to the nature of the RAID process, the usable space is the lowest common size of the drives. (You don't need to worry about differing disk geometries these days.) If the space difference is large, you can still use the extra space as non-RAID storage.

You don't need weird/custom kernels, and the easiest way to set this up has been as an option in the installation process -- it has been available for the past decade or so. (Ubuntu Server, like Debian before it, has this as an option. I am less confident about some of the GUI install processes.)

In Linux software RAID creates a block device. You can pretty much put any filesystem you want on it.

Cheers,
Steven Black

On Oct 29, 2012 7:46 PM, "Ben Shewmaker" <ben@shewbox.org> wrote:

I have a home server (sporting a dual core AMD E-350 and 4 gigs of
ram) running Ubuntu Server 12.04 and have been really happy with it
for the last 6 months.  The one area where I am not happy is my lack
of data redundancy.  I use this as my NAS as well, using offsite
backup for my personal and irreplaceable files (I use SpiderOak) but
it would still be nice to have some redundancy locally in case of a
single drive failure.  I'm wondering if anyone could recommend any
software solutions to doing some sort of RAID-like redundancy within
Ubuntu server?  I've been reading about ZFS, which seems interesting,
but I also see that it cannot be included in Linux because it is
incompatible with the GPL.  Is ZFS worth considering?  Are there other
solutions like ZFS out there that I can use?  That is, is there some
software solution that allows creating a pool of disks that will take
care of data redundancy and be able to recover from a drive failure?
Also, does using different capacity disks make this a much harder
challenge?

Thanks!

Ben
_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

Re: [BLUG] RAID-like redundancy for a linux?

If you are interested in RAID-5/6, Ben, you should definitely look at a hardware RAID card to do this. RAID-5/6 is not safe to do in software alone, unlike RAID 1, 0, or 10 (although I would not recommend RAID 0).

I love ZFS, but I'm unsure of its future under Oracle's leadership. Ironically, it seems like if you want ZFS, short of shelling out the dough for Solaris, your best bet is FreeBSD/FreeNAS where it seems a lot of people have been using it with no reported difficulties. There is also OpenIndiana, but I'm not convinced that its backing and support is all that great to really assure me that the future of some sort of non-commercial version of ZFS running in a Solaris based OS is in good hands. With many players interested in and working on BTRFS, it could also be that ZFS is eventually phased out for BTRFS  once it is deemed production ready.

However, since you are talking about Linux, all of this FreeBSD/Solaris stuff is moot. There is no great way to get ZFS in Linux now.


I would look at software RAID 1 or 10 in Linux as has been said, RAID-5/6 if you don't mind buying a RAID card (which includes a battery to perform the last of the writes in the case of a power failure, which is what makes it "safe").

How many drive bays do you have to work with, Ben? Is performance an issue here, or are you just looking for basic storage and reliability?

David M. Moore

October 29, 2012 10:40 PM

For the last two years my home media server has run Mandriva Linux
(yes, I know, I'm the last person still using Mandriva :-) ). I have
it set up currently as a RAID-6 system with 8 2TB drives along with 2
more hot spares, all formatted ext4. The one time a drive failed,
about 1.5 years ago, it automatically shut down the bad drive, pulled
in a hot spare, and rebuilt the missing drive, all with no attention
from me.

If you have a lot of data, I always recommend RAID-6 over RAID-5. With
2TB drives, it will take 8 to 10 hours to rebuild from a drive
failure. With RAID-5, that's 8 to 10 hours where you have no
redundancy to fall back on. And if you don't have hot spares, RAID-5
leaves you with no redundancy until you happen to notice there is a
problem. I don't like either of those scenarios.

I honestly don't know if it would work with differing drives sizes or
not. It was easier, in my opinion, to just stick with the same size of
drives right out of the gate.

I don't know of any reason you couldn't do the same with Ubuntu.

David
davmoo@gmail.com
_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

Lord Drachenblut

October 29, 2012 9:12 PM

amahi has an interesting option called grey-hole.
it uses multiple drives and writes data to all drives, but each drive
is readable outside of this pool

Jeremy L. Gaddis

October 29, 2012 7:54 PM

* Ben Shewmaker <ben@shewbox.org> wrote:  

solutions like ZFS out there that I can use?  That is, is there some  software solution that allows creating a pool of disks that will take  care of data redundancy and be able to recover from a drive failure?  

  https://raid.wiki.kernel.org/index.php/Linux_Raid    

Also, does using different capacity disks make this a much harder  challenge?  

  Identical drives are best, but you can use drives of different size.    

Ben Shewmaker

October 29, 2012 7:46 PM

I have a home server (sporting a dual core AMD E-350 and 4 gigs of
ram) running Ubuntu Server 12.04 and have been really happy with it
for the last 6 months. The one area where I am not happy is my lack
of data redundancy. I use this as my NAS as well, using offsite
backup for my personal and irreplaceable files (I use SpiderOak) but
it would still be nice to have some redundancy locally in case of a
single drive failure. I'm wondering if anyone could recommend any
software solutions to doing some sort of RAID-like redundancy within
Ubuntu server? I've been reading about ZFS, which seems interesting,
but I also see that it cannot be included in Linux because it is
incompatible with the GPL. Is ZFS worth considering? Are there other
solutions like ZFS out there that I can use? That is, is there some
software solution that allows creating a pool of disks that will take
care of data redundancy and be able to recover from a drive failure?
Also, does using different capacity disks make this a much harder
challenge?

Thanks!

Ben
_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

---

--
Joe Auty, NetMusician
NetMusician helps musicians, bands and artists create beautiful, professional, custom designed, career-essential websites that are easy to maintain and to integrate with popular social networks.
www.netmusician.org
joe@netmusician.org

Re: [BLUG] RAID-like redundancy for a linux?

Ben Shewmaker <ben@shewbox.org> wrote:
>>> solutions like ZFS out there that I can use? That is, is there some
>>> software solution that allows creating a pool of disks that will take
>>> care of data redundancy and be able to recover from a drive failure?

For the last two years my home media server has run Mandriva Linux
(yes, I know, I'm the last person still using Mandriva :-) ). I have
it set up currently as a RAID-6 system with 8 2TB drives along with 2
more hot spares, all formatted ext4. The one time a drive failed,
about 1.5 years ago, it automatically shut down the bad drive, pulled
in a hot spare, and rebuilt the missing drive, all with no attention
from me.

If you have a lot of data, I always recommend RAID-6 over RAID-5. With
2TB drives, it will take 8 to 10 hours to rebuild from a drive
failure. With RAID-5, that's 8 to 10 hours where you have no
redundancy to fall back on. And if you don't have hot spares, RAID-5
leaves you with no redundancy until you happen to notice there is a
problem. I don't like either of those scenarios.

I honestly don't know if it would work with differing drives sizes or
not. It was easier, in my opinion, to just stick with the same size of
drives right out of the gate.

I don't know of any reason you couldn't do the same with Ubuntu.

David
davmoo@gmail.com
_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

Re: [BLUG] RAID-like redundancy for a linux?

amahi has an interesting option called grey-hole.
it uses multiple drives and writes data to all drives, but each drive
is readable outside of this pool

On Mon, Oct 29, 2012 at 7:54 PM, Jeremy L. Gaddis <jlgaddis@gnu.org> wrote:
> * Ben Shewmaker <ben@shewbox.org> wrote:
>> solutions like ZFS out there that I can use? That is, is there some
>> software solution that allows creating a pool of disks that will take
>> care of data redundancy and be able to recover from a drive failure?
>
> https://raid.wiki.kernel.org/index.php/Linux_Raid
>
>> Also, does using different capacity disks make this a much harder
>> challenge?
>
> Identical drives are best, but you can use drives of different size.
>
> --
> Jeremy Gaddis e: jlgaddis@gnu.org
> Network Engineer m: +1.812.865.0581
> PGP: 0x95E2C8FE w: evilrouters.net
>
> _______________________________________________
> BLUG mailing list
> BLUG@linuxfan.com
> http://mailman.cs.indiana.edu/mailman/listinfo/blug



--
Mettle not in the Ways of dragons for thou art crunchy and taste good
with ketchup
_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

Re: [BLUG] RAID-like redundancy for a linux?

* Ben Shewmaker <ben@shewbox.org> wrote:
> solutions like ZFS out there that I can use? That is, is there some
> software solution that allows creating a pool of disks that will take
> care of data redundancy and be able to recover from a drive failure?

https://raid.wiki.kernel.org/index.php/Linux_Raid

> Also, does using different capacity disks make this a much harder
> challenge?

Identical drives are best, but you can use drives of different size.

--
Jeremy Gaddis e: jlgaddis@gnu.org
Network Engineer m: +1.812.865.0581
PGP: 0x95E2C8FE w: evilrouters.net

_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

[BLUG] RAID-like redundancy for a linux?

I have a home server (sporting a dual core AMD E-350 and 4 gigs of
ram) running Ubuntu Server 12.04 and have been really happy with it
for the last 6 months. The one area where I am not happy is my lack
of data redundancy. I use this as my NAS as well, using offsite
backup for my personal and irreplaceable files (I use SpiderOak) but
it would still be nice to have some redundancy locally in case of a
single drive failure. I'm wondering if anyone could recommend any
software solutions to doing some sort of RAID-like redundancy within
Ubuntu server? I've been reading about ZFS, which seems interesting,
but I also see that it cannot be included in Linux because it is
incompatible with the GPL. Is ZFS worth considering? Are there other
solutions like ZFS out there that I can use? That is, is there some
software solution that allows creating a pool of disks that will take
care of data redundancy and be able to recover from a drive failure?
Also, does using different capacity disks make this a much harder
challenge?

Thanks!

Ben
_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug