Sunday, September 27, 2009

Re: [BLUG] storage options

Lord Drachenblut wrote:
> On Sunday 27 September 2009 3:37:10 am Joe Auty wrote:
>> Hey Guys,
>>
>>
>> I'm looking for some perspectives on the storage options available
>> today, and on the horizon...
>>
>>
>> Specifically, I'm thinking about something that would be a little
>> handier and more future proof for my servers I run. Specifically, I run
>> VMWare Server on Linux and several virtual machines all on the same
>> hardware.
>>
>> There are several different variables and technologies at this point:
>>
>> - direct attach w. software or hardware RAID
>>
>> - file systems such as ZFS and BTRFS and the cool stuff you can do with
>> them
>>
>> - NAS
>>
>> - external RAID arrays
>>
>> - iSCSI
>>
>> - getting higher quality disks such as SCSI and solid state drives
>>
>>
>>
>> In a perfect world, here is what I'd like:
>>
>>
>> - A storage device not tethered to my servers so that I can add more
>> storage as needed without planning for downtime and running dd, and
>> upgrade server hardware without having to reinvest in storage.
>>
>> - Fast I/O suitable for running VMs. There are tricks that can be
>> employed which will help one get by with SATA, but you still have to be
>> careful not to do heavy disk operations on the host as to render the
>> guests unresponsive. This is a nuisance, and of course makes
>> hypothetically backing up complete VM disk images difficult.
>>
>> - Disk redundancy, i.e. some sort of RAID configuration.
>>
>> - Some sort of solution somewhere between Cletus the Slackjaw Yokel's
>> Windows XP box and high level enterprise stuff. I like the idea of
>> spreading out the I/O demand across several cheap disks, adding more
>> disk as needed, replacing failed disks as necessary. I don't need crazy
>> fast I/O, I just need something a little more than a single SATA disk.
>> So, enterprise level technology is not what I'm after here, just
>> convenience and decent performance.
>>
>> - Something that will be relatively future proof and not cost me a
>> fortune :)
>>
>> - I'm using nearly 250 gig right now so my capacity needs are not
>> tremendous, but it would cost me a lot more than what I'm paying now to
>> run within a VPS provider such as Linode or Slicehost, and I'd rather
>> not get into the position where my costs grow significantly just to add
>> a modest amount of disk space. My storage needs don't warrant an
>> expensive SAN such as the ones that are no doubt in use by these
>> providers, one of my VMs is running Windows, and I like the control I
>> have now and I like working with my server provider. So, I'm not really
>> keen on moving my eight servers to VMs provided by one of these companies
>>
>> - A way to backup all of this data (snapshots), preferably via the same
>> overall design so that I have something I can test with and perhaps even
>> fail over to in the event of an emergency. I know that offsite
>> redundancy is the golden egg for many companies, but hey, this is a
>> perfect world type wishlist! :) I use Amazon S3 for an offsite backup in
>> addition to my current backup to backup my most important data, but to
>> keep my costs to a minimum I'd prefer to just stay with rsync to
>> multiple cheap disks.
>>
>>
>> I'm interested in learning more about iSCSI, and am fascinated with
>> BTRFS and ZFS. Do any of you have any experience with any of this, have
>> any general recommendations, thoughts, predictions, anything? I don't
>> need to buy anything tomorrow, I'm just thinking that it would be nice
>> to think ahead a little.
>>
>> It seems like I'm kind of stuck in between not needing to invest heavily
>> in storage like a big company would, yet I'm pushing the limits of
>> run-of-the-mill consumer grade direct attach SATA type stuff - the kind
>> of solution that would be great for using with Time Machine to store
>> pictures of your kids.
>>
>
> You might try looking into using freenas. it can do alot of the things you
> are looking for and has support for zfs and iscsi at this point.
>

Thanks for the idea!

I've looked at FreeNAS in the past, and it is an interesting option and
seemingly useful to me since I use FreeBSD a lot, but what about the
hardware end of things? What would host the disks that FreeNAS would
manage and make available?

>
> ------------------------------------------------------------------------
>
> _______________________________________________
> BLUG mailing list
> BLUG@linuxfan.com
> http://mailman.cs.indiana.edu/mailman/listinfo/blug


--
Joe Auty
NetMusician: web publishing software for musicians
http://www.netmusician.org
joe@netmusician.org
_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

No comments: