Yet another documentary-ish post.

My own requirements:

• >=8TB to consolidate all of my drives and media on
• At least RAID5 level reliability (maybe one hot spare, if the case has the space)
• Live deduplication
• Easily expandable
• Selective backup/sync to S3
• Watched directories (i.e. /torrents, where I just save the file and let the server download or help seed, etc)
• Network failover (and, potentially, link aggregation)
• The ability to use it with Time Machine (CIFS, here)
• A way to tell the condition of the array at a glance, without having to connect the server to a monitor or log into it (got a case with a case LED for this one)

Most of the setup is covered in mschenck’s tutorial at http://www.tek-ops.com/archives/350.

This will end up being a CIFS and NFS server, however I’ll need to figure out the schema for content, as deduplication can be set on the dataset level (see http://blogs.sun.com/bonwick/entry/zfs_dedup ) .

I.e.:
I probably have drives that I’m going to copy to the array, that have almost all of the same photos on them,  but when dealing with photos, I probably want to verify the blocks, as you don’t get a second chance to take the same photo, so the tank/photo dataset will be set dedup=verify.

HOWEVER, I have something similar with my (non-itunes) music collection…and don’t much care if every block is absolutely the same - I can almost always go back to one of my old hard drives and get it off, or rip the CD, OR (worst case scenario) just buy it on iTunes so the not-so-friendly RIAA can be absolutely sure that I have the requisite rights to have the song.  dedup=on in this case, as with random tv/movies.

Also, in the process, I get to reduce my movie and tv collection to only what is /not/ on hulu and/or netflix.

Is there ever any reason not to turn on deduplication?