Dealing with digital mess: Local (offline) File Storage

If you have digital camera or digital camcorder, you probably have a problem where to store the pictures and video so that they are readily available for viewing and protected against loss caused by hardware failure. How much storage you need depends on how many and how large pictures do you take and how heavy camcorder user you are. In my case, I have switched to shooting RAW on my D70, doing about 2000-4000 shots a year and the camcorder produces about 4-5 GB unprocessed video per hour of recording. After adding some free capacity for the document archives, some MP3 files, anything smaller than 500 GB will be probably too small. To have some space, I decided I better go for 1 Terrabyte of space.

The main advantage of the local disk storage is speed. Regardless how fast your Internet connection is, it is nothing like 1 GB Ethernet, when it comes to to speedy access to your files. You would of course get most speed if the data were stored on local disks of the machine which uses them, but who uses only one machine :-). To separate data storage into dedicated box offers easier sharing and freedom to reinstall OS on the workstation without impacting data availability. Therefore, I want NAS.

Another important feature of the local storage is safety. To protect yourself against data loss, you want to do at minimum RAID storage – likely RAID-5. I was avoding RAID for some time – and as result ended up with many USB disks laying around, lots of cables, power adapters and very complicated cascade backup process, which need to be simplified.

The basic question about NAS is to build or to buy. Ideally, I would like to have an appliance, not another computer. There is interesting site on the Net – Small Network Builders which offers lot of information about commercial NAS solutions available. Unfortunately, those devices which were in my desired price range (~ $900 to $1100) for TB storage (like Buffalo Home Server), had serious issues either with read/write performance which was at the level of 100 MB network speed on 1GB LAN – or some multi platform compatibility problems. Another potential problem with some NAS boxes was noise. Running four high-performance hard disk drives inside small enclosure requires good cooling and often makes the NAS noisiest part of your office. My goal is to lower the noise, not increase it. An example of NAS with good performance is the Thecus N5200 – but you have to accept the noise. A system that promises quiet operation was Synology CS-406, but in both cases the enclosure (without disks) price tag was in the range $700-$900. If you assume price of the 5 disks about $600-$700, it is impossible to build 1TB in assumed budget.

All commercial NAS devices are computer running usually Linux or BSD Unix with Web based user interface for disk management. The speed of the box is determined (except speed of the disks) by the NIC adapter speed, CPU speed (and amount of RAM) and the overhead of the used OS. Disadvantage of appliance NAS is little control about these and about features supported. With building custom appliance, you have full control and for $800 you can build very decent computer these days.

I have decided to go with larger case – to allow more space around hard drives, better cooling and quieter fans. I asked Yan to recommend really good mainboard with fast enough CPU – but not too fast to need lots of noisy cooling and enough RAM to have space for buffers. Here is configuration I have ended up with – purchased from my favorite hardware supplier:

– AMD 64 3500+ AM2 Processor
– Asus M2NPV-VM AM2 Nf430 mATX MB (Raid5/GbE/Gf6150/VGA/DVI-D)
– 1GB PC2-5300 DDR2 Memory
– Lite-On LH-18A1P-185 DVD-RW 18×18
– Antec Atlas Server Case w/True 500w Power Supply
– 2xSilenX 92mm 14dBA 2000rpm 36cfm Case Fan

Alltogether the price for enclosure was $546. If you add five 320 GB SATAII Seagate HDD, 16MB you will end up with 1.2 TB NAS (900 GB after RAID) which is (hopefully) more powerful, quiter and at least 300-400 cheaper than the pre-built alternatives. The machine which does only fileserver is about 3-4 times faster than my current desktop from early 2002 (which did cost about 2.5K$ at that time). Oh boy …

Technically I did not need buy 5 hard drives for RAID-5, only 4. I just picked up one spare to make sure that when it comes to replacement, I have it available. I believe that having same disks has value and if the increase in HDD capacities will continue with current speed, the 320 GB disks may be unavailable soon.

As for software, I have not decided yet. The best candidate is the FreeNAS. It is not Linux based, but Free BSD. I have no problem with that, quite the opposite – thanks to Macbook, I have very good relationship with FreeBSD Unix now :-).

The machine is completed and running in my office now. It is amazingly quiet – very deep below the level of noise of the other computers. I have not installed OS yet, only booted FreeNAS from the CD. Time to read FreeNAS installation guide I guess.

Will be continued

Advertisements
Explore posts in the same categories: gadgets

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: