FreeNAS 9.10 – Getting down and dirty with DIY NAS

The ability to consume copious amounts of media, when combined wanting to protect critical data, has developed a considerable market for network-attached storage devices and appliances.  Many different tech sites have provided benchmarks and recommendations based upon the outcomes of various tests.  The reality is that most of the leading consumer offerings do so much more than just “storage”.  Synology, for example, provides app containers via Docker, and has introduced support for btrfs.  The additional layer of protection against digital bit rot and potential corruption using legacy file systems can be mitigated with this file system.  Other vendors excluding Netgear may not have a clear road map that defines when they’ll introduce support for next-generation file systems.

The only other primary alternative to commercial, off-the-shelf solutions pertains to building your own NAS.  Depending on the feature set and level of expertise one has with available software and hardware platforms, it is within the realm of reality that a solution can be constructed with a superior performance profile and a feature list that meets or exceeds the boxed solutions.  Prior experience with FreeNAS 8.x on a low cost platform that had limited disk expansion capability was excellent from the administration and functionality perspectives, but the hardware lacked the necessary compute.  Furthermore, the single NIC offered in this previous test environment was known to have issues.

In following along with the latest updates and releases, this felt like the right time to dive back into FreeNAS using some cost-effective new components, along with some components that were freed up from servers we’ve retired last year.  Before we committed to this endeavor and the construction of our solution, extensive reading, scouring, and researching through the FreeNAS community forums helped bring us up to speed with some of the nuances related to system design and optimizing ZFS performance.  The sheer body of work put together by active participants was incredibly helpful and ensures that newcomers avoid potential mistakes that can cost them their data.

Ultimately, our build used the components listed below.  Cost optimization can be achieved using search engines, coupon codes, eBay or forums where selling/trading is permitted.

Enclosure:  SuperMicro CSE-825TQ-563LPB

This rack-mount enclosure includes almost everything you’ll need for installation if it’s paired with a SuperMicro motherboard.  While this unit only comes with a single 560W power supply, it will be more than enough to address a solution which uses all eight hot-swap drive bays.  In addition to ensuring disks are easily removable in the event of a drive failure, this unit also provides 2 non-removable traditional 3.5″ slots for drives.  Overall space within this enclosure is ample when the SuperMicro cables are used to connect the required HBA to the back plane.  Four cables of varying lengths are included in the box.  The thin design ensures they route cleanly through and minimizes the amount of slack.  Another matching set of cables runs approximately $30 USD.  If you want things clean, don’t be cheap.  Make this investment.

Motherboard: SuperMicro MBD-X10SL7-F-O

As we had previously noted, Haswell-based offerings provide an incredible bang for the buck.  While newer Skylake offerings are supported for FreeNAS, we erred on the side of caution and went with a proven platform that has an essential bonus that will keep costs and cooling considerations under control.  The majority of the configuration of this board is standard fare for a server-class solution: 4 DDR3 memory slots, Socket 1150, dual Intel i210 gigabit NICs, and a dedicated port for out of band management.  The bonus here is the inclusion of an integrated LSI Logic 2308 HBA.  This controller, when flashed with the appropriate firmware, provides for trouble-free performance within FreeNAS.  Concerns raised about the operating temperature of a dedicated HBA in a non-server platform are effectively mitigated with our selected combination.  The SAS controller’s heatsink resides under the adjustable air shroud that’s included with the enclosure we’ve selected.  The exceptional airflow provided by the removable (and fairly quiet) 80mm fans addresses operational concerns.  You can buy a SAS HBA for close to the price of this motherboard.  For us – getting the motherboard with the HBA baked in was a no-brainer.

Memory: 32GB Crucial ECC DDR3-1600 DRAM

This was one of the items that we had available from a prior server.  It’s on the compatibility list for the motherboard, all modules were already tested and certified as good, and reusing this memory resulted in reducing the amount of e-waste we’re generating.

CPU: Intel Xeon E3-1225 v3

Another one of the items that we had on-hand.  If you’re erring on the side of caution or plan on doing “things” above and beyond storage that will benefit from hyperthreading, the E3-1230 v3 will be the target entry point for a Xeon E3 CPU.  It’s also possible to go downstream with a Pentium or Core i3, both of which support ECC memory.

Install target: (2) SanDisk Cruzer Fit 16GB USB Flash Drives

Form factor was the biggest consideration for our selection of this specific model.  There is 1 USB 3.0 header on the motherboard itself that can be used in conjunction with these incredibly low-profile flash drives.  The ability to create a mirror during install ensures that the least expensive component for this solution isn’t the single point of failure.

Hard Drives: (4) Seagate Constellation ES.3 Enterprise Capacity Drives

The premium for these units versus more recent and NAS-focused models has been reduced over the past few weeks.  Multiple sources are now selling these for ~$170 USD per drive.  The 5 year warranty, 7200 RPM spindle speed, 128MB cache, and available performance metrics will facilitate meeting the target performance profile for the disk layout we’ll be using.  Research does state that 5400 RPM NAS drives will still provide enough performance for most use cases while running cooler and consuming less power.  Our luck with Western Digital Red drives from two different NAS units has been less then stellar.  Shipping five out of eight back due to unexpected early life failures was an experience that we do not want to repeat this time.

A few notes from our experience in constructing this solution:

1.) SuperMicro’s Motherboard-to-Chassis relationship information on their website isn’t fully accurate nor all encompassing.  If you look up the 825TQ-563LPB, you’ll note that the motherboard we used isn’t listed as being designed for the case.  With the removal of an adjustable plate in the chassis and a quick relocation of some of the mounts, this board fit beautifully.  Apparently, we’re not the only ones to see the potential for this combination.

2.) The SuperMicro SATA Cable Set (P/N: CBL-0180L-01) should be purchased with the enclosure.  One kit is included with the chassis, and the second kit will resulted in saved time, connections being made with the optimum length, and risks of thick cables impeding the re-installation of cooling fans will be eliminated.

3.) Ensure the necessary firmware has been applied to all hard drives and/or SSDs that will be used in the solution.  While we may take for granted that enterprise-class solutions are capable of applying disk firmware updates while remaining attached to a RAID controller or HBA, this benefit does not exist in the consumer space.  For the solution we’ve designed, the hard drives came with the FN04 firmware, and the most recent version is FN06.  Transplantation of the hard drives from the enclosure to an available test bench was require in conjunction with customizing and modifying the vendor-provided tools.  While Seagate states that their bootable Linux USB utility includes the firmware, the reality is far from that truth.  The process we used to successfully update all four drives can be found here.

We’re currently performing the long SMART test on the drives, and will provide an update once we’ve completed the burn-in for the solution.