======OS - NAS - FreeNAS - Miscellaneous======
[[https://www.freenas.org/|FreeNAS]] \\
[[https://www.freenas.org/freenas-certified-servers/|FreeNAS certified servers]] \\
[[https://www.ixsystems.com/documentation/freenas/|FreeNAS Documentation]] \\
[[https://www.ixsystems.com/community/|iX Community Forums]] \\
[[https://www.ixsystems.com/blog/hardware-guide/|The Official FreeNAS Hardware Guide]] \\
[[https://www.freebsd.org/releases/|FreeBSD Releases]] for links to hardware notes for the supported hardware lists that apply to FreeNAS as well (FreeNAS 11.3 is based on FreeBSD 11.3).
[[https://www.broadcom.com/products/storage/host-bus-adapters|Broadcom - SAS/SATA/NVMe Host Bus Adapters]]. The LSI 9200 and 9300 line are often used (in IT-mode as a true HBA) with FreeNAS.
=====Blogposts/Articles=====
[[https://serverfault.com/questions/84043/zfs-sas-sata-controller-recommendations|Server Fault - ZFS SAS/SATA controller recommendations]] \\
[[http://blog.zorinaq.com/from-32-to-2-ports-ideal-satasas-controllers-for-zfs-linux-md-ra/|mrb's blog - From 32 to 2 ports: Ideal SATA/SAS Controllers for ZFS & Linux MD RAID]] \\
[[https://www.servethehome.com/buyers-guides/top-hardware-components-freenas-nas-servers/top-picks-freenas-hbas/|STH - Top Picks for FreeNAS HBAs (Host Bus Adapters)]] \\
[[https://www.servethehome.com/buyers-guides/|STH - Buyer’s Guides]] for FreeNAS, OmniOS, and other buying guides. \\
[[https://www.ixsystems.com/community/threads/dont-be-afraid-to-be-sas-sy-a-primer-on-basic-sas-and-sata.26145/|iX Community Forum - Don't be afraid to be SAS-sy ... a primer on basic SAS and SATA]] \\
[[https://www.servethehome.com/sas-sata-cables-guide-sff8087-8088-8470-8482-8484-single-device-connectors/|STH - SAS/ SATA Cables Guide – SFF-8087, 8088, 8470, 8482, 8484, and single device connectors]] \\
[[wp>Serial Attached SCSI]] \\
[[https://forums.servethehome.com/index.php?threads/lsi-raid-controller-and-hba-complete-listing-plus-oem-models.599/|STH - LSI RAID Controller and HBA Complete Listing Plus OEM Models]] \\
[[https://www.servethehome.com/lsi-sas-2008-raid-controller-hba-information/|STH - LSI SAS 2008 RAID Controller/ HBA Information]] \\
[[https://www.ixsystems.com/community/resources/detailed-newcomers-guide-to-crossflashing-lsi-9211-9300-9305-9311-hba-and-variants.54/|iX Community Forums - Detailed newcomers' guide to crossflashing LSI 9211/9300/9305/9311 HBA and variants]] \\
=====Notes=====
====LSI adapter list====
* LSI SAS 3081E (LSI SAS1068E, PCIe 1.0, 3Gbps, FreeBSD mpt driver)
* IBM ServeRAID M1015 (LSI SAS 2008, PCIe 2.0, 6 Gbps SAS, FreeBSD mps driver)
* LSI SAS 9211-8i / LSI SAS 9210-8i(-OEM) (LSI SAS 2008, PCIe 2.0, 6 Gbps SAS, FreeBSD mps driver)
* LSI SAS 9207-8i (LSI SAS 2308, PCIe 3.0, 6 Gbps SAS, FreeBSD mps driver)
* LSI SAS 9300-8i (LSI SAS 3008, PCIe 3.0, 12 Gbps SAS, FreeBSD mpr driver)
* LSI SAS 9305-16i (LSI SAS 3224, PCIe 3.0, 12 Gbps SAS, FreeBSD mpr driver)
* LSI SAS 9305-24i (LSI SAS 3224, PCIe 3.0, 12 Gbps SAS, FreeBSD mpr driver)
* LSI/BROADCOM 9400-8i (SAS 3408, PCIe 3.1 x8, 12 Gbps SAS, FreeBSD mpr driver)
====On NVMe's====
gamebrigada
Wed Nov 28 2018 23:25:46 GMT +0100
This is a bad idea in a lot of ways. Don't do it.
ZFS bottlenecks at the CPU and Memory in NVME pools. It's simply not fast enough to take advantage of those
drives. Falls flat on it's face with just 1 drive. You get little to no additional performance by spanning
more of them. For example I run 2 systems, 1 with 4 Toshiba's spanned, 1 with 6 Samsung PRO drives spanned.
Performance is identical, even though the Samsungs are far superior in performance. With just 6 Sata/SAS
SSD's on a decent LSI controller will match that performance and cost far less money.
Scaling is a huge problem with NVME pools. You are literally limited by power, PCIe slots on motherboards,
and support for bifurcation. Getting a big pool involves an AMD Epyc/Threadripper CPU. If you are willing
to spend the cash, you can pick up a pre-built that uses an NVME backplane for U.2 drives. You will go
broke buying these.
You mentioned this is for streaming. You will literally get 0 benefit of doing an SSD pool in general...
A single 3.5" spinning rust drive will keep up with that demand, and you can them in 10TB enterprise grade
flavors for around 300$. Want more? Add more, and the performance and capacity will easily scale for a
small amount of money.
Can I make a recommendation? Pick up a FreeNAS mini
Source: [[https://www.reddit.com/r/freenas/comments/a069cy/can_i_use_all_pcie_nvme_m2_ssd_for_a_nas/|reddit - Can I use all PCIe NVMe m.2 SSD for a NAS?]]
====On the LSI 9400 series====
jgreco
Dec 29, 2019
>geronimo said:
>You can switch the personality of the controller in order to make it act as HBA as it supports
>HBA/JBOD/RAID modes.
No, you cannot switch the personality of the controller to make it act as a true HBA - not interested in
arguing the point, it's just a fact. It might be an "'HBA'" but it isn't the HBA we need. It is a
hardware RAID controller, and there is not an option to omit the hardware RAID data path. Even if there
were, there is still the issue of driver support. There are billions of driver-hours of FreeNAS on the
LSI 6Gbps HBA 20.00.07.00 in IT mode and all the bugs are shook out. There's a lesser number of hours on
the 12Gbps HBA products but they're known to work with only a caveat or two. These things do have drivers
for FreeBSD, but they're really oriented towards typical "run a program, write a datafile" style usage.
ZFS can pound continuously on your controller for days doing a scrub or resilver. The driver can't be
99.999%. It really needs to be as close to 100.000000% perfect as possible. The 92xx and 93xx controllers
are definitely known to exhibit issues under ZFS-class loads.
You really need the true HBA product and it needs to be in the software-RAID-bludgeoned-out IT mode. If
"storcli" runs you have the wrong product. I am sorry that Avago has chosen to mislead people by trying to
market this card as a multimode card. It's quite possible that at some point the driver support will
actually be up to snuff, but ZFS is *incredibly* demanding. It will pummel your typical RAID controller
CPU and cache. It will stress the driver to the breaking point with massive I/O operations.
It's not that this might not be able to work at some point some day. Ideally it *ought* to. However, you
really do not want to be the guinea pig for a problematic new controller that no one else is using on
FreeNAS at this time. It is hazardous to be the guy with the unusual controller. Lots of users here have
learned that the hard way. :-(
Source: [[https://www.ixsystems.com/community/threads/lsi-megaraid-9460-16i-with-physical-drives-in-jbod-mode.80070/|iX Community Forums - LSI MegaRAID 9460-16i with physical drives in JBOD mode]]