1. Introduction — FreeNAS®11.2-RC1 User Guide Table of Contents

FreeNAS® is © 2011-2018 iXsystems

FreeNAS® and the FreeNAS® logo are registered trademarks of iXsystems

FreeBSD® is a registered trademark of the FreeBSD Foundation

Written by users of the FreeNAS® network-attached storage operating system.

Version 11.2

Copyright © 2011-2018 iXsystems

This Guide covers the installation and use of FreeNAS® 11.2.

The FreeNAS® User Guide is a work in progress and relies on the contributions of many individuals. If you are interested in helping us to improve the Guide, read the instructions in the README. IRC Freenode users are welcome to join the #freenas channel where you will find other FreeNAS® users.

The FreeNAS® User Guide is freely available for sharing and redistribution under the terms of the Creative Commons Attribution License. This means that you have permission to copy, distribute, translate, and adapt the work as long as you attribute iXsystems as the original source of the Guide.

FreeNAS® and the FreeNAS® logo are registered trademarks of iXsystems.

Active Directory® is a registered trademark or trademark of Microsoft Corporation in the United States and/or other countries.

Apple, Mac and Mac OS are trademarks of Apple Inc., registered in the U.S. and other countries.

Broadcom is a trademark of Broadcom Corporation.

Chelsio® is a registered trademark of Chelsio Communications.

Cisco® is a registered trademark or trademark of Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries.

Django® is a registered trademark of Django Software Foundation.

Facebook® is a registered trademark of Facebook Inc.

FreeBSD® and the FreeBSD® logo are registered trademarks of the FreeBSD Foundation®.

Intel, the Intel logo, Pentium Inside, and Pentium are trademarks of Intel Corporation in the U.S. and/or other countries.

LinkedIn® is a registered trademark of LinkedIn Corporation.

Linux® is a registered trademark of Linus Torvalds.

Oracle is a registered trademark of Oracle Corporation and/or its affiliates.

Twitter is a trademark of Twitter, Inc. in the United States and other countries.

UNIX® is a registered trademark of The Open Group.

VirtualBox® is a registered trademark of Oracle.

VMware® is a registered trademark of VMware, Inc.

Wikipedia® is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization.

Windows® is a registered trademark of Microsoft Corporation in the United States and other countries.

Typographic Conventions

The FreeNAS® 11.2 User Guide uses these typographic conventions:

Table 1 Text Format Examples
Item | Visual Example
--- | ---
Graphical elements: buttons, icons, fields, columns, and boxes | Click the Import CA button.
Menu selections | Select `System → Information`.
Commands | Use the scp command.
File names and volume and dataset names | Locate the `/etc/rc.conf` file.
Keyboard keys | Press the `Enter` key.
Important points | This is important.
Values entered into fields, or device names | Enter 127.0.0.1 in the address field.

# 1. Introduction

FreeNAS® is an embedded open source network-attached storage (NAS) operating system based on FreeBSD and released under a 2-clause BSD license. A NAS has an operating system optimized for file storage and sharing.

FreeNAS® provides a browser-based, graphical configuration interface. The built-in networking protocols provide storage access to multiple operating systems. A plugin system is provided for extending the built-in features by installing additional software.

## 1.1. New Features in 11.2

FreeNAS® 11.2 is a feature release, which includes several new significant features, many improvements and bug fixes to existing features, and version updates to the operating system, base applications, and drivers. Users are encouraged to Update to this release in order to take advantage of these improvements and bug fixes.

These major features are new in this version:

- The login screen defaults to the new, Angular-based UI. Users who wish to continue to use the classic UI can select “Legacy UI” in the login screen.
- Beginning with this release, the screenshots that appear in the published version of the Guide and in the Guide option within the new UI are for the new UI. However, users who click the Guide option while logged into the classic UI will continue to see screenshots for the old UI. The availability of both versions of the Guide is to assist users as they become familiar with the new UI during the transition period before the classic UI is deprecated in a future release.
- The rewrite from the old API to the new middlewared continues. Once the rewrite is complete, api.freenas.org will be deprecated and replaced by the new API documentation. In the mean time, to see the API documentation for the new middleware, log into the new UI, click on the URL for the FreeNAS system in your browser’s location bar, and add `:api/docs` to the end of that URL.
- The boot loader has changed from GRUB to the native FreeBSD boot loader. This should resolve several issues that some users experienced with GRUB. GRUB was introduced as a temporary solution until the FreeBSD boot loader had full support for boot environments, which it now has.
- The Plugins and Jails backend has switched from warden to iocage and warden will no longer receive bug fixes. The new UI will automatically use iocage to create and manage Plugins and Jails. Users are encouraged to recreate any existing Plugins and Jails using the new UI to ensure that they are running the latest supported application versions.
- Plugins have switched to FreeBSD 11.2-RELEASE and all Plugins have been rebuilt for this version.
- Virtual Machines are more crash-resistant. When a guest is started, the amount of available memory is checked and an initialization error will occur if there is insufficient system resources. When a guest is stopped, its resources are returned to the system. In addition, the UEFI boot menu fix allows Linux kernels 4.15 and higher to boot properly.
- Cloud Sync provides configuration options to encrypt data before it is transmitted and to keep it in the encrypted format while stored on the cloud. The filenames can also be encrypted.
- Preliminary support has been added for Self-Encrypting Drives (SEDs).

This software has been added or updated:

- The base operating system is the STABLE branch of FreeBSD 11.2, which brings in many updated drivers and bug fixes. This branch has been patched to include the FreeBSD security advisories up to FreeBSD-SA-18:12.elf.
- OpenZFS is up-to-date with Illumos and slightly ahead due to support for sorted scrubs which were ported from ZFS on Linux. Notable improvements include channel programs, data disk removal, more resilient volume import, the ability to import a pool with missing vdevs, pool checkpoints, improved compressed ARC performance, and ZIL batching. As part of this change, the default ZFS indirect block size is reduced to 32 KiB from 128 KiB. Note that many of these improvements need further testing so have not yet been integrated into the UI.
- The IPsec kernel module has been added. It can be manually loaded with kldload ipsec.
- Support for eMMC flash storage has been added.
- The em, igb, ixgbe, and ixl Intel drivers have been patched to resolve a performance degradation issue that occurs when the MTU is set to 9000 (9k jumbo clusters). Before configuring 9k jumbo clusters for cxgbe create a Tunables with a Variable of hw.cxgbe.largest_rx_cluster, a Type of Loader, and a Value of 4096. The cxgb driver does not support jumbo clusters and should not use an MTU greater than 4096.
- The bnxt driver has been added which provides support for Broadcom NetXtreme-C and NetXtreme-E Ethernet drivers.
- The vt terminal is now used by default and the syscons terminal is removed from the kernel.
- ncdu has been added to the base system. This CLI utility can be used to analyze disk usage from the console or an SSH session.
- drm-next-kmod has been added to the base system, adding support for UTF-8 fonts to the console for Intel graphic cards.
- Samba 4.7.6 has been patched to address the latest round of security vulnerabilities.
- Netatalk has been updated to the 3.1.12 development version which addresses known issues with Time Machine timeouts.
- rsync has been updated to version 3.1.3.
- Minio has been updated to version 2018-04-04T05.
- Netdata as been updated to version 1.10.1.
- iocage has been synced with upstream as of July 10, providing many bug fixes and improved IPv6 support.
- RancherOS has been updated to version 1.4.0.
- zsh is the root shell for new installations. Upgrades will continue to use the csh shell as the default root shell.
- xattr has been added to the base system and can be used to modify file extended attributes from the command line. Type xattr -h to view the available options.
- convmv has been added to the base system and can be used to convert the encoding of filenames from the command line. Type convmv to view the available options.
- These switches have been added to freenas-debug: -J for dumping iocage information, -M for dumping SATADOM info, and -Z to delete old debug information. The -G switch has been removed as the system no longer uses GRUB.
- These switches have been added to arcstat: -a for displaying all available statistics and -p for displaying raw numbers without suffixes.

These screen options have changed:

- The ATA Security User, SED Password, and Reset SED Password fields have been added to `System → Advanced`.
- The Enable screen saver field has been removed from `System → Advanced`.
- The Enable automatic upload of kernel crash dumps and daily telemetry checkbox has been removed from `System → Advanced`.
- Alerts has been added to System and can be used to list the available alert conditions and to configure the notification frequency on a per-alert basis.
- These Cloud Credentials have been added to `System → Cloud Credentials`: Amazon Cloud Drive, Box, Dropbox, FTP, Google Drive, HTTP, Hubic, Mega, Microsoft OneDrive, pCloud, SFTP, WebDAV, and Yandex.
- The Team Drive ID field has been added to `System → Cloud Credentials → Add` form when Google Drive is the Provider.
- The Endpoint URL has been added to `System -> Cloud Credentials -> Add Cloud Credential` but only appears when Amazon S3 is selected as the Provider. This can be used to configure a connection to another S3-compatible service, such as Wasabi.
- The Remote encryption, Filename encryption, Encryption password, and Encryption salt fields have been added to `Tasks → Cloud Sync Tasks → Add Cloud Sync`.
- The Exec field has been added to `Storage → Volumes → Create Dataset → Advanced Mode`.
- The Password for SED column has been added to `Storage → Volumes → View Disks`.
- The MSDOSFS locale drop-down menu has been added to `Storage → Import Disk`.
- The User Base and Group Base fields have been removed from `Directory Services → Active Directory → Advanced Mode`.
- The Enable home directories, Home directories, Home share name, and Home Share Time Machine fields have been removed from `Services → AFP` and the Time Machine Quota field has been removed from `Sharing → Apple (AFP) Shares`. These fields have been replaced by `Sharing → Apple (AFP) Shares → Use as home share`.
- The Umask field in `Services → TFTP` has changed to a File Permissions selector.
- Disk temperature graphs have been added to `Reporting → Disk`.

## 1.2. Path and Name Lengths

Names of files, directories, and devices are subject to some limits imposed by the FreeBSD operating system. The limits shown here are for names using plain-text characters that each occupy one byte of space. Some UTF-8 characters take more than a single byte of space, and using those characters reduces these limits proportionally. System overhead can also reduce the length of these limits by one or more bytes.

Table 1.2.1 Path and Name Lengths
Type | Maximum Length | Description
--- | --- | ---
File Paths | 1024 bytes | Total file path length (PATH_MAX). The full path includes directory separator slash characters, subdirectory names, and the name of the file itself. For example, the path `/mnt/tank/mydataset/mydirectory/myfile.txt` is 42 bytes long. Using very long file or directory names can be problematic. A complete path with long directory and file names can exceed the 1024-byte limit, preventing direct access to that file until the directory names or filename are shortened or the file is moved into a directory with a shorter total path length.
File and Directory Names | 255 bytes | Individual directory or file name length (NAME_MAX).
Mounted Filesystem Paths | 88 bytes | Mounted filesystem path length (MNAMELEN). Longer paths can prevent a device from being mounted or data from being accessible.
Device Filesystem Paths | 63 bytes | devfs(8) device path lengths (SPECNAMELEN). Longer paths can prevent a device from being created.

Note

88 bytes is equal to 88 ASCII characters. The number of characters will vary when using Unicode.

Warning

If the mounted path length for a snapshot exceeds 88 bytes the data in the snapshot will be safe but inaccessible. When the mounted path length of the snapshot is less than the 88 byte limit, the data will be accessible again.

The 88 byte limit affects automatic and manual snapshot mounts in slightly different ways:

- Automatic mount: ZFS temporarily mounts a snapshot whenever a user attempts to view or search the files within the snapshot. The mountpoint used will be in the hidden directory `.zfs/snapshot/name` within the same ZFS dataset. For example, the snapshot `mypool/dataset/snap1@snap2` is mounted at `/mnt/mypool/dataset/.zfs/snapshot/snap2/`. If the length of this path exceeds 88 bytes the snapshot will not be automatically mounted by ZFS and the snapshot contents will not be visible or searchable. This can be resolved by renaming the ZFS pool or dataset containing the snapshot to shorter names (`mypool` or `dataset`), or by shortening the second part of the snapshot name (`snap2`), so that the total mounted path length does not exceed 88 bytes. ZFS will automatically perform any necessary unmount or remount of the file system as part of the rename operation. After renaming, the snapshot data will be visible and searchable again.
- Manual mount: If the same example snapshot is mounted manually from the CLI, using mount -t zfs mypool/dataset/snap1@snap2 /mnt/mymountpoint the path `/mnt/mountpoint/` must not exceed 88 bytes, but the length of the snapshot name will be irrelevant. When renaming a manual mountpoint, any object mounted on the mountpoint must be manually unmounted (using the umount command in the CLI) before renaming the mountpoint and can be remounted afterwards.

Note

A snapshot that cannot be mounted automatically by ZFS, can still be mounted manually from the CLI using a shorter mountpoint path. This makes it possible to mount and access snapshots that cannot be accessed automatically in other ways, such as from the GUI or from features such as “File History” or “Versions”.

## 1.3. Hardware Recommendations

FreeNAS® 11.2 is based on FreeBSD 11.2 and supports the same hardware found in the FreeBSD Hardware Compatibility List. Supported processors are listed in section 2.1 amd64. FreeNAS® is only available for 64-bit processors. This architecture is called amd64 by AMD and Intel 64 by Intel.

Note

FreeNAS® boots from a GPT partition. This means that the system BIOS must be able to boot using either the legacy BIOS firmware interface or EFI.

Actual hardware requirements vary depending on the usage of the FreeNAS® system. This section provides some starter guidelines. The FreeNAS® Hardware Forum has performance tips from FreeNAS® users and is a place to post questions regarding the hardware best suited to meet specific requirements. Hardware Recommendations gives detailed recommendations for system components, with the FreeNAS® Quick Hardware Guide providing short lists of components for various configurations. Building, Burn-In, and Testing your FreeNAS® system has detailed instructions on testing new hardware.

### 1.3.1. RAM

The best way to get the most out of a FreeNAS® system is to install as much RAM as possible. More RAM allows ZFS to provide better performance. The FreeNAS® Forums provide anecdotal evidence from users on how much performance can be gained by adding more RAM.

General guidelines for RAM:

- A minimum of 8 GiB of RAM is required.

  Additional features require additional RAM, and large amounts of storage require more RAM for cache. An old, somewhat overstated guideline is 1 GiB of RAM per terabyte of disk capacity.

- To use Active Directory with many users, add an additional 2 GiB of RAM for the winbind internal cache.

- For iSCSI, install at least 16 GiB of RAM if performance is not critical, or at least 32 GiB of RAM if good performance is a requirement.

- Jails are very memory-efficient, but can still use memory that would otherwise be available for ZFS. If the system will be running many jails, or a few resource-intensive jails, adding 1 to 4 additional gigabytes of RAM can be helpful. This memory is shared by the host and will be used for ZFS when not being used by jails.

- Virtual Machines require additional RAM beyond any amounts listed here. Memory used by virtual machines is not available to the host while the VM is running, and is not included in the amounts described above. For example, a system that will be running two VMs that each need 1 GiB of RAM requires an additional 2 GiB of RAM.

- When installing FreeNAS® on a headless system, disable the shared memory settings for the video card in the BIOS.

- For ZFS deduplication, ensure the system has at least 5 GiB of RAM per terabyte of storage to be deduplicated.

If the hardware supports it, install ECC RAM. While more expensive, ECC RAM is highly recommended as it prevents in-flight corruption of data before the error-correcting properties of ZFS come into play, thus providing consistency for the checksumming and parity calculations performed by ZFS. If your data is important, use ECC RAM. This Case Study describes the risks associated with memory corruption.

Do not use FreeNAS® to store data without at least 8 GiB of RAM. Many users expect FreeNAS® to function with less memory, just at reduced performance. The bottom line is that these minimums are based on feedback from many users. Requests for help in the forums or IRC are sometimes ignored when the installed system does not have at least 8 GiB of RAM because of the abundance of information that FreeNAS® may not behave properly with less memory.

### 1.3.2. The Operating System Device

The FreeNAS® operating system is installed to at least one device that is separate from the storage disks. The device can be a SSD, USB memory stick, or DOM (Disk on Module). Installation to a hard drive is discouraged as that drive is then not available for data storage.

Note

To write the installation file to a USB stick, two USB ports are needed, each with an inserted USB device. One USB stick contains the installer, while the other USB stick is the destination for the FreeNAS® installation. Be careful to select the correct USB device for the FreeNAS® installation. FreeNAS® cannot be installed onto the same device that contains the installer. After installation, remove the installer USB stick. It might also be necessary to adjust the BIOS configuration to boot from the new FreeNAS® boot device.

When determining the type and size of the target device where FreeNAS® is to be installed, keep these points in mind:

- The absolute bare minimum size is 8 GiB. That does not provide much room. The recommended minimum is 16 GiB. This provides room for the operating system and several boot environments created by updates. More space provides room for more boot environments and 32 GiB or more is preferred.

- SSDs (Solid State Disks) are fast and reliable, and make very good FreeNAS® operating system devices. Their one disadvantage is that they require a disk connection which might be needed for storage disks.

  Even a relatively large SSD (120 or 128 GiB) is useful as a boot device. While it might appear that the unused space is wasted, that space is instead used internally by the SSD for wear leveling. This makes the SSD last longer and provides greater reliability.

- When planning to add your own boot environments, budget about 1 GiB of storage per boot environment. Consider deleting older boot environments after making sure they are no longer needed. Boot environments can be created and deleted using `System → Boot`.

- Use quality, name-brand USB sticks, as ZFS will quickly reveal errors on cheap, poorly-made sticks.

- For a more reliable boot disk, use two identical devices and select them both during the installation. This will create a mirrored boot device.

Note

Current versions of FreeNAS® run directly from the operating system device. Early versions of FreeNAS® ran from RAM, but that has not been the case for years.

### 1.3.3. Storage Disks and Controllers

The Disk section of the FreeBSD Hardware List lists the supported disk controllers. In addition, support for 3ware 6 Gbps RAID controllers has been added along with the CLI utility tw_cli for managing 3ware RAID controllers.

FreeNAS® supports hot pluggable drives. Using this feature requires enabling AHCI in the BIOS.

Reliable disk alerting and immediate reporting of a failed drive can be obtained by using an HBA such as an Broadcom MegaRAID controller or a 3Ware twa-compatible controller.

Note

Upgrading the firmware of Broadcom SAS HBAs to the latest version is recommended.

Some Highpoint RAID controllers do not support pass-through of S.M.A.R.T. data or other disk information, potentially including disk serial numbers. It is best to use a different disk controller with FreeNAS®.

Note

The system is configured to prefer the mrsas(4) driver for controller cards like the Dell PERC H330 and H730 which are supported by several drivers. Although not recommended, the mfi(4) driver can be used instead by removing the loader Tunable: `hw.mfi.mrsas_enable` or setting the Value to 0.

Suggestions for testing disks before adding them to a RAID array can be found in this forum post. Additionally, badblocks is installed with FreeNAS® for testing disks.

If the budget allows optimization of the disk subsystem, consider the read/write needs and RAID requirements:

- For steady, non-contiguous writes, use disks with low seek times. Examples are 10K or 15K SAS drives which cost about $1/GiB. An example configuration would be six 600 GiB 15K SAS drives in a RAID 10 which would yield 1.8 TiB of usable space, or eight 600 GiB 15K SAS drives in a RAID 10 which would yield 2.4 TiB of usable space.

For ZFS, Disk Space Requirements for ZFS Storage Pools recommends a minimum of 16 GiB of disk space. Due to the way that ZFS creates swap, it is not possible to format less than 3 GiB of space with ZFS. However, on a drive that is below the minimum recommended size, a fair amount of storage space is lost to swap: for example, on a 4 GiB drive, 2 GiB will be reserved for swap.

Users new to ZFS who are purchasing hardware should read through ZFS Storage Pools Recommendations first.

ZFS vdevs, groups of disks that act like a single device, can be created using disks of different sizes. However, the capacity available on each disk is limited to the same capacity as the smallest disk in the group. For example, a vdev with one 2 TiB and two 4 TiB disks will only be able to use 2 TiB of space on each disk. In general, use disks that are the same size for the best space usage and performance.

The ZFS Drive Size and Cost Comparison spreadsheet is available to compare usable space provided by different quantities and sizes of disks.

### 1.3.4. Network Interfaces

The Ethernet section of the FreeBSD Hardware Notes indicates which interfaces are supported by each driver. While many interfaces are supported, FreeNAS® users have seen the best performance from Intel and Chelsio interfaces, so consider these brands when purchasing a new NIC. Realtek cards often perform poorly under CPU load as interfaces with these chipsets do not provide their own processors.

At a minimum, a GigE interface is recommended. While GigE interfaces and switches are affordable for home use, modern disks can easily saturate their 110 MiB/s throughput. For higher network throughput, multiple GigE cards can be bonded together using the LACP type of Link Aggregations. The Ethernet switch must support LACP, which means a more expensive managed switch is required.

When network performance is a requirement and there is some money to spend, use 10 GigE interfaces and a managed switch. Managed switches with support for LACP and jumbo frames are preferred, as both can be used to increase network throughput. Refer to the 10 Gig Networking Primer for more information.

Note

At present, these are not supported: InfiniBand, FibreChannel over Ethernet, or wireless interfaces.

Both hardware and the type of shares can affect network performance. On the same hardware, SMB is slower than FTP or NFS because Samba is single-threaded. So a fast CPU can help with SMB performance.

Wake on LAN (WOL) support depends on the FreeBSD driver for the interface. If the driver supports WOL, it can be enabled using ifconfig(8). To determine if WOL is supported on a particular interface, use the interface name with the following command. In this example, the capabilities line indicates that WOL is supported for the re0 interface:

    ifconfig -m re0
    re0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
            options=42098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWTSO>
            capabilities=5399b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_UCAST,WOL_MCAST, WOL_MAGIC,VLAN_HWFILTER,VLAN_H WTSO>

If WOL support is shown but not working for a particular interface, create a bug report using the instructions in Support.

## 1.4. Getting Started with ZFS

Readers new to ZFS should take a moment to read the ZFS Primer.