Highly Available Storage (HAST), The Future of Transparent Storage.

High availability is a major requirement in critical business applications and is considered as a key component in such environments. In FreeBSD, the Highly Available STorage (HAST) framework enables for transparent storage of the same data through several physically separated machines connected over a TCP/IP network. HAST is considered as a network-based RAID1 (mirror), and is similar to the DRBD storage system used in the GNU/Linux platform. In combination with other high-availability features of FreeBSD, HAST makes it easy to build a storage cluster that is resistant to hardware failures.

HAST adds feature rich values in FreeBSD, as it can be used to mask I/O errors on local hard drives, it works with any file system supported by FreeBSD, Efficient and quick resynchronization as only the blocks that were modified during the downtime of a node are synchronized and finally it can be used in an already deployed environment to add additional redundancy.

HAST provides synchronous replacement between any two machines: the master node, and the slave node. These two machines are referred to as a cluster. This setup permits only one of the cluster nodes to be active at each time instant. The master node, also called the active, is the one which will handle all the I/O requests to HAST devices. The slave node is automatically synchronized from the primary node.

Each write, delete, or flush process is transferred to both the local disk and to the remote disk over TCP/IP. Each read operation is served from the local disk, unless the local disk is not up-to-date or an I/O error occurs. In such cases, the read operation is transferred to the slave node. HAST supports fast failure recovery. For this reason, it is critical to reduce the synchronization time after a node’s outage. To provide fast synchronization, HAST monitors an on-disk bitmap of dirty extents and only synchronizes those during a regular synchronization, with an exception of the initial sync.

Generally, HAST should work without issues. However, like any other software, there may be times when it does not work as it is supposed or crashes and completely stops functioning. The sources of the problems may be different, but the rule of thumb with HAST, is to make sure that the time is totally synchronized between the nodes of the cluster.

However, there is serious risk coming along with HAST usage,the Split-brain, that occurs when the nodes of each HAST cluster become unable to communicate with each other, and both are configured as master. This is a dangerous condition because it allows both nodes to make incompatible changes to the data. This problem must be corrected manually by the system administrator.