What is ZFS?

Introducing ZFS: Your Enterprise-Grade File System 

Many people have heard of ZFS, but are unsure what it actually is. So, what is ZFS? The Zeta File System (ZOL on Linux) is an enterprise-grade transactional file system that uses the concept of storage pools to manage physical storage space. Sun Microsystems began work on the product in 2001 and released it in 2005 as part of OpenSolaris. Even after OpenSolaris was discontinued, ZFS gained popularity since it is a user-friendly, scalable, and powerful data management system. Its strength lies in its simple administration model. 

What is ZFS?

The Oracle Corporation, famous for its cloud-based data management system, defines ZFS as “a revolutionary new file system that fundamentally changes the way file systems are administered, with features and benefits not found in any other file system found today.” It was designed to eliminate volume management by merging devices together into a storage pool. As such, ZFS operates similar to RAID technologies. 

Who uses ZFS? 

Companies or individuals who store massive amounts of data and those who prefer network-attached storage systems (NAS) often choose ZFS instead of a cloud. As a redundant storage system, ZFS protects data in the event of a disaster. Small businesses also choose ZFS since it gives them the privacy they are denied in a public cloud as well as complete control over their data. 

QSM, QSAN’s NAS Operating System, runs on this file system. 

How does ZFS work?

As an enterprise-grade file system, ZFS allows users to create and manage file systems with ease since by eradicating the need to edit configuration files or issue multiple commands. Users can set a quota to limit the amount of disk space used or reserve disk space for a specific file system. 

ZFS uses a hierarchical system layout defined by location- and- conductor-specific sub-module ports each of which is restricted in the locality. Basically, it manages the physical storage of data through storage pools, which allow file systems to share disk space within the pool. 

When users need a larger storage pool, they add disks similar to adding memory to a computer. As users add storage space, the file systems automatically use the additional memory without the user needing to configure that memory or assign individual processes. This system limits human touch, making it not only easy to use but also highly reliable. 

What are ZFS’ enterprise features?

As a transactional file system, ZFS maintains a consistent state for the file system on the disk. Instead of overwriting data, which leaves the file system in an inconsistent state, ZFS manages data with a copy-on-write mechanism. This approach means that neither power losses nor system crashes can corrupt the file system. 

A checksum algorithm can verify ZFS’ data and metadata in order to protect file integrity. Rather than performing checksum verification on a per-block basis, ZFS checksums work at a file systems level. The storage pools can self-heal data by detecting bad data blocks and replacing them with a redundant copy. Essentially, ZFS can silently check data and make repairs without user involvement. 

ZFS’ most significant feature may be its scalability. Since ZFS can manage 1 billion terabytes of data, nearly any size business or IT department can use the system. It designates all metadata dynamically, meaning users do not have to preallocate inodes, which could reduce scalability. 

As Oracle notes, “Directories can have up to 248 (256 trillion) entries, and no limit exists on the number of file systems or the number of files that can be contained within a file system.” 

ZFS also offers a built-in snapshot, which can grab a picture of stored data at a precise instant. Users find snapshots quick and easy. Snapshots take up no extra disk space in the pool at first, but as data evolves, they reference old data and thus consume some space. These snapshots prevent old data from slipping back into the current storage pool. ZFS can also send and receive file system snapshots, a process that allows users to optimize their disk space. 

Why do users choose ZFS?

Most users select ZFS because it offers a simple administration model. Creating and managing file systems becomes easy with no separate volume manager commands to learn. Plus, the system manages mount points automatically. The file system’s low costs let companies create new ones for each project and user, enabling finer data management. 

Data Integrity, ZFS vs. Other File Systems

Why data integrity take place nowadays?

In the digital era, small files I/O and requests have taken place for nowadays technologies, such as financial blockchains, artificial intelligence, internet of things and e-commerce. These happened everywhere in our daily lives, for example, paying grocery shopping via mobile payments, withdraw cash in ATM, activity sensors in offices, factories, and public transportation stations, and many more activities that you can’t imagine. The rise of mobile apps and innovative technology transformed nearly every aspect of our lives. The incredible convenience was created to serve people and hard to believe how it been working in the last decade. However, the more convenience we have experienced, the more data we have been created. However, have you ever imagined that once if the data has been incorrectly placed or recorded, what if the data was accidentally corrupted by itself, what if the wrong data has replaced the right one? With these ideas came out, data integrity has become the major challenge for IT for the devastating threats and the fundamental of the digital era.

Why XCubeNAS?

In the principle of designing XCubeNAS, data integrity was at the top of the list making the development team to face multiple trade-offs between safety and performance. We have crafted a unique design for user-defined pools, which fulfills different application requirements; each of them serves a different purpose: Databases, General, and Media Streaming.
The database pool is designed to have the highest performance with smaller files. The general pool fulfills requirements of 80% of applications, like backup and file sharing. The media streaming pool provides high performance for larger files and is optimized for 10GbE, Thunderbolt 3.

How XCubeNAS ensure data integrity?

Data integrity and data safety are essential for databases. QSAN’s storage management software, QSM, offers two mechanisms to ensure an absence of data failures between servers and storage. First, we set the data to write through the cache to prevent data loss from accidental disasters, like flash damage or power outage. The second, we apply ZFS checksum. ZFS’ checksum was designed to ensure data integrity. In other file systems, like EXT4, checksums are used for integrity checks and only protect against bit rots. ZFS’ checksum, however, prevents issues such as phantom writes, misdirected reads and writes, direct memory access parity errors and accidental overwrites. The checksum is accomplished by storing it in the parent block pointer instead of in the block itself that contains a self-validating SHA-256 checksum.

Additionally, ZFS also supports self-healing data by using a mirrored setup similar to RAID0. Apart from this, other file systems are not addressable if a block is corrupt unless it’s a metadata block, which means other file systems may pass bad blocks to the application and it might overwrite the accurate data in other mirrors. ZFS is a transaction-based file system, which means the operations are atomic, if operation crash or power outage occurs, the file system can quickly recover from corruption.

Beyond Security, the performance

Higher security requires higher resources. As mentioned above, ZFS is using the memory-based checksum, which claims higher memory capacity installed in the system. Also, ZFS has used ARC, Adaptive Replacement Cache algorithm, which was implemented in ZFS to outperform the traditional RAM algorithm by providing a superior hit rate. Moreover, high performance is not the only goal for ZFS, to ensure each file and blocks are correct, ZFS also requires longer computing time and RAM capacity to complete the process, which may impact the latency and impact performance.

Conclusion

In this summary, ZFS has its capacity to prevent silent data corruption which secures the data during transmission and storage. In any application scenario, sending and receiving the uncorrupt data is essential. That’s how the XCubeNAS was designed. Built on ZFS, it is a reliable and high-performance enterprise storage solution.