Physical storage needs to be assigned before virtual disks can be added. On FreeBSD Disks available through the CAM system (camcontrol devlist) and accessible using atacontrol list will be visible for addition or deletion. On Linux disks available through the SCSI subssytem (check /proc/scsi/scsi for reference) and created by mdadm (software raid) will be visible for addition or deletion. Disks with mounted partitions or of size less than 4GB are ignored.
Recently support for ZFS ZVOL has been added. Note that QUADStor assumes ownership of about 80% of the total memory. For example on a system with 8GB memory, 6.6 GB of memory shall be reserved by QUADStor for its own use. In order to avoid out of memory conditions the OS and ZFS will have to work with the remaining 1.4 GB. Ensure that the following parameters are set to restrict ZFS memory usage
For example for a 8GB memory system the following are good settings
When the ZVOL is create ensure that -o volblocksize 4K is specified if "Enable Compression" (described below) will not be selected. The default volblocksize is 8K but a significant part of the metadata written/read or random data of 4K size. Using a 8K or more volblocksize will lead to massive loss in performance.
If "Enable Compression" will be selected then specify -o volblocksize 1K . When "Enable Compression" is specified and if a VDisk compression is enabled and if the data written is compressible, data will be written in chunks of 1K, 2K, 4K etc. So even a 4K volblocksize will lead to performance degradation.
Adding a Disk
On QUADStor startup the system is scanned for all available disks. Available disks do not include disks with mounted partitions. Available disks include software raid disks created using gvinum or mdadm. On linux LVM volumes can also be configured, however once configured the LVM volume should not be resized. Newer versions of the software also allow disk partitions to be configured.
In order to view the available disks and to configure the storage perform the following steps
- Ensure that appropriate license keys have been added. See Adding License Keys
- Click on the “Physical Storage” menu and select the “View Storage” Submenu
The physical storage detected by the system is listed in the “View Storage” page and is as shown in the figure below
- Click on the “Add” link to add a disk which can be used for virtual disks
In the next page additional properties for the disk can be selected as shown in the figure below.
Select the storage pool the disk will belong to
Select "Log Disk" if the disk needs to be a log disk. Usually the system determines this, however selecting this gives additional hints which configuring the disk.
A "HA Disk" is a configured disk in the system which is used as a quorum disk, High Availability synchronization etc.. If High Availability is required then select "HA Disk" for atleast one disk. It is safe to select this option in any case.
Enabling compression for a disk means that data on this disk can be compressed prior to writing data to that disk. The eligibility criteria for a disk to support compression is that logical sector size of the disk is either 512 or 1024. This would mean that if the disk which has a physical sector size of 4k it should be able to emulating a sector size of 512 bytes or 1K to support compression
It is advisable not to enable compression for slower disks. Performance drop with compression enabled for a disk is significant and memory requirements go higher. However Fast SSDs combined with large amounts of compressible data can give additional benefit in capacity savings without a significant drop in performance.
If the sector size reported by a disk is not 512 or 1024, compression is not enabled even if the option above is selected.
A disk that was added can be removed later by clicking on the “Remove” link for the disk
In case newer disk storage has been attached to the system, click on the “Rescan” button to rescan for attached storage devices
The first disk added to the pool will also be used to store data deduplication finger print information (if the pool is configured to store deduplication tables). This disk can only be removed when all other configured disks and VDisks in the pool are removed. The best practice is to add the fastest disk as the first disk. This would ensure that deduplication tables are synced and read from disk with the least amount of latency.
The first few disks (the count is determined by the QUADStor software) are also added as log disks. Log disks contain reserved space which is used for writing and reading key transaction data. Once added, Log disks can only be removed from the pool when there no I/O is being performed.
Modifying Disk Properties
Certain properties of a configured disk can be changed. Click on the "Modify" link for a configured disk to change its properties and is as shown in the following figure:
"HA Disk" was described above. In order to configur e a disk as the new HA disk, the previous HA disk has to be unmarked first.
By default when a disk is configured Unmap is disabled for the disk. Unmap can be enabled/disabled for a disk here. When enabled if a certain range of blocks for the disk are unused by QUADStor, the system issues a UNMAP/TRIM (aka Discard) request for that range of blocks to the corresponding disk. It is a good idea to enable this if the underlying physical disk is a SSD/Flash disk or even a thin provisioned disk. However certain disks don't yet handle the UNMAP command well, disable this option for such disks.
Write Cache property determines the behavior of write requests issued and has a significant impact on write performance.
Write Cache by default is set to "Write Cache", which basically assumes that the attached physical storage has a battery backed cache.
Set it to "Write FUA" which ensures the integrity of log data. This would ensure that the VDisk metadata can be brought to a consistent state after a system crash
Set it to "Write Flush FUA" which ensure that the client data is flushed from the disk cache, and the log data is directly written to disk skipping the disk cache.
Note that "Write FUA/Write Flush FUA" are only used to ensure the integrity of log data on a system crash. Also these options aren't supported for FreeBSD. On a FreeBSD either option results in a flush operation sent to the disk.
If your disks do not support a battery backed cache, the disk driver completes a write request once the data is stored in the disk cache, but a power loss will lead to loss of data stored in the cache but not yet written to disk.
A another but much slower option is to altogether disable the disk cache (write-through) which needs to be set directly for a disk.