CloudBD disks are thinly provisioned. A newly created disk is initially 100% thin and only allocates a 1kB disk descriptor regardless of the disk size. Operations that write data over thin sections of the disk, such as formatting a new disk with a filesystem, will create blocks only as needed.
CloudBD disks support trim operations. Data deleted from your disks can be deleted from object storage and keep your storage costs proportional to your disk's data usage. To enable this feature, simply mount your filesystem with the discard option and/or periodically run fstrim.
The cost of object storage is typically half the cost of a cloud provider’s persistent disks. The combination of lower cost storage, as-needed data allocations, and enabling trim provides you with a disk that scales its costs with its usage. You wont need to pay for provisioned and unused disk storage anymore. Converting 128TiB of AWS EBS Throughput Optimized HDDs (st1) to CloudBD disks can save over $5,900 per month in total costs.
Costs of CloudBD disks are from CloudBD + AWS S3 (stacked bars)
Costs of AWS EBS st1 disks are with a single backup snapshot needed for durability (black line)
Our disks are highly durable because 100% of the data is stored in your cloud provider's object storage system. Object storage systems like S3 and GCS guarantee 11 nines (99.999999999%) annual durability for each object. CloudBD disks use multiple objects to store the data in block sized chunks. For a 1TiB CloudBD disk, combining the durability of all blocks results in a disk with 5 nines (99.9995%) annual durability. This level of durability matches the traditional gold standard of backup to magnetic tape. However, CloudBD disks do not degrade over time, have random access capability, and are highly available.
CloudBD’s disk driver is optimized for high throughput and low overhead. The disks are network devices and their throughput will depend on the network bandwidth and the latency to your object storage. On 25 gigabit AWS EC2 instances a CloudBD disk can achieve over 2 GiB/s read and write throughput. Each disk is tunable for resource control of its memory and CPU usage. It is built on an asynchronous IO framework that allows for 100s of object storage operations in parallel efficiently.
CloudBD disks are fully compatible with existing linux disk tools to create an enhanced virtual disk stack. Linux provides built in tools to add encryption, logical volume management, RAID, or any combination of those features to disks using the device-mapper system.
Encrypting the data on your disk before it ever is stored in your cloud provider is supported using dm-crypt. Dm-crypt is a block level encryption tool that uses the kernel crypto routines to securely encrypt blocks before they are sent to the underlying disk.
LVM provides high level support for snapshots and logical volume management. CloudBD disks can be added as a physical volume to LVM and used just the same as any attached physical hard disk. Instant point-in-time snapshots can be created through LVM without first requiring a full copy of the disk’s data. These snapshots are much faster and less expensive than AWS disk snapshots.
Many object storage systems, including AWS S3 and Openstack Swift, have an eventually consistent data model. After updating an object this data model allows the object storage systems to respond to reads with either the new or the old data for a period of time. The lack of strong data consistency makes object storage typically incompatible for disk storage. CloudBD has developed a patented algorithm to guarantee strong data consistency when storing and accessing data in an eventually consistent system. Using this algorithm we are able to provide a strongly consistent disk interface for eventually consistent object storage systems.
CloudBD disks are fully flush and sync compliant. This allows posix filesystems such as ext4 to work directly on our disks. When a flush or sync operation returns success to Linux, your data is safely persisted in the object storage system and all reads will always return the last written data.