Erasure Coding

Enterprise protection at lower storage footprint levels

Ideal for moderately sized content (1 MB+) and large content stores

As file sizes and data sets grow - operational resources such as data center space, power, cooling and hardware become more difficult to manage and copy based protection schemes become less efficient. The solution is erasure coding, which breaks a file into multiple data segments and computes parity segments, resulting in total segments that use less capacity and operational resources than an additional copy but still provide enterprise grade protections.

• Files split into multiple data segments (k) and additional parity segments (p) based on the content of the data segments. This results in m total segments (k + p = m) being distributed to m different CAStor nodes or sub-clusters.
• Capacity usage is less than an additional copy: calculated by (m total segments / k data segments) * Size = footprint
• For a 1GB object with 5:2 encoding, the footprint on disk would be: 7/5 * 1GB = 1.4GB
  For a 1 GB object with a 10:6 encoding, the footprint on disk would be: 16/10 * 1GB= 1.6GB

Replication

Responsive Access with Increased Protection

Ideal for content delivery and small files (KBs-MBs)

Responsiveness is enabled by rapid access to a file. This means that to ensure snappy response times whole copies is a better solution than splitting a file into segments. In addition there is overhead associated with splitting a file into segments which is why erasure coding is less efficient for smaller files. For these use cases replication is ideal. Replication uses copy-based data protection where complete copies of a piece of content are made and distributed across nodes or sub-clusters. Because replication stores data contiguously on disk once the first bit is identified, content is delivered rapidly and efficiently without the need for rehydration.

No Specialty Nodes,
No Single Points of Failure

ECP works directly with CAStor - the Object Storage Platform’s object storage engine - leveraging CAStor’s symmetric architecture where all nodes do the same thing and support massive parallel scale. Every node can replicate a file and every node can encode and decode an erasure-coded file. There are no specialty nodes and no bottlenecks. This also means that the inherent high availability nature of CAStor in addition to the self-optimizing, self-healing and rapid recovery functionality all work with replicated and erasure-coded content.

Protection that Meets Your Business Requirements

By leveraging ECP you can determine what level of protection or level of availability is required based upon your business requirements for a specific app or class of data. Use replication when content is first created and accessed frequently then shift to erasure-coding when content is archived and you need to conserve disk capacity and data center footprint. ECP is the only data protection scheme that seamlessly enables this in the storage system reducing complexity and giving you granular control of your content.