Variable-length deduplication is a data deduplication algorithm that breaks a file system into subfile, variable-length data segments to determine unique and repetitive segments. This dramatically reduces backup storage during the backup and recovery process.
Variable-length deduplication is an advanced method for breaking up a data stream via context-aware anchor points. This subfile intelligent segmentation method provides greater storage efficiency for redundant data regardless of where new data has been inserted. As the name suggests, the length of segments vary, thus achieving higher deduplication ratios.
Who uses variable-length deduplication and why
Organizations with fast data growth, highly virtualized environments, and remote offices greatly benefit from variable-length deduplication over a fixed-block approach. Variable-length deduplication reduces backup storage and, when performed at the client, also reduces network traffic, making it ideal for remote backup.
How variable-length deduplication works
Client software examines the file system and applies a secure hash algorithm (SHA)-1 to variable-length data segments. Each data segment is assigned a unique identifier (ID). The client software then determines whether this unique ID has already been stored. If this object exists, a link to the stored object is referenced in the backup. In this way, the same segment is never backed up twice.
Benefits of variable length deduplication
By providing the highest possible level of deduplication, variable-length deduplication reduces backup storage, improves backup times, and lowers costs. When deployed at the client, it enables organizations to leverage existing bandwidth while reducing resource contention in highly virtualized environments.