Skip to content

Data Compression

Working with large string values can affect both bandwidth, memory, and cost on every round trip to the server. GLIDE’s compression feature shrinks those values before they leave the client and expands them again on read, reducing the cost of large commands.

Compression is a client side feature. The server simply stores compressed data as raw bytes from GLIDE.

For setup instructions and code examples, see our section on Compressing Data.

Compression runs in GLIDE’s Rust core layer, sitting between your application code and the network.

When you issue a command like SET:

  1. The client checks whether the value exceeds minCompressionSize (default: 64 bytes). Small values are sent as-is because the compression overhead would make them larger.
  2. If the value is large enough, the client compresses it with the configured backend (ZSTD or LZ4).
  3. A short magic header is prepended to identify the data as compressed and record which backend was used.
  4. The server receives and stores the compressed payload without modification.

When a response comes back from a command like GET:

  1. The client checks the first bytes for the magic header.
  2. If present, it decompresses and returns the plain value.
  3. If absent, it returns the value unchanged.

Compressed and uncompressed values can coexist in the same database. The client handles both.

BackendCharacteristics
ZSTDBetter compression ratio, more CPU. Good default for most workloads.
LZ4Faster but compresses less. Better when latency matters more than size.

Only specific commands support compressions.

These commands work transparently with compressed values:

  • GET
  • GETDEL
  • GETEX
  • GETSET
  • MGET
  • MSET
  • MSETNX
  • PSETEX
  • SET
  • SETEX
  • SETNX

The server doesn’t know about compression, so commands that read, modify, or compare a stored string operate on the compressed bytes instead of the original value when compression is enabled.

These commands are disabled when compression is enabled:

  • String manipulation: APPEND, GETRANGE, LCS, SETRANGE, STRLEN, SUBSTR
  • Value comparison: DELIFEQ, SET with the IFEQ or IFNE
  • Numeric operations: DECR, DECRBY, INCR, INCRBY, INCRBYFLOAT
  • Bit operations: BITCOUNT, BITFIELD, BITFIELD_RO, BITOP, BITPOS, GETBIT, SETBIT

The incompatibility exists because the server operates on stored bytes directly. Since it doesn’t know the bytes are compressed, operations like APPEND would concatenate compressed data, and STRLEN would return the compressed length rather than the original length. Comparison commands fail for the same reason. Operations like DELIFEQ and SET ... IFEQ incorrectly compare the uncompressed comparison value against the compressed data.