synology_apm_repo.sdk.format.compression module

Chunk decompression dispatch (FORMAT-SPEC.md: SizeStore).

Decrypt before decompress — ciphertext length equals compressed length; decompression happens on the plaintext side. This module only decompresses; crypto only decrypts; callers sequence the two.

synology_apm_repo.sdk.format.compression.ZSTD_FRAME_MAGIC = b'(\xb5/\xfd'

Standard ZSTD frame magic — the strongest offline signal that a candidate vault key is correct for a ZSTD chunk (FORMAT-SPEC.md: chunk-pool-encryption). Unlike LZ4’s unmagicked block format, decrypting with the wrong key and getting something that both matches this magic and decompresses cleanly is negligibly unlikely.

class synology_apm_repo.sdk.format.compression.CompressType(value)

Bases: Enum

Per-chunk compression type, as recorded in SizeStore (FORMAT-SPEC.md: SizeStore). Values match the on-disk encoding exactly — note 3 is skipped.

NONE = 0

LZ4 = 1

ZSTD = 2

COMPACTED = 4

synology_apm_repo.sdk.format.compression.decompress(ctype, data)

Decompress one chunk’s ciphertext-removed bytes back to exactly FIXED_CHUNK_LENGTH (4096) bytes of plaintext.

Single-chunk path — see decompress_many for the batched form. data accepts a memoryview as well as bytes — like decompress_many, so a caller slicing a chunk out of its own larger read buffer can pass a view through without copying it first; only the NONE case below (no real decompression happening at all) needs an explicit copy to still return real bytes as promised.

Raises:

• ChunkCompactedError – For CompressType.COMPACTED (unrecoverable, reclaimed by compaction).

• DataCorruptError – Decompression succeeds but yields anything other than exactly 4096 bytes.

synology_apm_repo.sdk.format.compression.decompress_many(items)

Batch form of decompress: decompresses a whole sequence of (compress_type, ciphertext-removed data) pairs in one pass, preserving input order. data accepts a memoryview as well as bytes so a caller slicing chunks out of its own larger read buffer can pass a view through without copying it first.

A ZSTD entry’s returned ``memoryview`` is a view into that call’s own shared decode buffer, not an independent copy — deliberately: copying every frame out into its own bytes object is the single most expensive step in the whole decode path. Tradeoff: keeping a returned memoryview alive keeps the whole batch’s shared decode buffer alive too — up to one merged run’s worth of chunks, not just that one chunk’s 4096 bytes — so copy out via bytes(x) at any point one chunk needs to outlive this call.

This is the batch decode entry point BucketReader._decode_run uses.

synology_apm_repo.sdk.format.compression.decompress_zstd_stream(data, *, max_output_size=None)

Decompress a whole ZSTD frame (e.g. version.db.zst or a service-level-DB snapshot object), as opposed to a single fixed-4096-byte chunk.

max_output_size, when given, bounds how much the decompressor produces before giving up (zstandard.ZstdError) — for callers decompressing content of an unconfirmed type, a cap turns “this wasn’t really zstd-framed” into a fast failure instead of decompressing a corrupt or hostile oversized frame in full. Omit it (the default, genuinely unbounded) for a source already known to be a real, trusted db snapshot.

Always reads through the streaming reader, counting bytes as they come out, rather than ever calling zstandard’s one-shot ZstdDecompressor.decompress(data, max_output_size=N): that API only enforces max_output_size when the frame does not declare its own decompressed content size in its header: a frame that does (the common case — every frame this project’s own encoders and test fixtures produce, since that’s ZstdCompressor()’s default) is decompressed in full regardless, silently ignoring the cap. Reading incrementally and checking the running total ourselves enforces max_output_size unconditionally, independent of whatever the frame’s header claims — the frame’s own declared size is never trusted either way.