synology_apm_repo.sdk.dedup.fingerprint module

Fingerprint lookup: locates the stored SHA-256 digest for one chunk in its group’s .inf/.fgp files. Used by verify and by Pool.read_chunk when verify_fingerprint is enabled.

Correctly handles a chunk whose fingerprint straddles a .fgp 4 MiB segment boundary, across both plaintext and encrypted, vault- and object-store-backed repositories.

Group path indirection: .inf/.fgp are shared by an entire 1024-bucket group, keyed by the group’s starting bucket id (bucket_id & ~1023), not any individual bucket’s own id — the same 10-bit layering scheme as .buk paths, just applied to the group id instead. .fgp is additionally split into 4 MiB segments (<prefix>_<segIdx>.fgp) since a full group’s fingerprint data (1024 buckets * up to 8192 chunks * 32 bytes) would otherwise be a single multi-hundred-MB file.

``AllocationTableCache``: fingerprint()/fingerprints() already resolve a bucket’s own .inf header + allocation-table entry once per call regardless of how many chunks that call checks — but every one of up to GROUP_BUCKET_NUM (1024) distinct buckets sharing one group still pays for the same header validation and entry lookup again, once per bucket, since nothing persists that resolution across separate calls, even though the underlying bytes are identical for every bucket in the group. An optional AllocationTableCache (kept by Pool, so every caller reading fingerprints through one — browsing, export, and both verify orchestrators alike — shares it automatically) resolves the whole group’s allocation table once, the first time any of its buckets is touched, and answers every later bucket in that same group from memory.

class synology_apm_repo.sdk.dedup.fingerprint.AllocationTableCache

Bases: object

Caches each distinct (stream_id, group's starting bucket id)’s whole, header-validated .inf allocation table for this cache instance’s whole lifetime — since up to GROUP_BUCKET_NUM (1024) distinct buckets share one group’s identical allocation-table bytes, and without this they’d each re-read and re-validate them separately. Pure in-memory bookkeeping on top of AsyncKeyedCache; owns no store/dir_cache/pool_root of its own, since every call already has those in hand (mirrors dedup.pool.BucketReaderCache’s own reason for the same shape — see that class’s docstring).

clear()

Drop every cached allocation table — for a caller releasing a whole Pool’s memory (see Pool.release_caches).

async resolve_entry(store, dir_cache, pool_root, stream_id, bucket_id)

_resolve_bucket_allocation’s exact return shape (full_dir, prefix, inf_path, byte_off, rec_num), backed by this cache instead of a fresh per-call resolution.

async synology_apm_repo.sdk.dedup.fingerprint.fingerprint(store, dir_cache, pool_root, stream_id, bucket_id, chunk_idx, *, cache=None)

Look up the stored 32-byte SHA-256 fingerprint for one chunk.

cache, when given, shares one group’s already-resolved allocation table across every bucket in it instead of resolving fresh each call.

Raises DataCorruptError if chunk_idx is beyond the group’s recorded fingerprint count for this bucket, or if the .inf header fails validation; FormatError if either file is truncated.

async synology_apm_repo.sdk.dedup.fingerprint.fingerprints(store, dir_cache, pool_root, stream_id, bucket_id, chunk_indices, *, cache=None)

Batched fingerprint() for every chunk_idx in chunk_indices, all within the same (stream_id, bucket_id) — resolves the shared .inf header/allocation-table entry exactly once regardless of how many chunks are checked, instead of once per chunk the way calling fingerprint() in a loop would (a real gap for Pool.verify_fingerprints’s multi-chunk case: every chunk in one bucket needs the identical two reads).

The digests themselves are batched too: chunk_indices is grouped into maximal runs of consecutive integers that also stay within one .fgp segment (_group_contiguous_runs), and each run costs one store.read() spanning the whole run rather than one 32-byte read per chunk. This is the common case for a bucket-wide sweep (FULL-level verify’s own “every non-COMPACTED chunk” walk): a large bucket’s fingerprints fit in a single .fgp segment, so an unbatched per-chunk read would otherwise turn one bucket’s worth of checking into tens of thousands of separate 32-byte reads. A caller passing genuinely scattered indices (no two adjacent) still costs one read per index, same as before — grouping never makes an unmergeable case worse.

cache, when given, shares one group’s already-resolved allocation table across every bucket in it instead of resolving fresh each call.

Raises the same exceptions fingerprint() does, for the same reasons.