synology_apm_repo.sdk.dedup.composition_reader module

Composition sub-file access and record decoding.

Two collaborating classes:

  • CompositionReader — resolves (stream_id, session_id) to the right sequence of composition sub-files (via composition_path plus sequence-id resolution), and reads any byte range of the session’s global offset space, transparently crossing 16 MiB sub-file boundaries.

  • CompositionRecord — one backup version’s RecordHead plus page-cached, binary-searched access to its ChunkMapRecord array.

class synology_apm_repo.sdk.dedup.composition_reader.CompositionReader(store, dir_cache, comp_root, stream_id, session_id, *, composition_cache=None)

Bases: object

Read access to one (stream_id, session_id)’s composition sub-files. Does not validate the subID=0 header automatically (a failure there would already surface as consistently wrong data) — call verify_header explicitly for that stricter, opt-in check.

composition_cache, when given, is consulted by record() — see that method for the sharing contract.

async read_at(global_offset, n)

Read n bytes starting at the session-global global_offset, transparently crossing 16 MiB sub-file boundaries (FORMAT-SPEC.md: Composition file splitting).

Raises:
async verify_header()

Explicitly validate the subID=0 sub-file’s 64-byte header (major version, subFileSize constant match).

Raises:
async record(head_off, *, record_head=None)

Open the composition record at global offset head_off (db/file_map.comp_offset) — reads only the 32-byte RecordHead, not the (potentially huge) chunk-map array that follows it, and not even that when record_head (the record’s already-parsed head) is given.

With a composition_cache, every caller sharing the same (stream_id, session_id, head_off) gets the identical CompositionRecord (page cache warm), whatever record_head it passes, until the entry is LRU-evicted; the next call then builds a new one.

Raises:
class synology_apm_repo.sdk.dedup.composition_reader.CompositionRecord(head_off, record_head, reader, _pages=<factory>, _page_end_offsets=<factory>, _contiguous_scanned=0, _repaired_array=None, _page_locks=<factory>)

Bases: object

One backup version’s composition record: a RecordHead plus lazy, page-cached, binary-search-capable access to its ChunkMapRecord array.

Pages (_PAGE_SIZE entries each) are fetched with one read each and cached in _pages as raw bytes, bounded to _DEFAULT_PAGE_CACHE_MAXSIZE pages (LRU beyond that); a page’s ChunkMapEntry values are decoded only as a caller consumes them. prime_pages fills the cache from an array a caller already read; seed_pages_from_array switches every page, cached or not, to already-repaired bytes.

head_off: int
record_head: RecordHead
reader: CompositionReader
property status: CompositionStatus
property map_num: int
property attr_leng: int
async entries(start=0, end=None)

Stream ChunkMapEntry values whose range overlaps [start, end) (file offsets, not record indices; end=None runs to the last entry). The first entry is found by binary search (_locate), then entries are walked forward page by page.

async seed_pages_from_array(array_raw)

Serve every page from array_raw, chunk-map array bytes a parity repair already confirmed, so no later read sees the still-corrupted on-disk copy, including after a page’s eviction.

Raises:

ValueError – array_raw is not map_num *     CHUNK_MAP_RECORD_LENGTH bytes.

async prime_pages(array_raw, start, end)

Cache the pages of array_raw (this record’s whole chunk-map array, already read and trusted by the caller) that overlap the file-offset window [start, end), within the page cache’s free room: nothing is evicted and array_raw is not kept. Every page’s end-offset up to the window’s end is recorded, so _locate reaches the window with no I/O. Pages already cached, and a record seed_pages_from_array repaired, are left as they are.

Raises:

ValueError – array_raw is not map_num *     CHUNK_MAP_RECORD_LENGTH bytes.

async extent()

(start, end): the file-offset range this record covers, from its first entry’s file_offset to its last entry’s end_offset; reads only the first and last pages.

units/device_pcps.py uses it for a PC/PS disk fragment, whose file_meta.file_size is the whole disk’s capacity (FORMAT-SPEC.md: PC/PS disk fragments).