synology_apm_repo.sdk.dedup.composition_reader module¶
Composition sub-file access and record decoding.
Two collaborating classes:
CompositionReader— resolves(stream_id, session_id)to the right sequence of composition sub-files (viacomposition_pathplus sequence-id resolution), and reads any byte range of the session’s global offset space, transparently crossing 16 MiB sub-file boundaries.CompositionRecord— one backup version’sRecordHeadplus page-cached, binary-searched access to itsChunkMapRecordarray.
- class synology_apm_repo.sdk.dedup.composition_reader.CompositionReader(store, dir_cache, comp_root, stream_id, session_id, *, composition_cache=None)¶
Bases:
objectRead access to one
(stream_id, session_id)’s composition sub-files. Does not validate thesubID=0header automatically (a failure there would already surface as consistently wrong data) — callverify_headerexplicitly for that stricter, opt-in check.composition_cache, when given, is consulted byrecord()— see that method for the sharing contract.- async read_at(global_offset, n)¶
Read
nbytes starting at the session-globalglobal_offset, transparently crossing 16 MiB sub-file boundaries (FORMAT-SPEC.md: Composition file splitting).- Raises:
NotFoundError – A sub-file the range touches is missing.
FormatError – A sub-file ends before the range does.
- async verify_header()¶
Explicitly validate the
subID=0sub-file’s 64-byte header (major version,subFileSizeconstant match).- Raises:
UnsupportedVersionError – the header’s major version is unsupported.
DataCorruptError – A bad magic or header CRC, or
subFileSizeis not the fixed 16 MiB.NotFoundError – the
subID=0sub-file is missing.FormatError – the sub-file is truncated.
- async record(head_off, *, record_head=None)¶
Open the composition record at global offset
head_off(db/file_map.comp_offset) — reads only the 32-byteRecordHead, not the (potentially huge) chunk-map array that follows it, and not even that whenrecord_head(the record’s already-parsed head) is given.With a
composition_cache, every caller sharing the same(stream_id, session_id, head_off)gets the identicalCompositionRecord(page cache warm), whateverrecord_headit passes, until the entry is LRU-evicted; the next call then builds a new one.- Raises:
NotFoundError – the sub-file is missing.
FormatError – the sub-file is truncated.
DataCorruptError – bad
RecordHeadmagic or head CRC, or an unknown status.UnsupportedVersionError – the record lacks the
Redundancymode bit.
- class synology_apm_repo.sdk.dedup.composition_reader.CompositionRecord(head_off, record_head, reader, _pages=<factory>, _page_end_offsets=<factory>, _contiguous_scanned=0, _repaired_array=None, _page_locks=<factory>)¶
Bases:
objectOne backup version’s composition record: a
RecordHeadplus lazy, page-cached, binary-search-capable access to itsChunkMapRecordarray.Pages (
_PAGE_SIZEentries each) are fetched with one read each and cached in_pagesas raw bytes, bounded to_DEFAULT_PAGE_CACHE_MAXSIZEpages (LRU beyond that); a page’sChunkMapEntryvalues are decoded only as a caller consumes them.prime_pagesfills the cache from an array a caller already read;seed_pages_from_arrayswitches every page, cached or not, to already-repaired bytes.- record_head: RecordHead¶
- reader: CompositionReader¶
- property status: CompositionStatus¶
- async entries(start=0, end=None)¶
Stream
ChunkMapEntryvalues whose range overlaps[start, end)(file offsets, not record indices;end=Noneruns to the last entry). The first entry is found by binary search (_locate), then entries are walked forward page by page.
- async seed_pages_from_array(array_raw)¶
Serve every page from
array_raw, chunk-map array bytes a parity repair already confirmed, so no later read sees the still-corrupted on-disk copy, including after a page’s eviction.- Raises:
ValueError –
array_rawis notmap_num * CHUNK_MAP_RECORD_LENGTHbytes.
- async prime_pages(array_raw, start, end)¶
Cache the pages of
array_raw(this record’s whole chunk-map array, already read and trusted by the caller) that overlap the file-offset window[start, end), within the page cache’s free room: nothing is evicted andarray_rawis not kept. Every page’s end-offset up to the window’s end is recorded, so_locatereaches the window with no I/O. Pages already cached, and a recordseed_pages_from_arrayrepaired, are left as they are.- Raises:
ValueError –
array_rawis notmap_num * CHUNK_MAP_RECORD_LENGTHbytes.
- async extent()¶
(start, end): the file-offset range this record covers, from its first entry’sfile_offsetto its last entry’send_offset; reads only the first and last pages.units/device_pcps.pyuses it for a PC/PS disk fragment, whosefile_meta.file_sizeis the whole disk’s capacity (FORMAT-SPEC.md: PC/PS disk fragments).