synology_apm_repo.sdk.dedup.dedup_file module¶
DedupFile: the sole cross-layer read contract.
Any workload — VM disk image, FS file, SaaS raw object — ultimately
reduces to a (stream_id, session_id, comp_offset) triple plus a size.
Everything above this module (catalog, units, CLI, TUI) reads content
through read()/stream()/export_range(), never a bucket or
chunk. ByteRangeView is the second shared
primitive: FS’s content_dedup_id + file_size and SaaS’s
object_table.(offset, length) are both a named sub-range of a bigger
dedup file.
Every read() locates its starting chunk-map record by
CompositionRecord.entries’s binary search, never a linear scan.
- synology_apm_repo.sdk.dedup.dedup_file.DEFAULT_STREAM_BLOCK = 8388608¶
Default block size of every
ContentSourceimplementer’sstream().
- synology_apm_repo.sdk.dedup.dedup_file.MAX_SINGLE_READ_SIZE = 1073741824¶
Ceiling on one
DedupFile.read()/ByteRangeView.read()call’sbytearray(length)allocation, sincelengthtraces back to repository-declared metadata with no bound of its own. A caller needing more usesstream(), which allocates at most oneDEFAULT_STREAM_BLOCKat a time.
- async synology_apm_repo.sdk.dedup.dedup_file.stream_via_read(source, block=8388608)¶
Yield
(offset, bytes)blocks front-to-back via repeatedsource.read(offset, block)calls.- Raises:
ValueError –
source.sizeisNone.
- synology_apm_repo.sdk.dedup.dedup_file.validate_read_args(offset, length)¶
Rejects a negative
offset/length— theValueErrorhalf ofContentSource.read’s contract, shared by every implementer that doesn’t needDedupFile’s own per-argument messages.- Raises:
ValueError –
offsetorlengthis negative.
- synology_apm_repo.sdk.dedup.dedup_file.clamp_read_length(offset, length, size)¶
Resolve a
read(offset, length)request against a knownsize: fills in the default length (offsettosize) and clamps a request extending pastsizedown to what’s actually there. EveryContentSourceimplementer shares this contract: reading past the end returns fewer bytes, never an error — only a negativeoffset/lengthis left for the caller to reject before calling this.- Returns:
The byte count to read,
>= 0.- Return type:
- synology_apm_repo.sdk.dedup.dedup_file.validate_export_range(start, end, size)¶
Checks
0 <= start <= end <= size— the range aContentSource.export_rangecall may ask for.- Raises:
ValueError – The range is not inside
[0, size].
- async synology_apm_repo.sdk.dedup.dedup_file.read_blocks(source, start, end, block=8388608)¶
Yield
(offset, bytes)blocks ofsource’s[start, end)via repeatedsource.read(start + offset, block)calls, withoffsetcounted fromstart. The offset advances by the requested length, not the returned one, so a short read cannot stall the loop.
- class synology_apm_repo.sdk.dedup.dedup_file.DedupContent¶
Bases:
objectThe content methods
DedupFileandByteRangeViewshare, in the coordinates of the window_window()names within its base file: the whole file for aDedupFile, the view’s range for aByteRangeView.- async read(offset=0, length=None)¶
- stream(block=8388608)¶
Yield
(offset, bytes)blocks front-to-back viaread; needs a knownsize.
- export_window()¶
(base file, window start, size)— the range an export of this content covers.- Raises:
ValueError – The size is unknown.
- async export_range(writer, start, end, *, sparse=True, progress=None, tuning=None)¶
Write this content’s
[start, end)intowriter, at offsets relative tostart;startmust fall on a 4096-byte boundary of the base file. Keywords and errors: seeexport_scheduler.export_to_writer.- Raises:
ValueError – The range is not inside this content,
startis not chunk-aligned, or the size is unknown.
- async planned_bytes(start, end)¶
The real
DATAbytes an export of[start, end)writes, which is what its progress counts.
- pool_descriptor()¶
How a worker process rebuilds the base file’s
Pool, orNonewhen its store cannot be rebuilt in another process.
- class synology_apm_repo.sdk.dedup.dedup_file.DedupFile(comp_reader, pool, comp_offset, *, size=None)¶
Bases:
DedupContentA logical, byte-addressable file backed by one composition record.
sizeis supplied by the caller (file_meta.file_size,object_table.file_size, …), never inferred from the composition record, which may end before the file does (trailingHOLE).Nonemeans unknown:readthen needs an explicitlengthand exports raiseValueError.- async cached_record(*, record_head=None)¶
This file’s own
CompositionRecord, fetched on first call and cached for this file’s life; a givenrecord_head(the already-parsed head) saves that first fetch reading it.
- async read(offset=0, length=None)¶
Read
lengthbytes starting atoffset(default: fromoffsettosize).ZERO/HOLEranges read as zero bytes. A request extending pastsizeis clamped to the bytes that exist (seeclamp_read_length), never zero-padded.- Parameters:
- Returns:
The bytes read, possibly fewer than
length.- Raises:
ValueError –
offsetorlengthis negative, orlengthisNonewhilesizeis unknown.ResourceLimitExceededError – the resolved read size exceeds
MAX_SINGLE_READ_SIZE— usestream()instead for a legitimately large read.NotFoundError – A composition sub-file or bucket is missing.
ChunkCompactedError – A chunk in range was reclaimed by compaction.
DataCorruptError – The record or a chunk fails to decode or verify.
- Return type:
- view(offset, length)¶
A
ByteRangeViewoflengthbytes atoffset.
- class synology_apm_repo.sdk.dedup.dedup_file.ByteRangeView(base, offset, length)¶
Bases:
DedupContentAn offset+length window into a
DedupFile, with the same read contract (size/read/stream/export_range) in window-relative coordinates, for a workload whose “file” has no composition record of its own.- async read(offset=0, length=None)¶
See
units.base.ContentSource.read: a request past this view’ssizeis clamped, never an error.