synology_apm_repo.sdk.dedup.dedup_file module

DedupFile: the sole cross-layer read contract.

Any workload — VM disk image, FS file, SaaS raw object — ultimately reduces to a (stream_id, session_id, comp_offset) triple plus a size. Everything above this module (catalog, units, CLI, TUI) reads content through read()/stream()/export_range(), never a bucket or chunk. ByteRangeView is the second shared primitive: FS’s content_dedup_id + file_size and SaaS’s object_table.(offset, length) are both a named sub-range of a bigger dedup file.

Every read() locates its starting chunk-map record by CompositionRecord.entries’s binary search, never a linear scan.

synology_apm_repo.sdk.dedup.dedup_file.DEFAULT_STREAM_BLOCK = 8388608

Default block size of every ContentSource implementer’s stream().

synology_apm_repo.sdk.dedup.dedup_file.MAX_SINGLE_READ_SIZE = 1073741824

Ceiling on one DedupFile.read()/ByteRangeView.read() call’s bytearray(length) allocation, since length traces back to repository-declared metadata with no bound of its own. A caller needing more uses stream(), which allocates at most one DEFAULT_STREAM_BLOCK at a time.

async synology_apm_repo.sdk.dedup.dedup_file.stream_via_read(source, block=8388608)

Yield (offset, bytes) blocks front-to-back via repeated source.read(offset, block) calls.

Raises:

ValueError – source.size is None.

synology_apm_repo.sdk.dedup.dedup_file.validate_read_args(offset, length)

Rejects a negative offset/length — the ValueError half of ContentSource.read’s contract, shared by every implementer that doesn’t need DedupFile’s own per-argument messages.

Raises:

ValueError – offset or length is negative.

synology_apm_repo.sdk.dedup.dedup_file.clamp_read_length(offset, length, size)

Resolve a read(offset, length) request against a known size: fills in the default length (offset to size) and clamps a request extending past size down to what’s actually there. Every ContentSource implementer shares this contract: reading past the end returns fewer bytes, never an error — only a negative offset/length is left for the caller to reject before calling this.

Returns:

The byte count to read, >= 0.

Return type:

int

synology_apm_repo.sdk.dedup.dedup_file.validate_export_range(start, end, size)

Checks 0 <= start <= end <= size — the range a ContentSource.export_range call may ask for.

Raises:

ValueError – The range is not inside [0, size].

async synology_apm_repo.sdk.dedup.dedup_file.read_blocks(source, start, end, block=8388608)

Yield (offset, bytes) blocks of source’s [start, end) via repeated source.read(start + offset, block) calls, with offset counted from start. The offset advances by the requested length, not the returned one, so a short read cannot stall the loop.

class synology_apm_repo.sdk.dedup.dedup_file.DedupContent

Bases: object

The content methods DedupFile and ByteRangeView share, in the coordinates of the window _window() names within its base file: the whole file for a DedupFile, the view’s range for a ByteRangeView.

size: int | None
async read(offset=0, length=None)
stream(block=8388608)

Yield (offset, bytes) blocks front-to-back via read; needs a known size.

export_window()

(base file, window start, size) — the range an export of this content covers.

Raises:

ValueError – The size is unknown.

async export_range(writer, start, end, *, sparse=True, progress=None, tuning=None)

Write this content’s [start, end) into writer, at offsets relative to start; start must fall on a 4096-byte boundary of the base file. Keywords and errors: see export_scheduler.export_to_writer.

Raises:

ValueError – The range is not inside this content, start is not chunk-aligned, or the size is unknown.

async planned_bytes(start, end)

The real DATA bytes an export of [start, end) writes, which is what its progress counts.

pool_descriptor()

How a worker process rebuilds the base file’s Pool, or None when its store cannot be rebuilt in another process.

class synology_apm_repo.sdk.dedup.dedup_file.DedupFile(comp_reader, pool, comp_offset, *, size=None)

Bases: DedupContent

A logical, byte-addressable file backed by one composition record.

size is supplied by the caller (file_meta.file_size, object_table.file_size, …), never inferred from the composition record, which may end before the file does (trailing HOLE). None means unknown: read then needs an explicit length and exports raise ValueError.

property stream_id: StreamId
property session_id: SessionId
property comp_offset: int
property pool: Pool

The Pool this file’s chunks resolve through.

async cached_record(*, record_head=None)

This file’s own CompositionRecord, fetched on first call and cached for this file’s life; a given record_head (the already-parsed head) saves that first fetch reading it.

async read(offset=0, length=None)

Read length bytes starting at offset (default: from offset to size). ZERO/HOLE ranges read as zero bytes. A request extending past size is clamped to the bytes that exist (see clamp_read_length), never zero-padded.

Parameters:
  • offset (int) – Start offset in the file.

  • length (int | None) – Byte count; None reads to size (required when size is unknown).

Returns:

The bytes read, possibly fewer than length.

Raises:
  • ValueError – offset or length is negative, or length is None while size is unknown.

  • ResourceLimitExceededError – the resolved read size exceeds MAX_SINGLE_READ_SIZE — use stream() instead for a legitimately large read.

  • NotFoundError – A composition sub-file or bucket is missing.

  • ChunkCompactedError – A chunk in range was reclaimed by compaction.

  • DataCorruptError – The record or a chunk fails to decode or verify.

Return type:

bytes | bytearray

view(offset, length)

A ByteRangeView of length bytes at offset.

class synology_apm_repo.sdk.dedup.dedup_file.ByteRangeView(base, offset, length)

Bases: DedupContent

An offset+length window into a DedupFile, with the same read contract (size/read/stream/export_range) in window-relative coordinates, for a workload whose “file” has no composition record of its own.

size: int
property base: DedupFile

The DedupFile this view windows into.

property offset: int

This view’s start offset within base.

async read(offset=0, length=None)

See units.base.ContentSource.read: a request past this view’s size is clamped, never an error.