synology_apm_repo.sdk.dedup.repository module

DedupRepo: opens one repository root and ties RepoInfo/key material/Pool/CompositionReader together behind open_file()/open_composition().

Repository-root path constants (FORMAT-SPEC.md: repo-root-layout’s fixed directory names) are uniform across both layout kinds — <repo_root>/@data/Pool, <repo_root>/@data/Composition, <repo_root>/db/; only layout.repo_root itself differs (the vault root directly for RepoKind.VAULT, or @ActiveProtectData/<repoId> for RepoKind.OBJECT_STORE). These are the same relative paths pool and keys already use.

synology_apm_repo.sdk.dedup.repository.FILE_MAP_STATUS_COMPLETE = 2

file_map-status) — see locate_file’s own Raises: section for how each is handled. FILE_MAP_STATUS_COMPLETE is public: units.saas.stream also needs it, to pre-filter forward-resolution’s own generation candidates down to ones locate_file will actually accept.

Type:

db/file_map.status values (FORMAT-SPEC.md

class synology_apm_repo.sdk.dedup.repository.FileLocation(stream_id, session_id, comp_offset, block, file_size)

Bases: object

One db/file_map row, resolved to what DedupRepo.open_composition needs plus (when available) file_meta.file_size. status is not carried here — locate_file already rejects every row except Complete (FORMAT-SPEC.md: file_map-status) before constructing one.

stream_id: StreamId
session_id: SessionId
comp_offset: CompOffset
block: int
file_size: int | None
class synology_apm_repo.sdk.dedup.repository.DedupRepo(store, layout, info, dir_cache, *, vault_key=None, bucket_cache_size=16, chunk_cache_size=4096, verify_fingerprint=False)

Bases: object

Rounds out the module-level Pool/CompositionReader machinery with a small cache of read-only db/<name> sqlite connections — together, what open_file/open_composition need to hand back a DedupFile.

property store: ObjectStore

The underlying ObjectStore this repository was opened against — exposed for Catalog-Layer-and-above callers that occasionally need a raw path read outside the Pool/CompositionReader/db() machinery (e.g. catalog’s object-store link-key display-name lookup, which lists @ActiveProtectKey/link/ directly).

property vault_key: bytes | None

The resolved VaultKey, if any (None for an unencrypted repository or one opened without key material) — exposed for Unit-Layer-and-above callers that need to decrypt something outside the Pool/CompositionReader machinery (e.g. DeviceProvider peeling an aHlT-enveloped target.db).

property dir_cache: DirCache

This repository’s shared DirCache — exposed for the same reason as store/vault_key — a caller outside Pool/CompositionReader (verify_checks.check_repo_info, units/verify_reachable.py’s top-down walk building its own CompositionReader) needs to resolve a .<seqId>-suffixed path itself, without duplicating a second, uncached listdir.

property comp_root: str

<repo_root>/@data/Composition — exposed so units/verify_reachable.py’s top-down walk can build its own CompositionReader per visited composition record without open_composition()’s DedupFile wrapping (verify wants the raw record/header, not a readable byte range).

property pool_root: str

<repo_root>/@data/Pool — exposed for the same reason as comp_root: units/verify_reachable.py’s top-down walk constructs its own private Pool (forcing verify_fingerprint/verify_ciphertext_crc on for the run, rather than mutating this repository’s own shared one), which needs this repository’s pool_root to do so.

async classmethod open(store, layout, keys=None, *, bucket_cache_size=16, chunk_cache_size=4096, verify_fingerprint=False)

Open layout. Cheap by construction — reads only repo_info (and, if keys is given and the repository is encrypted, whatever KeyMaterial.resolve_vault_key touches: one sqlite row or one small key file); never scans Pool or Composition.

A keys whose GCM tag doesn’t check out raises KeyMismatchError immediately — better to fail here than to hand back a repository that will silently decrypt every chunk into garbage later (AES-CTR has no integrity check of its own). verify_fingerprint=True makes per-chunk .fgp verification this session’s default for every read through pool instead of it being off by default — or pass it per-call there instead.

async db(name)

Open (and cache) a connection to db/<name>.

Read-only against the store either way: the fast path opens the real file immutable, and the slow path opens a private materialized copy read-write (see storage/sqlite.py) so an index hint can take effect — writes there never reach the store.

name is resolved through PHYSICAL_NAME_ALIASES first (e.g. "copy_target_file" has no on-disk object of its own; its table lives inside whichever generation "copy_target_version" resolves to), so requesting either aliased name transparently shares one connection.

On a RepoKind.OBJECT_STORE layout, the real generation is selected by FORMAT-SPEC.md: generation-selection’s transaction-log algorithm (resolve_generation), not the naive “largest .<N> suffix” rule every other per-generation file uses — a db/<name>.<N> generation can exist on disk before the transaction referencing it commits.

db/<name> is always raw (never aHlT/zstd-enveloped), so SqliteSource.from_raw_store skips straight to materializing it — no envelope detection needed here, unlike every other envelope→SQLite path in this project.

async locate_file(path)

Look up path in db/file_map and supplement it with file_meta.file_size when a matching row exists there too. file_meta.path is only unique per connection_config_id — this takes the first match, fine for the common single-workload case but a placeholder pending the Catalog Layer’s proper disambiguation.

Raises:

  • NotFoundError – no row for path at all, or its status is Initialized/Written/Compacted (0/1/3) — not yet, or no longer, resolvable content (FORMAT-SPEC.md: file_map-status).

  • DataCorruptError – the row’s status is Corrupted/Tainted (4/5) — the format itself flags this data as known-bad.

async file_map_paths_with_prefix(prefix, *, status=None)

Every db/file_map path starting with prefix, via a LIKE-prefix scan — file_map.path is the table’s own primary key, so SQLite resolves this as an indexed range scan, not a table scan. %/_/\ in prefix are escaped so they match literally rather than as SQL wildcards.

status, when given, restricts to rows at exactly that file_map.status value (FORMAT-SPEC.md: file_map-status) — unlike locate_file, this method applies no status filtering on its own.

async open_file(path)

Resolve path via db/file_map (+ file_meta.file_size when available) and return a ready-to-read DedupFile.

Raises:
open_composition(stream_id, session_id, comp_offset, size=None)

Construct a DedupFile directly from a (stream_id, session_id, comp_offset) triple — the path any workload-specific Unit Layer provider ultimately takes once it has resolved its own metadata down to this triple.

async probe_encrypted()

Cheaply determine whether this repository is actually vault-encrypted — no key needed at all, and never raises KeyRequiredError. Reads the repository’s own encryption-key record directly (probe_encrypted) — never opens a bucket file, never touches the Pool. Not the compile-time RepoInfo.encrypt_algorithm field (see BucketFileHeader.is_vault_encrypted’s docstring for why that’s unreliable); this reads the same live, per-repository record KeyMaterial.resolve_vault_key already reads to find one specific candidate key’s wrapped VaultKey, just its latest entry rather than one looked up by id.

None means “couldn’t tell” (the encryption-key record itself is entirely absent, which should not happen for a properly initialized repository) — a genuinely different answer from “confirmed not encrypted” (False), the same distinction KeyStatus’s own NO_KEY_PROVIDED draws for the same reason.

Result is cached for the lifetime of this (read-only, never-changes-under-us) DedupRepo — repeated calls cost nothing after the first.

async close()

Release every cached sqlite connection (and any temp file/ directory a slow-path materialization created), then drop this repository’s in-memory Pool caches.

Settles every in-flight db() fetch first, not just what’s already landed in _db_sources – a fetch cancelled mid-flight (a TUI worker torn down while a catalog load was still running, say) can already have opened a real, connected SqliteSource that nothing else references; skipping straight to .values() would abandon exactly that connection’s aiosqlite background thread forever — the same reasoning api.repository.Repository.close() applies to its own dedup-catalog cache one layer up.

Every source gets a close attempt regardless of whether an earlier one raised or hung – same “attempt all, then report” posture as Repository.close(), for the same reason.