synology_apm_repo.sdk.dedup.repository module¶
DedupRepo: opens one repository root and ties
RepoInfo/key material/Pool/CompositionReader together behind
open_file()/open_composition().
Repository-root path constants (FORMAT-SPEC.md: Locating the repository
root, and directory layout) are uniform across both layout kinds —
<repo_root>/@data/Pool, <repo_root>/@data/Composition,
<repo_root>/db/; only layout.repo_root itself differs (the vault
root directly for RepoKind.VAULT, or
@ActiveProtectData/<repoId> for RepoKind.OBJECT_STORE).
- synology_apm_repo.sdk.dedup.repository.DB_SOURCE_NAMES = frozenset({'connection_config', 'copy_target_version', 'copy_target_version_meta', 'file_map', 'file_meta', 'vault_link_key', 'workload_config'})¶
Every
db/<name>DedupRepo.db()serves (copy_target_fileis aliased tocopy_target_versionbefore lookup). This closed set is what bounds_db_sources: add a name here when a new caller needs one.
- synology_apm_repo.sdk.dedup.repository.FILE_MAP_STATUS_COMPLETE = 2¶
The
db/file_map.statusof a complete file, the only onelocate_fileaccepts; see its Raises section for the others (FORMAT-SPEC.md: db/file_map). Public becauseunits.saas.streampre-filters candidates with it.
- class synology_apm_repo.sdk.dedup.repository.FileLocation(stream_id, session_id, comp_offset, block, file_size)¶
Bases:
objectOne
db/file_maprow, resolved to whatDedupRepo.open_compositionneeds plus (when available)file_meta.file_size.statusis not carried here —locate_filealready rejects every row except Complete (FORMAT-SPEC.md: db/file_map) before constructing one.- comp_offset: CompOffset¶
- class synology_apm_repo.sdk.dedup.repository.DedupRepo(store, layout, info, dir_cache, *, fixed_dir_cache=None, vault_key=None, limits=CacheLimits(bucket_readers=16, bucket_readers_interactive=64, chunks=4096, composition_records=64, composition_pages=128, saas_streams=8, dir_scan=256, dir_fixed=64, allocation_tables=256, forward_resolutions=4096), verify=VerifyPolicy(ciphertext_crc=False, fingerprint=False))¶
Bases:
AsyncClosingOne opened repository: its
Pool, a cache of read-onlydb/<name>sqlite connections, and a repo-wideCompositionRecordcache shared by everyCompositionReaderit builds. Hands back aDedupFileviaopen_file/open_composition. Async context manager;close()releases the connections and caches.- Variables:
layout – The
RepoLayoutthis repository was opened from.info – The parsed
repo_info.
- caches¶
Every cache this repository owns, by name; see
CacheManager.
- property store: ObjectStore¶
The
ObjectStorethis repository was opened against, for callers above Catalog that need a raw read outsidePool/CompositionReader/db()(e.g. catalog’s link-key display-name lookup).
- property vault_key: bytes | None¶
The resolved VaultKey (
Nonefor an unencrypted repository or one opened without key material), for callers that decrypt outsidePool(e.g.catalog.version.open_target_dbpeeling anaHlT-envelopedtarget.db).
- property dir_cache: DirCache¶
This repository’s
DirCachefor bulk-scannable directories (Pool leaves, Composition, a SaaS stream’sdb/).
- async repo_info_path()¶
The store path of the
repo_infogeneration this repository uses — on object storage the committed one, asopenreads it.- Raises:
NotFoundError – No usable
repo_infoexists.
- new_pool(*, verify)¶
A separate
Poolover this repository’s chunks, with its own caches (sized by this repository’sCacheLimits) andverifypolicy; it shares this repository’s directory cache.
- async classmethod open(store, layout, keys=None, *, limits=CacheLimits(bucket_readers=16, bucket_readers_interactive=64, chunks=4096, composition_records=64, composition_pages=128, saas_streams=8, dir_scan=256, dir_fixed=64, allocation_tables=256, forward_resolutions=4096), verify=VerifyPolicy(ciphertext_crc=False, fingerprint=False))¶
Open
layout. Reads onlyrepo_infoand, givenkeysfor an encrypted repository, the wrapped VaultKey record; never scans Pool or Composition.- Parameters:
store (ObjectStore) – The repository’s
ObjectStore.layout (RepoLayout) – Where the repository root lives.
keys (KeyMaterial | None) – Key material;
Noneopens without a VaultKey.limits (CacheLimits) – Bounds of every cache this repository owns: the
Pool’s, the directory caches and the composition-record cache.verify (VerifyPolicy) – The per-chunk checks every
Poolread runs.
- Raises:
NotFoundError –
repo_infois missing.DataCorruptError –
repo_infoor the key record is corrupt.FormatError –
repo_infois truncated.KeyMismatchError –
keysfails the GCM tag check (raised here because AES-CTR would otherwise decrypt to garbage silently).KeyRequiredError –
keysnames a user key with no wrapped VaultKey on record.
- async db(name)¶
Open (and cache) a connection to
db/<name>.Read-only against the store: the fast path opens the real file immutable; the slow path opens a private materialized copy read-write (so an index hint can take effect) whose writes never reach the store.
nameis first resolved throughPHYSICAL_NAME_ALIASES("copy_target_file"lives inside"copy_target_version"’s file), so both names share one connection.On a
RepoKind.OBJECT_STORElayout the generation is chosen by FORMAT-SPEC.md: Multi-generation selection (resolve_generation), not the largest.<N>suffix, since a generation can exist on disk before its transaction commits.db/<name>is always raw (neveraHlT/zstd-enveloped).- Parameters:
name (str) – Logical
db/<name>.- Returns:
The cached connection;
close()owns its lifetime.- Raises:
NotFoundError – No such db file.
DataCorruptError – The file is not a readable sqlite database, or is gone although db/ lists it.
ResourceLimitExceededError – The slow path’s private copy doesn’t fit in the temp directory with its free-space reserve left.
ValueError –
nameis not inDB_SOURCE_NAMES.RuntimeError – The repository is closed.
- Return type:
Connection
- async locate_file(path)¶
Look up
pathindb/file_mapand supplement it withfile_meta.file_sizewhen a matching row exists.file_meta.pathis only unique perconnection_config_id; this takes the first match.- Raises:
NotFoundError – no row for
pathat all, or itsstatusis Initialized/Written/Compacted (0/1/3) — not yet, or no longer, resolvable content (FORMAT-SPEC.md: db/file_map).DataCorruptError – the row’s
statusis Corrupted/Tainted (4/5) — the format itself flags this data as known-bad.
- async file_map_paths_with_prefix(prefix, *, status=None)¶
Every
db/file_mappath starting withprefix, sorted.%/_/\inprefixmatch literally.status, when given, restricts to rows at exactly thatfile_map.statusvalue (FORMAT-SPEC.md: db/file_map); without it, unlikelocate_file, no status filtering applies.
- async open_file(path, *, fallback_size=None)¶
Resolve
pathviadb/file_mapand return a ready-to-readDedupFile, sized byfile_meta.file_size, or byfallback_sizewhenfile_metarecords none for it.- Raises:
NotFoundError – see
locate_file.DataCorruptError – see
locate_file.
- open_composition(stream_id, session_id, comp_offset, size=None)¶
A
DedupFilefor a(stream_id, session_id, comp_offset)triple, withsizeas its known size (None: unknown). ItsCompositionReadershares this repository’s composition-record cache.
- composition_reader(stream_id, session_id, *, shared_cache=False)¶
A
CompositionReaderfor one session’s Composition, caching its records in this repository’s shared cache only withshared_cache.
- async close()¶
Close every cached sqlite connection (and any temp file or directory a slow-path materialization created) and drop every other cache this repository holds, via
caches.invalidate_all().In-flight fetches are settled first, and every resource gets a close attempt, bounded by a timeout, even if an earlier one failed.
- Raises:
ExceptionGroup – One or more resources failed to close.