synology_apm_repo.sdk.dedup.verify_checks module

Byte-level integrity check primitives for the reachability-scoped walk (dedup.verify_walk, driven by units/verify_reachable.py) behind Repository.verify()/Catalog.verify().

Each function checks what it is handed (an open reader, or for check_repo_info the repository) and returns Findings; choosing what to check is the walker’s job.

synology_apm_repo.sdk.dedup.verify_checks.STALE_ROTATED_SUFFIX = 'possibly a stale/rotated reference'

Appended (behind an em dash) to a NotFoundError’s message in a Symptom.DATA_MISSING Finding: unresolvable_finding’s default missing_suffix.

async synology_apm_repo.sdk.dedup.verify_checks.check_repo_info(repo)

repo_info’s header/CRC/JSON-parse check; repository-wide, so verify_reachable() runs it once.

async synology_apm_repo.sdk.dedup.verify_checks.check_composition_header(reader, *, path)

The session’s subID=0 sub-file header (CompositionReader.verify_header), which CompositionReader itself skips. Call once per distinct (stream_id, session_id).

async synology_apm_repo.sdk.dedup.verify_checks.check_record_head(reader, comp_offset, *, path)

Read and parse the 32-byte RecordHead at comp_offset (parse_record_head checks magic and head_crc). Returns (finding, None) on failure, (None, record_head) on success.

synology_apm_repo.sdk.dedup.verify_checks.should_thread_chunk_map_crc(map_array_bytes)

Whether verify_chunk_map_crc(map_array_bytes, ...) is worth running via asyncio.to_thread() (see _CRC_THREAD_HOP_MIN_BYTES).

async synology_apm_repo.sdk.dedup.verify_checks.verify_chunk_map_crc_threaded(map_array_bytes, expected_crc)

format.composition.verify_chunk_map_crc, moved to a thread when should_thread_chunk_map_crc says the array is large enough to block the event loop.

Raises:

DataCorruptError – map_array_bytes doesn’t match expected_crc.

async synology_apm_repo.sdk.dedup.verify_checks.check_map_and_attr_crc(reader, comp_offset, record_head, *, path)

mapCrc over the whole chunk-map array plus (when present) attrCrc over the trailing JSON attribute blob, in one merged read.

A map_crc mismatch first tries Redundancy-blob self-repair (_attempt_map_crc_repair); a confirmed repair is reported as Symptom.REPAIRED_VIA_PARITY. Nothing is checked when record_head.map_num == 0.

Returns:

(findings, verified_array, repaired_array), at most one array set: the chunk-map array bytes map_crc confirmed as read, or the ones a parity repair confirmed. A caller that re-reads this array must pass a non-None repaired_array to CompositionRecord.seed_pages_from_array, or it reads the still-corrupted on-disk bytes.

Return type:

tuple[list[Finding], bytes | None, bytes | None]

async synology_apm_repo.sdk.dedup.verify_checks.check_bucket_structure(store, reader, *, known_file_size=None)

The chunk-independent checks of one bucket: expected_bucket_size against the on-disk size, and the ChunkCrcStore trailer’s self-consistency (BucketReader.ensure_chunk_crc_store, which also warms the cache check_chunk_ciphertext_crc reuses). Header/SizeStore CRC are checked by BucketReader.open; this adds a Symptom.REPAIRED_VIA_PARITY finding when its SizeStore self-repair succeeded.

A legacy uncompressed-layout bucket has neither an expected_bucket_size formula nor a ChunkCrcStore trailer; both checks are skipped.

known_file_size is the on-disk size when the caller already has it (Pool.bucket_size, from the directory listing); without it, and unless a SizeStore repair fetched it, this asks store.size.

Takes a bare store so a worker process holding only an ObjectStore can call it.

async synology_apm_repo.sdk.dedup.verify_checks.check_chunk_ciphertext_crc(reader, chunk_idx, raw=None)

One chunk’s stored bytes against its ChunkCrcStore entry (FORMAT-SPEC.md: ChunkCrcStore & Redundancy): raw when the caller already holds them (e.g. from read_raw_chunks), else read here. Nothing is decrypted, so no vault key is needed.

async synology_apm_repo.sdk.dedup.verify_checks.check_chunk_ciphertext_crcs(reader, chunk_indices)

Batched check_chunk_ciphertext_crc: fetches every chunk via reader.read_raw_chunks (one merged read per contiguous run), then checks each against its ChunkCrcStore entry.

read_raw_chunks discards earlier runs when a later one fails, so any failure but a StorageBackendError (raised) falls back to one check_chunk_ciphertext_crc call per chunk.