synology_apm_repo.sdk.dedup.verify_checks module¶
Byte-level integrity check primitives for the reachability-scoped walk
(dedup.verify_walk, driven by units/verify_reachable.py) behind
Repository.verify()/Catalog.verify().
Each function checks what it is handed (an open reader, or for
check_repo_info the repository) and returns Findings; choosing what to check is the walker’s job.
- synology_apm_repo.sdk.dedup.verify_checks.STALE_ROTATED_SUFFIX = 'possibly a stale/rotated reference'¶
Appended (behind an em dash) to a
NotFoundError’s message in aSymptom.DATA_MISSINGFinding:unresolvable_finding’s defaultmissing_suffix.
- async synology_apm_repo.sdk.dedup.verify_checks.check_repo_info(repo)¶
repo_info’s header/CRC/JSON-parse check; repository-wide, soverify_reachable()runs it once.
- async synology_apm_repo.sdk.dedup.verify_checks.check_composition_header(reader, *, path)¶
The session’s
subID=0sub-file header (CompositionReader.verify_header), whichCompositionReaderitself skips. Call once per distinct(stream_id, session_id).
- async synology_apm_repo.sdk.dedup.verify_checks.check_record_head(reader, comp_offset, *, path)¶
Read and parse the 32-byte
RecordHeadatcomp_offset(parse_record_headchecks magic andhead_crc). Returns(finding, None)on failure,(None, record_head)on success.
- synology_apm_repo.sdk.dedup.verify_checks.should_thread_chunk_map_crc(map_array_bytes)¶
Whether
verify_chunk_map_crc(map_array_bytes, ...)is worth running viaasyncio.to_thread()(see_CRC_THREAD_HOP_MIN_BYTES).
- async synology_apm_repo.sdk.dedup.verify_checks.verify_chunk_map_crc_threaded(map_array_bytes, expected_crc)¶
format.composition.verify_chunk_map_crc, moved to a thread whenshould_thread_chunk_map_crcsays the array is large enough to block the event loop.- Raises:
DataCorruptError –
map_array_bytesdoesn’t matchexpected_crc.
- async synology_apm_repo.sdk.dedup.verify_checks.check_map_and_attr_crc(reader, comp_offset, record_head, *, path)¶
mapCrcover the whole chunk-map array plus (when present)attrCrcover the trailing JSON attribute blob, in one merged read.A
map_crcmismatch first tries Redundancy-blob self-repair (_attempt_map_crc_repair); a confirmed repair is reported asSymptom.REPAIRED_VIA_PARITY. Nothing is checked whenrecord_head.map_num == 0.- Returns:
(findings, verified_array, repaired_array), at most one array set: the chunk-map array bytesmap_crcconfirmed as read, or the ones a parity repair confirmed. A caller that re-reads this array must pass a non-Nonerepaired_arraytoCompositionRecord.seed_pages_from_array, or it reads the still-corrupted on-disk bytes.- Return type:
- async synology_apm_repo.sdk.dedup.verify_checks.check_bucket_structure(store, reader, *, known_file_size=None)¶
The chunk-independent checks of one bucket:
expected_bucket_sizeagainst the on-disk size, and the ChunkCrcStore trailer’s self-consistency (BucketReader.ensure_chunk_crc_store, which also warms the cachecheck_chunk_ciphertext_crcreuses). Header/SizeStore CRC are checked byBucketReader.open; this adds aSymptom.REPAIRED_VIA_PARITYfinding when its SizeStore self-repair succeeded.A legacy uncompressed-layout bucket has neither an
expected_bucket_sizeformula nor a ChunkCrcStore trailer; both checks are skipped.known_file_sizeis the on-disk size when the caller already has it (Pool.bucket_size, from the directory listing); without it, and unless a SizeStore repair fetched it, this asksstore.size.Takes a bare
storeso a worker process holding only anObjectStorecan call it.
- async synology_apm_repo.sdk.dedup.verify_checks.check_chunk_ciphertext_crc(reader, chunk_idx, raw=None)¶
One chunk’s stored bytes against its ChunkCrcStore entry (FORMAT-SPEC.md: ChunkCrcStore & Redundancy):
rawwhen the caller already holds them (e.g. fromread_raw_chunks), else read here. Nothing is decrypted, so no vault key is needed.
- async synology_apm_repo.sdk.dedup.verify_checks.check_chunk_ciphertext_crcs(reader, chunk_indices)¶
Batched
check_chunk_ciphertext_crc: fetches every chunk viareader.read_raw_chunks(one merged read per contiguous run), then checks each against itsChunkCrcStoreentry.read_raw_chunksdiscards earlier runs when a later one fails, so any failure but aStorageBackendError(raised) falls back to onecheck_chunk_ciphertext_crccall per chunk.