synology_apm_repo.sdk.dedup.local_file_io module

Primitives for a local-file export destination: create it at full size, reserve its space, open it, and zero a range of it.

Export writes land at bucket-major, non-monotonic offsets, so the destination is created at its full logical size first and every write is positional (positional_io.pwrite).

On POSIX creating the file is one ftruncate. On Windows it is up to three Win32 calls, because the CRT’s file-extending truncate writes the zeros for real, at a cost proportional to the whole logical size.

synology_apm_repo.sdk.dedup.local_file_io.create_presized(dst, size, *, sparse, on_open=None)

Create or replace dst as an all-zero file of exactly size bytes.

Synchronous; call it through one asyncio.to_thread.

Parameters:
  • dst (Path) – Destination path. Replaced outright if it already exists.

  • size (int) – Final logical size, in bytes.

  • sparse (bool) – Whether unwritten regions may stay unallocated. Only affects Windows, where marking the file sparse stops NTFS zero-filling behind an out-of-order write; POSIX runs the same ftruncate either way. Neither branch reserves space (see preallocate).

  • on_open (Callable[[], None] | None) – Called once dst is created or truncated, before it is sized; not called if opening it fails, which leaves dst as it was.

synology_apm_repo.sdk.dedup.local_file_io.preallocate(fd, size)

Reserve disk space for all size bytes of the file behind fd, so a dense export fails before decoding if there is no room. Reserved, never-written ranges read as zero, so on True no zero-fill is needed.

Uses posix_fallocate where the platform has one and fcntl(F_PREALLOCATE) on macOS.

Parameters:
  • fd (int) – A file opened for writing whose content is all zero (a file create_presized just made); on macOS it is emptied and re-extended to size in the process.

  • size (int) – The file’s logical size, in bytes.

Returns:

True once the space is reserved (or there is nothing to reserve); False when this platform or filesystem cannot reserve it, in which case nothing was changed and the zeros must be written instead.

Raises:

OSError – Out of space or quota, or any other real failure — only “cannot reserve on this platform/filesystem” is reported as False.

Return type:

bool

synology_apm_repo.sdk.dedup.local_file_io.open_destination(dst)

Open an already-created dst for positional writes, returning its fd.

O_BINARY (0 off Windows) is required: Windows’ default text mode rewrites \n as \r\n in the exported bytes.

Parameters:

dst (Path | str) – Path to an existing file, normally one create_presized just sized.

Returns:

A writable file descriptor the caller owns and must close.

Return type:

int

synology_apm_repo.sdk.dedup.local_file_io.ZERO_FILL_BLOCK = 1048576

Size of one zero-fill pwrite.

synology_apm_repo.sdk.dedup.local_file_io.write_zeros_at(fd, offset, length)

Writes length zero bytes starting at offset, in ZERO_FILL_BLOCK-sized pwrite calls.

synology_apm_repo.sdk.dedup.local_file_io.zero_range(fd, offset, length)

Makes [offset, offset + length) of the file behind fd read as zero: deallocated where the platform and filesystem can (nothing is written), else overwritten in ZERO_FILL_BLOCK-sized pwrite calls. macOS deallocates only a 4096-aligned range; another is written.