synology_apm_repo.sdk.dedup.export_workers module

The worker-process side of a multiprocess export: ExportExecutor (one repository, one destination) and the per-bucket-group task its workers run, export_bucket_group_worker. export_scheduler decides which windows go here.

Each worker process builds its Pool and opens the destination writer once, in its initializer; the module-level state below is that one process’s.

class synology_apm_repo.sdk.dedup.export_workers.ExportGroupWorkerArgs(stream_id, bucket_id, runs, size, dst_offset)

Bases: object

One (stream_id, bucket_id) group’s (picklable) work for export_bucket_group_worker.

stream_id: StreamId
bucket_id: BucketId
runs: list[ChunkRun]
size: int
dst_offset: int
class synology_apm_repo.sdk.dedup.export_workers.ExportExecutor(pool_descriptor, sink_descriptor, *, max_workers=None)

Bases: BoundProcessPool

Worker processes for export_to_writer, bound for their lifetime to one repository (pool_descriptor) and one destination (sink_descriptor). Share one across the calls of a single logical export to that destination. max_workers is the process count (concurrency.default_worker_count() when None).

Raises:

ValueError – A descriptor cannot be pickled for the workers.

accepts(pool_descriptor, target)

Whether an export of pool_descriptor’s repository into target may be dispatched to these workers.

synology_apm_repo.sdk.dedup.export_workers.export_bucket_group_worker(args)

The multiprocess path’s per-bucket-group work item; returns the bytes this group wrote.