synology_apm_repo.sdk.dedup.export_workers module¶
The worker-process side of a multiprocess export: ExportExecutor (one
repository, one destination) and the per-bucket-group task its workers run,
export_bucket_group_worker. export_scheduler decides which windows go
here.
Each worker process builds its Pool and opens the destination writer once,
in its initializer; the module-level state below is that one process’s.
- class synology_apm_repo.sdk.dedup.export_workers.ExportGroupWorkerArgs(stream_id, bucket_id, runs, size, dst_offset)¶
Bases:
objectOne
(stream_id, bucket_id)group’s (picklable) work forexport_bucket_group_worker.
- class synology_apm_repo.sdk.dedup.export_workers.ExportExecutor(pool_descriptor, sink_descriptor, *, max_workers=None)¶
Bases:
BoundProcessPoolWorker processes for
export_to_writer, bound for their lifetime to one repository (pool_descriptor) and one destination (sink_descriptor). Share one across the calls of a single logical export to that destination.max_workersis the process count (concurrency.default_worker_count()whenNone).- Raises:
ValueError – A descriptor cannot be pickled for the workers.
- accepts(pool_descriptor, target)¶
Whether an export of
pool_descriptor’s repository intotargetmay be dispatched to these workers.
- synology_apm_repo.sdk.dedup.export_workers.export_bucket_group_worker(args)¶
The multiprocess path’s per-bucket-group work item; returns the bytes this group wrote.