synology_apm_repo.sdk.units.saas.stream module

SaasStream: the browsing layer shared by every SaaS workload.

A “stream” is saas/<connection_config_id>/<streamUuid>/ — its own db/{saas_snapshot,saas_version} (plain SQLite, never aHlT- enveloped even on an encrypted repository, unlike copy_meta_file’s per-file envelope) track snapshots (one per distinct backup job) and versions (one per run of that job) independently of db/copy_target_version.

A catalog Version carries no key directly usable against file_map; resolving it to the actual saas_obj means two SQLite lookups (SaasStream.stream_version_for) followed by a file_map path with an ambiguous middle segment (SaasStream.open_saas_obj).

stream_version is a monotonically increasing per-stream generation counter, not one-per-catalog-Version — several catalog Version rows can share one (multiple application-layer backups landing in one write session before a checkpoint). version_info.stream_version only records which generation a given version was written into; it is not a live pointer to where that content currently resides. A later generation’s composition record is always a superset of an earlier one’s (unwritten regions are inherited via the INHERIT bit, not re-written) — see FORMAT-SPEC.md: saas-addressing for the rotation/GC behavior this implies for a reader (server-side generation GC). So a catalog Version whose own recorded stream_version no longer has a file_map row is expected, routine rotation, not necessarily data loss — open_saas_obj forward-resolves to the nearest later generation that’s still live (bounded by stream_info. latest_complete_version, to exclude not-yet-committed generations) rather than requiring the literal requested one to still exist.

Constructing a SaasStream directly, outside SaasStreamCache, discards this per-instance resolution cache — every real caller goes through SaasStreamCache for exactly that reason; it is the only place that ever constructs one directly.

class synology_apm_repo.sdk.units.saas.stream.SaasStream(repo, connection_config_id, stream_uuid)

Bases: object

async close()

Release every sqlite connection this instance opened.

Settles each cache before closing its entries: this instance can be shared across concurrent callers (see SaasStreamCache), so a concurrent open_saas_obj() call can genuinely still be resolving a connection at the moment the owning SaasStreamCache evicts and closes this stream, and a value that lands afterward must not go unclosed.

The table cache goes with them: each Table it holds is bound to one of these connections, so keeping them would turn this instance’s documented lazy-reopen (every _*_connection() getter rebuilds from scratch) into a failure against a closed connection.

async stream_version_for(version)

Resolve a catalog Version’s (saas_snapshot_uuid, saas_version_id) down to this stream’s own stream_version, via snapshot_info.snapshot_uuid -> snapshot_id then version_info.(snapshot_id, version_id) -> stream_version.

async open_saas_obj(version)

Locate and open this catalog Version’s saas_obj.

Resolves version’s own recorded stream_version (stream_version_for), then forward-resolves it to the nearest still-live generation at or after that value (safe because a later generation’s composition record is always a superset of an earlier one’s, per the module docstring above) — an older generation routinely superseded and server-side garbage-collected is not itself an error.

Raises:

NotFoundError – no live generation exists anywhere from version’s own stream_version through this stream’s stream_info.latest_complete_version — a genuine gap, not routine rotation. ref names the originally-requested (not last-tried) path.

last_open_resolution(version)

The (requested, resolved) stream_version pair the most recent successful open_saas_obj(version) call actually used — None if that wasn’t the last call this instance made for this exact version (a different version, or none yet). Purely synchronous: reads a plain instance attribute open_saas_obj already set as its own side effect, no I/O and no re-derivation — for a caller (verify_reachable._saas_extents) that wants to report which generation actually got read, immediately after its own open_saas_obj call, without touching that method’s return type (which every other SaaS content-reading caller also uses and has no reason to care about this).

class synology_apm_repo.sdk.units.saas.stream.SaasStreamCache(repo, *, maxsize=8)

Bases: object

One SaasStream per distinct (connection_config_id, saas_stream_uuid) pair, reused across every version sharing that pair — a SaasStream’s own forward-resolution caches (its candidate-middle list, generation list, and latest_complete_version, each fetched at most once per instance since none change within one stream’s lifetime) only pay off when the same instance resolves every catalog Version for its stream, not a fresh one per version. Two real callers construct one of these each: units.verify_reachable’s _ReachabilityWalker, private and scoped to one verify run, and api.repository.Repository, one shared instance per opened DedupRepo for the whole repository’s lifetime — every SaaS provider (RawObjectProvider, SaasWorkloadProvider, TeamsChatProvider) borrows from the latter rather than opening its own private stream.

Bounded to at most maxsize concurrently warm streams (default _DEFAULT_STREAM_CACHE_SIZE) — see _stream_for for the eviction policy and open_saas_obj/_in_use for what keeps a stream still being read from safe across concurrent callers.

Every opened stream is closed on exit regardless of how the batch ended (async with).

async open_saas_obj(version)

version’s saas_obj, via the shared SaasStream for its (connection_config_id, saas_stream_uuid) — see SaasStream.open_saas_obj for resolution/error semantics.

Marked in-use for this call’s duration before _stream_for even runs, not after it returns: _stream_for itself can await (closing an evicted sibling) between inserting this call’s own key and handing the stream back, and a concurrent call for a different key racing through its own eviction loop during that window would otherwise see this key as not-yet-in-use and evict-and-close it out from under this call before it ever gets to mark it (see _in_use).

The finally block below tolerates key already being gone from _in_use — a concurrent close() of this whole cache clears it unconditionally, and this call’s own bookkeeping must not raise KeyError on top of (and masking) whatever close() racing this call already did to the stream itself.

last_open_resolution(version)

See SaasStream.last_open_resolution — meaningful only right after this cache’s own open_saas_obj(version) succeeded for the same version. Synchronous, and doesn’t itself resolve a new stream: None (not a lookup at all) if none has been built yet for this version’s own pair, or if it was since evicted — which can only mean open_saas_obj was never actually called for it, or was but the stream has since been recycled under pressure from other streams.

async close()