DSM

Appearance

Sync Protocol ​

DSM replication has two related layers: the runtime control plane and the repair-oriented data plane.

End-To-End Flow ​

  1. Application calls register.put(...), lease.acquire(...), or crdt.update(...).
  2. DsmRuntime writes the change locally.
  3. RuntimePlatformSyncService emits a platform envelope.
  4. Peers apply the delta if they are already in sync.
  5. If a peer is behind, RuntimeDataPlaneReplicationService repairs it.
  6. The cluster returns to a converged state.

Two-Layer Picture ​

Repair and anti-entropy flow

Control Plane ​

RuntimePlatformSyncService replicates register, lease, and CRDT deltas over platform envelopes. This is the normal propagation path after the application mutates a collection handle.

Control-plane flow: a local mutation enters DsmRuntime, the runtime records it, and RuntimePlatformSyncService forwards the resulting delta for healthy peers to apply.

This path is for the healthy, steady-state case where peers are online and current.

Example mental model:

register.put(route-hint) commits locally first, then the sync service emits one delta envelope, and healthy peers apply the same update.

Data Plane Repair ​

RuntimeDataPlaneReplicationService repairs lagging peers using:

  • digest exchange
  • snapshot transfer
  • replay flows

That repair path matters when peers miss messages, rejoin after downtime, or need to reconcile state after cluster instability.

If peer-c falls behind, digest comparison detects the difference. DSM then uses replay when the peer can catch up from available history, or snapshot when replay is unsafe, unavailable, or more expensive.

Repair is why DSM does not rely only on best-effort delta delivery.

Typical cases that trigger repair:

  • a node restarted and missed some deltas
  • a node was partitioned briefly
  • a peer joined after the latest state already moved forward

Why The Split Exists ​

The control plane handles normal steady-state mutation flow. The data plane exists to restore correctness and convergence when normal flow is not enough.

More concretely:

  • control plane optimizes for fast mutation propagation
  • repair plane optimizes for correctness after loss, delay, or rejoin

Without the repair plane, a missed message could leave a node permanently stale.

Collection-Specific Behavior ​

  • registers replicate metadata-backed entity updates
  • leases replicate ownership state and lease transitions
  • CRDTs replicate updates plus state repair toward convergence

Examples:

  • register: route-hints delta announces a new address for edge-eu-west-1
  • lease: shard-owner delta announces owner change or renew result for shard-17
  • CRDT: request-counter delta carries update intent while repair transfers merged state when needed

Sequence Sketch ​

For a healthy peer, node-a writes a route hint and node-b applies the delta envelope. If node-b misses the delta, a later digest mismatch triggers replay or snapshot repair until node-b catches up.

Where To Read Executable Behavior ​

If you want to see the protocol working in code rather than prose, start with:

  • dsm-integration-test/.../TwoNodeIntegrationTest for two-node register replication
  • dsm-integration-test/.../RuntimeIntegrationTest for register, lease, and CRDT behavior together