Production Readiness
Use this checklist before treating a DSM-backed workflow as production ready. It is intentionally concrete: each item should be tied to configuration, code, tests, or observable runtime evidence.
Identity And Scope
| Check | Why it matters |
|---|---|
clusterId is stable and environment-specific. | Prevents staging, production, and local nodes from joining the same runtime fabric. |
serviceId is stable per service family. | Prevents unrelated services from exchanging collection traffic. |
| Node IDs are explicit where operational identity matters. | Makes diagnostics, lease ownership, and incident timelines understandable. |
| Collection locators are reviewed. | Prevents unrelated domains from sharing one register because the name was convenient. |
Collection Modeling
| Check | Why it matters |
|---|---|
| Register, lease, and CRDT choices are justified. | Wrong collection type creates correctness bugs that repair cannot fix. |
schema-id policy is defined. | Payload compatibility is a deployment contract. |
| Record entities use compatible schemas across rolling deployments. | RecordCodec fingerprint mismatches discard incompatible remote upserts. |
| Lease side effects are fenced. | Lease ownership without downstream fencing is not a correctness boundary. |
| QUORUM mode is considered for lease workflows that cannot tolerate partition-local ownership. | QUORUM rejects operations while membership is unstable or majority is unavailable. |
Membership And Repair
| Check | Recommended baseline |
|---|---|
| Local development uses standalone membership. | Keep first integration simple. |
| Cloud/Kubernetes deployments use unicast gossip with explicit seeds or DNS seeds. | Avoid relying on multicast where it is not supported. |
| SWIM suspicion is enabled unless there is a clear reason to disable it. | Reduces false failure transitions. |
| Proactive anti-entropy is enabled. | Default: dsm.sync.anti-entropy.sweep-enabled=true. |
| Anti-entropy outcomes are monitored. | Alert on sustained failed or skipped outcomes. |
| Relay health is wired where relay peers exist. | Failed relay peers should be excluded from repair selection. |
Security
| Check | Recommended baseline |
|---|---|
dsm.security.enabled=true in production clusters. | Enables secure envelope signing and replay protection. |
| Cluster secret comes from a secret manager or environment, not source control. | Prevents credential leakage. |
min-acceptable-key-version is part of key rotation. | Prevents key downgrade. |
| Nonce sender capacity is sized and bounded. | Default 1600, Spring-valid range 100..10000. |
| Sender rate limiting is enabled. | Default ban threshold is 10 consecutive verification failures. |
| Payload encryption is explicitly decided. | If enabled, plaintext inbound envelopes are rejected before HMAC verification. |
Observability
| Signal | Action |
|---|---|
| Runtime readiness and diagnostics | Check before serving traffic and during incidents. |
| Cluster size | Alert on unexpected shrink or churn. |
| Anti-entropy sweep outcomes | Alert on repeated repair failures. |
| Schema fingerprint mismatch | Treat as incompatible deployment until proven otherwise. |
| Lease acquire/renew/verify/fencing metrics | Watch for churn, stale holders, and downstream rejects. |
| Security auth failure, sender ban, replay rejection | Treat sustained rates as security or rollout incidents. |
| Trace context injection | Use to confirm DSM traffic is joined to request traces when enabled. |
Test Evidence To Require
| Area | Minimum evidence |
|---|---|
| Collection behavior | Unit tests for entity metadata, codecs, and collection wrapper semantics. |
| Two-node convergence | Integration test with fake membership and explicit sync services. |
| Partition healing | Chaos or integration test showing anti-entropy catch-up. |
| Lease correctness | Tests for acquire, renew, transfer, verify-current, and stale fencing rejection. |
| Federation | Tests for bootstrap, live forwarding, relay failure/recovery, and repair outcomes. |
| Security | Tests for invalid HMAC, nonce rejection, sender ban, and encryption downgrade behavior. |
| Soak | Seed-driven or long-running lane for consistency invariants under random operations. |
Release Gate
Before release:
bash
npm run build
DSM_SOURCE_ROOT=/path/to/dsm npm run docs:checkBefore production rollout:
- Confirm every DSM node runs a compatible binary and schema set.
- Confirm Spring properties match the intended cluster mode and security posture.
- Confirm alerts exist for repair, lease, security, and schema mismatch signals.
- Confirm rollback does not violate wire compatibility or schema compatibility.
- Confirm operators know which error code category maps to configuration, security, consistency, sync, transport, codec, lease, or change-stream failures.