Production safety
CloneDB only reads the source; it never writes to it. But reads still cost the source CPU, disk I/O and buffer cache, and a run holds one read snapshot open for its whole duration. If your source is a live production database, follow this guide so a clone never degrades production.
In short: point the source at a read replica, keep clones narrow and off-peak, run
unattended clones with --yes, and keep the session safety defaults.
Prefer a read replica
Section titled “Prefer a read replica”Point the source at a read replica, not the primary, so the read load lands off the hot path:
default_source = "replica"
[connections.replica]host = "replica.db.internal"port = 5432database = "app"user = "clonedb_ro"sslmode = "verify-full"
[connections.replica.auth] provider = "gcp_secret" # or any provider; secrets never live in this file secret = "app-db-readonly"Or pick it per run without editing the config: clonedb run --from replica (a connection
name or a URL).
A replica is not a free pass:
- With
hot_standby_feedback = on, a long clone on the replica holds back the primary’s vacuum too, because the replica’s snapshot propagates to the primary’s xmin horizon. - Long queries on a replica can be cancelled by recovery conflicts. Tune
max_standby_streaming_delayon the replica, or keep clones short (see below). - A replica dedicated to reporting or ETL is ideal.
Keep the read load gentle
Section titled “Keep the read load gentle”- Parallel readers. A run reads on
parallel_readersconnections at once (default4), all sharing the run’s snapshot. Parallelism concentrates load: shorter, but spikier. Against a busy primary, run serially with--parallel-readers 1or set[defaults] parallel_readers = 1. Raise it only against a replica or a box you’re free to saturate. - Connection budget. On the source, a run holds the snapshot owner (open for the whole
run), the main read connection, and the parallel readers: up to
parallel_readers+ 2 connections, so 6 with the default and never more than 8. The target load uses a separate pool. Account for this against a tightmax_connectionsor a pooler’s limit.
Mind the snapshot hold
Section titled “Mind the snapshot hold”A run opens one REPEATABLE READ transaction and holds it for the entire clone. Key
collection and row emission share that snapshot so the subset is referentially consistent.
While it’s open it holds back the xmin horizon, so autovacuum can’t reclaim dead tuples
created during the run. On a write-heavy source that means transient table and index
bloat for as long as the clone runs.
The cost scales with duration, not subset size as such. Keep runs short:
- Seed narrowly. A tightly scoped subset (a few accounts, a recent date window) finishes
fast. Check the
run --dry-runestimates first. - Use
--yesfor unattended clones. The interactive “write to target?” prompt waits with the snapshot open, pinning it (and its connections) for as long as nobody answers. The heartbeat keeps a live run’s transactions alive while it waits, so the idle timeout won’t rescue you here. Don’t leave a prompt hanging against a busy source. - Run large clones off-peak, when write volume (and so bloat) is low.
- Use a replica so any bloat lands there, subject to the
hot_standby_feedbackcaveat above.
Monitor a long clone in pg_stat_activity. clonedb’s sessions carry
application_name = 'clonedb' by default:
SELECT pid, state, now() - xact_start AS txn_ageFROM pg_stat_activityWHERE application_name = 'clonedb';clonedb also logs per-phase timing and warns about any query slower than
slow_query_warn_secs. Total run time is how long the snapshot
was held.
Session safety settings
Section titled “Session safety settings”Every source session gets protective defaults, tunable per connection (full reference: Session settings):
| Setting | Default | Protects against |
|---|---|---|
idle_in_transaction_session_timeout | 3min | A hung or killed clonedb pinning the snapshot. The server ends the session and vacuum can proceed. |
lock_timeout | 10s | A clonedb query queueing indefinitely behind a lock, such as a migration’s ACCESS EXCLUSIVE. |
tcp_keepalives_idle / _interval / _count | 60 / 10 / 6 | A dead peer going unnoticed: the server detects a dead clonedb, and clonedb detects a dead server, in about two minutes. |
application_name | "clonedb" | Not knowing which sessions are clonedb’s. |
tcp_user_timeout | off | Leave off: it can abort a healthy run whose COPY is stalled by a slow target. |
The server settings are applied with set_config(…, true) at the start of every
transaction clonedb opens, after any SET TRANSACTION SNAPSHOT and before the first data
query. They’re in force for all of clonedb’s work and end with the transaction; nothing is
set for the session, so nothing outlives clonedb’s transactions on a server connection.
The server keepalives therefore apply while clonedb has a transaction open, which is when a
lost clonedb would pin a snapshot or hold locks. application_name goes in the startup
packet and is set in every transaction too, so it shows even behind a pooler that doesn’t
pass the startup parameter on. clonedb’s own socket keepalives apply for the life of the
connection.
A live run doesn’t trip the idle timeout: clonedb sends SELECT 1 on every open source
transaction, busy or not, every min(60 s, timeout / 3), using the timeout the server
actually has. The timeout only fires once clonedb has actually stopped, and then the
server ends its transactions and releases the snapshot. clonedb’s values replace any
ALTER ROLE / ALTER DATABASE default; set a key to 0 to switch one off.
lock_timeout covers only clonedb waiting on others. The run’s transactions keep their
ACCESS SHARE locks until they end, so a migration’s ALTER TABLE on a table the run has
read waits for the run, and queries queued behind the migration wait too. Give migrations
their own lock_timeout, or don’t clone during deploys.
The target gets application_name and the keepalives, lock_timeout only if its
connection block sets it, and never the idle timeout or the heartbeat.
Refused settings. clonedb applies the settings inside a savepoint, so a refusal leaves the transaction usable. If the server refuses one (common on managed services), clonedb logs a warning naming it once and continues the run without it. Check the log on a managed source.
Connection poolers
Section titled “Connection poolers”clonedb works behind a session- or transaction-mode pooler (e.g. PgBouncer with
pool_mode = session or pool_mode = transaction) with no setting on its side:
- Its settings end with each transaction, so they never reach the pooler’s other clients.
- The pooler doesn’t need prepared-statement support (e.g. PgBouncer with
max_prepared_statements = 0is fine). clonedb runs every statement inside a transaction (catalog reads such as introspection, dry-run estimates, the--watchfingerprint and the target partition pre-flight each run in one shortREAD ONLYtransaction) and closes its prepared statements before the transaction ends. - Behind a pooler, clonedb’s socket keepalives (and client
tcp_user_timeout) reach only the pooler, and the server keepalive settings tune the pooler-to-server socket for the length of clonedb’s transaction. The idle timeout and heartbeat are the guard that works end to end; keepidle_in_transaction_session_timeouton. - Size the pooler’s pool for it. A run holds
parallel_readers+ 2 source transactions open for its whole duration (see the connection budget), each pinning a server connection, plus one on the target. The pooler’s pool must allow that many concurrent source server connections (and one for the target), or the readers wait in the pooler’s queue. - Statement-mode pooling is not supported. clonedb needs multi-statement transactions.
clonedb doesn’t use the startup options parameter for its settings, because PgBouncer
rejects it by default (unless it’s listed in ignore_startup_parameters). The same goes
for an options= you put in a URL. Of clonedb’s settings, only application_name goes in
the startup packet, and it’s set in every transaction as well. A pooler relays the same back-pressure as a direct connection, so leave
tcp_user_timeout off either way.
sslmode follows libpq semantics:
sslmode | Encrypts | Verifies | Use for |
|---|---|---|---|
verify-full | yes | chain and hostname | A direct cloud connection (the default for a direct host). |
verify-ca | yes | chain only | A tunnel to a host whose CA you can supply via sslrootcert. |
require / prefer | yes | nothing | A tunnel to localhost, or a server whose certificate chain verification rejects (e.g. AlloyDB’s CA-leaf certificate). |
disable | no | — | Local-only, plaintext. |
For a component-mode provider (one that fetches just a password or token), an omitted
sslmode defaults to require on a loopback/tunnel host and verify-full on a direct
host. A full URL with no sslmode encrypts opportunistically but does not verify,
like libpq. Put ?sslmode=verify-full in the URL to verify.
Checklist
Section titled “Checklist”- The source points at a read replica (or a box that’s safe to load), not the busy primary.
- On a primary,
parallel_readersis1. - The subset is scoped narrowly and
run --dry-runlooks right. - Large clones run off-peak.
- Unattended runs pass
--yes, so no prompt holds the snapshot. -
parallel_readers+ 2 connections fit the source’smax_connectionsor pooler budget. -
idle_in_transaction_session_timeoutandlock_timeoutare left on;tcp_user_timeoutis left off. - Any pooler runs in session or transaction mode (never statement mode), with a pool
of at least
parallel_readers+ 2 source server connections (and 1 target). -
sslmodefits:verify-fulldirect,verify-ca+sslrootcertfor a verified tunnel,requireonly where verification is impossible.