Configuration

CloneDB is driven by a single clonedb.toml. It’s generated from your schema and safe to hand-edit; clonedb sync-config reconciles it with schema changes while preserving your edits and comments.

The guiding idea: relationships are introspected, not enumerated. The engine walks the live foreign-key graph automatically. The config holds only what introspection can’t know — your filters, a little follow/boundary policy, virtual FKs, and connection details.

The blocks, top to bottom:

Block	Purpose
[meta]	Drift-detection metadata (maintained by sync-config).
[connections.*]	A named pool of databases; each names a credential provider for its secret.
default_source / default_target	Top-level keys: which connection a bare run reads from / writes to.
env_file	Top-level key: an opt-in .env file to load into the environment before resolving credentials.
[schemas]	Which schemas participate.
[defaults]	Policy applied to every table unless overridden.
[tables.*]	Per-table seeds, filters, and policy.
[scopes.*]	Named, persisted seed sets for day-to-day work.
[virtual_foreign_keys]	Relationships with no DB constraint (terse map + [[virtual_foreign_key_rules]]).
[[relationship_overrides]]	Stop or redirect traversal on one edge.

[meta]

Drift-detection metadata. Maintained by sync-config — you rarely touch it by hand, and the whole block may be omitted from a hand-written config (the hash falls back to an empty string, populated on the next sync).

[meta]
schema_hash = "sha256:8f3c9a2b…"   # hash of the introspected schema, for drift detection

Key	Default	Description
schema_hash	""	Hash of the introspected schema (sha256:<hex>), for drift detection.

Note

Older configs may still carry version and generated_at keys. These are legacy — they parse without error but are ignored (never read, never rewritten). schema_hash is the only live field.

Connections

Connections are a named pool. Each [connections.<name>] block describes one database: its non-secret target (host/port/db/user/sslmode) plus a single […auth] block whose provider supplies the secret. Secrets never live in the config. Names are arbitrary — production, staging, dev, local — and source/target are just conventional defaults. A run moves data between any two connections (see Named connections & defaults).

[connections.source]
host     = "app-replica.abc123.us-east-1.rds.amazonaws.com"
port     = 5432            # defaults to 5432 when omitted
database = "app"
user     = "readonly"
sslmode  = "verify-full"   # verify-full for a direct cloud host; require through a tunnel

  [connections.source.auth]
  provider = "aws_secret"  # password from AWS Secrets Manager
  region   = "us-east-1"
  secret   = "app/db/readonly"
  field    = "password"    # JSON key; omit for a raw-string secret

The simplest target is a full URL (including password) from an env var:

[connections.target]
  [connections.target.auth]
  provider = "env"
  url_env  = "CLONEDB_TARGET_URL"   # postgres://user:pass@host:5432/app_dev?sslmode=require

Named connections & defaults

Define as many connections as you like, then point two top-level keys at them. default_source/default_target must come before the [connections.*] blocks — they’re top-level keys, and TOML would otherwise scope them into the preceding connection table (clonedb errors loudly if they’re misplaced):

default_source = "production"   # what a bare `run` reads from
default_target = "local"        # what a bare `run` writes to

[connections.production]
host = "prod-replica.abc123.us-east-1.rds.amazonaws.com"
database = "app"
user = "readonly"
  [connections.production.auth]
  provider = "aws_secret"
  region   = "us-east-1"
  secret   = "prod/db/readonly"

[connections.local]
  [connections.local.auth]
  provider = "env"
  url_env  = "LOCAL_DATABASE_URL"

clonedb run resolves each side independently:

Side	Resolution order
source	--from <name|url> → default_source → a connection named source → $CLONEDB_SOURCE_URL
target	--to <name|url> (or --target) → default_target → a connection named target

--from/--to take either a connection name or a raw URL, so you can move between any two named connections (clonedb run --from production --to staging) or override with a one-off URL. default_source/default_target are optional — omit them and rely on the source/target convention or the flags.

Connection fields

These non-secret fields apply to any connection (the [connections.<name>] block itself):

Field	Default	Description
host	—	Non-secret connection host.
port	5432	Non-secret connection port.
database	—	Non-secret database name.
user	—	Non-secret database user.
sslmode	inferred	TLS mode: disable / require / prefer / verify-ca / verify-full.
sslrootcert	—	Path to a CA-certificate PEM (e.g. a private CA) that extends the trust store for verify-ca / verify-full. Ignored by the non-verifying modes.

CloneDB follows libpq’s sslmode semantics, so it behaves like psql:

Mode	Behavior
disable	No encryption.
require / prefer	Encrypt, but do not validate the certificate. Matches libpq — this is what lets CloneDB talk to servers with self-signed certs (e.g. AlloyDB, whose leaf cert is a CA). sslrootcert is ignored here.
verify-ca	Encrypt and verify the certificate chain, but skip the hostname check (so a tunnelled loopback whose cert names the real instance still verifies). Uses sslrootcert if set.
verify-full	Verify the chain and the hostname (strict). Uses sslrootcert if set.

When sslmode is omitted it’s inferred from the host: a loopback/tunnel host (127.0.0.1, ::1, localhost) defaults to require, a direct host to verify-full. sslmode is independent of the auth provider.

Auth providers

Pick exactly one provider per connection in its [connections.<name>.auth] block. All providers coexist; the right one depends on where the secret lives.

Provider	Yields	Key fields
url	full URL (stored inline)	url
env	full URL or password	url_env / password_env
dotenv	full URL	url_env, path
aws_secret	static password	region, secret, field
aws_iam	dynamic token (~15m)	region
gcp_secret	static password	secret, field
gcp_iam	dynamic token (~60m)	—
script	password or URL	command, emits

url stores the whole connection string inline in the config — including the password. It’s the quickest option for a local or throwaway database; for anything sensitive use a provider that fetches the secret (and keep a url config out of version control). env takes either url_env (a full URL) or password_env (component mode — password only, with the host/port/etc. from the target fields above). For the rest, which fields are required vs. optional and their defaults are spelled out in the full table below.

All auth fields:

Field	Providers	Description
provider	all	The provider scheme (one of the above).
url	url	Full connection URL stored inline in the config (includes the password — for local/dev).
url_env	env, dotenv	Env var holding a full connection URL.
path	dotenv	Path to the .env file (defaults to ./.env).
password_env	env	Component mode — env var holding just the password (host/port/etc. come from the target fields above).
region	aws_iam, aws_secret	The AWS region.
secret	aws_secret, gcp_secret	The secret name/identifier.
field	aws_secret, gcp_secret	JSON key to extract (default password; omit for a raw-string secret). Alias: secret_field.
command	script	The command (argv array) to run.
emits	script	What the command prints: password (default) or url.

The two IAM providers (aws_iam, gcp_iam) mint a dynamic token refreshed on each run (and each --watch tick); the rest yield a static secret.

env vs dotenv

Both read a variable, but from different places:

• env reads from the process environment ($VAR). It reads no file itself — the variable must already be exported (your shell, CI, direnv) or loaded via env-file auto-load. Use url_env for a whole URL, or password_env plus the connection’s host/port/etc. for password-only.
• dotenv reads a variable directly from a .env file (path, default .env), without mutating the process environment. Whole-URL only; each connection can point at its own file.

Rule of thumb: secret already in your environment → env; secret in a specific file → dotenv; one .env with everything for local dev → env (or the $CLONEDB_SOURCE_URL fallback) plus env-file auto-load.

AWS

Authenticate with aws sso login / standard AWS credentials. RDS and Aurora support both a stored password in Secrets Manager and passwordless IAM auth:

# AWS Secrets Manager — static password (most common)
[connections.source.auth]
provider = "aws_secret"
region   = "us-east-1"
secret   = "app/db/readonly"   # secret name or ARN
field    = "password"          # JSON key; omit for a raw-string secret

# AWS RDS IAM — dynamic SigV4 token, no stored password
[connections.source.auth]
provider = "aws_iam"
region   = "us-east-1"

GCP

GCP providers use Application Default Credentials (gcloud auth application-default login). Cloud SQL and AlloyDB support both:

# GCP Secret Manager — static password
[connections.source.auth]
provider = "gcp_secret"
secret   = "app-db-readonly"   # name, or projects/<p>/secrets/<n>/versions/latest
field    = "password"          # JSON key; omit for a raw-string secret

# Cloud SQL / AlloyDB IAM — dynamic ADC OAuth token
[connections.source.auth]
provider = "gcp_iam"

Script (Vault, etc.)

Run any command that prints the password (or a full URL with emits = "url"):

[connections.source.auth]
provider = "script"
command  = ["vault", "read", "-field=password", "secret/db/read"]
emits    = "password"

env-file auto-load

clonedb does not read a .env file automatically. Opt in with the top-level env_file key (or the --env-file <FILE> flag, which overrides it):

env_file = ".env"   # loaded into the environment before credentials resolve

Before resolving credentials, clonedb loads that file into the process environment, so the env provider and the $CLONEDB_SOURCE_URL fallback can see its variables. Loading is non-override — a variable already set in the environment wins — and a named-but-missing file is a hard error. It does not affect the dotenv provider, which reads its own file directly.

env_file is a top-level key: like default_source/default_target, place it before the [connections.*] blocks (clonedb errors if it’s nested inside one). The init wizard sets it for you and creates the .env, prompting for each env/dotenv variable’s value as you define the connections.

Schemas

Which PostgreSQL schemas participate. Cross-schema foreign keys are followed automatically. exclude wins over include.

[schemas]
include = ["public", "billing"]   # empty/omitted means "all non-system schemas"
# exclude = ["audit"]             # drop these even if otherwise included

Key	Description
include	Allow-list of schema names. Empty means all non-system schemas.
exclude	Schemas to drop even if included.

Partitioned tables

Requires PostgreSQL 12+. clonedb uses pg_partition_tree to model declarative partitioning and does not fall back for older servers.

A declarative-partition hierarchy is modeled as one logical table at its root. Introspection keeps the partition root and folds away every partition (leaf and intermediate), so a partitioned table appears once everywhere — the dry-run plan, the SQL dump, config virtual-fks infer, and explain — never once per partition.

• Read through the root. SELECT … FROM root spans all leaves, so a partitioned table is subset/copied exactly once — no duplicate rows, no duplicate-PK errors on load.
• Write through the root. A load INSERTs into the root and PostgreSQL routes each row into the correct leaf. clonedb loads into an existing target schema, so the target must already have the full partition tree (root + intermediates + leaves); clonedb creates schemas, not tables.
• De-inflated estimates. A partitioned root has pg_class.reltuples = 0 (no own storage), so the dry-run estimate sums the leaves’ reltuples for a correct count.
• Name the root, not a partition. Pointing any config reference — a [tables] block, a seed, a scope, an override, or a virtual-FK endpoint — at a leaf or intermediate partition is a hard error that names the partition and its root. Otherwise partitions are folded silently: a root reads as a normal table.

Defaults

Policy applied to every table unless overridden per-table. The whole block (and any field) is optional and falls back to the safe baseline.

[defaults]
follow_children         = false       # don't START a cascade from unmarked seeds
follow_nullable         = false       # don't traverse nullable FKs unless asked
max_children_per_parent = 5000        # fan-out cap (explosion guard)
global_row_limit        = 2_000_000   # hard ceiling across the whole run
on_missing_parent       = "skip"      # orphan policy: "skip" | "error"
parallel_readers        = 4           # parallel snapshot-shared source readers
slow_query_warn_secs    = 5           # warn on any query slower than this (0 disables)

Key	Default	Description
follow_children	false	Whether an unmarked seed starts a downstream cascade.
follow_nullable	false	Whether to traverse nullable foreign keys.
max_children_per_parent	5000	Max children collected per parent row (explosion guard).
global_row_limit	none	Hard ceiling on total keys collected across the run. Omit for no run-wide ceiling.
on_missing_parent	"skip"	Orphan policy when a referenced parent row is missing: skip or error.
parallel_readers	4	Parallel snapshot-shared source reader connections for the run’s read phases. 1 = fully serial; higher values open that many source connections (each adopts the run snapshot, so results are point-in-time identical). Must be ≥ 1; clamped to pool headroom at run time. Overridden per run by run --parallel-readers.
slow_query_warn_secs	5	Log a warning the moment any single query runs longer than this many seconds (fractional allowed, e.g. 0.5). 0 disables the live warning; the end-of-run slow-query report is emitted either way.

Reachable-fill: how a table you didn’t configure is treated

There’s one subsetting behaviour — no knob. A table’s fate follows from whether it’s reachable from a seed through the foreign-key graph (real or virtual):

• Reachable → scoped. A table you can walk to from a seed is trimmed to that seed’s closure — only the connected rows are copied.
• Not reachable → whole. A table nothing connects to a seed is pulled entire — unless the run is bounded by --scope/--scoped, which skips the unreachable complement instead of cloning it whole.
• Excluded → nothing. A table with exclude = true copies nothing at all (unless you name it explicitly in a --scope/--seed, which overrides the exclusion for that run).

So you scope a table by connecting it — give it (or something upstream of it) a restriction, or wire up a virtual FK — and you leave a table whole by leaving it disconnected. The natural consequence: a config with no seeds reaches nothing, so every non-excluded table is whole — a full clone. Add a seed to start scoping.

Upstream parents are always pulled for referential integrity (including the parents of a whole table), so the subset is always referentially complete. A big whole count in run --dry-run is the signal a table isn’t connected yet — add a virtual FK or a restriction and watch it flip to a small scoped count (or leave it whole on purpose, e.g. a reference table).

Tables

A [tables."schema.table"] block holds per-table filters and policy. Table identity is schema-qualified — tables."public.users", which TOML quotes for you because of the dot.

A table that carries any restriction (where / ids / date / limit / sql) is a seed — a starting point. Upstream parents are always pulled (for referential integrity); downstream children only where a cascade is active.

You don’t list every table. Because relationships are introspected, a table with no block is pulled by FK traversal as needed — so a generated config contains no per-table blocks, just a comment listing the available tables (one line per schema). Add a [tables."schema.table"] block only for a table you want to seed, filter, or drop (exclude = true). (Run generate-config --all-tables if you’d rather have an explicit empty block per table.)

Note

A table with no block is still copied — scoped if reachable from a seed, whole if not (see Reachable-fill). With no seeds at all, nothing is reachable, so every non-excluded table is pulled whole — a full clone. Give some table a restriction to start scoping.

# Anchor: a few enterprise accounts and EVERYTHING under them.
[tables."public.accounts"]
sql = """
SELECT id FROM public.accounts
WHERE plan_tier = 'enterprise' AND status = 'active'
ORDER BY created_at DESC
LIMIT 25
"""
follow_children = true

# Seed by explicit ids, with a column-level scrub.
[tables."public.users"]
ids = [1001, 1002, 1042]
follow_children = true
exclude_columns = ["password_hash", "mfa_secret"]

# Latest-N pattern: order_by + limit selects the most recent rows.
[tables."public.audit_events"]
order_by = "created_at DESC"
limit    = 5000

Restriction strategies

These define the seed rows. A table’s seed source is exactly one of where/date, ids, or sql — setting more than one of those groups is an error. Within the first group, where and date compose (AND-ed into one predicate). limit is not a seed source and combines freely with any of them (and with order_by).

Key	Description
where	Raw SQL predicate (user-trusted), e.g. "issued_at >= now() - interval '90 days'". Composes with date.
ids	Explicit primary-key id list, e.g. [1001, 1002, 1042].
sql	A full raw SELECT whose result supplies the seed keys.
date	A date-range restriction on a column — see below. Composes with where.
limit	Row cap for this table’s seed. Not exclusive — combine with order_by and a seed source.

The date restriction is an inline table:

[tables."public.events"]
date = { column = "created_at", start = "2026-01-01", end = "2026-04-01" }

date field	Description
column	The column the range applies to (required).
start	Inclusive lower bound (>=). Optional.
end	Exclusive upper bound (<). Optional.

Bare ISO date/timestamp literals are quoted for you (start = "2026-01-01" → created_at >= '2026-01-01'); a value that looks like a SQL expression (now(), interval, an operator, or already quoted) is passed through verbatim.

Per-table policy

Key	Falls back to	Description
order_by	—	Order-by clause (column + direction). Combine with limit for “the latest N”.
upsert_key	table PK	Natural-/business-key columns to use as the load-time upsert conflict target. Set this for a table with no primary key (otherwise skipped on load); the columns must match a real UNIQUE constraint. Ignored (with a warning) on a table that already has a PK.
follow_children	[defaults].follow_children	Override the cascade anchor for this table.
follow_nullable	[defaults].follow_nullable	Override whether nullable FK child edges into this table are followed.
on_missing_parent	[defaults].on_missing_parent	Override the orphan policy (skip / error) for this table’s parents.
exclude_columns	—	Columns to drop (column-level scrub; the masking seam lives here).
passthrough	false	Copy the table whole and never traverse from it (tiny lookup tables).
exclude	false	Exclude this table entirely — schema kept, no data. An unconditional cut: the table is never collected, not even as a mandatory FK parent or cascade child.

# Passthrough: tiny lookup tables copied whole, never traversed.
[tables."public.countries"]
passthrough = true

# Exclude entirely (schema kept, no data). Preserved across sync-config.
[tables."public.access_log"]
exclude = true              # 400M rows of PII we never want in dev

Caution

Excluding a table that other tables reference leaves those child rows with a foreign key pointing at a row that won’t be copied — a dangling reference that can fail a target load. Exclude leaf tables (nothing points at them), or exclude the referencing children too. clonedb honors exclude over referential integrity, by design.

Bulk exclude / passthrough lists

For a table that needs only to be excluded or passed through — no filter, no column policy — a whole [tables."x"] block is boilerplate. Two top-level arrays of schema-qualified table names give you the same effect in one line per table:

exclude     = ["public.audit_log", "public.sessions", "public.schema_migrations"]
passthrough = ["billing.countries", "billing.plans"]

These are additive to the per-table booleans: a table is excluded if it’s in the exclude list or its block sets exclude = true (same for passthrough). The two forms coexist — config writes the list for new bulk edits and never rewrites your existing blocks — and exclude wins over passthrough when a table is in both.

Note

Because they’re top-level keys, TOML requires these arrays above the first [section] header (e.g. at the very top of the file, before [meta]). Editing them with config exclude / config passthrough places them correctly for you.

Key	Description
exclude	Schema-qualified tables to drop entirely (the list form of [tables."x"].exclude = true).
passthrough	Schema-qualified tables to copy whole and never traverse from (the list form of [tables."x"].passthrough = true).

validate checks each entry is a real table (an error) and warns when a table is in both lists, or in a list and a per-table block (harmless — the forms are additive).

Note

sync-config maintains a present_in_schema = false marker on a table block whose table no longer exists in the live schema — it flags the block rather than deleting it, so your filters aren’t lost if the table comes back. You don’t set this by hand.

Follow policy

follow_children = true anchors a cascade that propagates transitively — child to grandchild and on — until a stop. You set it once on the anchor, not on every level. Stops: an explicit follow_children = false, a passthrough table, or a relationship_overrides entry. Cycles terminate automatically via key dedup. The [defaults] guards (max_children_per_parent, global_row_limit) bound total size.

Scopes

Named, focused subsets for day-to-day work — clonedb run --scope checkout. Scope tables become the seed set and pull upstream parents only (small and predictable) unless an override re-enables a cascade.

A --scope run is bounded: the scope’s tables become the run’s seeds and only they, their FK parents, and any cascade an override re-enables are pulled — every other table is skipped, not cloned whole. This is the key difference from a plain run, where an unreachable table is pulled whole. A scope also neutralizes the config’s own ambient seeds/cascade and its top-level passthrough list — a passthrough table that isn’t one of the scope’s tables is not copied under that scope. A table named directly in tables (or via --seed) is collected even if the config marks it exclude: naming it wins over the exclusion for that run (validate notes this with an advisory warning, not an error). Use run --scoped to get the same bounding for a seed-driven run (config seeds or ad-hoc --seed) without defining a named scope.

[scopes.checkout]
description = "Just the checkout tables for the payments refactor"
tables = ["public.orders", "public.order_items", "billing.invoices"]

  # Per-scope filter overrides, applied only while this scope runs.
  [scopes.checkout.overrides."public.orders"]
  where = "created_at >= '2026-06-01'"
  limit = 500

Key	Description
description	Human-readable note describing what the scope pulls.
tables	Schema-qualified tables that become the seed/root set.
overrides."schema.table"	Per-table filter blocks (table shape) layered over the main config while this scope is active.

--scope is repeatable; the union of the named scopes is used. When two scopes name (or override) the same table, their seeds are unioned — a row matching either scope is collected — and their per-table attributes merge most-inclusively: a cascade runs if either scope opts in, and the limit is uncapped if either is, else the larger of the two. More scopes therefore pull more (or equal), never less.

limit is per source, not a grand total

Each scope’s limit caps that scope’s seed source independently — it’s not a ceiling on the combined set. Two overlapping scopes each capped at 500 can collect up to 1000 rows for the shared table (fewer if they overlap). The per-table limit was never a cross-scope total.

Need a one-off without editing the file? Use ad-hoc CLI seeds — see run --seed:

clonedb run --seed public.accounts --where "id = 123" --follow-children

Virtual foreign keys

For relationships with no database constraint (polymorphic, app-enforced, JSON-embedded) — CloneDB can’t discover these, so you declare them. The run engine follows every virtual FK (upstream parents and downstream cascades), including when discriminators and SQL-expression / JSON-embedded child columns.

There are two forms, and a config can mix them freely:

Terse map — the common case

A single-line entry per FK: "child.schema.table.col" = "parent[.col]". Omit the parent column to target the parent’s primary key. This is what config virtual-fks infer writes.

[virtual_foreign_keys]
"public.comments.author_id" = "public.users"        # parent column omitted → public.users.<pk>
"public.orders.account_id"  = "public.accounts.id"  # explicit parent column

The key is the child schema.table.column (a plain identifier); the value is the parent schema.table (PK default) or schema.table.column.

Rules array — when, composite, or expression FKs

For anything the terse map can’t hold — a when discriminator, composite columns, or a SQL-expression child column — use [[virtual_foreign_key_rules]]:

# Polymorphic: comments.commentable_id → posts.id WHEN commentable_type = 'post'
[[virtual_foreign_key_rules]]
child  = { table = "public.comments", columns = ["commentable_id"] }
parent = { table = "public.posts",    columns = ["id"] }
when   = "commentable_type = 'post'"   # discriminator predicate

# JSON-embedded reference: webhook_logs.payload->>'order_id' → orders.id.
# The expression child column is followed: the referenced orders rows are pulled.
[[virtual_foreign_key_rules]]
child  = { table = "public.webhook_logs", columns = ["(payload ->> 'order_id')::int"] }
parent = { table = "public.orders",       columns = ["id"] }

Key	Description
child	The referencing side: { table = "schema.table", columns = [...] }.
parent	The referenced side, same shape.
when	Optional discriminator predicate (e.g. polymorphic commentable_type = 'post').

Child columns may be plain identifiers or SQL expressions (composite keys list multiple columns); parent columns must be plain identifiers. validate checks plain-identifier columns against the schema; SQL-expression columns are intentionally not column-checked.

Relationship overrides

Stop or redirect traversal on a specific introspected edge (rare). Here: when a cascade reaches events, don’t drag the whole users table back in through it.

[[relationship_overrides]]
child  = "public.events"
parent = "public.users"
follow = false

Key	Description
child	Schema-qualified child table of the edge.
parent	Schema-qualified parent table of the edge.
follow	Whether to follow this edge.

See also

• CLI reference — the commands that read and write this file.
• Getting started — generate your first config and run it.