Skip to content

table-repair

Performs repairs on tables of divergent nodes based on the diff report generated by table-diff.

Syntax

./ace table-repair <cluster_name> <schema.table_name> --diff-file=<diff_file> [--source-of-truth=<node>] [options]
  • cluster_name: Name of the cluster. Optional. Overrides default_cluster in ace.yaml if specified.
  • schema.table_name: Schema‑qualified table name to repair.
  • --diff-file: Path to the diff JSON file.
  • --source-of-truth (-r): Authoritative node for repairs. Required unless you run in bidirectional insert-only mode or --fix-nulls.

Options

Option Alias Description Default
--diff-file <path> -f Path to the diff file (required)
--dbname <name> -d Database name override
--nodes <list> -n Nodes to include when resolving cluster metadata all
--source-of-truth <node> -r Authoritative node for repairs (required unless --bidirectional)
--dry-run -y Emit a dry-run plan instead of executing changes false
--generate-report -g Write a JSON repair report to reports/<YYYY-MM-DD>/repair_report_<HHMMSS.mmm>.json false
--insert-only -i Only insert missing rows; skip updates/deletes false
--upsert-only -P Insert or update rows; skip deletes false
--repair-file <path> -p Path to an advanced repair plan (YAML/JSON). Overrides --source-of-truth and uses rule-based actions.
--fix-nulls -X Fill NULL columns on each node using non-NULL values from its peers (no source-of-truth needed) false
--bidirectional -Z Perform insert-only repairs in both directions false
--fire-triggers -t Execute triggers (otherwise runs with session_replication_role='replica') false
--recovery-mode Enable recovery-mode repair when the diff was generated with --only-origin; can auto-select a source of truth using Spock LSNs false
--quiet -q Suppress non-essential logging false
--debug -v Enable verbose logging false

Advanced repair plans

  • Use --repair-file to drive repairs from a plan. Source-of-truth becomes optional; the plan sets per-row decisions.
  • Mutually exclusive with --bidirectional and --fix-nulls (those are separate modes).
  • Dry-run and reports include rule usage counts per node.
  • See Advanced repair for grammar and Examples for recipes.

Example

./ace table-repair acctg public.customers_large \
  --diff-file=public_customers_large_diffs-20250718134542.json \
  --source-of-truth=n1 \
  --generate-report

Diff reports share the same prefix generated by table-diff (for example public_customers_large_diffs-20250718134542.json). When you request a dry run or report, ACE also writes JSON summaries under reports/<YYYY-MM-DD>/repair_report_<HHMMSS.mmm>.json (or dry_run_report_<...>.json).

Recovery-mode behavior

  • If the diff file indicates only_origin, table-repair refuses to run unless --recovery-mode is set.
  • In recovery-mode, if no --source-of-truth is provided, ACE probes surviving nodes for the failed node’s Spock origin LSN (preferred) and slot LSN (fallback) and picks the highest. Ties or missing LSNs require you to provide --source-of-truth.
  • The chosen source (auto or explicit) is recorded in the repair report along with the LSN probes.
  • Recovery-mode still accepts --repair-file; the plan is applied after the source of truth is determined. If no repair file is provided, ACE performs a standard repair with the recovery-mode source selection.

Sample Output

2025/07/22 12:05:24 INFO Starting table repair for public.customers_large on cluster acctg
2025/07/22 12:05:24 INFO Processing repairs for divergent node: n2
2025/07/22 12:05:24 INFO Executed 99 upsert operations on n2
2025/07/22 12:05:24 INFO Repair of public.customers_large complete in 0.003s. Nodes n2 repaired (99 upserted).

Fixing null-only drifts (--fix-nulls)

Replication hiccups can leave some columns NULL on one node while populated on another. The --fix-nulls mode cross-fills those NULLs in both directions using values from the paired node(s); it does not require a source-of-truth. Use it when the diff shows only NULL/NOT NULL mismatches and you want to reconcile columns without preferring a single node.