CockroachDB Source

Connect CockroachDB as a source to replicate tables from a CockroachDB database into your warehouse. Supaflow supports full refresh and cursor-based incremental sync for tables in one configured database.

For an overview of capabilities and use cases, see the CockroachDB connector page. To run pipelines natively inside Snowflake, see Snowflake Native ETL.

Prerequisites

Before you begin, ensure you have:

• An active CockroachDB cluster reachable from Supaflow
• Network connectivity between Supaflow and your CockroachDB cluster
• A SQL user with read access to the database, schema, and tables you want to sync
• Firewall or CockroachDB Cloud network authorization that allows inbound connections from Supaflow, if your cluster is not privately connected
• A reliable cursor column on tables you want to sync incrementally

Allow Supaflow Network Access

If your CockroachDB cluster restricts inbound connections, add the Supaflow IP to your firewall or CockroachDB Cloud IP allowlist:

18.214.240.61

CockroachDB commonly listens on TCP port 26257. CockroachDB Cloud also supports private connectivity options for supported cluster types. See Cockroach Labs' network authorization and connection parameter documentation for current networking and connection details.

Create Database User

Create a dedicated read-only SQL user for Supaflow:

CREATE USER supaflow_reader WITH PASSWORD 'your_secure_password';

Grant access to the database, schema, and existing tables you want to sync. Run the schema and table grants while connected to your_database:

GRANT CONNECT ON DATABASE your_database TO supaflow_reader;
GRANT USAGE ON SCHEMA your_schema TO supaflow_reader;
GRANT SELECT ON ALL TABLES IN SCHEMA your_schema TO supaflow_reader;

If future tables are created by a migration or application role, grant default table privileges for that table-creator role:

ALTER DEFAULT PRIVILEGES FOR ROLE your_table_creator IN SCHEMA your_schema
GRANT SELECT ON TABLES TO supaflow_reader;

Replace your_database, your_schema, and your_table_creator with the CockroachDB database, schema, and role that will create future tables. If you omit FOR ROLE, CockroachDB applies the default privileges to objects created by the current user. See Cockroach Labs' CREATE USER, GRANT, and ALTER DEFAULT PRIVILEGES documentation for account and privilege syntax.

What Gets Synced

Supaflow discovers tables in the configured CockroachDB database. Tables are available for selection when the configured user has permission to read them.

The CockroachDB source is scoped to one database per source connection. To sync tables from multiple CockroachDB databases, create one Supaflow source for each database.

CockroachDB schemas are represented in the Supaflow object namespace when CockroachDB reports one. Object names are qualified as database.schema.table when a schema is present.

The current connector discovers tables. Views are not exposed as supported objects on this source page.

Sync Modes

Full refresh: On each sync, all rows are read from the selected table. Use full refresh for small tables, lookup tables, or tables without a reliable cursor column.

Incremental sync: Only rows in the selected cursor window are fetched. Use incremental sync for tables with a reliable date or timestamp column that advances when rows are inserted or updated.

Supaflow tracks cursor positions between syncs. You can also configure a lookback period so each incremental run re-reads a short window before the last cursor position, which helps capture late-arriving writes.

The CockroachDB connector does not use CockroachDB changefeeds or CDC and does not detect hard deletes from the source. If a table does not have a reliable cursor column, use full refresh.

Schema Discovery

Supaflow reads table metadata from the configured CockroachDB database to discover table names, column names, data types, and primary keys. Schema metadata is refreshed according to the Schema Refresh Interval setting.

New tables and columns appear after a schema refresh and can then be selected in pipelines.

Configuration

Step 1: Connection

Host*

CockroachDB node, load balancer, or cloud cluster hostname
Example: db.example.com or your-cluster.cockroachlabs.cloud

Port

Port on which CockroachDB is listening
Default: 26257

User*

Database username
Example: supaflow_reader

Password

Password for the database user
Stored encrypted

Database Name*

Name of the CockroachDB database to sync
Example: production_db

SSL Mode

SSL connection mode for the CockroachDB connection
Options: disable, allow, prefer, require, verify-ca, verify-full
Default: prefer
Use disable only for insecure local or development clusters

---

Step 2: Sync Settings (Optional)

Lookback Period (seconds)

Number of seconds to re-fetch before the last cursor position on incremental syncs
Default: 0 (no lookback)
Range: 0 to 86400

---

Step 3: Advanced Settings (Optional)

Schema Refresh Interval

Interval in minutes for schema metadata refresh
0 = refresh before every pipeline execution
-1 = disable automatic schema refresh
Default: 60 (Range: -1 to 10080)

---

Step 4: Test & Save

After configuring all required properties, click Test & Save to verify the connection and save the source.

Performance and Source Load

CockroachDB syncs run SQL reads against the selected source tables. Initial syncs and full refreshes read every selected row, so they can add noticeable load on busy clusters.

For large production tables:

• Use incremental sync when a reliable date or timestamp cursor is available
• Schedule large initial loads and full refreshes during off-peak hours
• Review CockroachDB connection, statement, and resource controls for the Supaflow user
• Keep pipeline selection focused on the tables and columns you need

Troubleshooting

Common issues and their solutions:

Connection refused or timeout

Problem:

• Cannot connect to the CockroachDB cluster
• "Connection refused" or network timeout error
• Test connection takes too long

Solutions:

1. Verify network access:
   • Add Supaflow IP 18.214.240.61 to your firewall or CockroachDB Cloud IP allowlist
   • Confirm inbound traffic is allowed on your CockroachDB port, usually 26257
2. Check the connection values:
   • Confirm Host, Port, and Database Name match the values from your CockroachDB connection parameters
   • For CockroachDB Cloud, copy host and port from the cluster's Connect dialog
3. Use private connectivity when required:
   • If your cluster is not exposed publicly, connect through your approved private deployment or network path

---

Authentication failed

Problem:

• Password authentication failed
• User does not exist
• Credentials work in another tool but not in Supaflow

Solutions:

1. Verify the User, Password, and Database Name.
2. Confirm the SQL user can log in with password authentication.
3. Check SSL Mode:
   • CockroachDB Cloud and secure clusters usually require an encrypted connection
   • Use disable only for insecure local or development clusters
4. If your cluster requires client certificates only, contact support.
   • The current source configuration exposes password authentication and SSL Mode, not client certificate file fields.

---

Permission denied or tables missing

Problem:

• Connection succeeds, but tables are missing
• Permission errors during schema discovery or sync

Solutions:

1. Grant read access to the database, schema, and tables:

   GRANT CONNECT ON DATABASE your_database TO supaflow_reader;
   GRANT USAGE ON SCHEMA your_schema TO supaflow_reader;
   GRANT SELECT ON ALL TABLES IN SCHEMA your_schema TO supaflow_reader;

2. Verify the source is configured for the correct Database Name.
3. Refresh schema in Supaflow:
   • Set Schema Refresh Interval to 0
   • Save and test the source again
4. Remember the current connector discovers tables, not views.

---

Incremental sync misses expected rows

Problem:

• Incremental sync completes, but recent inserts or updates are missing
• Rows created or updated out of order are not picked up

Solutions:

1. Use a reliable cursor column:
   • Choose a date or timestamp column that changes whenever a row changes
   • If no such column exists, use full refresh
2. Increase the lookback period:
   • Use Lookback Period to re-read a short window before the previous cursor position
   • This is useful when writes can arrive slightly out of order
3. Do not rely on incremental sync for hard deletes:
   • The CockroachDB connector does not use changefeeds or CDC
   • Hard deletes are not detected by cursor-based incremental sync

---

SSL connection errors

Problem:

• SSL negotiation fails
• Certificate verification fails
• The cluster rejects insecure connections

Solutions:

1. Use an SSL Mode your cluster accepts:
   • prefer is the default
   • require, verify-ca, or verify-full may be required for stricter environments
   • disable should be limited to insecure local or development clusters
2. Confirm your CockroachDB connection parameters outside Supaflow.
3. If your environment requires custom certificate files, contact support.

---

Large table sync is slow or times out

Problem:

• A large table takes too long to read
• A sync fails with a query timeout or source resource error

Solutions:

1. Open Job Details and check the per-object failure to identify the table that timed out.
2. Use incremental sync for the failing table if it has a reliable cursor column.
3. Schedule the pipeline during off-peak hours when the CockroachDB cluster has more read capacity.
4. Review CockroachDB statement and resource controls for the Supaflow user.
5. If the failing table is not identified, temporarily narrow the pipeline selection to isolate the object before tuning sync behavior.

Support

Need help? Contact us at support@supa-flow.io