MariaDB Source

Connect MariaDB as a source to replicate tables from a MariaDB database into your warehouse. Supaflow supports full refresh and cursor-based incremental sync for tables in one configured database.

For an overview of capabilities and use cases, see the MariaDB connector page. To run pipelines natively inside Snowflake, see Snowflake Native ETL.

Prerequisites

Before you begin, ensure you have:

• An active MariaDB database reachable from Supaflow
• Network connectivity between Supaflow and your MariaDB server
• A database user with SELECT privileges on the tables you want to sync
• Firewall rules that allow inbound connections from Supaflow, if your database is not privately connected
• A reliable cursor column on tables you want to sync incrementally

Whitelist Supaflow IP

If your MariaDB server restricts inbound connections, add the Supaflow IP to your firewall rules:

18.214.240.61

MariaDB commonly listens on TCP port 3306. For self-managed MariaDB, also confirm the server is configured for remote client access. See MariaDB's Connecting to MariaDB and Remote Client Access guides.

Create Database User

Create a dedicated read-only user for Supaflow:

CREATE USER 'supaflow_reader'@'%' IDENTIFIED BY 'your_secure_password';

Grant read access to the database you want to sync:

GRANT SELECT ON `your_database`.* TO 'supaflow_reader'@'%';

Or grant read access to specific tables:

GRANT SELECT ON `your_database`.`your_table` TO 'supaflow_reader'@'%';

Replace your_database and your_table with your actual database and table names. For tighter network control, replace % with the host or address range that should be allowed to connect. See MariaDB's CREATE USER and GRANT documentation for account and privilege syntax.

What Gets Synced

Supaflow discovers tables in the configured MariaDB database. Tables are available for selection when the configured user has permission to read them.

The MariaDB source is scoped to one database per source connection. To sync tables from multiple MariaDB databases, create one Supaflow source for each database.

Sync Modes

Full refresh: On each sync, all rows are read from the selected table. Use full refresh for small tables, lookup tables, or tables without a reliable cursor column.

Incremental sync: Only rows in the selected cursor window are fetched. Use incremental sync for tables with a reliable date, datetime, or timestamp column that advances when rows are inserted or updated.

Supaflow tracks cursor positions between syncs. You can also configure a lookback period so each incremental run re-reads a short window before the last cursor position, which helps capture late-arriving writes.

The MariaDB connector does not use binary log CDC and does not detect hard deletes from the source. If a table does not have a reliable cursor column, use full refresh.

Schema Discovery

Supaflow reads table metadata from the configured MariaDB database to discover table names, column names, data types, and primary keys. Tables are grouped under the configured database name; MariaDB schemas are not exposed separately.

Schema metadata is refreshed according to the Schema Refresh Interval setting. New tables and columns appear after a schema refresh and can then be selected in pipelines.

Configuration

Step 1: Connection

Host*

MariaDB server hostname or IP address
Example: db.example.com or 192.168.1.100

Port

Port on which MariaDB is listening
Default: 3306

User*

Database username
Example: supaflow_reader

Password

Password for the database user
Stored encrypted

Database Name*

Name of the MariaDB database to sync
Example: production_db

---

Step 2: Sync Settings (Optional)

Lookback Period (seconds)

Number of seconds to re-fetch before the last cursor position on incremental syncs
Default: 0 (no lookback)
Range: 0 to 86400

---

Step 3: Advanced Settings (Optional)

Schema Refresh Interval

Interval in minutes for schema metadata refresh
0 = refresh before every pipeline execution
-1 = disable automatic schema refresh
Default: 60 (Range: -1 to 10080)

---

Step 4: Test & Save

After configuring all required properties, click Test & Save to verify the connection and save the source.

Troubleshooting

Common issues and their solutions:

Connection refused or timeout

Problem:

• Cannot connect to the MariaDB server
• "Connection refused" or network timeout error
• Test connection takes too long

Solutions:

1. Verify firewall rules:
   • Add Supaflow IP 18.214.240.61 to allowed inbound rules
   • Confirm inbound traffic is allowed on your MariaDB port, usually 3306
2. Check remote access configuration:
   • Confirm MariaDB is listening on a reachable network interface
   • Confirm the host and port in Supaflow match your MariaDB server
3. Use private connectivity when required:
   • If your database is not exposed publicly, connect through your approved private deployment or network path

---

Authentication failed

Problem:

• "Access denied" or authentication failed
• Username and password work locally but not from Supaflow

Solutions:

1. Verify the username, password, and database name.
2. Check the MariaDB account host component:
   • MariaDB accounts are scoped as user@host
   • Create or update the account so it can connect from the Supaflow network path
3. Test from an allowed host:

   mariadb -h your_host -P 3306 -u supaflow_reader -p your_database

---

Permission denied or tables missing

Problem:

• Connection succeeds, but tables are missing
• "SELECT command denied" or permission errors

Solutions:

1. Grant SELECT on the database or tables you want to sync:

   GRANT SELECT ON `your_database`.* TO 'supaflow_reader'@'%';

2. Verify the source is configured for the correct database.
3. Refresh schema in Supaflow:
   • Set Schema Refresh Interval to 0
   • Save and test the source again
4. Remember the current connector discovers tables, not views.

---

Incremental sync misses expected rows

Problem:

• Incremental sync completes, but recent updates are missing
• Rows created or updated out of order are not picked up

Solutions:

1. Use a reliable cursor column:
   • Choose a date, datetime, or timestamp column that changes whenever a row changes
   • If no such column exists, use full refresh
2. Increase the lookback period:
   • Use Lookback Period to re-read a short window before the previous cursor position
   • This is useful when writes can arrive slightly out of order
3. Do not rely on incremental sync for hard deletes:
   • The MariaDB connector does not use binary log CDC
   • Hard deletes are not detected by cursor-based incremental sync

---

TLS or certificate-required connections

Problem:

• Your MariaDB server or database user requires client-side TLS settings
• Connection fails even though host, port, username, and password are correct

Solutions:

1. Confirm whether your MariaDB account requires TLS.
2. Use an approved private connectivity path if your environment requires database connections to stay inside a private network.
3. Contact support if you need client certificate or SSL-mode configuration for this source.

---

Large syncs are slow

Problem:

• Initial syncs or full refreshes take longer than expected
• Production database load increases during sync

Solutions:

1. Narrow the selected tables or columns in your pipeline.
2. Use incremental sync for large tables that have reliable cursor columns.
3. Schedule large syncs during off-peak hours.
4. Review MariaDB connection and resource limits for the Supaflow database user.

Support

Need help? Contact us at support@supa-flow.io