MySQL Source

Connect MySQL as a source to replicate tables from a MySQL database into your warehouse. Supaflow supports full refresh and cursor-based incremental sync for tables in one configured database.

For an overview of capabilities and use cases, see the MySQL connector page. To run pipelines natively inside Snowflake, see Snowflake Native ETL.

Prerequisites

Before you begin, ensure you have:

• An active MySQL database reachable from Supaflow
• Network connectivity between Supaflow and your MySQL server
• A database user with SELECT privileges on the tables you want to sync
• Firewall rules that allow inbound connections from Supaflow, if your database is not privately connected
• A reliable cursor column on tables you want to sync incrementally

Whitelist Supaflow IP

If your MySQL server restricts inbound connections, add the Supaflow IP to your firewall rules:

18.214.240.61

MySQL commonly listens on TCP port 3306. For self-managed MySQL, also confirm the server accepts inbound client connections from your Supaflow network path.

Create Database User

Create a dedicated read-only user for Supaflow:

CREATE USER 'supaflow_reader'@'%' IDENTIFIED BY 'your_secure_password';

Grant read access to the database you want to sync:

GRANT SELECT ON `your_database`.* TO 'supaflow_reader'@'%';

Or grant read access to specific tables:

GRANT SELECT ON `your_database`.`your_table` TO 'supaflow_reader'@'%';

Replace your_database and your_table with your actual database and table names. For tighter network control, replace % with the host or address range that should be allowed to connect. See MySQL's CREATE USER, GRANT, and account name documentation for account and privilege syntax.

What Gets Synced

Supaflow discovers tables in the configured MySQL database. Tables are available for selection when the configured user has permission to read them.

The MySQL source is scoped to one database per source connection. To sync tables from multiple MySQL databases, create one Supaflow source for each database.

Sync Modes

Full refresh: On each sync, all rows are read from the selected table. Use full refresh for small tables, lookup tables, or tables without a reliable cursor column.

Incremental sync: Only rows in the selected cursor window are fetched. Use incremental sync for tables with a reliable date, datetime, or timestamp column that advances when rows are inserted or updated.

Supaflow tracks cursor positions between syncs. You can also configure a lookback period so each incremental run re-reads a short window before the last cursor position, which helps capture late-arriving writes.

The MySQL connector does not use binary log CDC and does not detect hard deletes from the source. If a table does not have a reliable cursor column, use full refresh.

Schema Discovery and Evolution

Supaflow reads table metadata from the configured MySQL database to discover table names, column names, data types, and primary keys. Tables are grouped under the configured database name; MySQL schemas are not exposed separately in Supaflow.

Schema metadata is refreshed according to the Schema Refresh Interval setting. New tables and columns appear after a schema refresh and can then be selected in pipelines.

Configuration

Step 1: Connection

Host*

MySQL server hostname or IP address
Example: db.example.com or 192.168.1.100

Port

Port on which MySQL is listening
Default: 3306

User*

Database username
Example: supaflow_reader

Password

Password for the database user
Stored encrypted

Database Name*

Name of the MySQL database to sync
Example: production_db

---

Step 2: Sync Settings (Optional)

Lookback Period (seconds)

Number of seconds to re-fetch before the last cursor position on incremental syncs
Default: 0 (no lookback)
Range: 0 to 86400

---

Step 3: Advanced Settings (Optional)

Schema Refresh Interval

Interval in minutes for schema metadata refresh
0 = refresh before every pipeline execution
-1 = disable automatic schema refresh
Default: 60 (Range: -1 to 10080)

---

Step 4: Test & Save

After configuring all required properties, click Test & Save to verify the connection and save the source.

Troubleshooting

Common issues and their solutions:

Connection refused or timeout

Problem:

• Cannot connect to the MySQL server
• "Connection refused" or network timeout error
• Test connection takes too long

Solutions:

1. Verify firewall rules:
   • Add Supaflow IP 18.214.240.61 to allowed inbound rules
   • Confirm inbound traffic is allowed on your MySQL port, usually 3306
2. Check remote access configuration:
   • Confirm MySQL is listening on a reachable network interface
   • Confirm the host and port in Supaflow match your MySQL server
3. Use private connectivity when required:
   • If your database is not exposed publicly, connect through your approved private deployment or network path

---

Authentication failed

Problem:

• "Access denied" or authentication failed
• Username and password work locally but not from Supaflow

Solutions:

1. Verify the username, password, and database name.
2. Check the MySQL account host component:
   • MySQL accounts are scoped as user@host
   • Create or update the account so it can connect from the Supaflow network path
3. Test from an allowed host:

   mysql -h your_host -P 3306 -u supaflow_reader -p your_database

---

Permission denied or tables missing

Problem:

• Connection succeeds, but tables are missing
• "SELECT command denied" or permission errors

Solutions:

1. Grant SELECT on the database or tables you want to sync:

   GRANT SELECT ON `your_database`.* TO 'supaflow_reader'@'%';

2. Verify the source is configured for the correct database.
3. Refresh schema in Supaflow:
   • Set Schema Refresh Interval to 0
   • Save and test the source again
4. Remember the current connector discovers tables, not views.

---

Incremental sync misses expected rows

Problem:

• Incremental sync completes, but recent updates are missing
• Rows created or updated out of order are not picked up

Solutions:

1. Use a reliable cursor column:
   • Choose a date, datetime, or timestamp column that changes whenever a row changes
   • If no such column exists, use full refresh
2. Increase the lookback period:
   • Use Lookback Period to re-read a short window before the previous cursor position
   • This is useful when writes can arrive slightly out of order
3. Do not rely on incremental sync for hard deletes:
   • The MySQL connector does not use binary log CDC
   • Hard deletes are not detected by cursor-based incremental sync

---

TLS or certificate-required connections

Problem:

• Your MySQL server or database user requires client-side TLS settings
• Connection fails even though host, port, username, and password are correct

Solutions:

1. Confirm whether your MySQL account requires TLS.
2. Use an approved private connectivity path if your environment requires database connections to stay inside a private network.
3. Contact support if you need client certificate or SSL-mode configuration for this source.

---

Large syncs are slow

Problem:

• Initial syncs or full refreshes take longer than expected
• Production database load increases during sync

Solutions:

1. Narrow the selected tables or columns in your pipeline.
2. Use incremental sync for large tables that have reliable cursor columns.
3. Schedule large syncs during off-peak hours.
4. Review MySQL connection and resource limits for the Supaflow database user.

Support

Need help? Contact us at support@supa-flow.io