SQL Server Source

Connect Microsoft SQL Server, Azure SQL Database, or Microsoft Fabric Warehouse as a source for ingestion and activation pipelines.

For Microsoft Fabric Warehouse, first follow Prepare Microsoft Fabric Warehouse for Supaflow to create the service principal and configure Fabric access. For capabilities and use cases, see the Microsoft Fabric Warehouse connector overview.

Prerequisites

Before you begin, ensure you have:

An active SQL Server instance, Azure SQL Database, or capacity-backed Fabric Warehouse
Network connectivity between Supaflow and the SQL endpoint
A database identity with read access to the schemas and tables you want to use
Firewall access for SQL Server or Azure SQL Database, when inbound connections are restricted

Whitelist Supaflow IP

Add the Supaflow IP to your firewall rules:

18.214.240.61

Azure SQL Database: In the Azure portal, go to your SQL server > Networking > Firewall rules and add the IP above. Without this, connections from the hosted Supaflow agent will be refused.

On-premises SQL Server: Add the IP to your Windows Firewall or network security group if inbound connections are restricted.

Microsoft Fabric Warehouse: Do not add this address as an Azure SQL firewall rule. Follow the Fabric network policy for your tenant and ensure outbound TCP port 1433 is available to the agent.

Microsoft Entra Service Principal for Azure SQL

Azure SQL can use the same Microsoft Entra service-principal authentication exposed by the connector. Register a single-tenant Entra application, create a client secret, and configure an Entra administrator on the Azure SQL logical server. The Azure SQL destination setup shows those portal steps and screenshots.

Connect to the intended source database as the Entra administrator, then create a contained user for the application and grant read access. Replace app-name with the app registration's display name:

CREATE USER [app-name]
FROM EXTERNAL PROVIDER;

ALTER ROLE db_datareader
ADD MEMBER [app-name];

Do not add Microsoft Graph or Azure Resource Manager API permissions to grant database access. Azure SQL permissions are assigned inside the database. See Microsoft Entra authentication for Azure SQL for current Microsoft requirements.

Create Database User

Create a dedicated read-only user for Supaflow.

On-premises SQL Server:

USE [master];
CREATE LOGIN supaflow_reader WITH PASSWORD = 'your_secure_password';

USE [your_database];
CREATE USER supaflow_reader FOR LOGIN supaflow_reader;

Azure SQL Database (contained database user):

USE [your_database];
CREATE USER supaflow_reader WITH PASSWORD = 'your_secure_password';

Grant SELECT permission on the entire database:

GRANT SELECT ON DATABASE::[your_database] TO [supaflow_reader];

Or grant SELECT on a specific schema:

GRANT SELECT ON SCHEMA::[dbo] TO [supaflow_reader];

Or grant SELECT on a specific table:

GRANT SELECT ON [dbo].[your_table] TO [supaflow_reader];

Replace your_database, dbo, and your_table with your actual names.

Enable Change Tracking (Optional)

If you plan to use Change Tracking mode for incremental syncs, enable it on the database and each table you want to track.

Enable on the database:

ALTER DATABASE [your_database]
SET CHANGE_TRACKING = ON
(CHANGE_RETENTION = 7 DAYS, AUTO_CLEANUP = ON);

Enable on each table (table must have a primary key):

ALTER TABLE [dbo].[your_table] ENABLE CHANGE_TRACKING;

The Supaflow user also needs VIEW CHANGE TRACKING permission:

GRANT VIEW CHANGE TRACKING ON SCHEMA::[dbo] TO [supaflow_reader];

What Gets Synced

Supaflow discovers tables in the configured SQL Server, Azure SQL, or Fabric Warehouse database. Tables are available for selection when the configured identity has permission to read them, grouped by schema.

The source is scoped to one database or Warehouse item per connection. To use objects from multiple databases or Warehouses, create one Supaflow source for each one.

Sync Modes

Sync behavior depends on the Query Mode you select for the source (see Query Modes):

Standard mode: Tables with a reliable date, datetime, or timestamp cursor column sync incrementally -- only rows past the last cursor position are fetched on each run. Tables without a usable cursor column run as full refresh and are marked with a Full Sync badge in the pipeline wizard's schema selection step.

Change Tracking mode: Tables with Change Tracking enabled sync incrementally without needing a cursor column, and deletes are captured. Tables without Change Tracking enabled are skipped during schema discovery with a clear reason shown in the schema selection step.

Microsoft Fabric Warehouse supports Standard mode only. It can run full refreshes or use a reliable date or timestamp cursor for incremental reads, but it does not support SQL Server Change Tracking or automatic hard-delete detection.

Configuration

Step 1: Connection

Database Host*

SQL Server hostname or IP address
Example: myserver.database.windows.net (Azure), db.example.com, 192.168.1.100

Database Port*

Port on which SQL Server is listening
Default: 1433

Database Name*

Name of the database to connect to
Example: production_db; for Fabric, use the Warehouse item name

Authentication Method*

Authentication used for the connection
Options: SQL Password (default), Entra Service Principal
Fabric Warehouse requires Entra Service Principal

Database Username*

Username for SQL Password authentication
Required for SQL Password; example: supaflow_reader

Database Password*

Password for SQL Password authentication
Required for SQL Password; stored encrypted

The following fields appear when Authentication Method is Entra Service Principal:

Tenant ID*

Directory (tenant) ID of the Microsoft Entra tenant that owns the application

Client Application ID*

Application (client) ID of the Microsoft Entra service principal

Client Secret*

Client-secret value for the Microsoft Entra service principal
Stored encrypted; enter the secret value, not the Secret ID

Fabric Workspace ID

Workspace identifier used to initialize Fabric service-principal access
Required for Fabric Warehouse; find it after /groups/ in the Fabric workspace URL. Leave blank for SQL Server and Azure SQL Database.

Query Mode

How incremental changes are detected
Options: STANDARD (default), CHANGE_TRACKING

Mode	Description	Requirements
STANDARD	Uses a cursor column (e.g., `LastModifiedDate`) to detect new and changed rows. Best when tables have a reliable timestamp column.	None
CHANGE_TRACKING	Uses SQL Server Change Tracking to detect inserts, updates, and deletes. Captures the full change history including deletions.	Change Tracking enabled on database and tables. Primary key on every synced table. Minimum 7-day retention recommended.

Use STANDARD for Microsoft Fabric Warehouse.

Encrypt

Whether to encrypt the connection to SQL Server
Options: true (default), false, strict
Recommended: true for Azure SQL, strict for TDS 8.0

Trust Server Certificate

Trust the server certificate without validation
Default: unchecked (false)
Enable only for self-signed certificates or development environments

Step 2: Advanced Settings (Optional)

Noop Query

Query to test connectivity without returning data
Default: SELECT 1
Used for connection validation

JDBC Connection Properties

Additional JDBC connection parameters
Format: key=value pairs separated by semicolons
Example: loginTimeout=30;queryTimeout=60

Database Flavor

Select the database behavior to use
Options: Auto (default), SQL Server, Fabric Warehouse
Keep Auto for standard Fabric hostnames; force Fabric Warehouse only for custom DNS or proxy endpoints

Chunked Read Enabled

Enable keyset-based chunked reads for large tables
Default: true
When Window Limit is enabled (default), splits full-table reads into bounded batches using the primary key. When disabled, uses unbounded keyset read with resume semantics. Each chunk is resumable on failure. Only applies to initial/full-refresh reads on tables with a primary key -- incremental cursor-based reads are not chunked.

Chunked Read Chunk Size

Maximum number of rows per chunk
Default: 1,000,000 (Range: 1,000 to 5,000,000)

Chunked Read Window Limit Enabled

Use SQL TOP/LIMIT to bound each chunk query
Default: true

Chunked Read Memory Aware Limit Enabled

Dynamically reduce chunk size for wide rows based on available memory
Default: false

Chunked Read Target Memory MB

Target memory budget per chunk when memory-aware limiting is enabled
Default: 50 MB (Range: 1 to 4,096)

Schema Refresh Interval

Interval in minutes for schema metadata refresh
0 = refresh before every pipeline execution
-1 = disable schema refresh
Default: 30 (Range: -1 to 10080)

Step 3: Test & Save

After configuring all required properties, click Test & Save to verify your connection and save the source.

Query Modes

Standard Mode

Standard mode works like any JDBC source connector. During incremental syncs, it uses a cursor column (typically a timestamp like LastModifiedDate or UpdatedAt) to fetch only rows that changed since the last sync.

Best for tables with reliable modification timestamps
Cannot detect deletes (only inserts and updates)
Works with any SQL Server version

Change Tracking Mode

Change Tracking mode uses SQL Server's built-in Change Tracking feature to detect all changes including inserts, updates, and deletes.

The first sync reads the current table contents. Subsequent syncs read changes retained by SQL Server Change Tracking, including deleted records, so downstream destinations can apply their configured delete behavior. If the saved Change Tracking position falls outside the database's retention window, run a full resync before continuing incrementally.

Requirements:

SQL Server 2016 or higher (or Azure SQL Database)
Change Tracking enabled on the database with at least 7-day retention
Change Tracking enabled on each table
Every synced table must have a primary key

Tables without Change Tracking: If a table does not have Change Tracking enabled, it is automatically skipped during schema discovery. The schema selection step shows the table as "Permanently skipped (change tracking not enabled)", and the job log includes the exact ALTER TABLE statement to enable it.

Troubleshooting

Common issues and their solutions:

Connection refused

Problem:

Cannot connect to SQL Server
"Connection refused" or network timeout error

Solutions:

Verify firewall rules:
- Add Supaflow IP (18.214.240.61) to allowed IPs
- For Azure SQL: Add to Networking > Firewall rules in the Azure portal
- For on-premises: Check Windows Firewall and network security groups
Check SQL Server is running:
```
# Windows
Get-Service MSSQLSERVER
```
Verify port is open:
- Default port is 1433
- Check SQL Server Configuration Manager for TCP/IP settings
- Ensure TCP/IP protocol is enabled
Azure SQL specific:
- Ensure "Allow Azure services" is enabled if connecting from Azure
- Check that the server firewall (not just database firewall) allows the IP

Authentication failed

Problem:

"Login failed for user"
Error 18456

Solutions:

Verify username and password:

Test login manually:

sqlcmd -S your_host -U supaflow_reader -P your_password -d your_database

On-premises: Check authentication mode:
- SQL Server must be in "SQL Server and Windows Authentication" mode
- Change in SQL Server Management Studio > Server Properties > Security
- Restart SQL Server after changing

Azure SQL: Verify user exists in the correct database:

SELECT name FROM sys.database_principals WHERE name = 'supaflow_reader';

Reset password if needed:

-- On-premises
ALTER LOGIN supaflow_reader WITH PASSWORD = 'new_secure_password';

-- Azure SQL (contained user)
ALTER USER supaflow_reader WITH PASSWORD = 'new_secure_password';

Permission denied on tables

Problem:

"SELECT permission denied"
Can connect but cannot read data
Some tables visible, others not

Solutions:

Grant SELECT on all tables in a schema:

GRANT SELECT ON SCHEMA::[dbo] TO [supaflow_reader];

Grant SELECT on the entire database:

GRANT SELECT ON DATABASE::[your_database] TO [supaflow_reader];

Check current permissions:

SELECT
    dp.name AS user_name,
    o.name AS object_name,
    p.permission_name,
    p.state_desc
FROM sys.database_permissions p
JOIN sys.database_principals dp ON p.grantee_principal_id = dp.principal_id
LEFT JOIN sys.objects o ON p.major_id = o.object_id
WHERE dp.name = 'supaflow_reader';

Add to db_datareader role (grants SELECT on all tables):

ALTER ROLE db_datareader ADD MEMBER [supaflow_reader];

TLS / Encryption errors

Problem:

"SSL connection failed"
"The driver could not establish a secure connection"
Certificate verification errors

Solutions:

For Azure SQL:
- Set Encrypt to true (default) -- Azure SQL requires encryption
- If using a corporate proxy, you may need to enable Trust Server Certificate
For on-premises with self-signed certificates:
- Enable Trust Server Certificate checkbox
- Or set Encrypt to false for testing (not recommended for production)
For strict TLS (TDS 8.0):
- Set Encrypt to strict
- Requires SQL Server 2022 or Azure SQL
- Server certificate must be properly signed

Change Tracking errors

Problem:

Tables shown as "Permanently skipped (change tracking not enabled)" in the schema selection step
"CT incremental read requires custom_state" error
CT version mismatch

Solutions:

Verify CT is enabled on the database:

SELECT is_change_tracking_enabled
FROM sys.databases
WHERE name = DB_NAME();

Verify CT is enabled on the table:

SELECT t.name, ct.is_track_columns_updated_on
FROM sys.change_tracking_tables ct
JOIN sys.tables t ON ct.object_id = t.object_id;

Enable CT on a missing table:

ALTER TABLE [dbo].[your_table] ENABLE CHANGE_TRACKING;

Check CT retention period:

SELECT change_tracking_retention, change_tracking_retention_period_units
FROM sys.change_tracking_databases;

If the retention period expired between syncs, trigger a full resync.

Grant VIEW CHANGE TRACKING:

GRANT VIEW CHANGE TRACKING ON SCHEMA::[dbo] TO [supaflow_reader];

Tables not appearing in schema

Problem:

Connected successfully but tables missing
Some tables visible, others not
Empty schema list

Solutions:

Verify SELECT permission:

GRANT SELECT ON SCHEMA::[dbo] TO [supaflow_reader];

List tables the user can access:

SELECT TABLE_SCHEMA, TABLE_NAME
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_TYPE = 'BASE TABLE'
ORDER BY TABLE_SCHEMA, TABLE_NAME;

Refresh schema in Supaflow:
- Set Schema Refresh Interval to 0
- Save and test connection again
Check for schema filtering:
- Supaflow discovers all schemas the user can read
- Verify the user has USAGE on the desired schemas

Fabric authentication or initialization fails

Problem:

The Entra token is rejected
Fabric reports that the workspace is missing or inaccessible

Solutions:

Confirm Tenant ID is the Directory (tenant) ID, Client Application ID is the Application (client) ID, and Client Secret contains the secret value rather than the Secret ID.
Confirm Fabric Workspace ID is the identifier after /groups/ in the workspace URL.
Confirm the tenant setting Service principals can call Fabric public APIs applies to the application.
Grant the service principal at least the Viewer workspace role and verify Database Name exactly matches the Warehouse item name.
Keep Query Mode set to Standard and Database Flavor set to Auto for a standard Fabric hostname.

Connection timeout

Problem:

Connection times out during setup
Slow query response
Test connection takes too long

Solutions:

Increase timeout in JDBC properties:
- loginTimeout=60 (60 seconds)
- queryTimeout=60
- Format: loginTimeout=60;queryTimeout=60
Check network latency:
```
ping your_database_host
```

Verify SQL Server performance:

-- Check for long-running queries
SELECT session_id, start_time, status, command, wait_type
FROM sys.dm_exec_requests
WHERE status = 'running'
ORDER BY start_time;

Azure SQL specific:
- Check DTU/vCore utilization in the Azure portal
- Scale up the database tier if consistently hitting resource limits

Frequently Asked Questions

How do I connect Supaflow to an on-premises SQL Server?

For production, deploy a self-hosted Docker Agent on a stable host inside the same private network as SQL Server. The agent connects to SQL Server over the private network and polls Supaflow over outbound HTTPS, so the database port stays private and the pipeline has a predictable network path.

For a quick proof-of-concept connectivity test, you can use a temporary TCP tunnel. Follow the ngrok or bore walkthrough, and do not use a temporary public tunnel for production or sensitive data.

Support

Need help? Contact us at support@supa-flow.io

Prerequisites​

Microsoft Entra Service Principal for Azure SQL​

What Gets Synced​

Sync Modes​

Configuration​

Step 1: Connection​

Step 2: Advanced Settings (Optional)​

Step 3: Test & Save​

Query Modes​

Standard Mode​

Change Tracking Mode​

Troubleshooting​

Connection refused​

Authentication failed​

Permission denied on tables​

TLS / Encryption errors​

Change Tracking errors​

Tables not appearing in schema​

Fabric authentication or initialization fails​

Connection timeout​

Frequently Asked Questions​

How do I connect Supaflow to an on-premises SQL Server?​

Related Pages​

Support​

Prerequisites

Microsoft Entra Service Principal for Azure SQL

What Gets Synced

Sync Modes

Configuration

Step 1: Connection

Step 2: Advanced Settings (Optional)

Step 3: Test & Save

Query Modes

Standard Mode

Change Tracking Mode

Troubleshooting

Connection refused

Authentication failed

Permission denied on tables

TLS / Encryption errors

Change Tracking errors

Tables not appearing in schema

Fabric authentication or initialization fails

Connection timeout

Frequently Asked Questions

How do I connect Supaflow to an on-premises SQL Server?

Related Pages

Support