Snowflake Destination
Connect Snowflake as a destination for loading data from your sources into your warehouse.
For a commercial overview of the connector, see the Snowflake connector page. If you are building a source-to-Snowflake pipeline, start with the Redshift to Snowflake, Google Drive to Snowflake, SQL Server to Snowflake, or Oracle TM to Snowflake guides.
Prerequisites
Before you begin, ensure you have:
- ✅ An active Snowflake account with admin privileges (Don't have one? Sign up for a free trial)
- ✅ A warehouse configured for data loading (or ability to create one)
- ✅ Target database and schema created
- ✅ Appropriate role with CREATE TABLE permissions
- ✅ Network policies configured (if applicable)
- ✅ Whitelist Supaflow IP (If Required)
If your Snowflake account has IP restrictions enabled, add this IP to your Network Access settings:
18.214.240.61
For reference: https://docs.snowflake.com/en/user-guide/network-policies
Follow these steps to set up your Snowflake environment:
- Log in to your Snowflake data warehouse
- Click "Copy Script" button below to copy the snowflake setup script
- In Snowflake, click Projects → Worksheets → + (New Worksheet)
- Paste the entire script into the new worksheet
- IMPORTANT: Modify these values as needed:
- Change the user_password (required)
- Optionally modify role_name, user_name, warehouse_name, database_name, or schema_name
- Click the dropdown arrow next to the Run button ▼ and select "Run All"
Don't use the regular "Run" button — it only executes the current statement. Use "Run All" to execute the entire script.
-- create variables for user / password / role / warehouse / database (needs to be uppercase for objects)
set role_name = 'SUPA_ROLE';
set user_name = 'SUPA_USER';
set warehouse_name = 'SUPA_WH';
set database_name = 'SUPA_DB';
set schema_name = 'SUPA_SCHEMA';
set fqn_schema_name = concat($database_name,'.',$schema_name);
-- change role to securityadmin for user/role steps
use role securityadmin;
-- create role for Supaflow
create role if not exists identifier($role_name);
grant role identifier($role_name) to role SYSADMIN;
-- create a user for SupaFlow
create user if not exists identifier($user_name)
type = SERVICE
default_role = $role_name
default_warehouse = $warehouse_name;
grant role identifier($role_name) to user identifier($user_name);
-- set binary_input_format to BASE64
ALTER USER identifier($user_name) SET BINARY_INPUT_FORMAT = 'BASE64';
-- change role to sysadmin for warehouse/database steps
use role sysadmin;
-- create a warehouse for Supaflow
create warehouse if not exists identifier($warehouse_name)
warehouse_size = xsmall
warehouse_type = standard
auto_suspend = 60
auto_resume = true
initially_suspended = true;
-- create database for Supaflow
create database if not exists identifier($database_name);
create schema if not exists identifier($fqn_schema_name);
-- grant supaflow role access to warehouse
grant USAGE on warehouse identifier($warehouse_name) to role identifier($role_name);
-- grant supaflow access to the database
grant ALL on database identifier($database_name) to role identifier($role_name);
grant ALL on ALL schemas in database identifier($database_name) to role identifier($role_name);
grant ALL on schema identifier($fqn_schema_name) to role identifier($role_name);
-- change role to ACCOUNTADMIN for STORAGE INTEGRATION support (only needed for Snowflake on GCP)
use role ACCOUNTADMIN;
-- transfer ownership of database and schema to SUPA_ROLE
GRANT OWNERSHIP ON DATABASE identifier($database_name)
TO ROLE identifier($role_name) COPY CURRENT GRANTS;
GRANT OWNERSHIP ON SCHEMA identifier($fqn_schema_name)
TO ROLE identifier($role_name) COPY CURRENT GRANTS;
grant CREATE INTEGRATION on account to role identifier($role_name);
grant CREATE EXTERNAL VOLUME on account to role identifier($role_name);
grant CREATE DATABASE on account to role identifier($role_name);
GRANT EXECUTE TASK ON ACCOUNT TO ROLE identifier($role_name);
GRANT EXECUTE MANAGED TASK ON ACCOUNT TO ROLE identifier($role_name);
GRANT APPLICATION ROLE SNOWFLAKE.EVENTS_VIEWER TO ROLE identifier($role_name);
grant all on future schemas in database identifier($database_name) to role identifier($role_name);
grant all on future tables in database identifier($database_name) to role identifier($role_name);
grant all on future views in database identifier($database_name) to role identifier($role_name);
-- grants for querying Iceberg tables via external volume + catalog integration
grant CREATE TABLE on all schemas in database identifier($database_name) to role identifier($role_name);
grant CREATE TABLE on future schemas in database identifier($database_name) to role identifier($role_name);
-- grant usage on future file formats and stages
GRANT USAGE ON FUTURE FILE FORMATS IN DATABASE identifier($database_name) TO ROLE identifier($role_name);
GRANT USAGE ON FUTURE STAGES IN DATABASE identifier($database_name) TO ROLE identifier($role_name);
Python Task Permissions
The current Snowflake setup script includes the permissions used by Python tasks:
- Permission to create procedures and tasks in the configured database and schema
EXECUTE TASKandEXECUTE MANAGED TASKfor Snowflake-managed serverless execution- The
SNOWFLAKE.EVENTS_VIEWERapplication role for reading Python log messages from the default event table
If you configured the Snowflake role with an older copy of the script, run the current script again or ask a Snowflake administrator to apply the missing grants. Your Python code also runs with the datasource role's privileges, so grant that role access to every table, view, schema, and operation the task needs.
The Logs tab in Supaflow reads SNOWFLAKE.TELEMETRY.EVENTS_VIEW. If the account uses a custom active event table, the Python task still runs, but its log entries are not displayed in Supaflow. See Snowflake's event table overview and CREATE TASK access requirements.
Optional: Direct S3 External-Stage Loads
Most Snowflake destinations do not need an external stage. Leave External Storage Integration blank unless the source connector writes Snowflake-readable load files to S3 and the Snowflake destination should load directly from that S3 path.
In this flow, the source writes each job's files to an S3 bucket and prefix. The Snowflake destination creates a temporary external stage over that same S3 location and runs COPY INTO from the stage. This avoids downloading the source output and re-uploading it into a Snowflake internal stage.
The first supported use case is Amazon Redshift to Snowflake: the Redshift source writes files with UNLOAD, and Snowflake reads those files through a storage integration. For the full source and destination walkthrough, see the Redshift to Snowflake tutorial. The same Snowflake destination setting can be used by other Supaflow source connectors when their docs say they support direct S3 external-stage loads.
Use this option when all of the following are true:
- The source connector supports direct S3 external-stage loads.
- The destination is Snowflake.
- You want Snowflake to read the source files directly from S3.
- Your Snowflake administrator can create a storage integration and your AWS administrator can update the IAM role trust policy.
Do not use this field for pipelines whose source connector does not explicitly support direct S3 external-stage loads. Those pipelines use the standard Snowflake loading path.
Step 1: Confirm the Source S3 Location
In the source connector configuration, note the S3 bucket and prefix that will hold the direct-load files. For Redshift sources, these fields are:
- S3 Staging Bucket
- S3 Staging Prefix
- Bucket Region
The Snowflake storage integration must allow the same S3 bucket and prefix. For example, if the source writes to s3://company-pipeline-stage/redshift/, the Snowflake integration's STORAGE_ALLOWED_LOCATIONS must include s3://company-pipeline-stage/redshift/ or a parent prefix that covers it.
Step 2: Create the Snowflake Storage Integration
Run the script below once in Snowflake. Update the variables before running it:
integration_name: the Snowflake integration name you will enter in Supaflow, for exampleSUPAFLOW_EXTERNAL_S3_INTEGRATIONstorage_role_arn: the AWS IAM role Snowflake will assume to read the S3 staging locationstorage_allowed_location: the S3 bucket and prefix used by the source connector's direct-load files
-- create_supa_storage_integration.sql
--
-- OPTIONAL. Enables Redshift -> Snowflake direct S3 external-stage loads.
-- Run this ONCE per Snowflake account, as SUPA_ROLE (which already holds
-- CREATE INTEGRATION; see create_supa_user.sql). The Supaflow connector does
-- NOT create this integration at runtime -- it only references it by name via
-- the Snowflake datasource advanced property `externalStorageIntegration`.
--
-- Two-step setup:
-- 1. Run this script. Fill in the three variables below first.
-- 2. Run DESC INTEGRATION (at the bottom) and copy STORAGE_AWS_IAM_USER_ARN
-- and STORAGE_AWS_EXTERNAL_ID into the AWS IAM role's trust policy so that
-- Snowflake's principal can assume the role (sts:AssumeRole with the
-- matching sts:ExternalId condition). The integration only works AFTER the
-- AWS trust policy is updated.
--
-- IMPORTANT: storage_allowed_location MUST cover whatever S3 bucket/prefix
-- Redshift UNLOADs to (the per-job stage URL must fall within it).
set integration_name = 'SUPAFLOW_EXTERNAL_S3_INTEGRATION';
set storage_role_arn = 'arn:aws:iam::<account-id>:role/<s3-read-role>'; -- s3:GetObject/ListBucket on the UNLOAD bucket; may be the SAME role your Redshift source already uses
set storage_allowed_location = 's3://<bucket>/<prefix>/'; -- MUST cover the Redshift UNLOAD bucket/prefix (the per-job stage URL must fall within it)
use role SUPA_ROLE;
CREATE STORAGE INTEGRATION IF NOT EXISTS identifier($integration_name)
TYPE = EXTERNAL_STAGE
STORAGE_PROVIDER = 'S3'
ENABLED = TRUE
STORAGE_AWS_ROLE_ARN = $storage_role_arn
STORAGE_ALLOWED_LOCATIONS = ($storage_allowed_location);
-- Redundant when SUPA_ROLE created (and therefore owns) the integration above, but
-- required if an operator runs this as ACCOUNTADMIN instead: without USAGE, SUPA_ROLE
-- cannot create the external stage at load time. Safe to keep here -- the CREATE above
-- guarantees the integration exists, and GRANT has no IF EXISTS form.
GRANT USAGE ON INTEGRATION identifier($integration_name) TO ROLE SUPA_ROLE;
-- Read these two values and add them to the IAM role's trust policy:
-- Principal AWS = STORAGE_AWS_IAM_USER_ARN
-- sts:ExternalId = STORAGE_AWS_EXTERNAL_ID
DESC INTEGRATION identifier($integration_name);
The IAM role must be able to list and read the source files in S3. If you later enable Purge External Stage After Load, the same role also needs permission to delete objects from the staging prefix.
The standard Snowflake setup script grants CREATE INTEGRATION to SUPA_ROLE. If your Snowflake administrator creates the storage integration with a different role, grant the Supaflow role access to it:
GRANT USAGE ON INTEGRATION SUPAFLOW_EXTERNAL_S3_INTEGRATION TO ROLE SUPA_ROLE;
GRANT CREATE STAGE ON SCHEMA SUPA_DB.SUPA_SCHEMA TO ROLE SUPA_ROLE;
Step 3: Update the AWS Trust Policy
After you run the script, Snowflake returns STORAGE_AWS_IAM_USER_ARN and STORAGE_AWS_EXTERNAL_ID from DESC INTEGRATION.
Add those values to the AWS IAM role's trust policy so Snowflake can assume the role:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "<STORAGE_AWS_IAM_USER_ARN>"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"sts:ExternalId": "<STORAGE_AWS_EXTERNAL_ID>"
}
}
}
]
}
If you use the same IAM role for Redshift, Supaflow, and Snowflake, keep the existing trust statements for Redshift and Supaflow and add Snowflake as an additional trusted principal. Do not replace the existing trust policy unless Snowflake is the only service that needs the role.
For Snowflake's full storage integration flow, see Configuring a Snowflake storage integration for Amazon S3.
Step 4: Configure the Snowflake Destination in Supaflow
In your Snowflake destination, open Advanced Settings and set:
- External Storage Integration: enter the Snowflake integration name, for example
SUPAFLOW_EXTERNAL_S3_INTEGRATION - Purge External Stage After Load: leave disabled unless the Snowflake IAM role has
s3:DeleteObjectpermission and you want Snowflake to delete the source files after a successful load
The field expects only the Snowflake integration name. Do not enter an S3 bucket, role ARN, or credentials in this field.
Configuration
Step 1: Connection
Account Identifier*Your full Snowflake account identifier.
Example: orgname-account_name.snowflakecomputing.com
or
Example: account_locator.cloud_region_id.cloud.snowflakecomputing.com
Compute warehouse for queries
Example: SUPA_WH (created by script)
Target database name
Example: SUPA_DB (created by script)
Target schema name
Example: SUPA_SCHEMA (created by script)
Step 2: Authentication
Username*Your Snowflake username
Example: SUPA_USER (created by script)
Choose authentication method
Options: keypair, basic (password), or spcs_oauth
Default: keypair
| Auth Type | Use Case |
|---|---|
| keypair | Recommended for all standard connections. Uses RSA key pair authentication. |
| basic | Password-based authentication. Requires a Snowflake user with password login enabled. |
| spcs_oauth | Most secure option. Used by Native App destinations where the agent runs inside Snowflake (SPCS). No credentials are stored -- authentication uses Snowflake-injected OAuth tokens. |
Key Pair Authentication (Recommended)
When keypair is selected, you have two options for providing the private key:
- Create Key -- Click to have Supaflow automatically generate a key pair and passphrase. This is the easiest option.
- Upload Key -- Click to upload an existing private key file.
After clicking Create Key, Supaflow generates a key pair and displays a SQL statement:
ALTER USER SUPA_USER SET RSA_PUBLIC_KEY='MIIBIjAN...';
- Click Copy to copy the SQL statement
- Open a Snowflake worksheet as SECURITYADMIN (or ACCOUNTADMIN)
- Paste and run the SQL statement
You must complete this step before the connection test will succeed.
Passphrase for the encrypted private key
Auto-generated when using Create Key. Only required if your uploaded key is encrypted.
Password Authentication
When basic is selected:
Your Snowflake password
Use the password set in the setup script
SPCS OAuth Authentication
When a Supaflow agent runs inside Snowflake as a Native App (via Snowpark Container Services), it uses spcs_oauth authentication. This is the most secure option:
- Zero stored credentials -- No passwords, keys, or secrets are configured in the datasource. Authentication uses OAuth tokens that Snowflake injects directly into the running container at runtime.
- No credential exposure risk -- Since no credentials exist in the datasource configuration, there is nothing that can be leaked, stolen, or rotated.
- Private network path -- Data flows directly from the agent to Snowflake within the Snowflake infrastructure, without traversing the public internet.
After you approve a Native App agent in Settings > Agents, Supaflow shows a Create Snowflake Destination prompt. Click Create Destination to create the spcs_oauth destination. You can also create it later from the agent menu. You do not need to select this auth type manually in the destination form. See the Snowflake Native App Deployment Guide for the full setup walkthrough.
Step 3: Advanced Settings (Optional)
RoleThe Snowflake role to use
Example: SUPA_ROLE (created by script)
Other options: ACCOUNTADMIN, SYSADMIN
Query to test connectivity
Default: SELECT 1
Maximum seconds a Snowflake query may run. Use 0 for no timeout.
Default: 0 (Range: 0 to 86400)
Tag to apply to all queries for tracking
Default: Supaflow
Enable query statistics logging
Default: Disabled
Snowflake storage integration name for direct S3 external-stage loads. Leave blank for the standard Snowflake loading path.
Create the integration in Snowflake before entering the name here. For Snowflake's setup flow, see Configuring a Snowflake storage integration for Amazon S3.
Example: SUPAFLOW_EXTERNAL_S3_INTEGRATION
Delete files from the external S3 stage after a successful direct load. Only enable this when External Storage Integration is set and the Snowflake IAM role has s3:DeleteObject permission.
Default: Disabled
Interval in minutes for schema metadata refresh
0 = refresh before every pipeline execution
-1 = disable schema refresh
Default: 30 (Range: -1 to 10080)
JDBC Read Controls
The Snowflake connector inherits these JDBC read controls. They can generally be left at their defaults for Snowflake destination loads. Change them only when Supaflow support asks you to tune a Snowflake read path.
Chunked Read EnabledEnable keyset-based chunked reads for full-table reads with a deterministic primary key.
Default: Disabled
Use SQL window limits for chunked reads.
Default: Disabled
Maximum rows per chunk when chunked read window limits are enabled.
Default: 1,000,000 (Range: 1,000 to 5,000,000)
Reduce chunk size for very wide rows based on the configured memory target.
Default: Disabled
Target memory budget used when memory-aware chunk limits are enabled.
Default: 50 MB (Range: 1 to 4096)
Step 4: Test & Save
After configuring all required properties, click Test & Save to verify your connection and save the destination.
Loading Behavior
Supaflow creates destination tables when needed and applies the pipeline's configured load behavior. Keyed merge paths update existing rows and add new rows. Overwrite and reset paths can recreate a target table when the selected operation requires it.
Schema Evolution
Supaflow adds compatible columns and applies supported type changes when schema evolution permits them. Removed source columns remain in Snowflake instead of being dropped automatically.
When an existing table must be recreated, Supaflow preserves a compatible Snowflake clustering key whose referenced columns remain available. It also restores a suspended automatic reclustering state when Snowflake allows it. Review the activity if a table recreation reports that its previous physical design could not be reapplied.
Troubleshooting
Common issues and their solutions:
Invalid account identifier
Problem:
- Connection fails with "Invalid account identifier" error
- Cannot connect to Snowflake account
Solutions:
- Use the full account identifier including region and cloud provider
- ✅ Correct:
xy12345.us-east-1.snowflakecomputing.com - ❌ Wrong:
xy12345
- ✅ Correct:
- Find your account identifier:
- Log into Snowflake Console
- Look at the bottom left corner
- Copy the full identifier shown
- Verify format:
- New format:
orgname-account_name.snowflakecomputing.com - Legacy format:
account_locator.region.cloud.snowflakecomputing.com
- New format:
Warehouse suspended or not running
Problem:
- Queries fail with "Warehouse not running" error
- Connection test times out
- Data loading fails intermittently
Solutions:
- Check warehouse status in Snowflake:
- Navigate to: Admin → Warehouses
- Verify warehouse state is "Started" or "Auto-Resume"
- Enable auto-resume:
ALTER WAREHOUSE SUPA_WH SET AUTO_RESUME = TRUE; - Manually start warehouse:
ALTER WAREHOUSE SUPA_WH RESUME; - Verify warehouse name:
- Ensure the warehouse name in your config matches exactly (case-sensitive)
Network policy blocking connection
Problem:
- Connection refused due to IP restrictions
- "IP address not allowed" error
- Connection timeout
Solutions:
- Add Supaflow IP to your network policy:
-- Create network policy (if doesn't exist)
CREATE NETWORK POLICY supaflow_policy
ALLOWED_IP_LIST = ('18.214.240.61');
-- Or modify existing policy
ALTER NETWORK POLICY your_existing_policy
SET ALLOWED_IP_LIST = ('your.existing.ip', '18.214.240.61'); - Verify network policies:
- Go to: Admin → Security → Network Policies
- Ensure
18.214.240.61is in the allowed list
- Contact your Snowflake administrator if you don't have permission to modify network policies
Insufficient permissions
Problem:
- Cannot create tables in target schema
- "Access denied" or "Insufficient privileges" error
- Data loading fails with permission error
Solutions:
- Re-run the setup script provided in Prerequisites section
- Manually verify permissions:
-- Check current grants
SHOW GRANTS TO ROLE SUPA_ROLE;
-- Grant necessary permissions
GRANT CREATE TABLE, INSERT, UPDATE, DELETE ON SCHEMA SUPA_DB.SUPA_SCHEMA TO ROLE SUPA_ROLE;
GRANT USAGE ON DATABASE SUPA_DB TO ROLE SUPA_ROLE;
GRANT USAGE ON WAREHOUSE SUPA_WH TO ROLE SUPA_ROLE; - Verify role assignment:
-- Check user's default role
SHOW USERS LIKE 'SUPA_USER';
-- Assign role if missing
GRANT ROLE SUPA_ROLE TO USER SUPA_USER;
Authentication failed
Problem:
- "Invalid username or password" error
- Key-pair authentication fails
- Connection refused
Solutions:
- For password authentication:
- Verify password matches what you set in setup script
- Check for special characters that might need escaping
- Reset password if needed:
ALTER USER SUPA_USER SET PASSWORD = 'your_new_password';
- For key-pair authentication:
- Ensure private key is in correct PEM format
- Verify public key is assigned to user in Snowflake
- Check if private key is encrypted (requires passphrase)
- Test key pair:
-- Assign public key to user
ALTER USER SUPA_USER SET RSA_PUBLIC_KEY='your_public_key';
Database or schema does not exist
Problem:
- "Object does not exist" error
- Cannot find database or schema
- Setup test fails
Solutions:
- Verify database exists:
SHOW DATABASES LIKE 'SUPA_DB'; - Create database if missing:
CREATE DATABASE IF NOT EXISTS SUPA_DB; - Verify schema exists:
SHOW SCHEMAS IN DATABASE SUPA_DB; - Create schema if missing:
CREATE SCHEMA IF NOT EXISTS SUPA_DB.SUPA_SCHEMA; - Check naming:
- Database and schema names are case-sensitive
- Ensure exact match with configuration
External storage integration cannot access S3
Problem:
- A pipeline fails during the direct external-stage load
- Error mentions a storage integration, allowed locations,
CREATE STAGE,COPY INTO, or S3 access denied
Solutions:
- Confirm the Snowflake destination has the integration name.
- In Advanced Settings, External Storage Integration should contain only the Snowflake integration name, such as
SUPAFLOW_EXTERNAL_S3_INTEGRATION - Leave the field blank for pipelines whose source does not support direct S3 external-stage loads
- In Advanced Settings, External Storage Integration should contain only the Snowflake integration name, such as
- Confirm the allowed S3 location covers the source staging path.
- The integration's
STORAGE_ALLOWED_LOCATIONSmust include the S3 bucket and prefix used by the source connector's direct-load files - For Redshift sources, this means the Redshift source S3 Staging Bucket and S3 Staging Prefix
- If the source writes to
s3://company-pipeline-stage/redshift/jobs/..., an allowed location ofs3://company-pipeline-stage/other-prefix/will fail
- The integration's
- Update the AWS IAM role trust policy after
DESC INTEGRATION.- The trust policy must allow the Snowflake
STORAGE_AWS_IAM_USER_ARN - The trust policy must use the matching
STORAGE_AWS_EXTERNAL_ID
- The trust policy must allow the Snowflake
- Check Snowflake privileges.
GRANT USAGE ON INTEGRATION SUPAFLOW_EXTERNAL_S3_INTEGRATION TO ROLE SUPA_ROLE;
GRANT CREATE STAGE ON SCHEMA SUPA_DB.SUPA_SCHEMA TO ROLE SUPA_ROLE; - Keep purge disabled unless delete access is configured.
- If Purge External Stage After Load is enabled, the IAM role must have
s3:DeleteObjecton the staging prefix - If delete access is not approved, leave purge disabled and use an S3 lifecycle policy or external cleanup process
- If Purge External Stage After Load is enabled, the IAM role must have
Connection timeout
Problem:
- Connection times out after waiting
- No response from Snowflake
- Intermittent connectivity issues
Solutions:
- Check Snowflake service status:
- Visit: Snowflake Status Page
- Verify network connectivity:
- Ensure your network allows outbound HTTPS (443)
- Check if corporate firewall is blocking Snowflake
- Test from different network:
- Try from a different internet connection
- Verify if issue is network-specific
- Contact support if timeout persists
Support
Need help? Contact us at support@supa-flow.io