S3 Connector

Load data into Amazon S3 as Parquet files with optional AWS Glue Data Catalog integration for querying with Athena, Spark, or other analytics tools. Authenticates via cross-account IAM role assumption.

DestinationBronze

Start Free View Documentation

Why Supaflow

All connectors included

No per-connector fees. Every connector is available on every plan.

Pay for compute, not rows

Credit-based pricing. No per-row charges, no MAR surprises.

One platform

Ingestion, dbt Core transformation, reverse ETL, and orchestration in a single workspace.

Capabilities

Cross-Account IAM Role Assumption

Authenticates by assuming an IAM role in your AWS account. You create the role with a trust policy allowing Supaflow to assume it, plus S3 and optional Glue permissions.

Parquet File Output

Data is written as Parquet files to your S3 bucket with an optional path prefix. Concurrent uploads are configurable for throughput.

AWS Glue Data Catalog Integration

Optionally registers tables and partitions in the AWS Glue Data Catalog. Databases are created with a configurable prefix. Enables querying via Athena, Spark, Redshift Spectrum, and other Glue-compatible tools.

S3 Path Prefix and Partitioning

Configure an optional S3 path prefix for all files. Data is organized by source schema and table name within the prefix.

Supported Objects

AWS Resources

S3 Bucket

Target S3 bucket where Parquet files are written.

IAM Role

Cross-account IAM role with S3 write and optional Glue permissions.

Glue Data Catalog

Optional catalog registration for Athena/Spark queryability.

How It Works

Create S3 and Glue IAM policies

In AWS IAM, create a policy granting s3:PutObject, s3:GetObject, s3:DeleteObject, and s3:ListBucket on your target bucket. Optionally create a Glue policy granting glue:CreateDatabase, glue:CreateTable, glue:UpdateTable, glue:GetDatabase, glue:GetTable, and glue:BatchCreatePartition.

Create an IAM role with trust policy

Create an IAM role with a trust policy allowing Supaflow to assume it (using the Supaflow AWS account ID and external ID shown in the Supaflow UI). Attach the S3 and Glue policies to the role.

Enter configuration in Supaflow

Provide the S3 bucket name, AWS region, optional path prefix, and the IAM Role ARN. Enable Glue sync and set the database prefix if using the Data Catalog.

Test and save

Click Test & Save to verify that Supaflow can assume the role and write to the bucket.

Use Cases

Data lake ingestion

Load data from SaaS sources into S3 as Parquet files, registered in the Glue Data Catalog for querying with Athena or Spark without managing a traditional warehouse.

Archival and compliance

Write source data to S3 for long-term storage and compliance. S3 lifecycle policies can move data to cheaper storage tiers automatically.

Multi-tool analytics

Store data in S3 with Glue Catalog metadata so it is accessible from Athena, Redshift Spectrum, EMR, Databricks, and other tools simultaneously.

Frequently Asked Questions

What file format does Supaflow write to S3?

Supaflow writes data as Parquet files. Parquet is a columnar format that is compact and supports efficient querying by analytics tools.

Is AWS Glue required?

No. Glue Data Catalog integration is optional. Without it, data is still written to S3 as Parquet files but is not registered in any catalog. You can use external tools to catalog the files later.

How does authentication work?

Supaflow uses cross-account IAM role assumption. You create an IAM role in your AWS account with a trust policy allowing Supaflow to assume it. No AWS access keys are stored in Supaflow.