SFTP Connector

Sync structured data from CSV, TSV, JSON, and JSONL files on SFTP servers. Supports gzip and ZIP compression, password and RSA key authentication, and incremental sync based on file modification time.

SourceBronze

Start Free View Documentation

Why Supaflow

All connectors included

No per-connector fees. Every connector is available on every plan.

Pay for compute, not rows

Credit-based pricing. No per-row charges, no MAR surprises.

One platform

Ingestion, dbt Core transformation, reverse ETL, and orchestration in a single workspace.

Capabilities

Password and RSA Key Authentication

Connect with SSH password or RSA private key (PEM format). Supports passphrase-protected keys and automatic fallback from key to password authentication. Credentials are encrypted at rest and never written to disk.

Multiple File Formats

Reads CSV, TSV, JSON, and JSONL files. Gzip-compressed files (.csv.gz, .json.gz) are decompressed transparently. ZIP archives containing CSV/TSV entries are extracted and parsed. Configurable delimiter, encoding, and header settings for CSV/TSV.

Multi-Folder Table Discovery

Files directly in the configured root folder become one table. Each immediate child folder becomes a separate table and is scanned recursively for files. Column names are unioned across files within each table.

Incremental Sync via File Modification Time

Uses SFTP file modification timestamps (st_mtime) to detect changed files. After the initial full sync, only files modified since the last sync are re-read. Per-file checkpointing tracks progress for crash recovery.

Supported Objects

File Types

CSV

Comma-separated values. Configurable delimiter (comma, pipe, semicolon, tab), encoding (UTF-8, Latin-1, Windows-1252, ASCII), and header row settings.

TSV

Tab-separated values. Tab delimiter applied automatically.

JSON

JSON files containing an array of objects or a single object. Nested values serialized as JSON strings.

JSONL / NDJSON

Newline-delimited JSON. One JSON object per line. Files with .jsonl or .ndjson extensions.

Compression

Gzip (.gz)

Gzip-compressed files decompressed transparently. Format detected from extension (e.g., .csv.gz, .json.gz).

ZIP (.zip)

ZIP archives with CSV/TSV entries extracted and parsed. Non-matching entries skipped. Protected against ZIP bombs with size and entry count limits.

How It Works

Prepare SFTP credentials

Ensure you have SSH access to the SFTP server with a username and either a password or RSA private key. The user account needs read access to the folder containing your data files.

Configure the source in Supaflow

Enter the SFTP host, port, and authentication credentials. Specify the root folder path, file type (CSV, TSV, JSON, or JSONL), and optional file pattern regex. Configure CSV-specific settings like delimiter and encoding if needed.

Test and save

Click Test & Save to verify that Supaflow can connect to the SFTP server and list the configured folder. Supaflow discovers child folders as separate tables and infers schemas from file content.

Use Cases

Automated file drop ingestion

Ingest CSV or JSON files dropped into SFTP folders by upstream systems, partners, or vendors. Supaflow detects new and modified files automatically on each sync.

Legacy system data extraction

Many legacy ERP, financial, and logistics systems export data as flat files to SFTP. Supaflow reads these exports directly without custom scripts or ETL code.

Multi-source file consolidation

Combine files from multiple sources landing in different SFTP subfolders into separate warehouse tables from a single pipeline.

Frequently Asked Questions

How are folders mapped to tables?

Files directly in the configured root folder become one table (named after the root folder). Each immediate child folder with matching files becomes a separate table. Child folders are scanned recursively, so files in nested subfolders are included in the child folder's table.

What happens if files in the same folder have different columns?

Supaflow unions all discovered columns across files and fills missing values with null. This allows files with slightly different schemas to coexist in the same folder.

Are gzip and ZIP files supported?

Yes. Gzip-compressed files (.csv.gz, .json.gz, etc.) are decompressed transparently based on the file extension. ZIP archives containing CSV or TSV entries are extracted and parsed. ZIP bomb protection is built in with configurable size and entry count limits.

What key types are supported?

RSA private keys in PEM format are supported, including passphrase-protected keys. ED25519 and ECDSA keys are not currently supported.