Agents
Monitor and manage the services that execute your data pipelines.
If you are deploying via the Snowflake Native App (installed from the Snowflake Marketplace), follow the dedicated Snowflake Native App Deployment Guide for a step-by-step walkthrough with screenshots.
Overview
Agents are deployment services that run your Supaflow jobs. Each agent operates in an isolated environment (AWS ECS or Snowflake SPCS) with its own encryption keys and compute resources.
Supaflow is built with security at its core — customer data never leaves your environment. Only metadata is shared with Supaflow Cloud, encrypted with keys you control.
To access: Navigate to Settings → Agents in the sidebar.
How Agents Work
Supaflow uses strict control-plane and data-plane separation:
- Control plane - Coordinates work and manages job orchestration (Supaflow Cloud)
- Data plane - Processes all customer data (your environment only)
Agent Communication Flow
- Agent polls for jobs - Uses outbound HTTPS only. No inbound network access, firewall changes, or IP allowlists required.
- Agent decrypts job metadata - All job metadata is encrypted using your tenant's public key. The matching private key is stored securely with the agent, so only your agent can decrypt and run jobs.
- Agent executes jobs - Connects directly to your sources and destinations within your network. Data flows from source to destination without ever passing through Supaflow Cloud.
This architecture ensures your data stays within your environment while Supaflow coordinates the work.
What You Can Do
- Monitor agent health - View real-time status and connectivity
- Control Snowflake services - Suspend or resume SPCS agents to manage costs
- Manage encryption - Rotate keys and migrate datasource encryption
- Approve new agents - Activate registered agents for job execution
- Track metrics - See total agents, running count, jobs completed, and uptime
- Deploy agents - Set up agents in multiple environments
Deploying a Snowflake Agent
Why Deploy an Agent in Snowflake?
By deploying the agent within your Snowflake environment, you ensure that all data remains within your Snowflake account. This is important for:
- Compliance with regulatory requirements (HIPAA, SOC 2, etc.)
- Sensitive data that cannot be shared with another data processor
- Organizations that require complete data sovereignty
If these constraints don't apply to your use case, you can use a Supaflow-managed agent instead.
Deployment Workflow
Step 1: Start Deployment
- Go to Settings → Agents
- Click Deploy to Snowflake
Step 2: Run Snowflake Setup Script
Supaflow generates a deployment script that must run directly in your Snowflake account. This script:
- Grants required Snowpark Container Services privileges
- Creates schemas used by the agent
- Sets up the image repository and stage
- Defines outbound network rules and external access integration
To run the script:
- Click Copy Script in Supaflow
- Log in to Snowflake using Snowsight as an account admin
- Create a new worksheet and paste the script
- Review the script carefully - This is where you control network rules and restrict outbound access to only the systems the agent needs to reach
- Customize network rules if needed (add or remove endpoints for your specific data sources and destinations)
- Click Run All to execute the script
Important: The script provisions Snowflake resources required to run the agent as a native service. Review it thoroughly before running.
Step 3: Configure Agent Settings
After running the Snowflake script, return to Supaflow and configure how the agent will run:
- Controller datasource - Select the Snowflake datasource used to deploy and manage the agent
- Node size - Choose the agent compute size (determines processing power)
- Autoscaling limits - Configure min/max instances based on workload
These settings determine how much compute the agent has available and how it scales as workloads increase. The default settings work well for most workloads. You can always redeploy the agent later if you need to adjust these based on data volume or pipeline concurrency.
Step 4: Deploy the Agent
- Review your configuration
- Click Deploy Agent
- Monitor the deployment progress as Supaflow executes SQL statements in Snowflake to create and start the agent service
- Wait for the agent to finish starting and register successfully
- Click Done to return to the agents page
For Snowflake Native App agents: After deployment, grant the app role to your Supaflow users role so the controller datasource can manage the service:
GRANT APPLICATION ROLE <native_app_database>.app_public TO ROLE SUPA_ROLE;
-- Example (production):
GRANT APPLICATION ROLE SUPAFLOW_DI_AGENT.app_public TO ROLE SUPA_ROLE;
Step 5: Approve the Agent
- On the Agents page, you'll see your new agent with "Registered" or "Pending Approval" status
- Click the green Approve button
Why approval is required: Approval acts as a security gate, ensuring only explicitly trusted agents can run jobs. Once approved, the agent completes activation and prepares to accept work (this may take a few minutes).
Step 6: Verify Deployment
After deployment, verify everything is set up correctly:
In Supaflow:
- Refresh the agents page and confirm the agent shows "Running" status
- Navigate to Settings → Public Keys and verify there's an active public key for your tenant (this is used to encrypt job metadata)
In Snowflake:
- Confirm a dedicated Supaflow compute pool has been created
- Verify the
SUPAFLOW_AGENT_SERVICEis running - Optionally review the service configuration to confirm it matches your deployment settings
Your agent is now ready to process jobs!
Network Security
The deployment script creates network rules that control which external systems your agent can access. By default, the agent can only reach:
- Supaflow Cloud (for job polling and status updates)
- Endpoints you explicitly allow in the network rules
To add or remove allowed endpoints:
- Edit the deployment script before running it
- Add network rules for your specific data sources and destinations
- Remove any rules for systems you won't be connecting to
This ensures the agent can only communicate with approved systems, maintaining strict network security.
Understanding Agent Status
Lifecycle Status
| Status | What It Means |
|---|---|
| Registered | Agent connected but not yet approved for use |
| Pending Approval | Waiting for manual approval |
| Active | Approved and ready to process jobs |
| Suspended | Temporarily paused (Snowflake SPCS only) |
| Deactivated | Permanently stopped |
| Terminated | Service has been terminated |
Connectivity Status
Your agent's connectivity is shown on its card:
- Running - Agent is connected and processing jobs
- Connecting - Agent is initializing
- Stopped - No heartbeat received, agent may be down
Each agent card shows when the last heartbeat was received (e.g., "2 minutes ago") so you can monitor connection health.
Deployment Types
Supaflow supports two deployment options for agents:
Snowflake SPCS
What it is: Native Snowflake deployment running on dedicated Snowflake compute resources.
Key features:
- Private encryption keys per service
- Suspend/resume controls to manage costs
- Automatic service cleanup when deactivated
- Runs directly in your Snowflake account
Best for: Organizations already using Snowflake who want native integration and maximum data security.
AWS ECS
What it is: Managed container infrastructure running on AWS Fargate.
Key features:
- Serverless containers (no server management)
- Automatic scaling
- Managed deployment and updates
- Uses system encryption keys (shared across agents)
Best for: AWS-based infrastructure with minimal management overhead.
Agent Actions
Approve Agent
When to use: After deploying a new agent that shows "Registered" or "Pending Approval" status.
What it does:
- Activates the agent's encryption key
- Authorizes the agent to process jobs (security gate)
- Allows the agent to start accepting work from the job queue
How to do it: Click the green Approve button next to the agent's status badge.
Security note: Approval ensures only explicitly trusted agents can run jobs in your organization. After approval, you may see a suggestion to rotate keys if you have existing datasources that could benefit from private encryption.
Rotate Encryption Keys
When to use: When you want to improve security by migrating datasources to private encryption or updating to newer keys.
Three scenarios:
Switch to Private Key
Use this when you have datasources encrypted with the system key and want to migrate them to your agent's private key for better security isolation.
What happens: Datasources are re-encrypted using your agent's private key instead of the shared system key.
Switch to Latest Key
Use this when datasources are using an older version of your private key and you want to catch them up to the current version.
What happens: Datasources are re-encrypted with the most recent key version.
Rotate Private Key
Use this when you want to generate a completely new encryption key (e.g., after a security incident or as part of regular security maintenance).
What happens:
- Agent generates a new key pair
- All datasources are re-encrypted with the new key
- Old key is retired
Note: The button label changes based on which scenario applies to your situation, and shows how many datasources will be affected.
Suspend Service (Snowflake SPCS Only)
When to use: Temporarily stop a Snowflake agent to save on compute costs.
What it does:
- Pauses the Snowflake service
- Stops compute billing immediately
- Preserves all configuration and state
- Prevents new jobs from running until resumed
Common use cases:
- Stop non-production agents outside business hours
- Pause testing environments when not in use
- Respond to incidents by preventing further job execution
How to resume: Use the Resume Service action when you're ready to restart.
Resume Service (Snowflake SPCS Only)
When to use: Restart a suspended Snowflake agent.
What it does:
- Restarts the Snowflake service
- Resumes compute billing
- Agent reconnects and starts processing queued jobs
Note: Resume typically takes less than 1 minute.
Deactivate Agent
When to use: Permanently remove an agent from your organization.
What it does:
- Optionally migrates datasources to system encryption (if this is your last agent with a particular key)
- Drops the Snowflake service (if applicable)
- Marks the agent as deactivated
- Stops all job processing
Important considerations:
- If you have datasources encrypted with this agent's private key, you'll be prompted to either:
- Leave them as-is if you have other agents with the same key
- Migrate to system encryption if this is your last agent with that key
- Deactivation is permanent — you cannot reactivate a deactivated agent
- Requires organization admin role
Monitoring Agent Health
Metrics Dashboard
At the top of the Agents page, you'll see key metrics:
- Total Agents - All agents including deactivated ones
- Running - Currently connected and active agents
- Jobs Completed - Total jobs executed across all agents
- Total Uptime - Cumulative uptime hours for all agents
These metrics help you understand your overall agent fleet health at a glance.
Agent Cards
Each agent displays:
- Agent identifier and current status
- Last heartbeat timestamp
- Deployment type description
- Service type badge (SPCS or ECS)
- Node size (if configured)
- Current session uptime
Use the three-dot menu on each card to access available actions based on the agent's status.
Troubleshooting
Agent Shows "Stopped" But Service is Running
What it means: The agent hasn't sent a heartbeat recently, even though the underlying service appears to be active.
How to resolve:
- Check the "last heartbeat" timestamp — if it's been more than 5 minutes, there's a connectivity issue
- For Snowflake agents, check if the service was manually suspended
- Review Snowflake service logs to identify container issues
- Verify network connectivity between your deployment environment and Supaflow
- Try resuming the service if it was suspended
- Contact support if the issue persists
Common causes:
- Snowflake network policies blocking outbound connections
- Service suspended manually in Snowflake
- Agent container crashed
Key Rotation Shows Count But Reports No Datasources
What it means: The displayed count may be stale or datasources may have been deleted/deactivated.
How to resolve:
- Refresh the agents page to update the encryption summary
- Verify datasources are in an active state (not draft or deleted)
- Check the Public Keys page to see which datasources are actually using each key
- Try the rotation again after refreshing
Deactivate Fails with Permission Error
What it means: You don't have the required permissions to deactivate agents.
How to resolve:
- Agent deactivation requires organization admin role
- Check your role in Settings → Organization
- Ask your organization owner to grant you admin permissions
- If you're the owner, contact support
Service Suspend/Resume Times Out
What it means: The Snowflake operation is taking longer than expected (>2 minutes).
How to resolve:
- Verify the Snowflake warehouse is running and has available resources
- Check that the agent's controller datasource is still valid
- Test the operation manually in Snowflake to see if there are account-level issues
- Check for locks on the service from other users or processes
- Review the service status in Snowflake
Common causes:
- Warehouse is suspended or out of resources
- Another service operation is in progress
- Snowflake account experiencing issues
Best Practices
Monitor Agent Connectivity Daily
What to check:
- All production agents show "Running" status
- Last heartbeat timestamps are recent (< 5 minutes)
- Uptime metrics are consistent (no frequent restarts)
Recommended:
- Set up external monitoring for critical agents
- Alert when an agent is disconnected for more than 10 minutes
- Track job completion rates to identify performance issues
Plan Key Rotations During Maintenance Windows
Why: Re-encryption jobs can take 1-2 minutes per 100 datasources.
Best approach:
- Schedule rotation during low-activity periods
- Notify your team about potential brief job delays
- Execute the rotation via the "Rotate Keys" button
- Monitor the operation until completion
- Verify the new key is active on the Public Keys page
Recommended frequency:
- System → Private migration: One-time per environment
- Private key rotation: Annually or after security incidents
Use Suspend/Resume for Cost Control
For Snowflake SPCS agents:
- Suspended services stop compute billing immediately
- Configuration and state are preserved
- Resume takes less than 1 minute
Good use cases:
- Suspend non-production agents outside business hours
- Stop testing environments when not actively used
- Temporarily halt job execution during incidents
Automation options:
- Use Snowflake scheduled tasks to suspend/resume on a schedule
- Call Supaflow API from your automation tools
- Set up cost monitoring alerts to catch unexpected usage
Deploy Multiple Agents for High Availability
Recommended setup:
Production:
- 2-3 agents in the same region
- Share the same encryption key
- Jobs automatically load balance across available agents
Staging:
- 1 agent (cost-effective for lower volume)
Development:
- 1 agent, suspend when not in use to save costs
Benefits:
- Zero-downtime deployments (deactivate one agent at a time)
- Automatic failover if an agent disconnects
- Higher job concurrency and throughput
- Better resilience during upgrades
Related Pages
- Public Keys - View encryption keys and datasource assignments
- Settings - Settings overview
- Activities - Monitor activities executed by agents
Support
Need help with agent management? Contact us at support@supa-flow.io
Ask Claude