Skip to main content

Python Tasks

Use Python tasks to run Snowpark transformations and other Python logic directly in Snowflake. Python tasks use Snowflake-managed serverless compute and can run manually, on a schedule, or between pipeline steps in an orchestration.

Prerequisites​

You need an active Snowflake datasource. The Snowflake role must be able to create procedures and serverless tasks in the datasource schema and access every object used by the code. See Snowflake Destination for the current setup script and required privileges.

Creating a Python Task​

  1. Navigate to Tasks in the sidebar
  2. Click + Create Task
  3. Select Python Task
  4. Select an active Snowflake datasource
  5. Configure the handler and timeout
  6. Write the Python code
  7. Click Run
  8. Verify success in the Result panel
  9. Click Next, name the task, and click Save Task

Every task must run successfully before you can continue. The Next button is enabled only after the current code and configuration complete successfully. If you change the code, datasource, handler, or timeout after a successful test, run the task again before continuing.

Python Task Configuration​

Handler (required): Name of the Python function Snowflake should call. The default is run.

Timeout (required): Maximum run time. Choose from 2 minutes through 7 days. The default is 1 hour.

Python code (required): Inline Python source that defines the handler function.

Python tasks use Snowflake-managed serverless compute, so they do not use the virtual warehouse configured on the datasource.

The handler receives a Snowpark Session as its first argument and should return a string. The value in Handler must match the function name in the code.

import logging

from snowflake.snowpark import Session


logger = logging.getLogger("customer_summary")
logger.setLevel(logging.WARNING)


def run(session: Session) -> str:
    customer_count = session.table("SUPA_DB.SUPA_SCHEMA.CUSTOMERS").count()
    logger.warning("customer summary processed %s rows", customer_count)
    return f"Processed {customer_count} customers"

For this example, keep Handler set to run. Replace the table name with a table that the Snowflake role can read.

Current Python Environment​

  • Python 3.12
  • Python standard library
  • snowflake-snowpark-python

Supaflow adds snowflake-snowpark-python to the Snowflake procedure environment, and Snowflake resolves that package's dependencies. The task editor does not currently provide a package selector, staged imports, or external secret injection. Use Run to verify any non-standard import before relying on it in a saved task.

Test Results and Logs​

The Result panel shows whether the test run completed, failed, timed out, or was cancelled. Use the Logs tab to view messages collected from Snowflake telemetry.

The Logs tab displays up to 200 entries. Snowflake collects telemetry asynchronously, so a successful task can finish before its logs become visible. During creation and editing, Supaflow waits briefly for those entries and reports when logs are still pending or unavailable.

What happens when you click Run:

  • The task is saved in draft state
  • The code executes against the selected Snowflake datasource
  • If successful, the task transitions to active state
  • If unsuccessful, the task remains in draft state

Note: If you leave the flow without clicking Next, the task remains saved in its current state. You can edit or delete it from the tasks list later.

How Python Tasks Run​

Each Python task run uses the database, schema, role, and authentication from the selected Snowflake datasource.

For each run, Supaflow:

  1. Creates a Python stored procedure containing your code in the datasource schema
  2. Creates a one-time Snowflake serverless task that calls the procedure
  3. Starts the task and waits for Snowflake to report a terminal state
  4. Collects available telemetry logs
  5. Removes the generated task and procedure after execution stops

The procedure runs with the task owner's Snowflake privileges. Grant the datasource role access only to the tables, views, schemas, and operations that the Python code needs.

If Snowflake cannot confirm that a cancelled or timed-out run has stopped, Supaflow leaves the generated objects in place instead of removing code that may still be executing. Contact support@supa-flow.io if generated objects remain after the run reaches a terminal state.

After the task becomes active, you can run it manually, add it to an orchestration, or assign it to a schedule.

Troubleshooting​

No Snowflake Datasources Found​

Problem: The Python task datasource menu is empty.

Solution: Python tasks currently support Snowflake only. Confirm that the Snowflake datasource is active and uses a connector version that supports Python task execution. If the datasource was created before this feature was released, test the connection again after updating the connector.

Python Task Permission Error​

Problem: Snowflake rejects procedure or task creation, task execution, or a query inside the handler.

Solution: Use the current setup script in Snowflake Destination. The datasource role needs permission to create procedures and tasks in the configured schema, the account-level privileges required to execute serverless tasks, and access to every object referenced by your code.

Python Handler Error​

Problem: The run reports that the handler is invalid or cannot be found.

Solution: The handler must be a simple Python function name, such as run or build_customer_summary. Confirm that the Handler field exactly matches a function defined in the editor and that the function accepts the Snowpark Session as its first argument.

Python Package Not Available​

Problem: The task fails while importing a library.

Solution: Supaflow declares snowflake-snowpark-python for the task environment. Snowflake may also resolve dependencies required by that package, but the task editor does not provide general package selection or staged imports. Test non-standard imports with Run, rewrite the task using the available environment, or use a SQL task when the operation can be expressed in SQL.

Python Logs Are Empty or Unavailable​

Problem: The task completes, but the Logs tab has no entries.

Solution: Confirm all of the following:

  • The code emits messages through Python's logging module
  • The Snowflake log level includes the severity you emit; logger.warning(...) is a useful first test
  • The Snowflake role has the SNOWFLAKE.EVENTS_VIEWER application role
  • The account uses Snowflake's default active event table

Supaflow currently reads SNOWFLAKE.TELEMETRY.EVENTS_VIEW. It does not read a custom active event table. Snowflake telemetry can also arrive after the task finishes, and the telemetry query can occasionally be unavailable or cancelled. In those cases, the Result tab remains the authoritative task outcome; unavailable logs do not mean the Python code failed. See Viewing log messages in the Snowflake documentation.

Python Task Timed Out​

Problem: The task reaches its configured timeout.

Solution: Increase the timeout in the task editor or reduce the work performed in one run. Supaflow requests cancellation when the timeout is reached and waits for Snowflake to confirm the terminal state.

Python Execution Fails​

Problem: The Result panel shows a failed status.

Solution: Review the error in the Result panel. Common causes include syntax errors, missing objects, insufficient permissions, unavailable packages, or timeouts. Test the same operation with the datasource role in Snowflake when you need to isolate a permissions or object-access problem.

For task naming, states, manual runs, editing, and deletion, see Tasks.

Support​

Need help? Contact us at support@supa-flow.io

Last updated on by pungaa
AI Tools
Ask ChatGPTClaudeAsk Claude
On this page