Skip to main content

Activities

Monitor pipeline runs, task executions, and orchestrations with detailed status tracking and object-level metrics.

Overview

Activities track every pipeline run, task execution, and orchestration with execution status, duration, row counts, and detailed object-level breakdowns.

Activity types:

  • Pipeline Run - Ingestion pipeline execution (clickable for details)
  • Activation Run - Activation pipeline execution (clickable for details)
  • Task Run - SQL task execution (list view only, not clickable)

Each pipeline run and activation run shows object-level details including ingestion mode (Historical or Incremental), incremental sync timestamps, and per-stage metrics (Ingest → Load).

To access: Navigate to Activities in the sidebar.


Activities List

The activities list shows all executions with search, filters, and auto-refresh.

List Columns

ColumnDescription
NameActivity name with icon (Pipeline Run, Activation Run, or Task Run)
Activity ID shown below in gray
StartedRelative time (e.g., "6 days ago")
DurationExecution time (e.g., "14s", "39m 59s")
ObjectsNumber of tables/objects synced (pipeline/activation runs only)
Shows "-" for task runs
RowsTotal rows processed (pipeline/activation runs only)
Shows "-" for task runs
StatusStatus badge: Completed, Failed, Timed out, Running, etc.

Search and Filters

Search Bar:

  • Search by activity name
  • Real-time filtering as you type

Type Filter:

  • Default shows 2 types: Pipeline Run and Task Run
  • Click to select/deselect: Pipeline Run, Activation Run, Task Run
  • Shows count of selected types (e.g., "Type (2)")

Status Filter:

  • Select multiple statuses: Completed, Failed, Timed out, Running, Queued, etc.
  • Click Apply to filter
  • Click Clear to reset

Refresh:

  • Manual refresh button
  • Shows "Last refreshed: less than a minute ago"
  • Auto-refresh: ON when activities are running, OFF when all completed

Clicking an Activity

Pipeline Run and Activation Run:

  • Click the row to open the activity detail page
  • Shows detailed object-level breakdowns, ingestion modes, and stage metrics

Task Run:

  • Not clickable - no detail page available
  • View summary information in the list only

Canceling a Running Activity

To cancel:

  1. Hover over the status badge on any running activity (shows "In progress", "Queued", or "Picked")
  2. Badge transforms to show "Cancel" with an X icon
  3. Click the "Cancel" button
  4. Confirm cancellation in the modal

What happens:

  • Activity stops executing
  • Status changes to "Cancelled"
  • Cannot be undone

Note: There may be a slight delay before execution fully stops.


Activity Detail Page

To open: Click any Pipeline Run or Activation Run in the activities list

The detail page shows comprehensive execution information with object-level breakdowns.


Elements:

  • Back arrow - Returns to activities list
  • Pipeline name - Clickable link to open pipeline in IDE
  • Status badge - Current status (Completed, Failed, Running, etc.)

Cancel button (running activities only):

  • Hover over the status badge (transforms to "Cancel")
  • Click to cancel execution
  • Confirmation required before stopping

Summary Cards

Three cards display aggregate metrics:

Details Card

  • Source → Destination with connector icons
  • Started at - When execution began
  • Duration - Total execution time

Objects Card

  • Successful - Objects completed
  • Queued - Objects waiting
  • Failed - Objects with errors
  • Running - Objects currently processing

Rows Card

  • Ingested - Total rows read from source
  • Loaded - Total rows written to destination
  • Failed - Total rows that failed
  • Max Latency - Highest latency observed

Objects List

Display:

  • Search bar to filter objects
  • Status filter dropdown
  • Refresh button with auto-refresh indicator
  • Table showing all synced objects

Table Columns:

ColumnDescription
Object NameTable/object name with ingestion mode label
Historical - Full sync
Incremental - Only changes synced
Hover to see incremental timestamp (e.g., "UPDATE_DATE: 2025-12-12T22:06:07.167535642")
DurationTime to sync this object
Max LatencyLatency for this object
# IngestedRows read during ingestion
# LoadedRows written to destination
# FailedRows that failed
StatusCompleted, Failed, Running, etc.

Expand Object Details:

Click the chevron (▸) to expand any object and see stage-level breakdown:

Stage: Ingest

  • Rows ingested from source
  • Duration and latency metrics

Stage: Load

  • Rows loaded to destination
  • Duration and latency metrics

Incremental Sync Positions

For objects synced incrementally, the ingestion mode shows "Incremental" label.

To view the cursor timestamp:

  • Hover over the object name row
  • Tooltip displays the incremental position (e.g., "UPDATE_DATE: 2025-12-12T22:06:07.167535642")

This timestamp shows where the last sync ended. The next incremental sync will start from this position.


Activity Statuses

StatusDescription
CompletedSuccessfully finished
FailedExecution encountered errors
Timed outExceeded maximum execution time
RunningCurrently executing (clickable to cancel)
QueuedWaiting to start

Troubleshooting

Activity Stuck in "Running"

Problem: Activity shows "Running" status for extended period.

Solution:

  • Hover over the status badge and click "Cancel" to stop the activity
  • Check pipeline configuration for issues
  • Verify source/destination connectivity
  • Retry the pipeline run

Activity Failed

Problem: Activity shows "Failed" status.

Solution:

  1. Click the activity to view detail page
  2. Look for failed objects in the objects list
  3. Expand failed objects to see stage-level errors
  4. Common causes:
    • Expired credentials
    • Insufficient permissions
    • Schema mismatches
    • Connection timeouts
  5. Fix the underlying issue and retry

No Objects or Rows Shown

Problem: Objects and Rows columns show "-".

Solution:

  • This is normal for Task Run activities (only pipeline/activation runs have object-level metrics)
  • Ensure Type filter includes "Pipeline Run" or "Activation Run"
  • If activity is still running, metrics populate when complete

Incremental Sync Not Working

Problem: Every run syncs full data instead of only changes.

Solution:

  • Click activity to view detail page
  • Check if objects show "Historical" instead of "Incremental"
  • Verify pipeline has incremental cursor field configured
  • Check source supports incremental sync


Support

Need help? Contact us at support@supa-flow.io