Finding the Slow Step in a GitHub Actions Workflow

The slow step in a GitHub Actions job is the one with the largest gap between its started_at and completed_at timestamps, and those timestamps are available for every step of every run through the GitHub REST API. Before you rank them, split the job's wall clock into queue wait and execution time, because a pipeline that grew in the queue has no slow step to find.

This guide covers the order of operations: separate queue time from execution time, attribute execution time to individual steps, then use runner utilization to name the resource holding the slow step. The last section prices the change so you know what a size bump or a cache fix does to the bill before you merge it.

Diagnosis

Three numbers answer three different questions, and each one lives in a different place. Reading them out of order is how teams end up buying a bigger runner for a job that was waiting in a queue.

Start with Queue Timings. It aggregates per runner label and stack, with a daily bar chart of average queue time alongside the job count for that day. If end to end pipeline time grew while duration p90 in the Jobs report stayed flat, the growth is queue wait and no step changed.

Move to the Jobs report second. It aggregates per unique repository, workflow, and job name combination and reports run count, success rate, and p75 and p90 for duration, queue time, CPU, and memory. Sort by duration p90 descending and take the top job. Percentiles matter here because averages hide the run shape you actually care about: a job with a 4 minute average and a 19 minute p90 has a step that is sometimes slow, which is a different investigation from a step that is always slow.

Only then attribute execution time to steps. A single representative run gives a timeline like this one, where the job executed for 15 minutes:

Shares are rounded to whole percent. Two steps own 62 percent of the run, and they have nothing else in common, so they need separate diagnoses.

That is where utilization comes in. WarpBuild runners carry an agent that records CPU, memory, filesystem, disk I/O, and network utilization per instance over OpenTelemetry, and the Recommendations view filters and highlights the instances that cross fixed thresholds.

Those labels highlight instances in the UI so you can filter to them. They do not send notifications, so the review is something you schedule rather than something that pages you. Network utilization is recorded per instance as well and carries no threshold, which is why a download bound step is identified by reading the chart against the step boundary rather than by waiting for a label to appear.

Two collection limits are worth knowing before you go hunting. Metrics and logs are collected for jobs longer than about one minute, so a 40 second step inside a 40 second job produces no chart. Collection can also be paused, which stops system logs and GitHub Actions logs along with the metrics.

Fix

The sequence below turns one slow step into one change.

Step 1: confirm the time is in execution. Compare queue time p90 from Queue Timings against duration p90 from the Jobs report for the same window. If queue wait moved, stop here and work the capacity path instead.

Step 2: pick the job by duration p90, then pick the run. Take the top job by duration p90 and pull a recent run whose duration sits near that percentile. Profiling the fastest run of the week teaches you nothing about the p90.

Step 3: rank the steps. Subtract started_at from completed_at for each step of that run and sort descending. The two heaviest steps are the whole investigation.

Step 4: line the metrics up with the step boundary. Open the Usage view for that runner instance. The metrics chart and the GitHub Actions logs sit in the same view, so a plateau in CPU or disk I/O can be matched to the step that was running when it started. A step at 85 percent memory for 20 seconds is a different problem from a step that holds 85 percent for 9 minutes.

Step 5: choose the lever the metric points at. Sustained CPU at or above 80 percent for the duration of the step means the step is core bound and runner size is the lever. Memory at or above 80 percent means the size change should be about RAM, and it usually explains test workers that die without a stack trace. Filesystem at or above 80 percent means the working set outgrew the volume. Disk I/O at or above 80 percent of supported throughput while CPU sits low means the step is moving bytes, and the cache path is the lever.

Step 6: change one thing and re-measure. Run count in the Jobs report tells you how long to wait. A job with 40 runs a week needs a full week before its new p90 means anything.

Here is the worked case for the timeline above. The 3:41 restore step ran at 22 percent CPU and 31 percent memory with disk I/O at 84 percent of supported throughput, so cores would not have moved it. Switching that step to the WarpBuild cache action and the cache enabled setup-node fork brought it to 0:52. The 5:36 build step ran at 94 percent sustained CPU on a 4 vCPU runner, which is the size signal, so the job moved to warp-ubuntu-latest-x64-8x and the step came back at 3:20.

Execution time for the job goes from 15:00 to 9:55. The cache change alone accounts for 2:49 of that and the size change for 2:16. The next section prices both.

Every runner WarpBuild operates carries the same agent, and BYOC runs on AWS, GCP, and Azure, so metrics for a runner inside your own cloud account land in the same reports as a hosted one. The step attribution method is identical on all four.

Configuration

Step durations come straight from the GitHub REST API. This lists every step of every job in a run with its duration in seconds, sorted longest first:

gh api "repos/OWNER/REPO/actions/runs/$RUN_ID/jobs" --paginate \
  --jq '.jobs[] | .name as $job | .steps[]
    | select(.started_at != null and .completed_at != null)
    | [((.completed_at | fromdateiso8601) - (.started_at | fromdateiso8601)), $job, .name]
    | @tsv' \
| sort -rn | head -20

Pick RUN_ID from a run whose duration sits near the job's p90 rather than a random recent run. The Jobs report API exposes that percentile, so the ranking can be scripted with an API key carrying the ci scope:

curl -sS -G 'https://api.warpbuild.com/api/v1/reports/jobs' \
  -H 'Authorization: Bearer wkey-xxxx' \
  --data-urlencode 'start_date=2026-07-13T00:00:00Z' \
  --data-urlencode 'end_date=2026-08-13T00:00:00Z' \
  --data-urlencode 'sort_by=duration_p90' \
  --data-urlencode 'sort_order=desc' \
| jq -r '.table.items[]
    | [.repo, .workflow_name, .job_name, .run_count,
       .duration_p90, .queue_time_p90, .cpu_p90, .memory_p90]
    | @tsv'

Reading duration_p90 and queue_time_p90 side by side in that output is the step 1 check in a single command. Add --data-urlencode 'format=csv' when you want the rows in a spreadsheet or a ticket.

The workflow below is the state after both fixes. The build job carries the larger label, the restore step uses the WarpBuild cache path, and the test job stays on the smaller size because its metrics never crossed a threshold:

name: ci
on:
  pull_request:

jobs:
  build:
    runs-on: warp-ubuntu-latest-x64-8x
    steps:
      - uses: actions/checkout@v4
      - uses: WarpBuilds/setup-node@v6
        with:
          node-version: 22
          cache: npm
      - run: npm ci
      - run: npm run build

  test:
    needs: build
    runs-on: warp-ubuntu-latest-x64-4x
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3, 4]
    steps:
      - uses: actions/checkout@v4
      - uses: WarpBuilds/setup-node@v6
        with:
          node-version: 22
          cache: npm
      - run: npm ci
      - run: npm test -- --shard=${{ matrix.shard }}/4

Splitting a slow test step across shards adds jobs rather than minutes per job. Run as many jobs as your workflows need. Generally available Linux and Windows runners do not have plan-level concurrency caps, so a 4 way shard becomes an 8 way shard without a queue penalty appearing somewhere else.

The telemetry agent uses port 33931. It is open by default on WarpBuild operated runners, and on BYOC runners you confirm egress on that port from the subnet the runners launch in, otherwise the Usage view stays empty while jobs run normally. Field definitions for every metric live in the Observability documentation, and the report tabs are documented in the Reports documentation.

Cost or Time Model

Runner size prices in a straight line, so a size change is arithmetic. Linux x64 rates from the WarpBuild pricing page, billed per minute:

Assumptions. The job from the diagnosis section runs 700 times a month. Execution time is the 15:00 measured above, which is 15.0 minutes. The post fix step timings of 0:52 and 3:20 are the observed replacements described in the fix section. Queue time is excluded because it is not billed, and billing is per minute of runner time.

Baseline. 700 runs at 15.0 minutes on warp-ubuntu-latest-x64-4x is 10,500 minutes at $0.008, which is $84.00 a month.

Change one, the cache path only. Execution drops to 12.2 minutes on the same 4 vCPU label. That is 8,540 minutes at $0.008, which is $68.32 a month, and it returns 1,960 minutes of wall clock a month to the people waiting on the pull request. This change lowers both numbers, which is why the cache lever goes first whenever disk I/O rather than CPU crossed the threshold.

Change two, the size bump on top. Execution drops to 9.92 minutes on warp-ubuntu-latest-x64-8x. That is 6,944 minutes at $0.016, which is $111.10 a month. Against the $84.00 baseline it costs $27.10 more and returns 3,556 minutes of wall clock a month. The dollar break even is the number to keep: at 700 runs, the 8 vCPU label matches the old bill when execution reaches 7.5 minutes per run, so this job is paying $27.10 for the 2.4 minutes it did not get there.

For a list price baseline, GitHub publishes these per minute prices for GitHub-hosted Linux runners, checked on 2026-08-13 in the GitHub billing reference:

The same 6,944 minutes on the 8-core GitHub-hosted larger runner costs $152.77 a month against $111.10 on warp-ubuntu-latest-x64-8x. That is the same 8 vCPU and 32 GB shape at $0.016 per minute against $0.022, which is 27 percent lower list price, checked on 2026-08-13.

For the percentile reading that picks the run to profile, see read GitHub Actions build duration percentiles. For the per instance metric definitions behind step 4, see runner level metrics for GitHub Actions jobs. For the estate wide view, start at GitHub Actions observability with WarpBuild, and for the broader set of levers once the slow step is named, see speed up GitHub Actions workflows.

FAQ

How do I find which step in a GitHub Actions job is slow?

Pull the job through the GitHub REST API and subtract each step's started_at from its completed_at. That gives every step's duration for one run, sorted. Rank jobs first by duration p90 in the Jobs report so you profile a run that is representative rather than the one you happened to open.

Why does the job total not match the sum of its step durations?

Queue wait sits outside the steps and is reported separately in Queue Timings. Inside the job, Set up job, Post steps, and Complete job carry real time that is easy to skip when reading the log. Billed time is also reported separately from execution time in the CI billing table, so reconcile against billed time when you compare with an invoice.

Can the runner tell me why a step is slow rather than only how long it took?

Yes. WarpBuild runners run an agent that records CPU, memory, filesystem, disk I/O, and network utilization per instance and shows it beside the GitHub Actions logs for the same job, so you can line a metric plateau up with a step boundary. Metrics and logs are collected for jobs longer than about one minute.

Should I move a slow step to a bigger runner or fix caching first?

Let the metric decide. Sustained CPU at or above 80 percent points at runner size. A slow restore or download step at low CPU and high disk I/O points at the cache path, which is usually cheaper to fix and lowers the bill instead of raising it.