Sharding pytest Suites Across Runners

A pytest suite runs parallel in GitHub Actions through two independent levers: pytest-xdist worker processes inside one job, and matrix shards that spread the suite across several jobs. Workers divide the suite across the vCPUs of one runner, shards divide it across runners, and a suite over roughly 20 minutes usually wants both set deliberately rather than one of them turned up until it stops helping.

This guide covers how to tell which lever is mis-set, the edits that fix each case, a workflow that runs four shards with four workers each and merges the results into one report, and a cost model that prices one 16 vCPU runner against four 4 vCPU runners. For the language-neutral form of the technique see the guide to matrix sharding and the test sharding definition, and for the rest of the run see the hub on speeding up GitHub Actions.

Diagnosis

The two levers fail in different ways, so the first job is deciding which one you are looking at.

Worker processes inside one job

pytest -n 8 starts eight worker processes on the machine running the job. They are separate processes, so the CPython global interpreter lock does not cap them, and the real ceilings are vCPU count, memory, disk, and any shared service the suite talks to. Each worker imports the test modules on startup, so collection cost is paid once per worker rather than once per job.

-n auto reads the CPU count of the machine, which means the same workflow line produces 2 workers on warp-ubuntu-latest-x64-2x and 16 on warp-ubuntu-latest-x64-16x. That is convenient until the suite is memory bound, at which point the larger label starts killing workers.

Distribution mode matters as much as the count. The default --dist load sends each test to whichever worker is free, which rebalances at run time and also spreads one module across several workers, so module and class scoped fixtures are set up more than once. --dist loadfile and --dist loadscope keep tests grouped by file or by class (pytest-xdist distribution modes, checked on 2026-08-13).

Shards across jobs

A shard is a separate job on a separate runner with its own checkout, its own dependency install, and its own slice of the suite decided before the run starts. That fixed split is where wall clock leaks. Splitting by test count gives four jobs of four different lengths, and the run waits for the longest one while the other three sit finished.

Duration, queue time, CPU, and memory land at P75 and P90 for every repository, workflow, and job name combination in the Jobs report, with CSV export (reports documentation); the CPU and memory columns need observability enabled. On GitHub-hosted runners, read peak memory yourself inside the job and subtract created_at from started_at per job for queue wait. A single job stops at 6 hours (GitHub Actions usage limits, checked on 2026-08-13), which is the hard reason a long suite has to shard rather than only add workers.

Fix

Work these in order. The first three change the worker lever, the last three change the shard lever.

Record durations before splitting anything. pytest --store-durations writes a .test_durations file that pytest-split reads to build balanced groups (pytest-split, checked on 2026-08-13). Commit the file and refresh it on a schedule, since a split built from last quarter's timings drifts as tests are added.

Set the worker count from the label, then check memory. Start at the vCPU count and read Memory P90 after a day of runs. Both layouts in the cost model below give each worker 4 GB: 16 workers across the 64 GB of warp-ubuntu-latest-x64-16x, or 4 workers across the 16 GB of warp-ubuntu-latest-x64-4x.

Pick the distribution mode from fixture scope. Suites built on module or class scoped fixtures belong on --dist loadfile or --dist loadscope. Suites of independent function scoped tests keep the default and get run-time rebalancing for free.

Give each worker its own state. xdist sets PYTEST_XDIST_WORKER to gw0, gw1, and so on inside each worker. Derive the database name, the temp directory, and any bound port from it in a session fixture, and the cross-worker failures in the table above stop.

Shard once one machine is saturated. The trigger is a machine ceiling rather than a preference: memory the largest label cannot hold, a suite approaching the 6 hour job limit, or a wall clock target below what the vCPU count of one runner can reach.

Merge the results in a dedicated job. Four shards produce four coverage files and four JUnit XML files, and branch protection wants one check. The merge job in the next section runs with if: always() so a failing shard still reports.

A Python suite with platform specific paths can keep the same shard and worker structure on a warp-macos-latest-arm64-6x or warp-windows-latest-x64-4x label and change only runs-on.

Configuration

Four shards, four workers per shard, durations-based splitting, and a merge job.

name: pytest
on:
  pull_request:
  push:
    branches: [main]

jobs:
  test:
    name: pytest ${{ matrix.shard }}/4
    runs-on: warp-ubuntu-latest-x64-4x
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3, 4]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.13"
          cache: pip
      - run: pip install -r requirements.txt -r requirements-dev.txt
      - name: Run shard
        env:
          COVERAGE_FILE: .coverage.${{ matrix.shard }}
        run: |
          pytest -n 4 --dist loadfile \
            --splits 4 --group ${{ matrix.shard }} \
            --durations-path .test_durations \
            --junitxml=junit-${{ matrix.shard }}.xml \
            --cov=src --cov-report=
      - uses: actions/upload-artifact@v4
        with:
          name: pytest-results-${{ matrix.shard }}
          path: |
            junit-${{ matrix.shard }}.xml
            .coverage.${{ matrix.shard }}
          include-hidden-files: true

  merge:
    needs: test
    if: always()
    runs-on: warp-ubuntu-latest-x64-2x
    steps:
      - uses: actions/setup-python@v5
        with:
          python-version: "3.13"
      - run: pip install coverage junitparser
      - uses: actions/download-artifact@v4
        with:
          pattern: pytest-results-*
          merge-multiple: true
      - run: |
          junitparser merge junit-*.xml junit.xml
          coverage combine
          coverage report --fail-under=85

What each piece does:

• --splits 4 --group N comes from pytest-split and partitions the suite by recorded time from .test_durations, so the four jobs aim at equal duration rather than equal test count.
• -n 4 matches the 4 vCPU of warp-ubuntu-latest-x64-4x, so the two levers multiply to 16 worker processes across the matrix.
• --dist loadfile keeps every test in a module on one worker, which is what makes module scoped fixtures cost one setup each.
• COVERAGE_FILE per shard avoids four jobs writing the same .coverage name. coverage combine looks for files prefixed .coverage. in the working directory and merges them (coverage combine, checked on 2026-08-13).
• Artifact names are unique per shard because actions/upload-artifact@v4 writes immutable artifacts and rejects a second upload under an existing name (upload-artifact, checked on 2026-08-13). include-hidden-files: true is required for the dotfile coverage data.
• merge-multiple: true on the download unpacks every matching artifact into one directory, which is what lets the two merge commands use globs.
• fail-fast: false keeps the other three shards running after one fails, so a single flaky test does not cost a full rerun of the grid.

Labels resolve to these machines, from the WarpBuild cloud runners documentation, checked on 2026-08-13. The last column is the per-minute rate divided by the vCPU count.

Cost or Time Model

Two formulas cover the pair of levers. Wall clock per run is s + c + T / (N * w), and billed minutes are N * (s + c + T / (N * w)), where T is serial suite execution time, N is the shard count, w is workers per shard, s is per-job setup, and c is the collection each worker pays before its first test.

Assumptions for the model: T of 64 minutes, s of 3 minutes for checkout, setup-python, and a warm pip cache, c of 0.5 minutes, an even durations-based split, no retries, and queue wait excluded. Replace all five with your own numbers from the Jobs report.

Rows one and two are the comparison the question usually turns on, and they price identically. The Linux x64 catalog above charges $0.002 per vCPU minute at every size, so 16 vCPU for 7.5 minutes costs $0.24 whether it arrives as one machine or four. The repeated setup in the four-job layout is paid on quarter-size machines, which is why it does not show up as a premium: 4 * 3 * $0.008 is the same $0.096 as 1 * 3 * $0.032.

The price separation appears when the worker total goes up rather than when it is rearranged. Going from 16 workers to 32 buys 2 minutes of wall clock and adds $0.11 per run; the next doubling to 64 workers buys 1 more minute and adds $0.23. Stop where per-shard test time approaches s + c, which is 32 workers for this suite.

Three things break the tie between rows one and two at equal price. An uneven split hurts the shard layout, because xdist rebalances at run time inside a job while a matrix split is fixed before the run: shards holding 24, 16, 14, and 10 minutes of work finish at 9.5, 7.5, 7, and 6 minutes, so the run takes 9.5 minutes against 7.5 for the single machine, at the same 30 billed minutes. Rerunning one failed shard costs a quarter of the grid, where rerunning the single job costs all of it. And a suite that needs more than 64 GB or more than 6 hours has no single-machine option left.

Those 30 billed minutes carry a list price. GitHub rates below are from the GitHub Actions billing reference, checked on 2026-08-13.

warp-ubuntu-latest-x64-4x (4 vCPU, 16 GB) costs $0.008 per minute against $0.012 per minute for the 4-core Linux larger runner (4 vCPU, 16 GB): 33 percent lower list price. warp-ubuntu-latest-x64-16x (16 vCPU, 64 GB) costs $0.032 per minute against $0.042 per minute for the 16-core Linux larger runner (16 vCPU, 64 GB): 24 percent lower list price. GitHub list prices checked on 2026-08-13.

Scale the four-shard layout to a working month at 40 runs a day: 1,200 billed minutes a day, $9.60 on warp-ubuntu-latest-x64-4x and $14.40 on the 4-core GitHub-hosted larger runner, or $211.20 against $316.80 over 22 working days.

Full rates by runner type are on the pricing page.

FAQ

Should I use pytest-xdist workers or matrix shards?

Use workers first, because they cost one setup and rebalance at run time. Add shards once the largest runner you are willing to rent is saturated, once the suite needs more memory than one machine holds, or once a single job approaches the 6 hour GitHub Actions job limit. Most suites end up with both: a shard matrix for the fan-out and a worker count inside each shard.

How many pytest-xdist workers should I run per runner?

Start at the vCPU count of the label, so 4 on warp-ubuntu-latest-x64-4x and 16 on warp-ubuntu-latest-x64-16x, then lower it if memory peaks near the ceiling. Each worker is a separate process holding its own imports and fixtures, so a suite that needs 4 GB per worker fits 4 workers in the 16 GB of a 4 vCPU runner and 16 workers in the 64 GB of a 16 vCPU runner.

How do I merge coverage and JUnit XML from pytest shards?

Write one file per shard, upload each under a unique artifact name, then merge in a dependent job that runs with if: always() so a failed shard still reports. Set COVERAGE_FILE to a per-shard path and run coverage combine over the downloaded files, and merge the per-shard JUnit XML into one report before the branch protection check reads it.