Faster Python Test Suites on GitHub Actions

Python test suites slow down on GitHub Actions for two reasons that compound: the standard hosted Linux runner for private repositories carries 2 vCPUs, so pytest runs close to serially, and every job starts with a cold cache, so pip or uv rebuilds the environment from scratch. The fix is a larger WarpBuild runner plus a persistent dependency cache: point runs-on at a warp- label, cache the resolver directory and the virtualenv, and let pytest -n auto use every core.

Overview

A typical Python job spends its time in four places: dependency install, test collection, test execution, and service startup for database-backed tests. On GitHub's standard hosted Linux runner for private repositories, which carries 2 vCPUs and 8GB of RAM per GitHub's published specs (checked on 2026-08-13), pytest -n auto gets two workers. A CPU-bound suite queues behind the interpreter, and a memory-hungry fixture set swaps.

WarpBuild runners register against your organization under warp- labels, and the Linux runner images carry the same tooling as GitHub-hosted runners, so setup-python, tox, and pytest run unchanged. Runners are ephemeral virtual machines: freshly allocated for each job and destroyed when the job completes. The full label list, sizes, and rates are in the cloud runners documentation.

If you want the broader tour of what makes any workflow slow, the checklist for speeding up GitHub Actions covers the general case; this page stays on Python.

The rest of this page walks through a working pytest configuration, sizing guidance for parallel runs from 4 vCPUs to 16 vCPUs, the four bottlenecks that dominate Python suites, and the public evidence that the setup holds up.

Configuration

The workflow below runs pytest on warp-ubuntu-latest-x64-8x (8 vCPUs, 32GB RAM) with uv as the package manager. It caches the uv download cache and the project virtualenv through WarpBuilds/cache, keyed on the hash of uv.lock. On a warm run, restore replaces resolution and install almost entirely, and uv sync --frozen only reconciles the delta.

name: python-tests

on:
  push:
    branches: [main]
  pull_request:

jobs:
  pytest:
    runs-on: warp-ubuntu-latest-x64-8x
    steps:
      - uses: actions/checkout@v4

      - name: Install uv
        uses: astral-sh/setup-uv@v5
        with:
          python-version: "3.12"

      - name: Restore uv cache and virtualenv
        id: uv-cache
        uses: WarpBuilds/cache@v1
        with:
          path: |
            ~/.cache/uv
            .venv
          key: ${{ runner.os }}-uv-${{ hashFiles('uv.lock') }}
          restore-keys: |
            ${{ runner.os }}-uv-

      - name: Sync dependencies
        run: uv sync --frozen

      - name: Run tests
        run: uv run pytest -n auto --dist worksteal

WarpBuilds/cache is a drop-in replacement for actions/cache@v4, so the inputs you already know (path, key, restore-keys, fail-on-cache-miss) behave the same way. The cache is enabled by default on Linux runners, entries are scoped to key, version, and branch, and an entry expires 7 days after its last use. Details and the split restore/save variants are in the caching documentation. Because the cache has no fixed size ceiling, large virtualenvs with compiled scientific packages fit without eviction gymnastics; if you have fought the hosted cache quota before, the guide to the GitHub Actions cache size limit explains the difference in detail.

If you use pip, pipenv, or poetry instead of uv, WarpBuilds/setup-python is a drop-in replacement for actions/setup-python that routes toolchain and dependency caching through the WarpBuild cache automatically:

      - uses: WarpBuilds/setup-python@v6
        with:
          python-version: "3.12"
          cache: pip

The restore-keys fallback in the first workflow matters more for Python than for most ecosystems. When uv.lock changes, the partial match restores the previous environment and the sync step installs only the packages that changed, so a one-line dependency bump avoids a full reinstall.

Sizing

pytest -n auto with pytest-xdist starts one worker per vCPU, so the runner size sets your parallelism directly. These are the Linux sizes that matter for test suites, with rates as listed on the pricing page:

How to pick:

4x (4 workers). Suites under roughly ten single-threaded minutes, mostly unit tests, light fixtures. With 16GB of RAM you get about 4GB per worker, which is comfortable for plain pytest and tight for anything loading large models or dataframes per worker.

8x (8 workers). The default recommendation for suites already on pytest-xdist. 32GB leaves 4GB per worker at full parallelism, enough for a per-worker database connection pool and mid-sized fixtures. Most teams land here.

16x (16 workers). Large monorepo suites and integration matrices. Before paying for 16 vCPUs, time your collection phase: collection runs in a single process no matter how many workers you buy, and a suite that spends three minutes collecting will keep spending three minutes collecting. The Bottlenecks section covers the fix.

The ARM64 rows carry lower per-minute rates at every size. If your dependencies publish ARM64 wheels, which numpy, pandas, and most of the scientific stack now do, the same suite runs on warp-ubuntu-latest-arm64-8x at $0.012 per minute. The Linux ARM64 runner page covers wheel availability and the differences worth knowing before switching architectures.

To verify a sizing choice instead of guessing, WarpBuild's CI observability reports CPU and memory metrics from the runner agent correlated with your job logs, so you can see whether all eight workers saturate the machine or whether four of them wait on a database.

A worked cost model

Assume a suite that takes 18 minutes on GitHub's standard 2 vCPU hosted runner and runs 400 times a month. Assume your own measurement shows pytest -n auto on 8 vCPUs brings wall clock to 6 minutes; substitute your real numbers, since parallel efficiency depends on the suite.

GitHub's per-minute list prices for x64 Linux are $0.006 for the standard 2 vCPU runner on private repositories, $0.012 for the 4-core larger runner, $0.022 for the 8-core larger runner, and $0.042 for the 16-core larger runner, per the GitHub Actions billing reference, checked on 2026-08-13. WarpBuild rates come from the table above.

Two things fall out of the arithmetic. Buying more vCPUs from GitHub raises the monthly bill even when the suite finishes sooner, because the rate climbs faster than the wall clock drops. And the equal-shape rows are cheaper on WarpBuild: warp-ubuntu-latest-x64-8x (8 vCPU, 32 GB) costs $0.016 per minute against $0.022 per minute for the 8-core Linux larger runner (8 vCPU, 32 GB): 27 percent lower list price. GitHub list price checked on 2026-08-13. On ARM64, warp-ubuntu-latest-arm64-8x (8 vCPU, 32 GB) costs $0.012 per minute against $0.014 per minute for the 8-core Linux ARM64 larger runner (8 vCPU, 32 GB): 14 percent lower list price. GitHub list price checked on 2026-08-13.

Every rate above is on the pricing page.

Bottlenecks

Four bottlenecks account for most slow Python suites on GitHub Actions. Each has a specific fix.

Wheel builds for native extensions

When pip or uv finds no wheel matching your platform, Python version, and architecture, it falls back to the sdist and compiles the extension in the job. This is common with internal packages, niche C extensions, and brand-new Python releases where the ecosystem has not caught up. The symptoms are install steps that swing from seconds to many minutes depending on the package set.

Three mitigations, in order of value: cache the build products, since both ~/.cache/pip and ~/.cache/uv keep built wheels and the workflow above restores them; pin Python versions with wide wheel coverage rather than adopting a new minor release in the first weeks; and prebuild internal packages into wheels published to your index so jobs never see an sdist. When a compile is unavoidable, more vCPUs help directly, because extension builds parallelize across cores.

Cold dependency resolution

Without a restored cache, every job pays for resolution and download of the full dependency set. Lockfiles make the cache exact: key on hashFiles('uv.lock') or your poetry.lock or hashed requirements.txt, and the cache hits precisely when the environment is unchanged. The carrying cost is small and visible: cache storage is $0.20 per GB-month and cache operations are $0.0001 each, so keeping a 500MB virtualenv cache warm costs about $0.10 a month in storage, and entries expire 7 days after last use anyway.

Service containers for database-backed tests

Suites backed by Postgres or Redis pay a startup tax and a contention tax. The startup tax is bounded with health checks, so the job proceeds the moment the service is ready rather than after a fixed sleep:

jobs:
  integration:
    runs-on: warp-ubuntu-latest-x64-8x
    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_PASSWORD: test
        ports:
          - 5432:5432
        options: >-
          --health-cmd "pg_isready -U postgres"
          --health-interval 5s
          --health-timeout 5s
          --health-retries 10

The contention tax appears when xdist workers share one database and deadlock on fixtures. Give each worker its own database, named from the PYTEST_XDIST_WORKER environment variable, and create them from a template database so per-worker setup stays cheap. This is also where RAM sizing earns its keep: eight workers each holding a connection pool and a fixture set is the workload the 32GB on an 8x runner exists for.

Serialized test collection

Before xdist distributes anything, pytest collects the whole suite in a single process, importing every test module along the way. Heavy imports at module scope, such as a framework app object or a large settings graph, turn collection into a fixed serial cost that no runner size reduces. Measure it with time pytest --collect-only -q. If it is more than a few percent of the run, restrict testpaths in your pytest configuration, move expensive imports inside fixtures or functions, and avoid importing the application at collection time. This is the one phase where a bigger machine buys nothing, so fix it in the code.

When a failure only reproduces in GitHub Actions, the WarpBuild Action Debugger pauses the workflow and opens an SSH session on the live runner, which beats adding print statements and re-running a 15 minute suite.

Proof

Public repositories are checkable evidence. LanceDB runs the Python macOS 3.14 ARM job of its open source lance repository on the warp-macos-14-arm64-6x label; the job builds the macOS Python wheel and runs the test suite, and you can read the configuration directly in the python.yml workflow file (checked on 2026-08-13). The same label carries the wheel build in lancedb/lancedb's pypi-publish.yml, the workflow that produces the published PyPI artifact. The label appears in the runner catalog on the cloud runners page.

Teams running JavaScript alongside Python can apply the same runner and cache approach; the Node.js on GitHub Actions page covers that ecosystem's equivalents.

FAQ

Do I need to change my pytest setup to run on WarpBuild runners?

No. Change the runs-on label in your workflow to a warp- label such as warp-ubuntu-latest-x64-8x. Steps, actions, and pytest configuration stay the same.

How many pytest-xdist workers should I run?

Start with -n auto, which launches one worker per vCPU. That means 4 workers on a 4x runner, 8 on an 8x, and 16 on a 16x. Lower the count if workers contend for a shared database or run out of RAM, and raise the runner size when every worker sits at full CPU.

Does the WarpBuild cache work with uv and poetry as well as pip?

Yes. WarpBuilds/cache stores any directory, so ~/.cache/uv, ~/.cache/pip, ~/.cache/pypoetry, and .venv all cache the same way. WarpBuilds/setup-python handles pip, pipenv, and poetry caching without extra configuration.

What does it cost to run Python tests on WarpBuild?

Linux x64 runners start at $0.004 per minute and Linux ARM64 runners at $0.003 per minute, billed per minute of runtime.