CI map · public data
pandas-dev/pandas
Today the median run takes 66m 28s; resharded on Boat the plan estimates 58m 56s, 11% less wall clock. Estimated, not measured. Measured on boat: 77,286 tests (a subset of the suite) ran in 86.6 s on 1 machine and 17.2 s on 8 machines (runs below).
- Future configs legacy values44m 08s
- Future configs upcoming values42m 12s
- ubuntu-24.04 py31237m 42s
- windows-2025 py31236m 47s
- ubuntu-24.04 py31135m 03s
- windows-2025 py31134m 48s
- macos-15-intel py31365m 12s
- macos-15-intel py31258m 21s
- macos-15-intel py31146m 46s
- macos-15 py31136m 26s
- macos-15 py31235m 38s
- ubuntu-24.04-arm py31234m 17s
Show all 40 jobs
- Minimum Versions25m 49s
- Clang (ASAN/UBSAN)20m 22s
- Linux-Musl10m 12s
- Future configs legacy values44m 08s
- Future configs upcoming values42m 12s
- ubuntu-24.04 py31237m 42s
- windows-2025 py31236m 47s
- ubuntu-24.04 py31135m 03s
- windows-2025 py31134m 48s
- Locale: it_IT30m 58s
- ubuntu-24.04 py31430m 14s
- ubuntu-24.04 py31329m 53s
- Pyarrow Nightly26m 58s
- Clang (ASAN/LSAN)24m 47s
- windows-2025 py31420m 28s
- Numpy Nightly16m 44s
- python-dev (windows-2025)16m 24s
- Linux-32-bit15m 42s
- Without PyArrow15m 37s
- Pyodide build11m 14s
- python-dev (ubuntu-24.04)10m 47s
- Freethreading9m 43s
- Windows-MinGW7m 51s
- Downstream Compat6m 46s
- macos-15-intel py31365m 12s
- macos-15-intel py31258m 21s
- macos-15-intel py31146m 46s
- macos-15 py31136m 26s
- macos-15 py31235m 38s
- windows-2025 py31334m 16s
- macos-15 py31331m 21s
- Locale: zh_CN21m 11s
- python-dev (macos-15-intel)19m 37s
- macos-15 py31419m 19s
- python-dev (macos-15)13m 54s
- ubuntu-24.04-arm py31234m 17s
- ubuntu-24.04-arm py31330m 51s
- ubuntu-24.04-arm py31130m 46s
- macos-15-intel py31428m 01s
- ubuntu-24.04-arm py31420m 17s
- Future configs legacy values34m 40s
- Future configs upcoming values33m 35s
- ubuntu-24.04 py31230m 04s
- windows-2025 py31233m 13s
- windows-2025 py31131m 25s
- macos-15-intel py31358m 52s
- macos-15-intel py31252m 41s
- macos-15-intel py31142m 14s
- macos-15 py31132m 54s
- macos-15 py31232m 11s
- windows-2025 py31330m 57s
- macos-15 py31328m 19s
Show all 40 jobs
- Minimum Versions18m 41s
- Clang (ASAN/UBSAN)14m 50s
- Linux-Musl5m 25s
- Future configs legacy values34m 40s
- Future configs upcoming values33m 35s
- ubuntu-24.04 py31230m 04s
- windows-2025 py31233m 13s
- ubuntu-24.04 py31127m 44s
- windows-2025 py31131m 25s
- Locale: it_IT22m 53s
- ubuntu-24.04 py31422m 21s
- ubuntu-24.04 py31323m 28s
- Pyarrow Nightly19m 11s
- Clang (ASAN/LSAN)16m 42s
- windows-2025 py31418m 29s
- Numpy Nightly11m 42s
- python-dev (windows-2025)14m 49s
- Linux-32-bit6m 55s
- Without PyArrow10m 57s
- Pyodide build7m 46s
- python-dev (ubuntu-24.04)6m 27s
- Freethreading5m 41s
- Windows-MinGW7m 05s
- Downstream Compat1m 26s
- macos-15-intel py31358m 52s
- macos-15-intel py31252m 41s
- macos-15-intel py31142m 14s
- macos-15 py31132m 54s
- macos-15 py31232m 11s
- windows-2025 py31330m 57s
- macos-15 py31328m 19s
- Locale: zh_CN14m 11s
- python-dev (macos-15-intel)17m 43s
- macos-15 py31417m 27s
- python-dev (macos-15)12m 33s
- ubuntu-24.04-arm py31227m 17s
- ubuntu-24.04-arm py31324m 20s
- ubuntu-24.04-arm py31124m 14s
- macos-15-intel py31425m 18s
- ubuntu-24.04-arm py31414m 57s
Times: medians of 184 sampled runs; bars: one real pull request run (#44562983054) from public GitHub Actions data; recommended plan estimated. Analyzed 2026-11-20.
Measured runs
On boat, 432 pandas test files (77,286 tests, a subset) ran in 86.6 s on 1 machine and 17.2 s on 8 forked machines. 8 machines ran 259,601 tests in 32.7 s.
Every run, failures, GitHub Actions and limits
Real runs of pandas-dev/pandas, one run each. Compare within one group only. Bars share one scale.
On boat · each run alone · measured · 4 test directories: frame, series, indexes, groupby (432 files), a subset
- 1× default1m 27s77,286 tests, 1 failed
- 8× default17s77,286 tests, 1 failed, shards balanced by test file size in bytes
On boat · each run alone · measured · whole pandas/tests tree except io/pytables (966 files), a subset
- 8× default1m 24s259,603 tests, 26 failed, shards balanced by measured seconds per test file from an earlier run
- 8× default33s259,601 tests, 26 failed, shards balanced by measured seconds per test file from an earlier run
On boat · each run alone · measured · whole pandas/tests tree except io/pytables (966 files) · killed before finishing
- 1× default2m 20skilled, no result
- 1× default3m 24skilled, no result
GitHub Actions · p50 from history · side by side, not like for like
- ubuntu-24.04 py312 (Unit Tests workflow)31m 33s
Not like for like: the GitHub job installs every optional dependency, runs the whole test tree and uploads coverage; the boat runs install the required dependencies plus pyarrow, numexpr and bottleneck, so the two are listed side by side, not as a like-for-like speedup.
Failed tests: 1 in every run of 4 test directories: frame, series, indexes, groupby (432 files); 26 in every run of whole pandas/tests tree except io/pytables (966 files).
- every number is one run, not a median; boat 1 machine and boat N machines use the same warm snapshot, the same machine size (default: 4 vCPU, 8 GB) and the same pytest command
- the 1-machine against 8-machine comparison covers a subset: 432 test files in 4 directories (77,286 tests); it says nothing about the whole CI run
- the whole tree on one default machine was killed twice before finishing (exit code 137, after 138 s and 204 s), so the whole-tree runs have no 1-machine baseline and no speedup figure
- no run is free of failures: 1 test fails in the 4-directory scope on 1 machine and on 8 machines alike (test_tz_localize_overflow_tzlocal), and 26 tests fail in the whole-tree scope in both 8-machine runs with different shard layouts (time zone and pyarrow conversion tests); the failures follow the environment, not the sharding, and were not investigated further
- tests that need optional dependencies not installed here are skipped (13,391 skipped of 259,601 in the whole-tree scope); pandas/tests/io/pytables is left out because its conftest needs PyTables
- markers follow the project's CI default: not slow, not network, not db, not single_cpu
- wallClockSeconds is measured from dispatch to the last machine finishing and excludes forkSeconds, which is listed next to it; machineSeconds sums the test seconds of every machine and costUsd prices it at the boat default rate ($0.036 per machine hour)
- the first 8-machine run of each scope was balanced by file size or by partial timings; the 32.7 s whole-tree run was balanced by seconds per test file measured in the 84.2 s run before it
More detail
What to change
Recommended: 30 machines, $0.378 per run estimated The recommended plan keeps the 9 jobs of the workflow, starts each from a warm snapshot and splits the test work of the slowest job over more machines: 30 machines against 30 today.
- Fork from a warm snapshot instead of repeating setupestimated
Each job on the critical path spends its first steps on checkout, tool setup and installs. A boat.dev sandbox forks from a snapshot that already holds that state, so the job starts at the first test step.
Saves 6m 18s per run - Balance shards by measured test timeestimated
A sharded job finishes when its slowest shard does. Splitting tests by recorded duration brings the slowest shard down to the mean shard time.
Saves 7m 57s per run - Run the critical-path tests on 12 machinesestimated
Sandboxes are billed per second, so the same test work spread over more machines costs about the same and finishes sooner. Only per-shard setup stays constant.
Saves 38m 15s per run - Cache dependencies between runsestimated
The workflow reinstalls dependencies on every job. Restoring them from a cache removes part of the setup time of every job.
82.8 runner hours a day - Retry failed tests in parallel inside the same runestimated
A rerun repeats the whole workflow after someone notices the failure. Retrying only the failed tests on a fresh fork while the run is still going removes that second cycle.
Saves 60 s per run - Test merges speculatively before they landestimated
A red default branch costs at least one more full CI cycle to fix or revert. Testing each merge result on a fork before it lands keeps those failures off the default branch.
Saves 1m 42s per run
How this CI evolved
Daily wall time of successful runs, p50 and p95, 2026-11-16 to 2026-11-20.
p50 p95
What's next: the same climb, automatic
pandas-dev/pandas can climb the same hill. Boat's agent will run that loop for you: try a plan, measure it, keep what is faster, try again. This is not live yet; when measured rounds exist, they will appear on this page.