Benchmarks are tests to measure the performance of pandas. There are two different kinds of benchmarks relevant to pandas:

pandas benchmarks

pandas benchmarks are implemented in the asv_bench directory of our repository. The benchmarks are implemented for the airspeed velocity (asv for short) framework.

The benchmarks can be run locally by any pandas developer. This can be done with the asv run command, and it can be useful to detect if local changes have an impact in performance, by running the benchmarks before and after the changes. More information on running the performance test suite is found here.

Note that benchmarks are not deterministic, and running in different hardware or running in the same hardware with different levels of stress have a big impact in the result. Even running the benchmarks with identical hardware and almost identical conditions produces significant differences when running the same exact code.

pandas benchmarks servers

We currently have two physical servers running the benchmarks of pandas for every (or almost every) commit to the main branch. The servers run independently from each other. The original server has been running for a long time, and it is physically located with one of the pandas maintainers. The newer server is in a datacenter kindly sponsored by OVHCloud. More information about pandas sponsors, and how your company can support the development of pandas is available at the pandas sponsors page.

Results of the benchmarks are available at:

Original server configuration

The machine can be configured with the Ansible playbook in tomaugspurger/asv-runner. The results are published to another GitHub repository, tomaugspurger/asv-collection.

The benchmarks are scheduled by Airflow. It has a dashboard for viewing and debugging the results. You’ll need to setup an SSH tunnel to view them:

ssh -L 8080:localhost:8080

OVH server configuration

The server used to run the benchmarks has been configured to reduce system noise and maximize the stability of the benchmarks times.

The details on how the server is configured can be found in the pandas-benchmarks repository. There is a quick summary here:

Community benchmarks

The main benchmarks comparing dataframe tools that include pandas are: