Skip to content

Framework for easily and relatively quickly creating optimized and fully-featured CPython Python builds, supporting the latest Python versions (3.13, 3.14+) / backports across Debian and Debian-based Linux operating systems including Ubuntu and Raspberry Pi OS.

Notifications You must be signed in to change notification settings

ziesemer/python-builder

Repository files navigation

Python Builder

Summary

Framework for easily and relatively quickly creating optimized and fully-featured CPython Python builds, supporting the latest Python versions (3.13, 3.14+) / backports across Debian and Debian-based Linux operating systems including Ubuntu and Raspberry Pi OS.

This is a Docker-based framework for creating builds. I do not have plans to provide binary output builds at this time.

OS-provided support benefits are utilized where possible, but this project aims to remain as much of a standard / vanilla build of the original Python source as possible. This includes no unexplained patching of the source tree, etc., and no patching at all of the source tree at this time.

Support for free-threaded Python (no-gil / python3t) is optionally provided as a secondary installable package with only the additional needed binaries that appends upon the primary.

Packages are configured to install to /opt/python/python${PYTHON_VER_MAJ_MIN}/, e.g. /opt/python/python3.14/.

While I've built some of my own Python builds from source since at least Python 3.7 (2018), this is the first that I've tried to formalize this process with these efforts here. As such, this project and any builds produced from it should be considered experimental.

Feedback

Any feedback or recommendations to further improve this process are welcome - though especially from anyone involved in upstream packaging (Python, Debian, or Ubuntu maintainers).

Aspects specific to Docker and debuild are probably the most likely areas that could benefit from further efforts.

I'm especially hoping for any improvements that allow for further simplification of this process, and am not looking to grow into some of the additional complexities I've observed across most of the referenced efforts I included below.

Features

Feature Here Python
Source
Default
Ubuntu 25.10
Questing Quokka
Python 3.13
Debian 13
Trixie
Python 3.13
--enable-optimizations
(Profile Guided Optimizations / PGO)
--with-lto
(Link Time Optimization / LTO)
--with-system-expat)
(Expat XML parser)
--with-system-libmpdec
--with-dtrace
--enable-shared
  • Ubuntu 25.10 (Questing Quokka) Python 3.14 (universe) is configured the same as Python 3.13 (main) in respect to the above options.
  1. --with-system-libmpdec is the Python default since 3.13.
  2. DTrace: See also, Instrumenting CPython with DTrace and SystemTap.
  3. Shared: May revisit, though I haven't seen a need to override this yet. (libpythonMAJOR.MINOR.a and python.o are still built and provided.)

The configuration information details (installation paths, configuration variables, etc.) for any Python 3 build can be obtained by running sysconfig:

python3 -m sysconfig

CONFIG_ARGS is likely the most significant for at least any initial comparative reference.

Dynamically Shared Linked Libraries

I had spent considerable time looking at the different configurations surrounding the use of dynamically-shared linked libraries / dynamic depdendencies (*.so files). Specifically, seeing what ldd reported for each python3 binary.

  • It is important to review not only each python3 binary, but also each library .so file under lib\python3.*\lib-dynload\ - as each of these then link to further dependencies.
  • Adding a Modules/Setup.local (next to Modules/Setup) can be used to modify various module build options, force them to be built as static vs. shared, or completely disable them, etc.

The build options used here do not make use of any of these configuration options, but uses the defaults provided by CPython.

Build Arguments

  • OS_DISTRO_BASE:
    • For example, debian:trixie-slim.
    • Used to direct which Docker base image to use for the build environment.
  • OS_DISTRO_DISP:
    • For example, Debian Trixie.
    • Used only as the display name to include in the built package description.
  • PYTHON_VER_MAJ_MIN:
    • For example, 3.14.
    • Used to direct which Python version to build.
  • PYTHON_VER_MIC:
    • For example, 2 - as in 3.14.2.
    • Used to direct which micro / patch version of the above Python version to build.
  • FAST_BUILD. Defaults to unset. If set to true:
    • Updates OPTIMIZATION_FLAGS:
      • Removes --enable-optimizations.
      • Removes --with-lto.
      • Adds --disable-test-modules.
    • Skips dh_auto_test.
  • BUILD_NOGIL. Defaults to true. If set to false:
    • Skips running separate configure, build, and install steps for a free-threaded Python version.

Sample Build Times

build args Type Laptop Time Raspberry Pi 5 Time
FAST_BUILD=true
BUILD_NOGIL=false
--no-cache 2m:52s 5m:33s
FAST_BUILD=true
BUILD_NOGIL=false
PYTHON_VER_MIC=1
Build another micro version, following the above. 2m:04s 3m:47s
(empty) Default, full, optimized build, following the above. 24m:17s 57m:16s
PYTHON_VER_MIC=1 Default, full, optimized build of another micro version, following the above. 21:04s 56m:55s

These times:

  • Are provided only as a rough approximation of build time requirements. They are unaveraged, taken from a single run, and measured without experimental controls or adherence to scientific methodology, and should not be treated as precise or reproducible benchmarks.
  • Were taken on 2025-12-27.
  • Assume that the needed Docker base images are already pulled. Metrics started after running docker builder prune -a.

The Raspberry Pi 5 is:

  • A model B 16 GB Rev 1.1 e04171.
  • Using a Samsung 512 GB Pro Ultimate C10/V30/U3/A2 microSDXC memory card (MB-MY512SA/AM).
  • Running Raspberry Pi OS Debian Trixie (13) 64-bit.

References

Referenced Efforts

This effort is inspired by a combination of:

Build References

Python Versions

Author

Mark Ziesemer, CISSP, CCSP, CSSLP

About

Framework for easily and relatively quickly creating optimized and fully-featured CPython Python builds, supporting the latest Python versions (3.13, 3.14+) / backports across Debian and Debian-based Linux operating systems including Ubuntu and Raspberry Pi OS.

Topics

Resources

Stars

Watchers

Forks