Python Packaging
Comprehensive guide to creating, structuring, and distributing Python packages using modern packaging tools, pyproject.toml, and publishing to PyPI.
When to Use This Skill
- Creating Python libraries for distribution
- Building command-line tools with entry points
- Publishing packages to PyPI or private repositories
- Setting up Python project structure
- Creating installable packages with dependencies
- Building wheels and source distributions
- Versioning and releasing Python packages
- Creating namespace packages
- Implementing package metadata and classifiers
Core Concepts
1. Package Structure
- Source layout:
src/package_name/(recommended) - Flat layout:
package_name/(simpler but less flexible) - Package metadata: pyproject.toml, setup.py, or setup.cfg
- Distribution formats: wheel (.whl) and source distribution (.tar.gz)
2. Modern Packaging Standards
- PEP 517/518: Build system requirements
- PEP 621: Metadata in pyproject.toml
- PEP 660: Editable installs
- pyproject.toml: Single source of configuration
3. Build Backends
- setuptools: Traditional, widely used
- hatchling: Modern, opinionated
- flit: Lightweight, for pure Python
- poetry: Dependency management + packaging
4. Distribution
- PyPI: Python Package Index (public)
- TestPyPI: Testing before production
- Private repositories: JFrog, AWS CodeArtifact, etc.
Quick Start
Minimal Package Structure
my-package/
βββ pyproject.toml
βββ README.md
βββ LICENSE
βββ src/
β βββ my_package/
β βββ __init__.py
β βββ module.py
βββ tests/
βββ test_module.pyMinimal pyproject.toml
[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"
[project]
name = "my-package"
version = "0.1.0"
description = "A short description"
authors = [{name = "Your Name", email = "you@example.com"}]
readme = "README.md"
requires-python = ">=3.8"
dependencies = [
"requests>=2.28.0",
]
[project.optional-dependencies]
dev = [
"pytest>=7.0",
"black>=22.0",
]Package Structure Patterns
Pattern 1: Source Layout (Recommended)
my-package/
βββ pyproject.toml
βββ README.md
βββ LICENSE
βββ .gitignore
βββ src/
β βββ my_package/
β βββ __init__.py
β βββ core.py
β βββ utils.py
β βββ py.typed # For type hints
βββ tests/
β βββ __init__.py
β βββ test_core.py
β βββ test_utils.py
βββ docs/
βββ index.mdAdvantages:
- Prevents accidentally importing from source
- Cleaner test imports
- Better isolation
pyproject.toml for source layout:
[tool.setuptools.packages.find]
where = ["src"]Pattern 2: Flat Layout
my-package/
βββ pyproject.toml
βββ README.md
βββ my_package/
β βββ __init__.py
β βββ module.py
βββ tests/
βββ test_module.pySimpler but:
- Can import package without installing
- Less professional for libraries
Pattern 3: Multi-Package Project
project/
βββ pyproject.toml
βββ packages/
β βββ package-a/
β β βββ src/
β β βββ package_a/
β βββ package-b/
β βββ src/
β βββ package_b/
βββ tests/Detailed patterns and worked examples
Detailed pattern documentation lives in references/details.md. Read that file when the navigation tier above is insufficient.