A high-performance configuration framework for Python applications, built with Rust.

Lerna is a rewrite of Facebook's Hydra configuration framework. It provides the same powerful API with significantly improved performance through a Rust core.

• Same Hydra API: Drop-in replacement for Hydra - just change import hydra to import lerna
• Rust-powered: Core config parsing and loading implemented in Rust via PyO3
• Full Compatibility: 2,854 tests passing, nearly 100% Hydra compatibility
• No ANTLR: Override parser completely rewritten in Rust (~2,400 LOC removed)
• Zero Warnings: Clean Rust codebase with no compiler warnings
• Extension Points: Rust traits for Callback, ConfigSource, Launcher, and Sweeper with Python interoperability

pip install lerna

import lerna
from omegaconf import DictConfig

@lerna.main(config_path="conf", config_name="config")
def my_app(cfg: DictConfig) -> None:
    print(cfg.db.driver)
    print(cfg.db.user)

if __name__ == "__main__":
    my_app()

Lerna is a drop-in replacement for Hydra. To migrate:

# Before (Hydra)
import hydra
from hydra import compose, initialize
from hydra.core.config_store import ConfigStore

# After (Lerna)
import lerna
from lerna import compose, initialize
from lerna.core.config_store import ConfigStore

All your existing configs, overrides, and patterns work unchanged:

# Same CLI interface
python my_app.py db=postgres server.port=8080

# Same multirun syntax
python my_app.py -m db=mysql,postgres server.port=8080,8081

# Same sweep functions
python my_app.py -m learning_rate=interval(0.001,0.1) batch_size=choice(16,32,64)

These are shell-specific completion behaviors, not functional differences.

Lerna addresses several long-standing Hydra issues that have been open for years:

Lerna adds intuitive, cross-platform list operations:

# Append items to a list
python app.py 'tags=append(new_tag)'
python app.py 'tags=append(a,b,c)'  # Multiple items

# Prepend items
python app.py 'tags=prepend(first)'

# Insert at specific index
python app.py 'tags=insert(0,first_item)'

# Extend from another config list
python app.py 'tags=extend(${defaults.tags})'

# Remove by index
python app.py 'tags=pop(0)'      # Remove first
python app.py 'tags=pop(-1)'     # Remove last

# Remove by value
python app.py 'tags=remove(old_tag)'

# Delete a slice (equivalent to del tags[1:3])
python app.py 'tags=delete_slice(1,3)'

# Clear entire list
python app.py 'tags=clear()'

remove_at, remove_value, list_clear, and extend_from remain available as compatibility aliases for pop, remove, clear, and extend, respectively.

These functions use shell-safe syntax (quote the entire override) and work on bash, zsh, fish, PowerShell, and cmd.

Hydra's ANTLR-based parser breaks when PYTHONOPTIMIZE=1 or PYTHONOPTIMIZE=2 is set. Lerna's Rust parser has no Python dependencies and works in all environments.

# This breaks Hydra but works with Lerna
PYTHONOPTIMIZE=2 python app.py db=postgres

Lerna adds an overrides parameter to @lerna.main() for setting default overrides that can be overridden from CLI:

@lerna.main(
    config_path="conf",
    config_name="config",
    overrides=["db.driver=postgres", "server.port=8080"]  # Default overrides
)
def my_app(cfg: DictConfig) -> None:
    print(cfg.db.driver)  # "postgres" by default, CLI can override

# Uses decorator defaults
python app.py                        # db.driver=postgres

# CLI overrides take precedence
python app.py db.driver=mysql        # db.driver=mysql

Lerna adds _call_=False to instantiate() for importing non-callable objects (like torch.int64):

from lerna.utils import instantiate
from omegaconf import OmegaConf

# Import a non-callable object directly
cfg = OmegaConf.create({
    "_target_": "torch.int64",
    "_call_": False,  # Don't try to call it
})
dtype = instantiate(cfg)  # Returns torch.int64 directly

Lerna discovers plugins from both lerna_plugins and hydra_plugins namespaces, enabling gradual migration:

# Both work:
# - lerna_plugins.my_plugin.MyPlugin  (new Lerna plugins)
# - hydra_plugins.my_plugin.MyPlugin  (existing Hydra plugins)

Hydra incorrectly treats appended defaults as relative paths when the main config is in a subfolder:

# Hydra bug: this fails because it looks for server/db/postgresql
python app.py --config-name=server/alpha +db@db_2=postgresql

# Lerna: correctly treats appended configs as absolute paths
python app.py --config-name=server/alpha +db@db_2=postgresql  # Works!

Hydra provides no way to remove or modify specific keys/values inherited from composed configs via the defaults list. Lerna adds a _patch_ directive that lets you apply override operations to the composed config before CLI overrides are applied.

# config.yaml
defaults:
  - some_lib/defaults    # pulls in a library config
  - _self_
  - _patch_:
    - ~unwanted_key                # delete a key
    - ~status=deprecated           # delete key only if value matches
    - items=remove(stale)          # remove a list item by value
    - items=pop(0)                 # remove a list item by index
    - +new_key=injected            # add a new key
    - setting=new_value            # change a value

Key resolution rules:

For root-level configs (no @ package), bare keys and _here_ are equivalent since the parent package is empty.

Patch entries accept the existing override string syntax or a structured mapping. Both forms execute in sequence and use the same package-scoping rules.

In delete_slice(start, stop?), omitting stop deletes from start through the end. Negative and out-of-range indexes follow Python slice behavior.

Structured value and values fields preserve YAML mappings, lists, booleans, nulls, and interpolations without serializing them through the override grammar. A mapping or list in values is one item. extend() is the operation that splices the selected source list into the destination; nested lists inside the source remain nested.

defaults:
  - _patch_@gateway:
    - op: append
      path: modules
      values:
        - /modules/rest
        - /modules/outputs
    - op: change
      path: settings
      value:
        enabled: true
        retries: 3
        label: null

append_unique() resolves values for structural comparison but preserves the unresolved value when it appends it. Existing duplicates remain. remove_all() removes every occurrence of each argument. extend() snapshots its source before copying, so extending a list from itself is well-defined.

Example with packaged config:

# config.yaml — using _patch_@vendor to scope bare keys to the vendor package
defaults:
  - vendor/large_defaults@vendor
  - _self_
  - _patch_@vendor:
    - ~debug_mode           # bare key → targets vendor.debug_mode
    - items=remove(x)       # bare key → targets vendor.items

# Multiple scoped patches can target different packages:
# - _patch_@db:
#   - ~debug
# - _patch_@server:
#   - port=9090

Nested patches: _patch_ directives in sub-configs accumulate naturally. If lib/refined.yaml has its own _patch_ that removes beta, and your root config adds _patch_@lib: to remove gamma, both patches apply — beta and gamma are both removed from the final config.

compose_with_provenance() returns a CompositionResult containing the unresolved composed config and its composition metadata. Consumers such as csp-gateway and ccflow can build configuration explanations without importing from lerna._internal, patching YAML loaders, or disabling Rust YAML parsing.

from lerna import compose_with_provenance, initialize_config_dir

with initialize_config_dir(config_dir="conf", version_base=None):
  result = compose_with_provenance("config", overrides=["gateway.modules=append(metrics)"])

print(result.config)                         # unresolved DictConfig
print(result.resolved_copy())                # independently resolved copy
print(result.selected_defaults)              # composition order and sources
print(result.available_options["gateway"])  # options seen for the group
print(result.provenance("gateway.modules[0]"))
print(result.patch_operations)               # config-authored operations
print(result.cli_overrides)                  # command-line operations

Node provenance includes the source identifier, source config, package, source key, origin, and operation index. List-item provenance follows surviving items as list operations move their indexes. Patch history retains removed items and their provenance.

Hydra produces empty string keys when using .. in defaults list paths:

# Hydra bug with ../dir2 produces config with empty string keys
# Lerna normalizes paths correctly
defaults:
  - ../dir2: child.yaml  # Now works correctly

Hydra breaks with importlib-resources 6.2+ due to OrphanPath objects not having is_file()/is_dir() methods. Lerna handles this gracefully.

Lerna provides a bridge that allows plugins registered via lerna to work with hydra-core. This enables you to write plugins once and have them work with both frameworks.

Add your plugin to pyproject.toml using the hydra.lernaplugins entry point group:

# For SearchPathPlugin modules:
[project.entry-points."hydra.lernaplugins"]
my-plugin = "my_package.plugin_module"

# For package-style config directories:
[project.entry-points."hydra.lernaplugins"]
my-plugin = "pkg:my_package.hydra"

# If only using lerna, you can also register under lerna.plugins:
[project.entry-points."lerna.plugins"]
my-plugin = "my_package.plugin_module"

Module-style entry points (like my_package.plugin_module) are imported and scanned for SearchPathPlugin subclasses.

Package-style entry points (like pkg:my_package.hydra) register config search paths directly.

When hydra-core is used, lerna's LernaGenericSearchPathPlugin (installed in the hydra_plugins namespace) discovers all plugins registered under hydra.lernaplugins and makes them available to hydra's plugin system.

This enables gradual migration: you can write plugins for lerna and they'll automatically work with existing hydra-core installations.

Hydra's plugin ecosystem (Optuna, Ray, Submitit, etc.) references hydra internally. To use them with Lerna:

# Option 1: Import aliasing (recommended)
import lerna as hydra  # Alias for plugin compatibility

# Option 2: Use Lerna's built-in extensions
from lerna import RustBasicLauncher, RustBasicSweeper

Lerna requires OmegaConf (same as Hydra):

pip install lerna omegaconf

The override parser is fully implemented in Rust with support for:

• All sweep types: choice(), range(), interval(), glob()
• Cast functions: int(), float(), str(), bool(), json_str()
• Modifiers: shuffle(), sort(), tag(), extend_list()
• User-defined functions via Python callbacks (with proper shadowing)
• Complex nested structures and interpolations

• High-performance YAML parsing in Rust
• Defaults list processing with proper package resolution
• Config merging and override application
• Full interpolation support via OmegaConf

• Job context management
• Output directory computation and creation
• Config/override file serialization

Pluggable architecture allowing both Rust and Python implementations:

• Callback: Lifecycle hooks (on_job_start, on_job_end, on_run_start, etc.)
• ConfigSource: Config loading from file://, pkg://, structured:// sources
• Launcher: Job execution orchestration (BasicLauncher included)
• Sweeper: Parameter sweep strategies (BasicSweeper with cartesian product included)

lerna/
├── lerna/              # Python package (Hydra API)
├── rust/               # Pure Rust core library (no Python deps)
│   └── src/
│       ├── parser/     # Override parser (2,800 LOC)
│       ├── config/     # Config loading
│       ├── omegaconf/  # OmegaConf compatibility
│       └── ...
└── src/                # PyO3 bindings

All remaining xfails are known shell-specific limitations, not bugs:

• 16 zsh completion tests (tilde handling in shells)
• 1 multirun completion test (partial override parsing)

# Build Rust extension
make develop

# Run tests
make test

This project is licensed under the MIT License - see the LICENSE file for details.

This project is based on Hydra by Facebook Research.