Recipe Rendering Basics#
Welcome to the rattler-build Python bindings tutorial! This tutorial will teach you how to:
- Load recipes from YAML strings and Python dictionaries
- Configure variants (different build configurations)
- Render recipes to produce fully evaluated build specifications
- Understand the difference between Stage0 (template) and Stage1 (evaluated) recipes
Let's get started!
import json
import pprint
import yaml
from rattler_build import (
MultiOutputRecipe,
PlatformConfig,
RenderConfig,
SingleOutputRecipe,
Stage0Recipe,
VariantConfig,
)
Example 1: Loading a Simple Recipe from YAML#
The most common way to define a recipe is using YAML format. Let's create a simple package recipe:
# Define a simple recipe in YAML format with Jinja templates
simple_recipe_yaml = """
package:
name: my-simple-package
version: "1.0.0"
build:
number: 0
script:
- echo "Building my package"
requirements:
host:
- python ${{ python }}.*
- numpy ${{ numpy }}.*
run:
- python
- numpy >=${{ numpy }}
about:
homepage: https://github.com/example/my-package
license: MIT
summary: A simple example package
"""
# Parse the YAML into a Stage0Recipe object
simple_recipe = Stage0Recipe.from_yaml(simple_recipe_yaml)
print("Recipe loaded successfully!")
print(f"Type: {type(simple_recipe).__name__}")
print(f"Is single output: {isinstance(simple_recipe, SingleOutputRecipe)}")
print(f"Is multi output: {isinstance(simple_recipe, MultiOutputRecipe)}")
print("\nRecipe structure (as dict):")
print(json.dumps(simple_recipe.to_dict(), indent=2))
Recipe loaded successfully!
Type: SingleOutputRecipe
Is single output: True
Is multi output: False
Recipe structure (as dict):
{
"package": {
"name": "my-simple-package",
"version": "1.0.0"
},
"build": {
"number": 0,
"script": {
"content": [
"echo \"Building my package\""
]
},
"python": {
"entry_points": [],
"skip_pyc_compilation": [],
"use_python_app_entrypoint": false
},
"skip": [],
"always_copy_files": [],
"always_include_files": [],
"merge_build_and_host_envs": false,
"files": [],
"dynamic_linking": {
"rpaths": [],
"binary_relocation": true,
"missing_dso_allowlist": [],
"rpath_allowlist": []
},
"variant": {
"use_keys": [],
"ignore_keys": []
},
"prefix_detection": {
"force_file_type": {
"text": [],
"binary": []
},
"ignore": false,
"ignore_binary_files": false
},
"post_process": []
},
"requirements": {
"host": [
"python ${{ python }}.*",
"numpy ${{ numpy }}.*"
],
"run": [
"python",
"numpy >=${{ numpy }}"
]
},
"about": {
"homepage": "https://github.com/example/my-package",
"license": "MIT",
"summary": "A simple example package"
},
"extra": {}
}
Example 2: Creating a Recipe from a Python Dictionary#
You can also create recipes from Python dictionaries. Let's verify that Recipe.from_yaml() and Recipe.from_dict() produce the same result when given the same data:
# Parse the same YAML as a Python dictionary
recipe_dict = yaml.safe_load(simple_recipe_yaml)
# Create Stage0Recipe from dictionary
dict_recipe = Stage0Recipe.from_dict(recipe_dict)
print("Recipe created from dictionary!")
# Assert that both recipes are the same
yaml_dict = simple_recipe.to_dict()
dict_dict = dict_recipe.to_dict()
assert yaml_dict == dict_dict, "Recipes should be identical!"
print("\nBoth recipes are identical!")
Recipe created from dictionary!
Both recipes are identical!
Example 3: Understanding VariantConfig — Merging and Zip Keys#
Variants allow you to build the same package with different configurations (e.g., different Python versions, compilers, or dependencies).
In practice, variant configs often come from separate sources — for example, one config defines Python versions and another defines NumPy versions. You can combine them with merge(), which returns a new config where the other config's values take precedence for overlapping keys.
By default, VariantConfig creates all possible combinations (Cartesian product), but we can use zip_keys to pair specific variants together:
# Variant configs often come from separate sources — merge them together
python_variants = VariantConfig({"python": ["3.9", "3.10", "3.11"]})
numpy_variants = VariantConfig({"numpy": ["1.21", "1.22", "1.23"]})
# merge() returns a new config — originals are not modified
variant_config_without_zip = python_variants.merge(numpy_variants)
print("Variant Configuration (merged from two configs)")
print("=" * 60)
print(f"Variant keys: {variant_config_without_zip.keys()}")
print(f"Python versions: {variant_config_without_zip.get_values('python')}")
print(f"Numpy versions: {variant_config_without_zip.get_values('numpy')}")
print("\nWITHOUT zip_keys (Cartesian product):")
print(f"Total combinations: {len(variant_config_without_zip.combinations())} (3 x 3)")
print("\nAll possible combinations:")
pprint.pprint(variant_config_without_zip.combinations())
# Create a new VariantConfig with zip_keys (python and numpy zipped together by index)
variant_config = VariantConfig(
variant_config_without_zip.to_dict(),
zip_keys=[["python", "numpy"]],
)
print("\n" + "=" * 60)
print("WITH zip_keys (paired by index):")
print(f"Zip keys: {variant_config.zip_keys}")
print(f"Total combinations: {len(variant_config.combinations())} (paired)")
print("\nPaired combinations:")
pprint.pprint(variant_config.combinations())
print("\nVariant config as dict:")
print(json.dumps(variant_config.to_dict(), indent=2))
Variant Configuration (merged from two configs)
============================================================
Variant keys: ['numpy', 'python']
Python versions: ['3.9', '3.10', '3.11']
Numpy versions: ['1.21', '1.22', '1.23']
WITHOUT zip_keys (Cartesian product):
Total combinations: 9 (3 x 3)
All possible combinations:
[{'numpy': '1.21', 'python': '3.10'},
{'numpy': '1.21', 'python': '3.11'},
{'numpy': '1.21', 'python': '3.9'},
{'numpy': '1.22', 'python': '3.10'},
{'numpy': '1.22', 'python': '3.11'},
{'numpy': '1.22', 'python': '3.9'},
{'numpy': '1.23', 'python': '3.10'},
{'numpy': '1.23', 'python': '3.11'},
{'numpy': '1.23', 'python': '3.9'}]
============================================================
WITH zip_keys (paired by index):
Zip keys: [['python', 'numpy']]
Total combinations: 3 (paired)
Paired combinations:
[{'numpy': '1.21', 'python': '3.9'},
{'numpy': '1.22', 'python': '3.10'},
{'numpy': '1.23', 'python': '3.11'}]
Variant config as dict:
{
"numpy": [
"1.21",
"1.22",
"1.23"
],
"python": [
"3.9",
"3.10",
"3.11"
]
}
Example 4: RenderConfig - Controlling the Build Environment#
RenderConfig lets you specify the target platform and add custom context variables for recipe rendering:
# Create a render config with custom settings
platform_config = PlatformConfig(
target_platform="linux-64",
build_platform="linux-64",
host_platform="linux-64",
experimental=False,
)
render_config = RenderConfig(
platform=platform_config,
extra_context={
"custom_var": "custom_value",
"build_timestamp": "2024-01-01",
"my_number": 42,
},
)
print("Render Configuration")
print("=" * 60)
print(f"Target platform: {render_config.target_platform}")
print(f"Build platform: {render_config.build_platform}")
print(f"Host platform: {render_config.host_platform}")
print(f"Experimental: {render_config.experimental}")
print("\nCustom context variables:")
print(json.dumps(render_config.get_all_context(), indent=2))
Render Configuration
============================================================
Target platform: linux-64
Build platform: linux-64
Host platform: linux-64
Experimental: False
Custom context variables:
{
"custom_var": "custom_value",
"build_timestamp": "2024-01-01",
"my_number": 42
}
Example 5: Rendering Recipe with Variants#
Now let's put it all together! We'll use the recipe from Example 1, the variant config from Example 3, and the render config from Example 4 to render our recipe with multiple variants.
Stage0 is the parsed recipe with Jinja templates still intact (e.g., ${{ python }}).
Stage1 is the fully evaluated recipe with all templates resolved to actual values.
# Render the recipe with all the configurations we've created
rendered_variants = simple_recipe.render(variant_config, render_config)
print("STAGE 0 (Parsed, templates intact)")
print("=" * 60)
print(f"Package name (raw): {simple_recipe.package.name}")
print(f"Package version (raw): {simple_recipe.package.version}")
print(f"Host requirements (raw): {simple_recipe.requirements.host}")
print(f"\nRendered {len(rendered_variants)} variant(s)")
print("=" * 60)
for i, rendered_variant in enumerate(rendered_variants, 1):
variant_values = rendered_variant.variant
stage1_recipe = rendered_variant.recipe
print(f"\nSTAGE 1 - Variant {i} (Rendered, templates evaluated)")
print("-" * 60)
print(f" Variant config: {json.dumps(variant_values, indent=4)}")
print(f" Package name: {stage1_recipe.package.name}")
print(f" Package version: {stage1_recipe.package.version}")
print(f" Python: {variant_values.get('python')}")
print(f" Numpy: {variant_values.get('numpy')}")
print(f" Host requirements: {stage1_recipe.requirements.host}")
print(f" Run requirements: {stage1_recipe.requirements.run}")
print(f" Build string: {stage1_recipe.build.string}")
print("\n" + "=" * 60)
print("Recipe rendering complete!")
STAGE 0 (Parsed, templates intact)
============================================================
Package name (raw): my-simple-package
Package version (raw): 1.0.0
Host requirements (raw): ['python ${{ python }}.*', 'numpy ${{ numpy }}.*']
Rendered 3 variant(s)
============================================================
STAGE 1 - Variant 1 (Rendered, templates evaluated)
------------------------------------------------------------
Variant config: {
"numpy": "1.21",
"python": "3.9",
"target_platform": "linux-64"
}
Package name: my-simple-package
Package version: 1.0.0
Python: 3.9
Numpy: 1.21
Host requirements: ['python 3.9.*', 'numpy 1.21.*']
Run requirements: ['python', 'numpy >=1.21']
Build string: np121py39h13fef27_0
STAGE 1 - Variant 2 (Rendered, templates evaluated)
------------------------------------------------------------
Variant config: {
"numpy": "1.22",
"python": "3.10",
"target_platform": "linux-64"
}
Package name: my-simple-package
Package version: 1.0.0
Python: 3.10
Numpy: 1.22
Host requirements: ['python 3.10.*', 'numpy 1.22.*']
Run requirements: ['python', 'numpy >=1.22']
Build string: np122py310h27bf20f_0
STAGE 1 - Variant 3 (Rendered, templates evaluated)
------------------------------------------------------------
Variant config: {
"numpy": "1.23",
"python": "3.11",
"target_platform": "linux-64"
}
Package name: my-simple-package
Package version: 1.0.0
Python: 3.11
Numpy: 1.23
Host requirements: ['python 3.11.*', 'numpy 1.23.*']
Run requirements: ['python', 'numpy >=1.23']
Build string: np123py311hadc6f11_0
============================================================
Recipe rendering complete!
Example 6: Building the Package#
Finally, let's actually build the package! We'll take the rendered variants and build them into conda packages:
# Build each variant
print("Building packages...")
print("=" * 60)
print(f"Recipe path: {simple_recipe.recipe_path}")
for i, variant in enumerate(rendered_variants, 1):
print(f"\nBuilding variant {i}/{len(rendered_variants)}")
stage1_recipe = variant.recipe
package = stage1_recipe.package
build = stage1_recipe.build
print(f" Package: {package.name}")
print(f" Version: {package.version}")
print(f" Build string: {build.string}")
result = variant.run_build()
# Display build result information
print(f" Build complete in {result.build_time:.2f}s!")
print(f" Package: {result.packages[0]}")
if result.variant:
print(f" Variant: {result.variant}")
# Display build log
if result.log:
print(f" Build log: {len(result.log)} messages captured")
print("\n Build log details:")
for log_entry in result.log[:10]: # Show first 10 log entries
print(f" {log_entry}")
if len(result.log) > 10:
print(f" ... and {len(result.log) - 10} more messages")
print("\n" + "=" * 60)
print("All builds completed successfully!")
print(f"\nBuilt packages are available in: {result.output_dir}")
Building packages...
============================================================
Recipe path: /tmp/rattler_build_m4ir_946/recipe.yaml
Building variant 1/3
Package: my-simple-package
Version: 1.0.0
Build string: np121py39h13fef27_0
Build complete in 2.94s!
Package: /tmp/rattler_build_m4ir_946/output/linux-64/my-simple-package-1.0.0-np121py39h13fef27_0.conda
Variant: {'numpy': '1.21', 'target_platform': 'linux-64', 'python': '3.9'}
Build log: 52 messages captured
Build log details:
Starting build of 1 outputs
No sources to fetch
Could not find noarch/repodata.json. Creating new one.
Adding 0 packages to subdir noarch.
Successfully added 0 packages to subdir noarch.
Writing repodata to noarch/repodata.json
Platform: linux-64 [2m[__unix=0=0, __linux=6.17.0=0, __glibc=2.39=0, __archspec=1=zen2][0m
Channels:
- file:///tmp/rattler_build_m4ir_946/output/
- conda-forge
... and 42 more messages
Building variant 2/3
Package: my-simple-package
Version: 1.0.0
Build string: np122py310h27bf20f_0
Build complete in 3.36s!
Package: /tmp/rattler_build_m4ir_946/output/linux-64/my-simple-package-1.0.0-np122py310h27bf20f_0.conda
Variant: {'python': '3.10', 'target_platform': 'linux-64', 'numpy': '1.22'}
Build log: 52 messages captured
Build log details:
Starting build of 1 outputs
No sources to fetch
Adding 0 packages to subdir linux-64.
Successfully added 0 packages to subdir linux-64.
Writing repodata to linux-64/repodata.json
Adding 0 packages to subdir noarch.
Successfully added 0 packages to subdir noarch.
Writing repodata to noarch/repodata.json
Platform: linux-64 [2m[__unix=0=0, __linux=6.17.0=0, __glibc=2.39=0, __archspec=1=zen2][0m
Channels:
... and 42 more messages
Building variant 3/3
Package: my-simple-package
Version: 1.0.0
Build string: np123py311hadc6f11_0
Build complete in 2.38s!
Package: /tmp/rattler_build_m4ir_946/output/linux-64/my-simple-package-1.0.0-np123py311hadc6f11_0.conda
Variant: {'target_platform': 'linux-64', 'numpy': '1.23', 'python': '3.11'}
Build log: 52 messages captured
Build log details:
Starting build of 1 outputs
No sources to fetch
Adding 0 packages to subdir linux-64.
Successfully added 0 packages to subdir linux-64.
Writing repodata to linux-64/repodata.json
Adding 0 packages to subdir noarch.
Successfully added 0 packages to subdir noarch.
Writing repodata to noarch/repodata.json
Platform: linux-64 [2m[__unix=0=0, __linux=6.17.0=0, __glibc=2.39=0, __archspec=1=zen2][0m
Channels:
... and 42 more messages
============================================================
All builds completed successfully!
Built packages are available in: /tmp/rattler_build_m4ir_946/output
Summary#
In this tutorial, you learned:
- Recipe Creation: Load recipes from YAML strings (
Stage0Recipe.from_yaml()) or Python dicts (Stage0Recipe.from_dict()) - VariantConfig: Define build variants, use
zip_keysto pair specific combinations, and load/merge multiple variant config files - RenderConfig: Configure target platforms and add custom context variables
- Stage0 vs Stage1: Understand the difference between parsed templates and evaluated recipes
- Rendering: Use
recipe.render()to transform Stage0 -> Stage1 with variants - Building: Use
variant.run_build()to build conda packages, which returns aBuildResultwith package paths, metadata, timing information, and captured build logs