Recipe Rendering Basics#
Welcome to the rattler-build Python bindings tutorial! This tutorial will teach you how to:
- Load recipes from YAML strings and Python dictionaries
- Configure variants (different build configurations)
- Render recipes to produce fully evaluated build specifications
- Understand the difference between Stage0 (template) and Stage1 (evaluated) recipes
Let's get started!
import json
import pprint
import yaml
from rattler_build import (
MultiOutputRecipe,
PlatformConfig,
RenderConfig,
SingleOutputRecipe,
Stage0Recipe,
VariantConfig,
)
Example 1: Loading a Simple Recipe from YAML#
The most common way to define a recipe is using YAML format. Let's create a simple package recipe:
# Define a simple recipe in YAML format with Jinja templates
simple_recipe_yaml = """
package:
name: my-simple-package
version: "1.0.0"
build:
number: 0
script:
- echo "Building my package"
requirements:
host:
- python ${{ python }}.*
- numpy ${{ numpy }}.*
run:
- python
- numpy >=${{ numpy }}
about:
homepage: https://github.com/example/my-package
license: MIT
summary: A simple example package
"""
# Parse the YAML into a Stage0Recipe object
simple_recipe = Stage0Recipe.from_yaml(simple_recipe_yaml)
print("Recipe loaded successfully!")
print(f"Type: {type(simple_recipe).__name__}")
print(f"Is single output: {isinstance(simple_recipe, SingleOutputRecipe)}")
print(f"Is multi output: {isinstance(simple_recipe, MultiOutputRecipe)}")
print("\nRecipe structure (as dict):")
print(json.dumps(simple_recipe.to_dict(), indent=2))
Recipe loaded successfully!
Type: SingleOutputRecipe
Is single output: True
Is multi output: False
Recipe structure (as dict):
{
"package": {
"name": "my-simple-package",
"version": "1.0.0"
},
"build": {
"number": 0,
"string": null,
"script": {
"content": [
"echo \"Building my package\""
]
},
"noarch": null,
"python": {
"entry_points": [],
"skip_pyc_compilation": [],
"use_python_app_entrypoint": false,
"site_packages_path": null
},
"skip": [],
"always_copy_files": [],
"always_include_files": [],
"merge_build_and_host_envs": false,
"files": [],
"dynamic_linking": {
"rpaths": [],
"binary_relocation": true,
"missing_dso_allowlist": [],
"rpath_allowlist": [],
"overdepending_behavior": null,
"overlinking_behavior": null
},
"variant": {
"use_keys": [],
"ignore_keys": [],
"down_prioritize_variant": null
},
"prefix_detection": {
"force_file_type": {
"text": [],
"binary": []
},
"ignore": false,
"ignore_binary_files": false
},
"post_process": []
},
"requirements": {
"host": [
"python ${{ python }}.*",
"numpy ${{ numpy }}.*"
],
"run": [
"python",
"numpy >=${{ numpy }}"
]
},
"about": {
"homepage": "https://github.com/example/my-package",
"license": "MIT",
"license_family": null,
"summary": "A simple example package",
"description": null,
"documentation": null,
"repository": null
},
"extra": {}
}
Example 2: Creating a Recipe from a Python Dictionary#
You can also create recipes from Python dictionaries. Let's verify that Recipe.from_yaml() and Recipe.from_dict() produce the same result when given the same data:
# Parse the same YAML as a Python dictionary
recipe_dict = yaml.safe_load(simple_recipe_yaml)
# Create Stage0Recipe from dictionary
dict_recipe = Stage0Recipe.from_dict(recipe_dict)
print("Recipe created from dictionary!")
# Assert that both recipes are the same
yaml_dict = simple_recipe.to_dict()
dict_dict = dict_recipe.to_dict()
assert yaml_dict == dict_dict, "Recipes should be identical!"
print("\nBoth recipes are identical!")
Recipe created from dictionary!
Both recipes are identical!
Example 3: Understanding VariantConfig with Zip Keys#
Variants allow you to build the same package with different configurations (e.g., different Python versions, compilers, or dependencies). By default, VariantConfig creates all possible combinations (Cartesian product), but we can use zip_keys to pair specific variants together:
# Create a VariantConfig from a dictionary
variant_dict = {
"python": ["3.9", "3.10", "3.11"],
"numpy": ["1.21", "1.22", "1.23"],
}
variant_config_without_zip = VariantConfig(variant_dict)
print("Variant Configuration")
print("=" * 60)
print(f"Variant keys: {variant_config_without_zip.keys()}")
print(f"Python versions: {variant_config_without_zip.get_values('python')}")
print(f"Numpy versions: {variant_config_without_zip.get_values('numpy')}")
print("\nWITHOUT zip_keys (Cartesian product):")
print(f"Total combinations: {len(variant_config_without_zip.combinations())} (3 x 3)")
print("\nAll possible combinations:")
pprint.pprint(variant_config_without_zip.combinations())
# Create a new VariantConfig with zip_keys (python and numpy zipped together by index)
variant_config = VariantConfig(variant_dict, zip_keys=[["python", "numpy"]])
print("\n" + "=" * 60)
print("WITH zip_keys (paired by index):")
print(f"Zip keys: {variant_config.zip_keys}")
print(f"Total combinations: {len(variant_config.combinations())} (paired)")
print("\nPaired combinations:")
pprint.pprint(variant_config.combinations())
print("\nVariant config as dict:")
print(json.dumps(variant_config.to_dict(), indent=2))
Variant Configuration
============================================================
Variant keys: ['numpy', 'python']
Python versions: ['3.9', '3.10', '3.11']
Numpy versions: ['1.21', '1.22', '1.23']
WITHOUT zip_keys (Cartesian product):
Total combinations: 9 (3 x 3)
All possible combinations:
[{'numpy': '1.21', 'python': '3.10'},
{'numpy': '1.21', 'python': '3.11'},
{'numpy': '1.21', 'python': '3.9'},
{'numpy': '1.22', 'python': '3.10'},
{'numpy': '1.22', 'python': '3.11'},
{'numpy': '1.22', 'python': '3.9'},
{'numpy': '1.23', 'python': '3.10'},
{'numpy': '1.23', 'python': '3.11'},
{'numpy': '1.23', 'python': '3.9'}]
============================================================
WITH zip_keys (paired by index):
Zip keys: [['python', 'numpy']]
Total combinations: 3 (paired)
Paired combinations:
[{'numpy': '1.21', 'python': '3.9'},
{'numpy': '1.22', 'python': '3.10'},
{'numpy': '1.23', 'python': '3.11'}]
Variant config as dict:
{
"numpy": [
"1.21",
"1.22",
"1.23"
],
"python": [
"3.9",
"3.10",
"3.11"
]
}
Example 4: RenderConfig - Controlling the Build Environment#
RenderConfig lets you specify the target platform and add custom context variables for recipe rendering:
# Create a render config with custom settings
platform_config = PlatformConfig(
target_platform="linux-64",
build_platform="linux-64",
host_platform="linux-64",
experimental=False,
)
render_config = RenderConfig(
platform=platform_config,
extra_context={
"custom_var": "custom_value",
"build_timestamp": "2024-01-01",
"my_number": 42,
},
)
print("Render Configuration")
print("=" * 60)
print(f"Target platform: {render_config.target_platform}")
print(f"Build platform: {render_config.build_platform}")
print(f"Host platform: {render_config.host_platform}")
print(f"Experimental: {render_config.experimental}")
print("\nCustom context variables:")
print(json.dumps(render_config.get_all_context(), indent=2))
Render Configuration
============================================================
Target platform: linux-64
Build platform: linux-64
Host platform: linux-64
Experimental: False
Custom context variables:
{
"custom_var": "custom_value",
"build_timestamp": "2024-01-01",
"my_number": 42
}
Example 5: Rendering Recipe with Variants#
Now let's put it all together! We'll use the recipe from Example 1, the variant config from Example 3, and the render config from Example 4 to render our recipe with multiple variants.
Stage0 is the parsed recipe with Jinja templates still intact (e.g., ${{ python }}).
Stage1 is the fully evaluated recipe with all templates resolved to actual values.
# Render the recipe with all the configurations we've created
rendered_variants = simple_recipe.render(variant_config, render_config)
print("STAGE 0 (Parsed, templates intact)")
print("=" * 60)
print(f"Package name (raw): {simple_recipe.package.name}")
print(f"Package version (raw): {simple_recipe.package.version}")
print(f"Host requirements (raw): {simple_recipe.requirements.host}")
print(f"\nRendered {len(rendered_variants)} variant(s)")
print("=" * 60)
for i, rendered_variant in enumerate(rendered_variants, 1):
variant_values = rendered_variant.variant
stage1_recipe = rendered_variant.recipe
print(f"\nSTAGE 1 - Variant {i} (Rendered, templates evaluated)")
print("-" * 60)
print(f" Variant config: {json.dumps(variant_values, indent=4)}")
print(f" Package name: {stage1_recipe.package.name}")
print(f" Package version: {stage1_recipe.package.version}")
print(f" Python: {variant_values.get('python')}")
print(f" Numpy: {variant_values.get('numpy')}")
print(f" Host requirements: {stage1_recipe.requirements.host}")
print(f" Run requirements: {stage1_recipe.requirements.run}")
print(f" Build string: {stage1_recipe.build.string}")
print("\n" + "=" * 60)
print("Recipe rendering complete!")
STAGE 0 (Parsed, templates intact)
============================================================
Package name (raw): my-simple-package
Package version (raw): 1.0.0
Host requirements (raw): ['python ${{ python }}.*', 'numpy ${{ numpy }}.*']
Rendered 3 variant(s)
============================================================
STAGE 1 - Variant 1 (Rendered, templates evaluated)
------------------------------------------------------------
Variant config: {
"numpy": "1.21",
"python": "3.9",
"target_platform": "linux-64"
}
Package name: my-simple-package
Package version: 1.0.0
Python: 3.9
Numpy: 1.21
Host requirements: ['python 3.9.*', 'numpy 1.21.*']
Run requirements: ['python', 'numpy >=1.21']
Build string: np121py39h13fef27_0
STAGE 1 - Variant 2 (Rendered, templates evaluated)
------------------------------------------------------------
Variant config: {
"numpy": "1.22",
"python": "3.10",
"target_platform": "linux-64"
}
Package name: my-simple-package
Package version: 1.0.0
Python: 3.10
Numpy: 1.22
Host requirements: ['python 3.10.*', 'numpy 1.22.*']
Run requirements: ['python', 'numpy >=1.22']
Build string: np122py310h27bf20f_0
STAGE 1 - Variant 3 (Rendered, templates evaluated)
------------------------------------------------------------
Variant config: {
"numpy": "1.23",
"python": "3.11",
"target_platform": "linux-64"
}
Package name: my-simple-package
Package version: 1.0.0
Python: 3.11
Numpy: 1.23
Host requirements: ['python 3.11.*', 'numpy 1.23.*']
Run requirements: ['python', 'numpy >=1.23']
Build string: np123py311hadc6f11_0
============================================================
Recipe rendering complete!
Example 6: Building the Package#
Finally, let's actually build the package! We'll take the rendered variants and build them into conda packages:
# Build each variant
print("Building packages...")
print("=" * 60)
print(f"Recipe path: {simple_recipe.recipe_path}")
for i, variant in enumerate(rendered_variants, 1):
print(f"\nBuilding variant {i}/{len(rendered_variants)}")
stage1_recipe = variant.recipe
package = stage1_recipe.package
build = stage1_recipe.build
print(f" Package: {package.name}")
print(f" Version: {package.version}")
print(f" Build string: {build.string}")
result = variant.run_build()
# Display build result information
print(f" Build complete in {result.build_time:.2f}s!")
print(f" Package: {result.packages[0]}")
if result.variant:
print(f" Variant: {result.variant}")
# Display build log
if result.log:
print(f" Build log: {len(result.log)} messages captured")
print("\n Build log details:")
for log_entry in result.log[:10]: # Show first 10 log entries
print(f" {log_entry}")
if len(result.log) > 10:
print(f" ... and {len(result.log) - 10} more messages")
print("\n" + "=" * 60)
print("All builds completed successfully!")
print(f"\nBuilt packages are available in: {result.output_dir}")
Building packages...
============================================================
Recipe path: /tmp/rattler_build_mp4jfdci/recipe.yaml
Building variant 1/3
Package: my-simple-package
Version: 1.0.0
Build string: np121py39h13fef27_0
Build complete in 2.78s!
Package: /tmp/rattler_build_mp4jfdci/output/linux-64/my-simple-package-1.0.0-np121py39h13fef27_0.conda
Variant: {'numpy': '1.21', 'target_platform': 'linux-64', 'python': '3.9'}
Build log: 52 messages captured
Build log details:
Starting build of 1 outputs
No sources to fetch
Could not find linux-64/repodata.json. Creating new one.
Adding 0 packages to subdir linux-64.
Successfully added 0 packages to subdir linux-64.
Writing repodata to linux-64/repodata.json
Could not find noarch/repodata.json. Creating new one.
Adding 0 packages to subdir noarch.
Successfully added 0 packages to subdir noarch.
Writing repodata to noarch/repodata.json
... and 42 more messages
Building variant 2/3
Package: my-simple-package
Version: 1.0.0
Build string: np122py310h27bf20f_0
Build complete in 2.56s!
Package: /tmp/rattler_build_mp4jfdci/output/linux-64/my-simple-package-1.0.0-np122py310h27bf20f_0.conda
Variant: {'target_platform': 'linux-64', 'python': '3.10', 'numpy': '1.22'}
Build log: 47 messages captured
Build log details:
Starting build of 1 outputs
No sources to fetch
Adding 0 packages to subdir linux-64.
Successfully added 0 packages to subdir linux-64.
Writing repodata to linux-64/repodata.json
Resolving host environment:
Platform: linux-64 [2m[__unix=0=0, __linux=6.14.0=0, __glibc=2.39=0, __archspec=1=zen2][0m
Channels:
- file:///tmp/rattler_build_mp4jfdci/output/
- conda-forge
... and 37 more messages
Building variant 3/3
Package: my-simple-package
Version: 1.0.0
Build string: np123py311hadc6f11_0
Build complete in 2.32s!
Package: /tmp/rattler_build_mp4jfdci/output/linux-64/my-simple-package-1.0.0-np123py311hadc6f11_0.conda
Variant: {'python': '3.11', 'target_platform': 'linux-64', 'numpy': '1.23'}
Build log: 47 messages captured
Build log details:
Starting build of 1 outputs
No sources to fetch
Adding 0 packages to subdir linux-64.
Successfully added 0 packages to subdir linux-64.
Writing repodata to linux-64/repodata.json
Resolving host environment:
Platform: linux-64 [2m[__unix=0=0, __linux=6.14.0=0, __glibc=2.39=0, __archspec=1=zen2][0m
Channels:
- file:///tmp/rattler_build_mp4jfdci/output/
- conda-forge
... and 37 more messages
============================================================
All builds completed successfully!
Built packages are available in: /tmp/rattler_build_mp4jfdci/output
Summary#
In this tutorial, you learned:
- Recipe Creation: Load recipes from YAML strings (
Stage0Recipe.from_yaml()) or Python dicts (Stage0Recipe.from_dict()) - VariantConfig: Define build variants and use
zip_keysto pair specific combinations - RenderConfig: Configure target platforms and add custom context variables
- Stage0 vs Stage1: Understand the difference between parsed templates and evaluated recipes
- Rendering: Use
recipe.render()to transform Stage0 -> Stage1 with variants - Building: Use
variant.run_build()to build conda packages, which returns aBuildResultwith package paths, metadata, timing information, and captured build logs