Skip to content

Quickstart

Welcome to the FlowerPower quickstart guide! This guide will walk you through the process of creating a "Hello World" project to demonstrate the core functionalities of the library.

Installation

First, ensure you have FlowerPower installed. We recommend using uv for a fast and reliable installation.

1
2
3
4
5
6
# Create and activate a virtual environment
uv venv
source .venv/bin/activate

# Install FlowerPower
uv pip install flowerpower

1. Initialize Your Project

You can create a new project using either the CLI or the Python API.

Using the CLI

1
2
flowerpower init --name hello-flowerpower
cd hello-flowerpower

Using the Python API

1
2
3
4
5
6
from flowerpower import FlowerPowerProject

# Initialize a new project
project = FlowerPowerProject.new(
    name='hello-flowerpower'
)

This creates a standard project structure with conf/ and pipelines/ directories.

2. Configure Your Project

The conf/project.yml file contains global settings for your project.

1
2
# conf/project.yml
name: hello-flowerpower

3. Create a Pipeline

Next, create a pipeline to define your data processing logic.

Using the CLI

1
flowerpower pipeline new hello_world

Using the Python API

1
2
3
4
from flowerpower import FlowerPowerProject

project = FlowerPowerProject.load('.')
project.pipeline_manager.new(name='hello_world')

This generates pipelines/hello_world.py for your pipeline logic and conf/pipelines/hello_world.yml for its configuration.

4. Implement the Pipeline

Open pipelines/hello_world.py and add your Hamilton functions.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# pipelines/hello_world.py
from pathlib import Path
from hamilton.function_modifiers import parameterize
from flowerpower.cfg.project import ProjectConfig
from flowerpower.cfg.pipeline import PipelineConfig

# Load project and pipeline configurations separately
project_cfg = ProjectConfig.load(name="hello-flowerpower")
pipeline_cfg = PipelineConfig.load(name="hello_world")
PARAMS = pipeline_cfg.params  # Access params from pipeline config

@parameterize(**PARAMS.get("greeting_message", {}))
def greeting_message(message: str) -> str:
    return f"{message},"

@parameterize(**PARAMS.target_name)
def target_name(name: str) -> str:
    return f"{name}!"

def full_greeting(greeting_message: str, target_name: str) -> str:
    """Combines the greeting and target."""
    print(f"Executing pipeline: {greeting_message} {target_name}")
    return f"{greeting_message} {target_name}"

5. Configure the Pipeline

In conf/pipelines/hello_world.yml, define the parameters and execution details for your pipeline.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
# conf/pipelines/hello_world.yml
params:
  greeting_message:
    message: "Hello"
  target_name:
    name: "World"

run:
  final_vars:
    - full_greeting

6. Run the Pipeline

You can run your pipeline synchronously for quick tests.

Synchronous Execution

This is useful for debugging and local development.

Using the CLI

The run command now primarily accepts a RunConfig object, but also allows individual parameters to be passed via **kwargs which override RunConfig attributes.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
# Basic pipeline execution
flowerpower pipeline run hello_world

# Run with individual parameters (kwargs)
flowerpower pipeline run hello_world --inputs '{"greeting_message": "Hi", "target_name": "FlowerPower"}' --final-vars '["full_greeting"]' --log-level DEBUG

# Run using a RunConfig from a YAML file
# Assuming you have a run_config.yaml like:
# inputs:
#   greeting_message: "Hola"
#   target_name: "Amigo"
# log_level: "INFO"
flowerpower pipeline run hello_world --run-config ./run_config.yaml

# Run using a RunConfig provided as a JSON string
flowerpower pipeline run hello_world --run-config '{"inputs": {"greeting_message": "Bonjour", "target_name": "Monde"}, "log_level": "INFO"}'

# Mixing RunConfig with individual parameters (kwargs overrides RunConfig)
# This will run with log_level="DEBUG" and inputs={"greeting_message": "Howdy", "target_name": "Partner"}
flowerpower pipeline run hello_world --run-config '{"inputs": {"greeting_message": "Original", "target_name": "Value"}, "log_level": "INFO"}' --inputs '{"greeting_message": "Howdy", "target_name": "Partner"}' --log-level DEBUG

Using the Python API

1
2
3
4
5
from flowerpower import FlowerPowerProject

project = FlowerPowerProject.load('.')
result = project.run('hello_world')
print(result)

Advanced Pipeline Execution with RunConfig

For more control over pipeline execution, you can use the RunConfig class to configure execution parameters.

Using RunConfig Directly

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
from flowerpower import FlowerPowerProject
from flowerpower.cfg.pipeline.run import RunConfig

project = FlowerPowerProject.load('.')

# Create a configuration with custom parameters
config = RunConfig(
    inputs={"greeting_message": "Hi", "target_name": "FlowerPower"},
    final_vars=["full_greeting"],
    log_level="DEBUG"
)

result = project.run('hello_world', run_config=config)
print(result)

The RunConfigBuilder provides a fluent interface for building complex configurations:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
from flowerpower import FlowerPowerProject
from flowerpower.cfg.pipeline.builder import RunConfigBuilder

project = FlowerPowerProject.load('.')

# Build a configuration using the builder pattern
config = (
    RunConfigBuilder(pipeline_name='hello_world')
    .with_inputs({"greeting_message": "Hello", "target_name": "World"})
    .with_final_vars(["full_greeting"])
    .with_log_level("DEBUG")
    .with_retry_config(max_retries=3, retry_delay=1.0)
    .build()
)

result = project.run('hello_world', run_config=config)
print(result)

Mixing RunConfig with Individual Parameters

You can also combine RunConfig with individual parameters, where individual parameters take precedence:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
from flowerpower import FlowerPowerProject
from flowerpower.cfg.pipeline.builder import RunConfigBuilder

project = FlowerPowerProject.load('.')

# Create a base configuration
base_config = RunConfigBuilder().with_log_level("INFO").build()

# Run with base config but override specific parameters
result = project.run(
    'hello_world',
    run_config=base_config,
    inputs={"greeting_message": "Greetings", "target_name": "Universe"}
)
print(result)

For more details on managing your project, refer to the API documentation for FlowerPowerProject, PipelineManager, and RunConfig.