Installation

Requirements

• Python 3.12

• uv package manager (recommended)

Installing with uv

The recommended way to install Dataflow is using uv:

# Clone the repository
git clone <repository-url>
cd dataflow

# Install dependencies with uv
uv sync

Installing with pip

Alternatively, you can install using pip:

# Clone the repository
git clone <repository-url>
cd dataflow

# Create a virtual environment
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install in development mode
pip install -e .

Running Dataflow

Important

Ensure that your .env file is properly configured with database connection details and other necessary environment variables for the desired workflow.

Once installed and configured, run your workflows:

# Set the workflow to run
export WORKFLOW=your_workflow_name

# Run with Python
python -m dataflow.main

# Or with uv
uv run python -m dataflow.main

Docker

Dataflow includes Docker support for containerized deployments:

# Build the image
docker build -t dataflow .

# Run with environment file
docker run --env-file .env dataflow