Skip to main content

Overview

Each deployment strategy is designed for specific use cases. The system can auto-select the best strategy or you can specify one explicitly.

LOCAL

Run as a local Python process. Fastest for development and testing.

Features

  • No containerization overhead
  • Direct file system access
  • Easiest debugging
  • No additional setup required

Usage

How It Works

  1. Adapter creates a run.py entry point
  2. Runner imports and calls the main function
  3. Results returned directly

Configuration

DOCKER

Run in an isolated Docker container.

Features

  • Environment isolation
  • Reproducible builds
  • Easy dependency management
  • Cross-platform compatibility

Prerequisites

  • Docker installed and running

Usage

How It Works

  1. Adapter creates Dockerfile and docker-compose.yml
  2. Runner builds and starts container
  3. Inputs/outputs mapped via volumes

Dockerfile Template

Deploy to Modal.com for serverless GPU compute.

Features

  • Serverless scaling
  • GPU support (A100, H100)
  • Pay-per-use pricing
  • Fast cold starts

Prerequisites

  • Modal account and CLI
  • modal package installed

Usage

How It Works

  1. Adapter creates modal_app.py with Modal decorators
  2. Runner deploys to Modal
  3. HTTP endpoint for inference

BENTOML

Deploy with BentoML for production ML serving.

Features

  • Model versioning
  • Automatic batching
  • Prometheus metrics
  • Kubernetes-ready

Prerequisites

  • bentoml package installed

Usage

How It Works

  1. Adapter creates service.py with BentoML decorators
  2. Runner builds and serves the Bento
  3. HTTP API for inference

BentoML Service Template

LANGGRAPH

Deploy as a LangGraph agent workflow.

Features

  • Stateful agent execution
  • Tool integration
  • Streaming support
  • Checkpoint persistence

Prerequisites

  • langgraph package installed

Usage

How It Works

  1. Adapter creates graph.py with LangGraph nodes
  2. Runner compiles and runs the graph
  3. Streaming or batch execution

AUTO Selection

When DeployStrategy.AUTO is used, the selector analyzes the code:

Selection Criteria

Strategy Comparison

Run Interface

Each strategy defines how to call the deployed code:

Environment Variables

Pass environment variables to deployments: