Status snapshot · 2 June 2026

Forja

Governed control plane for LLM agents and models — registry, guardrails, HITL, training orchestration, and promotion gates.

Product direction

Forja is evolving from an agent governance runtime into a full model lifecycle forge: define agents and policies (Studio), validate with guardrails and human review (Runs, Approvals), orchestrate external fine-tuning (Azure ML), evaluate on canonical scenarios, and promote draft versions to production (Promotions).

Completed in this iteration

Phase 0 — Professional UI (English)

• Removed gamified copy (Hearth, Anvil, blades, levels, etc.)
• Navigation: Agents, Runs, Approvals, Promotions, Run history, Deployments
• docs/product-voice.md and karpathy.md referenced from README
• Terminal runs from the UI persist to data/executions.jsonl

Phase A — MLOps orchestration (foundation)

• Versioned datasets/ and models/ registries
• TrainingBackend strategy: mock and azure_ml
• REST: /api/datasets, /api/models, /api/training/runs
• Training runs stored under data/training_runs/

Phase B — Eval gate and governance promotion

• Post-train evaluation over agents/<name>/examples/*.txt
• POST /api/training/runs/{id}/evaluate — blocks promotion if guardrails fail
• /api/promotions — request, approve, reject; activates agent version in registry
• HTMX page /promotions for the governance queue

Azure ML SDK integration

• azure-ai-ml + azure-identity (DefaultAzureCredential)
• Submits command jobs when AZURE_ML_* workspace settings are set
• Falls back to stub mode locally when workspace is not configured
• Placeholder script: core/src/forja_core/training/_azure_job_bundle/train.py

Already in place (MVP core)

• LangGraph runtime with SQLite checkpoints (HITL survives restarts)
• Guardrails AI engine + policy YAML
• Agent registry with Git-like versioning
• REST API under /api and HTMX UI on port 8000

How to run the project

Option 1 — Docker (recommended)

cp .env.example .env
docker compose up

Open http://localhost:8000. Default LLM_PROVIDER=mock works without API keys.

Option 2 — Local Python

cp .env.example .env
python -m venv venv
source venv/bin/activate
pip install -r core/requirements.txt
pip install pytest pytest-asyncio ruff mypy freezegun

cd core/src
uvicorn forja_core.main:app --reload --host 0.0.0.0 --port 8000

Run from repo root with PYTHONPATH=core/src if needed:

PYTHONPATH=core/src uvicorn forja_core.main:app --reload --port 8000

Tests

source venv/bin/activate
./venv/bin/pytest tests/unit -q          # unit
./venv/bin/pytest tests/integration -q   # integration (slower)
make lint                                 # ruff + mypy (if tools on PATH)

Quick demo (3 steps)

1. Agents — /agents → inspect incident_analyzer
2. Runs — /run → scenario 01_sip_registration_drop (may trigger Approvals)
3. Approvals / Promotions — /approvals and /promotions

Key API endpoints

Area	Endpoint
Health	GET /health
Invoke agent	POST /api/agents/{name}/invoke
Training run	POST /api/training/runs
Post-train eval	POST /api/training/runs/{id}/evaluate
Promotion	POST /api/promotions → .../approve

Next steps (when you resume)

• Wire Forja datasets to Azure ML data assets (not only env vars)
• Replace placeholder train.py with real fine-tuning script
• Multi-tenant, OIDC, Postgres (enterprise roadmap)
• LLM-as-judge regression suite in CI

Documentation

• README.md — quickstart
• ARCHITECTURE.md — technical decisions
• docs/product-voice.md — UI language rules
• karpathy.md — development guidelines
• docs/futuro.md — roadmap