Compare commits

...
17 Commits
Author SHA1 Message Date
juan f67eb68e3e cambio de enfoque para hacer una API multitenant 2026-06-11 16:34:48 +02:00
juan 37633920ce pagina unica 2026-06-10 18:21:33 +02:00
juan 7d0f10d21f the forgeeeee 2026-05-27 21:28:10 +02:00
juan 34b11dfc66 cambios profundos 2026-05-27 16:29:02 +02:00
juan d95ed15766 añadido guidelines de karpathy y repensado del proyecto 2026-05-25 19:28:58 +02:00
juan 8c594f8ddf migracion de streamlit a niceguy 2026-05-23 21:46:44 +02:00
juan b4964261ec proyecto generalizado 2026-05-23 16:44:45 +02:00
juan 2f990ea636 added plan of transformation 2026-05-21 20:19:18 +02:00
juan 4244b00ce7 añadido recorrido codificación a walkthrough 2026-05-19 18:13:13 +02:00
juan dd5cfd9799 added walkthrough 2026-05-12 15:04:47 +02:00
Juan 9377730de4 algunas soluciones 2026-05-11 17:30:44 +02:00
JuanandClaude Opus 4.7 ce629139c2 docs: añade docs/componentes.md — referencia de cableado a bajo nivel
Complementa a docs/explicacion.md (alto nivel) con la vista de bajo nivel:
- grafo de dependencias de módulos por niveles topológicos (imports internos)
- el grafo de inyección de dependencias de api/deps.py (lru_cache + factories)
- las firmas exactas de los contratos: LLMProvider, GuardrailEngine, los
  validadores, el registry/policy store, el AgentOrchestrator, AgentState y el
  cableado del grafo LangGraph, y cómo orchestrator._build_execution mapea el
  StateSnapshot a AgentExecution
- la cadena de llamada de cada endpoint (qué deps, qué llama, qué persiste)
- mapa de persistencia (quién escribe/lee cada artefacto YAML/JSON/JSONL/SQLite)
- mapa CoreClient ↔ endpoints ↔ páginas del dashboard
- arranque/ciclo de vida y tabla Settings → consumidor
- "gotchas" conocidos
README y docs/explicacion.md enlazan al nuevo documento.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-11 15:20:00 +02:00
JuanandClaude Opus 4.7 57ba136c70 docs: añade docs/explicacion.md — guía didáctica del proyecto y sus módulos
Documento de alto nivel para terceros: las seis ideas clave, vista de los dos
servicios y las capas del core, recorrido módulo a módulo con sus dependencias
(quién depende de quién), el patrón Protocol+factory+Settings, el viaje completo
de una petición (con diagrama de secuencia), la estrategia de persistencia (YAML/
JSON/JSONL/SQLite), glosario, mapa del repo y ruta de lectura. README enlaza a él.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-11 15:01:01 +02:00
JuanandClaude Opus 4.7 9dd2a8f9bd docs(plan): registra el walkthrough manual del dashboard y los 3 fixes derivados
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-11 14:39:26 +02:00
JuanandClaude Opus 4.7 c860984ae4 fix(api): /executions incluye también las ejecuciones HITL pendientes
`GET /executions` solo leía `executions.jsonl`, donde nunca se escribe una ejecución
en `awaiting_approval` (esas viven solo en el checkpointer). La página de Aprobaciones
del dashboard hace `[e for e in list_executions() if e.status=="awaiting_approval"]`,
así que nunca encontraba aprobaciones pendientes. Ahora `list_executions` reconstruye
las no-terminales desde `execution_index.json` + `orchestrator.snapshot()` y las
fusiona con las del JSONL. Añade `test_list_executions_incluye_la_awaiting`.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-11 14:38:52 +02:00
JuanandClaude Opus 4.7 c414e16357 fix(policies): detect_pii/pii_leakage solo con recognizers de patrón fiables
Con Presidio real (imagen Docker) y `en_core_web_sm`, el recognizer de PERSON daba
falsos positivos sobre los textos de los escenarios (técnicos, en español, analizados
con el modelo `en`) → `01_sip` y `02_mos` salían `blocked_by_guardrail` en lugar de
`awaiting_approval`/`completed`. Se quitan `PERSON` y `PHONE_NUMBER` de
`detect_pii.entities` (input) y `PHONE_NUMBER` de `pii_leakage.entities` (output);
quedan los recognizers puramente de patrón (`EMAIL_ADDRESS`, `ES_NIF`, `IP_ADDRESS`,
`IBAN_CODE`), fiables. El bloqueo por NIF/email del demo sigue funcionando.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-11 14:38:52 +02:00
JuanandClaude Opus 4.7 eac2ed02a4 fix(guardrails): Presidio usa en_core_web_sm cacheado (en_core_web_lg no está en la imagen)
`detect_pii` instanciaba `AnalyzerEngine()` sin configuración, así que Presidio
intentaba cargar su modelo por defecto `en_core_web_lg` (~560 MB), que no está en
la imagen Docker — `core/Dockerfile` instala `en_core_web_sm`. Resultado: el
validador fallaba con `[E050] Can't find model 'en_core_web_lg'` y, al ser la
política fail-closed, *toda* invocación quedaba `blocked_by_guardrail`. Además se
reinstanciaba el engine en cada llamada (caro; probablemente la causa del crash del
proceso al segundo invoke). Ahora:
  - `_presidio_analyzer()`: singleton perezoso con `NlpEngineProvider` → `en_core_web_sm`.
  - si Presidio no está instalado o el modelo no carga, se hace fallback a la
    detección por regex (antes solo se hacía fallback ante ImportError).
Detectado ejecutando el smoke de docker-compose (el venv local no tiene Presidio,
así que los tests usan la rama de regex y no lo veían).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-11 14:38:52 +02:00
169 changed files with 5705 additions and 7918 deletions
+18 -3
View File
@@ -16,12 +16,27 @@ OPENAI_MODEL=gpt-4o
# Guardrails
GUARDRAILS_NEMO_ENABLED=false
# Keywords CSV para topical rails de NeMo (vacío = sin chequeo off-topic)
GUARDRAILS_NEMO_ALLOWED_KEYWORDS=
# Observabilidad
LOG_LEVEL=INFO
# Training orchestration (mock | azure_ml)
TRAINING_BACKEND=mock
# Azure ML (required when TRAINING_BACKEND=azure_ml and using live SDK; else stub mode)
AZURE_ML_SUBSCRIPTION_ID=
AZURE_ML_RESOURCE_GROUP=
AZURE_ML_WORKSPACE_NAME=
AZURE_ML_COMPUTE=cpu-cluster
AZURE_ML_ENVIRONMENT=AzureML-sklearn-1.5
AZURE_ML_EXPERIMENT_NAME=forja-training
AZURE_ML_COMMAND=python train.py
# Persistencia
DATA_DIR=./data
# URL del core (la usa el dashboard)
AGENTFORGE_CORE_URL=http://core:8000
AGENTS_DIR=./agents
POLICIES_DIR=./policies
DATASETS_DIR=./datasets
MODELS_DIR=./models
+3 -5
View File
@@ -4,11 +4,12 @@
*.key
*.pem
# Datos runtime (registry, checkpoints, logs)
# Datos runtime (registry, checkpoints, logs).
# Nota: sin patrón global *.jsonl — los records de datasets/ (p. ej.
# datasets/<name>/records/*.jsonl) son contenido versionado, no runtime.
data/
*.sqlite
*.sqlite-*
*.jsonl
# Python
__pycache__/
@@ -38,8 +39,5 @@ env/
.DS_Store
Thumbs.db
# Streamlit
.streamlit/secrets.toml
# Notas de hand-off entre sesiones de Claude Code
claude-session.txt
+30 -19
View File
@@ -1,26 +1,23 @@
# AgentForge — Arquitectura técnica
# Forja — Arquitectura técnica
> Documento "vivo" con las decisiones técnicas. El spec original está en
> [`docs/superpowers/specs/2026-05-09-agentforge-design.md`](docs/superpowers/specs/2026-05-09-agentforge-design.md).
> Documento vivo con las decisiones técnicas.
## Visión general
Two-tier:
- `agentforge-core` — FastAPI 8000. Dominio de gobierno, runtime LangGraph,
guardrails y persistencia.
- `agentforge-dashboard` — Streamlit 8501. Cliente HTTP del core.
Forja es un **único servicio FastAPI** (:8000):
- Todo el dominio de gobierno, runtime LangGraph, guardrails y persistencia.
- UI embebida (HTMX) para uso humano + API REST completa bajo `/api`.
## Diagrama
```
┌─────────────────────┐ HTTP/JSON ┌─────────────────────┐
Dashboard (Stream.) │ ───────────► │ Core (FastAPI)
└─────────────────────┘ └─────────┬───────────┘
┌─────────────────────────┬────────┴──────┬────────────┐
▼ ▼ ▼ ▼
LLM layer Guardrails layer LangGraph State
(Strategy pattern) (Strategy pattern) Runtime JSON+SQL+JSONL
┌────────────────────────────────────────────────────────────┐
forja-core :8000
│ UI HTMX (humanos) + REST API (/api) + dominio completo │
│ LLM (strategy) │ Guardrails (strategy) │ LangGraph │
│ │ │ + checkpoints│
└────────────────────────────────────────────────────────────┘
```
## Patrones aplicados
@@ -38,11 +35,24 @@ Two-tier:
| Capa | Tecnología | Por qué |
|---|---|---|
| Agent Registry | JSON + YAML | Humano lo escribe (YAML); máquina lo cataloga (index.yaml). |
| Versionado | YAML por versión + index.yaml | Diff legible; hash SHA-256 normalizado. |
| Registries (agents/policies/datasets/models) | YAML por versión + index.yaml | Humano lo escribe; diff legible; hash SHA-256 normalizado. Base común en `registry/yaml_store.py`. |
| Logs de violaciones | JSONL append-only | Inmutable, auditable, fácil de shipear. |
| Logs de ejecuciones | JSONL append-only | Idem. |
| Checkpoints LangGraph | SQLite | Tool right; persistencia entre restarts (HITL). |
| Training runs / evaluaciones / promociones | JSON por entidad bajo `data/` | Estado mutable consultable por id; los registros de promoción aprobada van además a JSONL append-only. |
## Ciclo MLOps (training → evaluación → promoción)
1. `POST /api/training/runs` envía un job al backend configurado
(`TRAINING_BACKEND=mock|azure_ml`) con dataset, modelo base y, opcionalmente,
el agente y su **versión candidata**.
2. `POST /api/training/runs/{id}/evaluate` ejecuta los escenarios canónicos
(`agents/<name>/examples/*.txt`) contra la versión candidata por el runtime
gobernado completo; pasa si no hay violaciones bloqueantes ni fallos.
3. `POST /api/promotions` crea la petición; exige run `succeeded`, evaluación
`passed` y que la evaluación cubra **exactamente** la versión a promocionar.
4. Un operador aprueba/rechaza (UI `/promotions`); al aprobar, la versión pasa a
`active` en el registry y queda un registro de auditoría append-only.
## HITL — Patrón
@@ -50,7 +60,7 @@ Se usa la API dinámica `interrupt(payload)` de LangGraph ≥0.2 dentro del
nodo `approve_gate`. Sólo pausa cuando hay acciones de alto riesgo
(`risk_score >= threshold` o `requires_approval=True`). El cliente envía
`Command(resume={"approved_action_ids": [...]})` vía
`POST /executions/{trace_id}/approve`.
`POST /api/executions/{trace_id}/approve`.
## Observabilidad
@@ -60,7 +70,8 @@ nodo `approve_gate`. Sólo pausa cuando hay acciones de alto riesgo
## Errores
- Azure OpenAI / OpenAI: retry exponencial 1s/2s/4s, fallback opcional, luego
- Azure OpenAI / OpenAI: retry exponencial 1s/2s/4s; si `LLM_FALLBACK_PROVIDER`
está configurado, `FallbackLLMProvider` delega en el secundario; luego
`status=failed`.
- Validador: `fail_closed` por defecto.
- Schema mismatch en LLM output: 1 retry, luego `failed`.
+4 -4
View File
@@ -4,16 +4,16 @@ help:
@echo "Targets: install lint format test test-all smoke up down logs clean"
install:
pip install -r core/requirements.txt -r dashboard/requirements.txt
pip install -r core/requirements.txt
pip install pytest pytest-asyncio ruff mypy freezegun
lint:
ruff check core/src dashboard/src tests
ruff check core/src tests
mypy core/src
format:
ruff format core/src dashboard/src tests
ruff check --fix core/src dashboard/src tests
ruff format core/src tests
ruff check --fix core/src tests
test:
pytest tests/unit -v
+75 -49
View File
@@ -1,34 +1,37 @@
# 🛡️ AgentForge
# 🔨 Forja
Plataforma profesional de **gobernanza de agentes IA**: catalogación, versionado
de prompts y políticas, guardrails runtime, ejecución stateful con
Human-in-the-Loop, observabilidad y trazabilidad end-to-end.
## ¿Por qué?
Poner agentes IA en producción sin una capa de gobierno produce sistemas
opacos: prompts que cambian sin historial, validaciones inconsistentes,
acciones de alto impacto sin supervisión, sin auditoría de decisiones.
AgentForge aporta el plano de control mínimo que un equipo de plataforma
necesita antes de operar agentes con impacto real.
**Forja de agentes IA** — plataforma de gobernanza genérica para agentes de cualquier tipo.
Catalogación y versionado (estilo Git) de definiciones, políticas y guardrails;
ejecución controlada con runtime stateful, Human-in-the-Loop y observabilidad completa.
Úsala como plano de control en cualquier proyecto que necesite agentes gobernados.
## Arquitectura
Forja es un **único servicio** (FastAPI en :8000) que expone:
- API REST completa bajo `/api` (agentes, políticas, ejecuciones, aprobaciones, violaciones, datasets, models, training runs, promotions).
- UI embebida de **una sola página** (HTMX + Tailwind + Jinja) que recorre el
ciclo de vida completo en seis etapas: **Define → Run → Approve → Train →
Promote → Audit**. Cada panel se refresca solo tras cada acción; las rutas
antiguas (`/agents`, `/run`, ...) redirigen a su etapa.
```
┌───────────────────────── docker-compose ──────────────────────────┐
│ ┌─────────────────────┐ HTTP/JSON ┌────────────────────┐ │
│ │ agentforge-dashboard│ ────────────────► │ agentforge-core │ │
│ Streamlit :8501 │ ◄──────────────── │ FastAPI :8000 │
└─────────────────────┘ └────────────────────┘
Strategy pattern (Protocol) para LLMProvider, GuardrailEngine,
│ AgentRegistry. Persistencia mixta: YAML (definiciones), JSON │
│ (registry), JSONL (logs append-only), SQLite (checkpoints). │
└────────────────────────────────────────────────────────────────────┘
┌───────────────────────────────────────────────────────────────┐
forja-core :8000
│ ┌─────────────────────────────────────────────────────────┐
│ │ UI HTMX (navegador humano) │ API REST (/api/*)
└─────────────────────────────────────────────────────────┘
Strategy factories (LLM / Guardrails / Registry)
LangGraph runtime + SQLite checkpoints (HITL real)
Persistencia: YAML versionado + JSONL + SQLite
└───────────────────────────────────────────────────────────────┘
```
Detalles completos en [`ARCHITECTURE.md`](ARCHITECTURE.md).
Documentación viva:
- [`ARCHITECTURE.md`](ARCHITECTURE.md) — decisiones técnicas.
- [`docs/explicacion.md`](docs/explicacion.md) — explicación didáctica completa.
- [`docs/componentes.md`](docs/componentes.md) — referencia de cableado a bajo nivel.
- [`docs/futuro.md`](docs/futuro.md) — roadmap.
## Quickstart
@@ -37,38 +40,55 @@ cp .env.example .env
docker compose up
```
Abre [http://localhost:8501](http://localhost:8501).
Abre [http://localhost:8000](http://localhost:8000).
Funciona out-of-the-box (`LLM_PROVIDER=mock`, sin API keys). Si quieres usar
Azure OpenAI real, edita `.env`.
## Demo guiada (3 pasos)
## Demo guiada: el ciclo completo en 6 clicks
1. **Registro** → ver `incident_analyzer` y comparar `v1` vs `v2`.
2. **Ejecutar** → seleccionar el escenario `01_sip_registration_drop` y
pulsar *Invocar*. Verás el grafo recorrer validate_input → llm_reason
→ validate_output → propose_actions → approve_gate y pausarse en HITL.
3. **Aprobaciones** → revisar las acciones propuestas (con risk_score y
rollback_plan), aprobar las seguras y comprobar que la ejecución
completa.
Todo ocurre en `http://localhost:8000`, de arriba abajo, sin escribir nada:
1. **▶ Run demo incident** (etapa 02) — lanza un incidente HSS pregrabado por el
pipeline gobernado. La acción propuesta tiene risk 5 → la ejecución se pausa
en el approval gate.
2. **Approve all** (etapa 03) — el grafo se reanuda desde su checkpoint y la
ejecución completa (aparece en la etapa 06 — Audit).
3. **Submit training run** (etapa 04) — el formulario viene pre-rellenado con el
dataset `incident_sft`, el modelo `gpt4o_lora_base` y la versión candidata
`v3` (draft). Con `TRAINING_BACKEND=mock` termina al instante.
4. **Run evaluation** (etapa 04) — ejecuta los escenarios canónicos del agente
(`agents/<name>/examples/`) sobre la candidata `v3`; pasa si no hay
violaciones bloqueantes.
5. **Request promotion** (etapa 04) — crea la petición de gobernanza, que
aparece en la etapa 05.
6. **Approve promotion** (etapa 05) — `v3` pasa a `active` en el registry;
compruébalo en los chips de versión de la etapa 01.
## Capacidades implementadas
| Feature | Ubicación |
|---|---|
| Agent Registry | `core/src/agentforge_core/registry/repository.py` |
| Versionado tipo Git | `agents/<name>/versions/` + `registry/versioning.py` |
| Guardrails runtime | `core/src/agentforge_core/guardrails/` |
| LangGraph stateful + checkpointing | `core/src/agentforge_core/runtime/` |
| Human-in-the-Loop | nodo `approve_gate` + endpoints `/approve` y `/reject` |
| Observabilidad | `observability/logging.py` (structlog + trace_id) |
| LLM provider abstraction | `core/src/agentforge_core/llm/` |
| Política versionada | `policies/default/versions/v1.yaml` |
| Agent / Policy / Dataset / Model Registry | `core/src/forja_core/registry/` (base genérica `yaml_store.py`) |
| Versionado Git-like (YAML) | `agents/`, `policies/`, `datasets/`, `models/` |
| Guardrails runtime (strategy) | `core/src/forja_core/guardrails/` |
| LangGraph + HITL + checkpoints | `core/src/forja_core/runtime/` |
| Observabilidad (trace + structlog) | `core/src/forja_core/observability/` |
| LLM abstraction (+fallback opcional) | `core/src/forja_core/llm/` |
| Training backends (mock / Azure ML) | `core/src/forja_core/training/` |
| Evaluación post-training (escenarios) | `core/src/forja_core/evaluation/` |
| Promoción gobernada draft → active | `core/src/forja_core/api/promotions.py` |
| UI embebida (HTMX) | `core/src/forja_core/web/` (ruta raíz) |
| API REST completa | bajo `/api` en forja-core :8000 |
## Variables de entorno
Ver [`.env.example`](.env.example).
### Azure ML training
Set `TRAINING_BACKEND=azure_ml` and fill `AZURE_ML_*` workspace settings. Forja submits an Azure ML **command job** via `azure-ai-ml` using `DefaultAzureCredential` (Azure CLI, managed identity, etc.). Replace the bundled script in `core/src/forja_core/training/_azure_job_bundle/` or override `AZURE_ML_COMMAND` for your fine-tuning pipeline. Without workspace config, the backend stays in **stub mode** for local development.
## Roadmap
Ver [`docs/futuro.md`](docs/futuro.md).
@@ -76,16 +96,22 @@ Ver [`docs/futuro.md`](docs/futuro.md).
## Estructura del repositorio
```
agentforge/
├── core/ servicio FastAPI
├── dashboard/ servicio Streamlit
├── agents/ definiciones declarativas (YAML)
├── policies/ políticas de guardrails
├── data/ estado runtime (gitignored)
├── tests/ pytest unit + integration
── docs/ documentación adicional
forja/
├── core/ servicio FastAPI único (API + UI HTMX embebida)
├── agents/ definiciones de agentes (YAML versionados)
├── policies/ políticas de guardrails (YAML versionados)
├── datasets/ datasets para fine-tuning (YAML versionados)
├── models/ modelos base / destino de entrenamiento (YAML versionados)
├── data/ runtime state (gitignored)
── tests/
└── docs/ (explicacion.md, componentes.md, futuro.md, ARCHITECTURE.md)
```
## Development
- Implementation guidelines: [`karpathy.md`](karpathy.md)
- UI copy and language: [`docs/product-voice.md`](docs/product-voice.md) (English only in the web app)
## Tests
```bash
+5
View File
@@ -10,4 +10,9 @@ versions:
author: Juan
message: Añade detección de codec mismatch y refuerzo de prompts.
created_at: 2026-05-01T12:00:00Z
- id: v3
hash: pending
author: Juan
message: Candidata SFT sobre incident_sft (LoRA); pendiente de evaluación y promoción.
created_at: 2026-06-10T10:00:00Z
active_version: v2
@@ -37,4 +37,6 @@ output_schema:
type: object
required: [id, action, target, risk_score, rollback_plan, requires_approval]
risk_threshold_for_hitl: 4
icon: "🤖"
color: "#64748b"
updated_at: 2026-04-12T10:00:00Z
@@ -35,4 +35,8 @@ output_schema:
type: object
required: [id, action, target, risk_score, rollback_plan, requires_approval]
risk_threshold_for_hitl: 4
input_label: Incident description
input_placeholder: "Sudden 80% drop in SIP registrations after deploying image 4.7.2 on the CSCF cluster."
icon: "🤖"
color: "#f59e0b"
updated_at: 2026-05-01T12:00:00Z
+43
View File
@@ -0,0 +1,43 @@
name: incident_analyzer
version: v3
owner: Juan
purpose: |
Candidata re-entrenada (SFT sobre incident_sft, LoRA) con la misma cobertura
que v2: codec mismatch, tormentas de registro post-handover, degradación de
MOS y saturación replicada en HSS active-active.
state: draft
guardrails: [default]
llm:
provider: mock
model: gpt-4o
temperature: 0.1
max_tokens: 2000
system_prompt: |
Eres un analista senior de operaciones de plataforma de voz virtualizada (IMS,
CSCF, SBC, HSS). Recibes una descripción de incidente. Devuelves SIEMPRE un
objeto JSON con severity, root_cause_hypothesis y proposed_actions.
Considera específicamente:
- Codec mismatch G.711/G.729 entre SBC peering y core IMS.
- Tormentas de registro post-handover con CSCF tras despliegues.
- Degradación de MOS en pools SBC durante picos.
- Saturación replicada en HSS active-active.
Prioriza hipótesis respaldadas por los ejemplos del dataset incident_sft.
Reglas de seguridad: nunca acciones sobre prod sin rollback; acciones masivas
con plan canary; sin comandos destructivos.
output_schema:
type: object
required: [severity, root_cause_hypothesis, proposed_actions]
properties:
severity: {type: string, enum: [low, medium, high, critical]}
root_cause_hypothesis: {type: string}
proposed_actions:
type: array
items:
type: object
required: [id, action, target, risk_score, rollback_plan, requires_approval]
risk_threshold_for_hitl: 4
input_label: Incident description
input_placeholder: "Sudden 80% drop in SIP registrations after deploying image 4.7.2 on the CSCF cluster."
icon: "🤖"
color: "#f59e0b"
updated_at: 2026-06-10T10:00:00Z
+3 -1
View File
@@ -30,7 +30,9 @@ USER agent
EXPOSE 8000
# Nota: Este servicio ahora sirve tanto la API REST como la UI HTMX (Opción A)
HEALTHCHECK --interval=10s --timeout=3s --start-period=15s --retries=3 \
CMD curl -fsS http://localhost:8000/health || exit 1
CMD ["uvicorn", "agentforge_core.main:app", "--host", "0.0.0.0", "--port", "8000"]
CMD ["uvicorn", "forja_core.main:app", "--host", "0.0.0.0", "--port", "8000"]
+8 -4
View File
@@ -9,14 +9,18 @@ langgraph-checkpoint-sqlite>=2.0,<3.0
# AsyncSqliteSaver (langgraph-checkpoint-sqlite 2.x) usa Connection.is_alive(),
# eliminado en aiosqlite 0.21. Fijado a la última serie compatible.
aiosqlite>=0.20,<0.21
guardrails-ai>=0.5,<0.6
# typer 0.12.x (pin transitivo de guardrails-ai <0.6) rompe con click >= 8.2
# ("Secondary flag is not valid for non-boolean flag" al ejecutar `spacy download`
# y la CLI de typer/weasel). Fijado a la última serie 8.1.
click>=8.1,<8.2
presidio-analyzer>=2.2,<3.0
presidio-anonymizer>=2.2,<3.0
openai>=1.50,<2.0
azure-ai-ml>=1.16,<2.0
azure-identity>=1.16,<2.0
PyYAML>=6.0,<7.0
jsonschema>=4.0,<5.0
nemoguardrails>=0.10,<0.12
# nemoguardrails se reintroducirá cuando se integre Colang real (docs/futuro.md);
# el engine actual es un stub por keywords que no importa el paquete.
# HTMX UI embebida (Opción A)
jinja2>=3.1,<4.0
python-multipart>=0.0.9,<0.1
-23
View File
@@ -1,23 +0,0 @@
"""Router /policies: listado de políticas y de sus versiones."""
from __future__ import annotations
from fastapi import APIRouter, HTTPException
from agentforge_core.api.deps import PolicyStoreDep
from agentforge_core.domain.policy import PolicyDefinition, PolicyVersionMeta
router = APIRouter()
@router.get("", response_model=list[PolicyDefinition])
def list_policies(store: PolicyStoreDep) -> list[PolicyDefinition]:
return store.list_policies()
@router.get("/{name}/versions", response_model=list[PolicyVersionMeta])
def list_versions(name: str, store: PolicyStoreDep) -> list[PolicyVersionMeta]:
try:
return store.list_versions(name)
except FileNotFoundError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@@ -1,33 +0,0 @@
"""Factory que ensambla el GuardrailEngine según settings."""
from __future__ import annotations
from agentforge_core.config import Settings
from agentforge_core.guardrails.base import GuardrailEngine
from agentforge_core.guardrails.composite import CompositeGuardrailEngine
from agentforge_core.guardrails.guardrails_ai import GuardrailsAIEngine
from agentforge_core.guardrails.nemo import NeMoGuardrailsEngine
def build_guardrail_engine(settings: Settings) -> GuardrailEngine:
"""Ensambla el engine: GuardrailsAIEngine siempre; NeMo si está habilitado."""
engines: list[GuardrailEngine] = [GuardrailsAIEngine()]
if settings.guardrails_nemo_enabled:
engines.append(
NeMoGuardrailsEngine(
allowed_keywords=[
"sip",
"ims",
"cscf",
"sbc",
"mos",
"hss",
"registro",
"incidente",
"voz",
"codec",
"rollback",
],
)
)
return CompositeGuardrailEngine(engines)
-28
View File
@@ -1,28 +0,0 @@
"""Factory que selecciona el LLMProvider según settings."""
from __future__ import annotations
from agentforge_core.config import Settings
from agentforge_core.llm.azure import AzureOpenAIProvider
from agentforge_core.llm.base import LLMProvider
from agentforge_core.llm.mock import MockProvider
from agentforge_core.llm.openai import OpenAIProvider
def build_llm_provider(settings: Settings) -> LLMProvider:
"""Materializa el proveedor activo. Falla en arranque si la config es inconsistente."""
match settings.llm_provider:
case "mock":
return MockProvider()
case "azure":
return AzureOpenAIProvider(
endpoint=settings.azure_openai_endpoint,
api_key=settings.azure_openai_api_key,
deployment=settings.azure_openai_deployment,
api_version=settings.azure_openai_api_version,
)
case "openai":
return OpenAIProvider(
api_key=settings.openai_api_key,
model=settings.openai_model,
)
-37
View File
@@ -1,37 +0,0 @@
"""Composición raíz de la aplicación FastAPI (agentforge-core)."""
from __future__ import annotations
from fastapi import FastAPI
from agentforge_core.api.middlewares import TraceIdMiddleware
from agentforge_core.config import Settings
from agentforge_core.observability.logging import configure_logging
def create_app() -> FastAPI:
"""Construye la app: logging, middleware de trace_id, /health y routers."""
settings = Settings()
configure_logging(level=settings.log_level)
app = FastAPI(
title="AgentForge Core",
version="0.1.0",
description="Plataforma de gobernanza de agentes IA — API REST.",
)
app.add_middleware(TraceIdMiddleware)
@app.get("/health", tags=["meta"])
async def health() -> dict[str, str]:
return {"status": "ok"}
from agentforge_core.api import agents, executions, policies, violations
app.include_router(agents.router, prefix="/agents", tags=["agents"])
app.include_router(executions.invoke_router, prefix="/agents", tags=["agents"])
app.include_router(executions.router, prefix="/executions", tags=["executions"])
app.include_router(policies.router, prefix="/policies", tags=["policies"])
app.include_router(violations.router, prefix="/violations", tags=["violations"])
return app
app = create_app()
@@ -1,17 +0,0 @@
"""Factories para AgentRegistry y PolicyStore basados en Settings."""
from __future__ import annotations
from agentforge_core.config import Settings
from agentforge_core.registry.policy_store import FileSystemPolicyStore
from agentforge_core.registry.repository import FileSystemAgentRegistry
def build_agent_registry(settings: Settings) -> FileSystemAgentRegistry:
"""Construye el registry de agentes apuntando a ``settings.agents_dir``."""
return FileSystemAgentRegistry(settings.agents_dir)
def build_policy_store(settings: Settings) -> FileSystemPolicyStore:
"""Construye el store de políticas apuntando a ``settings.policies_dir``."""
return FileSystemPolicyStore(settings.policies_dir)
@@ -1,64 +0,0 @@
"""PolicyStore basado en sistema de ficheros (YAML por versión + index.yaml)."""
from __future__ import annotations
from datetime import datetime
from pathlib import Path
from typing import Any
import yaml
from agentforge_core.domain.policy import PolicyDefinition, PolicyVersionMeta
class FileSystemPolicyStore:
"""Lee políticas de policies/<name>/{index.yaml,versions/vN.yaml}."""
def __init__(self, root: Path) -> None:
self._root = Path(root)
def list_policies(self) -> list[PolicyDefinition]:
if not self._root.exists():
return []
result: list[PolicyDefinition] = []
for d in sorted(self._root.iterdir()):
if d.is_dir() and (d / "index.yaml").exists():
result.append(self.get_policy(d.name))
return result
def get_policy(self, name: str, version: str | None = None) -> PolicyDefinition:
index = self._load_index(name)
v = version or index["active_version"]
path = self._root / name / "versions" / f"{v}.yaml"
if not path.exists():
raise FileNotFoundError(f"policy version not found: {name}@{v}")
with path.open(encoding="utf-8") as f:
data = yaml.safe_load(f)
return PolicyDefinition.model_validate(data)
def list_versions(self, name: str) -> list[PolicyVersionMeta]:
index = self._load_index(name)
return [
PolicyVersionMeta(
id=v["id"],
hash=v["hash"],
author=v["author"],
message=v["message"],
created_at=self._parse_dt(v["created_at"]),
)
for v in index["versions"]
]
def _load_index(self, name: str) -> dict[str, Any]:
index_path = self._root / name / "index.yaml"
if not index_path.exists():
raise FileNotFoundError(f"policy not found: {name}")
with index_path.open(encoding="utf-8") as f:
data: dict[str, Any] = yaml.safe_load(f)
return data
@staticmethod
def _parse_dt(value: str | datetime) -> datetime:
if isinstance(value, datetime):
return value
return datetime.fromisoformat(value.replace("Z", "+00:00"))
@@ -2,15 +2,22 @@
from __future__ import annotations
from fastapi import APIRouter, HTTPException
from fastapi import APIRouter, HTTPException, status
from pydantic import BaseModel
from agentforge_core.api.deps import RegistryDep
from agentforge_core.domain.agent import AgentDefinition, AgentVersionMeta
from agentforge_core.registry.versioning import DiffResult
from forja_core.api.deps import RegistryDep
from forja_core.domain.agent import AgentDefinition, AgentVersionMeta
from forja_core.registry.versioning import DiffResult
router = APIRouter()
class CreateAgentVersionRequest(BaseModel):
agent: AgentDefinition
message: str
author: str = "dashboard"
@router.get("", response_model=list[AgentDefinition])
def list_agents(registry: RegistryDep) -> list[AgentDefinition]:
return registry.list_agents()
@@ -43,3 +50,15 @@ def get_version(name: str, version: str, registry: RegistryDep) -> AgentDefiniti
@router.get("/{name}/versions/{v_from}/diff/{v_to}", response_model=DiffResult)
def diff_versions(name: str, v_from: str, v_to: str, registry: RegistryDep) -> DiffResult:
return registry.diff_versions(name, v_from, v_to)
@router.post(
"/{name}/versions", response_model=AgentVersionMeta, status_code=status.HTTP_201_CREATED
)
def create_agent_version(
name: str, req: CreateAgentVersionRequest, registry: RegistryDep
) -> AgentVersionMeta:
try:
return registry.upsert_version(name, req.agent, req.message, req.author)
except Exception as exc:
raise HTTPException(status_code=500, detail=str(exc)) from exc
+39
View File
@@ -0,0 +1,39 @@
"""Dataset registry API (read-only)."""
from __future__ import annotations
from fastapi import APIRouter, HTTPException
from forja_core.api.deps import DatasetStoreDep
from forja_core.domain.dataset import DatasetDefinition, DatasetVersionMeta
router = APIRouter()
@router.get("", response_model=list[DatasetDefinition])
def list_datasets(store: DatasetStoreDep) -> list[DatasetDefinition]:
return store.list_datasets()
@router.get("/{name}", response_model=DatasetDefinition)
def get_dataset(name: str, store: DatasetStoreDep) -> DatasetDefinition:
try:
return store.get_dataset(name)
except FileNotFoundError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@router.get("/{name}/versions", response_model=list[DatasetVersionMeta])
def list_versions(name: str, store: DatasetStoreDep) -> list[DatasetVersionMeta]:
try:
return store.list_versions(name)
except FileNotFoundError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@router.get("/{name}/versions/{version}", response_model=DatasetDefinition)
def get_version(name: str, version: str, store: DatasetStoreDep) -> DatasetDefinition:
try:
return store.get_version(name, version)
except FileNotFoundError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@@ -12,15 +12,24 @@ from typing import Annotated
from fastapi import Depends
from agentforge_core.config import Settings
from agentforge_core.guardrails.base import GuardrailEngine
from agentforge_core.guardrails.factory import build_guardrail_engine
from agentforge_core.llm.base import LLMProvider
from agentforge_core.llm.factory import build_llm_provider
from agentforge_core.registry.factory import build_agent_registry, build_policy_store
from agentforge_core.registry.policy_store import FileSystemPolicyStore
from agentforge_core.registry.repository import FileSystemAgentRegistry
from agentforge_core.runtime.orchestrator import AgentOrchestrator
from forja_core.config import Settings
from forja_core.guardrails.base import GuardrailEngine
from forja_core.guardrails.factory import build_guardrail_engine
from forja_core.llm.base import LLMProvider
from forja_core.llm.factory import build_llm_provider
from forja_core.registry.dataset_store import FileSystemDatasetStore
from forja_core.registry.factory import (
build_agent_registry,
build_dataset_store,
build_model_store,
build_policy_store,
)
from forja_core.registry.model_store import FileSystemModelStore
from forja_core.registry.policy_store import FileSystemPolicyStore
from forja_core.registry.repository import FileSystemAgentRegistry
from forja_core.runtime.orchestrator import AgentOrchestrator
from forja_core.training.base import TrainingBackend
from forja_core.training.factory import build_training_backend
@lru_cache(maxsize=1)
@@ -48,6 +57,21 @@ def get_guardrail_engine() -> GuardrailEngine:
return build_guardrail_engine(get_settings())
@lru_cache(maxsize=1)
def get_dataset_store() -> FileSystemDatasetStore:
return build_dataset_store(get_settings())
@lru_cache(maxsize=1)
def get_model_store() -> FileSystemModelStore:
return build_model_store(get_settings())
@lru_cache(maxsize=1)
def get_training_backend() -> TrainingBackend:
return build_training_backend(get_settings())
@lru_cache(maxsize=1)
def get_orchestrator() -> AgentOrchestrator:
settings = get_settings()
@@ -63,3 +87,6 @@ SettingsDep = Annotated[Settings, Depends(get_settings)]
RegistryDep = Annotated[FileSystemAgentRegistry, Depends(get_registry)]
PolicyStoreDep = Annotated[FileSystemPolicyStore, Depends(get_policy_store)]
OrchestratorDep = Annotated[AgentOrchestrator, Depends(get_orchestrator)]
DatasetStoreDep = Annotated[FileSystemDatasetStore, Depends(get_dataset_store)]
ModelStoreDep = Annotated[FileSystemModelStore, Depends(get_model_store)]
TrainingBackendDep = Annotated[TrainingBackend, Depends(get_training_backend)]
@@ -0,0 +1,47 @@
"""Persist evaluation reports under data/evaluations/."""
from __future__ import annotations
import json
from pathlib import Path
from uuid import UUID
from forja_core.domain.training import EvaluationReport
def _eval_dir(data_dir: Path) -> Path:
return data_dir / "evaluations"
def save_evaluation(data_dir: Path, report: EvaluationReport) -> None:
directory = _eval_dir(data_dir)
directory.mkdir(parents=True, exist_ok=True)
path = directory / f"{report.id}.json"
path.write_text(
json.dumps(report.model_dump(mode="json"), indent=2, default=str),
encoding="utf-8",
)
index_path = directory / "by_training_run.json"
index: dict[str, str] = {}
if index_path.exists():
index = json.loads(index_path.read_text(encoding="utf-8"))
index[str(report.training_run_id)] = str(report.id)
index_path.write_text(json.dumps(index, indent=2), encoding="utf-8")
def load_evaluation(data_dir: Path, report_id: UUID) -> EvaluationReport | None:
path = _eval_dir(data_dir) / f"{report_id}.json"
if not path.exists():
return None
return EvaluationReport.model_validate(json.loads(path.read_text(encoding="utf-8")))
def load_evaluation_for_run(data_dir: Path, training_run_id: UUID) -> EvaluationReport | None:
index_path = _eval_dir(data_dir) / "by_training_run.json"
if not index_path.exists():
return None
index = json.loads(index_path.read_text(encoding="utf-8"))
report_id = index.get(str(training_run_id))
if report_id is None:
return None
return load_evaluation(data_dir, UUID(report_id))
@@ -9,23 +9,23 @@ from uuid import UUID
from fastapi import APIRouter, HTTPException
from pydantic import BaseModel, Field
from agentforge_core.api.deps import (
from forja_core.api.deps import (
OrchestratorDep,
PolicyStoreDep,
RegistryDep,
SettingsDep,
)
from agentforge_core.api.persistence import (
from forja_core.api.persistence import (
append_execution,
append_violation,
read_execution_summaries,
)
from agentforge_core.domain.agent import AgentDefinition
from agentforge_core.domain.execution import AgentExecution, AgentExecutionSummary
from agentforge_core.domain.policy import PolicyDefinition
from agentforge_core.registry.policy_store import FileSystemPolicyStore
from agentforge_core.registry.repository import FileSystemAgentRegistry
from agentforge_core.runtime.orchestrator import AgentOrchestrator
from forja_core.domain.agent import AgentDefinition
from forja_core.domain.execution import AgentExecution, AgentExecutionSummary
from forja_core.domain.policy import PolicyDefinition
from forja_core.registry.policy_store import FileSystemPolicyStore
from forja_core.registry.repository import FileSystemAgentRegistry
from forja_core.runtime.orchestrator import AgentOrchestrator
router = APIRouter()
# POST /agents/{name}/invoke vive aquí pero se monta en main bajo el prefijo /agents.
@@ -133,12 +133,8 @@ async def invoke_agent(
except FileNotFoundError as exc:
raise HTTPException(status_code=422, detail=str(exc)) from exc
execution = await orchestrator.invoke(
agent_def=agent_def, policy=policy, user_input=body.input
)
_record_execution(
settings.data_dir, str(execution.trace_id), agent_def.name, agent_def.version
)
execution = await orchestrator.invoke(agent_def=agent_def, policy=policy, user_input=body.input)
_record_execution(settings.data_dir, str(execution.trace_id), agent_def.name, agent_def.version)
if execution.status in _TERMINAL_STATUSES:
append_execution(settings.data_dir, execution)
for violation in execution.violations:
@@ -146,9 +142,57 @@ async def invoke_agent(
return execution
async def collect_execution_summaries(
registry: FileSystemAgentRegistry,
policies: FileSystemPolicyStore,
orchestrator: AgentOrchestrator,
data_dir: Path,
) -> list[AgentExecutionSummary]:
"""Resumen de ejecuciones: las terminales del JSONL más las pausadas en HITL.
Una ejecución en ``awaiting_approval`` no se escribe en el log append-only
(`executions.jsonl`); solo vive en el checkpointer. Aquí se reconstruyen esas
desde el índice `execution_index.json` + el checkpointer. Lo usan tanto la API
como la página de Approvals de la UI.
"""
summaries = read_execution_summaries(data_dir)
seen = {str(s.trace_id) for s in summaries}
for trace_id, meta in _load_index(data_dir).items():
if trace_id in seen:
continue
try:
agent_def = registry.get_version(meta["agent_name"], meta["version"])
policy = policies.get_policy(agent_def.guardrails[0])
execution = await orchestrator.snapshot(
agent_def=agent_def, policy=policy, trace_id=UUID(trace_id)
)
except (FileNotFoundError, IndexError, KeyError, ValueError):
continue
if execution is None:
continue
summaries.append(
AgentExecutionSummary(
trace_id=execution.trace_id,
agent_name=execution.agent_name,
agent_version=execution.agent_version,
status=execution.status,
started_at=execution.started_at,
finished_at=execution.finished_at,
n_violations=len(execution.violations),
n_proposed_actions=len(execution.proposed_actions),
)
)
return summaries
@router.get("", response_model=list[AgentExecutionSummary])
def list_executions(settings: SettingsDep) -> list[AgentExecutionSummary]:
return read_execution_summaries(settings.data_dir)
async def list_executions(
registry: RegistryDep,
policies: PolicyStoreDep,
orchestrator: OrchestratorDep,
settings: SettingsDep,
) -> list[AgentExecutionSummary]:
return await collect_execution_summaries(registry, policies, orchestrator, settings.data_dir)
@router.get("/{trace_id}", response_model=AgentExecution)
@@ -177,12 +221,19 @@ async def approve_execution(
) -> AgentExecution:
agent_def, policy = _resolve(registry, policies, settings.data_dir, trace_id)
await _ensure_awaiting(orchestrator, agent_def, policy, trace_id)
# Empty approved_action_ids from UI "Aprobar todo" → approve everything that was waiting
approved = body.approved_action_ids
if not approved:
# Try to resolve the actual ids from current snapshot (best effort)
snap = await orchestrator.snapshot(agent_def=agent_def, policy=policy, trace_id=trace_id)
if snap and snap.needs_human_for:
approved = [a.id for a in snap.needs_human_for]
execution = await orchestrator.resume(
agent_def=agent_def,
policy=policy,
trace_id=trace_id,
decision={
"approved_action_ids": body.approved_action_ids,
"approved_action_ids": approved,
"comment": body.comment,
"rejected": False,
},
@@ -8,15 +8,13 @@ from starlette.middleware.base import BaseHTTPMiddleware, RequestResponseEndpoin
from starlette.requests import Request
from starlette.responses import Response
from agentforge_core.observability.logging import bind_trace_id, clear_trace_id
from forja_core.observability.logging import bind_trace_id, clear_trace_id
class TraceIdMiddleware(BaseHTTPMiddleware):
"""Lee o genera `X-Trace-Id`, lo bind-ea al contexto de structlog y lo devuelve en la respuesta."""
async def dispatch(
self, request: Request, call_next: RequestResponseEndpoint
) -> Response:
async def dispatch(self, request: Request, call_next: RequestResponseEndpoint) -> Response:
trace_id = request.headers.get("X-Trace-Id") or str(uuid4())
bind_trace_id(trace_id)
try:
+39
View File
@@ -0,0 +1,39 @@
"""Foundation model registry API (read-only)."""
from __future__ import annotations
from fastapi import APIRouter, HTTPException
from forja_core.api.deps import ModelStoreDep
from forja_core.domain.foundation_model import ModelDefinition, ModelVersionMeta
router = APIRouter()
@router.get("", response_model=list[ModelDefinition])
def list_models(store: ModelStoreDep) -> list[ModelDefinition]:
return store.list_models()
@router.get("/{name}", response_model=ModelDefinition)
def get_model(name: str, store: ModelStoreDep) -> ModelDefinition:
try:
return store.get_model(name)
except FileNotFoundError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@router.get("/{name}/versions", response_model=list[ModelVersionMeta])
def list_versions(name: str, store: ModelStoreDep) -> list[ModelVersionMeta]:
try:
return store.list_versions(name)
except FileNotFoundError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@router.get("/{name}/versions/{version}", response_model=ModelDefinition)
def get_version(name: str, version: str, store: ModelStoreDep) -> ModelDefinition:
try:
return store.get_version(name, version)
except FileNotFoundError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@@ -8,8 +8,8 @@ from __future__ import annotations
from pathlib import Path
from agentforge_core.domain.execution import AgentExecution, AgentExecutionSummary
from agentforge_core.domain.guardrail import GuardrailViolation
from forja_core.domain.execution import AgentExecution, AgentExecutionSummary
from forja_core.domain.guardrail import GuardrailViolation
def append_violation(data_dir: Path, violation: GuardrailViolation) -> None:
+55
View File
@@ -0,0 +1,55 @@
"""Router /policies: listado de políticas y de sus versiones."""
from __future__ import annotations
from fastapi import APIRouter, HTTPException, status
from pydantic import BaseModel
from forja_core.api.deps import PolicyStoreDep
from forja_core.domain.policy import PolicyDefinition, PolicyVersionMeta
router = APIRouter()
class CreatePolicyVersionRequest(BaseModel):
policy: PolicyDefinition
message: str
author: str = "dashboard"
@router.get("", response_model=list[PolicyDefinition])
def list_policies(store: PolicyStoreDep) -> list[PolicyDefinition]:
return store.list_policies()
@router.get("/{name}/versions", response_model=list[PolicyVersionMeta])
def list_versions(name: str, store: PolicyStoreDep) -> list[PolicyVersionMeta]:
try:
return store.list_versions(name)
except FileNotFoundError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@router.post(
"/{name}/versions", response_model=PolicyVersionMeta, status_code=status.HTTP_201_CREATED
)
def create_policy_version(
name: str, req: CreatePolicyVersionRequest, store: PolicyStoreDep
) -> PolicyVersionMeta:
try:
return store.upsert_version(name, req.policy, req.message, req.author)
except Exception as exc:
raise HTTPException(status_code=500, detail=str(exc)) from exc
@router.get("/validators")
def list_validators() -> dict[str, list[str]]:
return {
"input": ["detect_pii", "prompt_injection", "toxic_language", "forbidden_topics"],
"output": [
"schema_match",
"pii_leakage",
"forbidden_action_keywords",
"telco_safety_rules",
],
}
@@ -0,0 +1,51 @@
"""Persist promotion requests under data/promotions/."""
from __future__ import annotations
import json
from pathlib import Path
from uuid import UUID
from forja_core.domain.training import PromotionRecord, PromotionRequest, PromotionStatus
def _promo_dir(data_dir: Path) -> Path:
return data_dir / "promotions"
def save_promotion_request(data_dir: Path, request: PromotionRequest) -> None:
directory = _promo_dir(data_dir)
directory.mkdir(parents=True, exist_ok=True)
path = directory / f"{request.id}.json"
path.write_text(
json.dumps(request.model_dump(mode="json"), indent=2, default=str),
encoding="utf-8",
)
def load_promotion_request(data_dir: Path, request_id: UUID) -> PromotionRequest | None:
path = _promo_dir(data_dir) / f"{request_id}.json"
if not path.exists():
return None
return PromotionRequest.model_validate(json.loads(path.read_text(encoding="utf-8")))
def list_promotion_requests(
data_dir: Path,
status: PromotionStatus | None = None,
) -> list[PromotionRequest]:
directory = _promo_dir(data_dir)
if not directory.exists():
return []
out: list[PromotionRequest] = []
for path in sorted(directory.glob("*.json"), reverse=True):
req = PromotionRequest.model_validate(json.loads(path.read_text(encoding="utf-8")))
if status is None or req.status == status:
out.append(req)
return out
def append_promotion_record(data_dir: Path, record: PromotionRecord) -> None:
data_dir.mkdir(parents=True, exist_ok=True)
with (data_dir / "promotion_records.jsonl").open("a", encoding="utf-8") as f:
f.write(record.model_dump_json() + "\n")
+202
View File
@@ -0,0 +1,202 @@
"""Governance promotions: draft agent versions to active after eval gate."""
from __future__ import annotations
from datetime import UTC, datetime
from typing import Annotated
from uuid import UUID, uuid4
from fastapi import APIRouter, HTTPException, Query, status
from pydantic import BaseModel
from forja_core.api.deps import RegistryDep, SettingsDep
from forja_core.api.evaluation_persistence import load_evaluation
from forja_core.api.promotion_persistence import (
append_promotion_record,
list_promotion_requests,
load_promotion_request,
save_promotion_request,
)
from forja_core.api.training_persistence import load_training_run
from forja_core.config import Settings
from forja_core.domain.training import (
PromotionRecord,
PromotionRequest,
PromotionStatus,
TrainingRun,
)
router = APIRouter()
class CreatePromotionRequest(BaseModel):
agent_name: str
agent_version: str
training_run_id: UUID
evaluation_report_id: UUID
requested_by: str = "api"
class PromotionDecision(BaseModel):
approved_by: str = "api"
comment: str | None = None
class RejectPromotion(BaseModel):
rejected_by: str = "api"
reason: str | None = None
def _require_succeeded_run(run_id: UUID, settings: Settings) -> TrainingRun:
run = load_training_run(settings.data_dir, run_id)
if run is None:
raise HTTPException(status_code=404, detail="training run not found")
if run.status != "succeeded":
raise HTTPException(
status_code=409,
detail=f"training run status '{run.status}' must be 'succeeded' for this operation",
)
return run
@router.get("", response_model=list[PromotionRequest])
def list_promotions(
settings: SettingsDep,
status: Annotated[PromotionStatus | None, Query()] = None,
) -> list[PromotionRequest]:
return list_promotion_requests(settings.data_dir, status=status)
@router.get("/{request_id}", response_model=PromotionRequest)
def get_promotion(request_id: UUID, settings: SettingsDep) -> PromotionRequest:
req = load_promotion_request(settings.data_dir, request_id)
if req is None:
raise HTTPException(status_code=404, detail="promotion request not found")
return req
@router.post("", response_model=PromotionRequest, status_code=status.HTTP_201_CREATED)
def request_promotion(
body: CreatePromotionRequest,
settings: SettingsDep,
registry: RegistryDep,
) -> PromotionRequest:
_require_succeeded_run(body.training_run_id, settings)
report = load_evaluation(settings.data_dir, body.evaluation_report_id)
if report is None:
raise HTTPException(status_code=404, detail="evaluation report not found")
if report.training_run_id != body.training_run_id:
raise HTTPException(status_code=422, detail="evaluation report does not match training run")
if not report.passed:
raise HTTPException(
status_code=422,
detail="evaluation did not pass; promotion blocked by guardrail gate",
)
try:
agent = registry.get_version(body.agent_name, body.agent_version)
except FileNotFoundError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
if agent.state == "active":
raise HTTPException(status_code=409, detail="agent version is already active")
if agent.name != report.agent_name:
raise HTTPException(
status_code=422, detail="evaluation agent does not match promotion target"
)
if report.agent_version != body.agent_version:
raise HTTPException(
status_code=422,
detail=(
f"evaluation covered version '{report.agent_version}' but promotion "
f"targets '{body.agent_version}'; evaluate the candidate version first"
),
)
now = datetime.now(UTC)
req = PromotionRequest(
id=uuid4(),
agent_name=body.agent_name,
agent_version=body.agent_version,
training_run_id=body.training_run_id,
evaluation_report_id=body.evaluation_report_id,
status="pending_approval",
requested_by=body.requested_by,
created_at=now,
updated_at=now,
)
save_promotion_request(settings.data_dir, req)
return req
@router.post("/{request_id}/approve", response_model=PromotionRecord)
def approve_promotion(
request_id: UUID,
body: PromotionDecision,
settings: SettingsDep,
registry: RegistryDep,
) -> PromotionRecord:
req = load_promotion_request(settings.data_dir, request_id)
if req is None:
raise HTTPException(status_code=404, detail="promotion request not found")
if req.status != "pending_approval":
raise HTTPException(status_code=409, detail=f"promotion status is '{req.status}'")
report = load_evaluation(settings.data_dir, req.evaluation_report_id)
if report is None or not report.passed:
raise HTTPException(status_code=422, detail="evaluation gate no longer satisfied")
registry.promote_to_active(
req.agent_name,
req.agent_version,
author=body.approved_by,
comment=body.comment or "Approved via governance promotion",
)
now = datetime.now(UTC)
updated_req = req.model_copy(
update={
"status": "approved",
"decided_by": body.approved_by,
"comment": body.comment,
"updated_at": now,
},
)
save_promotion_request(settings.data_dir, updated_req)
record = PromotionRecord(
id=uuid4(),
agent_name=req.agent_name,
agent_version=req.agent_version,
training_run_id=req.training_run_id,
evaluation_report_id=req.evaluation_report_id,
approved_by=body.approved_by,
comment=body.comment,
created_at=now,
)
append_promotion_record(settings.data_dir, record)
return record
@router.post("/{request_id}/reject", response_model=PromotionRequest)
def reject_promotion(
request_id: UUID,
body: RejectPromotion,
settings: SettingsDep,
) -> PromotionRequest:
req = load_promotion_request(settings.data_dir, request_id)
if req is None:
raise HTTPException(status_code=404, detail="promotion request not found")
if req.status != "pending_approval":
raise HTTPException(status_code=409, detail=f"promotion status is '{req.status}'")
updated = req.model_copy(
update={
"status": "rejected",
"decided_by": body.rejected_by,
"comment": body.reason,
"updated_at": datetime.now(UTC),
},
)
save_promotion_request(settings.data_dir, updated)
return updated
+237
View File
@@ -0,0 +1,237 @@
"""Training runs API — submit jobs to external MLOps backends."""
from __future__ import annotations
from datetime import UTC, datetime
from uuid import UUID, uuid4
from fastapi import APIRouter, HTTPException, status
from pydantic import BaseModel, field_validator
from forja_core.api.deps import (
DatasetStoreDep,
ModelStoreDep,
OrchestratorDep,
PolicyStoreDep,
RegistryDep,
SettingsDep,
TrainingBackendDep,
)
from forja_core.api.evaluation_persistence import load_evaluation_for_run, save_evaluation
from forja_core.api.training_persistence import (
list_training_runs,
load_training_run,
save_training_run,
)
from forja_core.domain.training import (
EvaluationReport,
TrainingHyperparameters,
TrainingJobSpec,
TrainingRun,
)
from forja_core.evaluation.post_train import run_post_train_evaluation
from forja_core.registry.dataset_store import FileSystemDatasetStore
from forja_core.registry.model_store import FileSystemModelStore
from forja_core.registry.repository import FileSystemAgentRegistry
from forja_core.training.base import TrainingBackend
router = APIRouter()
class CreateTrainingRunRequest(BaseModel):
dataset_name: str
dataset_version: str | None = None
model_name: str
model_version: str | None = None
agent_name: str | None = None
agent_version: str | None = None
hyperparameters: TrainingHyperparameters | None = None
@field_validator(
"dataset_version", "model_version", "agent_name", "agent_version", mode="before"
)
@classmethod
def _empty_string_is_none(cls, value: object) -> object:
# HTML form selects submit "" for the unselected option.
return value or None
def _resolve_spec(
body: CreateTrainingRunRequest,
datasets: FileSystemDatasetStore,
models: FileSystemModelStore,
registry: FileSystemAgentRegistry,
) -> TrainingJobSpec:
try:
dataset = datasets.get_dataset(body.dataset_name, body.dataset_version)
except FileNotFoundError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
try:
model = models.get_model(body.model_name, body.model_version)
except FileNotFoundError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
if body.agent_version and not body.agent_name:
raise HTTPException(status_code=422, detail="agent_version requires agent_name")
if body.agent_name:
try:
registry.get_agent(body.agent_name, body.agent_version)
except FileNotFoundError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
return TrainingJobSpec(
dataset_name=dataset.name,
dataset_version=dataset.version,
model_name=model.name,
model_version=model.version,
agent_name=body.agent_name,
agent_version=body.agent_version,
hyperparameters=body.hyperparameters or TrainingHyperparameters(),
)
async def _apply_backend_status(
backend: TrainingBackend,
run: TrainingRun,
) -> TrainingRun:
if run.external_job_id is None:
return run
job = await backend.status(run.external_job_id)
now = datetime.now(UTC)
return run.model_copy(
update={
"status": job.status,
"updated_at": now,
"artifact_refs": job.artifacts,
"error_message": job.message if job.status == "failed" else None,
},
)
@router.get("", response_model=list[TrainingRun])
def list_runs(settings: SettingsDep) -> list[TrainingRun]:
return list_training_runs(settings.data_dir)
@router.get("/{run_id}", response_model=TrainingRun)
def get_run(run_id: UUID, settings: SettingsDep) -> TrainingRun:
run = load_training_run(settings.data_dir, run_id)
if run is None:
raise HTTPException(status_code=404, detail="training run not found")
return run
@router.post("", response_model=TrainingRun, status_code=status.HTTP_201_CREATED)
async def create_run(
body: CreateTrainingRunRequest,
settings: SettingsDep,
datasets: DatasetStoreDep,
models: ModelStoreDep,
registry: RegistryDep,
backend: TrainingBackendDep,
) -> TrainingRun:
spec = _resolve_spec(body, datasets, models, registry)
now = datetime.now(UTC)
run_id = uuid4()
run = TrainingRun(
id=run_id,
backend=settings.training_backend,
status="submitted",
spec=spec,
created_at=now,
updated_at=now,
)
external_id = await backend.submit(spec, run_id)
run = run.model_copy(
update={
"external_job_id": external_id,
"status": "queued",
"updated_at": datetime.now(UTC),
},
)
run = await _apply_backend_status(backend, run)
save_training_run(settings.data_dir, run)
return run
@router.post("/{run_id}/refresh", response_model=TrainingRun)
async def refresh_run(
run_id: UUID,
settings: SettingsDep,
backend: TrainingBackendDep,
) -> TrainingRun:
run = load_training_run(settings.data_dir, run_id)
if run is None:
raise HTTPException(status_code=404, detail="training run not found")
if run.backend != settings.training_backend:
raise HTTPException(
status_code=409,
detail=(
f"run backend '{run.backend}' does not match "
f"configured '{settings.training_backend}'"
),
)
run = await _apply_backend_status(backend, run)
save_training_run(settings.data_dir, run)
return run
def _require_succeeded_run(run_id: UUID, settings: SettingsDep) -> TrainingRun:
run = load_training_run(settings.data_dir, run_id)
if run is None:
raise HTTPException(status_code=404, detail="training run not found")
if run.status != "succeeded":
raise HTTPException(
status_code=409,
detail=f"training run status '{run.status}' must be 'succeeded'",
)
return run
@router.post("/{run_id}/evaluate", response_model=EvaluationReport)
async def evaluate_run(
run_id: UUID,
settings: SettingsDep,
registry: RegistryDep,
policies: PolicyStoreDep,
orchestrator: OrchestratorDep,
) -> EvaluationReport:
run = _require_succeeded_run(run_id, settings)
agent_name = run.spec.agent_name
if not agent_name:
raise HTTPException(
status_code=422,
detail="training run has no agent_name; set it when creating the run",
)
existing = load_evaluation_for_run(settings.data_dir, run_id)
if existing is not None:
return existing
# Evaluate the candidate version tied to the run (falls back to active).
try:
agent_def = registry.get_agent(agent_name, run.spec.agent_version)
except FileNotFoundError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
if not agent_def.guardrails:
raise HTTPException(status_code=422, detail="agent has no guardrail policy")
try:
policy = policies.get_policy(agent_def.guardrails[0])
except FileNotFoundError as exc:
raise HTTPException(status_code=422, detail=str(exc)) from exc
report = await run_post_train_evaluation(
training_run_id=run_id,
agent_def=agent_def,
policy=policy,
orchestrator=orchestrator,
agents_dir=settings.agents_dir,
)
save_evaluation(settings.data_dir, report)
return report
@router.get("/{run_id}/evaluation", response_model=EvaluationReport)
def get_run_evaluation(run_id: UUID, settings: SettingsDep) -> EvaluationReport:
report = load_evaluation_for_run(settings.data_dir, run_id)
if report is None:
raise HTTPException(status_code=404, detail="evaluation not found for this training run")
return report
@@ -0,0 +1,42 @@
"""Persist training runs as one JSON file per run under data/training_runs/."""
from __future__ import annotations
import json
from pathlib import Path
from uuid import UUID
from forja_core.domain.training import TrainingRun
def _runs_dir(data_dir: Path) -> Path:
return data_dir / "training_runs"
def save_training_run(data_dir: Path, run: TrainingRun) -> None:
directory = _runs_dir(data_dir)
directory.mkdir(parents=True, exist_ok=True)
path = directory / f"{run.id}.json"
path.write_text(
json.dumps(run.model_dump(mode="json"), indent=2, default=str),
encoding="utf-8",
)
def load_training_run(data_dir: Path, run_id: UUID) -> TrainingRun | None:
path = _runs_dir(data_dir) / f"{run_id}.json"
if not path.exists():
return None
data = json.loads(path.read_text(encoding="utf-8"))
return TrainingRun.model_validate(data)
def list_training_runs(data_dir: Path) -> list[TrainingRun]:
directory = _runs_dir(data_dir)
if not directory.exists():
return []
runs: list[TrainingRun] = []
for path in sorted(directory.glob("*.json"), reverse=True):
data = json.loads(path.read_text(encoding="utf-8"))
runs.append(TrainingRun.model_validate(data))
return runs
@@ -7,9 +7,9 @@ from uuid import UUID
from fastapi import APIRouter, Query
from agentforge_core.api.deps import SettingsDep
from agentforge_core.api.persistence import read_violations
from agentforge_core.domain.guardrail import GuardrailViolation
from forja_core.api.deps import SettingsDep
from forja_core.api.persistence import read_violations
from forja_core.domain.guardrail import GuardrailViolation
router = APIRouter()
@@ -35,16 +35,26 @@ class Settings(BaseSettings):
# Guardrails
guardrails_nemo_enabled: bool = False
# Comma-separated keywords for the NeMo topical-rails stub. Empty disables
# the off-topic check (no domain assumptions baked into the core).
guardrails_nemo_allowed_keywords: str = ""
# Observabilidad
log_level: Literal["DEBUG", "INFO", "WARNING", "ERROR"] = "INFO"
# Training / MLOps orchestration
training_backend: Literal["mock", "azure_ml"] = "mock"
azure_ml_subscription_id: str = ""
azure_ml_resource_group: str = ""
azure_ml_workspace_name: str = ""
azure_ml_compute: str = ""
azure_ml_environment: str = "AzureML-sklearn-1.5"
azure_ml_experiment_name: str = "forja-training"
azure_ml_command: str = "python train.py"
# Persistencia
data_dir: Path = Field(default=Path("./data"))
agents_dir: Path = Field(default=Path("./agents"))
policies_dir: Path = Field(default=Path("./policies"))
def get_settings() -> Settings:
"""Helper para inyección por dependencia en FastAPI."""
return Settings()
datasets_dir: Path = Field(default=Path("./datasets"))
models_dir: Path = Field(default=Path("./models"))
@@ -7,6 +7,8 @@ from typing import Any, Literal
from pydantic import BaseModel, Field
from forja_core.domain.version_meta import VersionMeta
class LLMConfig(BaseModel):
"""Configuración del proveedor LLM utilizado por un agente."""
@@ -17,14 +19,8 @@ class LLMConfig(BaseModel):
max_tokens: int = Field(default=2000, ge=1, le=128_000)
class AgentVersionMeta(BaseModel):
"""Metadatos de una versión concreta de un agente (estilo commit Git)."""
id: str
hash: str
author: str
message: str
created_at: datetime
# Alias de compatibilidad: los metadatos de versión son comunes a todos los registries.
AgentVersionMeta = VersionMeta
class AgentDefinition(BaseModel):
@@ -41,3 +37,10 @@ class AgentDefinition(BaseModel):
output_schema: dict[str, Any]
risk_threshold_for_hitl: int = Field(default=4, ge=1, le=5)
updated_at: datetime
input_label: str | None = None
input_placeholder: str | None = None
category: str | None = None
tags: list[str] = Field(default_factory=list)
icon: str | None = "🤖"
color: str | None = None
template: str = "governed_llm"
+28
View File
@@ -0,0 +1,28 @@
"""Dataset definitions for fine-tuning (versioned YAML)."""
from __future__ import annotations
from datetime import datetime
from typing import Literal
from pydantic import BaseModel, Field
from forja_core.domain.version_meta import VersionMeta
# Compatibility alias: version metadata is shared across all registries.
DatasetVersionMeta = VersionMeta
class DatasetDefinition(BaseModel):
"""Declarative dataset used to submit training jobs."""
name: str
version: str
owner: str
purpose: str
state: Literal["draft", "active", "deprecated"] = "active"
format: Literal["jsonl", "conversation"] = "jsonl"
records_path: str = Field(
description="Path relative to the dataset directory, e.g. records/train.jsonl",
)
updated_at: datetime
@@ -8,7 +8,7 @@ from uuid import UUID
from pydantic import BaseModel, Field
from agentforge_core.domain.guardrail import GuardrailViolation
from forja_core.domain.guardrail import GuardrailViolation
class ProposedAction(BaseModel):
@@ -0,0 +1,28 @@
"""Base / fine-tune target model definitions (versioned YAML)."""
from __future__ import annotations
from datetime import datetime
from typing import Literal
from pydantic import BaseModel, Field
from forja_core.domain.version_meta import VersionMeta
# Compatibility alias: version metadata is shared across all registries.
ModelVersionMeta = VersionMeta
class ModelDefinition(BaseModel):
"""Registered base model or training target reference."""
name: str
version: str
owner: str
purpose: str
state: Literal["draft", "active", "deprecated"] = "active"
base_model_id: str = Field(
description="Provider model id, e.g. HuggingFace repo or Azure deployment name",
)
adaptation: Literal["full", "lora", "qlora"] = "lora"
updated_at: datetime
@@ -2,11 +2,12 @@
from __future__ import annotations
from datetime import datetime
from typing import Any, Literal
from pydantic import BaseModel, Field
from forja_core.domain.version_meta import VersionMeta
class PolicyValidator(BaseModel):
"""Validador individual configurado en una política."""
@@ -15,14 +16,8 @@ class PolicyValidator(BaseModel):
config: dict[str, Any] = Field(default_factory=dict)
class PolicyVersionMeta(BaseModel):
"""Metadatos de una versión concreta de una política (estilo commit Git)."""
id: str
hash: str
author: str
message: str
created_at: datetime
# Alias de compatibilidad: los metadatos de versión son comunes a todos los registries.
PolicyVersionMeta = VersionMeta
class PolicyDefinition(BaseModel):
+127
View File
@@ -0,0 +1,127 @@
"""Training jobs, runs, and promotion-related records."""
from __future__ import annotations
from datetime import datetime
from typing import Literal
from uuid import UUID
from pydantic import BaseModel, Field
TrainingRunStatus = Literal[
"submitted",
"queued",
"running",
"succeeded",
"failed",
"cancelled",
]
TrainingBackendKind = Literal["mock", "azure_ml"]
class TrainingHyperparameters(BaseModel):
"""Minimal hyperparameters passed to an external trainer."""
learning_rate: float = Field(default=1e-4, gt=0.0, le=1.0)
epochs: int = Field(default=1, ge=1, le=100)
batch_size: int = Field(default=4, ge=1, le=512)
class TrainingJobSpec(BaseModel):
"""Input specification for a training backend."""
dataset_name: str
dataset_version: str
model_name: str
model_version: str
agent_name: str | None = None
# Candidate agent version this run is meant to produce/validate. Post-train
# evaluation runs against this version, and promotion requires the
# evaluation report to match it.
agent_version: str | None = None
hyperparameters: TrainingHyperparameters = Field(default_factory=TrainingHyperparameters)
class ArtifactRef(BaseModel):
"""Output artifact from a completed training job."""
uri: str
kind: Literal["checkpoint", "adapter", "metrics"] = "adapter"
class BackendJobStatus(BaseModel):
"""Status returned by an external MLOps backend."""
status: TrainingRunStatus
message: str | None = None
artifacts: list[ArtifactRef] = Field(default_factory=list)
class TrainingRun(BaseModel):
"""Persisted training run orchestrated by Forja."""
id: UUID
backend: TrainingBackendKind
status: TrainingRunStatus
spec: TrainingJobSpec
external_job_id: str | None = None
created_at: datetime
updated_at: datetime
artifact_refs: list[ArtifactRef] = Field(default_factory=list)
error_message: str | None = None
class ScenarioEvalResult(BaseModel):
"""Result of running one canonical scenario through the governed runtime."""
scenario: str
status: str
blocking_violations: int = 0
class EvaluationReport(BaseModel):
"""Post-training evaluation summary (gate before promotion)."""
id: UUID
training_run_id: UUID
agent_name: str
agent_version: str
passed: bool
scenario_count: int = 0
violation_count: int = 0
scenarios: list[ScenarioEvalResult] = Field(default_factory=list)
created_at: datetime
notes: str | None = None
PromotionStatus = Literal["pending_approval", "approved", "rejected"]
class PromotionRequest(BaseModel):
"""Governance HITL request to promote a draft agent version to active."""
id: UUID
agent_name: str
agent_version: str
training_run_id: UUID
evaluation_report_id: UUID
status: PromotionStatus
requested_by: str
created_at: datetime
updated_at: datetime
# Operator who approved or rejected (audit trail for both outcomes).
decided_by: str | None = None
comment: str | None = None
class PromotionRecord(BaseModel):
"""Audit record when a model/agent version is promoted to active."""
id: UUID
agent_name: str
agent_version: str
training_run_id: UUID | None = None
evaluation_report_id: UUID | None = None
approved_by: str
comment: str | None = None
created_at: datetime
@@ -0,0 +1,17 @@
"""Metadatos de versión compartidos por todos los registries versionados."""
from __future__ import annotations
from datetime import datetime
from pydantic import BaseModel
class VersionMeta(BaseModel):
"""Metadatos de una versión concreta de un artefacto versionado (estilo commit Git)."""
id: str
hash: str
author: str
message: str
created_at: datetime
@@ -0,0 +1 @@
"""Post-training evaluation and promotion gates."""
@@ -0,0 +1,85 @@
"""Run post-training evaluation: governed agent runs over canonical scenarios."""
from __future__ import annotations
from datetime import UTC, datetime
from pathlib import Path
from uuid import UUID, uuid4
from forja_core.domain.agent import AgentDefinition
from forja_core.domain.policy import PolicyDefinition
from forja_core.domain.training import EvaluationReport, ScenarioEvalResult
from forja_core.evaluation.scenarios import load_agent_scenarios
from forja_core.runtime.orchestrator import AgentOrchestrator
def _blocking_violation_count(execution: object) -> int:
violations = getattr(execution, "violations", [])
return sum(1 for v in violations if v.blocked)
async def run_post_train_evaluation(
*,
training_run_id: UUID,
agent_def: AgentDefinition,
policy: PolicyDefinition,
orchestrator: AgentOrchestrator,
agents_dir: Path,
) -> EvaluationReport:
"""Execute each scenario; pass when there are no blocking guardrail violations."""
scenarios = load_agent_scenarios(agents_dir, agent_def.name)
if not scenarios:
now = datetime.now(UTC)
return EvaluationReport(
id=uuid4(),
training_run_id=training_run_id,
agent_name=agent_def.name,
agent_version=agent_def.version,
passed=False,
scenario_count=0,
violation_count=0,
created_at=now,
notes="No scenarios found under agents/<name>/examples/",
)
results: list[ScenarioEvalResult] = []
total_blocking = 0
for name, user_input in scenarios:
execution = await orchestrator.invoke(
agent_def=agent_def,
policy=policy,
user_input=user_input,
)
blocking = _blocking_violation_count(execution)
total_blocking += blocking
results.append(
ScenarioEvalResult(
scenario=name,
status=execution.status,
blocking_violations=blocking,
)
)
failed_count = sum(1 for r in results if r.status == "failed")
passed = total_blocking == 0 and failed_count == 0
notes = None
if not passed:
parts: list[str] = []
if total_blocking:
parts.append(f"{total_blocking} blocking guardrail violation(s)")
if failed_count:
parts.append(f"{failed_count} scenario(s) failed")
notes = "; ".join(parts)
return EvaluationReport(
id=uuid4(),
training_run_id=training_run_id,
agent_name=agent_def.name,
agent_version=agent_def.version,
passed=passed,
scenario_count=len(results),
violation_count=total_blocking,
scenarios=results,
created_at=datetime.now(UTC),
notes=notes,
)
@@ -0,0 +1,16 @@
"""Load canonical evaluation scenarios from agent example files."""
from __future__ import annotations
from pathlib import Path
def load_agent_scenarios(agents_dir: Path, agent_name: str) -> list[tuple[str, str]]:
"""Return (scenario_name, input_text) pairs from agents/<name>/examples/*.txt."""
examples_dir = agents_dir / agent_name / "examples"
if not examples_dir.is_dir():
return []
scenarios: list[tuple[str, str]] = []
for path in sorted(examples_dir.glob("*.txt")):
scenarios.append((path.name, path.read_text(encoding="utf-8")))
return scenarios
@@ -5,8 +5,8 @@ from __future__ import annotations
from typing import Any, Protocol
from uuid import UUID
from agentforge_core.domain.guardrail import GuardrailViolation
from agentforge_core.domain.policy import PolicyDefinition
from forja_core.domain.guardrail import GuardrailViolation
from forja_core.domain.policy import PolicyDefinition
class GuardrailEngine(Protocol):
@@ -6,9 +6,9 @@ import asyncio
from typing import Any
from uuid import UUID
from agentforge_core.domain.guardrail import GuardrailViolation
from agentforge_core.domain.policy import PolicyDefinition
from agentforge_core.guardrails.base import GuardrailEngine
from forja_core.domain.guardrail import GuardrailViolation
from forja_core.domain.policy import PolicyDefinition
from forja_core.guardrails.base import GuardrailEngine
class CompositeGuardrailEngine:
+24
View File
@@ -0,0 +1,24 @@
"""Factory que ensambla el GuardrailEngine según settings."""
from __future__ import annotations
from forja_core.config import Settings
from forja_core.guardrails.base import GuardrailEngine
from forja_core.guardrails.composite import CompositeGuardrailEngine
from forja_core.guardrails.guardrails_ai import GuardrailsAIEngine
from forja_core.guardrails.nemo import NeMoGuardrailsEngine
def build_guardrail_engine(settings: Settings) -> GuardrailEngine:
"""Ensambla el engine: GuardrailsAIEngine siempre; NeMo si está habilitado.
Los keywords de topical rails vienen de configuración (CSV en
``GUARDRAILS_NEMO_ALLOWED_KEYWORDS``); el core no asume ningún dominio.
"""
engines: list[GuardrailEngine] = [GuardrailsAIEngine()]
if settings.guardrails_nemo_enabled:
keywords = [
k.strip() for k in settings.guardrails_nemo_allowed_keywords.split(",") if k.strip()
]
engines.append(NeMoGuardrailsEngine(allowed_keywords=keywords))
return CompositeGuardrailEngine(engines)
@@ -9,9 +9,9 @@ from uuid import UUID
import structlog
from agentforge_core.domain.guardrail import GuardrailViolation
from agentforge_core.domain.policy import PolicyDefinition, PolicyValidator
from agentforge_core.guardrails.validators import (
from forja_core.domain.guardrail import GuardrailViolation
from forja_core.domain.policy import PolicyDefinition, PolicyValidator
from forja_core.guardrails.validators import (
detect_pii,
forbidden_action_keywords,
forbidden_topics,
@@ -12,8 +12,8 @@ from uuid import UUID
import structlog
from agentforge_core.domain.guardrail import GuardrailViolation
from agentforge_core.domain.policy import PolicyDefinition
from forja_core.domain.guardrail import GuardrailViolation
from forja_core.domain.policy import PolicyDefinition
log = structlog.get_logger(__name__)
@@ -48,7 +48,7 @@ class NeMoGuardrailsEngine:
stage="input",
validator="NeMoTopicalRails",
severity="warning",
message="Input fuera de los temas permitidos.",
message="Input outside the allowed topics.",
blocked=False,
)
]
@@ -10,7 +10,7 @@ from uuid import UUID
import jsonschema
from agentforge_core.domain.guardrail import GuardrailViolation
from forja_core.domain.guardrail import GuardrailViolation
_INJECTION_PATTERNS = [
r"ignore (the )?(previous|prior|above) (instruction|prompt|message)",
@@ -21,6 +21,33 @@ _INJECTION_PATTERNS = [
r"reveal (the )?(system|hidden) (prompt|instruction)",
]
# Singleton perezoso del AnalyzerEngine de Presidio. Construirlo es caro (carga el
# modelo spaCy y los recognizers), así que se reutiliza entre llamadas.
_PRESIDIO_ANALYZER: Any = None
def _presidio_analyzer() -> Any:
"""``AnalyzerEngine`` de Presidio configurado con el modelo spaCy ``en_core_web_sm``.
Presidio usa por defecto ``en_core_web_lg`` (~560 MB), que **no** está en la imagen
Docker: ``core/Dockerfile`` instala ``en_core_web_sm``. Si Presidio no está instalado
o el modelo no se puede cargar, esto lanza y el llamador (`detect_pii`) hace fallback
a la detección por regex.
"""
global _PRESIDIO_ANALYZER
if _PRESIDIO_ANALYZER is None:
from presidio_analyzer import AnalyzerEngine
from presidio_analyzer.nlp_engine import NlpEngineProvider
nlp_engine = NlpEngineProvider(
nlp_configuration={
"nlp_engine_name": "spacy",
"models": [{"lang_code": "en", "model_name": "en_core_web_sm"}],
}
).create_engine()
_PRESIDIO_ANALYZER = AnalyzerEngine(nlp_engine=nlp_engine)
return _PRESIDIO_ANALYZER
def _violation(
*,
@@ -45,11 +72,11 @@ def _violation(
def detect_pii(
text: str, config: dict[str, Any], trace_id: UUID, stage: str
) -> list[GuardrailViolation]:
"""Detección PII vía Presidio Analyzer (con fallback a regex si no está instalado)."""
"""Detección PII vía Presidio Analyzer (con fallback a regex si no está disponible)."""
try:
from presidio_analyzer import AnalyzerEngine
analyzer = _presidio_analyzer()
except Exception:
# Si Presidio no está, fallback a regex básica
# Presidio no instalado o modelo spaCy no disponible → fallback a regex básica.
return _pii_regex_fallback(text, config, trace_id, stage)
entities = config.get(
@@ -59,7 +86,6 @@ def detect_pii(
sev = config.get("severity_on_match", "block")
blocked = sev == "block"
analyzer = AnalyzerEngine()
results = analyzer.analyze(text=text, entities=entities, language="en")
if not results:
return []
@@ -70,7 +96,7 @@ def detect_pii(
stage=stage,
validator="DetectPII",
severity=sev,
message=f"PII detectada: {matched}",
message=f"PII detected: {matched}",
blocked=blocked,
)
]
@@ -97,7 +123,7 @@ def _pii_regex_fallback(
stage=stage,
validator="DetectPII",
severity=sev,
message=f"PII detectada (regex fallback): {', '.join(matched)}",
message=f"PII detected (regex fallback): {', '.join(matched)}",
blocked=blocked,
)
]
@@ -118,7 +144,7 @@ def prompt_injection(
stage=stage,
validator="PromptInjection",
severity=sev,
message=f"Patrón de inyección detectado: {pattern}",
message=f"Injection pattern detected: {pattern}",
blocked=blocked,
)
]
@@ -149,7 +175,7 @@ def toxic_language(
stage=stage,
validator="ToxicLanguage",
severity=sev,
message=f"Lenguaje tóxico (score={score:.2f}, hits={hits})",
message=f"Toxic language (score={score:.2f}, hits={hits})",
blocked=blocked,
)
]
@@ -172,7 +198,7 @@ def forbidden_topics(
stage=stage,
validator="ForbiddenTopics",
severity=sev,
message=f"Temas prohibidos: {', '.join(matched)}",
message=f"Forbidden topics: {', '.join(matched)}",
blocked=blocked,
)
]
@@ -230,7 +256,7 @@ def forbidden_action_keywords(
stage=stage,
validator="ForbiddenActionKeywords",
severity=sev,
message=f"Palabras prohibidas en acciones: {', '.join(matched)}",
message=f"Forbidden keywords in actions: {', '.join(matched)}",
blocked=blocked,
)
]
@@ -249,7 +275,7 @@ def telco_safety_rules(
target = str(a.get("target", "")).lower()
rollback = str(a.get("rollback_plan", "")).strip()
if ("prod" in target or "production" in target) and (not rollback or rollback == "n/a"):
issues.append(f"acción sobre prod sin rollback: {a.get('action')}")
issues.append(f"production action without rollback: {a.get('action')}")
if "never_propose_mass_action_without_canary" in rules:
mass_keywords = {"all", "todos", "*", "mass"}
@@ -257,7 +283,7 @@ def telco_safety_rules(
target = str(a.get("target", "")).lower()
plan = str(a.get("rollback_plan", "")).lower()
if any(k in target for k in mass_keywords) and "canary" not in plan:
issues.append(f"acción masiva sin canary: {a.get('action')}")
issues.append(f"mass action without canary plan: {a.get('action')}")
if not issues:
return []
@@ -8,7 +8,7 @@ from typing import Any
from openai import AsyncAzureOpenAI
from agentforge_core.llm.base import CompletionResult, Message
from forja_core.llm.base import CompletionResult, Message
class AzureOpenAIProvider:
+40
View File
@@ -0,0 +1,40 @@
"""Factory que selecciona el LLMProvider según settings."""
from __future__ import annotations
from typing import Literal
from forja_core.config import Settings
from forja_core.llm.azure import AzureOpenAIProvider
from forja_core.llm.base import LLMProvider
from forja_core.llm.fallback import FallbackLLMProvider
from forja_core.llm.mock import MockProvider
from forja_core.llm.openai import OpenAIProvider
def _build_single(provider: Literal["mock", "azure", "openai"], settings: Settings) -> LLMProvider:
match provider:
case "mock":
return MockProvider()
case "azure":
return AzureOpenAIProvider(
endpoint=settings.azure_openai_endpoint,
api_key=settings.azure_openai_api_key,
deployment=settings.azure_openai_deployment,
api_version=settings.azure_openai_api_version,
)
case "openai":
return OpenAIProvider(
api_key=settings.openai_api_key,
model=settings.openai_model,
)
def build_llm_provider(settings: Settings) -> LLMProvider:
"""Materializa el proveedor activo (con fallback opcional). Falla en arranque si la config es inconsistente."""
primary = _build_single(settings.llm_provider, settings)
fallback_name = settings.llm_fallback_provider
if not fallback_name or fallback_name == settings.llm_provider:
return primary
fallback = _build_single(fallback_name, settings)
return FallbackLLMProvider(primary, fallback)
+46
View File
@@ -0,0 +1,46 @@
"""Proveedor LLM con fallback: intenta el primario y, si agota sus reintentos, usa el secundario."""
from __future__ import annotations
from typing import Any
import structlog
from forja_core.llm.base import CompletionResult, LLMProvider, Message
log = structlog.get_logger(__name__)
class FallbackLLMProvider:
"""Envuelve dos providers: si el primario lanza, delega en el de fallback.
Cada provider interno conserva su propia política de retries; este wrapper
solo decide el cambio de proveedor cuando el primario falla definitivamente.
"""
def __init__(self, primary: LLMProvider, fallback: LLMProvider) -> None:
self._primary = primary
self._fallback = fallback
self.name = f"{primary.name}+fallback:{fallback.name}"
async def complete(
self,
messages: list[Message],
schema: dict[str, Any] | None = None,
temperature: float = 0.2,
max_tokens: int = 2000,
) -> CompletionResult:
try:
return await self._primary.complete(
messages, schema=schema, temperature=temperature, max_tokens=max_tokens
)
except Exception as exc:
log.warning(
"llm_primary_failed_falling_back",
primary=self._primary.name,
fallback=self._fallback.name,
error=str(exc),
)
return await self._fallback.complete(
messages, schema=schema, temperature=temperature, max_tokens=max_tokens
)
@@ -7,7 +7,7 @@ import json
import time
from typing import Any
from agentforge_core.llm.base import CompletionResult, Message
from forja_core.llm.base import CompletionResult, Message
_CANONICAL_RESPONSES: dict[str, dict[str, Any]] = {
"sip": {
@@ -8,7 +8,7 @@ from typing import Any
from openai import AsyncOpenAI
from agentforge_core.llm.base import CompletionResult, Message
from forja_core.llm.base import CompletionResult, Message
class OpenAIProvider:
+56
View File
@@ -0,0 +1,56 @@
"""Composición raíz de la aplicación FastAPI (forja-core)."""
from __future__ import annotations
from fastapi import FastAPI
from forja_core.api.middlewares import TraceIdMiddleware
from forja_core.config import Settings
from forja_core.observability.logging import configure_logging
def create_app() -> FastAPI:
"""Construye la app: logging, middleware de trace_id, /health y routers."""
settings = Settings()
configure_logging(level=settings.log_level)
app = FastAPI(
title="Forja Core",
version="0.1.0",
description="Plataforma de gobernanza de agentes IA — API REST.",
)
app.add_middleware(TraceIdMiddleware)
@app.get("/health", tags=["meta"])
async def health() -> dict[str, str]:
return {"status": "ok"}
from forja_core.api import (
agents,
datasets,
executions,
models,
policies,
promotions,
training,
violations,
)
from forja_core.web import ui
# UI HTMX embebida (Opción A) en rutas amigables para humanos.
# Toda la API REST vive bajo /api para eliminar ambigüedad y conflictos de ruta.
app.include_router(ui.router)
app.include_router(agents.router, prefix="/api/agents", tags=["agents"])
app.include_router(executions.invoke_router, prefix="/api/agents", tags=["agents"])
app.include_router(executions.router, prefix="/api/executions", tags=["executions"])
app.include_router(policies.router, prefix="/api/policies", tags=["policies"])
app.include_router(violations.router, prefix="/api/violations", tags=["violations"])
app.include_router(datasets.router, prefix="/api/datasets", tags=["datasets"])
app.include_router(models.router, prefix="/api/models", tags=["models"])
app.include_router(training.router, prefix="/api/training/runs", tags=["training"])
app.include_router(promotions.router, prefix="/api/promotions", tags=["promotions"])
return app
app = create_app()
@@ -0,0 +1,25 @@
"""Read-only dataset registry (YAML versioned, same layout as agents/)."""
from __future__ import annotations
from forja_core.domain.dataset import DatasetDefinition, DatasetVersionMeta
from forja_core.registry.yaml_store import VersionedYamlStore
class FileSystemDatasetStore(VersionedYamlStore[DatasetDefinition]):
"""Reads datasets from datasets/<name>/{index.yaml,versions/vN.yaml}."""
model_cls = DatasetDefinition
kind = "dataset"
def list_datasets(self) -> list[DatasetDefinition]:
return self._list_all()
def get_dataset(self, name: str, version: str | None = None) -> DatasetDefinition:
return self._get(name, version)
def get_version(self, name: str, version: str) -> DatasetDefinition:
return self._get_version(name, version)
def list_versions(self, name: str) -> list[DatasetVersionMeta]:
return self._list_versions(name)
+27
View File
@@ -0,0 +1,27 @@
"""Factories para AgentRegistry y PolicyStore basados en Settings."""
from __future__ import annotations
from forja_core.config import Settings
from forja_core.registry.dataset_store import FileSystemDatasetStore
from forja_core.registry.model_store import FileSystemModelStore
from forja_core.registry.policy_store import FileSystemPolicyStore
from forja_core.registry.repository import FileSystemAgentRegistry
def build_agent_registry(settings: Settings) -> FileSystemAgentRegistry:
"""Construye el registry de agentes apuntando a ``settings.agents_dir``."""
return FileSystemAgentRegistry(settings.agents_dir)
def build_policy_store(settings: Settings) -> FileSystemPolicyStore:
"""Construye el store de políticas apuntando a ``settings.policies_dir``."""
return FileSystemPolicyStore(settings.policies_dir)
def build_dataset_store(settings: Settings) -> FileSystemDatasetStore:
return FileSystemDatasetStore(settings.datasets_dir)
def build_model_store(settings: Settings) -> FileSystemModelStore:
return FileSystemModelStore(settings.models_dir)
@@ -0,0 +1,25 @@
"""Read-only foundation model registry (YAML versioned)."""
from __future__ import annotations
from forja_core.domain.foundation_model import ModelDefinition, ModelVersionMeta
from forja_core.registry.yaml_store import VersionedYamlStore
class FileSystemModelStore(VersionedYamlStore[ModelDefinition]):
"""Reads models from models/<name>/{index.yaml,versions/vN.yaml}."""
model_cls = ModelDefinition
kind = "model"
def list_models(self) -> list[ModelDefinition]:
return self._list_all()
def get_model(self, name: str, version: str | None = None) -> ModelDefinition:
return self._get(name, version)
def get_version(self, name: str, version: str) -> ModelDefinition:
return self._get_version(name, version)
def list_versions(self, name: str) -> list[ModelVersionMeta]:
return self._list_versions(name)
@@ -0,0 +1,32 @@
"""PolicyStore basado en sistema de ficheros (YAML por versión + index.yaml)."""
from __future__ import annotations
from forja_core.domain.policy import PolicyDefinition, PolicyVersionMeta
from forja_core.registry.yaml_store import VersionedYamlStore
class FileSystemPolicyStore(VersionedYamlStore[PolicyDefinition]):
"""Lee políticas de policies/<name>/{index.yaml,versions/vN.yaml}."""
model_cls = PolicyDefinition
kind = "policy"
def list_policies(self) -> list[PolicyDefinition]:
return self._list_all()
def get_policy(self, name: str, version: str | None = None) -> PolicyDefinition:
return self._get(name, version)
def list_versions(self, name: str) -> list[PolicyVersionMeta]:
return self._list_versions(name)
def upsert_version(
self,
name: str,
body: PolicyDefinition,
message: str,
author: str,
) -> PolicyVersionMeta:
# Las políticas no tienen estado draft/active: la última versión escrita manda.
return self._upsert_version(name, body, body.version, message, author, set_active=True)
@@ -0,0 +1,67 @@
"""AgentRegistry basado en sistema de ficheros (YAML versionado, simulación tipo Git)."""
from __future__ import annotations
from datetime import UTC, datetime
from forja_core.domain.agent import AgentDefinition, AgentVersionMeta
from forja_core.registry.versioning import DiffResult, unified_diff
from forja_core.registry.yaml_store import VersionedYamlStore
class FileSystemAgentRegistry(VersionedYamlStore[AgentDefinition]):
"""Lee/escribe agentes en agents/<name>/{index.yaml,versions/vN.yaml}."""
model_cls = AgentDefinition
kind = "agent"
def list_agents(self) -> list[AgentDefinition]:
return self._list_all()
def get_agent(self, name: str, version: str | None = None) -> AgentDefinition:
return self._get(name, version)
def get_version(self, name: str, version: str) -> AgentDefinition:
return self._get_version(name, version)
def list_versions(self, name: str) -> list[AgentVersionMeta]:
return self._list_versions(name)
def upsert_version(
self,
name: str,
body: AgentDefinition,
message: str,
author: str,
) -> AgentVersionMeta:
return self._upsert_version(
name,
body,
body.version,
message,
author,
set_active=body.state == "active",
)
def promote_to_active(
self,
name: str,
version: str,
author: str,
comment: str,
) -> AgentVersionMeta:
"""Set a version to active and update the registry index (governance promotion)."""
agent = self.get_version(name, version)
updated = agent.model_copy(update={"state": "active", "updated_at": datetime.now(UTC)})
return self.upsert_version(name, updated, message=comment, author=author)
def diff_versions(self, name: str, v1: str, v2: str) -> DiffResult:
path_a = self._root / name / "versions" / f"{v1}.yaml"
path_b = self._root / name / "versions" / f"{v2}.yaml"
text_a = path_a.read_text(encoding="utf-8") if path_a.exists() else ""
text_b = path_b.read_text(encoding="utf-8") if path_b.exists() else ""
return DiffResult(
from_version=v1,
to_version=v2,
unified_diff=unified_diff(text_a, text_b, f"{name}@{v1}", f"{name}@{v2}"),
)
@@ -1,49 +1,62 @@
"""AgentRegistry basado en sistema de ficheros (YAML versionado, simulación tipo Git)."""
"""Base genérica de los registries versionados en YAML.
Layout común en disco: ``<root>/<name>/{index.yaml, versions/<vN>.yaml}``.
Agentes, políticas, datasets y modelos comparten esta mecánica; cada store
concreto fija ``model_cls`` (el modelo Pydantic de la definición) y ``kind``
(el nombre del artefacto en los mensajes de error) y expone wrappers con
nombres de dominio (``get_agent``, ``get_policy``, ...).
"""
from __future__ import annotations
from datetime import UTC, datetime
from pathlib import Path
from typing import Any
from typing import Any, Generic, TypeVar
import yaml
from pydantic import BaseModel
from agentforge_core.domain.agent import AgentDefinition, AgentVersionMeta
from agentforge_core.registry.versioning import DiffResult, compute_hash, unified_diff
from forja_core.domain.version_meta import VersionMeta
from forja_core.registry.versioning import compute_hash
M = TypeVar("M", bound=BaseModel)
class FileSystemAgentRegistry:
"""Lee/escribe agentes en agents/<name>/{index.yaml,versions/vN.yaml}."""
class VersionedYamlStore(Generic[M]):
"""Operaciones de lectura sobre un árbol de artefactos versionados."""
model_cls: type[M]
kind: str = "item"
def __init__(self, root: Path) -> None:
self._root = Path(root)
def list_agents(self) -> list[AgentDefinition]:
def _list_all(self) -> list[M]:
if not self._root.exists():
return []
agents: list[AgentDefinition] = []
out: list[M] = []
for d in sorted(self._root.iterdir()):
if d.is_dir() and (d / "index.yaml").exists():
agents.append(self.get_agent(d.name))
return agents
out.append(self._get(d.name))
return out
def get_agent(self, name: str, version: str | None = None) -> AgentDefinition:
def _get(self, name: str, version: str | None = None) -> M:
index = self._load_index(name)
v = version or index["active_version"]
return self.get_version(name, v)
v: str = version or index["active_version"]
return self._get_version(name, v)
def get_version(self, name: str, version: str) -> AgentDefinition:
def _get_version(self, name: str, version: str) -> M:
path = self._root / name / "versions" / f"{version}.yaml"
if not path.exists():
raise FileNotFoundError(f"agent version not found: {name}@{version}")
raise FileNotFoundError(f"{self.kind} version not found: {name}@{version}")
with path.open(encoding="utf-8") as f:
data = yaml.safe_load(f)
return AgentDefinition.model_validate(data)
return self.model_cls.model_validate(data)
def list_versions(self, name: str) -> list[AgentVersionMeta]:
def _list_versions(self, name: str) -> list[VersionMeta]:
index = self._load_index(name)
return [
AgentVersionMeta(
VersionMeta(
id=v["id"],
hash=v["hash"],
author=v["author"],
@@ -53,68 +66,10 @@ class FileSystemAgentRegistry:
for v in index["versions"]
]
def upsert_version(
self,
name: str,
body: AgentDefinition,
message: str,
author: str,
) -> AgentVersionMeta:
agent_dir = self._root / name
versions_dir = agent_dir / "versions"
versions_dir.mkdir(parents=True, exist_ok=True)
version_id = body.version
yaml_text = yaml.safe_dump(
body.model_dump(mode="json"),
sort_keys=False,
allow_unicode=True,
)
h = compute_hash(yaml_text)
version_path = versions_dir / f"{version_id}.yaml"
version_path.write_text(yaml_text, encoding="utf-8")
meta = AgentVersionMeta(
id=version_id,
hash=h,
author=author,
message=message,
created_at=datetime.now(UTC),
)
index_path = agent_dir / "index.yaml"
if index_path.exists():
with index_path.open(encoding="utf-8") as f:
index: dict[str, Any] = yaml.safe_load(f)
else:
index = {"name": name, "versions": [], "active_version": version_id}
# reemplaza si existía la versión, si no añade
index["versions"] = [v for v in index["versions"] if v["id"] != version_id]
index["versions"].append(meta.model_dump(mode="json"))
if body.state == "active":
index["active_version"] = version_id
with index_path.open("w", encoding="utf-8") as f:
yaml.safe_dump(index, f, sort_keys=False, allow_unicode=True)
return meta
def diff_versions(self, name: str, v1: str, v2: str) -> DiffResult:
path_a = self._root / name / "versions" / f"{v1}.yaml"
path_b = self._root / name / "versions" / f"{v2}.yaml"
text_a = path_a.read_text(encoding="utf-8") if path_a.exists() else ""
text_b = path_b.read_text(encoding="utf-8") if path_b.exists() else ""
return DiffResult(
from_version=v1,
to_version=v2,
unified_diff=unified_diff(text_a, text_b, f"{name}@{v1}", f"{name}@{v2}"),
)
def _load_index(self, name: str) -> dict[str, Any]:
index_path = self._root / name / "index.yaml"
if not index_path.exists():
raise FileNotFoundError(f"agent not found: {name}")
raise FileNotFoundError(f"{self.kind} not found: {name}")
with index_path.open(encoding="utf-8") as f:
data: dict[str, Any] = yaml.safe_load(f)
return data
@@ -124,3 +79,55 @@ class FileSystemAgentRegistry:
if isinstance(value, datetime):
return value
return datetime.fromisoformat(str(value).replace("Z", "+00:00"))
# --- Escritura (usada por agentes y políticas) -----------------------------------
def _upsert_version(
self,
name: str,
body: M,
version_id: str,
message: str,
author: str,
*,
set_active: bool,
) -> VersionMeta:
"""Escribe ``versions/<version_id>.yaml`` y actualiza ``index.yaml``.
Reemplaza la entrada del índice si la versión ya existía; si
``set_active`` es True, la marca como ``active_version``.
"""
artifact_dir = self._root / name
versions_dir = artifact_dir / "versions"
versions_dir.mkdir(parents=True, exist_ok=True)
yaml_text = yaml.safe_dump(
body.model_dump(mode="json"),
sort_keys=False,
allow_unicode=True,
)
(versions_dir / f"{version_id}.yaml").write_text(yaml_text, encoding="utf-8")
meta = VersionMeta(
id=version_id,
hash=compute_hash(yaml_text),
author=author,
message=message,
created_at=datetime.now(UTC),
)
index_path = artifact_dir / "index.yaml"
if index_path.exists():
with index_path.open(encoding="utf-8") as f:
index: dict[str, Any] = yaml.safe_load(f)
else:
index = {"name": name, "versions": [], "active_version": version_id}
index["versions"] = [v for v in index["versions"] if v["id"] != version_id]
index["versions"].append(meta.model_dump(mode="json"))
if set_active:
index["active_version"] = version_id
with index_path.open("w", encoding="utf-8") as f:
yaml.safe_dump(index, f, sort_keys=False, allow_unicode=True)
return meta
@@ -7,11 +7,11 @@ from typing import Any
from langgraph.checkpoint.base import BaseCheckpointSaver
from langgraph.graph import END, START, StateGraph
from agentforge_core.domain.agent import AgentDefinition
from agentforge_core.domain.policy import PolicyDefinition
from agentforge_core.guardrails.base import GuardrailEngine
from agentforge_core.llm.base import LLMProvider
from agentforge_core.runtime.nodes import (
from forja_core.domain.agent import AgentDefinition
from forja_core.domain.policy import PolicyDefinition
from forja_core.guardrails.base import GuardrailEngine
from forja_core.llm.base import LLMProvider
from forja_core.runtime.nodes import (
build_node_approve_gate,
build_node_finalize,
build_node_llm_reason,
@@ -19,7 +19,7 @@ from agentforge_core.runtime.nodes import (
build_node_validate_input,
build_node_validate_output,
)
from agentforge_core.runtime.state import AgentState
from forja_core.runtime.state import AgentState
def build_graph(
@@ -12,11 +12,11 @@ from uuid import UUID
import structlog
from langgraph.types import interrupt
from agentforge_core.domain.agent import AgentDefinition
from agentforge_core.domain.policy import PolicyDefinition
from agentforge_core.guardrails.base import GuardrailEngine
from agentforge_core.llm.base import LLMProvider, Message
from agentforge_core.runtime.state import AgentState
from forja_core.domain.agent import AgentDefinition
from forja_core.domain.policy import PolicyDefinition
from forja_core.guardrails.base import GuardrailEngine
from forja_core.llm.base import LLMProvider, Message
from forja_core.runtime.state import AgentState
log = structlog.get_logger(__name__)
@@ -18,14 +18,14 @@ from uuid import UUID, uuid4
import structlog
from langgraph.types import Command
from agentforge_core.domain.agent import AgentDefinition
from agentforge_core.domain.execution import AgentExecution, DecisionStep, ProposedAction
from agentforge_core.domain.guardrail import GuardrailViolation
from agentforge_core.domain.policy import PolicyDefinition
from agentforge_core.guardrails.base import GuardrailEngine
from agentforge_core.llm.base import LLMProvider
from agentforge_core.runtime.checkpointer import build_checkpointer
from agentforge_core.runtime.graph import build_graph
from forja_core.domain.agent import AgentDefinition
from forja_core.domain.execution import AgentExecution, DecisionStep, ProposedAction
from forja_core.domain.guardrail import GuardrailViolation
from forja_core.domain.policy import PolicyDefinition
from forja_core.guardrails.base import GuardrailEngine
from forja_core.llm.base import LLMProvider
from forja_core.runtime.checkpointer import build_checkpointer
from forja_core.runtime.graph import build_graph
log = structlog.get_logger(__name__)
@@ -39,9 +39,7 @@ def _thread_config(trace_id: UUID) -> dict[str, Any]:
class AgentOrchestrator:
"""Punto único de entrada para invocar agentes, reanudar HITL y leer su estado."""
def __init__(
self, *, provider: LLMProvider, engine: GuardrailEngine, data_dir: Path
) -> None:
def __init__(self, *, provider: LLMProvider, engine: GuardrailEngine, data_dir: Path) -> None:
self._provider = provider
self._engine = engine
self._data_dir = data_dir
+1
View File
@@ -0,0 +1 @@
"""External MLOps training backends (Strategy pattern)."""
@@ -0,0 +1,25 @@
"""Placeholder training script uploaded with Azure ML command jobs.
Replace this script in your workspace or override ``AZURE_ML_COMMAND`` for real fine-tuning.
"""
from __future__ import annotations
import json
import os
def main() -> None:
payload = {
"forja_run_id": os.environ.get("FORJA_RUN_ID"),
"dataset": os.environ.get("FORJA_DATASET"),
"model": os.environ.get("FORJA_MODEL"),
"agent": os.environ.get("FORJA_AGENT"),
"learning_rate": os.environ.get("FORJA_LEARNING_RATE"),
"epochs": os.environ.get("FORJA_EPOCHS"),
}
print(json.dumps(payload))
if __name__ == "__main__":
main()
+67
View File
@@ -0,0 +1,67 @@
"""Azure ML training backend — SDK when configured, stub otherwise."""
from __future__ import annotations
import asyncio
from uuid import UUID
import structlog
from forja_core.config import Settings
from forja_core.domain.training import ArtifactRef, BackendJobStatus, TrainingJobSpec
from forja_core.training.azure_ml_config import is_azure_ml_configured
log = structlog.get_logger(__name__)
class AzureMLTrainingBackend:
"""Submits command jobs via Azure ML SDK v2 when workspace settings are present."""
name = "azure_ml"
def __init__(self, settings: Settings | None = None) -> None:
self._settings = settings or Settings()
self._use_sdk = is_azure_ml_configured(self._settings)
if not self._use_sdk:
log.info(
"azure_ml_stub_mode",
reason="missing workspace config, compute, or azure-ai-ml packages",
)
async def submit(self, spec: TrainingJobSpec, run_id: UUID) -> str:
if self._use_sdk:
from forja_core.training.azure_ml_sdk import submit_command_job
return await asyncio.to_thread(submit_command_job, self._settings, spec, run_id)
return f"azureml-stub:{run_id}"
async def status(self, external_job_id: str) -> BackendJobStatus:
if external_job_id.startswith("azureml-stub:"):
return _stub_status(external_job_id)
if external_job_id.startswith("azureml:"):
job_name = external_job_id.removeprefix("azureml:")
if self._use_sdk:
from forja_core.training.azure_ml_sdk import poll_job_status
return await asyncio.to_thread(poll_job_status, self._settings, job_name)
return BackendJobStatus(
status="failed",
message="Azure ML SDK not configured; cannot poll live job",
)
return BackendJobStatus(
status="failed",
message=f"Unknown Azure ML job id: {external_job_id}",
)
def _stub_status(external_job_id: str) -> BackendJobStatus:
return BackendJobStatus(
status="succeeded",
message="Azure ML stub job completed (workspace not configured)",
artifacts=[
ArtifactRef(
uri=f"azureml://workspaces/stub/jobs/{external_job_id}/outputs/adapter",
kind="adapter",
),
],
)
@@ -0,0 +1,25 @@
"""Azure ML workspace settings helpers."""
from __future__ import annotations
from forja_core.config import Settings
def sdk_packages_available() -> bool:
try:
import azure.ai.ml
import azure.identity # noqa: F401
return True
except ImportError:
return False
def is_azure_ml_configured(settings: Settings) -> bool:
return bool(
settings.azure_ml_subscription_id.strip()
and settings.azure_ml_resource_group.strip()
and settings.azure_ml_workspace_name.strip()
and settings.azure_ml_compute.strip()
and sdk_packages_available()
)
@@ -0,0 +1,100 @@
"""Synchronous Azure ML SDK v2 client (invoked via asyncio.to_thread from the backend)."""
from __future__ import annotations
from pathlib import Path
from uuid import UUID
import structlog
from azure.ai.ml import MLClient, command
from azure.identity import DefaultAzureCredential
from forja_core.config import Settings
from forja_core.domain.training import (
ArtifactRef,
BackendJobStatus,
TrainingJobSpec,
TrainingRunStatus,
)
log = structlog.get_logger(__name__)
JOB_CODE_DIR = Path(__file__).parent / "_azure_job_bundle"
_AZURE_TO_FORJA_STATUS: dict[str, TrainingRunStatus] = {
"notstarted": "queued",
"starting": "queued",
"provisioning": "queued",
"queued": "queued",
"running": "running",
"finalizing": "running",
"completed": "succeeded",
"failed": "failed",
"canceled": "cancelled",
"cancelled": "cancelled",
}
def _ml_client(settings: Settings) -> MLClient:
return MLClient(
credential=DefaultAzureCredential(),
subscription_id=settings.azure_ml_subscription_id,
resource_group_name=settings.azure_ml_resource_group,
workspace_name=settings.azure_ml_workspace_name,
)
def submit_command_job(settings: Settings, spec: TrainingJobSpec, run_id: UUID) -> str:
"""Create an Azure ML command job; returns external id ``azureml:<job_name>``."""
client = _ml_client(settings)
display_name = f"forja-{run_id}"
env_vars = {
"FORJA_RUN_ID": str(run_id),
"FORJA_DATASET": f"{spec.dataset_name}@{spec.dataset_version}",
"FORJA_MODEL": f"{spec.model_name}@{spec.model_version}",
"FORJA_AGENT": spec.agent_name or "",
"FORJA_AGENT_VERSION": spec.agent_version or "",
"FORJA_LEARNING_RATE": str(spec.hyperparameters.learning_rate),
"FORJA_EPOCHS": str(spec.hyperparameters.epochs),
"FORJA_BATCH_SIZE": str(spec.hyperparameters.batch_size),
}
job_def = command(
code=str(JOB_CODE_DIR),
command=settings.azure_ml_command,
environment=settings.azure_ml_environment,
compute=settings.azure_ml_compute,
display_name=display_name,
experiment_name=settings.azure_ml_experiment_name,
environment_variables=env_vars,
)
created = client.jobs.create_or_update(job_def)
job_name = created.name
if job_name is None:
raise RuntimeError("Azure ML job created without a name")
log.info("azure_ml_job_submitted", job_name=job_name, run_id=str(run_id))
return f"azureml:{job_name}"
def poll_job_status(settings: Settings, job_name: str) -> BackendJobStatus:
client = _ml_client(settings)
job = client.jobs.get(job_name)
raw_status = str(job.status) if job.status is not None else "Queued"
raw = raw_status.split(".")[-1].lower()
forja_status = _AZURE_TO_FORJA_STATUS.get(raw, "running")
message = None
if forja_status == "failed":
message = "Azure ML job failed"
artifacts: list[ArtifactRef] = []
if forja_status == "succeeded":
artifacts.append(
ArtifactRef(
uri=(
f"azureml://subscriptions/{settings.azure_ml_subscription_id}"
f"/resourcegroups/{settings.azure_ml_resource_group}"
f"/workspaces/{settings.azure_ml_workspace_name}"
f"/jobs/{job_name}/outputs/default"
),
kind="adapter",
)
)
return BackendJobStatus(status=forja_status, message=message, artifacts=artifacts)
+22
View File
@@ -0,0 +1,22 @@
"""TrainingBackend protocol for external fine-tuning platforms."""
from __future__ import annotations
from typing import Protocol
from uuid import UUID
from forja_core.domain.training import BackendJobStatus, TrainingJobSpec
class TrainingBackend(Protocol):
"""Submit and poll jobs on Azure ML, SageMaker, Hugging Face Jobs, etc."""
name: str
async def submit(self, spec: TrainingJobSpec, run_id: UUID) -> str:
"""Start a remote job; returns provider job id."""
...
async def status(self, external_job_id: str) -> BackendJobStatus:
"""Poll remote job status."""
...
+14
View File
@@ -0,0 +1,14 @@
"""Build TrainingBackend from Settings."""
from __future__ import annotations
from forja_core.config import Settings
from forja_core.training.azure_ml import AzureMLTrainingBackend
from forja_core.training.base import TrainingBackend
from forja_core.training.mock import MockTrainingBackend
def build_training_backend(settings: Settings) -> TrainingBackend:
if settings.training_backend == "azure_ml":
return AzureMLTrainingBackend(settings)
return MockTrainingBackend()
+28
View File
@@ -0,0 +1,28 @@
"""Mock training backend for local development and tests."""
from __future__ import annotations
from uuid import UUID
from forja_core.domain.training import ArtifactRef, BackendJobStatus, TrainingJobSpec
class MockTrainingBackend:
"""Immediately succeeds with a placeholder artifact URI."""
name = "mock"
async def submit(self, spec: TrainingJobSpec, run_id: UUID) -> str:
return f"mock:{run_id}"
async def status(self, external_job_id: str) -> BackendJobStatus:
return BackendJobStatus(
status="succeeded",
message="Mock training completed",
artifacts=[
ArtifactRef(
uri=f"{external_job_id}/adapter",
kind="adapter",
),
],
)
+1
View File
@@ -0,0 +1 @@
"""Paquete de UI HTMX embebida (FastAPI + Jinja2 + HTMX)."""
@@ -0,0 +1,79 @@
<div class="grid grid-cols-1 lg:grid-cols-3 gap-4">
<div class="lg:col-span-2 space-y-4">
{% for card in agent_cards %}
{% set agent = card.agent %}
<div class="forge-card border border-forge-iron rounded-2xl p-5">
<div class="flex items-start justify-between gap-3">
<div>
<div class="flex items-center gap-2 flex-wrap">
<span class="text-lg font-semibold tracking-tight">{{ agent.name }}</span>
{% for v in card.versions %}
<span class="text-[11px] px-2 py-0.5 rounded border font-mono
{% if v.state == 'active' %}bg-emerald-950 text-emerald-400 border-emerald-900
{% elif v.state == 'draft' %}bg-amber-950/60 text-amber-400 border-amber-900
{% else %}bg-forge-surface2 text-forge-muted border-forge-iron{% endif %}">
{{ v.id }} · {{ v.state }}
</span>
{% endfor %}
</div>
<p class="text-sm text-forge-muted mt-2 max-w-xl">{{ agent.purpose }}</p>
</div>
</div>
<div class="mt-3 flex flex-wrap gap-x-6 gap-y-1 text-xs text-forge-muted">
<span>LLM: <span class="font-mono text-forge-steel">{{ agent.llm.provider }}/{{ agent.llm.model }}</span></span>
<span>HITL threshold: <span class="font-mono text-forge-steel">risk ≥ {{ agent.risk_threshold_for_hitl }}</span></span>
<span>Policy:
{% for g in agent.guardrails %}
<span class="font-mono text-forge-steel">{{ g }}</span>{% if not loop.last %}, {% endif %}
{% endfor %}
</span>
<span>Owner: {{ agent.owner }}</span>
</div>
<details class="mt-3 group">
<summary class="cursor-pointer select-none text-xs text-forge-muted hover:text-forge-ember">
<span class="group-open:hidden">Show system prompt and output schema</span>
<span class="hidden group-open:inline">Hide system prompt and output schema</span>
</summary>
<div class="mt-2 p-3 bg-black/60 border border-forge-iron rounded-xl text-xs whitespace-pre-wrap font-light leading-relaxed">{{ agent.system_prompt }}</div>
<pre class="mt-2 text-[10px] bg-black/40 border border-forge-iron p-3 rounded-xl overflow-auto max-h-44 text-forge-steel">{{ agent.output_schema | tojson(indent=2) }}</pre>
</details>
</div>
{% else %}
<div class="forge-card border border-forge-iron rounded-2xl p-8 text-center text-forge-muted">No agents registered.</div>
{% endfor %}
</div>
<div class="space-y-4">
{% for p in policies %}
<div class="forge-card border border-forge-iron rounded-2xl p-5">
<div class="flex items-baseline justify-between gap-2">
<div class="font-medium">{{ p.name }} <span class="text-xs text-forge-muted font-mono">{{ p.version }}</span></div>
<span class="text-[10px] px-2 py-0.5 rounded border font-mono
{{ 'bg-red-950/60 border-red-900 text-red-400' if p.on_validator_error == 'fail_closed' else 'bg-forge-surface2 border-forge-iron text-forge-steel' }}">
{{ p.on_validator_error }}
</span>
</div>
<p class="text-xs text-forge-muted mt-1">{{ p.description }}</p>
<div class="mt-3 text-[11px]">
<div class="uppercase tracking-[1.5px] text-forge-muted mb-1">Input validators</div>
<div class="flex flex-wrap gap-1">
{% for v in p.input_validators %}
<span class="px-2 py-0.5 bg-black/40 border border-forge-iron rounded font-mono">{{ v.type }}</span>
{% endfor %}
</div>
<div class="uppercase tracking-[1.5px] text-forge-muted mb-1 mt-2">Output validators</div>
<div class="flex flex-wrap gap-1">
{% for v in p.output_validators %}
<span class="px-2 py-0.5 bg-black/40 border border-forge-iron rounded font-mono">{{ v.type }}</span>
{% endfor %}
</div>
</div>
</div>
{% else %}
<div class="forge-card border border-forge-iron rounded-2xl p-6 text-center text-forge-muted text-sm">No policies registered.</div>
{% endfor %}
</div>
</div>
@@ -0,0 +1,53 @@
<div class="space-y-4">
{% for ex in pending %}
<div class="forge-card border border-amber-900/60 rounded-2xl p-5">
<div class="flex justify-between items-start">
<div>
<div class="font-medium">{{ ex.agent_name }} <span class="text-forge-muted">v{{ ex.agent_version }}</span></div>
<div class="text-xs text-forge-muted font-mono mt-0.5">{{ ex.trace_id }}</div>
</div>
<div class="text-amber-400 text-xs font-mono tracking-wide">Pending approval</div>
</div>
{% if ex.needs_human_for %}
<div class="mt-4 space-y-1.5">
<div class="uppercase text-xs tracking-[1.5px] text-forge-muted">Actions awaiting approval</div>
{% for a in ex.needs_human_for %}
<div class="text-xs bg-black/50 border border-forge-iron rounded p-2.5">
<div>
<span class="font-mono">{{ a.action }}</span><span class="font-mono text-forge-steel">{{ a.target }}</span>
<span class="font-mono {{ 'text-amber-400' if a.risk_score < 5 else 'text-red-400' }}">(risk={{ a.risk_score }})</span>
</div>
<div class="text-forge-muted mt-1">Rollback: {{ a.rollback_plan }}</div>
</div>
{% endfor %}
</div>
{% endif %}
<div class="mt-4 flex gap-3">
<button
hx-post="/api/executions/{{ ex.trace_id }}/approve"
hx-ext="json-enc"
hx-vals='{"approved_action_ids": [], "comment": "Approved from UI"}'
hx-swap="none"
hx-on::after-request="htmx.trigger(document.body, 'forja-refresh')"
class="px-4 py-1.5 text-xs font-medium bg-emerald-600 hover:bg-emerald-500 text-white rounded transition">
Approve all
</button>
<button
hx-post="/api/executions/{{ ex.trace_id }}/reject"
hx-ext="json-enc"
hx-vals='{"reason": "Rejected from web UI"}'
hx-swap="none"
hx-on::after-request="htmx.trigger(document.body, 'forja-refresh')"
class="px-4 py-1.5 text-xs font-medium bg-red-700 hover:bg-red-600 text-white rounded transition">
Reject
</button>
</div>
</div>
{% else %}
<div class="forge-card border border-forge-iron rounded-2xl p-6 text-center text-forge-muted text-sm">
No runs are waiting for approval. Run the demo incident in stage <a href="#run" class="text-forge-ember hover:underline">02 — Run</a>.
</div>
{% endfor %}
</div>
@@ -0,0 +1,27 @@
<div class="space-y-3">
{% for ex in runs %}
<div class="forge-card border border-forge-iron rounded-2xl p-4">
<div class="flex items-center justify-between gap-4">
<div>
<span class="font-medium">{{ ex.agent_name }}</span>
<span class="text-xs text-forge-muted">v{{ ex.agent_version }}</span>
<span class="text-xs text-forge-muted font-mono ml-2">{{ ex.trace_id }}</span>
</div>
<div class="flex items-center gap-4 text-xs shrink-0">
<span class="text-forge-muted">{{ ex.n_proposed_actions }} action(s)</span>
<span class="{{ 'text-amber-400' if ex.n_violations > 0 else 'text-forge-muted' }}">{{ ex.n_violations }} violation(s)</span>
<span class="text-emerald-400 font-mono">Completed</span>
<span class="text-forge-muted">{{ ex.finished_at.strftime('%H:%M:%S') if ex.finished_at else '' }}</span>
<a href="/api/executions/{{ ex.trace_id }}" target="_blank" class="text-forge-ember hover:underline">JSON →</a>
</div>
</div>
</div>
{% else %}
<div class="forge-card border border-forge-iron rounded-2xl p-6 text-center text-forge-muted text-sm">
No completed runs yet. They appear here after stage <a href="#approve" class="text-forge-ember hover:underline">03 — Approve</a> (or directly when no approval is needed).
</div>
{% endfor %}
{% if total_completed > runs | length %}
<div class="text-xs text-forge-muted text-center pt-1">Showing the {{ runs | length }} most recent of {{ total_completed }} completed runs.</div>
{% endif %}
</div>
@@ -0,0 +1,61 @@
<div class="space-y-4">
{% for item in pending %}
{% set req = item.request %}
{% set ev = item.evaluation %}
<div class="forge-card border border-amber-900/60 rounded-2xl p-5">
<div class="flex justify-between items-start gap-4">
<div>
<div class="font-medium text-lg">{{ req.agent_name }} <span class="text-forge-muted">{{ req.agent_version }} → active</span></div>
<div class="text-xs text-forge-muted font-mono mt-1">request {{ req.id }}</div>
</div>
<div class="text-amber-400 text-xs font-mono tracking-wide shrink-0">Pending promotion</div>
</div>
<div class="mt-4 grid grid-cols-2 sm:grid-cols-3 gap-3 text-xs">
<div>
<span class="text-forge-muted uppercase tracking-wide">Training run</span>
<div class="font-mono mt-0.5 truncate">{{ req.training_run_id }}</div>
</div>
<div>
<span class="text-forge-muted uppercase tracking-wide">Evaluation</span>
<div class="mt-0.5 {% if ev and ev.passed %}text-emerald-400{% else %}text-red-400{% endif %}">
{% if ev %}{{ "Passed" if ev.passed else "Failed" }} ({{ ev.agent_name }}@{{ ev.agent_version }}){% else %}Unknown{% endif %}
</div>
</div>
<div>
<span class="text-forge-muted uppercase tracking-wide">Scenarios</span>
<div class="mt-0.5">{% if ev %}{{ ev.scenario_count }} run, {{ ev.violation_count }} blocking{% else %}—{% endif %}</div>
</div>
</div>
<div class="text-xs text-forge-muted mt-3">
Requested by {{ req.requested_by }} · {{ req.created_at.strftime('%Y-%m-%d %H:%M') if req.created_at else '' }}
</div>
<div class="mt-4 flex gap-3">
<button
hx-post="/api/promotions/{{ req.id }}/approve"
hx-ext="json-enc"
hx-vals='{"approved_by": "web-ui", "comment": "Approved from Promotions UI"}'
hx-swap="none"
hx-on::after-request="htmx.trigger(document.body, 'forja-refresh')"
class="px-4 py-1.5 text-xs font-medium bg-emerald-600 hover:bg-emerald-500 text-white rounded transition">
Approve promotion
</button>
<button
hx-post="/api/promotions/{{ req.id }}/reject"
hx-ext="json-enc"
hx-vals='{"rejected_by": "web-ui", "reason": "Rejected from Promotions UI"}'
hx-swap="none"
hx-on::after-request="htmx.trigger(document.body, 'forja-refresh')"
class="px-4 py-1.5 text-xs font-medium bg-red-700 hover:bg-red-600 text-white rounded transition">
Reject
</button>
</div>
</div>
{% else %}
<div class="forge-card border border-forge-iron rounded-2xl p-6 text-center text-forge-muted text-sm">
No promotion requests are waiting. Request one from stage <a href="#train" class="text-forge-ember hover:underline">04 — Train</a> after a passing evaluation.
</div>
{% endfor %}
</div>
@@ -0,0 +1,98 @@
<div class="space-y-4">
{% for item in items %}
{% set run = item.run %}
{% set ev = item.evaluation %}
<div class="forge-card border border-forge-iron rounded-2xl p-5">
<div class="flex justify-between items-start gap-4">
<div>
<div class="font-medium">
{{ run.spec.dataset_name }}@{{ run.spec.dataset_version }}
<span class="text-forge-muted"></span>
{{ run.spec.model_name }}@{{ run.spec.model_version }}
</div>
<div class="text-xs text-forge-muted font-mono mt-1">run {{ run.id }}</div>
{% if run.spec.agent_name %}
<div class="text-xs text-forge-muted mt-1">
Candidate: <span class="font-mono text-forge-steel">{{ run.spec.agent_name }}{% if run.spec.agent_version %}@{{ run.spec.agent_version }}{% endif %}</span>
{% if item.candidate_state %}
<span class="ml-1 {{ 'text-emerald-400' if item.candidate_state == 'active' else 'text-amber-400' }}">({{ item.candidate_state }})</span>
{% endif %}
</div>
{% endif %}
</div>
<span class="text-xs font-mono tracking-wide shrink-0
{{ 'text-emerald-400' if run.status == 'succeeded'
else ('text-red-400' if run.status in ['failed', 'cancelled']
else 'text-amber-400') }}">
{{ run.status }}
</span>
</div>
{% if run.error_message %}
<p class="mt-2 text-xs text-red-400">{{ run.error_message }}</p>
{% endif %}
{% if ev %}
<div class="mt-3 text-xs">
<span class="uppercase tracking-wide text-forge-muted">Evaluation</span>
<span class="ml-2 {{ 'text-emerald-400' if ev.passed else 'text-red-400' }}">
{{ "Passed" if ev.passed else "Failed" }}
</span>
<span class="text-forge-muted ml-2">{{ ev.scenario_count }} scenario(s), {{ ev.violation_count }} blocking violation(s) — covers {{ ev.agent_name }}@{{ ev.agent_version }}</span>
{% if ev.notes %}<span class="text-forge-muted ml-2">— {{ ev.notes }}</span>{% endif %}
</div>
{% endif %}
<div class="mt-4 flex flex-wrap items-center gap-3">
{% if run.status not in ['succeeded', 'failed', 'cancelled'] %}
<button
hx-post="/api/training/runs/{{ run.id }}/refresh"
hx-ext="json-enc"
hx-vals='{}'
hx-swap="none"
hx-on::after-request="htmx.trigger(document.body, 'forja-refresh')"
class="px-4 py-1.5 text-xs font-medium bg-forge-surface2 border border-forge-iron hover:border-forge-ember rounded transition">
Refresh status
</button>
{% endif %}
{% if run.status == 'succeeded' and run.spec.agent_name and not ev %}
<button
hx-post="/api/training/runs/{{ run.id }}/evaluate"
hx-ext="json-enc"
hx-vals='{}'
hx-swap="none"
hx-on::after-request="htmx.trigger(document.body, 'forja-refresh')"
class="px-4 py-1.5 text-xs font-medium bg-forge-ember text-black hover:brightness-105 rounded transition">
Run evaluation
</button>
<span class="text-xs text-forge-muted">Runs the candidate over its canonical scenarios.</span>
{% endif %}
{% if ev and ev.passed and run.spec.agent_version %}
{% if item.candidate_state == 'active' %}
<span class="text-xs text-emerald-400">✓ Promoted — {{ run.spec.agent_name }}@{{ run.spec.agent_version }} is active (see stage 01).</span>
{% elif item.promotion_pending %}
<span class="text-xs text-amber-400">Promotion pending approval in stage <a href="#promote" class="underline">05 — Promote</a>.</span>
{% else %}
<button
hx-post="/api/promotions"
hx-ext="json-enc"
hx-vals='{"agent_name": "{{ run.spec.agent_name }}", "agent_version": "{{ run.spec.agent_version }}", "training_run_id": "{{ run.id }}", "evaluation_report_id": "{{ ev.id }}", "requested_by": "web-ui"}'
hx-swap="none"
hx-on::after-request="htmx.trigger(document.body, 'forja-refresh')"
class="px-4 py-1.5 text-xs font-medium bg-emerald-600 hover:bg-emerald-500 text-white rounded transition">
Request promotion
</button>
{% endif %}
{% elif ev and ev.passed and not run.spec.agent_version %}
<span class="text-xs text-forge-muted">Set a candidate version on the run to request promotion.</span>
{% endif %}
</div>
</div>
{% else %}
<div class="forge-card border border-forge-iron rounded-2xl p-6 text-center text-forge-muted text-sm">
No training runs yet. Submit one above — the candidate version is prefilled with the agent's draft.
</div>
{% endfor %}
</div>
@@ -0,0 +1,95 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Forja • {{ title or "Forja" }}</title>
<style>
:root {
--forge-bg: #1c1c1c;
--forge-surface: #151515;
--forge-surface2: #242424;
--forge-iron: #333333;
--forge-ember: #f97316;
--forge-gold: #fcd34d;
--forge-steel: #94a3b8;
--forge-text: #e7e5e4;
--forge-muted: #78716c;
}
</style>
<script src="https://cdn.tailwindcss.com"></script>
<script>
/* Must run right after the CDN script (before the body is parsed) so the
custom palette applies to the first paint. Single source of truth:
the CSS variables above. */
tailwind.config = {
theme: {
extend: {
colors: {
'forge': {
'bg': 'var(--forge-bg)',
'surface': 'var(--forge-surface)',
'surface2': 'var(--forge-surface2)',
'iron': 'var(--forge-iron)',
'ember': 'var(--forge-ember)',
'gold': 'var(--forge-gold)',
'steel': 'var(--forge-steel)',
'text': 'var(--forge-text)',
'muted': 'var(--forge-muted)',
}
}
}
}
};
</script>
<script src="https://unpkg.com/htmx.org@2.0.3/dist/htmx.min.js"></script>
<!-- Encode HTMX requests as JSON so buttons/forms can target the /api endpoints. -->
<script src="https://unpkg.com/htmx-ext-json-enc@2.0.2/json-enc.js"></script>
<style>
html { scroll-behavior: smooth; scroll-padding-top: 5rem; }
body { font-family: ui-sans-serif, system-ui, sans-serif; }
.htmx-indicator { display: none; }
.htmx-request .htmx-indicator { display: inline; }
.htmx-request.htmx-indicator { display: inline; }
body {
background-color: var(--forge-bg);
color: var(--forge-text);
}
.forge-header {
background-color: var(--forge-surface);
border-color: var(--forge-iron);
}
.forge-card {
background-color: var(--forge-surface);
border-color: var(--forge-iron);
}
</style>
</head>
<body>
<div class="min-h-screen">
<header class="forge-header border-b sticky top-0 z-50">
<div class="max-w-5xl mx-auto px-6 h-14 flex items-center justify-between">
<a href="/" class="flex flex-col hover:text-forge-ember transition">
<span class="font-semibold tracking-wide text-lg">Forja</span>
<span class="text-[10px] text-forge-muted">Governed models and agents</span>
</a>
<nav class="flex items-center gap-x-5 text-sm">
<a href="/#define" class="hover:text-forge-ember transition">Define</a>
<a href="/#run" class="hover:text-forge-ember transition">Run</a>
<a href="/#approve" class="hover:text-forge-ember transition">Approve</a>
<a href="/#train" class="hover:text-forge-ember transition">Train</a>
<a href="/#promote" class="hover:text-forge-ember transition">Promote</a>
<a href="/#audit" class="hover:text-forge-ember transition">Audit</a>
</nav>
</div>
</header>
<main class="max-w-5xl mx-auto px-6 py-8 text-forge-text">
{% block content %}{% endblock %}
</main>
</div>
</body>
</html>
@@ -0,0 +1,223 @@
{% extends "base.html" %}
{% macro stage_header(number, anchor, name, description) %}
<div class="flex items-center gap-3 mb-1 pt-2" id="{{ anchor }}">
<span class="font-mono text-xs px-2 py-px border border-forge-ember/60 text-forge-ember rounded bg-forge-surface2">{{ number }}</span>
<h2 class="text-2xl font-semibold tracking-tight">{{ name }}</h2>
</div>
<p class="text-sm text-forge-muted mb-4 max-w-2xl">{{ description }}</p>
{% endmacro %}
{% block content %}
<div class="pb-16">
<div class="mb-10">
<h1 class="text-3xl font-medium tracking-tight">The governed agent lifecycle, on one page.</h1>
<p class="text-sm text-forge-muted mt-2 max-w-2xl">
Six stages, top to bottom. Every action below is a click — run the demo incident,
approve it, train the draft candidate, evaluate it, and promote it to active.
Each panel refreshes itself after every action.
</p>
</div>
<!-- ============================ 01 DEFINE ============================ -->
{{ stage_header("01", "define", "Define",
"Agents and the guardrail policies they run under — all versioned YAML.
The draft version below is the candidate that the rest of this page trains,
evaluates, and promotes.") }}
<div id="agents-panel"
hx-get="/fragments/agents"
hx-trigger="forja-refresh from:body"
hx-swap="innerHTML">
{% include "_agents.html" %}
</div>
<!-- ============================ 02 RUN =============================== -->
<div class="mt-12"></div>
{{ stage_header("02", "run", "Run",
"Execute the active agent through the governed pipeline: input guardrails →
LLM → output guardrails → proposed actions → approval gate. The demo incident
proposes a risk-5 action, so the run pauses for human approval in stage 03.") }}
<div class="forge-card border border-forge-iron rounded-2xl p-6">
<div class="flex flex-wrap items-center gap-3 mb-5 pb-5 border-b border-forge-iron">
<button
hx-post="/run"
hx-vals='{"agent_name": "incident_analyzer", "input": "{{ demo_input }}"}'
hx-target="#run-result"
hx-swap="innerHTML"
hx-on::after-request="htmx.trigger(document.body, 'forja-refresh')"
class="px-5 py-2.5 bg-forge-ember text-black font-semibold rounded-lg text-sm hover:brightness-105 active:scale-[0.985] transition">
▶ Run demo incident
<span class="htmx-indicator"></span>
</button>
<span class="text-xs text-forge-muted">One click — sends a replicated HSS capacity alarm and pauses at the approval gate.</span>
</div>
<form hx-post="/run" hx-target="#run-result" hx-swap="innerHTML"
hx-on::after-request="htmx.trigger(document.body, 'forja-refresh')"
class="space-y-4">
<div class="grid grid-cols-1 sm:grid-cols-2 gap-4">
<div>
<label class="block text-xs tracking-widest text-forge-muted mb-1.5 uppercase">Agent</label>
<select name="agent_name" id="agent-select"
class="w-full bg-black/50 border border-forge-iron rounded-lg px-3 py-2.5 text-sm focus:border-forge-ember outline-none">
{% for a in agents %}
<option value="{{ a.name }}"
data-label="{{ a.input_label or 'Input' }}"
data-placeholder="{{ a.input_placeholder or '' }}">
{{ a.name }} (v{{ a.version }})
</option>
{% endfor %}
</select>
</div>
<div>
<label id="input-label" class="block text-xs tracking-widest text-forge-muted mb-1.5 uppercase">Input</label>
<textarea name="input" id="input-text" rows="3"
class="w-full bg-black/50 border border-forge-iron rounded-lg px-3 py-2 font-mono text-sm focus:border-forge-ember outline-none"></textarea>
</div>
</div>
<button type="submit"
class="px-5 py-2 bg-forge-surface2 border border-forge-iron hover:border-forge-ember rounded-lg text-sm transition">
Run with custom input
<span class="htmx-indicator"></span>
</button>
</form>
</div>
<div id="run-result" class="mt-4"></div>
<!-- ============================ 03 APPROVE =========================== -->
<div class="mt-12"></div>
{{ stage_header("03", "approve", "Approve",
"Runs paused by the approval gate (HITL). The state lives in a checkpoint on
disk, so it survives restarts. Approving resumes the graph exactly where it
stopped; the completed run lands in stage 06.") }}
<div id="approvals-panel"
hx-get="/fragments/approvals"
hx-trigger="forja-refresh from:body"
hx-swap="innerHTML">
{% include "_approvals.html" %}
</div>
<!-- ============================ 04 TRAIN ============================= -->
<div class="mt-12"></div>
{{ stage_header("04", "train", "Train & evaluate",
"Submit a fine-tuning job for the draft candidate to the configured MLOps
backend, then evaluate it over the agent's canonical scenarios through the
full governed runtime. A passing evaluation unlocks promotion.") }}
<div class="forge-card border border-forge-iron rounded-2xl p-6 mb-4">
<form hx-post="/api/training/runs"
hx-ext="json-enc"
hx-swap="none"
hx-on::after-request="htmx.trigger(document.body, 'forja-refresh');
this.querySelector('.form-error').textContent =
event.detail.successful ? '' : 'Request failed: ' + event.detail.xhr.responseText.slice(0, 200)"
class="grid grid-cols-2 sm:grid-cols-4 gap-4 items-end">
<div>
<label class="block text-xs tracking-widest text-forge-muted mb-1.5 uppercase">Dataset</label>
<select name="dataset_name" class="w-full bg-black/50 border border-forge-iron rounded-lg px-3 py-2 text-sm focus:border-forge-ember outline-none">
{% for d in datasets %}
<option value="{{ d.name }}">{{ d.name }}</option>
{% endfor %}
</select>
</div>
<div>
<label class="block text-xs tracking-widest text-forge-muted mb-1.5 uppercase">Base model</label>
<select name="model_name" class="w-full bg-black/50 border border-forge-iron rounded-lg px-3 py-2 text-sm focus:border-forge-ember outline-none">
{% for m in models %}
<option value="{{ m.name }}">{{ m.name }}</option>
{% endfor %}
</select>
</div>
<div>
<label class="block text-xs tracking-widest text-forge-muted mb-1.5 uppercase">Agent</label>
<select name="agent_name" id="train-agent-select"
class="w-full bg-black/50 border border-forge-iron rounded-lg px-3 py-2 text-sm focus:border-forge-ember outline-none">
{% for card in agent_cards %}
{% set draft = card.versions | selectattr('state', 'equalto', 'draft') | map(attribute='id') | first %}
<option value="{{ card.agent.name }}" data-draft="{{ draft or '' }}">{{ card.agent.name }}</option>
{% endfor %}
</select>
</div>
<div>
<label class="block text-xs tracking-widest text-forge-muted mb-1.5 uppercase">Candidate version</label>
<input name="agent_version" id="candidate-version" type="text"
class="w-full bg-black/50 border border-forge-iron rounded-lg px-3 py-2 text-sm font-mono focus:border-forge-ember outline-none">
</div>
<div class="col-span-2 sm:col-span-4 flex items-center gap-4">
<button type="submit"
class="px-5 py-2 bg-forge-ember text-black font-semibold rounded-lg text-sm hover:brightness-105 active:scale-[0.985] transition">
Submit training run
</button>
<span class="text-xs text-forge-muted">backend: <span class="font-mono">{{ backend }}</span> — the mock backend completes instantly.</span>
<span class="form-error text-xs text-red-400"></span>
</div>
</form>
</div>
<div id="training-panel"
hx-get="/fragments/training"
hx-trigger="forja-refresh from:body"
hx-swap="innerHTML">
{% include "_training.html" %}
</div>
<!-- ============================ 05 PROMOTE =========================== -->
<div class="mt-12"></div>
{{ stage_header("05", "promote", "Promote",
"Governance queue. A promotion requires a succeeded training run and a passing
evaluation of the exact candidate version. Approving flips the candidate to
active in the registry — check stage 01 after approving.") }}
<div id="promotions-panel"
hx-get="/fragments/promotions"
hx-trigger="forja-refresh from:body"
hx-swap="innerHTML">
{% include "_promotions.html" %}
</div>
<!-- ============================ 06 AUDIT ============================= -->
<div class="mt-12"></div>
{{ stage_header("06", "audit", "Audit",
"Completed runs, persisted append-only with their full decision trail,
violations, and approved actions.") }}
<div id="history-panel"
hx-get="/fragments/history"
hx-trigger="forja-refresh from:body"
hx-swap="innerHTML">
{% include "_history.html" %}
</div>
</div>
<script>
/* Run form: apply the selected agent's input label and placeholder (from its YAML). */
const agentSelect = document.getElementById("agent-select");
const inputLabel = document.getElementById("input-label");
const inputText = document.getElementById("input-text");
function syncAgentHints() {
const opt = agentSelect.selectedOptions[0];
if (!opt) return;
inputLabel.textContent = opt.dataset.label || "Input";
inputText.placeholder = opt.dataset.placeholder || "";
}
agentSelect.addEventListener("change", syncAgentHints);
syncAgentHints();
/* Train form: prefill the candidate version with the agent's first draft version. */
const trainAgentSelect = document.getElementById("train-agent-select");
const candidateVersion = document.getElementById("candidate-version");
function syncCandidateVersion() {
const opt = trainAgentSelect.selectedOptions[0];
if (!opt) return;
candidateVersion.value = opt.dataset.draft || "";
}
trainAgentSelect.addEventListener("change", syncCandidateVersion);
syncCandidateVersion();
</script>
{% endblock %}
@@ -0,0 +1,68 @@
<div class="forge-card border border-forge-iron rounded-2xl p-5">
<div class="flex items-center justify-between">
<div class="flex items-center gap-2">
<span class="font-semibold tracking-tight">Execution result</span>
<span class="text-[10px] px-2 py-px rounded border font-mono {{ status_badge_style }} {{ status_text_color }}">{{ status_label }}</span>
</div>
<div class="text-[10px] text-forge-muted font-mono">trace {{ execution.trace_id }}</div>
</div>
<div class="mt-4">
<div class="uppercase text-xs tracking-[1.5px] text-forge-muted mb-1">Decision path</div>
<div class="font-mono text-[10px] bg-black/60 border border-forge-iron rounded p-2.5 space-y-px">
{% for step in execution.decision_path %}
<div class="flex justify-between text-xs py-0.5 border-b border-forge-iron/60">
<span class="font-mono text-forge-text">{{ step.step }}</span>
<span class="text-forge-muted">
{%- if step.duration_ms %}{{ step.duration_ms }}ms{% endif -%}
{%- if step.detail.get("n_approved") is not none %} · {{ step.detail["n_approved"] }} approved{% endif -%}
{%- if step.detail.get("rejected") %} · rejected{% endif -%}
{%- if step.detail.get("hitl") %} · HITL{% endif -%}
{%- if step.detail.get("n_violations") %} · {{ step.detail["n_violations"] }} violation(s){% endif -%}
</span>
</div>
{% else %}
<span class="text-forge-muted"></span>
{% endfor %}
</div>
</div>
{% if execution.violations %}
<div class="mt-4">
<div class="uppercase text-xs tracking-[1.5px] text-red-400 mb-1">Violations</div>
{% for v in execution.violations %}
<div class="text-xs {{ 'text-red-400' if v.severity == 'block' else 'text-amber-400' }}">
• {{ v.validator }}: {{ v.message }} ({{ v.severity }})
</div>
{% endfor %}
</div>
{% endif %}
{% if execution.proposed_actions %}
<div class="mt-4">
<div class="uppercase text-xs tracking-[1.5px] text-amber-400 mb-1">Proposed actions</div>
<div class="space-y-1">
{% for a in execution.proposed_actions %}
<div class="text-xs bg-black/50 border border-forge-iron rounded p-2">
• {{ a.action }} → <span class="font-mono">{{ a.target }}</span>
<span class="font-mono {{ 'text-emerald-400' if a.risk_score < 3 else ('text-amber-400' if a.risk_score < 5 else 'text-red-400') }}">(risk={{ a.risk_score }})</span>
</div>
{% endfor %}
</div>
</div>
{% endif %}
{% if execution.needs_human_for %}
<div class="mt-3 p-3 border border-amber-900 bg-amber-950/40 rounded text-amber-300 text-sm">
{{ execution.needs_human_for | length }} action(s) require approval.
<a href="#approve" class="underline hover:text-amber-200">Go to stage 03 — Approve ↓</a>
</div>
{% endif %}
{% if execution.status == "completed" %}
<div class="mt-4 text-emerald-400 text-sm">
Run finished successfully. {{ execution.proposed_actions | length }} proposed action(s).
<a href="#audit" class="underline">See it in stage 06 — Audit ↓</a>
</div>
{% endif %}
</div>
+294
View File
@@ -0,0 +1,294 @@
"""HTMX UI router — single-page lifecycle console mounted inside forja-core.
The UI is one page (``/``) organised as the six stages of the governed agent
lifecycle: Define → Run → Approve → Train → Promote → Audit. Each stage is a
fragment that re-fetches itself when any action button fires the global
``forja-refresh`` event, so the whole cycle can be driven with clicks only.
"""
from __future__ import annotations
from collections.abc import Callable, Coroutine
from pathlib import Path
from typing import Any
from uuid import UUID
from fastapi import APIRouter, Form, Request
from fastapi.responses import HTMLResponse, RedirectResponse
from fastapi.templating import Jinja2Templates
from forja_core.api.deps import (
get_dataset_store,
get_model_store,
get_orchestrator,
get_policy_store,
get_registry,
get_settings,
)
from forja_core.api.evaluation_persistence import load_evaluation, load_evaluation_for_run
from forja_core.api.executions import _load_index, _record_execution
from forja_core.api.persistence import append_execution, append_violation, read_execution_summaries
from forja_core.api.promotion_persistence import list_promotion_requests
from forja_core.api.training_persistence import list_training_runs
from forja_core.domain.execution import AgentExecution
router = APIRouter(prefix="", tags=["ui"])
# Template path works in dev and inside the container.
TEMPLATES_DIR = Path(__file__).parent / "templates"
templates = Jinja2Templates(directory=str(TEMPLATES_DIR))
_STATUS_BADGES = {
"completed": ("text-emerald-400", "bg-emerald-950/60 border-emerald-900", "Completed"),
"awaiting_approval": ("text-amber-400", "bg-amber-950/60 border-amber-900", "Pending approval"),
"failed": ("text-red-400", "bg-red-950/60 border-red-900", "Failed"),
"blocked_by_guardrail": ("text-red-400", "bg-red-950/60 border-red-900", "Blocked by policy"),
"running": ("text-forge-steel", "bg-forge-surface2 border-forge-iron", "Running"),
}
# Demo input for the one-click walkthrough: "hss" maps to a risk-5 action in the
# mock provider, so the run pauses in HITL and exercises the approval stage.
DEMO_INPUT = (
"Replicated capacity alarm on the HSS active-active pair in Madrid; "
"subscriber database replication lag is growing."
)
# --- Context builders (shared by the index page and the fragments) ---------------------
def _define_ctx() -> dict[str, Any]:
"""Agents with all their versions/states, plus the guardrail policies."""
registry = get_registry()
cards = []
for agent in registry.list_agents():
versions = []
for meta in registry.list_versions(agent.name):
try:
version_def = registry.get_version(agent.name, meta.id)
except FileNotFoundError:
continue
versions.append({"id": meta.id, "state": version_def.state, "message": meta.message})
cards.append({"agent": agent, "versions": versions})
return {"agent_cards": cards, "policies": get_policy_store().list_policies()}
async def _approvals_ctx() -> dict[str, Any]:
return {"pending": await _pending_executions()}
def _training_ctx() -> dict[str, Any]:
settings = get_settings()
registry = get_registry()
pending_promo_runs = {
str(req.training_run_id)
for req in list_promotion_requests(settings.data_dir, status="pending_approval")
}
items = []
for run in list_training_runs(settings.data_dir):
candidate_state = None
if run.spec.agent_name and run.spec.agent_version:
try:
candidate_state = registry.get_version(
run.spec.agent_name, run.spec.agent_version
).state
except FileNotFoundError:
candidate_state = None
items.append(
{
"run": run,
"evaluation": load_evaluation_for_run(settings.data_dir, run.id),
"candidate_state": candidate_state,
"promotion_pending": str(run.id) in pending_promo_runs,
}
)
return {
"items": items,
"datasets": get_dataset_store().list_datasets(),
"models": get_model_store().list_models(),
"backend": settings.training_backend,
}
def _promotions_ctx() -> dict[str, Any]:
data_dir = get_settings().data_dir
pending_items = []
for req in list_promotion_requests(data_dir, status="pending_approval"):
pending_items.append(
{"request": req, "evaluation": load_evaluation(data_dir, req.evaluation_report_id)}
)
return {"pending": pending_items}
def _history_ctx() -> dict[str, Any]:
data_dir = get_settings().data_dir
all_execs = read_execution_summaries(data_dir)
completed = [e for e in all_execs if e.status == "completed"]
completed.sort(key=lambda e: e.finished_at or e.started_at, reverse=True)
return {"runs": completed[:10], "total_completed": len(completed)}
async def _pending_executions() -> list[AgentExecution]:
"""Executions paused in HITL. They only live in the checkpointer (not in the
append-only JSONL, which records terminal runs), so they are reconstructed
from the execution index + a checkpointer snapshot."""
registry = get_registry()
policy_store = get_policy_store()
orchestrator = get_orchestrator()
data_dir = get_settings().data_dir
pending: list[AgentExecution] = []
for trace_id, meta in _load_index(data_dir).items():
try:
agent_def = registry.get_version(meta["agent_name"], meta["version"])
policy = policy_store.get_policy(agent_def.guardrails[0])
execution = await orchestrator.snapshot(
agent_def=agent_def, policy=policy, trace_id=UUID(trace_id)
)
except (FileNotFoundError, IndexError, KeyError, ValueError):
continue
if execution is not None and execution.status == "awaiting_approval":
pending.append(execution)
pending.sort(key=lambda e: e.started_at, reverse=True)
return pending
# --- The single page --------------------------------------------------------------------
@router.get("/", response_class=HTMLResponse)
async def index(request: Request) -> HTMLResponse:
registry = get_registry()
ctx: dict[str, Any] = {
"request": request,
"title": "Lifecycle",
"agents": registry.list_agents(),
"demo_input": DEMO_INPUT,
}
ctx.update(_define_ctx())
ctx.update(await _approvals_ctx())
ctx.update(_training_ctx())
ctx.update(_promotions_ctx())
ctx.update(_history_ctx())
return templates.TemplateResponse("index.html", ctx)
@router.post("/run", response_class=HTMLResponse)
async def run_invoke(
request: Request,
agent_name: str = Form(...),
input: str = Form(...),
) -> HTMLResponse:
"""Execute the agent using the orchestrator directly (same process)."""
registry = get_registry()
policy_store = get_policy_store()
orchestrator = get_orchestrator()
settings = get_settings()
try:
agent_def = registry.get_agent(agent_name)
policy_name = agent_def.guardrails[0] if agent_def.guardrails else "default"
policy = policy_store.get_policy(policy_name)
execution: AgentExecution = await orchestrator.invoke(
agent_def=agent_def,
policy=policy,
user_input=input,
)
# Persist like the API does: index always, JSONL only for terminal runs.
_record_execution(
settings.data_dir, str(execution.trace_id), agent_def.name, agent_def.version
)
if execution.status in {"completed", "failed", "blocked_by_guardrail"}:
append_execution(settings.data_dir, execution)
for v in execution.violations:
append_violation(settings.data_dir, v)
except Exception as e:
return HTMLResponse(
f'<div class="text-red-400 border border-red-900 bg-red-950/40 rounded p-3 text-sm">'
f"Execution error: {str(e)[:300]}</div>",
status_code=400,
)
text_color, badge_style, status_label = _STATUS_BADGES.get(
execution.status,
(
"text-forge-steel",
"bg-forge-surface2 border-forge-iron",
execution.status.replace("_", " ").title(),
),
)
return templates.TemplateResponse(
"run_result.html",
{
"request": request,
"execution": execution,
"status_label": status_label,
"status_text_color": text_color,
"status_badge_style": badge_style,
},
)
# --- Fragments (re-fetched on the global forja-refresh event) ---------------------------
@router.get("/fragments/agents", response_class=HTMLResponse)
async def fragment_agents(request: Request) -> HTMLResponse:
return templates.TemplateResponse("_agents.html", {"request": request, **_define_ctx()})
@router.get("/fragments/approvals", response_class=HTMLResponse)
async def fragment_approvals(request: Request) -> HTMLResponse:
return templates.TemplateResponse(
"_approvals.html", {"request": request, **(await _approvals_ctx())}
)
@router.get("/fragments/training", response_class=HTMLResponse)
async def fragment_training(request: Request) -> HTMLResponse:
return templates.TemplateResponse("_training.html", {"request": request, **_training_ctx()})
@router.get("/fragments/promotions", response_class=HTMLResponse)
async def fragment_promotions(request: Request) -> HTMLResponse:
return templates.TemplateResponse("_promotions.html", {"request": request, **_promotions_ctx()})
@router.get("/fragments/history", response_class=HTMLResponse)
async def fragment_history(request: Request) -> HTMLResponse:
return templates.TemplateResponse("_history.html", {"request": request, **_history_ctx()})
# --- Legacy routes from the multi-page UI → anchors on the single page ------------------
_LEGACY_ROUTES = {
"/agents": "/#define",
"/policies": "/#define",
"/run": "/#run",
"/approvals": "/#approve",
"/training": "/#train",
"/promotions": "/#promote",
"/history": "/#audit",
"/armory": "/#audit",
"/tutorial": "/",
}
def _redirect_endpoint(target: str) -> Callable[[], Coroutine[Any, Any, RedirectResponse]]:
async def _redirect() -> RedirectResponse:
return RedirectResponse(url=target, status_code=301)
return _redirect
for _path, _target in _LEGACY_ROUTES.items():
router.add_api_route(
_path, _redirect_endpoint(_target), methods=["GET"], include_in_schema=False
)
# Old per-agent detail pages → Define stage.
router.add_api_route(
"/agents/{name}", _redirect_endpoint("/#define"), methods=["GET"], include_in_schema=False
)
-31
View File
@@ -1,31 +0,0 @@
# syntax=docker/dockerfile:1.7
FROM python:3.11-slim
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
PIP_NO_CACHE_DIR=1
WORKDIR /app
RUN apt-get update && apt-get install -y --no-install-recommends \
curl \
&& rm -rf /var/lib/apt/lists/*
COPY dashboard/requirements.txt /app/requirements.txt
RUN pip install -r /app/requirements.txt
COPY dashboard/src /app/src
ENV PYTHONPATH=/app/src
RUN useradd --create-home --shell /bin/bash dash && chown -R dash:dash /app
USER dash
EXPOSE 8501
HEALTHCHECK --interval=10s --timeout=3s --start-period=10s --retries=3 \
CMD curl -fsS http://localhost:8501/_stcore/health || exit 1
CMD ["streamlit", "run", "/app/src/agentforge_dashboard/app.py", \
"--server.address=0.0.0.0", "--server.port=8501", \
"--server.headless=true", "--browser.gatherUsageStats=false"]
-4
View File
@@ -1,4 +0,0 @@
streamlit>=1.38,<2.0
httpx>=0.27,<0.28
pydantic>=2.7,<3.0
PyYAML>=6.0,<7.0
-47
View File
@@ -1,47 +0,0 @@
"""Entry point del dashboard Streamlit con sidebar de branding y health."""
from __future__ import annotations
import streamlit as st
from agentforge_dashboard.client import CoreClient
@st.cache_resource
def get_client() -> CoreClient:
return CoreClient()
def main() -> None:
st.set_page_config(
page_title="AgentForge",
page_icon="🛡️",
layout="wide",
initial_sidebar_state="expanded",
)
st.sidebar.markdown("## 🛡️ AgentForge")
st.sidebar.caption("Plataforma de gobernanza de agentes IA")
client = get_client()
try:
client.health()
st.sidebar.success("Core API: OK")
except Exception as exc:
st.sidebar.error(f"Core API unreachable: {exc}")
st.title("🛡️ AgentForge")
st.markdown(
"""
Bienvenido al panel de gobernanza de agentes IA.
Usa la barra lateral para navegar:
- **Registro**: catálogo de agentes y versiones.
- **Ejecutar**: lanzar un agente con guardrails completos.
- **Aprobaciones**: ejecuciones pausadas a la espera de revisión humana.
- **Historial**: trazas, violaciones y resultados.
- **Politicas**: políticas de guardrails y diff entre versiones.
"""
)
if __name__ == "__main__":
main()
@@ -1,71 +0,0 @@
"""Cliente HTTP tipado al core. Encapsula httpx y mapea respuestas a dicts."""
from __future__ import annotations
import os
from typing import Any
import httpx
class CoreClient:
"""Cliente sincrono (Streamlit es sync). 2 retries con httpx Transport."""
def __init__(self, base_url: str | None = None, timeout: float = 30.0) -> None:
self.base_url = base_url or os.getenv("AGENTFORGE_CORE_URL", "http://localhost:8000")
transport = httpx.HTTPTransport(retries=2)
self._client = httpx.Client(base_url=self.base_url, timeout=timeout, transport=transport)
def health(self) -> dict:
return self._get("/health")
def list_agents(self) -> list[dict]:
return self._get("/agents")
def get_agent(self, name: str) -> dict:
return self._get(f"/agents/{name}")
def list_versions(self, name: str) -> list[dict]:
return self._get(f"/agents/{name}/versions")
def diff_versions(self, name: str, v_from: str, v_to: str) -> dict:
return self._get(f"/agents/{name}/versions/{v_from}/diff/{v_to}")
def invoke_agent(self, name: str, body: dict) -> dict:
return self._post(f"/agents/{name}/invoke", body)
def get_execution(self, trace_id: str) -> dict:
return self._get(f"/executions/{trace_id}")
def list_executions(self) -> list[dict]:
return self._get("/executions")
def approve(self, trace_id: str, body: dict) -> dict:
return self._post(f"/executions/{trace_id}/approve", body)
def reject(self, trace_id: str, body: dict) -> dict:
return self._post(f"/executions/{trace_id}/reject", body)
def list_violations(self, **filters: Any) -> list[dict]:
params = {k: v for k, v in filters.items() if v is not None}
return self._get("/violations", params=params)
def list_policies(self) -> list[dict]:
return self._get("/policies")
def list_policy_versions(self, name: str) -> list[dict]:
return self._get(f"/policies/{name}/versions")
# ----- helpers privados -----
def _get(self, path: str, params: dict | None = None) -> Any:
r = self._client.get(path, params=params)
r.raise_for_status()
return r.json()
def _post(self, path: str, body: dict) -> Any:
r = self._client.post(path, json=body)
if r.status_code in {404, 409, 422}:
return {"error": r.json()}
r.raise_for_status()
return r.json()

Some files were not shown because too many files have changed in this diff Show More