Compare commits
17
Commits
4a8eb34874
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f67eb68e3e | ||
|
|
37633920ce | ||
|
|
7d0f10d21f | ||
|
|
34b11dfc66 | ||
|
|
d95ed15766 | ||
|
|
8c594f8ddf | ||
|
|
b4964261ec | ||
|
|
2f990ea636 | ||
|
|
4244b00ce7 | ||
|
|
dd5cfd9799 | ||
|
|
9377730de4 | ||
|
|
ce629139c2 | ||
|
|
57ba136c70 | ||
|
|
9dd2a8f9bd | ||
|
|
c860984ae4 | ||
|
|
c414e16357 | ||
|
|
eac2ed02a4 |
+18
-3
@@ -16,12 +16,27 @@ OPENAI_MODEL=gpt-4o
|
|||||||
|
|
||||||
# Guardrails
|
# Guardrails
|
||||||
GUARDRAILS_NEMO_ENABLED=false
|
GUARDRAILS_NEMO_ENABLED=false
|
||||||
|
# Keywords CSV para topical rails de NeMo (vacío = sin chequeo off-topic)
|
||||||
|
GUARDRAILS_NEMO_ALLOWED_KEYWORDS=
|
||||||
|
|
||||||
# Observabilidad
|
# Observabilidad
|
||||||
LOG_LEVEL=INFO
|
LOG_LEVEL=INFO
|
||||||
|
|
||||||
|
# Training orchestration (mock | azure_ml)
|
||||||
|
TRAINING_BACKEND=mock
|
||||||
|
|
||||||
|
# Azure ML (required when TRAINING_BACKEND=azure_ml and using live SDK; else stub mode)
|
||||||
|
AZURE_ML_SUBSCRIPTION_ID=
|
||||||
|
AZURE_ML_RESOURCE_GROUP=
|
||||||
|
AZURE_ML_WORKSPACE_NAME=
|
||||||
|
AZURE_ML_COMPUTE=cpu-cluster
|
||||||
|
AZURE_ML_ENVIRONMENT=AzureML-sklearn-1.5
|
||||||
|
AZURE_ML_EXPERIMENT_NAME=forja-training
|
||||||
|
AZURE_ML_COMMAND=python train.py
|
||||||
|
|
||||||
# Persistencia
|
# Persistencia
|
||||||
DATA_DIR=./data
|
DATA_DIR=./data
|
||||||
|
AGENTS_DIR=./agents
|
||||||
# URL del core (la usa el dashboard)
|
POLICIES_DIR=./policies
|
||||||
AGENTFORGE_CORE_URL=http://core:8000
|
DATASETS_DIR=./datasets
|
||||||
|
MODELS_DIR=./models
|
||||||
|
|||||||
+3
-5
@@ -4,11 +4,12 @@
|
|||||||
*.key
|
*.key
|
||||||
*.pem
|
*.pem
|
||||||
|
|
||||||
# Datos runtime (registry, checkpoints, logs)
|
# Datos runtime (registry, checkpoints, logs).
|
||||||
|
# Nota: sin patrón global *.jsonl — los records de datasets/ (p. ej.
|
||||||
|
# datasets/<name>/records/*.jsonl) son contenido versionado, no runtime.
|
||||||
data/
|
data/
|
||||||
*.sqlite
|
*.sqlite
|
||||||
*.sqlite-*
|
*.sqlite-*
|
||||||
*.jsonl
|
|
||||||
|
|
||||||
# Python
|
# Python
|
||||||
__pycache__/
|
__pycache__/
|
||||||
@@ -38,8 +39,5 @@ env/
|
|||||||
.DS_Store
|
.DS_Store
|
||||||
Thumbs.db
|
Thumbs.db
|
||||||
|
|
||||||
# Streamlit
|
|
||||||
.streamlit/secrets.toml
|
|
||||||
|
|
||||||
# Notas de hand-off entre sesiones de Claude Code
|
# Notas de hand-off entre sesiones de Claude Code
|
||||||
claude-session.txt
|
claude-session.txt
|
||||||
|
|||||||
+30
-19
@@ -1,26 +1,23 @@
|
|||||||
# AgentForge — Arquitectura técnica
|
# Forja — Arquitectura técnica
|
||||||
|
|
||||||
> Documento "vivo" con las decisiones técnicas. El spec original está en
|
> Documento vivo con las decisiones técnicas.
|
||||||
> [`docs/superpowers/specs/2026-05-09-agentforge-design.md`](docs/superpowers/specs/2026-05-09-agentforge-design.md).
|
|
||||||
|
|
||||||
## Visión general
|
## Visión general
|
||||||
|
|
||||||
Two-tier:
|
Forja es un **único servicio FastAPI** (:8000):
|
||||||
- `agentforge-core` — FastAPI 8000. Dominio de gobierno, runtime LangGraph,
|
- Todo el dominio de gobierno, runtime LangGraph, guardrails y persistencia.
|
||||||
guardrails y persistencia.
|
- UI embebida (HTMX) para uso humano + API REST completa bajo `/api`.
|
||||||
- `agentforge-dashboard` — Streamlit 8501. Cliente HTTP del core.
|
|
||||||
|
|
||||||
## Diagrama
|
## Diagrama
|
||||||
|
|
||||||
```
|
```
|
||||||
┌─────────────────────┐ HTTP/JSON ┌─────────────────────┐
|
┌────────────────────────────────────────────────────────────┐
|
||||||
│ Dashboard (Stream.) │ ───────────► │ Core (FastAPI) │
|
│ forja-core :8000 │
|
||||||
└─────────────────────┘ └─────────┬───────────┘
|
│ UI HTMX (humanos) + REST API (/api) + dominio completo │
|
||||||
│
|
│ │
|
||||||
┌─────────────────────────┬────────┴──────┬────────────┐
|
│ LLM (strategy) │ Guardrails (strategy) │ LangGraph │
|
||||||
▼ ▼ ▼ ▼
|
│ │ │ + checkpoints│
|
||||||
LLM layer Guardrails layer LangGraph State
|
└────────────────────────────────────────────────────────────┘
|
||||||
(Strategy pattern) (Strategy pattern) Runtime JSON+SQL+JSONL
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Patrones aplicados
|
## Patrones aplicados
|
||||||
@@ -38,11 +35,24 @@ Two-tier:
|
|||||||
|
|
||||||
| Capa | Tecnología | Por qué |
|
| Capa | Tecnología | Por qué |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| Agent Registry | JSON + YAML | Humano lo escribe (YAML); máquina lo cataloga (index.yaml). |
|
| Registries (agents/policies/datasets/models) | YAML por versión + index.yaml | Humano lo escribe; diff legible; hash SHA-256 normalizado. Base común en `registry/yaml_store.py`. |
|
||||||
| Versionado | YAML por versión + index.yaml | Diff legible; hash SHA-256 normalizado. |
|
|
||||||
| Logs de violaciones | JSONL append-only | Inmutable, auditable, fácil de shipear. |
|
| Logs de violaciones | JSONL append-only | Inmutable, auditable, fácil de shipear. |
|
||||||
| Logs de ejecuciones | JSONL append-only | Idem. |
|
| Logs de ejecuciones | JSONL append-only | Idem. |
|
||||||
| Checkpoints LangGraph | SQLite | Tool right; persistencia entre restarts (HITL). |
|
| Checkpoints LangGraph | SQLite | Tool right; persistencia entre restarts (HITL). |
|
||||||
|
| Training runs / evaluaciones / promociones | JSON por entidad bajo `data/` | Estado mutable consultable por id; los registros de promoción aprobada van además a JSONL append-only. |
|
||||||
|
|
||||||
|
## Ciclo MLOps (training → evaluación → promoción)
|
||||||
|
|
||||||
|
1. `POST /api/training/runs` envía un job al backend configurado
|
||||||
|
(`TRAINING_BACKEND=mock|azure_ml`) con dataset, modelo base y, opcionalmente,
|
||||||
|
el agente y su **versión candidata**.
|
||||||
|
2. `POST /api/training/runs/{id}/evaluate` ejecuta los escenarios canónicos
|
||||||
|
(`agents/<name>/examples/*.txt`) contra la versión candidata por el runtime
|
||||||
|
gobernado completo; pasa si no hay violaciones bloqueantes ni fallos.
|
||||||
|
3. `POST /api/promotions` crea la petición; exige run `succeeded`, evaluación
|
||||||
|
`passed` y que la evaluación cubra **exactamente** la versión a promocionar.
|
||||||
|
4. Un operador aprueba/rechaza (UI `/promotions`); al aprobar, la versión pasa a
|
||||||
|
`active` en el registry y queda un registro de auditoría append-only.
|
||||||
|
|
||||||
## HITL — Patrón
|
## HITL — Patrón
|
||||||
|
|
||||||
@@ -50,7 +60,7 @@ Se usa la API dinámica `interrupt(payload)` de LangGraph ≥0.2 dentro del
|
|||||||
nodo `approve_gate`. Sólo pausa cuando hay acciones de alto riesgo
|
nodo `approve_gate`. Sólo pausa cuando hay acciones de alto riesgo
|
||||||
(`risk_score >= threshold` o `requires_approval=True`). El cliente envía
|
(`risk_score >= threshold` o `requires_approval=True`). El cliente envía
|
||||||
`Command(resume={"approved_action_ids": [...]})` vía
|
`Command(resume={"approved_action_ids": [...]})` vía
|
||||||
`POST /executions/{trace_id}/approve`.
|
`POST /api/executions/{trace_id}/approve`.
|
||||||
|
|
||||||
## Observabilidad
|
## Observabilidad
|
||||||
|
|
||||||
@@ -60,7 +70,8 @@ nodo `approve_gate`. Sólo pausa cuando hay acciones de alto riesgo
|
|||||||
|
|
||||||
## Errores
|
## Errores
|
||||||
|
|
||||||
- Azure OpenAI / OpenAI: retry exponencial 1s/2s/4s, fallback opcional, luego
|
- Azure OpenAI / OpenAI: retry exponencial 1s/2s/4s; si `LLM_FALLBACK_PROVIDER`
|
||||||
|
está configurado, `FallbackLLMProvider` delega en el secundario; luego
|
||||||
`status=failed`.
|
`status=failed`.
|
||||||
- Validador: `fail_closed` por defecto.
|
- Validador: `fail_closed` por defecto.
|
||||||
- Schema mismatch en LLM output: 1 retry, luego `failed`.
|
- Schema mismatch en LLM output: 1 retry, luego `failed`.
|
||||||
|
|||||||
@@ -4,16 +4,16 @@ help:
|
|||||||
@echo "Targets: install lint format test test-all smoke up down logs clean"
|
@echo "Targets: install lint format test test-all smoke up down logs clean"
|
||||||
|
|
||||||
install:
|
install:
|
||||||
pip install -r core/requirements.txt -r dashboard/requirements.txt
|
pip install -r core/requirements.txt
|
||||||
pip install pytest pytest-asyncio ruff mypy freezegun
|
pip install pytest pytest-asyncio ruff mypy freezegun
|
||||||
|
|
||||||
lint:
|
lint:
|
||||||
ruff check core/src dashboard/src tests
|
ruff check core/src tests
|
||||||
mypy core/src
|
mypy core/src
|
||||||
|
|
||||||
format:
|
format:
|
||||||
ruff format core/src dashboard/src tests
|
ruff format core/src tests
|
||||||
ruff check --fix core/src dashboard/src tests
|
ruff check --fix core/src tests
|
||||||
|
|
||||||
test:
|
test:
|
||||||
pytest tests/unit -v
|
pytest tests/unit -v
|
||||||
|
|||||||
@@ -1,34 +1,37 @@
|
|||||||
# 🛡️ AgentForge
|
# 🔨 Forja
|
||||||
|
|
||||||
Plataforma profesional de **gobernanza de agentes IA**: catalogación, versionado
|
**Forja de agentes IA** — plataforma de gobernanza genérica para agentes de cualquier tipo.
|
||||||
de prompts y políticas, guardrails runtime, ejecución stateful con
|
Catalogación y versionado (estilo Git) de definiciones, políticas y guardrails;
|
||||||
Human-in-the-Loop, observabilidad y trazabilidad end-to-end.
|
ejecución controlada con runtime stateful, Human-in-the-Loop y observabilidad completa.
|
||||||
|
Úsala como plano de control en cualquier proyecto que necesite agentes gobernados.
|
||||||
## ¿Por qué?
|
|
||||||
|
|
||||||
Poner agentes IA en producción sin una capa de gobierno produce sistemas
|
|
||||||
opacos: prompts que cambian sin historial, validaciones inconsistentes,
|
|
||||||
acciones de alto impacto sin supervisión, sin auditoría de decisiones.
|
|
||||||
AgentForge aporta el plano de control mínimo que un equipo de plataforma
|
|
||||||
necesita antes de operar agentes con impacto real.
|
|
||||||
|
|
||||||
## Arquitectura
|
## Arquitectura
|
||||||
|
|
||||||
|
Forja es un **único servicio** (FastAPI en :8000) que expone:
|
||||||
|
|
||||||
|
- API REST completa bajo `/api` (agentes, políticas, ejecuciones, aprobaciones, violaciones, datasets, models, training runs, promotions).
|
||||||
|
- UI embebida de **una sola página** (HTMX + Tailwind + Jinja) que recorre el
|
||||||
|
ciclo de vida completo en seis etapas: **Define → Run → Approve → Train →
|
||||||
|
Promote → Audit**. Cada panel se refresca solo tras cada acción; las rutas
|
||||||
|
antiguas (`/agents`, `/run`, ...) redirigen a su etapa.
|
||||||
|
|
||||||
```
|
```
|
||||||
┌───────────────────────── docker-compose ──────────────────────────┐
|
┌───────────────────────────────────────────────────────────────┐
|
||||||
│ │
|
│ forja-core :8000 │
|
||||||
│ ┌─────────────────────┐ HTTP/JSON ┌────────────────────┐ │
|
│ ┌─────────────────────────────────────────────────────────┐ │
|
||||||
│ │ agentforge-dashboard│ ────────────────► │ agentforge-core │ │
|
│ │ UI HTMX (navegador humano) │ API REST (/api/*) │ │
|
||||||
│ │ Streamlit :8501 │ ◄──────────────── │ FastAPI :8000 │ │
|
│ └─────────────────────────────────────────────────────────┘ │
|
||||||
│ └─────────────────────┘ └────────────────────┘ │
|
│ Strategy factories (LLM / Guardrails / Registry) │
|
||||||
│ │
|
│ LangGraph runtime + SQLite checkpoints (HITL real) │
|
||||||
│ Strategy pattern (Protocol) para LLMProvider, GuardrailEngine, │
|
│ Persistencia: YAML versionado + JSONL + SQLite │
|
||||||
│ AgentRegistry. Persistencia mixta: YAML (definiciones), JSON │
|
└───────────────────────────────────────────────────────────────┘
|
||||||
│ (registry), JSONL (logs append-only), SQLite (checkpoints). │
|
|
||||||
└────────────────────────────────────────────────────────────────────┘
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Detalles completos en [`ARCHITECTURE.md`](ARCHITECTURE.md).
|
Documentación viva:
|
||||||
|
- [`ARCHITECTURE.md`](ARCHITECTURE.md) — decisiones técnicas.
|
||||||
|
- [`docs/explicacion.md`](docs/explicacion.md) — explicación didáctica completa.
|
||||||
|
- [`docs/componentes.md`](docs/componentes.md) — referencia de cableado a bajo nivel.
|
||||||
|
- [`docs/futuro.md`](docs/futuro.md) — roadmap.
|
||||||
|
|
||||||
## Quickstart
|
## Quickstart
|
||||||
|
|
||||||
@@ -37,38 +40,55 @@ cp .env.example .env
|
|||||||
docker compose up
|
docker compose up
|
||||||
```
|
```
|
||||||
|
|
||||||
Abre [http://localhost:8501](http://localhost:8501).
|
Abre [http://localhost:8000](http://localhost:8000).
|
||||||
|
|
||||||
Funciona out-of-the-box (`LLM_PROVIDER=mock`, sin API keys). Si quieres usar
|
Funciona out-of-the-box (`LLM_PROVIDER=mock`, sin API keys). Si quieres usar
|
||||||
Azure OpenAI real, edita `.env`.
|
Azure OpenAI real, edita `.env`.
|
||||||
|
|
||||||
## Demo guiada (3 pasos)
|
## Demo guiada: el ciclo completo en 6 clicks
|
||||||
|
|
||||||
1. **Registro** → ver `incident_analyzer` y comparar `v1` vs `v2`.
|
Todo ocurre en `http://localhost:8000`, de arriba abajo, sin escribir nada:
|
||||||
2. **Ejecutar** → seleccionar el escenario `01_sip_registration_drop` y
|
|
||||||
pulsar *Invocar*. Verás el grafo recorrer validate_input → llm_reason
|
1. **▶ Run demo incident** (etapa 02) — lanza un incidente HSS pregrabado por el
|
||||||
→ validate_output → propose_actions → approve_gate y pausarse en HITL.
|
pipeline gobernado. La acción propuesta tiene risk 5 → la ejecución se pausa
|
||||||
3. **Aprobaciones** → revisar las acciones propuestas (con risk_score y
|
en el approval gate.
|
||||||
rollback_plan), aprobar las seguras y comprobar que la ejecución
|
2. **Approve all** (etapa 03) — el grafo se reanuda desde su checkpoint y la
|
||||||
completa.
|
ejecución completa (aparece en la etapa 06 — Audit).
|
||||||
|
3. **Submit training run** (etapa 04) — el formulario viene pre-rellenado con el
|
||||||
|
dataset `incident_sft`, el modelo `gpt4o_lora_base` y la versión candidata
|
||||||
|
`v3` (draft). Con `TRAINING_BACKEND=mock` termina al instante.
|
||||||
|
4. **Run evaluation** (etapa 04) — ejecuta los escenarios canónicos del agente
|
||||||
|
(`agents/<name>/examples/`) sobre la candidata `v3`; pasa si no hay
|
||||||
|
violaciones bloqueantes.
|
||||||
|
5. **Request promotion** (etapa 04) — crea la petición de gobernanza, que
|
||||||
|
aparece en la etapa 05.
|
||||||
|
6. **Approve promotion** (etapa 05) — `v3` pasa a `active` en el registry;
|
||||||
|
compruébalo en los chips de versión de la etapa 01.
|
||||||
|
|
||||||
## Capacidades implementadas
|
## Capacidades implementadas
|
||||||
|
|
||||||
| Feature | Ubicación |
|
| Feature | Ubicación |
|
||||||
|---|---|
|
|---|---|
|
||||||
| Agent Registry | `core/src/agentforge_core/registry/repository.py` |
|
| Agent / Policy / Dataset / Model Registry | `core/src/forja_core/registry/` (base genérica `yaml_store.py`) |
|
||||||
| Versionado tipo Git | `agents/<name>/versions/` + `registry/versioning.py` |
|
| Versionado Git-like (YAML) | `agents/`, `policies/`, `datasets/`, `models/` |
|
||||||
| Guardrails runtime | `core/src/agentforge_core/guardrails/` |
|
| Guardrails runtime (strategy) | `core/src/forja_core/guardrails/` |
|
||||||
| LangGraph stateful + checkpointing | `core/src/agentforge_core/runtime/` |
|
| LangGraph + HITL + checkpoints | `core/src/forja_core/runtime/` |
|
||||||
| Human-in-the-Loop | nodo `approve_gate` + endpoints `/approve` y `/reject` |
|
| Observabilidad (trace + structlog) | `core/src/forja_core/observability/` |
|
||||||
| Observabilidad | `observability/logging.py` (structlog + trace_id) |
|
| LLM abstraction (+fallback opcional) | `core/src/forja_core/llm/` |
|
||||||
| LLM provider abstraction | `core/src/agentforge_core/llm/` |
|
| Training backends (mock / Azure ML) | `core/src/forja_core/training/` |
|
||||||
| Política versionada | `policies/default/versions/v1.yaml` |
|
| Evaluación post-training (escenarios) | `core/src/forja_core/evaluation/` |
|
||||||
|
| Promoción gobernada draft → active | `core/src/forja_core/api/promotions.py` |
|
||||||
|
| UI embebida (HTMX) | `core/src/forja_core/web/` (ruta raíz) |
|
||||||
|
| API REST completa | bajo `/api` en forja-core :8000 |
|
||||||
|
|
||||||
## Variables de entorno
|
## Variables de entorno
|
||||||
|
|
||||||
Ver [`.env.example`](.env.example).
|
Ver [`.env.example`](.env.example).
|
||||||
|
|
||||||
|
### Azure ML training
|
||||||
|
|
||||||
|
Set `TRAINING_BACKEND=azure_ml` and fill `AZURE_ML_*` workspace settings. Forja submits an Azure ML **command job** via `azure-ai-ml` using `DefaultAzureCredential` (Azure CLI, managed identity, etc.). Replace the bundled script in `core/src/forja_core/training/_azure_job_bundle/` or override `AZURE_ML_COMMAND` for your fine-tuning pipeline. Without workspace config, the backend stays in **stub mode** for local development.
|
||||||
|
|
||||||
## Roadmap
|
## Roadmap
|
||||||
|
|
||||||
Ver [`docs/futuro.md`](docs/futuro.md).
|
Ver [`docs/futuro.md`](docs/futuro.md).
|
||||||
@@ -76,16 +96,22 @@ Ver [`docs/futuro.md`](docs/futuro.md).
|
|||||||
## Estructura del repositorio
|
## Estructura del repositorio
|
||||||
|
|
||||||
```
|
```
|
||||||
agentforge/
|
forja/
|
||||||
├── core/ servicio FastAPI
|
├── core/ servicio FastAPI único (API + UI HTMX embebida)
|
||||||
├── dashboard/ servicio Streamlit
|
├── agents/ definiciones de agentes (YAML versionados)
|
||||||
├── agents/ definiciones declarativas (YAML)
|
├── policies/ políticas de guardrails (YAML versionados)
|
||||||
├── policies/ políticas de guardrails
|
├── datasets/ datasets para fine-tuning (YAML versionados)
|
||||||
├── data/ estado runtime (gitignored)
|
├── models/ modelos base / destino de entrenamiento (YAML versionados)
|
||||||
├── tests/ pytest unit + integration
|
├── data/ runtime state (gitignored)
|
||||||
└── docs/ documentación adicional
|
├── tests/
|
||||||
|
└── docs/ (explicacion.md, componentes.md, futuro.md, ARCHITECTURE.md)
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Development
|
||||||
|
|
||||||
|
- Implementation guidelines: [`karpathy.md`](karpathy.md)
|
||||||
|
- UI copy and language: [`docs/product-voice.md`](docs/product-voice.md) (English only in the web app)
|
||||||
|
|
||||||
## Tests
|
## Tests
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|||||||
@@ -10,4 +10,9 @@ versions:
|
|||||||
author: Juan
|
author: Juan
|
||||||
message: Añade detección de codec mismatch y refuerzo de prompts.
|
message: Añade detección de codec mismatch y refuerzo de prompts.
|
||||||
created_at: 2026-05-01T12:00:00Z
|
created_at: 2026-05-01T12:00:00Z
|
||||||
|
- id: v3
|
||||||
|
hash: pending
|
||||||
|
author: Juan
|
||||||
|
message: Candidata SFT sobre incident_sft (LoRA); pendiente de evaluación y promoción.
|
||||||
|
created_at: 2026-06-10T10:00:00Z
|
||||||
active_version: v2
|
active_version: v2
|
||||||
|
|||||||
@@ -37,4 +37,6 @@ output_schema:
|
|||||||
type: object
|
type: object
|
||||||
required: [id, action, target, risk_score, rollback_plan, requires_approval]
|
required: [id, action, target, risk_score, rollback_plan, requires_approval]
|
||||||
risk_threshold_for_hitl: 4
|
risk_threshold_for_hitl: 4
|
||||||
|
icon: "🤖"
|
||||||
|
color: "#64748b"
|
||||||
updated_at: 2026-04-12T10:00:00Z
|
updated_at: 2026-04-12T10:00:00Z
|
||||||
|
|||||||
@@ -35,4 +35,8 @@ output_schema:
|
|||||||
type: object
|
type: object
|
||||||
required: [id, action, target, risk_score, rollback_plan, requires_approval]
|
required: [id, action, target, risk_score, rollback_plan, requires_approval]
|
||||||
risk_threshold_for_hitl: 4
|
risk_threshold_for_hitl: 4
|
||||||
|
input_label: Incident description
|
||||||
|
input_placeholder: "Sudden 80% drop in SIP registrations after deploying image 4.7.2 on the CSCF cluster."
|
||||||
|
icon: "🤖"
|
||||||
|
color: "#f59e0b"
|
||||||
updated_at: 2026-05-01T12:00:00Z
|
updated_at: 2026-05-01T12:00:00Z
|
||||||
|
|||||||
@@ -0,0 +1,43 @@
|
|||||||
|
name: incident_analyzer
|
||||||
|
version: v3
|
||||||
|
owner: Juan
|
||||||
|
purpose: |
|
||||||
|
Candidata re-entrenada (SFT sobre incident_sft, LoRA) con la misma cobertura
|
||||||
|
que v2: codec mismatch, tormentas de registro post-handover, degradación de
|
||||||
|
MOS y saturación replicada en HSS active-active.
|
||||||
|
state: draft
|
||||||
|
guardrails: [default]
|
||||||
|
llm:
|
||||||
|
provider: mock
|
||||||
|
model: gpt-4o
|
||||||
|
temperature: 0.1
|
||||||
|
max_tokens: 2000
|
||||||
|
system_prompt: |
|
||||||
|
Eres un analista senior de operaciones de plataforma de voz virtualizada (IMS,
|
||||||
|
CSCF, SBC, HSS). Recibes una descripción de incidente. Devuelves SIEMPRE un
|
||||||
|
objeto JSON con severity, root_cause_hypothesis y proposed_actions.
|
||||||
|
Considera específicamente:
|
||||||
|
- Codec mismatch G.711/G.729 entre SBC peering y core IMS.
|
||||||
|
- Tormentas de registro post-handover con CSCF tras despliegues.
|
||||||
|
- Degradación de MOS en pools SBC durante picos.
|
||||||
|
- Saturación replicada en HSS active-active.
|
||||||
|
Prioriza hipótesis respaldadas por los ejemplos del dataset incident_sft.
|
||||||
|
Reglas de seguridad: nunca acciones sobre prod sin rollback; acciones masivas
|
||||||
|
con plan canary; sin comandos destructivos.
|
||||||
|
output_schema:
|
||||||
|
type: object
|
||||||
|
required: [severity, root_cause_hypothesis, proposed_actions]
|
||||||
|
properties:
|
||||||
|
severity: {type: string, enum: [low, medium, high, critical]}
|
||||||
|
root_cause_hypothesis: {type: string}
|
||||||
|
proposed_actions:
|
||||||
|
type: array
|
||||||
|
items:
|
||||||
|
type: object
|
||||||
|
required: [id, action, target, risk_score, rollback_plan, requires_approval]
|
||||||
|
risk_threshold_for_hitl: 4
|
||||||
|
input_label: Incident description
|
||||||
|
input_placeholder: "Sudden 80% drop in SIP registrations after deploying image 4.7.2 on the CSCF cluster."
|
||||||
|
icon: "🤖"
|
||||||
|
color: "#f59e0b"
|
||||||
|
updated_at: 2026-06-10T10:00:00Z
|
||||||
+3
-1
@@ -30,7 +30,9 @@ USER agent
|
|||||||
|
|
||||||
EXPOSE 8000
|
EXPOSE 8000
|
||||||
|
|
||||||
|
# Nota: Este servicio ahora sirve tanto la API REST como la UI HTMX (Opción A)
|
||||||
|
|
||||||
HEALTHCHECK --interval=10s --timeout=3s --start-period=15s --retries=3 \
|
HEALTHCHECK --interval=10s --timeout=3s --start-period=15s --retries=3 \
|
||||||
CMD curl -fsS http://localhost:8000/health || exit 1
|
CMD curl -fsS http://localhost:8000/health || exit 1
|
||||||
|
|
||||||
CMD ["uvicorn", "agentforge_core.main:app", "--host", "0.0.0.0", "--port", "8000"]
|
CMD ["uvicorn", "forja_core.main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||||
|
|||||||
@@ -9,14 +9,18 @@ langgraph-checkpoint-sqlite>=2.0,<3.0
|
|||||||
# AsyncSqliteSaver (langgraph-checkpoint-sqlite 2.x) usa Connection.is_alive(),
|
# AsyncSqliteSaver (langgraph-checkpoint-sqlite 2.x) usa Connection.is_alive(),
|
||||||
# eliminado en aiosqlite 0.21. Fijado a la última serie compatible.
|
# eliminado en aiosqlite 0.21. Fijado a la última serie compatible.
|
||||||
aiosqlite>=0.20,<0.21
|
aiosqlite>=0.20,<0.21
|
||||||
guardrails-ai>=0.5,<0.6
|
|
||||||
# typer 0.12.x (pin transitivo de guardrails-ai <0.6) rompe con click >= 8.2
|
|
||||||
# ("Secondary flag is not valid for non-boolean flag" al ejecutar `spacy download`
|
# ("Secondary flag is not valid for non-boolean flag" al ejecutar `spacy download`
|
||||||
# y la CLI de typer/weasel). Fijado a la última serie 8.1.
|
# y la CLI de typer/weasel). Fijado a la última serie 8.1.
|
||||||
click>=8.1,<8.2
|
|
||||||
presidio-analyzer>=2.2,<3.0
|
presidio-analyzer>=2.2,<3.0
|
||||||
presidio-anonymizer>=2.2,<3.0
|
presidio-anonymizer>=2.2,<3.0
|
||||||
openai>=1.50,<2.0
|
openai>=1.50,<2.0
|
||||||
|
azure-ai-ml>=1.16,<2.0
|
||||||
|
azure-identity>=1.16,<2.0
|
||||||
PyYAML>=6.0,<7.0
|
PyYAML>=6.0,<7.0
|
||||||
jsonschema>=4.0,<5.0
|
jsonschema>=4.0,<5.0
|
||||||
nemoguardrails>=0.10,<0.12
|
# nemoguardrails se reintroducirá cuando se integre Colang real (docs/futuro.md);
|
||||||
|
# el engine actual es un stub por keywords que no importa el paquete.
|
||||||
|
|
||||||
|
# HTMX UI embebida (Opción A)
|
||||||
|
jinja2>=3.1,<4.0
|
||||||
|
python-multipart>=0.0.9,<0.1
|
||||||
|
|||||||
@@ -1,23 +0,0 @@
|
|||||||
"""Router /policies: listado de políticas y de sus versiones."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from fastapi import APIRouter, HTTPException
|
|
||||||
|
|
||||||
from agentforge_core.api.deps import PolicyStoreDep
|
|
||||||
from agentforge_core.domain.policy import PolicyDefinition, PolicyVersionMeta
|
|
||||||
|
|
||||||
router = APIRouter()
|
|
||||||
|
|
||||||
|
|
||||||
@router.get("", response_model=list[PolicyDefinition])
|
|
||||||
def list_policies(store: PolicyStoreDep) -> list[PolicyDefinition]:
|
|
||||||
return store.list_policies()
|
|
||||||
|
|
||||||
|
|
||||||
@router.get("/{name}/versions", response_model=list[PolicyVersionMeta])
|
|
||||||
def list_versions(name: str, store: PolicyStoreDep) -> list[PolicyVersionMeta]:
|
|
||||||
try:
|
|
||||||
return store.list_versions(name)
|
|
||||||
except FileNotFoundError as exc:
|
|
||||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
|
||||||
@@ -1,33 +0,0 @@
|
|||||||
"""Factory que ensambla el GuardrailEngine según settings."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from agentforge_core.config import Settings
|
|
||||||
from agentforge_core.guardrails.base import GuardrailEngine
|
|
||||||
from agentforge_core.guardrails.composite import CompositeGuardrailEngine
|
|
||||||
from agentforge_core.guardrails.guardrails_ai import GuardrailsAIEngine
|
|
||||||
from agentforge_core.guardrails.nemo import NeMoGuardrailsEngine
|
|
||||||
|
|
||||||
|
|
||||||
def build_guardrail_engine(settings: Settings) -> GuardrailEngine:
|
|
||||||
"""Ensambla el engine: GuardrailsAIEngine siempre; NeMo si está habilitado."""
|
|
||||||
engines: list[GuardrailEngine] = [GuardrailsAIEngine()]
|
|
||||||
if settings.guardrails_nemo_enabled:
|
|
||||||
engines.append(
|
|
||||||
NeMoGuardrailsEngine(
|
|
||||||
allowed_keywords=[
|
|
||||||
"sip",
|
|
||||||
"ims",
|
|
||||||
"cscf",
|
|
||||||
"sbc",
|
|
||||||
"mos",
|
|
||||||
"hss",
|
|
||||||
"registro",
|
|
||||||
"incidente",
|
|
||||||
"voz",
|
|
||||||
"codec",
|
|
||||||
"rollback",
|
|
||||||
],
|
|
||||||
)
|
|
||||||
)
|
|
||||||
return CompositeGuardrailEngine(engines)
|
|
||||||
@@ -1,28 +0,0 @@
|
|||||||
"""Factory que selecciona el LLMProvider según settings."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from agentforge_core.config import Settings
|
|
||||||
from agentforge_core.llm.azure import AzureOpenAIProvider
|
|
||||||
from agentforge_core.llm.base import LLMProvider
|
|
||||||
from agentforge_core.llm.mock import MockProvider
|
|
||||||
from agentforge_core.llm.openai import OpenAIProvider
|
|
||||||
|
|
||||||
|
|
||||||
def build_llm_provider(settings: Settings) -> LLMProvider:
|
|
||||||
"""Materializa el proveedor activo. Falla en arranque si la config es inconsistente."""
|
|
||||||
match settings.llm_provider:
|
|
||||||
case "mock":
|
|
||||||
return MockProvider()
|
|
||||||
case "azure":
|
|
||||||
return AzureOpenAIProvider(
|
|
||||||
endpoint=settings.azure_openai_endpoint,
|
|
||||||
api_key=settings.azure_openai_api_key,
|
|
||||||
deployment=settings.azure_openai_deployment,
|
|
||||||
api_version=settings.azure_openai_api_version,
|
|
||||||
)
|
|
||||||
case "openai":
|
|
||||||
return OpenAIProvider(
|
|
||||||
api_key=settings.openai_api_key,
|
|
||||||
model=settings.openai_model,
|
|
||||||
)
|
|
||||||
@@ -1,37 +0,0 @@
|
|||||||
"""Composición raíz de la aplicación FastAPI (agentforge-core)."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from fastapi import FastAPI
|
|
||||||
|
|
||||||
from agentforge_core.api.middlewares import TraceIdMiddleware
|
|
||||||
from agentforge_core.config import Settings
|
|
||||||
from agentforge_core.observability.logging import configure_logging
|
|
||||||
|
|
||||||
|
|
||||||
def create_app() -> FastAPI:
|
|
||||||
"""Construye la app: logging, middleware de trace_id, /health y routers."""
|
|
||||||
settings = Settings()
|
|
||||||
configure_logging(level=settings.log_level)
|
|
||||||
app = FastAPI(
|
|
||||||
title="AgentForge Core",
|
|
||||||
version="0.1.0",
|
|
||||||
description="Plataforma de gobernanza de agentes IA — API REST.",
|
|
||||||
)
|
|
||||||
app.add_middleware(TraceIdMiddleware)
|
|
||||||
|
|
||||||
@app.get("/health", tags=["meta"])
|
|
||||||
async def health() -> dict[str, str]:
|
|
||||||
return {"status": "ok"}
|
|
||||||
|
|
||||||
from agentforge_core.api import agents, executions, policies, violations
|
|
||||||
|
|
||||||
app.include_router(agents.router, prefix="/agents", tags=["agents"])
|
|
||||||
app.include_router(executions.invoke_router, prefix="/agents", tags=["agents"])
|
|
||||||
app.include_router(executions.router, prefix="/executions", tags=["executions"])
|
|
||||||
app.include_router(policies.router, prefix="/policies", tags=["policies"])
|
|
||||||
app.include_router(violations.router, prefix="/violations", tags=["violations"])
|
|
||||||
return app
|
|
||||||
|
|
||||||
|
|
||||||
app = create_app()
|
|
||||||
@@ -1,17 +0,0 @@
|
|||||||
"""Factories para AgentRegistry y PolicyStore basados en Settings."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from agentforge_core.config import Settings
|
|
||||||
from agentforge_core.registry.policy_store import FileSystemPolicyStore
|
|
||||||
from agentforge_core.registry.repository import FileSystemAgentRegistry
|
|
||||||
|
|
||||||
|
|
||||||
def build_agent_registry(settings: Settings) -> FileSystemAgentRegistry:
|
|
||||||
"""Construye el registry de agentes apuntando a ``settings.agents_dir``."""
|
|
||||||
return FileSystemAgentRegistry(settings.agents_dir)
|
|
||||||
|
|
||||||
|
|
||||||
def build_policy_store(settings: Settings) -> FileSystemPolicyStore:
|
|
||||||
"""Construye el store de políticas apuntando a ``settings.policies_dir``."""
|
|
||||||
return FileSystemPolicyStore(settings.policies_dir)
|
|
||||||
@@ -1,64 +0,0 @@
|
|||||||
"""PolicyStore basado en sistema de ficheros (YAML por versión + index.yaml)."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from datetime import datetime
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import yaml
|
|
||||||
|
|
||||||
from agentforge_core.domain.policy import PolicyDefinition, PolicyVersionMeta
|
|
||||||
|
|
||||||
|
|
||||||
class FileSystemPolicyStore:
|
|
||||||
"""Lee políticas de policies/<name>/{index.yaml,versions/vN.yaml}."""
|
|
||||||
|
|
||||||
def __init__(self, root: Path) -> None:
|
|
||||||
self._root = Path(root)
|
|
||||||
|
|
||||||
def list_policies(self) -> list[PolicyDefinition]:
|
|
||||||
if not self._root.exists():
|
|
||||||
return []
|
|
||||||
result: list[PolicyDefinition] = []
|
|
||||||
for d in sorted(self._root.iterdir()):
|
|
||||||
if d.is_dir() and (d / "index.yaml").exists():
|
|
||||||
result.append(self.get_policy(d.name))
|
|
||||||
return result
|
|
||||||
|
|
||||||
def get_policy(self, name: str, version: str | None = None) -> PolicyDefinition:
|
|
||||||
index = self._load_index(name)
|
|
||||||
v = version or index["active_version"]
|
|
||||||
path = self._root / name / "versions" / f"{v}.yaml"
|
|
||||||
if not path.exists():
|
|
||||||
raise FileNotFoundError(f"policy version not found: {name}@{v}")
|
|
||||||
with path.open(encoding="utf-8") as f:
|
|
||||||
data = yaml.safe_load(f)
|
|
||||||
return PolicyDefinition.model_validate(data)
|
|
||||||
|
|
||||||
def list_versions(self, name: str) -> list[PolicyVersionMeta]:
|
|
||||||
index = self._load_index(name)
|
|
||||||
return [
|
|
||||||
PolicyVersionMeta(
|
|
||||||
id=v["id"],
|
|
||||||
hash=v["hash"],
|
|
||||||
author=v["author"],
|
|
||||||
message=v["message"],
|
|
||||||
created_at=self._parse_dt(v["created_at"]),
|
|
||||||
)
|
|
||||||
for v in index["versions"]
|
|
||||||
]
|
|
||||||
|
|
||||||
def _load_index(self, name: str) -> dict[str, Any]:
|
|
||||||
index_path = self._root / name / "index.yaml"
|
|
||||||
if not index_path.exists():
|
|
||||||
raise FileNotFoundError(f"policy not found: {name}")
|
|
||||||
with index_path.open(encoding="utf-8") as f:
|
|
||||||
data: dict[str, Any] = yaml.safe_load(f)
|
|
||||||
return data
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _parse_dt(value: str | datetime) -> datetime:
|
|
||||||
if isinstance(value, datetime):
|
|
||||||
return value
|
|
||||||
return datetime.fromisoformat(value.replace("Z", "+00:00"))
|
|
||||||
@@ -2,15 +2,22 @@
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from fastapi import APIRouter, HTTPException
|
from fastapi import APIRouter, HTTPException, status
|
||||||
|
from pydantic import BaseModel
|
||||||
|
|
||||||
from agentforge_core.api.deps import RegistryDep
|
from forja_core.api.deps import RegistryDep
|
||||||
from agentforge_core.domain.agent import AgentDefinition, AgentVersionMeta
|
from forja_core.domain.agent import AgentDefinition, AgentVersionMeta
|
||||||
from agentforge_core.registry.versioning import DiffResult
|
from forja_core.registry.versioning import DiffResult
|
||||||
|
|
||||||
router = APIRouter()
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
class CreateAgentVersionRequest(BaseModel):
|
||||||
|
agent: AgentDefinition
|
||||||
|
message: str
|
||||||
|
author: str = "dashboard"
|
||||||
|
|
||||||
|
|
||||||
@router.get("", response_model=list[AgentDefinition])
|
@router.get("", response_model=list[AgentDefinition])
|
||||||
def list_agents(registry: RegistryDep) -> list[AgentDefinition]:
|
def list_agents(registry: RegistryDep) -> list[AgentDefinition]:
|
||||||
return registry.list_agents()
|
return registry.list_agents()
|
||||||
@@ -43,3 +50,15 @@ def get_version(name: str, version: str, registry: RegistryDep) -> AgentDefiniti
|
|||||||
@router.get("/{name}/versions/{v_from}/diff/{v_to}", response_model=DiffResult)
|
@router.get("/{name}/versions/{v_from}/diff/{v_to}", response_model=DiffResult)
|
||||||
def diff_versions(name: str, v_from: str, v_to: str, registry: RegistryDep) -> DiffResult:
|
def diff_versions(name: str, v_from: str, v_to: str, registry: RegistryDep) -> DiffResult:
|
||||||
return registry.diff_versions(name, v_from, v_to)
|
return registry.diff_versions(name, v_from, v_to)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/{name}/versions", response_model=AgentVersionMeta, status_code=status.HTTP_201_CREATED
|
||||||
|
)
|
||||||
|
def create_agent_version(
|
||||||
|
name: str, req: CreateAgentVersionRequest, registry: RegistryDep
|
||||||
|
) -> AgentVersionMeta:
|
||||||
|
try:
|
||||||
|
return registry.upsert_version(name, req.agent, req.message, req.author)
|
||||||
|
except Exception as exc:
|
||||||
|
raise HTTPException(status_code=500, detail=str(exc)) from exc
|
||||||
@@ -0,0 +1,39 @@
|
|||||||
|
"""Dataset registry API (read-only)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from fastapi import APIRouter, HTTPException
|
||||||
|
|
||||||
|
from forja_core.api.deps import DatasetStoreDep
|
||||||
|
from forja_core.domain.dataset import DatasetDefinition, DatasetVersionMeta
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("", response_model=list[DatasetDefinition])
|
||||||
|
def list_datasets(store: DatasetStoreDep) -> list[DatasetDefinition]:
|
||||||
|
return store.list_datasets()
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{name}", response_model=DatasetDefinition)
|
||||||
|
def get_dataset(name: str, store: DatasetStoreDep) -> DatasetDefinition:
|
||||||
|
try:
|
||||||
|
return store.get_dataset(name)
|
||||||
|
except FileNotFoundError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{name}/versions", response_model=list[DatasetVersionMeta])
|
||||||
|
def list_versions(name: str, store: DatasetStoreDep) -> list[DatasetVersionMeta]:
|
||||||
|
try:
|
||||||
|
return store.list_versions(name)
|
||||||
|
except FileNotFoundError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{name}/versions/{version}", response_model=DatasetDefinition)
|
||||||
|
def get_version(name: str, version: str, store: DatasetStoreDep) -> DatasetDefinition:
|
||||||
|
try:
|
||||||
|
return store.get_version(name, version)
|
||||||
|
except FileNotFoundError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
@@ -12,15 +12,24 @@ from typing import Annotated
|
|||||||
|
|
||||||
from fastapi import Depends
|
from fastapi import Depends
|
||||||
|
|
||||||
from agentforge_core.config import Settings
|
from forja_core.config import Settings
|
||||||
from agentforge_core.guardrails.base import GuardrailEngine
|
from forja_core.guardrails.base import GuardrailEngine
|
||||||
from agentforge_core.guardrails.factory import build_guardrail_engine
|
from forja_core.guardrails.factory import build_guardrail_engine
|
||||||
from agentforge_core.llm.base import LLMProvider
|
from forja_core.llm.base import LLMProvider
|
||||||
from agentforge_core.llm.factory import build_llm_provider
|
from forja_core.llm.factory import build_llm_provider
|
||||||
from agentforge_core.registry.factory import build_agent_registry, build_policy_store
|
from forja_core.registry.dataset_store import FileSystemDatasetStore
|
||||||
from agentforge_core.registry.policy_store import FileSystemPolicyStore
|
from forja_core.registry.factory import (
|
||||||
from agentforge_core.registry.repository import FileSystemAgentRegistry
|
build_agent_registry,
|
||||||
from agentforge_core.runtime.orchestrator import AgentOrchestrator
|
build_dataset_store,
|
||||||
|
build_model_store,
|
||||||
|
build_policy_store,
|
||||||
|
)
|
||||||
|
from forja_core.registry.model_store import FileSystemModelStore
|
||||||
|
from forja_core.registry.policy_store import FileSystemPolicyStore
|
||||||
|
from forja_core.registry.repository import FileSystemAgentRegistry
|
||||||
|
from forja_core.runtime.orchestrator import AgentOrchestrator
|
||||||
|
from forja_core.training.base import TrainingBackend
|
||||||
|
from forja_core.training.factory import build_training_backend
|
||||||
|
|
||||||
|
|
||||||
@lru_cache(maxsize=1)
|
@lru_cache(maxsize=1)
|
||||||
@@ -48,6 +57,21 @@ def get_guardrail_engine() -> GuardrailEngine:
|
|||||||
return build_guardrail_engine(get_settings())
|
return build_guardrail_engine(get_settings())
|
||||||
|
|
||||||
|
|
||||||
|
@lru_cache(maxsize=1)
|
||||||
|
def get_dataset_store() -> FileSystemDatasetStore:
|
||||||
|
return build_dataset_store(get_settings())
|
||||||
|
|
||||||
|
|
||||||
|
@lru_cache(maxsize=1)
|
||||||
|
def get_model_store() -> FileSystemModelStore:
|
||||||
|
return build_model_store(get_settings())
|
||||||
|
|
||||||
|
|
||||||
|
@lru_cache(maxsize=1)
|
||||||
|
def get_training_backend() -> TrainingBackend:
|
||||||
|
return build_training_backend(get_settings())
|
||||||
|
|
||||||
|
|
||||||
@lru_cache(maxsize=1)
|
@lru_cache(maxsize=1)
|
||||||
def get_orchestrator() -> AgentOrchestrator:
|
def get_orchestrator() -> AgentOrchestrator:
|
||||||
settings = get_settings()
|
settings = get_settings()
|
||||||
@@ -63,3 +87,6 @@ SettingsDep = Annotated[Settings, Depends(get_settings)]
|
|||||||
RegistryDep = Annotated[FileSystemAgentRegistry, Depends(get_registry)]
|
RegistryDep = Annotated[FileSystemAgentRegistry, Depends(get_registry)]
|
||||||
PolicyStoreDep = Annotated[FileSystemPolicyStore, Depends(get_policy_store)]
|
PolicyStoreDep = Annotated[FileSystemPolicyStore, Depends(get_policy_store)]
|
||||||
OrchestratorDep = Annotated[AgentOrchestrator, Depends(get_orchestrator)]
|
OrchestratorDep = Annotated[AgentOrchestrator, Depends(get_orchestrator)]
|
||||||
|
DatasetStoreDep = Annotated[FileSystemDatasetStore, Depends(get_dataset_store)]
|
||||||
|
ModelStoreDep = Annotated[FileSystemModelStore, Depends(get_model_store)]
|
||||||
|
TrainingBackendDep = Annotated[TrainingBackend, Depends(get_training_backend)]
|
||||||
@@ -0,0 +1,47 @@
|
|||||||
|
"""Persist evaluation reports under data/evaluations/."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from forja_core.domain.training import EvaluationReport
|
||||||
|
|
||||||
|
|
||||||
|
def _eval_dir(data_dir: Path) -> Path:
|
||||||
|
return data_dir / "evaluations"
|
||||||
|
|
||||||
|
|
||||||
|
def save_evaluation(data_dir: Path, report: EvaluationReport) -> None:
|
||||||
|
directory = _eval_dir(data_dir)
|
||||||
|
directory.mkdir(parents=True, exist_ok=True)
|
||||||
|
path = directory / f"{report.id}.json"
|
||||||
|
path.write_text(
|
||||||
|
json.dumps(report.model_dump(mode="json"), indent=2, default=str),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
index_path = directory / "by_training_run.json"
|
||||||
|
index: dict[str, str] = {}
|
||||||
|
if index_path.exists():
|
||||||
|
index = json.loads(index_path.read_text(encoding="utf-8"))
|
||||||
|
index[str(report.training_run_id)] = str(report.id)
|
||||||
|
index_path.write_text(json.dumps(index, indent=2), encoding="utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def load_evaluation(data_dir: Path, report_id: UUID) -> EvaluationReport | None:
|
||||||
|
path = _eval_dir(data_dir) / f"{report_id}.json"
|
||||||
|
if not path.exists():
|
||||||
|
return None
|
||||||
|
return EvaluationReport.model_validate(json.loads(path.read_text(encoding="utf-8")))
|
||||||
|
|
||||||
|
|
||||||
|
def load_evaluation_for_run(data_dir: Path, training_run_id: UUID) -> EvaluationReport | None:
|
||||||
|
index_path = _eval_dir(data_dir) / "by_training_run.json"
|
||||||
|
if not index_path.exists():
|
||||||
|
return None
|
||||||
|
index = json.loads(index_path.read_text(encoding="utf-8"))
|
||||||
|
report_id = index.get(str(training_run_id))
|
||||||
|
if report_id is None:
|
||||||
|
return None
|
||||||
|
return load_evaluation(data_dir, UUID(report_id))
|
||||||
+68
-17
@@ -9,23 +9,23 @@ from uuid import UUID
|
|||||||
from fastapi import APIRouter, HTTPException
|
from fastapi import APIRouter, HTTPException
|
||||||
from pydantic import BaseModel, Field
|
from pydantic import BaseModel, Field
|
||||||
|
|
||||||
from agentforge_core.api.deps import (
|
from forja_core.api.deps import (
|
||||||
OrchestratorDep,
|
OrchestratorDep,
|
||||||
PolicyStoreDep,
|
PolicyStoreDep,
|
||||||
RegistryDep,
|
RegistryDep,
|
||||||
SettingsDep,
|
SettingsDep,
|
||||||
)
|
)
|
||||||
from agentforge_core.api.persistence import (
|
from forja_core.api.persistence import (
|
||||||
append_execution,
|
append_execution,
|
||||||
append_violation,
|
append_violation,
|
||||||
read_execution_summaries,
|
read_execution_summaries,
|
||||||
)
|
)
|
||||||
from agentforge_core.domain.agent import AgentDefinition
|
from forja_core.domain.agent import AgentDefinition
|
||||||
from agentforge_core.domain.execution import AgentExecution, AgentExecutionSummary
|
from forja_core.domain.execution import AgentExecution, AgentExecutionSummary
|
||||||
from agentforge_core.domain.policy import PolicyDefinition
|
from forja_core.domain.policy import PolicyDefinition
|
||||||
from agentforge_core.registry.policy_store import FileSystemPolicyStore
|
from forja_core.registry.policy_store import FileSystemPolicyStore
|
||||||
from agentforge_core.registry.repository import FileSystemAgentRegistry
|
from forja_core.registry.repository import FileSystemAgentRegistry
|
||||||
from agentforge_core.runtime.orchestrator import AgentOrchestrator
|
from forja_core.runtime.orchestrator import AgentOrchestrator
|
||||||
|
|
||||||
router = APIRouter()
|
router = APIRouter()
|
||||||
# POST /agents/{name}/invoke vive aquí pero se monta en main bajo el prefijo /agents.
|
# POST /agents/{name}/invoke vive aquí pero se monta en main bajo el prefijo /agents.
|
||||||
@@ -133,12 +133,8 @@ async def invoke_agent(
|
|||||||
except FileNotFoundError as exc:
|
except FileNotFoundError as exc:
|
||||||
raise HTTPException(status_code=422, detail=str(exc)) from exc
|
raise HTTPException(status_code=422, detail=str(exc)) from exc
|
||||||
|
|
||||||
execution = await orchestrator.invoke(
|
execution = await orchestrator.invoke(agent_def=agent_def, policy=policy, user_input=body.input)
|
||||||
agent_def=agent_def, policy=policy, user_input=body.input
|
_record_execution(settings.data_dir, str(execution.trace_id), agent_def.name, agent_def.version)
|
||||||
)
|
|
||||||
_record_execution(
|
|
||||||
settings.data_dir, str(execution.trace_id), agent_def.name, agent_def.version
|
|
||||||
)
|
|
||||||
if execution.status in _TERMINAL_STATUSES:
|
if execution.status in _TERMINAL_STATUSES:
|
||||||
append_execution(settings.data_dir, execution)
|
append_execution(settings.data_dir, execution)
|
||||||
for violation in execution.violations:
|
for violation in execution.violations:
|
||||||
@@ -146,9 +142,57 @@ async def invoke_agent(
|
|||||||
return execution
|
return execution
|
||||||
|
|
||||||
|
|
||||||
|
async def collect_execution_summaries(
|
||||||
|
registry: FileSystemAgentRegistry,
|
||||||
|
policies: FileSystemPolicyStore,
|
||||||
|
orchestrator: AgentOrchestrator,
|
||||||
|
data_dir: Path,
|
||||||
|
) -> list[AgentExecutionSummary]:
|
||||||
|
"""Resumen de ejecuciones: las terminales del JSONL más las pausadas en HITL.
|
||||||
|
|
||||||
|
Una ejecución en ``awaiting_approval`` no se escribe en el log append-only
|
||||||
|
(`executions.jsonl`); solo vive en el checkpointer. Aquí se reconstruyen esas
|
||||||
|
desde el índice `execution_index.json` + el checkpointer. Lo usan tanto la API
|
||||||
|
como la página de Approvals de la UI.
|
||||||
|
"""
|
||||||
|
summaries = read_execution_summaries(data_dir)
|
||||||
|
seen = {str(s.trace_id) for s in summaries}
|
||||||
|
for trace_id, meta in _load_index(data_dir).items():
|
||||||
|
if trace_id in seen:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
agent_def = registry.get_version(meta["agent_name"], meta["version"])
|
||||||
|
policy = policies.get_policy(agent_def.guardrails[0])
|
||||||
|
execution = await orchestrator.snapshot(
|
||||||
|
agent_def=agent_def, policy=policy, trace_id=UUID(trace_id)
|
||||||
|
)
|
||||||
|
except (FileNotFoundError, IndexError, KeyError, ValueError):
|
||||||
|
continue
|
||||||
|
if execution is None:
|
||||||
|
continue
|
||||||
|
summaries.append(
|
||||||
|
AgentExecutionSummary(
|
||||||
|
trace_id=execution.trace_id,
|
||||||
|
agent_name=execution.agent_name,
|
||||||
|
agent_version=execution.agent_version,
|
||||||
|
status=execution.status,
|
||||||
|
started_at=execution.started_at,
|
||||||
|
finished_at=execution.finished_at,
|
||||||
|
n_violations=len(execution.violations),
|
||||||
|
n_proposed_actions=len(execution.proposed_actions),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return summaries
|
||||||
|
|
||||||
|
|
||||||
@router.get("", response_model=list[AgentExecutionSummary])
|
@router.get("", response_model=list[AgentExecutionSummary])
|
||||||
def list_executions(settings: SettingsDep) -> list[AgentExecutionSummary]:
|
async def list_executions(
|
||||||
return read_execution_summaries(settings.data_dir)
|
registry: RegistryDep,
|
||||||
|
policies: PolicyStoreDep,
|
||||||
|
orchestrator: OrchestratorDep,
|
||||||
|
settings: SettingsDep,
|
||||||
|
) -> list[AgentExecutionSummary]:
|
||||||
|
return await collect_execution_summaries(registry, policies, orchestrator, settings.data_dir)
|
||||||
|
|
||||||
|
|
||||||
@router.get("/{trace_id}", response_model=AgentExecution)
|
@router.get("/{trace_id}", response_model=AgentExecution)
|
||||||
@@ -177,12 +221,19 @@ async def approve_execution(
|
|||||||
) -> AgentExecution:
|
) -> AgentExecution:
|
||||||
agent_def, policy = _resolve(registry, policies, settings.data_dir, trace_id)
|
agent_def, policy = _resolve(registry, policies, settings.data_dir, trace_id)
|
||||||
await _ensure_awaiting(orchestrator, agent_def, policy, trace_id)
|
await _ensure_awaiting(orchestrator, agent_def, policy, trace_id)
|
||||||
|
# Empty approved_action_ids from UI "Aprobar todo" → approve everything that was waiting
|
||||||
|
approved = body.approved_action_ids
|
||||||
|
if not approved:
|
||||||
|
# Try to resolve the actual ids from current snapshot (best effort)
|
||||||
|
snap = await orchestrator.snapshot(agent_def=agent_def, policy=policy, trace_id=trace_id)
|
||||||
|
if snap and snap.needs_human_for:
|
||||||
|
approved = [a.id for a in snap.needs_human_for]
|
||||||
execution = await orchestrator.resume(
|
execution = await orchestrator.resume(
|
||||||
agent_def=agent_def,
|
agent_def=agent_def,
|
||||||
policy=policy,
|
policy=policy,
|
||||||
trace_id=trace_id,
|
trace_id=trace_id,
|
||||||
decision={
|
decision={
|
||||||
"approved_action_ids": body.approved_action_ids,
|
"approved_action_ids": approved,
|
||||||
"comment": body.comment,
|
"comment": body.comment,
|
||||||
"rejected": False,
|
"rejected": False,
|
||||||
},
|
},
|
||||||
+2
-4
@@ -8,15 +8,13 @@ from starlette.middleware.base import BaseHTTPMiddleware, RequestResponseEndpoin
|
|||||||
from starlette.requests import Request
|
from starlette.requests import Request
|
||||||
from starlette.responses import Response
|
from starlette.responses import Response
|
||||||
|
|
||||||
from agentforge_core.observability.logging import bind_trace_id, clear_trace_id
|
from forja_core.observability.logging import bind_trace_id, clear_trace_id
|
||||||
|
|
||||||
|
|
||||||
class TraceIdMiddleware(BaseHTTPMiddleware):
|
class TraceIdMiddleware(BaseHTTPMiddleware):
|
||||||
"""Lee o genera `X-Trace-Id`, lo bind-ea al contexto de structlog y lo devuelve en la respuesta."""
|
"""Lee o genera `X-Trace-Id`, lo bind-ea al contexto de structlog y lo devuelve en la respuesta."""
|
||||||
|
|
||||||
async def dispatch(
|
async def dispatch(self, request: Request, call_next: RequestResponseEndpoint) -> Response:
|
||||||
self, request: Request, call_next: RequestResponseEndpoint
|
|
||||||
) -> Response:
|
|
||||||
trace_id = request.headers.get("X-Trace-Id") or str(uuid4())
|
trace_id = request.headers.get("X-Trace-Id") or str(uuid4())
|
||||||
bind_trace_id(trace_id)
|
bind_trace_id(trace_id)
|
||||||
try:
|
try:
|
||||||
@@ -0,0 +1,39 @@
|
|||||||
|
"""Foundation model registry API (read-only)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from fastapi import APIRouter, HTTPException
|
||||||
|
|
||||||
|
from forja_core.api.deps import ModelStoreDep
|
||||||
|
from forja_core.domain.foundation_model import ModelDefinition, ModelVersionMeta
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("", response_model=list[ModelDefinition])
|
||||||
|
def list_models(store: ModelStoreDep) -> list[ModelDefinition]:
|
||||||
|
return store.list_models()
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{name}", response_model=ModelDefinition)
|
||||||
|
def get_model(name: str, store: ModelStoreDep) -> ModelDefinition:
|
||||||
|
try:
|
||||||
|
return store.get_model(name)
|
||||||
|
except FileNotFoundError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{name}/versions", response_model=list[ModelVersionMeta])
|
||||||
|
def list_versions(name: str, store: ModelStoreDep) -> list[ModelVersionMeta]:
|
||||||
|
try:
|
||||||
|
return store.list_versions(name)
|
||||||
|
except FileNotFoundError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{name}/versions/{version}", response_model=ModelDefinition)
|
||||||
|
def get_version(name: str, version: str, store: ModelStoreDep) -> ModelDefinition:
|
||||||
|
try:
|
||||||
|
return store.get_version(name, version)
|
||||||
|
except FileNotFoundError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
+2
-2
@@ -8,8 +8,8 @@ from __future__ import annotations
|
|||||||
|
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from agentforge_core.domain.execution import AgentExecution, AgentExecutionSummary
|
from forja_core.domain.execution import AgentExecution, AgentExecutionSummary
|
||||||
from agentforge_core.domain.guardrail import GuardrailViolation
|
from forja_core.domain.guardrail import GuardrailViolation
|
||||||
|
|
||||||
|
|
||||||
def append_violation(data_dir: Path, violation: GuardrailViolation) -> None:
|
def append_violation(data_dir: Path, violation: GuardrailViolation) -> None:
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
"""Router /policies: listado de políticas y de sus versiones."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from fastapi import APIRouter, HTTPException, status
|
||||||
|
from pydantic import BaseModel
|
||||||
|
|
||||||
|
from forja_core.api.deps import PolicyStoreDep
|
||||||
|
from forja_core.domain.policy import PolicyDefinition, PolicyVersionMeta
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
class CreatePolicyVersionRequest(BaseModel):
|
||||||
|
policy: PolicyDefinition
|
||||||
|
message: str
|
||||||
|
author: str = "dashboard"
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("", response_model=list[PolicyDefinition])
|
||||||
|
def list_policies(store: PolicyStoreDep) -> list[PolicyDefinition]:
|
||||||
|
return store.list_policies()
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{name}/versions", response_model=list[PolicyVersionMeta])
|
||||||
|
def list_versions(name: str, store: PolicyStoreDep) -> list[PolicyVersionMeta]:
|
||||||
|
try:
|
||||||
|
return store.list_versions(name)
|
||||||
|
except FileNotFoundError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/{name}/versions", response_model=PolicyVersionMeta, status_code=status.HTTP_201_CREATED
|
||||||
|
)
|
||||||
|
def create_policy_version(
|
||||||
|
name: str, req: CreatePolicyVersionRequest, store: PolicyStoreDep
|
||||||
|
) -> PolicyVersionMeta:
|
||||||
|
try:
|
||||||
|
return store.upsert_version(name, req.policy, req.message, req.author)
|
||||||
|
except Exception as exc:
|
||||||
|
raise HTTPException(status_code=500, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/validators")
|
||||||
|
def list_validators() -> dict[str, list[str]]:
|
||||||
|
return {
|
||||||
|
"input": ["detect_pii", "prompt_injection", "toxic_language", "forbidden_topics"],
|
||||||
|
"output": [
|
||||||
|
"schema_match",
|
||||||
|
"pii_leakage",
|
||||||
|
"forbidden_action_keywords",
|
||||||
|
"telco_safety_rules",
|
||||||
|
],
|
||||||
|
}
|
||||||
@@ -0,0 +1,51 @@
|
|||||||
|
"""Persist promotion requests under data/promotions/."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from forja_core.domain.training import PromotionRecord, PromotionRequest, PromotionStatus
|
||||||
|
|
||||||
|
|
||||||
|
def _promo_dir(data_dir: Path) -> Path:
|
||||||
|
return data_dir / "promotions"
|
||||||
|
|
||||||
|
|
||||||
|
def save_promotion_request(data_dir: Path, request: PromotionRequest) -> None:
|
||||||
|
directory = _promo_dir(data_dir)
|
||||||
|
directory.mkdir(parents=True, exist_ok=True)
|
||||||
|
path = directory / f"{request.id}.json"
|
||||||
|
path.write_text(
|
||||||
|
json.dumps(request.model_dump(mode="json"), indent=2, default=str),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def load_promotion_request(data_dir: Path, request_id: UUID) -> PromotionRequest | None:
|
||||||
|
path = _promo_dir(data_dir) / f"{request_id}.json"
|
||||||
|
if not path.exists():
|
||||||
|
return None
|
||||||
|
return PromotionRequest.model_validate(json.loads(path.read_text(encoding="utf-8")))
|
||||||
|
|
||||||
|
|
||||||
|
def list_promotion_requests(
|
||||||
|
data_dir: Path,
|
||||||
|
status: PromotionStatus | None = None,
|
||||||
|
) -> list[PromotionRequest]:
|
||||||
|
directory = _promo_dir(data_dir)
|
||||||
|
if not directory.exists():
|
||||||
|
return []
|
||||||
|
out: list[PromotionRequest] = []
|
||||||
|
for path in sorted(directory.glob("*.json"), reverse=True):
|
||||||
|
req = PromotionRequest.model_validate(json.loads(path.read_text(encoding="utf-8")))
|
||||||
|
if status is None or req.status == status:
|
||||||
|
out.append(req)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def append_promotion_record(data_dir: Path, record: PromotionRecord) -> None:
|
||||||
|
data_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
with (data_dir / "promotion_records.jsonl").open("a", encoding="utf-8") as f:
|
||||||
|
f.write(record.model_dump_json() + "\n")
|
||||||
@@ -0,0 +1,202 @@
|
|||||||
|
"""Governance promotions: draft agent versions to active after eval gate."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from typing import Annotated
|
||||||
|
from uuid import UUID, uuid4
|
||||||
|
|
||||||
|
from fastapi import APIRouter, HTTPException, Query, status
|
||||||
|
from pydantic import BaseModel
|
||||||
|
|
||||||
|
from forja_core.api.deps import RegistryDep, SettingsDep
|
||||||
|
from forja_core.api.evaluation_persistence import load_evaluation
|
||||||
|
from forja_core.api.promotion_persistence import (
|
||||||
|
append_promotion_record,
|
||||||
|
list_promotion_requests,
|
||||||
|
load_promotion_request,
|
||||||
|
save_promotion_request,
|
||||||
|
)
|
||||||
|
from forja_core.api.training_persistence import load_training_run
|
||||||
|
from forja_core.config import Settings
|
||||||
|
from forja_core.domain.training import (
|
||||||
|
PromotionRecord,
|
||||||
|
PromotionRequest,
|
||||||
|
PromotionStatus,
|
||||||
|
TrainingRun,
|
||||||
|
)
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
class CreatePromotionRequest(BaseModel):
|
||||||
|
agent_name: str
|
||||||
|
agent_version: str
|
||||||
|
training_run_id: UUID
|
||||||
|
evaluation_report_id: UUID
|
||||||
|
requested_by: str = "api"
|
||||||
|
|
||||||
|
|
||||||
|
class PromotionDecision(BaseModel):
|
||||||
|
approved_by: str = "api"
|
||||||
|
comment: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class RejectPromotion(BaseModel):
|
||||||
|
rejected_by: str = "api"
|
||||||
|
reason: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
def _require_succeeded_run(run_id: UUID, settings: Settings) -> TrainingRun:
|
||||||
|
run = load_training_run(settings.data_dir, run_id)
|
||||||
|
if run is None:
|
||||||
|
raise HTTPException(status_code=404, detail="training run not found")
|
||||||
|
if run.status != "succeeded":
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=409,
|
||||||
|
detail=f"training run status '{run.status}' must be 'succeeded' for this operation",
|
||||||
|
)
|
||||||
|
return run
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("", response_model=list[PromotionRequest])
|
||||||
|
def list_promotions(
|
||||||
|
settings: SettingsDep,
|
||||||
|
status: Annotated[PromotionStatus | None, Query()] = None,
|
||||||
|
) -> list[PromotionRequest]:
|
||||||
|
return list_promotion_requests(settings.data_dir, status=status)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{request_id}", response_model=PromotionRequest)
|
||||||
|
def get_promotion(request_id: UUID, settings: SettingsDep) -> PromotionRequest:
|
||||||
|
req = load_promotion_request(settings.data_dir, request_id)
|
||||||
|
if req is None:
|
||||||
|
raise HTTPException(status_code=404, detail="promotion request not found")
|
||||||
|
return req
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("", response_model=PromotionRequest, status_code=status.HTTP_201_CREATED)
|
||||||
|
def request_promotion(
|
||||||
|
body: CreatePromotionRequest,
|
||||||
|
settings: SettingsDep,
|
||||||
|
registry: RegistryDep,
|
||||||
|
) -> PromotionRequest:
|
||||||
|
_require_succeeded_run(body.training_run_id, settings)
|
||||||
|
|
||||||
|
report = load_evaluation(settings.data_dir, body.evaluation_report_id)
|
||||||
|
if report is None:
|
||||||
|
raise HTTPException(status_code=404, detail="evaluation report not found")
|
||||||
|
if report.training_run_id != body.training_run_id:
|
||||||
|
raise HTTPException(status_code=422, detail="evaluation report does not match training run")
|
||||||
|
if not report.passed:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=422,
|
||||||
|
detail="evaluation did not pass; promotion blocked by guardrail gate",
|
||||||
|
)
|
||||||
|
|
||||||
|
try:
|
||||||
|
agent = registry.get_version(body.agent_name, body.agent_version)
|
||||||
|
except FileNotFoundError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
if agent.state == "active":
|
||||||
|
raise HTTPException(status_code=409, detail="agent version is already active")
|
||||||
|
if agent.name != report.agent_name:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=422, detail="evaluation agent does not match promotion target"
|
||||||
|
)
|
||||||
|
if report.agent_version != body.agent_version:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=422,
|
||||||
|
detail=(
|
||||||
|
f"evaluation covered version '{report.agent_version}' but promotion "
|
||||||
|
f"targets '{body.agent_version}'; evaluate the candidate version first"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
req = PromotionRequest(
|
||||||
|
id=uuid4(),
|
||||||
|
agent_name=body.agent_name,
|
||||||
|
agent_version=body.agent_version,
|
||||||
|
training_run_id=body.training_run_id,
|
||||||
|
evaluation_report_id=body.evaluation_report_id,
|
||||||
|
status="pending_approval",
|
||||||
|
requested_by=body.requested_by,
|
||||||
|
created_at=now,
|
||||||
|
updated_at=now,
|
||||||
|
)
|
||||||
|
save_promotion_request(settings.data_dir, req)
|
||||||
|
return req
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{request_id}/approve", response_model=PromotionRecord)
|
||||||
|
def approve_promotion(
|
||||||
|
request_id: UUID,
|
||||||
|
body: PromotionDecision,
|
||||||
|
settings: SettingsDep,
|
||||||
|
registry: RegistryDep,
|
||||||
|
) -> PromotionRecord:
|
||||||
|
req = load_promotion_request(settings.data_dir, request_id)
|
||||||
|
if req is None:
|
||||||
|
raise HTTPException(status_code=404, detail="promotion request not found")
|
||||||
|
if req.status != "pending_approval":
|
||||||
|
raise HTTPException(status_code=409, detail=f"promotion status is '{req.status}'")
|
||||||
|
|
||||||
|
report = load_evaluation(settings.data_dir, req.evaluation_report_id)
|
||||||
|
if report is None or not report.passed:
|
||||||
|
raise HTTPException(status_code=422, detail="evaluation gate no longer satisfied")
|
||||||
|
|
||||||
|
registry.promote_to_active(
|
||||||
|
req.agent_name,
|
||||||
|
req.agent_version,
|
||||||
|
author=body.approved_by,
|
||||||
|
comment=body.comment or "Approved via governance promotion",
|
||||||
|
)
|
||||||
|
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
updated_req = req.model_copy(
|
||||||
|
update={
|
||||||
|
"status": "approved",
|
||||||
|
"decided_by": body.approved_by,
|
||||||
|
"comment": body.comment,
|
||||||
|
"updated_at": now,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
save_promotion_request(settings.data_dir, updated_req)
|
||||||
|
|
||||||
|
record = PromotionRecord(
|
||||||
|
id=uuid4(),
|
||||||
|
agent_name=req.agent_name,
|
||||||
|
agent_version=req.agent_version,
|
||||||
|
training_run_id=req.training_run_id,
|
||||||
|
evaluation_report_id=req.evaluation_report_id,
|
||||||
|
approved_by=body.approved_by,
|
||||||
|
comment=body.comment,
|
||||||
|
created_at=now,
|
||||||
|
)
|
||||||
|
append_promotion_record(settings.data_dir, record)
|
||||||
|
return record
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{request_id}/reject", response_model=PromotionRequest)
|
||||||
|
def reject_promotion(
|
||||||
|
request_id: UUID,
|
||||||
|
body: RejectPromotion,
|
||||||
|
settings: SettingsDep,
|
||||||
|
) -> PromotionRequest:
|
||||||
|
req = load_promotion_request(settings.data_dir, request_id)
|
||||||
|
if req is None:
|
||||||
|
raise HTTPException(status_code=404, detail="promotion request not found")
|
||||||
|
if req.status != "pending_approval":
|
||||||
|
raise HTTPException(status_code=409, detail=f"promotion status is '{req.status}'")
|
||||||
|
|
||||||
|
updated = req.model_copy(
|
||||||
|
update={
|
||||||
|
"status": "rejected",
|
||||||
|
"decided_by": body.rejected_by,
|
||||||
|
"comment": body.reason,
|
||||||
|
"updated_at": datetime.now(UTC),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
save_promotion_request(settings.data_dir, updated)
|
||||||
|
return updated
|
||||||
@@ -0,0 +1,237 @@
|
|||||||
|
"""Training runs API — submit jobs to external MLOps backends."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from uuid import UUID, uuid4
|
||||||
|
|
||||||
|
from fastapi import APIRouter, HTTPException, status
|
||||||
|
from pydantic import BaseModel, field_validator
|
||||||
|
|
||||||
|
from forja_core.api.deps import (
|
||||||
|
DatasetStoreDep,
|
||||||
|
ModelStoreDep,
|
||||||
|
OrchestratorDep,
|
||||||
|
PolicyStoreDep,
|
||||||
|
RegistryDep,
|
||||||
|
SettingsDep,
|
||||||
|
TrainingBackendDep,
|
||||||
|
)
|
||||||
|
from forja_core.api.evaluation_persistence import load_evaluation_for_run, save_evaluation
|
||||||
|
from forja_core.api.training_persistence import (
|
||||||
|
list_training_runs,
|
||||||
|
load_training_run,
|
||||||
|
save_training_run,
|
||||||
|
)
|
||||||
|
from forja_core.domain.training import (
|
||||||
|
EvaluationReport,
|
||||||
|
TrainingHyperparameters,
|
||||||
|
TrainingJobSpec,
|
||||||
|
TrainingRun,
|
||||||
|
)
|
||||||
|
from forja_core.evaluation.post_train import run_post_train_evaluation
|
||||||
|
from forja_core.registry.dataset_store import FileSystemDatasetStore
|
||||||
|
from forja_core.registry.model_store import FileSystemModelStore
|
||||||
|
from forja_core.registry.repository import FileSystemAgentRegistry
|
||||||
|
from forja_core.training.base import TrainingBackend
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
class CreateTrainingRunRequest(BaseModel):
|
||||||
|
dataset_name: str
|
||||||
|
dataset_version: str | None = None
|
||||||
|
model_name: str
|
||||||
|
model_version: str | None = None
|
||||||
|
agent_name: str | None = None
|
||||||
|
agent_version: str | None = None
|
||||||
|
hyperparameters: TrainingHyperparameters | None = None
|
||||||
|
|
||||||
|
@field_validator(
|
||||||
|
"dataset_version", "model_version", "agent_name", "agent_version", mode="before"
|
||||||
|
)
|
||||||
|
@classmethod
|
||||||
|
def _empty_string_is_none(cls, value: object) -> object:
|
||||||
|
# HTML form selects submit "" for the unselected option.
|
||||||
|
return value or None
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_spec(
|
||||||
|
body: CreateTrainingRunRequest,
|
||||||
|
datasets: FileSystemDatasetStore,
|
||||||
|
models: FileSystemModelStore,
|
||||||
|
registry: FileSystemAgentRegistry,
|
||||||
|
) -> TrainingJobSpec:
|
||||||
|
try:
|
||||||
|
dataset = datasets.get_dataset(body.dataset_name, body.dataset_version)
|
||||||
|
except FileNotFoundError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
try:
|
||||||
|
model = models.get_model(body.model_name, body.model_version)
|
||||||
|
except FileNotFoundError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
if body.agent_version and not body.agent_name:
|
||||||
|
raise HTTPException(status_code=422, detail="agent_version requires agent_name")
|
||||||
|
if body.agent_name:
|
||||||
|
try:
|
||||||
|
registry.get_agent(body.agent_name, body.agent_version)
|
||||||
|
except FileNotFoundError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
return TrainingJobSpec(
|
||||||
|
dataset_name=dataset.name,
|
||||||
|
dataset_version=dataset.version,
|
||||||
|
model_name=model.name,
|
||||||
|
model_version=model.version,
|
||||||
|
agent_name=body.agent_name,
|
||||||
|
agent_version=body.agent_version,
|
||||||
|
hyperparameters=body.hyperparameters or TrainingHyperparameters(),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def _apply_backend_status(
|
||||||
|
backend: TrainingBackend,
|
||||||
|
run: TrainingRun,
|
||||||
|
) -> TrainingRun:
|
||||||
|
if run.external_job_id is None:
|
||||||
|
return run
|
||||||
|
job = await backend.status(run.external_job_id)
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
return run.model_copy(
|
||||||
|
update={
|
||||||
|
"status": job.status,
|
||||||
|
"updated_at": now,
|
||||||
|
"artifact_refs": job.artifacts,
|
||||||
|
"error_message": job.message if job.status == "failed" else None,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("", response_model=list[TrainingRun])
|
||||||
|
def list_runs(settings: SettingsDep) -> list[TrainingRun]:
|
||||||
|
return list_training_runs(settings.data_dir)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{run_id}", response_model=TrainingRun)
|
||||||
|
def get_run(run_id: UUID, settings: SettingsDep) -> TrainingRun:
|
||||||
|
run = load_training_run(settings.data_dir, run_id)
|
||||||
|
if run is None:
|
||||||
|
raise HTTPException(status_code=404, detail="training run not found")
|
||||||
|
return run
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("", response_model=TrainingRun, status_code=status.HTTP_201_CREATED)
|
||||||
|
async def create_run(
|
||||||
|
body: CreateTrainingRunRequest,
|
||||||
|
settings: SettingsDep,
|
||||||
|
datasets: DatasetStoreDep,
|
||||||
|
models: ModelStoreDep,
|
||||||
|
registry: RegistryDep,
|
||||||
|
backend: TrainingBackendDep,
|
||||||
|
) -> TrainingRun:
|
||||||
|
spec = _resolve_spec(body, datasets, models, registry)
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
run_id = uuid4()
|
||||||
|
run = TrainingRun(
|
||||||
|
id=run_id,
|
||||||
|
backend=settings.training_backend,
|
||||||
|
status="submitted",
|
||||||
|
spec=spec,
|
||||||
|
created_at=now,
|
||||||
|
updated_at=now,
|
||||||
|
)
|
||||||
|
external_id = await backend.submit(spec, run_id)
|
||||||
|
run = run.model_copy(
|
||||||
|
update={
|
||||||
|
"external_job_id": external_id,
|
||||||
|
"status": "queued",
|
||||||
|
"updated_at": datetime.now(UTC),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
run = await _apply_backend_status(backend, run)
|
||||||
|
save_training_run(settings.data_dir, run)
|
||||||
|
return run
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{run_id}/refresh", response_model=TrainingRun)
|
||||||
|
async def refresh_run(
|
||||||
|
run_id: UUID,
|
||||||
|
settings: SettingsDep,
|
||||||
|
backend: TrainingBackendDep,
|
||||||
|
) -> TrainingRun:
|
||||||
|
run = load_training_run(settings.data_dir, run_id)
|
||||||
|
if run is None:
|
||||||
|
raise HTTPException(status_code=404, detail="training run not found")
|
||||||
|
if run.backend != settings.training_backend:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=409,
|
||||||
|
detail=(
|
||||||
|
f"run backend '{run.backend}' does not match "
|
||||||
|
f"configured '{settings.training_backend}'"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
run = await _apply_backend_status(backend, run)
|
||||||
|
save_training_run(settings.data_dir, run)
|
||||||
|
return run
|
||||||
|
|
||||||
|
|
||||||
|
def _require_succeeded_run(run_id: UUID, settings: SettingsDep) -> TrainingRun:
|
||||||
|
run = load_training_run(settings.data_dir, run_id)
|
||||||
|
if run is None:
|
||||||
|
raise HTTPException(status_code=404, detail="training run not found")
|
||||||
|
if run.status != "succeeded":
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=409,
|
||||||
|
detail=f"training run status '{run.status}' must be 'succeeded'",
|
||||||
|
)
|
||||||
|
return run
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{run_id}/evaluate", response_model=EvaluationReport)
|
||||||
|
async def evaluate_run(
|
||||||
|
run_id: UUID,
|
||||||
|
settings: SettingsDep,
|
||||||
|
registry: RegistryDep,
|
||||||
|
policies: PolicyStoreDep,
|
||||||
|
orchestrator: OrchestratorDep,
|
||||||
|
) -> EvaluationReport:
|
||||||
|
run = _require_succeeded_run(run_id, settings)
|
||||||
|
agent_name = run.spec.agent_name
|
||||||
|
if not agent_name:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=422,
|
||||||
|
detail="training run has no agent_name; set it when creating the run",
|
||||||
|
)
|
||||||
|
|
||||||
|
existing = load_evaluation_for_run(settings.data_dir, run_id)
|
||||||
|
if existing is not None:
|
||||||
|
return existing
|
||||||
|
|
||||||
|
# Evaluate the candidate version tied to the run (falls back to active).
|
||||||
|
try:
|
||||||
|
agent_def = registry.get_agent(agent_name, run.spec.agent_version)
|
||||||
|
except FileNotFoundError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
if not agent_def.guardrails:
|
||||||
|
raise HTTPException(status_code=422, detail="agent has no guardrail policy")
|
||||||
|
try:
|
||||||
|
policy = policies.get_policy(agent_def.guardrails[0])
|
||||||
|
except FileNotFoundError as exc:
|
||||||
|
raise HTTPException(status_code=422, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
report = await run_post_train_evaluation(
|
||||||
|
training_run_id=run_id,
|
||||||
|
agent_def=agent_def,
|
||||||
|
policy=policy,
|
||||||
|
orchestrator=orchestrator,
|
||||||
|
agents_dir=settings.agents_dir,
|
||||||
|
)
|
||||||
|
save_evaluation(settings.data_dir, report)
|
||||||
|
return report
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{run_id}/evaluation", response_model=EvaluationReport)
|
||||||
|
def get_run_evaluation(run_id: UUID, settings: SettingsDep) -> EvaluationReport:
|
||||||
|
report = load_evaluation_for_run(settings.data_dir, run_id)
|
||||||
|
if report is None:
|
||||||
|
raise HTTPException(status_code=404, detail="evaluation not found for this training run")
|
||||||
|
return report
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
"""Persist training runs as one JSON file per run under data/training_runs/."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from forja_core.domain.training import TrainingRun
|
||||||
|
|
||||||
|
|
||||||
|
def _runs_dir(data_dir: Path) -> Path:
|
||||||
|
return data_dir / "training_runs"
|
||||||
|
|
||||||
|
|
||||||
|
def save_training_run(data_dir: Path, run: TrainingRun) -> None:
|
||||||
|
directory = _runs_dir(data_dir)
|
||||||
|
directory.mkdir(parents=True, exist_ok=True)
|
||||||
|
path = directory / f"{run.id}.json"
|
||||||
|
path.write_text(
|
||||||
|
json.dumps(run.model_dump(mode="json"), indent=2, default=str),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def load_training_run(data_dir: Path, run_id: UUID) -> TrainingRun | None:
|
||||||
|
path = _runs_dir(data_dir) / f"{run_id}.json"
|
||||||
|
if not path.exists():
|
||||||
|
return None
|
||||||
|
data = json.loads(path.read_text(encoding="utf-8"))
|
||||||
|
return TrainingRun.model_validate(data)
|
||||||
|
|
||||||
|
|
||||||
|
def list_training_runs(data_dir: Path) -> list[TrainingRun]:
|
||||||
|
directory = _runs_dir(data_dir)
|
||||||
|
if not directory.exists():
|
||||||
|
return []
|
||||||
|
runs: list[TrainingRun] = []
|
||||||
|
for path in sorted(directory.glob("*.json"), reverse=True):
|
||||||
|
data = json.loads(path.read_text(encoding="utf-8"))
|
||||||
|
runs.append(TrainingRun.model_validate(data))
|
||||||
|
return runs
|
||||||
@@ -7,9 +7,9 @@ from uuid import UUID
|
|||||||
|
|
||||||
from fastapi import APIRouter, Query
|
from fastapi import APIRouter, Query
|
||||||
|
|
||||||
from agentforge_core.api.deps import SettingsDep
|
from forja_core.api.deps import SettingsDep
|
||||||
from agentforge_core.api.persistence import read_violations
|
from forja_core.api.persistence import read_violations
|
||||||
from agentforge_core.domain.guardrail import GuardrailViolation
|
from forja_core.domain.guardrail import GuardrailViolation
|
||||||
|
|
||||||
router = APIRouter()
|
router = APIRouter()
|
||||||
|
|
||||||
@@ -35,16 +35,26 @@ class Settings(BaseSettings):
|
|||||||
|
|
||||||
# Guardrails
|
# Guardrails
|
||||||
guardrails_nemo_enabled: bool = False
|
guardrails_nemo_enabled: bool = False
|
||||||
|
# Comma-separated keywords for the NeMo topical-rails stub. Empty disables
|
||||||
|
# the off-topic check (no domain assumptions baked into the core).
|
||||||
|
guardrails_nemo_allowed_keywords: str = ""
|
||||||
|
|
||||||
# Observabilidad
|
# Observabilidad
|
||||||
log_level: Literal["DEBUG", "INFO", "WARNING", "ERROR"] = "INFO"
|
log_level: Literal["DEBUG", "INFO", "WARNING", "ERROR"] = "INFO"
|
||||||
|
|
||||||
|
# Training / MLOps orchestration
|
||||||
|
training_backend: Literal["mock", "azure_ml"] = "mock"
|
||||||
|
azure_ml_subscription_id: str = ""
|
||||||
|
azure_ml_resource_group: str = ""
|
||||||
|
azure_ml_workspace_name: str = ""
|
||||||
|
azure_ml_compute: str = ""
|
||||||
|
azure_ml_environment: str = "AzureML-sklearn-1.5"
|
||||||
|
azure_ml_experiment_name: str = "forja-training"
|
||||||
|
azure_ml_command: str = "python train.py"
|
||||||
|
|
||||||
# Persistencia
|
# Persistencia
|
||||||
data_dir: Path = Field(default=Path("./data"))
|
data_dir: Path = Field(default=Path("./data"))
|
||||||
agents_dir: Path = Field(default=Path("./agents"))
|
agents_dir: Path = Field(default=Path("./agents"))
|
||||||
policies_dir: Path = Field(default=Path("./policies"))
|
policies_dir: Path = Field(default=Path("./policies"))
|
||||||
|
datasets_dir: Path = Field(default=Path("./datasets"))
|
||||||
|
models_dir: Path = Field(default=Path("./models"))
|
||||||
def get_settings() -> Settings:
|
|
||||||
"""Helper para inyección por dependencia en FastAPI."""
|
|
||||||
return Settings()
|
|
||||||
@@ -7,6 +7,8 @@ from typing import Any, Literal
|
|||||||
|
|
||||||
from pydantic import BaseModel, Field
|
from pydantic import BaseModel, Field
|
||||||
|
|
||||||
|
from forja_core.domain.version_meta import VersionMeta
|
||||||
|
|
||||||
|
|
||||||
class LLMConfig(BaseModel):
|
class LLMConfig(BaseModel):
|
||||||
"""Configuración del proveedor LLM utilizado por un agente."""
|
"""Configuración del proveedor LLM utilizado por un agente."""
|
||||||
@@ -17,14 +19,8 @@ class LLMConfig(BaseModel):
|
|||||||
max_tokens: int = Field(default=2000, ge=1, le=128_000)
|
max_tokens: int = Field(default=2000, ge=1, le=128_000)
|
||||||
|
|
||||||
|
|
||||||
class AgentVersionMeta(BaseModel):
|
# Alias de compatibilidad: los metadatos de versión son comunes a todos los registries.
|
||||||
"""Metadatos de una versión concreta de un agente (estilo commit Git)."""
|
AgentVersionMeta = VersionMeta
|
||||||
|
|
||||||
id: str
|
|
||||||
hash: str
|
|
||||||
author: str
|
|
||||||
message: str
|
|
||||||
created_at: datetime
|
|
||||||
|
|
||||||
|
|
||||||
class AgentDefinition(BaseModel):
|
class AgentDefinition(BaseModel):
|
||||||
@@ -41,3 +37,10 @@ class AgentDefinition(BaseModel):
|
|||||||
output_schema: dict[str, Any]
|
output_schema: dict[str, Any]
|
||||||
risk_threshold_for_hitl: int = Field(default=4, ge=1, le=5)
|
risk_threshold_for_hitl: int = Field(default=4, ge=1, le=5)
|
||||||
updated_at: datetime
|
updated_at: datetime
|
||||||
|
input_label: str | None = None
|
||||||
|
input_placeholder: str | None = None
|
||||||
|
category: str | None = None
|
||||||
|
tags: list[str] = Field(default_factory=list)
|
||||||
|
icon: str | None = "🤖"
|
||||||
|
color: str | None = None
|
||||||
|
template: str = "governed_llm"
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
"""Dataset definitions for fine-tuning (versioned YAML)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime
|
||||||
|
from typing import Literal
|
||||||
|
|
||||||
|
from pydantic import BaseModel, Field
|
||||||
|
|
||||||
|
from forja_core.domain.version_meta import VersionMeta
|
||||||
|
|
||||||
|
# Compatibility alias: version metadata is shared across all registries.
|
||||||
|
DatasetVersionMeta = VersionMeta
|
||||||
|
|
||||||
|
|
||||||
|
class DatasetDefinition(BaseModel):
|
||||||
|
"""Declarative dataset used to submit training jobs."""
|
||||||
|
|
||||||
|
name: str
|
||||||
|
version: str
|
||||||
|
owner: str
|
||||||
|
purpose: str
|
||||||
|
state: Literal["draft", "active", "deprecated"] = "active"
|
||||||
|
format: Literal["jsonl", "conversation"] = "jsonl"
|
||||||
|
records_path: str = Field(
|
||||||
|
description="Path relative to the dataset directory, e.g. records/train.jsonl",
|
||||||
|
)
|
||||||
|
updated_at: datetime
|
||||||
+1
-1
@@ -8,7 +8,7 @@ from uuid import UUID
|
|||||||
|
|
||||||
from pydantic import BaseModel, Field
|
from pydantic import BaseModel, Field
|
||||||
|
|
||||||
from agentforge_core.domain.guardrail import GuardrailViolation
|
from forja_core.domain.guardrail import GuardrailViolation
|
||||||
|
|
||||||
|
|
||||||
class ProposedAction(BaseModel):
|
class ProposedAction(BaseModel):
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
"""Base / fine-tune target model definitions (versioned YAML)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime
|
||||||
|
from typing import Literal
|
||||||
|
|
||||||
|
from pydantic import BaseModel, Field
|
||||||
|
|
||||||
|
from forja_core.domain.version_meta import VersionMeta
|
||||||
|
|
||||||
|
# Compatibility alias: version metadata is shared across all registries.
|
||||||
|
ModelVersionMeta = VersionMeta
|
||||||
|
|
||||||
|
|
||||||
|
class ModelDefinition(BaseModel):
|
||||||
|
"""Registered base model or training target reference."""
|
||||||
|
|
||||||
|
name: str
|
||||||
|
version: str
|
||||||
|
owner: str
|
||||||
|
purpose: str
|
||||||
|
state: Literal["draft", "active", "deprecated"] = "active"
|
||||||
|
base_model_id: str = Field(
|
||||||
|
description="Provider model id, e.g. HuggingFace repo or Azure deployment name",
|
||||||
|
)
|
||||||
|
adaptation: Literal["full", "lora", "qlora"] = "lora"
|
||||||
|
updated_at: datetime
|
||||||
@@ -2,11 +2,12 @@
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from datetime import datetime
|
|
||||||
from typing import Any, Literal
|
from typing import Any, Literal
|
||||||
|
|
||||||
from pydantic import BaseModel, Field
|
from pydantic import BaseModel, Field
|
||||||
|
|
||||||
|
from forja_core.domain.version_meta import VersionMeta
|
||||||
|
|
||||||
|
|
||||||
class PolicyValidator(BaseModel):
|
class PolicyValidator(BaseModel):
|
||||||
"""Validador individual configurado en una política."""
|
"""Validador individual configurado en una política."""
|
||||||
@@ -15,14 +16,8 @@ class PolicyValidator(BaseModel):
|
|||||||
config: dict[str, Any] = Field(default_factory=dict)
|
config: dict[str, Any] = Field(default_factory=dict)
|
||||||
|
|
||||||
|
|
||||||
class PolicyVersionMeta(BaseModel):
|
# Alias de compatibilidad: los metadatos de versión son comunes a todos los registries.
|
||||||
"""Metadatos de una versión concreta de una política (estilo commit Git)."""
|
PolicyVersionMeta = VersionMeta
|
||||||
|
|
||||||
id: str
|
|
||||||
hash: str
|
|
||||||
author: str
|
|
||||||
message: str
|
|
||||||
created_at: datetime
|
|
||||||
|
|
||||||
|
|
||||||
class PolicyDefinition(BaseModel):
|
class PolicyDefinition(BaseModel):
|
||||||
@@ -0,0 +1,127 @@
|
|||||||
|
"""Training jobs, runs, and promotion-related records."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime
|
||||||
|
from typing import Literal
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from pydantic import BaseModel, Field
|
||||||
|
|
||||||
|
TrainingRunStatus = Literal[
|
||||||
|
"submitted",
|
||||||
|
"queued",
|
||||||
|
"running",
|
||||||
|
"succeeded",
|
||||||
|
"failed",
|
||||||
|
"cancelled",
|
||||||
|
]
|
||||||
|
TrainingBackendKind = Literal["mock", "azure_ml"]
|
||||||
|
|
||||||
|
|
||||||
|
class TrainingHyperparameters(BaseModel):
|
||||||
|
"""Minimal hyperparameters passed to an external trainer."""
|
||||||
|
|
||||||
|
learning_rate: float = Field(default=1e-4, gt=0.0, le=1.0)
|
||||||
|
epochs: int = Field(default=1, ge=1, le=100)
|
||||||
|
batch_size: int = Field(default=4, ge=1, le=512)
|
||||||
|
|
||||||
|
|
||||||
|
class TrainingJobSpec(BaseModel):
|
||||||
|
"""Input specification for a training backend."""
|
||||||
|
|
||||||
|
dataset_name: str
|
||||||
|
dataset_version: str
|
||||||
|
model_name: str
|
||||||
|
model_version: str
|
||||||
|
agent_name: str | None = None
|
||||||
|
# Candidate agent version this run is meant to produce/validate. Post-train
|
||||||
|
# evaluation runs against this version, and promotion requires the
|
||||||
|
# evaluation report to match it.
|
||||||
|
agent_version: str | None = None
|
||||||
|
hyperparameters: TrainingHyperparameters = Field(default_factory=TrainingHyperparameters)
|
||||||
|
|
||||||
|
|
||||||
|
class ArtifactRef(BaseModel):
|
||||||
|
"""Output artifact from a completed training job."""
|
||||||
|
|
||||||
|
uri: str
|
||||||
|
kind: Literal["checkpoint", "adapter", "metrics"] = "adapter"
|
||||||
|
|
||||||
|
|
||||||
|
class BackendJobStatus(BaseModel):
|
||||||
|
"""Status returned by an external MLOps backend."""
|
||||||
|
|
||||||
|
status: TrainingRunStatus
|
||||||
|
message: str | None = None
|
||||||
|
artifacts: list[ArtifactRef] = Field(default_factory=list)
|
||||||
|
|
||||||
|
|
||||||
|
class TrainingRun(BaseModel):
|
||||||
|
"""Persisted training run orchestrated by Forja."""
|
||||||
|
|
||||||
|
id: UUID
|
||||||
|
backend: TrainingBackendKind
|
||||||
|
status: TrainingRunStatus
|
||||||
|
spec: TrainingJobSpec
|
||||||
|
external_job_id: str | None = None
|
||||||
|
created_at: datetime
|
||||||
|
updated_at: datetime
|
||||||
|
artifact_refs: list[ArtifactRef] = Field(default_factory=list)
|
||||||
|
error_message: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class ScenarioEvalResult(BaseModel):
|
||||||
|
"""Result of running one canonical scenario through the governed runtime."""
|
||||||
|
|
||||||
|
scenario: str
|
||||||
|
status: str
|
||||||
|
blocking_violations: int = 0
|
||||||
|
|
||||||
|
|
||||||
|
class EvaluationReport(BaseModel):
|
||||||
|
"""Post-training evaluation summary (gate before promotion)."""
|
||||||
|
|
||||||
|
id: UUID
|
||||||
|
training_run_id: UUID
|
||||||
|
agent_name: str
|
||||||
|
agent_version: str
|
||||||
|
passed: bool
|
||||||
|
scenario_count: int = 0
|
||||||
|
violation_count: int = 0
|
||||||
|
scenarios: list[ScenarioEvalResult] = Field(default_factory=list)
|
||||||
|
created_at: datetime
|
||||||
|
notes: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
PromotionStatus = Literal["pending_approval", "approved", "rejected"]
|
||||||
|
|
||||||
|
|
||||||
|
class PromotionRequest(BaseModel):
|
||||||
|
"""Governance HITL request to promote a draft agent version to active."""
|
||||||
|
|
||||||
|
id: UUID
|
||||||
|
agent_name: str
|
||||||
|
agent_version: str
|
||||||
|
training_run_id: UUID
|
||||||
|
evaluation_report_id: UUID
|
||||||
|
status: PromotionStatus
|
||||||
|
requested_by: str
|
||||||
|
created_at: datetime
|
||||||
|
updated_at: datetime
|
||||||
|
# Operator who approved or rejected (audit trail for both outcomes).
|
||||||
|
decided_by: str | None = None
|
||||||
|
comment: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class PromotionRecord(BaseModel):
|
||||||
|
"""Audit record when a model/agent version is promoted to active."""
|
||||||
|
|
||||||
|
id: UUID
|
||||||
|
agent_name: str
|
||||||
|
agent_version: str
|
||||||
|
training_run_id: UUID | None = None
|
||||||
|
evaluation_report_id: UUID | None = None
|
||||||
|
approved_by: str
|
||||||
|
comment: str | None = None
|
||||||
|
created_at: datetime
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
"""Metadatos de versión compartidos por todos los registries versionados."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime
|
||||||
|
|
||||||
|
from pydantic import BaseModel
|
||||||
|
|
||||||
|
|
||||||
|
class VersionMeta(BaseModel):
|
||||||
|
"""Metadatos de una versión concreta de un artefacto versionado (estilo commit Git)."""
|
||||||
|
|
||||||
|
id: str
|
||||||
|
hash: str
|
||||||
|
author: str
|
||||||
|
message: str
|
||||||
|
created_at: datetime
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
"""Post-training evaluation and promotion gates."""
|
||||||
@@ -0,0 +1,85 @@
|
|||||||
|
"""Run post-training evaluation: governed agent runs over canonical scenarios."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from pathlib import Path
|
||||||
|
from uuid import UUID, uuid4
|
||||||
|
|
||||||
|
from forja_core.domain.agent import AgentDefinition
|
||||||
|
from forja_core.domain.policy import PolicyDefinition
|
||||||
|
from forja_core.domain.training import EvaluationReport, ScenarioEvalResult
|
||||||
|
from forja_core.evaluation.scenarios import load_agent_scenarios
|
||||||
|
from forja_core.runtime.orchestrator import AgentOrchestrator
|
||||||
|
|
||||||
|
|
||||||
|
def _blocking_violation_count(execution: object) -> int:
|
||||||
|
violations = getattr(execution, "violations", [])
|
||||||
|
return sum(1 for v in violations if v.blocked)
|
||||||
|
|
||||||
|
|
||||||
|
async def run_post_train_evaluation(
|
||||||
|
*,
|
||||||
|
training_run_id: UUID,
|
||||||
|
agent_def: AgentDefinition,
|
||||||
|
policy: PolicyDefinition,
|
||||||
|
orchestrator: AgentOrchestrator,
|
||||||
|
agents_dir: Path,
|
||||||
|
) -> EvaluationReport:
|
||||||
|
"""Execute each scenario; pass when there are no blocking guardrail violations."""
|
||||||
|
scenarios = load_agent_scenarios(agents_dir, agent_def.name)
|
||||||
|
if not scenarios:
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
return EvaluationReport(
|
||||||
|
id=uuid4(),
|
||||||
|
training_run_id=training_run_id,
|
||||||
|
agent_name=agent_def.name,
|
||||||
|
agent_version=agent_def.version,
|
||||||
|
passed=False,
|
||||||
|
scenario_count=0,
|
||||||
|
violation_count=0,
|
||||||
|
created_at=now,
|
||||||
|
notes="No scenarios found under agents/<name>/examples/",
|
||||||
|
)
|
||||||
|
|
||||||
|
results: list[ScenarioEvalResult] = []
|
||||||
|
total_blocking = 0
|
||||||
|
for name, user_input in scenarios:
|
||||||
|
execution = await orchestrator.invoke(
|
||||||
|
agent_def=agent_def,
|
||||||
|
policy=policy,
|
||||||
|
user_input=user_input,
|
||||||
|
)
|
||||||
|
blocking = _blocking_violation_count(execution)
|
||||||
|
total_blocking += blocking
|
||||||
|
results.append(
|
||||||
|
ScenarioEvalResult(
|
||||||
|
scenario=name,
|
||||||
|
status=execution.status,
|
||||||
|
blocking_violations=blocking,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
failed_count = sum(1 for r in results if r.status == "failed")
|
||||||
|
passed = total_blocking == 0 and failed_count == 0
|
||||||
|
notes = None
|
||||||
|
if not passed:
|
||||||
|
parts: list[str] = []
|
||||||
|
if total_blocking:
|
||||||
|
parts.append(f"{total_blocking} blocking guardrail violation(s)")
|
||||||
|
if failed_count:
|
||||||
|
parts.append(f"{failed_count} scenario(s) failed")
|
||||||
|
notes = "; ".join(parts)
|
||||||
|
|
||||||
|
return EvaluationReport(
|
||||||
|
id=uuid4(),
|
||||||
|
training_run_id=training_run_id,
|
||||||
|
agent_name=agent_def.name,
|
||||||
|
agent_version=agent_def.version,
|
||||||
|
passed=passed,
|
||||||
|
scenario_count=len(results),
|
||||||
|
violation_count=total_blocking,
|
||||||
|
scenarios=results,
|
||||||
|
created_at=datetime.now(UTC),
|
||||||
|
notes=notes,
|
||||||
|
)
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
"""Load canonical evaluation scenarios from agent example files."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
def load_agent_scenarios(agents_dir: Path, agent_name: str) -> list[tuple[str, str]]:
|
||||||
|
"""Return (scenario_name, input_text) pairs from agents/<name>/examples/*.txt."""
|
||||||
|
examples_dir = agents_dir / agent_name / "examples"
|
||||||
|
if not examples_dir.is_dir():
|
||||||
|
return []
|
||||||
|
scenarios: list[tuple[str, str]] = []
|
||||||
|
for path in sorted(examples_dir.glob("*.txt")):
|
||||||
|
scenarios.append((path.name, path.read_text(encoding="utf-8")))
|
||||||
|
return scenarios
|
||||||
+2
-2
@@ -5,8 +5,8 @@ from __future__ import annotations
|
|||||||
from typing import Any, Protocol
|
from typing import Any, Protocol
|
||||||
from uuid import UUID
|
from uuid import UUID
|
||||||
|
|
||||||
from agentforge_core.domain.guardrail import GuardrailViolation
|
from forja_core.domain.guardrail import GuardrailViolation
|
||||||
from agentforge_core.domain.policy import PolicyDefinition
|
from forja_core.domain.policy import PolicyDefinition
|
||||||
|
|
||||||
|
|
||||||
class GuardrailEngine(Protocol):
|
class GuardrailEngine(Protocol):
|
||||||
+3
-3
@@ -6,9 +6,9 @@ import asyncio
|
|||||||
from typing import Any
|
from typing import Any
|
||||||
from uuid import UUID
|
from uuid import UUID
|
||||||
|
|
||||||
from agentforge_core.domain.guardrail import GuardrailViolation
|
from forja_core.domain.guardrail import GuardrailViolation
|
||||||
from agentforge_core.domain.policy import PolicyDefinition
|
from forja_core.domain.policy import PolicyDefinition
|
||||||
from agentforge_core.guardrails.base import GuardrailEngine
|
from forja_core.guardrails.base import GuardrailEngine
|
||||||
|
|
||||||
|
|
||||||
class CompositeGuardrailEngine:
|
class CompositeGuardrailEngine:
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
"""Factory que ensambla el GuardrailEngine según settings."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from forja_core.config import Settings
|
||||||
|
from forja_core.guardrails.base import GuardrailEngine
|
||||||
|
from forja_core.guardrails.composite import CompositeGuardrailEngine
|
||||||
|
from forja_core.guardrails.guardrails_ai import GuardrailsAIEngine
|
||||||
|
from forja_core.guardrails.nemo import NeMoGuardrailsEngine
|
||||||
|
|
||||||
|
|
||||||
|
def build_guardrail_engine(settings: Settings) -> GuardrailEngine:
|
||||||
|
"""Ensambla el engine: GuardrailsAIEngine siempre; NeMo si está habilitado.
|
||||||
|
|
||||||
|
Los keywords de topical rails vienen de configuración (CSV en
|
||||||
|
``GUARDRAILS_NEMO_ALLOWED_KEYWORDS``); el core no asume ningún dominio.
|
||||||
|
"""
|
||||||
|
engines: list[GuardrailEngine] = [GuardrailsAIEngine()]
|
||||||
|
if settings.guardrails_nemo_enabled:
|
||||||
|
keywords = [
|
||||||
|
k.strip() for k in settings.guardrails_nemo_allowed_keywords.split(",") if k.strip()
|
||||||
|
]
|
||||||
|
engines.append(NeMoGuardrailsEngine(allowed_keywords=keywords))
|
||||||
|
return CompositeGuardrailEngine(engines)
|
||||||
+3
-3
@@ -9,9 +9,9 @@ from uuid import UUID
|
|||||||
|
|
||||||
import structlog
|
import structlog
|
||||||
|
|
||||||
from agentforge_core.domain.guardrail import GuardrailViolation
|
from forja_core.domain.guardrail import GuardrailViolation
|
||||||
from agentforge_core.domain.policy import PolicyDefinition, PolicyValidator
|
from forja_core.domain.policy import PolicyDefinition, PolicyValidator
|
||||||
from agentforge_core.guardrails.validators import (
|
from forja_core.guardrails.validators import (
|
||||||
detect_pii,
|
detect_pii,
|
||||||
forbidden_action_keywords,
|
forbidden_action_keywords,
|
||||||
forbidden_topics,
|
forbidden_topics,
|
||||||
+3
-3
@@ -12,8 +12,8 @@ from uuid import UUID
|
|||||||
|
|
||||||
import structlog
|
import structlog
|
||||||
|
|
||||||
from agentforge_core.domain.guardrail import GuardrailViolation
|
from forja_core.domain.guardrail import GuardrailViolation
|
||||||
from agentforge_core.domain.policy import PolicyDefinition
|
from forja_core.domain.policy import PolicyDefinition
|
||||||
|
|
||||||
log = structlog.get_logger(__name__)
|
log = structlog.get_logger(__name__)
|
||||||
|
|
||||||
@@ -48,7 +48,7 @@ class NeMoGuardrailsEngine:
|
|||||||
stage="input",
|
stage="input",
|
||||||
validator="NeMoTopicalRails",
|
validator="NeMoTopicalRails",
|
||||||
severity="warning",
|
severity="warning",
|
||||||
message="Input fuera de los temas permitidos.",
|
message="Input outside the allowed topics.",
|
||||||
blocked=False,
|
blocked=False,
|
||||||
)
|
)
|
||||||
]
|
]
|
||||||
+39
-13
@@ -10,7 +10,7 @@ from uuid import UUID
|
|||||||
|
|
||||||
import jsonschema
|
import jsonschema
|
||||||
|
|
||||||
from agentforge_core.domain.guardrail import GuardrailViolation
|
from forja_core.domain.guardrail import GuardrailViolation
|
||||||
|
|
||||||
_INJECTION_PATTERNS = [
|
_INJECTION_PATTERNS = [
|
||||||
r"ignore (the )?(previous|prior|above) (instruction|prompt|message)",
|
r"ignore (the )?(previous|prior|above) (instruction|prompt|message)",
|
||||||
@@ -21,6 +21,33 @@ _INJECTION_PATTERNS = [
|
|||||||
r"reveal (the )?(system|hidden) (prompt|instruction)",
|
r"reveal (the )?(system|hidden) (prompt|instruction)",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
# Singleton perezoso del AnalyzerEngine de Presidio. Construirlo es caro (carga el
|
||||||
|
# modelo spaCy y los recognizers), así que se reutiliza entre llamadas.
|
||||||
|
_PRESIDIO_ANALYZER: Any = None
|
||||||
|
|
||||||
|
|
||||||
|
def _presidio_analyzer() -> Any:
|
||||||
|
"""``AnalyzerEngine`` de Presidio configurado con el modelo spaCy ``en_core_web_sm``.
|
||||||
|
|
||||||
|
Presidio usa por defecto ``en_core_web_lg`` (~560 MB), que **no** está en la imagen
|
||||||
|
Docker: ``core/Dockerfile`` instala ``en_core_web_sm``. Si Presidio no está instalado
|
||||||
|
o el modelo no se puede cargar, esto lanza y el llamador (`detect_pii`) hace fallback
|
||||||
|
a la detección por regex.
|
||||||
|
"""
|
||||||
|
global _PRESIDIO_ANALYZER
|
||||||
|
if _PRESIDIO_ANALYZER is None:
|
||||||
|
from presidio_analyzer import AnalyzerEngine
|
||||||
|
from presidio_analyzer.nlp_engine import NlpEngineProvider
|
||||||
|
|
||||||
|
nlp_engine = NlpEngineProvider(
|
||||||
|
nlp_configuration={
|
||||||
|
"nlp_engine_name": "spacy",
|
||||||
|
"models": [{"lang_code": "en", "model_name": "en_core_web_sm"}],
|
||||||
|
}
|
||||||
|
).create_engine()
|
||||||
|
_PRESIDIO_ANALYZER = AnalyzerEngine(nlp_engine=nlp_engine)
|
||||||
|
return _PRESIDIO_ANALYZER
|
||||||
|
|
||||||
|
|
||||||
def _violation(
|
def _violation(
|
||||||
*,
|
*,
|
||||||
@@ -45,11 +72,11 @@ def _violation(
|
|||||||
def detect_pii(
|
def detect_pii(
|
||||||
text: str, config: dict[str, Any], trace_id: UUID, stage: str
|
text: str, config: dict[str, Any], trace_id: UUID, stage: str
|
||||||
) -> list[GuardrailViolation]:
|
) -> list[GuardrailViolation]:
|
||||||
"""Detección PII vía Presidio Analyzer (con fallback a regex si no está instalado)."""
|
"""Detección PII vía Presidio Analyzer (con fallback a regex si no está disponible)."""
|
||||||
try:
|
try:
|
||||||
from presidio_analyzer import AnalyzerEngine
|
analyzer = _presidio_analyzer()
|
||||||
except Exception:
|
except Exception:
|
||||||
# Si Presidio no está, fallback a regex básica
|
# Presidio no instalado o modelo spaCy no disponible → fallback a regex básica.
|
||||||
return _pii_regex_fallback(text, config, trace_id, stage)
|
return _pii_regex_fallback(text, config, trace_id, stage)
|
||||||
|
|
||||||
entities = config.get(
|
entities = config.get(
|
||||||
@@ -59,7 +86,6 @@ def detect_pii(
|
|||||||
sev = config.get("severity_on_match", "block")
|
sev = config.get("severity_on_match", "block")
|
||||||
blocked = sev == "block"
|
blocked = sev == "block"
|
||||||
|
|
||||||
analyzer = AnalyzerEngine()
|
|
||||||
results = analyzer.analyze(text=text, entities=entities, language="en")
|
results = analyzer.analyze(text=text, entities=entities, language="en")
|
||||||
if not results:
|
if not results:
|
||||||
return []
|
return []
|
||||||
@@ -70,7 +96,7 @@ def detect_pii(
|
|||||||
stage=stage,
|
stage=stage,
|
||||||
validator="DetectPII",
|
validator="DetectPII",
|
||||||
severity=sev,
|
severity=sev,
|
||||||
message=f"PII detectada: {matched}",
|
message=f"PII detected: {matched}",
|
||||||
blocked=blocked,
|
blocked=blocked,
|
||||||
)
|
)
|
||||||
]
|
]
|
||||||
@@ -97,7 +123,7 @@ def _pii_regex_fallback(
|
|||||||
stage=stage,
|
stage=stage,
|
||||||
validator="DetectPII",
|
validator="DetectPII",
|
||||||
severity=sev,
|
severity=sev,
|
||||||
message=f"PII detectada (regex fallback): {', '.join(matched)}",
|
message=f"PII detected (regex fallback): {', '.join(matched)}",
|
||||||
blocked=blocked,
|
blocked=blocked,
|
||||||
)
|
)
|
||||||
]
|
]
|
||||||
@@ -118,7 +144,7 @@ def prompt_injection(
|
|||||||
stage=stage,
|
stage=stage,
|
||||||
validator="PromptInjection",
|
validator="PromptInjection",
|
||||||
severity=sev,
|
severity=sev,
|
||||||
message=f"Patrón de inyección detectado: {pattern}",
|
message=f"Injection pattern detected: {pattern}",
|
||||||
blocked=blocked,
|
blocked=blocked,
|
||||||
)
|
)
|
||||||
]
|
]
|
||||||
@@ -149,7 +175,7 @@ def toxic_language(
|
|||||||
stage=stage,
|
stage=stage,
|
||||||
validator="ToxicLanguage",
|
validator="ToxicLanguage",
|
||||||
severity=sev,
|
severity=sev,
|
||||||
message=f"Lenguaje tóxico (score={score:.2f}, hits={hits})",
|
message=f"Toxic language (score={score:.2f}, hits={hits})",
|
||||||
blocked=blocked,
|
blocked=blocked,
|
||||||
)
|
)
|
||||||
]
|
]
|
||||||
@@ -172,7 +198,7 @@ def forbidden_topics(
|
|||||||
stage=stage,
|
stage=stage,
|
||||||
validator="ForbiddenTopics",
|
validator="ForbiddenTopics",
|
||||||
severity=sev,
|
severity=sev,
|
||||||
message=f"Temas prohibidos: {', '.join(matched)}",
|
message=f"Forbidden topics: {', '.join(matched)}",
|
||||||
blocked=blocked,
|
blocked=blocked,
|
||||||
)
|
)
|
||||||
]
|
]
|
||||||
@@ -230,7 +256,7 @@ def forbidden_action_keywords(
|
|||||||
stage=stage,
|
stage=stage,
|
||||||
validator="ForbiddenActionKeywords",
|
validator="ForbiddenActionKeywords",
|
||||||
severity=sev,
|
severity=sev,
|
||||||
message=f"Palabras prohibidas en acciones: {', '.join(matched)}",
|
message=f"Forbidden keywords in actions: {', '.join(matched)}",
|
||||||
blocked=blocked,
|
blocked=blocked,
|
||||||
)
|
)
|
||||||
]
|
]
|
||||||
@@ -249,7 +275,7 @@ def telco_safety_rules(
|
|||||||
target = str(a.get("target", "")).lower()
|
target = str(a.get("target", "")).lower()
|
||||||
rollback = str(a.get("rollback_plan", "")).strip()
|
rollback = str(a.get("rollback_plan", "")).strip()
|
||||||
if ("prod" in target or "production" in target) and (not rollback or rollback == "n/a"):
|
if ("prod" in target or "production" in target) and (not rollback or rollback == "n/a"):
|
||||||
issues.append(f"acción sobre prod sin rollback: {a.get('action')}")
|
issues.append(f"production action without rollback: {a.get('action')}")
|
||||||
|
|
||||||
if "never_propose_mass_action_without_canary" in rules:
|
if "never_propose_mass_action_without_canary" in rules:
|
||||||
mass_keywords = {"all", "todos", "*", "mass"}
|
mass_keywords = {"all", "todos", "*", "mass"}
|
||||||
@@ -257,7 +283,7 @@ def telco_safety_rules(
|
|||||||
target = str(a.get("target", "")).lower()
|
target = str(a.get("target", "")).lower()
|
||||||
plan = str(a.get("rollback_plan", "")).lower()
|
plan = str(a.get("rollback_plan", "")).lower()
|
||||||
if any(k in target for k in mass_keywords) and "canary" not in plan:
|
if any(k in target for k in mass_keywords) and "canary" not in plan:
|
||||||
issues.append(f"acción masiva sin canary: {a.get('action')}")
|
issues.append(f"mass action without canary plan: {a.get('action')}")
|
||||||
|
|
||||||
if not issues:
|
if not issues:
|
||||||
return []
|
return []
|
||||||
@@ -8,7 +8,7 @@ from typing import Any
|
|||||||
|
|
||||||
from openai import AsyncAzureOpenAI
|
from openai import AsyncAzureOpenAI
|
||||||
|
|
||||||
from agentforge_core.llm.base import CompletionResult, Message
|
from forja_core.llm.base import CompletionResult, Message
|
||||||
|
|
||||||
|
|
||||||
class AzureOpenAIProvider:
|
class AzureOpenAIProvider:
|
||||||
@@ -0,0 +1,40 @@
|
|||||||
|
"""Factory que selecciona el LLMProvider según settings."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Literal
|
||||||
|
|
||||||
|
from forja_core.config import Settings
|
||||||
|
from forja_core.llm.azure import AzureOpenAIProvider
|
||||||
|
from forja_core.llm.base import LLMProvider
|
||||||
|
from forja_core.llm.fallback import FallbackLLMProvider
|
||||||
|
from forja_core.llm.mock import MockProvider
|
||||||
|
from forja_core.llm.openai import OpenAIProvider
|
||||||
|
|
||||||
|
|
||||||
|
def _build_single(provider: Literal["mock", "azure", "openai"], settings: Settings) -> LLMProvider:
|
||||||
|
match provider:
|
||||||
|
case "mock":
|
||||||
|
return MockProvider()
|
||||||
|
case "azure":
|
||||||
|
return AzureOpenAIProvider(
|
||||||
|
endpoint=settings.azure_openai_endpoint,
|
||||||
|
api_key=settings.azure_openai_api_key,
|
||||||
|
deployment=settings.azure_openai_deployment,
|
||||||
|
api_version=settings.azure_openai_api_version,
|
||||||
|
)
|
||||||
|
case "openai":
|
||||||
|
return OpenAIProvider(
|
||||||
|
api_key=settings.openai_api_key,
|
||||||
|
model=settings.openai_model,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def build_llm_provider(settings: Settings) -> LLMProvider:
|
||||||
|
"""Materializa el proveedor activo (con fallback opcional). Falla en arranque si la config es inconsistente."""
|
||||||
|
primary = _build_single(settings.llm_provider, settings)
|
||||||
|
fallback_name = settings.llm_fallback_provider
|
||||||
|
if not fallback_name or fallback_name == settings.llm_provider:
|
||||||
|
return primary
|
||||||
|
fallback = _build_single(fallback_name, settings)
|
||||||
|
return FallbackLLMProvider(primary, fallback)
|
||||||
@@ -0,0 +1,46 @@
|
|||||||
|
"""Proveedor LLM con fallback: intenta el primario y, si agota sus reintentos, usa el secundario."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import structlog
|
||||||
|
|
||||||
|
from forja_core.llm.base import CompletionResult, LLMProvider, Message
|
||||||
|
|
||||||
|
log = structlog.get_logger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class FallbackLLMProvider:
|
||||||
|
"""Envuelve dos providers: si el primario lanza, delega en el de fallback.
|
||||||
|
|
||||||
|
Cada provider interno conserva su propia política de retries; este wrapper
|
||||||
|
solo decide el cambio de proveedor cuando el primario falla definitivamente.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, primary: LLMProvider, fallback: LLMProvider) -> None:
|
||||||
|
self._primary = primary
|
||||||
|
self._fallback = fallback
|
||||||
|
self.name = f"{primary.name}+fallback:{fallback.name}"
|
||||||
|
|
||||||
|
async def complete(
|
||||||
|
self,
|
||||||
|
messages: list[Message],
|
||||||
|
schema: dict[str, Any] | None = None,
|
||||||
|
temperature: float = 0.2,
|
||||||
|
max_tokens: int = 2000,
|
||||||
|
) -> CompletionResult:
|
||||||
|
try:
|
||||||
|
return await self._primary.complete(
|
||||||
|
messages, schema=schema, temperature=temperature, max_tokens=max_tokens
|
||||||
|
)
|
||||||
|
except Exception as exc:
|
||||||
|
log.warning(
|
||||||
|
"llm_primary_failed_falling_back",
|
||||||
|
primary=self._primary.name,
|
||||||
|
fallback=self._fallback.name,
|
||||||
|
error=str(exc),
|
||||||
|
)
|
||||||
|
return await self._fallback.complete(
|
||||||
|
messages, schema=schema, temperature=temperature, max_tokens=max_tokens
|
||||||
|
)
|
||||||
@@ -7,7 +7,7 @@ import json
|
|||||||
import time
|
import time
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from agentforge_core.llm.base import CompletionResult, Message
|
from forja_core.llm.base import CompletionResult, Message
|
||||||
|
|
||||||
_CANONICAL_RESPONSES: dict[str, dict[str, Any]] = {
|
_CANONICAL_RESPONSES: dict[str, dict[str, Any]] = {
|
||||||
"sip": {
|
"sip": {
|
||||||
@@ -8,7 +8,7 @@ from typing import Any
|
|||||||
|
|
||||||
from openai import AsyncOpenAI
|
from openai import AsyncOpenAI
|
||||||
|
|
||||||
from agentforge_core.llm.base import CompletionResult, Message
|
from forja_core.llm.base import CompletionResult, Message
|
||||||
|
|
||||||
|
|
||||||
class OpenAIProvider:
|
class OpenAIProvider:
|
||||||
@@ -0,0 +1,56 @@
|
|||||||
|
"""Composición raíz de la aplicación FastAPI (forja-core)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from fastapi import FastAPI
|
||||||
|
|
||||||
|
from forja_core.api.middlewares import TraceIdMiddleware
|
||||||
|
from forja_core.config import Settings
|
||||||
|
from forja_core.observability.logging import configure_logging
|
||||||
|
|
||||||
|
|
||||||
|
def create_app() -> FastAPI:
|
||||||
|
"""Construye la app: logging, middleware de trace_id, /health y routers."""
|
||||||
|
settings = Settings()
|
||||||
|
configure_logging(level=settings.log_level)
|
||||||
|
app = FastAPI(
|
||||||
|
title="Forja Core",
|
||||||
|
version="0.1.0",
|
||||||
|
description="Plataforma de gobernanza de agentes IA — API REST.",
|
||||||
|
)
|
||||||
|
app.add_middleware(TraceIdMiddleware)
|
||||||
|
|
||||||
|
@app.get("/health", tags=["meta"])
|
||||||
|
async def health() -> dict[str, str]:
|
||||||
|
return {"status": "ok"}
|
||||||
|
|
||||||
|
from forja_core.api import (
|
||||||
|
agents,
|
||||||
|
datasets,
|
||||||
|
executions,
|
||||||
|
models,
|
||||||
|
policies,
|
||||||
|
promotions,
|
||||||
|
training,
|
||||||
|
violations,
|
||||||
|
)
|
||||||
|
from forja_core.web import ui
|
||||||
|
|
||||||
|
# UI HTMX embebida (Opción A) en rutas amigables para humanos.
|
||||||
|
# Toda la API REST vive bajo /api para eliminar ambigüedad y conflictos de ruta.
|
||||||
|
app.include_router(ui.router)
|
||||||
|
|
||||||
|
app.include_router(agents.router, prefix="/api/agents", tags=["agents"])
|
||||||
|
app.include_router(executions.invoke_router, prefix="/api/agents", tags=["agents"])
|
||||||
|
app.include_router(executions.router, prefix="/api/executions", tags=["executions"])
|
||||||
|
app.include_router(policies.router, prefix="/api/policies", tags=["policies"])
|
||||||
|
app.include_router(violations.router, prefix="/api/violations", tags=["violations"])
|
||||||
|
app.include_router(datasets.router, prefix="/api/datasets", tags=["datasets"])
|
||||||
|
app.include_router(models.router, prefix="/api/models", tags=["models"])
|
||||||
|
app.include_router(training.router, prefix="/api/training/runs", tags=["training"])
|
||||||
|
app.include_router(promotions.router, prefix="/api/promotions", tags=["promotions"])
|
||||||
|
|
||||||
|
return app
|
||||||
|
|
||||||
|
|
||||||
|
app = create_app()
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
"""Read-only dataset registry (YAML versioned, same layout as agents/)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from forja_core.domain.dataset import DatasetDefinition, DatasetVersionMeta
|
||||||
|
from forja_core.registry.yaml_store import VersionedYamlStore
|
||||||
|
|
||||||
|
|
||||||
|
class FileSystemDatasetStore(VersionedYamlStore[DatasetDefinition]):
|
||||||
|
"""Reads datasets from datasets/<name>/{index.yaml,versions/vN.yaml}."""
|
||||||
|
|
||||||
|
model_cls = DatasetDefinition
|
||||||
|
kind = "dataset"
|
||||||
|
|
||||||
|
def list_datasets(self) -> list[DatasetDefinition]:
|
||||||
|
return self._list_all()
|
||||||
|
|
||||||
|
def get_dataset(self, name: str, version: str | None = None) -> DatasetDefinition:
|
||||||
|
return self._get(name, version)
|
||||||
|
|
||||||
|
def get_version(self, name: str, version: str) -> DatasetDefinition:
|
||||||
|
return self._get_version(name, version)
|
||||||
|
|
||||||
|
def list_versions(self, name: str) -> list[DatasetVersionMeta]:
|
||||||
|
return self._list_versions(name)
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
"""Factories para AgentRegistry y PolicyStore basados en Settings."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from forja_core.config import Settings
|
||||||
|
from forja_core.registry.dataset_store import FileSystemDatasetStore
|
||||||
|
from forja_core.registry.model_store import FileSystemModelStore
|
||||||
|
from forja_core.registry.policy_store import FileSystemPolicyStore
|
||||||
|
from forja_core.registry.repository import FileSystemAgentRegistry
|
||||||
|
|
||||||
|
|
||||||
|
def build_agent_registry(settings: Settings) -> FileSystemAgentRegistry:
|
||||||
|
"""Construye el registry de agentes apuntando a ``settings.agents_dir``."""
|
||||||
|
return FileSystemAgentRegistry(settings.agents_dir)
|
||||||
|
|
||||||
|
|
||||||
|
def build_policy_store(settings: Settings) -> FileSystemPolicyStore:
|
||||||
|
"""Construye el store de políticas apuntando a ``settings.policies_dir``."""
|
||||||
|
return FileSystemPolicyStore(settings.policies_dir)
|
||||||
|
|
||||||
|
|
||||||
|
def build_dataset_store(settings: Settings) -> FileSystemDatasetStore:
|
||||||
|
return FileSystemDatasetStore(settings.datasets_dir)
|
||||||
|
|
||||||
|
|
||||||
|
def build_model_store(settings: Settings) -> FileSystemModelStore:
|
||||||
|
return FileSystemModelStore(settings.models_dir)
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
"""Read-only foundation model registry (YAML versioned)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from forja_core.domain.foundation_model import ModelDefinition, ModelVersionMeta
|
||||||
|
from forja_core.registry.yaml_store import VersionedYamlStore
|
||||||
|
|
||||||
|
|
||||||
|
class FileSystemModelStore(VersionedYamlStore[ModelDefinition]):
|
||||||
|
"""Reads models from models/<name>/{index.yaml,versions/vN.yaml}."""
|
||||||
|
|
||||||
|
model_cls = ModelDefinition
|
||||||
|
kind = "model"
|
||||||
|
|
||||||
|
def list_models(self) -> list[ModelDefinition]:
|
||||||
|
return self._list_all()
|
||||||
|
|
||||||
|
def get_model(self, name: str, version: str | None = None) -> ModelDefinition:
|
||||||
|
return self._get(name, version)
|
||||||
|
|
||||||
|
def get_version(self, name: str, version: str) -> ModelDefinition:
|
||||||
|
return self._get_version(name, version)
|
||||||
|
|
||||||
|
def list_versions(self, name: str) -> list[ModelVersionMeta]:
|
||||||
|
return self._list_versions(name)
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
"""PolicyStore basado en sistema de ficheros (YAML por versión + index.yaml)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from forja_core.domain.policy import PolicyDefinition, PolicyVersionMeta
|
||||||
|
from forja_core.registry.yaml_store import VersionedYamlStore
|
||||||
|
|
||||||
|
|
||||||
|
class FileSystemPolicyStore(VersionedYamlStore[PolicyDefinition]):
|
||||||
|
"""Lee políticas de policies/<name>/{index.yaml,versions/vN.yaml}."""
|
||||||
|
|
||||||
|
model_cls = PolicyDefinition
|
||||||
|
kind = "policy"
|
||||||
|
|
||||||
|
def list_policies(self) -> list[PolicyDefinition]:
|
||||||
|
return self._list_all()
|
||||||
|
|
||||||
|
def get_policy(self, name: str, version: str | None = None) -> PolicyDefinition:
|
||||||
|
return self._get(name, version)
|
||||||
|
|
||||||
|
def list_versions(self, name: str) -> list[PolicyVersionMeta]:
|
||||||
|
return self._list_versions(name)
|
||||||
|
|
||||||
|
def upsert_version(
|
||||||
|
self,
|
||||||
|
name: str,
|
||||||
|
body: PolicyDefinition,
|
||||||
|
message: str,
|
||||||
|
author: str,
|
||||||
|
) -> PolicyVersionMeta:
|
||||||
|
# Las políticas no tienen estado draft/active: la última versión escrita manda.
|
||||||
|
return self._upsert_version(name, body, body.version, message, author, set_active=True)
|
||||||
@@ -0,0 +1,67 @@
|
|||||||
|
"""AgentRegistry basado en sistema de ficheros (YAML versionado, simulación tipo Git)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
|
||||||
|
from forja_core.domain.agent import AgentDefinition, AgentVersionMeta
|
||||||
|
from forja_core.registry.versioning import DiffResult, unified_diff
|
||||||
|
from forja_core.registry.yaml_store import VersionedYamlStore
|
||||||
|
|
||||||
|
|
||||||
|
class FileSystemAgentRegistry(VersionedYamlStore[AgentDefinition]):
|
||||||
|
"""Lee/escribe agentes en agents/<name>/{index.yaml,versions/vN.yaml}."""
|
||||||
|
|
||||||
|
model_cls = AgentDefinition
|
||||||
|
kind = "agent"
|
||||||
|
|
||||||
|
def list_agents(self) -> list[AgentDefinition]:
|
||||||
|
return self._list_all()
|
||||||
|
|
||||||
|
def get_agent(self, name: str, version: str | None = None) -> AgentDefinition:
|
||||||
|
return self._get(name, version)
|
||||||
|
|
||||||
|
def get_version(self, name: str, version: str) -> AgentDefinition:
|
||||||
|
return self._get_version(name, version)
|
||||||
|
|
||||||
|
def list_versions(self, name: str) -> list[AgentVersionMeta]:
|
||||||
|
return self._list_versions(name)
|
||||||
|
|
||||||
|
def upsert_version(
|
||||||
|
self,
|
||||||
|
name: str,
|
||||||
|
body: AgentDefinition,
|
||||||
|
message: str,
|
||||||
|
author: str,
|
||||||
|
) -> AgentVersionMeta:
|
||||||
|
return self._upsert_version(
|
||||||
|
name,
|
||||||
|
body,
|
||||||
|
body.version,
|
||||||
|
message,
|
||||||
|
author,
|
||||||
|
set_active=body.state == "active",
|
||||||
|
)
|
||||||
|
|
||||||
|
def promote_to_active(
|
||||||
|
self,
|
||||||
|
name: str,
|
||||||
|
version: str,
|
||||||
|
author: str,
|
||||||
|
comment: str,
|
||||||
|
) -> AgentVersionMeta:
|
||||||
|
"""Set a version to active and update the registry index (governance promotion)."""
|
||||||
|
agent = self.get_version(name, version)
|
||||||
|
updated = agent.model_copy(update={"state": "active", "updated_at": datetime.now(UTC)})
|
||||||
|
return self.upsert_version(name, updated, message=comment, author=author)
|
||||||
|
|
||||||
|
def diff_versions(self, name: str, v1: str, v2: str) -> DiffResult:
|
||||||
|
path_a = self._root / name / "versions" / f"{v1}.yaml"
|
||||||
|
path_b = self._root / name / "versions" / f"{v2}.yaml"
|
||||||
|
text_a = path_a.read_text(encoding="utf-8") if path_a.exists() else ""
|
||||||
|
text_b = path_b.read_text(encoding="utf-8") if path_b.exists() else ""
|
||||||
|
return DiffResult(
|
||||||
|
from_version=v1,
|
||||||
|
to_version=v2,
|
||||||
|
unified_diff=unified_diff(text_a, text_b, f"{name}@{v1}", f"{name}@{v2}"),
|
||||||
|
)
|
||||||
+84
-77
@@ -1,49 +1,62 @@
|
|||||||
"""AgentRegistry basado en sistema de ficheros (YAML versionado, simulación tipo Git)."""
|
"""Base genérica de los registries versionados en YAML.
|
||||||
|
|
||||||
|
Layout común en disco: ``<root>/<name>/{index.yaml, versions/<vN>.yaml}``.
|
||||||
|
Agentes, políticas, datasets y modelos comparten esta mecánica; cada store
|
||||||
|
concreto fija ``model_cls`` (el modelo Pydantic de la definición) y ``kind``
|
||||||
|
(el nombre del artefacto en los mensajes de error) y expone wrappers con
|
||||||
|
nombres de dominio (``get_agent``, ``get_policy``, ...).
|
||||||
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any
|
from typing import Any, Generic, TypeVar
|
||||||
|
|
||||||
import yaml
|
import yaml
|
||||||
|
from pydantic import BaseModel
|
||||||
|
|
||||||
from agentforge_core.domain.agent import AgentDefinition, AgentVersionMeta
|
from forja_core.domain.version_meta import VersionMeta
|
||||||
from agentforge_core.registry.versioning import DiffResult, compute_hash, unified_diff
|
from forja_core.registry.versioning import compute_hash
|
||||||
|
|
||||||
|
M = TypeVar("M", bound=BaseModel)
|
||||||
|
|
||||||
|
|
||||||
class FileSystemAgentRegistry:
|
class VersionedYamlStore(Generic[M]):
|
||||||
"""Lee/escribe agentes en agents/<name>/{index.yaml,versions/vN.yaml}."""
|
"""Operaciones de lectura sobre un árbol de artefactos versionados."""
|
||||||
|
|
||||||
|
model_cls: type[M]
|
||||||
|
kind: str = "item"
|
||||||
|
|
||||||
def __init__(self, root: Path) -> None:
|
def __init__(self, root: Path) -> None:
|
||||||
self._root = Path(root)
|
self._root = Path(root)
|
||||||
|
|
||||||
def list_agents(self) -> list[AgentDefinition]:
|
def _list_all(self) -> list[M]:
|
||||||
if not self._root.exists():
|
if not self._root.exists():
|
||||||
return []
|
return []
|
||||||
agents: list[AgentDefinition] = []
|
out: list[M] = []
|
||||||
for d in sorted(self._root.iterdir()):
|
for d in sorted(self._root.iterdir()):
|
||||||
if d.is_dir() and (d / "index.yaml").exists():
|
if d.is_dir() and (d / "index.yaml").exists():
|
||||||
agents.append(self.get_agent(d.name))
|
out.append(self._get(d.name))
|
||||||
return agents
|
return out
|
||||||
|
|
||||||
def get_agent(self, name: str, version: str | None = None) -> AgentDefinition:
|
def _get(self, name: str, version: str | None = None) -> M:
|
||||||
index = self._load_index(name)
|
index = self._load_index(name)
|
||||||
v = version or index["active_version"]
|
v: str = version or index["active_version"]
|
||||||
return self.get_version(name, v)
|
return self._get_version(name, v)
|
||||||
|
|
||||||
def get_version(self, name: str, version: str) -> AgentDefinition:
|
def _get_version(self, name: str, version: str) -> M:
|
||||||
path = self._root / name / "versions" / f"{version}.yaml"
|
path = self._root / name / "versions" / f"{version}.yaml"
|
||||||
if not path.exists():
|
if not path.exists():
|
||||||
raise FileNotFoundError(f"agent version not found: {name}@{version}")
|
raise FileNotFoundError(f"{self.kind} version not found: {name}@{version}")
|
||||||
with path.open(encoding="utf-8") as f:
|
with path.open(encoding="utf-8") as f:
|
||||||
data = yaml.safe_load(f)
|
data = yaml.safe_load(f)
|
||||||
return AgentDefinition.model_validate(data)
|
return self.model_cls.model_validate(data)
|
||||||
|
|
||||||
def list_versions(self, name: str) -> list[AgentVersionMeta]:
|
def _list_versions(self, name: str) -> list[VersionMeta]:
|
||||||
index = self._load_index(name)
|
index = self._load_index(name)
|
||||||
return [
|
return [
|
||||||
AgentVersionMeta(
|
VersionMeta(
|
||||||
id=v["id"],
|
id=v["id"],
|
||||||
hash=v["hash"],
|
hash=v["hash"],
|
||||||
author=v["author"],
|
author=v["author"],
|
||||||
@@ -53,68 +66,10 @@ class FileSystemAgentRegistry:
|
|||||||
for v in index["versions"]
|
for v in index["versions"]
|
||||||
]
|
]
|
||||||
|
|
||||||
def upsert_version(
|
|
||||||
self,
|
|
||||||
name: str,
|
|
||||||
body: AgentDefinition,
|
|
||||||
message: str,
|
|
||||||
author: str,
|
|
||||||
) -> AgentVersionMeta:
|
|
||||||
agent_dir = self._root / name
|
|
||||||
versions_dir = agent_dir / "versions"
|
|
||||||
versions_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
|
|
||||||
version_id = body.version
|
|
||||||
yaml_text = yaml.safe_dump(
|
|
||||||
body.model_dump(mode="json"),
|
|
||||||
sort_keys=False,
|
|
||||||
allow_unicode=True,
|
|
||||||
)
|
|
||||||
h = compute_hash(yaml_text)
|
|
||||||
|
|
||||||
version_path = versions_dir / f"{version_id}.yaml"
|
|
||||||
version_path.write_text(yaml_text, encoding="utf-8")
|
|
||||||
|
|
||||||
meta = AgentVersionMeta(
|
|
||||||
id=version_id,
|
|
||||||
hash=h,
|
|
||||||
author=author,
|
|
||||||
message=message,
|
|
||||||
created_at=datetime.now(UTC),
|
|
||||||
)
|
|
||||||
|
|
||||||
index_path = agent_dir / "index.yaml"
|
|
||||||
if index_path.exists():
|
|
||||||
with index_path.open(encoding="utf-8") as f:
|
|
||||||
index: dict[str, Any] = yaml.safe_load(f)
|
|
||||||
else:
|
|
||||||
index = {"name": name, "versions": [], "active_version": version_id}
|
|
||||||
|
|
||||||
# reemplaza si existía la versión, si no añade
|
|
||||||
index["versions"] = [v for v in index["versions"] if v["id"] != version_id]
|
|
||||||
index["versions"].append(meta.model_dump(mode="json"))
|
|
||||||
if body.state == "active":
|
|
||||||
index["active_version"] = version_id
|
|
||||||
|
|
||||||
with index_path.open("w", encoding="utf-8") as f:
|
|
||||||
yaml.safe_dump(index, f, sort_keys=False, allow_unicode=True)
|
|
||||||
return meta
|
|
||||||
|
|
||||||
def diff_versions(self, name: str, v1: str, v2: str) -> DiffResult:
|
|
||||||
path_a = self._root / name / "versions" / f"{v1}.yaml"
|
|
||||||
path_b = self._root / name / "versions" / f"{v2}.yaml"
|
|
||||||
text_a = path_a.read_text(encoding="utf-8") if path_a.exists() else ""
|
|
||||||
text_b = path_b.read_text(encoding="utf-8") if path_b.exists() else ""
|
|
||||||
return DiffResult(
|
|
||||||
from_version=v1,
|
|
||||||
to_version=v2,
|
|
||||||
unified_diff=unified_diff(text_a, text_b, f"{name}@{v1}", f"{name}@{v2}"),
|
|
||||||
)
|
|
||||||
|
|
||||||
def _load_index(self, name: str) -> dict[str, Any]:
|
def _load_index(self, name: str) -> dict[str, Any]:
|
||||||
index_path = self._root / name / "index.yaml"
|
index_path = self._root / name / "index.yaml"
|
||||||
if not index_path.exists():
|
if not index_path.exists():
|
||||||
raise FileNotFoundError(f"agent not found: {name}")
|
raise FileNotFoundError(f"{self.kind} not found: {name}")
|
||||||
with index_path.open(encoding="utf-8") as f:
|
with index_path.open(encoding="utf-8") as f:
|
||||||
data: dict[str, Any] = yaml.safe_load(f)
|
data: dict[str, Any] = yaml.safe_load(f)
|
||||||
return data
|
return data
|
||||||
@@ -124,3 +79,55 @@ class FileSystemAgentRegistry:
|
|||||||
if isinstance(value, datetime):
|
if isinstance(value, datetime):
|
||||||
return value
|
return value
|
||||||
return datetime.fromisoformat(str(value).replace("Z", "+00:00"))
|
return datetime.fromisoformat(str(value).replace("Z", "+00:00"))
|
||||||
|
|
||||||
|
# --- Escritura (usada por agentes y políticas) -----------------------------------
|
||||||
|
|
||||||
|
def _upsert_version(
|
||||||
|
self,
|
||||||
|
name: str,
|
||||||
|
body: M,
|
||||||
|
version_id: str,
|
||||||
|
message: str,
|
||||||
|
author: str,
|
||||||
|
*,
|
||||||
|
set_active: bool,
|
||||||
|
) -> VersionMeta:
|
||||||
|
"""Escribe ``versions/<version_id>.yaml`` y actualiza ``index.yaml``.
|
||||||
|
|
||||||
|
Reemplaza la entrada del índice si la versión ya existía; si
|
||||||
|
``set_active`` es True, la marca como ``active_version``.
|
||||||
|
"""
|
||||||
|
artifact_dir = self._root / name
|
||||||
|
versions_dir = artifact_dir / "versions"
|
||||||
|
versions_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
yaml_text = yaml.safe_dump(
|
||||||
|
body.model_dump(mode="json"),
|
||||||
|
sort_keys=False,
|
||||||
|
allow_unicode=True,
|
||||||
|
)
|
||||||
|
(versions_dir / f"{version_id}.yaml").write_text(yaml_text, encoding="utf-8")
|
||||||
|
|
||||||
|
meta = VersionMeta(
|
||||||
|
id=version_id,
|
||||||
|
hash=compute_hash(yaml_text),
|
||||||
|
author=author,
|
||||||
|
message=message,
|
||||||
|
created_at=datetime.now(UTC),
|
||||||
|
)
|
||||||
|
|
||||||
|
index_path = artifact_dir / "index.yaml"
|
||||||
|
if index_path.exists():
|
||||||
|
with index_path.open(encoding="utf-8") as f:
|
||||||
|
index: dict[str, Any] = yaml.safe_load(f)
|
||||||
|
else:
|
||||||
|
index = {"name": name, "versions": [], "active_version": version_id}
|
||||||
|
|
||||||
|
index["versions"] = [v for v in index["versions"] if v["id"] != version_id]
|
||||||
|
index["versions"].append(meta.model_dump(mode="json"))
|
||||||
|
if set_active:
|
||||||
|
index["active_version"] = version_id
|
||||||
|
|
||||||
|
with index_path.open("w", encoding="utf-8") as f:
|
||||||
|
yaml.safe_dump(index, f, sort_keys=False, allow_unicode=True)
|
||||||
|
return meta
|
||||||
@@ -7,11 +7,11 @@ from typing import Any
|
|||||||
from langgraph.checkpoint.base import BaseCheckpointSaver
|
from langgraph.checkpoint.base import BaseCheckpointSaver
|
||||||
from langgraph.graph import END, START, StateGraph
|
from langgraph.graph import END, START, StateGraph
|
||||||
|
|
||||||
from agentforge_core.domain.agent import AgentDefinition
|
from forja_core.domain.agent import AgentDefinition
|
||||||
from agentforge_core.domain.policy import PolicyDefinition
|
from forja_core.domain.policy import PolicyDefinition
|
||||||
from agentforge_core.guardrails.base import GuardrailEngine
|
from forja_core.guardrails.base import GuardrailEngine
|
||||||
from agentforge_core.llm.base import LLMProvider
|
from forja_core.llm.base import LLMProvider
|
||||||
from agentforge_core.runtime.nodes import (
|
from forja_core.runtime.nodes import (
|
||||||
build_node_approve_gate,
|
build_node_approve_gate,
|
||||||
build_node_finalize,
|
build_node_finalize,
|
||||||
build_node_llm_reason,
|
build_node_llm_reason,
|
||||||
@@ -19,7 +19,7 @@ from agentforge_core.runtime.nodes import (
|
|||||||
build_node_validate_input,
|
build_node_validate_input,
|
||||||
build_node_validate_output,
|
build_node_validate_output,
|
||||||
)
|
)
|
||||||
from agentforge_core.runtime.state import AgentState
|
from forja_core.runtime.state import AgentState
|
||||||
|
|
||||||
|
|
||||||
def build_graph(
|
def build_graph(
|
||||||
@@ -12,11 +12,11 @@ from uuid import UUID
|
|||||||
import structlog
|
import structlog
|
||||||
from langgraph.types import interrupt
|
from langgraph.types import interrupt
|
||||||
|
|
||||||
from agentforge_core.domain.agent import AgentDefinition
|
from forja_core.domain.agent import AgentDefinition
|
||||||
from agentforge_core.domain.policy import PolicyDefinition
|
from forja_core.domain.policy import PolicyDefinition
|
||||||
from agentforge_core.guardrails.base import GuardrailEngine
|
from forja_core.guardrails.base import GuardrailEngine
|
||||||
from agentforge_core.llm.base import LLMProvider, Message
|
from forja_core.llm.base import LLMProvider, Message
|
||||||
from agentforge_core.runtime.state import AgentState
|
from forja_core.runtime.state import AgentState
|
||||||
|
|
||||||
log = structlog.get_logger(__name__)
|
log = structlog.get_logger(__name__)
|
||||||
|
|
||||||
+9
-11
@@ -18,14 +18,14 @@ from uuid import UUID, uuid4
|
|||||||
import structlog
|
import structlog
|
||||||
from langgraph.types import Command
|
from langgraph.types import Command
|
||||||
|
|
||||||
from agentforge_core.domain.agent import AgentDefinition
|
from forja_core.domain.agent import AgentDefinition
|
||||||
from agentforge_core.domain.execution import AgentExecution, DecisionStep, ProposedAction
|
from forja_core.domain.execution import AgentExecution, DecisionStep, ProposedAction
|
||||||
from agentforge_core.domain.guardrail import GuardrailViolation
|
from forja_core.domain.guardrail import GuardrailViolation
|
||||||
from agentforge_core.domain.policy import PolicyDefinition
|
from forja_core.domain.policy import PolicyDefinition
|
||||||
from agentforge_core.guardrails.base import GuardrailEngine
|
from forja_core.guardrails.base import GuardrailEngine
|
||||||
from agentforge_core.llm.base import LLMProvider
|
from forja_core.llm.base import LLMProvider
|
||||||
from agentforge_core.runtime.checkpointer import build_checkpointer
|
from forja_core.runtime.checkpointer import build_checkpointer
|
||||||
from agentforge_core.runtime.graph import build_graph
|
from forja_core.runtime.graph import build_graph
|
||||||
|
|
||||||
log = structlog.get_logger(__name__)
|
log = structlog.get_logger(__name__)
|
||||||
|
|
||||||
@@ -39,9 +39,7 @@ def _thread_config(trace_id: UUID) -> dict[str, Any]:
|
|||||||
class AgentOrchestrator:
|
class AgentOrchestrator:
|
||||||
"""Punto único de entrada para invocar agentes, reanudar HITL y leer su estado."""
|
"""Punto único de entrada para invocar agentes, reanudar HITL y leer su estado."""
|
||||||
|
|
||||||
def __init__(
|
def __init__(self, *, provider: LLMProvider, engine: GuardrailEngine, data_dir: Path) -> None:
|
||||||
self, *, provider: LLMProvider, engine: GuardrailEngine, data_dir: Path
|
|
||||||
) -> None:
|
|
||||||
self._provider = provider
|
self._provider = provider
|
||||||
self._engine = engine
|
self._engine = engine
|
||||||
self._data_dir = data_dir
|
self._data_dir = data_dir
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
"""External MLOps training backends (Strategy pattern)."""
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
"""Placeholder training script uploaded with Azure ML command jobs.
|
||||||
|
|
||||||
|
Replace this script in your workspace or override ``AZURE_ML_COMMAND`` for real fine-tuning.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
payload = {
|
||||||
|
"forja_run_id": os.environ.get("FORJA_RUN_ID"),
|
||||||
|
"dataset": os.environ.get("FORJA_DATASET"),
|
||||||
|
"model": os.environ.get("FORJA_MODEL"),
|
||||||
|
"agent": os.environ.get("FORJA_AGENT"),
|
||||||
|
"learning_rate": os.environ.get("FORJA_LEARNING_RATE"),
|
||||||
|
"epochs": os.environ.get("FORJA_EPOCHS"),
|
||||||
|
}
|
||||||
|
print(json.dumps(payload))
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -0,0 +1,67 @@
|
|||||||
|
"""Azure ML training backend — SDK when configured, stub otherwise."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
import structlog
|
||||||
|
|
||||||
|
from forja_core.config import Settings
|
||||||
|
from forja_core.domain.training import ArtifactRef, BackendJobStatus, TrainingJobSpec
|
||||||
|
from forja_core.training.azure_ml_config import is_azure_ml_configured
|
||||||
|
|
||||||
|
log = structlog.get_logger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class AzureMLTrainingBackend:
|
||||||
|
"""Submits command jobs via Azure ML SDK v2 when workspace settings are present."""
|
||||||
|
|
||||||
|
name = "azure_ml"
|
||||||
|
|
||||||
|
def __init__(self, settings: Settings | None = None) -> None:
|
||||||
|
self._settings = settings or Settings()
|
||||||
|
self._use_sdk = is_azure_ml_configured(self._settings)
|
||||||
|
if not self._use_sdk:
|
||||||
|
log.info(
|
||||||
|
"azure_ml_stub_mode",
|
||||||
|
reason="missing workspace config, compute, or azure-ai-ml packages",
|
||||||
|
)
|
||||||
|
|
||||||
|
async def submit(self, spec: TrainingJobSpec, run_id: UUID) -> str:
|
||||||
|
if self._use_sdk:
|
||||||
|
from forja_core.training.azure_ml_sdk import submit_command_job
|
||||||
|
|
||||||
|
return await asyncio.to_thread(submit_command_job, self._settings, spec, run_id)
|
||||||
|
return f"azureml-stub:{run_id}"
|
||||||
|
|
||||||
|
async def status(self, external_job_id: str) -> BackendJobStatus:
|
||||||
|
if external_job_id.startswith("azureml-stub:"):
|
||||||
|
return _stub_status(external_job_id)
|
||||||
|
if external_job_id.startswith("azureml:"):
|
||||||
|
job_name = external_job_id.removeprefix("azureml:")
|
||||||
|
if self._use_sdk:
|
||||||
|
from forja_core.training.azure_ml_sdk import poll_job_status
|
||||||
|
|
||||||
|
return await asyncio.to_thread(poll_job_status, self._settings, job_name)
|
||||||
|
return BackendJobStatus(
|
||||||
|
status="failed",
|
||||||
|
message="Azure ML SDK not configured; cannot poll live job",
|
||||||
|
)
|
||||||
|
return BackendJobStatus(
|
||||||
|
status="failed",
|
||||||
|
message=f"Unknown Azure ML job id: {external_job_id}",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _stub_status(external_job_id: str) -> BackendJobStatus:
|
||||||
|
return BackendJobStatus(
|
||||||
|
status="succeeded",
|
||||||
|
message="Azure ML stub job completed (workspace not configured)",
|
||||||
|
artifacts=[
|
||||||
|
ArtifactRef(
|
||||||
|
uri=f"azureml://workspaces/stub/jobs/{external_job_id}/outputs/adapter",
|
||||||
|
kind="adapter",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
)
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
"""Azure ML workspace settings helpers."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from forja_core.config import Settings
|
||||||
|
|
||||||
|
|
||||||
|
def sdk_packages_available() -> bool:
|
||||||
|
try:
|
||||||
|
import azure.ai.ml
|
||||||
|
import azure.identity # noqa: F401
|
||||||
|
|
||||||
|
return True
|
||||||
|
except ImportError:
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def is_azure_ml_configured(settings: Settings) -> bool:
|
||||||
|
return bool(
|
||||||
|
settings.azure_ml_subscription_id.strip()
|
||||||
|
and settings.azure_ml_resource_group.strip()
|
||||||
|
and settings.azure_ml_workspace_name.strip()
|
||||||
|
and settings.azure_ml_compute.strip()
|
||||||
|
and sdk_packages_available()
|
||||||
|
)
|
||||||
@@ -0,0 +1,100 @@
|
|||||||
|
"""Synchronous Azure ML SDK v2 client (invoked via asyncio.to_thread from the backend)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
import structlog
|
||||||
|
from azure.ai.ml import MLClient, command
|
||||||
|
from azure.identity import DefaultAzureCredential
|
||||||
|
|
||||||
|
from forja_core.config import Settings
|
||||||
|
from forja_core.domain.training import (
|
||||||
|
ArtifactRef,
|
||||||
|
BackendJobStatus,
|
||||||
|
TrainingJobSpec,
|
||||||
|
TrainingRunStatus,
|
||||||
|
)
|
||||||
|
|
||||||
|
log = structlog.get_logger(__name__)
|
||||||
|
|
||||||
|
JOB_CODE_DIR = Path(__file__).parent / "_azure_job_bundle"
|
||||||
|
|
||||||
|
_AZURE_TO_FORJA_STATUS: dict[str, TrainingRunStatus] = {
|
||||||
|
"notstarted": "queued",
|
||||||
|
"starting": "queued",
|
||||||
|
"provisioning": "queued",
|
||||||
|
"queued": "queued",
|
||||||
|
"running": "running",
|
||||||
|
"finalizing": "running",
|
||||||
|
"completed": "succeeded",
|
||||||
|
"failed": "failed",
|
||||||
|
"canceled": "cancelled",
|
||||||
|
"cancelled": "cancelled",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _ml_client(settings: Settings) -> MLClient:
|
||||||
|
return MLClient(
|
||||||
|
credential=DefaultAzureCredential(),
|
||||||
|
subscription_id=settings.azure_ml_subscription_id,
|
||||||
|
resource_group_name=settings.azure_ml_resource_group,
|
||||||
|
workspace_name=settings.azure_ml_workspace_name,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def submit_command_job(settings: Settings, spec: TrainingJobSpec, run_id: UUID) -> str:
|
||||||
|
"""Create an Azure ML command job; returns external id ``azureml:<job_name>``."""
|
||||||
|
client = _ml_client(settings)
|
||||||
|
display_name = f"forja-{run_id}"
|
||||||
|
env_vars = {
|
||||||
|
"FORJA_RUN_ID": str(run_id),
|
||||||
|
"FORJA_DATASET": f"{spec.dataset_name}@{spec.dataset_version}",
|
||||||
|
"FORJA_MODEL": f"{spec.model_name}@{spec.model_version}",
|
||||||
|
"FORJA_AGENT": spec.agent_name or "",
|
||||||
|
"FORJA_AGENT_VERSION": spec.agent_version or "",
|
||||||
|
"FORJA_LEARNING_RATE": str(spec.hyperparameters.learning_rate),
|
||||||
|
"FORJA_EPOCHS": str(spec.hyperparameters.epochs),
|
||||||
|
"FORJA_BATCH_SIZE": str(spec.hyperparameters.batch_size),
|
||||||
|
}
|
||||||
|
job_def = command(
|
||||||
|
code=str(JOB_CODE_DIR),
|
||||||
|
command=settings.azure_ml_command,
|
||||||
|
environment=settings.azure_ml_environment,
|
||||||
|
compute=settings.azure_ml_compute,
|
||||||
|
display_name=display_name,
|
||||||
|
experiment_name=settings.azure_ml_experiment_name,
|
||||||
|
environment_variables=env_vars,
|
||||||
|
)
|
||||||
|
created = client.jobs.create_or_update(job_def)
|
||||||
|
job_name = created.name
|
||||||
|
if job_name is None:
|
||||||
|
raise RuntimeError("Azure ML job created without a name")
|
||||||
|
log.info("azure_ml_job_submitted", job_name=job_name, run_id=str(run_id))
|
||||||
|
return f"azureml:{job_name}"
|
||||||
|
|
||||||
|
|
||||||
|
def poll_job_status(settings: Settings, job_name: str) -> BackendJobStatus:
|
||||||
|
client = _ml_client(settings)
|
||||||
|
job = client.jobs.get(job_name)
|
||||||
|
raw_status = str(job.status) if job.status is not None else "Queued"
|
||||||
|
raw = raw_status.split(".")[-1].lower()
|
||||||
|
forja_status = _AZURE_TO_FORJA_STATUS.get(raw, "running")
|
||||||
|
message = None
|
||||||
|
if forja_status == "failed":
|
||||||
|
message = "Azure ML job failed"
|
||||||
|
artifacts: list[ArtifactRef] = []
|
||||||
|
if forja_status == "succeeded":
|
||||||
|
artifacts.append(
|
||||||
|
ArtifactRef(
|
||||||
|
uri=(
|
||||||
|
f"azureml://subscriptions/{settings.azure_ml_subscription_id}"
|
||||||
|
f"/resourcegroups/{settings.azure_ml_resource_group}"
|
||||||
|
f"/workspaces/{settings.azure_ml_workspace_name}"
|
||||||
|
f"/jobs/{job_name}/outputs/default"
|
||||||
|
),
|
||||||
|
kind="adapter",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return BackendJobStatus(status=forja_status, message=message, artifacts=artifacts)
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
"""TrainingBackend protocol for external fine-tuning platforms."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Protocol
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from forja_core.domain.training import BackendJobStatus, TrainingJobSpec
|
||||||
|
|
||||||
|
|
||||||
|
class TrainingBackend(Protocol):
|
||||||
|
"""Submit and poll jobs on Azure ML, SageMaker, Hugging Face Jobs, etc."""
|
||||||
|
|
||||||
|
name: str
|
||||||
|
|
||||||
|
async def submit(self, spec: TrainingJobSpec, run_id: UUID) -> str:
|
||||||
|
"""Start a remote job; returns provider job id."""
|
||||||
|
...
|
||||||
|
|
||||||
|
async def status(self, external_job_id: str) -> BackendJobStatus:
|
||||||
|
"""Poll remote job status."""
|
||||||
|
...
|
||||||
@@ -0,0 +1,14 @@
|
|||||||
|
"""Build TrainingBackend from Settings."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from forja_core.config import Settings
|
||||||
|
from forja_core.training.azure_ml import AzureMLTrainingBackend
|
||||||
|
from forja_core.training.base import TrainingBackend
|
||||||
|
from forja_core.training.mock import MockTrainingBackend
|
||||||
|
|
||||||
|
|
||||||
|
def build_training_backend(settings: Settings) -> TrainingBackend:
|
||||||
|
if settings.training_backend == "azure_ml":
|
||||||
|
return AzureMLTrainingBackend(settings)
|
||||||
|
return MockTrainingBackend()
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
"""Mock training backend for local development and tests."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from forja_core.domain.training import ArtifactRef, BackendJobStatus, TrainingJobSpec
|
||||||
|
|
||||||
|
|
||||||
|
class MockTrainingBackend:
|
||||||
|
"""Immediately succeeds with a placeholder artifact URI."""
|
||||||
|
|
||||||
|
name = "mock"
|
||||||
|
|
||||||
|
async def submit(self, spec: TrainingJobSpec, run_id: UUID) -> str:
|
||||||
|
return f"mock:{run_id}"
|
||||||
|
|
||||||
|
async def status(self, external_job_id: str) -> BackendJobStatus:
|
||||||
|
return BackendJobStatus(
|
||||||
|
status="succeeded",
|
||||||
|
message="Mock training completed",
|
||||||
|
artifacts=[
|
||||||
|
ArtifactRef(
|
||||||
|
uri=f"{external_job_id}/adapter",
|
||||||
|
kind="adapter",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
)
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
"""Paquete de UI HTMX embebida (FastAPI + Jinja2 + HTMX)."""
|
||||||
@@ -0,0 +1,79 @@
|
|||||||
|
<div class="grid grid-cols-1 lg:grid-cols-3 gap-4">
|
||||||
|
|
||||||
|
<div class="lg:col-span-2 space-y-4">
|
||||||
|
{% for card in agent_cards %}
|
||||||
|
{% set agent = card.agent %}
|
||||||
|
<div class="forge-card border border-forge-iron rounded-2xl p-5">
|
||||||
|
<div class="flex items-start justify-between gap-3">
|
||||||
|
<div>
|
||||||
|
<div class="flex items-center gap-2 flex-wrap">
|
||||||
|
<span class="text-lg font-semibold tracking-tight">{{ agent.name }}</span>
|
||||||
|
{% for v in card.versions %}
|
||||||
|
<span class="text-[11px] px-2 py-0.5 rounded border font-mono
|
||||||
|
{% if v.state == 'active' %}bg-emerald-950 text-emerald-400 border-emerald-900
|
||||||
|
{% elif v.state == 'draft' %}bg-amber-950/60 text-amber-400 border-amber-900
|
||||||
|
{% else %}bg-forge-surface2 text-forge-muted border-forge-iron{% endif %}">
|
||||||
|
{{ v.id }} · {{ v.state }}
|
||||||
|
</span>
|
||||||
|
{% endfor %}
|
||||||
|
</div>
|
||||||
|
<p class="text-sm text-forge-muted mt-2 max-w-xl">{{ agent.purpose }}</p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="mt-3 flex flex-wrap gap-x-6 gap-y-1 text-xs text-forge-muted">
|
||||||
|
<span>LLM: <span class="font-mono text-forge-steel">{{ agent.llm.provider }}/{{ agent.llm.model }}</span></span>
|
||||||
|
<span>HITL threshold: <span class="font-mono text-forge-steel">risk ≥ {{ agent.risk_threshold_for_hitl }}</span></span>
|
||||||
|
<span>Policy:
|
||||||
|
{% for g in agent.guardrails %}
|
||||||
|
<span class="font-mono text-forge-steel">{{ g }}</span>{% if not loop.last %}, {% endif %}
|
||||||
|
{% endfor %}
|
||||||
|
</span>
|
||||||
|
<span>Owner: {{ agent.owner }}</span>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<details class="mt-3 group">
|
||||||
|
<summary class="cursor-pointer select-none text-xs text-forge-muted hover:text-forge-ember">
|
||||||
|
<span class="group-open:hidden">Show system prompt and output schema</span>
|
||||||
|
<span class="hidden group-open:inline">Hide system prompt and output schema</span>
|
||||||
|
</summary>
|
||||||
|
<div class="mt-2 p-3 bg-black/60 border border-forge-iron rounded-xl text-xs whitespace-pre-wrap font-light leading-relaxed">{{ agent.system_prompt }}</div>
|
||||||
|
<pre class="mt-2 text-[10px] bg-black/40 border border-forge-iron p-3 rounded-xl overflow-auto max-h-44 text-forge-steel">{{ agent.output_schema | tojson(indent=2) }}</pre>
|
||||||
|
</details>
|
||||||
|
</div>
|
||||||
|
{% else %}
|
||||||
|
<div class="forge-card border border-forge-iron rounded-2xl p-8 text-center text-forge-muted">No agents registered.</div>
|
||||||
|
{% endfor %}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="space-y-4">
|
||||||
|
{% for p in policies %}
|
||||||
|
<div class="forge-card border border-forge-iron rounded-2xl p-5">
|
||||||
|
<div class="flex items-baseline justify-between gap-2">
|
||||||
|
<div class="font-medium">{{ p.name }} <span class="text-xs text-forge-muted font-mono">{{ p.version }}</span></div>
|
||||||
|
<span class="text-[10px] px-2 py-0.5 rounded border font-mono
|
||||||
|
{{ 'bg-red-950/60 border-red-900 text-red-400' if p.on_validator_error == 'fail_closed' else 'bg-forge-surface2 border-forge-iron text-forge-steel' }}">
|
||||||
|
{{ p.on_validator_error }}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
<p class="text-xs text-forge-muted mt-1">{{ p.description }}</p>
|
||||||
|
<div class="mt-3 text-[11px]">
|
||||||
|
<div class="uppercase tracking-[1.5px] text-forge-muted mb-1">Input validators</div>
|
||||||
|
<div class="flex flex-wrap gap-1">
|
||||||
|
{% for v in p.input_validators %}
|
||||||
|
<span class="px-2 py-0.5 bg-black/40 border border-forge-iron rounded font-mono">{{ v.type }}</span>
|
||||||
|
{% endfor %}
|
||||||
|
</div>
|
||||||
|
<div class="uppercase tracking-[1.5px] text-forge-muted mb-1 mt-2">Output validators</div>
|
||||||
|
<div class="flex flex-wrap gap-1">
|
||||||
|
{% for v in p.output_validators %}
|
||||||
|
<span class="px-2 py-0.5 bg-black/40 border border-forge-iron rounded font-mono">{{ v.type }}</span>
|
||||||
|
{% endfor %}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
{% else %}
|
||||||
|
<div class="forge-card border border-forge-iron rounded-2xl p-6 text-center text-forge-muted text-sm">No policies registered.</div>
|
||||||
|
{% endfor %}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
@@ -0,0 +1,53 @@
|
|||||||
|
<div class="space-y-4">
|
||||||
|
{% for ex in pending %}
|
||||||
|
<div class="forge-card border border-amber-900/60 rounded-2xl p-5">
|
||||||
|
<div class="flex justify-between items-start">
|
||||||
|
<div>
|
||||||
|
<div class="font-medium">{{ ex.agent_name }} <span class="text-forge-muted">v{{ ex.agent_version }}</span></div>
|
||||||
|
<div class="text-xs text-forge-muted font-mono mt-0.5">{{ ex.trace_id }}</div>
|
||||||
|
</div>
|
||||||
|
<div class="text-amber-400 text-xs font-mono tracking-wide">Pending approval</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
{% if ex.needs_human_for %}
|
||||||
|
<div class="mt-4 space-y-1.5">
|
||||||
|
<div class="uppercase text-xs tracking-[1.5px] text-forge-muted">Actions awaiting approval</div>
|
||||||
|
{% for a in ex.needs_human_for %}
|
||||||
|
<div class="text-xs bg-black/50 border border-forge-iron rounded p-2.5">
|
||||||
|
<div>
|
||||||
|
<span class="font-mono">{{ a.action }}</span> → <span class="font-mono text-forge-steel">{{ a.target }}</span>
|
||||||
|
<span class="font-mono {{ 'text-amber-400' if a.risk_score < 5 else 'text-red-400' }}">(risk={{ a.risk_score }})</span>
|
||||||
|
</div>
|
||||||
|
<div class="text-forge-muted mt-1">Rollback: {{ a.rollback_plan }}</div>
|
||||||
|
</div>
|
||||||
|
{% endfor %}
|
||||||
|
</div>
|
||||||
|
{% endif %}
|
||||||
|
|
||||||
|
<div class="mt-4 flex gap-3">
|
||||||
|
<button
|
||||||
|
hx-post="/api/executions/{{ ex.trace_id }}/approve"
|
||||||
|
hx-ext="json-enc"
|
||||||
|
hx-vals='{"approved_action_ids": [], "comment": "Approved from UI"}'
|
||||||
|
hx-swap="none"
|
||||||
|
hx-on::after-request="htmx.trigger(document.body, 'forja-refresh')"
|
||||||
|
class="px-4 py-1.5 text-xs font-medium bg-emerald-600 hover:bg-emerald-500 text-white rounded transition">
|
||||||
|
Approve all
|
||||||
|
</button>
|
||||||
|
<button
|
||||||
|
hx-post="/api/executions/{{ ex.trace_id }}/reject"
|
||||||
|
hx-ext="json-enc"
|
||||||
|
hx-vals='{"reason": "Rejected from web UI"}'
|
||||||
|
hx-swap="none"
|
||||||
|
hx-on::after-request="htmx.trigger(document.body, 'forja-refresh')"
|
||||||
|
class="px-4 py-1.5 text-xs font-medium bg-red-700 hover:bg-red-600 text-white rounded transition">
|
||||||
|
Reject
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
{% else %}
|
||||||
|
<div class="forge-card border border-forge-iron rounded-2xl p-6 text-center text-forge-muted text-sm">
|
||||||
|
No runs are waiting for approval. Run the demo incident in stage <a href="#run" class="text-forge-ember hover:underline">02 — Run</a>.
|
||||||
|
</div>
|
||||||
|
{% endfor %}
|
||||||
|
</div>
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
<div class="space-y-3">
|
||||||
|
{% for ex in runs %}
|
||||||
|
<div class="forge-card border border-forge-iron rounded-2xl p-4">
|
||||||
|
<div class="flex items-center justify-between gap-4">
|
||||||
|
<div>
|
||||||
|
<span class="font-medium">{{ ex.agent_name }}</span>
|
||||||
|
<span class="text-xs text-forge-muted">v{{ ex.agent_version }}</span>
|
||||||
|
<span class="text-xs text-forge-muted font-mono ml-2">{{ ex.trace_id }}</span>
|
||||||
|
</div>
|
||||||
|
<div class="flex items-center gap-4 text-xs shrink-0">
|
||||||
|
<span class="text-forge-muted">{{ ex.n_proposed_actions }} action(s)</span>
|
||||||
|
<span class="{{ 'text-amber-400' if ex.n_violations > 0 else 'text-forge-muted' }}">{{ ex.n_violations }} violation(s)</span>
|
||||||
|
<span class="text-emerald-400 font-mono">Completed</span>
|
||||||
|
<span class="text-forge-muted">{{ ex.finished_at.strftime('%H:%M:%S') if ex.finished_at else '' }}</span>
|
||||||
|
<a href="/api/executions/{{ ex.trace_id }}" target="_blank" class="text-forge-ember hover:underline">JSON →</a>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
{% else %}
|
||||||
|
<div class="forge-card border border-forge-iron rounded-2xl p-6 text-center text-forge-muted text-sm">
|
||||||
|
No completed runs yet. They appear here after stage <a href="#approve" class="text-forge-ember hover:underline">03 — Approve</a> (or directly when no approval is needed).
|
||||||
|
</div>
|
||||||
|
{% endfor %}
|
||||||
|
{% if total_completed > runs | length %}
|
||||||
|
<div class="text-xs text-forge-muted text-center pt-1">Showing the {{ runs | length }} most recent of {{ total_completed }} completed runs.</div>
|
||||||
|
{% endif %}
|
||||||
|
</div>
|
||||||
@@ -0,0 +1,61 @@
|
|||||||
|
<div class="space-y-4">
|
||||||
|
{% for item in pending %}
|
||||||
|
{% set req = item.request %}
|
||||||
|
{% set ev = item.evaluation %}
|
||||||
|
<div class="forge-card border border-amber-900/60 rounded-2xl p-5">
|
||||||
|
<div class="flex justify-between items-start gap-4">
|
||||||
|
<div>
|
||||||
|
<div class="font-medium text-lg">{{ req.agent_name }} <span class="text-forge-muted">{{ req.agent_version }} → active</span></div>
|
||||||
|
<div class="text-xs text-forge-muted font-mono mt-1">request {{ req.id }}</div>
|
||||||
|
</div>
|
||||||
|
<div class="text-amber-400 text-xs font-mono tracking-wide shrink-0">Pending promotion</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="mt-4 grid grid-cols-2 sm:grid-cols-3 gap-3 text-xs">
|
||||||
|
<div>
|
||||||
|
<span class="text-forge-muted uppercase tracking-wide">Training run</span>
|
||||||
|
<div class="font-mono mt-0.5 truncate">{{ req.training_run_id }}</div>
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<span class="text-forge-muted uppercase tracking-wide">Evaluation</span>
|
||||||
|
<div class="mt-0.5 {% if ev and ev.passed %}text-emerald-400{% else %}text-red-400{% endif %}">
|
||||||
|
{% if ev %}{{ "Passed" if ev.passed else "Failed" }} ({{ ev.agent_name }}@{{ ev.agent_version }}){% else %}Unknown{% endif %}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<span class="text-forge-muted uppercase tracking-wide">Scenarios</span>
|
||||||
|
<div class="mt-0.5">{% if ev %}{{ ev.scenario_count }} run, {{ ev.violation_count }} blocking{% else %}—{% endif %}</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="text-xs text-forge-muted mt-3">
|
||||||
|
Requested by {{ req.requested_by }} · {{ req.created_at.strftime('%Y-%m-%d %H:%M') if req.created_at else '' }}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="mt-4 flex gap-3">
|
||||||
|
<button
|
||||||
|
hx-post="/api/promotions/{{ req.id }}/approve"
|
||||||
|
hx-ext="json-enc"
|
||||||
|
hx-vals='{"approved_by": "web-ui", "comment": "Approved from Promotions UI"}'
|
||||||
|
hx-swap="none"
|
||||||
|
hx-on::after-request="htmx.trigger(document.body, 'forja-refresh')"
|
||||||
|
class="px-4 py-1.5 text-xs font-medium bg-emerald-600 hover:bg-emerald-500 text-white rounded transition">
|
||||||
|
Approve promotion
|
||||||
|
</button>
|
||||||
|
<button
|
||||||
|
hx-post="/api/promotions/{{ req.id }}/reject"
|
||||||
|
hx-ext="json-enc"
|
||||||
|
hx-vals='{"rejected_by": "web-ui", "reason": "Rejected from Promotions UI"}'
|
||||||
|
hx-swap="none"
|
||||||
|
hx-on::after-request="htmx.trigger(document.body, 'forja-refresh')"
|
||||||
|
class="px-4 py-1.5 text-xs font-medium bg-red-700 hover:bg-red-600 text-white rounded transition">
|
||||||
|
Reject
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
{% else %}
|
||||||
|
<div class="forge-card border border-forge-iron rounded-2xl p-6 text-center text-forge-muted text-sm">
|
||||||
|
No promotion requests are waiting. Request one from stage <a href="#train" class="text-forge-ember hover:underline">04 — Train</a> after a passing evaluation.
|
||||||
|
</div>
|
||||||
|
{% endfor %}
|
||||||
|
</div>
|
||||||
@@ -0,0 +1,98 @@
|
|||||||
|
<div class="space-y-4">
|
||||||
|
{% for item in items %}
|
||||||
|
{% set run = item.run %}
|
||||||
|
{% set ev = item.evaluation %}
|
||||||
|
<div class="forge-card border border-forge-iron rounded-2xl p-5">
|
||||||
|
<div class="flex justify-between items-start gap-4">
|
||||||
|
<div>
|
||||||
|
<div class="font-medium">
|
||||||
|
{{ run.spec.dataset_name }}@{{ run.spec.dataset_version }}
|
||||||
|
<span class="text-forge-muted">→</span>
|
||||||
|
{{ run.spec.model_name }}@{{ run.spec.model_version }}
|
||||||
|
</div>
|
||||||
|
<div class="text-xs text-forge-muted font-mono mt-1">run {{ run.id }}</div>
|
||||||
|
{% if run.spec.agent_name %}
|
||||||
|
<div class="text-xs text-forge-muted mt-1">
|
||||||
|
Candidate: <span class="font-mono text-forge-steel">{{ run.spec.agent_name }}{% if run.spec.agent_version %}@{{ run.spec.agent_version }}{% endif %}</span>
|
||||||
|
{% if item.candidate_state %}
|
||||||
|
<span class="ml-1 {{ 'text-emerald-400' if item.candidate_state == 'active' else 'text-amber-400' }}">({{ item.candidate_state }})</span>
|
||||||
|
{% endif %}
|
||||||
|
</div>
|
||||||
|
{% endif %}
|
||||||
|
</div>
|
||||||
|
<span class="text-xs font-mono tracking-wide shrink-0
|
||||||
|
{{ 'text-emerald-400' if run.status == 'succeeded'
|
||||||
|
else ('text-red-400' if run.status in ['failed', 'cancelled']
|
||||||
|
else 'text-amber-400') }}">
|
||||||
|
{{ run.status }}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
{% if run.error_message %}
|
||||||
|
<p class="mt-2 text-xs text-red-400">{{ run.error_message }}</p>
|
||||||
|
{% endif %}
|
||||||
|
|
||||||
|
{% if ev %}
|
||||||
|
<div class="mt-3 text-xs">
|
||||||
|
<span class="uppercase tracking-wide text-forge-muted">Evaluation</span>
|
||||||
|
<span class="ml-2 {{ 'text-emerald-400' if ev.passed else 'text-red-400' }}">
|
||||||
|
{{ "Passed" if ev.passed else "Failed" }}
|
||||||
|
</span>
|
||||||
|
<span class="text-forge-muted ml-2">{{ ev.scenario_count }} scenario(s), {{ ev.violation_count }} blocking violation(s) — covers {{ ev.agent_name }}@{{ ev.agent_version }}</span>
|
||||||
|
{% if ev.notes %}<span class="text-forge-muted ml-2">— {{ ev.notes }}</span>{% endif %}
|
||||||
|
</div>
|
||||||
|
{% endif %}
|
||||||
|
|
||||||
|
<div class="mt-4 flex flex-wrap items-center gap-3">
|
||||||
|
{% if run.status not in ['succeeded', 'failed', 'cancelled'] %}
|
||||||
|
<button
|
||||||
|
hx-post="/api/training/runs/{{ run.id }}/refresh"
|
||||||
|
hx-ext="json-enc"
|
||||||
|
hx-vals='{}'
|
||||||
|
hx-swap="none"
|
||||||
|
hx-on::after-request="htmx.trigger(document.body, 'forja-refresh')"
|
||||||
|
class="px-4 py-1.5 text-xs font-medium bg-forge-surface2 border border-forge-iron hover:border-forge-ember rounded transition">
|
||||||
|
Refresh status
|
||||||
|
</button>
|
||||||
|
{% endif %}
|
||||||
|
|
||||||
|
{% if run.status == 'succeeded' and run.spec.agent_name and not ev %}
|
||||||
|
<button
|
||||||
|
hx-post="/api/training/runs/{{ run.id }}/evaluate"
|
||||||
|
hx-ext="json-enc"
|
||||||
|
hx-vals='{}'
|
||||||
|
hx-swap="none"
|
||||||
|
hx-on::after-request="htmx.trigger(document.body, 'forja-refresh')"
|
||||||
|
class="px-4 py-1.5 text-xs font-medium bg-forge-ember text-black hover:brightness-105 rounded transition">
|
||||||
|
Run evaluation
|
||||||
|
</button>
|
||||||
|
<span class="text-xs text-forge-muted">Runs the candidate over its canonical scenarios.</span>
|
||||||
|
{% endif %}
|
||||||
|
|
||||||
|
{% if ev and ev.passed and run.spec.agent_version %}
|
||||||
|
{% if item.candidate_state == 'active' %}
|
||||||
|
<span class="text-xs text-emerald-400">✓ Promoted — {{ run.spec.agent_name }}@{{ run.spec.agent_version }} is active (see stage 01).</span>
|
||||||
|
{% elif item.promotion_pending %}
|
||||||
|
<span class="text-xs text-amber-400">Promotion pending approval in stage <a href="#promote" class="underline">05 — Promote</a>.</span>
|
||||||
|
{% else %}
|
||||||
|
<button
|
||||||
|
hx-post="/api/promotions"
|
||||||
|
hx-ext="json-enc"
|
||||||
|
hx-vals='{"agent_name": "{{ run.spec.agent_name }}", "agent_version": "{{ run.spec.agent_version }}", "training_run_id": "{{ run.id }}", "evaluation_report_id": "{{ ev.id }}", "requested_by": "web-ui"}'
|
||||||
|
hx-swap="none"
|
||||||
|
hx-on::after-request="htmx.trigger(document.body, 'forja-refresh')"
|
||||||
|
class="px-4 py-1.5 text-xs font-medium bg-emerald-600 hover:bg-emerald-500 text-white rounded transition">
|
||||||
|
Request promotion
|
||||||
|
</button>
|
||||||
|
{% endif %}
|
||||||
|
{% elif ev and ev.passed and not run.spec.agent_version %}
|
||||||
|
<span class="text-xs text-forge-muted">Set a candidate version on the run to request promotion.</span>
|
||||||
|
{% endif %}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
{% else %}
|
||||||
|
<div class="forge-card border border-forge-iron rounded-2xl p-6 text-center text-forge-muted text-sm">
|
||||||
|
No training runs yet. Submit one above — the candidate version is prefilled with the agent's draft.
|
||||||
|
</div>
|
||||||
|
{% endfor %}
|
||||||
|
</div>
|
||||||
@@ -0,0 +1,95 @@
|
|||||||
|
<!doctype html>
|
||||||
|
<html lang="en">
|
||||||
|
<head>
|
||||||
|
<meta charset="utf-8">
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||||
|
<title>Forja • {{ title or "Forja" }}</title>
|
||||||
|
<style>
|
||||||
|
:root {
|
||||||
|
--forge-bg: #1c1c1c;
|
||||||
|
--forge-surface: #151515;
|
||||||
|
--forge-surface2: #242424;
|
||||||
|
--forge-iron: #333333;
|
||||||
|
--forge-ember: #f97316;
|
||||||
|
--forge-gold: #fcd34d;
|
||||||
|
--forge-steel: #94a3b8;
|
||||||
|
--forge-text: #e7e5e4;
|
||||||
|
--forge-muted: #78716c;
|
||||||
|
}
|
||||||
|
</style>
|
||||||
|
<script src="https://cdn.tailwindcss.com"></script>
|
||||||
|
<script>
|
||||||
|
/* Must run right after the CDN script (before the body is parsed) so the
|
||||||
|
custom palette applies to the first paint. Single source of truth:
|
||||||
|
the CSS variables above. */
|
||||||
|
tailwind.config = {
|
||||||
|
theme: {
|
||||||
|
extend: {
|
||||||
|
colors: {
|
||||||
|
'forge': {
|
||||||
|
'bg': 'var(--forge-bg)',
|
||||||
|
'surface': 'var(--forge-surface)',
|
||||||
|
'surface2': 'var(--forge-surface2)',
|
||||||
|
'iron': 'var(--forge-iron)',
|
||||||
|
'ember': 'var(--forge-ember)',
|
||||||
|
'gold': 'var(--forge-gold)',
|
||||||
|
'steel': 'var(--forge-steel)',
|
||||||
|
'text': 'var(--forge-text)',
|
||||||
|
'muted': 'var(--forge-muted)',
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
</script>
|
||||||
|
<script src="https://unpkg.com/htmx.org@2.0.3/dist/htmx.min.js"></script>
|
||||||
|
<!-- Encode HTMX requests as JSON so buttons/forms can target the /api endpoints. -->
|
||||||
|
<script src="https://unpkg.com/htmx-ext-json-enc@2.0.2/json-enc.js"></script>
|
||||||
|
<style>
|
||||||
|
html { scroll-behavior: smooth; scroll-padding-top: 5rem; }
|
||||||
|
body { font-family: ui-sans-serif, system-ui, sans-serif; }
|
||||||
|
.htmx-indicator { display: none; }
|
||||||
|
.htmx-request .htmx-indicator { display: inline; }
|
||||||
|
.htmx-request.htmx-indicator { display: inline; }
|
||||||
|
|
||||||
|
body {
|
||||||
|
background-color: var(--forge-bg);
|
||||||
|
color: var(--forge-text);
|
||||||
|
}
|
||||||
|
|
||||||
|
.forge-header {
|
||||||
|
background-color: var(--forge-surface);
|
||||||
|
border-color: var(--forge-iron);
|
||||||
|
}
|
||||||
|
|
||||||
|
.forge-card {
|
||||||
|
background-color: var(--forge-surface);
|
||||||
|
border-color: var(--forge-iron);
|
||||||
|
}
|
||||||
|
</style>
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<div class="min-h-screen">
|
||||||
|
<header class="forge-header border-b sticky top-0 z-50">
|
||||||
|
<div class="max-w-5xl mx-auto px-6 h-14 flex items-center justify-between">
|
||||||
|
<a href="/" class="flex flex-col hover:text-forge-ember transition">
|
||||||
|
<span class="font-semibold tracking-wide text-lg">Forja</span>
|
||||||
|
<span class="text-[10px] text-forge-muted">Governed models and agents</span>
|
||||||
|
</a>
|
||||||
|
<nav class="flex items-center gap-x-5 text-sm">
|
||||||
|
<a href="/#define" class="hover:text-forge-ember transition">Define</a>
|
||||||
|
<a href="/#run" class="hover:text-forge-ember transition">Run</a>
|
||||||
|
<a href="/#approve" class="hover:text-forge-ember transition">Approve</a>
|
||||||
|
<a href="/#train" class="hover:text-forge-ember transition">Train</a>
|
||||||
|
<a href="/#promote" class="hover:text-forge-ember transition">Promote</a>
|
||||||
|
<a href="/#audit" class="hover:text-forge-ember transition">Audit</a>
|
||||||
|
</nav>
|
||||||
|
</div>
|
||||||
|
</header>
|
||||||
|
|
||||||
|
<main class="max-w-5xl mx-auto px-6 py-8 text-forge-text">
|
||||||
|
{% block content %}{% endblock %}
|
||||||
|
</main>
|
||||||
|
</div>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
@@ -0,0 +1,223 @@
|
|||||||
|
{% extends "base.html" %}
|
||||||
|
|
||||||
|
{% macro stage_header(number, anchor, name, description) %}
|
||||||
|
<div class="flex items-center gap-3 mb-1 pt-2" id="{{ anchor }}">
|
||||||
|
<span class="font-mono text-xs px-2 py-px border border-forge-ember/60 text-forge-ember rounded bg-forge-surface2">{{ number }}</span>
|
||||||
|
<h2 class="text-2xl font-semibold tracking-tight">{{ name }}</h2>
|
||||||
|
</div>
|
||||||
|
<p class="text-sm text-forge-muted mb-4 max-w-2xl">{{ description }}</p>
|
||||||
|
{% endmacro %}
|
||||||
|
|
||||||
|
{% block content %}
|
||||||
|
<div class="pb-16">
|
||||||
|
|
||||||
|
<div class="mb-10">
|
||||||
|
<h1 class="text-3xl font-medium tracking-tight">The governed agent lifecycle, on one page.</h1>
|
||||||
|
<p class="text-sm text-forge-muted mt-2 max-w-2xl">
|
||||||
|
Six stages, top to bottom. Every action below is a click — run the demo incident,
|
||||||
|
approve it, train the draft candidate, evaluate it, and promote it to active.
|
||||||
|
Each panel refreshes itself after every action.
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- ============================ 01 DEFINE ============================ -->
|
||||||
|
{{ stage_header("01", "define", "Define",
|
||||||
|
"Agents and the guardrail policies they run under — all versioned YAML.
|
||||||
|
The draft version below is the candidate that the rest of this page trains,
|
||||||
|
evaluates, and promotes.") }}
|
||||||
|
|
||||||
|
<div id="agents-panel"
|
||||||
|
hx-get="/fragments/agents"
|
||||||
|
hx-trigger="forja-refresh from:body"
|
||||||
|
hx-swap="innerHTML">
|
||||||
|
{% include "_agents.html" %}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- ============================ 02 RUN =============================== -->
|
||||||
|
<div class="mt-12"></div>
|
||||||
|
{{ stage_header("02", "run", "Run",
|
||||||
|
"Execute the active agent through the governed pipeline: input guardrails →
|
||||||
|
LLM → output guardrails → proposed actions → approval gate. The demo incident
|
||||||
|
proposes a risk-5 action, so the run pauses for human approval in stage 03.") }}
|
||||||
|
|
||||||
|
<div class="forge-card border border-forge-iron rounded-2xl p-6">
|
||||||
|
<div class="flex flex-wrap items-center gap-3 mb-5 pb-5 border-b border-forge-iron">
|
||||||
|
<button
|
||||||
|
hx-post="/run"
|
||||||
|
hx-vals='{"agent_name": "incident_analyzer", "input": "{{ demo_input }}"}'
|
||||||
|
hx-target="#run-result"
|
||||||
|
hx-swap="innerHTML"
|
||||||
|
hx-on::after-request="htmx.trigger(document.body, 'forja-refresh')"
|
||||||
|
class="px-5 py-2.5 bg-forge-ember text-black font-semibold rounded-lg text-sm hover:brightness-105 active:scale-[0.985] transition">
|
||||||
|
▶ Run demo incident
|
||||||
|
<span class="htmx-indicator">⋯</span>
|
||||||
|
</button>
|
||||||
|
<span class="text-xs text-forge-muted">One click — sends a replicated HSS capacity alarm and pauses at the approval gate.</span>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<form hx-post="/run" hx-target="#run-result" hx-swap="innerHTML"
|
||||||
|
hx-on::after-request="htmx.trigger(document.body, 'forja-refresh')"
|
||||||
|
class="space-y-4">
|
||||||
|
<div class="grid grid-cols-1 sm:grid-cols-2 gap-4">
|
||||||
|
<div>
|
||||||
|
<label class="block text-xs tracking-widest text-forge-muted mb-1.5 uppercase">Agent</label>
|
||||||
|
<select name="agent_name" id="agent-select"
|
||||||
|
class="w-full bg-black/50 border border-forge-iron rounded-lg px-3 py-2.5 text-sm focus:border-forge-ember outline-none">
|
||||||
|
{% for a in agents %}
|
||||||
|
<option value="{{ a.name }}"
|
||||||
|
data-label="{{ a.input_label or 'Input' }}"
|
||||||
|
data-placeholder="{{ a.input_placeholder or '' }}">
|
||||||
|
{{ a.name }} (v{{ a.version }})
|
||||||
|
</option>
|
||||||
|
{% endfor %}
|
||||||
|
</select>
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<label id="input-label" class="block text-xs tracking-widest text-forge-muted mb-1.5 uppercase">Input</label>
|
||||||
|
<textarea name="input" id="input-text" rows="3"
|
||||||
|
class="w-full bg-black/50 border border-forge-iron rounded-lg px-3 py-2 font-mono text-sm focus:border-forge-ember outline-none"></textarea>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<button type="submit"
|
||||||
|
class="px-5 py-2 bg-forge-surface2 border border-forge-iron hover:border-forge-ember rounded-lg text-sm transition">
|
||||||
|
Run with custom input
|
||||||
|
<span class="htmx-indicator">⋯</span>
|
||||||
|
</button>
|
||||||
|
</form>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div id="run-result" class="mt-4"></div>
|
||||||
|
|
||||||
|
<!-- ============================ 03 APPROVE =========================== -->
|
||||||
|
<div class="mt-12"></div>
|
||||||
|
{{ stage_header("03", "approve", "Approve",
|
||||||
|
"Runs paused by the approval gate (HITL). The state lives in a checkpoint on
|
||||||
|
disk, so it survives restarts. Approving resumes the graph exactly where it
|
||||||
|
stopped; the completed run lands in stage 06.") }}
|
||||||
|
|
||||||
|
<div id="approvals-panel"
|
||||||
|
hx-get="/fragments/approvals"
|
||||||
|
hx-trigger="forja-refresh from:body"
|
||||||
|
hx-swap="innerHTML">
|
||||||
|
{% include "_approvals.html" %}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- ============================ 04 TRAIN ============================= -->
|
||||||
|
<div class="mt-12"></div>
|
||||||
|
{{ stage_header("04", "train", "Train & evaluate",
|
||||||
|
"Submit a fine-tuning job for the draft candidate to the configured MLOps
|
||||||
|
backend, then evaluate it over the agent's canonical scenarios through the
|
||||||
|
full governed runtime. A passing evaluation unlocks promotion.") }}
|
||||||
|
|
||||||
|
<div class="forge-card border border-forge-iron rounded-2xl p-6 mb-4">
|
||||||
|
<form hx-post="/api/training/runs"
|
||||||
|
hx-ext="json-enc"
|
||||||
|
hx-swap="none"
|
||||||
|
hx-on::after-request="htmx.trigger(document.body, 'forja-refresh');
|
||||||
|
this.querySelector('.form-error').textContent =
|
||||||
|
event.detail.successful ? '' : 'Request failed: ' + event.detail.xhr.responseText.slice(0, 200)"
|
||||||
|
class="grid grid-cols-2 sm:grid-cols-4 gap-4 items-end">
|
||||||
|
<div>
|
||||||
|
<label class="block text-xs tracking-widest text-forge-muted mb-1.5 uppercase">Dataset</label>
|
||||||
|
<select name="dataset_name" class="w-full bg-black/50 border border-forge-iron rounded-lg px-3 py-2 text-sm focus:border-forge-ember outline-none">
|
||||||
|
{% for d in datasets %}
|
||||||
|
<option value="{{ d.name }}">{{ d.name }}</option>
|
||||||
|
{% endfor %}
|
||||||
|
</select>
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<label class="block text-xs tracking-widest text-forge-muted mb-1.5 uppercase">Base model</label>
|
||||||
|
<select name="model_name" class="w-full bg-black/50 border border-forge-iron rounded-lg px-3 py-2 text-sm focus:border-forge-ember outline-none">
|
||||||
|
{% for m in models %}
|
||||||
|
<option value="{{ m.name }}">{{ m.name }}</option>
|
||||||
|
{% endfor %}
|
||||||
|
</select>
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<label class="block text-xs tracking-widest text-forge-muted mb-1.5 uppercase">Agent</label>
|
||||||
|
<select name="agent_name" id="train-agent-select"
|
||||||
|
class="w-full bg-black/50 border border-forge-iron rounded-lg px-3 py-2 text-sm focus:border-forge-ember outline-none">
|
||||||
|
{% for card in agent_cards %}
|
||||||
|
{% set draft = card.versions | selectattr('state', 'equalto', 'draft') | map(attribute='id') | first %}
|
||||||
|
<option value="{{ card.agent.name }}" data-draft="{{ draft or '' }}">{{ card.agent.name }}</option>
|
||||||
|
{% endfor %}
|
||||||
|
</select>
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<label class="block text-xs tracking-widest text-forge-muted mb-1.5 uppercase">Candidate version</label>
|
||||||
|
<input name="agent_version" id="candidate-version" type="text"
|
||||||
|
class="w-full bg-black/50 border border-forge-iron rounded-lg px-3 py-2 text-sm font-mono focus:border-forge-ember outline-none">
|
||||||
|
</div>
|
||||||
|
<div class="col-span-2 sm:col-span-4 flex items-center gap-4">
|
||||||
|
<button type="submit"
|
||||||
|
class="px-5 py-2 bg-forge-ember text-black font-semibold rounded-lg text-sm hover:brightness-105 active:scale-[0.985] transition">
|
||||||
|
Submit training run
|
||||||
|
</button>
|
||||||
|
<span class="text-xs text-forge-muted">backend: <span class="font-mono">{{ backend }}</span> — the mock backend completes instantly.</span>
|
||||||
|
<span class="form-error text-xs text-red-400"></span>
|
||||||
|
</div>
|
||||||
|
</form>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div id="training-panel"
|
||||||
|
hx-get="/fragments/training"
|
||||||
|
hx-trigger="forja-refresh from:body"
|
||||||
|
hx-swap="innerHTML">
|
||||||
|
{% include "_training.html" %}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- ============================ 05 PROMOTE =========================== -->
|
||||||
|
<div class="mt-12"></div>
|
||||||
|
{{ stage_header("05", "promote", "Promote",
|
||||||
|
"Governance queue. A promotion requires a succeeded training run and a passing
|
||||||
|
evaluation of the exact candidate version. Approving flips the candidate to
|
||||||
|
active in the registry — check stage 01 after approving.") }}
|
||||||
|
|
||||||
|
<div id="promotions-panel"
|
||||||
|
hx-get="/fragments/promotions"
|
||||||
|
hx-trigger="forja-refresh from:body"
|
||||||
|
hx-swap="innerHTML">
|
||||||
|
{% include "_promotions.html" %}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- ============================ 06 AUDIT ============================= -->
|
||||||
|
<div class="mt-12"></div>
|
||||||
|
{{ stage_header("06", "audit", "Audit",
|
||||||
|
"Completed runs, persisted append-only with their full decision trail,
|
||||||
|
violations, and approved actions.") }}
|
||||||
|
|
||||||
|
<div id="history-panel"
|
||||||
|
hx-get="/fragments/history"
|
||||||
|
hx-trigger="forja-refresh from:body"
|
||||||
|
hx-swap="innerHTML">
|
||||||
|
{% include "_history.html" %}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<script>
|
||||||
|
/* Run form: apply the selected agent's input label and placeholder (from its YAML). */
|
||||||
|
const agentSelect = document.getElementById("agent-select");
|
||||||
|
const inputLabel = document.getElementById("input-label");
|
||||||
|
const inputText = document.getElementById("input-text");
|
||||||
|
function syncAgentHints() {
|
||||||
|
const opt = agentSelect.selectedOptions[0];
|
||||||
|
if (!opt) return;
|
||||||
|
inputLabel.textContent = opt.dataset.label || "Input";
|
||||||
|
inputText.placeholder = opt.dataset.placeholder || "";
|
||||||
|
}
|
||||||
|
agentSelect.addEventListener("change", syncAgentHints);
|
||||||
|
syncAgentHints();
|
||||||
|
|
||||||
|
/* Train form: prefill the candidate version with the agent's first draft version. */
|
||||||
|
const trainAgentSelect = document.getElementById("train-agent-select");
|
||||||
|
const candidateVersion = document.getElementById("candidate-version");
|
||||||
|
function syncCandidateVersion() {
|
||||||
|
const opt = trainAgentSelect.selectedOptions[0];
|
||||||
|
if (!opt) return;
|
||||||
|
candidateVersion.value = opt.dataset.draft || "";
|
||||||
|
}
|
||||||
|
trainAgentSelect.addEventListener("change", syncCandidateVersion);
|
||||||
|
syncCandidateVersion();
|
||||||
|
</script>
|
||||||
|
{% endblock %}
|
||||||
@@ -0,0 +1,68 @@
|
|||||||
|
<div class="forge-card border border-forge-iron rounded-2xl p-5">
|
||||||
|
<div class="flex items-center justify-between">
|
||||||
|
<div class="flex items-center gap-2">
|
||||||
|
<span class="font-semibold tracking-tight">Execution result</span>
|
||||||
|
<span class="text-[10px] px-2 py-px rounded border font-mono {{ status_badge_style }} {{ status_text_color }}">{{ status_label }}</span>
|
||||||
|
</div>
|
||||||
|
<div class="text-[10px] text-forge-muted font-mono">trace {{ execution.trace_id }}</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="mt-4">
|
||||||
|
<div class="uppercase text-xs tracking-[1.5px] text-forge-muted mb-1">Decision path</div>
|
||||||
|
<div class="font-mono text-[10px] bg-black/60 border border-forge-iron rounded p-2.5 space-y-px">
|
||||||
|
{% for step in execution.decision_path %}
|
||||||
|
<div class="flex justify-between text-xs py-0.5 border-b border-forge-iron/60">
|
||||||
|
<span class="font-mono text-forge-text">{{ step.step }}</span>
|
||||||
|
<span class="text-forge-muted">
|
||||||
|
{%- if step.duration_ms %}{{ step.duration_ms }}ms{% endif -%}
|
||||||
|
{%- if step.detail.get("n_approved") is not none %} · {{ step.detail["n_approved"] }} approved{% endif -%}
|
||||||
|
{%- if step.detail.get("rejected") %} · rejected{% endif -%}
|
||||||
|
{%- if step.detail.get("hitl") %} · HITL{% endif -%}
|
||||||
|
{%- if step.detail.get("n_violations") %} · {{ step.detail["n_violations"] }} violation(s){% endif -%}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
{% else %}
|
||||||
|
<span class="text-forge-muted">—</span>
|
||||||
|
{% endfor %}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
{% if execution.violations %}
|
||||||
|
<div class="mt-4">
|
||||||
|
<div class="uppercase text-xs tracking-[1.5px] text-red-400 mb-1">Violations</div>
|
||||||
|
{% for v in execution.violations %}
|
||||||
|
<div class="text-xs {{ 'text-red-400' if v.severity == 'block' else 'text-amber-400' }}">
|
||||||
|
• {{ v.validator }}: {{ v.message }} ({{ v.severity }})
|
||||||
|
</div>
|
||||||
|
{% endfor %}
|
||||||
|
</div>
|
||||||
|
{% endif %}
|
||||||
|
|
||||||
|
{% if execution.proposed_actions %}
|
||||||
|
<div class="mt-4">
|
||||||
|
<div class="uppercase text-xs tracking-[1.5px] text-amber-400 mb-1">Proposed actions</div>
|
||||||
|
<div class="space-y-1">
|
||||||
|
{% for a in execution.proposed_actions %}
|
||||||
|
<div class="text-xs bg-black/50 border border-forge-iron rounded p-2">
|
||||||
|
• {{ a.action }} → <span class="font-mono">{{ a.target }}</span>
|
||||||
|
<span class="font-mono {{ 'text-emerald-400' if a.risk_score < 3 else ('text-amber-400' if a.risk_score < 5 else 'text-red-400') }}">(risk={{ a.risk_score }})</span>
|
||||||
|
</div>
|
||||||
|
{% endfor %}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
{% endif %}
|
||||||
|
|
||||||
|
{% if execution.needs_human_for %}
|
||||||
|
<div class="mt-3 p-3 border border-amber-900 bg-amber-950/40 rounded text-amber-300 text-sm">
|
||||||
|
{{ execution.needs_human_for | length }} action(s) require approval.
|
||||||
|
<a href="#approve" class="underline hover:text-amber-200">Go to stage 03 — Approve ↓</a>
|
||||||
|
</div>
|
||||||
|
{% endif %}
|
||||||
|
|
||||||
|
{% if execution.status == "completed" %}
|
||||||
|
<div class="mt-4 text-emerald-400 text-sm">
|
||||||
|
Run finished successfully. {{ execution.proposed_actions | length }} proposed action(s).
|
||||||
|
<a href="#audit" class="underline">See it in stage 06 — Audit ↓</a>
|
||||||
|
</div>
|
||||||
|
{% endif %}
|
||||||
|
</div>
|
||||||
@@ -0,0 +1,294 @@
|
|||||||
|
"""HTMX UI router — single-page lifecycle console mounted inside forja-core.
|
||||||
|
|
||||||
|
The UI is one page (``/``) organised as the six stages of the governed agent
|
||||||
|
lifecycle: Define → Run → Approve → Train → Promote → Audit. Each stage is a
|
||||||
|
fragment that re-fetches itself when any action button fires the global
|
||||||
|
``forja-refresh`` event, so the whole cycle can be driven with clicks only.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from collections.abc import Callable, Coroutine
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Form, Request
|
||||||
|
from fastapi.responses import HTMLResponse, RedirectResponse
|
||||||
|
from fastapi.templating import Jinja2Templates
|
||||||
|
|
||||||
|
from forja_core.api.deps import (
|
||||||
|
get_dataset_store,
|
||||||
|
get_model_store,
|
||||||
|
get_orchestrator,
|
||||||
|
get_policy_store,
|
||||||
|
get_registry,
|
||||||
|
get_settings,
|
||||||
|
)
|
||||||
|
from forja_core.api.evaluation_persistence import load_evaluation, load_evaluation_for_run
|
||||||
|
from forja_core.api.executions import _load_index, _record_execution
|
||||||
|
from forja_core.api.persistence import append_execution, append_violation, read_execution_summaries
|
||||||
|
from forja_core.api.promotion_persistence import list_promotion_requests
|
||||||
|
from forja_core.api.training_persistence import list_training_runs
|
||||||
|
from forja_core.domain.execution import AgentExecution
|
||||||
|
|
||||||
|
router = APIRouter(prefix="", tags=["ui"])
|
||||||
|
|
||||||
|
# Template path works in dev and inside the container.
|
||||||
|
TEMPLATES_DIR = Path(__file__).parent / "templates"
|
||||||
|
templates = Jinja2Templates(directory=str(TEMPLATES_DIR))
|
||||||
|
|
||||||
|
_STATUS_BADGES = {
|
||||||
|
"completed": ("text-emerald-400", "bg-emerald-950/60 border-emerald-900", "Completed"),
|
||||||
|
"awaiting_approval": ("text-amber-400", "bg-amber-950/60 border-amber-900", "Pending approval"),
|
||||||
|
"failed": ("text-red-400", "bg-red-950/60 border-red-900", "Failed"),
|
||||||
|
"blocked_by_guardrail": ("text-red-400", "bg-red-950/60 border-red-900", "Blocked by policy"),
|
||||||
|
"running": ("text-forge-steel", "bg-forge-surface2 border-forge-iron", "Running"),
|
||||||
|
}
|
||||||
|
|
||||||
|
# Demo input for the one-click walkthrough: "hss" maps to a risk-5 action in the
|
||||||
|
# mock provider, so the run pauses in HITL and exercises the approval stage.
|
||||||
|
DEMO_INPUT = (
|
||||||
|
"Replicated capacity alarm on the HSS active-active pair in Madrid; "
|
||||||
|
"subscriber database replication lag is growing."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# --- Context builders (shared by the index page and the fragments) ---------------------
|
||||||
|
|
||||||
|
|
||||||
|
def _define_ctx() -> dict[str, Any]:
|
||||||
|
"""Agents with all their versions/states, plus the guardrail policies."""
|
||||||
|
registry = get_registry()
|
||||||
|
cards = []
|
||||||
|
for agent in registry.list_agents():
|
||||||
|
versions = []
|
||||||
|
for meta in registry.list_versions(agent.name):
|
||||||
|
try:
|
||||||
|
version_def = registry.get_version(agent.name, meta.id)
|
||||||
|
except FileNotFoundError:
|
||||||
|
continue
|
||||||
|
versions.append({"id": meta.id, "state": version_def.state, "message": meta.message})
|
||||||
|
cards.append({"agent": agent, "versions": versions})
|
||||||
|
return {"agent_cards": cards, "policies": get_policy_store().list_policies()}
|
||||||
|
|
||||||
|
|
||||||
|
async def _approvals_ctx() -> dict[str, Any]:
|
||||||
|
return {"pending": await _pending_executions()}
|
||||||
|
|
||||||
|
|
||||||
|
def _training_ctx() -> dict[str, Any]:
|
||||||
|
settings = get_settings()
|
||||||
|
registry = get_registry()
|
||||||
|
pending_promo_runs = {
|
||||||
|
str(req.training_run_id)
|
||||||
|
for req in list_promotion_requests(settings.data_dir, status="pending_approval")
|
||||||
|
}
|
||||||
|
items = []
|
||||||
|
for run in list_training_runs(settings.data_dir):
|
||||||
|
candidate_state = None
|
||||||
|
if run.spec.agent_name and run.spec.agent_version:
|
||||||
|
try:
|
||||||
|
candidate_state = registry.get_version(
|
||||||
|
run.spec.agent_name, run.spec.agent_version
|
||||||
|
).state
|
||||||
|
except FileNotFoundError:
|
||||||
|
candidate_state = None
|
||||||
|
items.append(
|
||||||
|
{
|
||||||
|
"run": run,
|
||||||
|
"evaluation": load_evaluation_for_run(settings.data_dir, run.id),
|
||||||
|
"candidate_state": candidate_state,
|
||||||
|
"promotion_pending": str(run.id) in pending_promo_runs,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"items": items,
|
||||||
|
"datasets": get_dataset_store().list_datasets(),
|
||||||
|
"models": get_model_store().list_models(),
|
||||||
|
"backend": settings.training_backend,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _promotions_ctx() -> dict[str, Any]:
|
||||||
|
data_dir = get_settings().data_dir
|
||||||
|
pending_items = []
|
||||||
|
for req in list_promotion_requests(data_dir, status="pending_approval"):
|
||||||
|
pending_items.append(
|
||||||
|
{"request": req, "evaluation": load_evaluation(data_dir, req.evaluation_report_id)}
|
||||||
|
)
|
||||||
|
return {"pending": pending_items}
|
||||||
|
|
||||||
|
|
||||||
|
def _history_ctx() -> dict[str, Any]:
|
||||||
|
data_dir = get_settings().data_dir
|
||||||
|
all_execs = read_execution_summaries(data_dir)
|
||||||
|
completed = [e for e in all_execs if e.status == "completed"]
|
||||||
|
completed.sort(key=lambda e: e.finished_at or e.started_at, reverse=True)
|
||||||
|
return {"runs": completed[:10], "total_completed": len(completed)}
|
||||||
|
|
||||||
|
|
||||||
|
async def _pending_executions() -> list[AgentExecution]:
|
||||||
|
"""Executions paused in HITL. They only live in the checkpointer (not in the
|
||||||
|
append-only JSONL, which records terminal runs), so they are reconstructed
|
||||||
|
from the execution index + a checkpointer snapshot."""
|
||||||
|
registry = get_registry()
|
||||||
|
policy_store = get_policy_store()
|
||||||
|
orchestrator = get_orchestrator()
|
||||||
|
data_dir = get_settings().data_dir
|
||||||
|
pending: list[AgentExecution] = []
|
||||||
|
for trace_id, meta in _load_index(data_dir).items():
|
||||||
|
try:
|
||||||
|
agent_def = registry.get_version(meta["agent_name"], meta["version"])
|
||||||
|
policy = policy_store.get_policy(agent_def.guardrails[0])
|
||||||
|
execution = await orchestrator.snapshot(
|
||||||
|
agent_def=agent_def, policy=policy, trace_id=UUID(trace_id)
|
||||||
|
)
|
||||||
|
except (FileNotFoundError, IndexError, KeyError, ValueError):
|
||||||
|
continue
|
||||||
|
if execution is not None and execution.status == "awaiting_approval":
|
||||||
|
pending.append(execution)
|
||||||
|
pending.sort(key=lambda e: e.started_at, reverse=True)
|
||||||
|
return pending
|
||||||
|
|
||||||
|
|
||||||
|
# --- The single page --------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/", response_class=HTMLResponse)
|
||||||
|
async def index(request: Request) -> HTMLResponse:
|
||||||
|
registry = get_registry()
|
||||||
|
ctx: dict[str, Any] = {
|
||||||
|
"request": request,
|
||||||
|
"title": "Lifecycle",
|
||||||
|
"agents": registry.list_agents(),
|
||||||
|
"demo_input": DEMO_INPUT,
|
||||||
|
}
|
||||||
|
ctx.update(_define_ctx())
|
||||||
|
ctx.update(await _approvals_ctx())
|
||||||
|
ctx.update(_training_ctx())
|
||||||
|
ctx.update(_promotions_ctx())
|
||||||
|
ctx.update(_history_ctx())
|
||||||
|
return templates.TemplateResponse("index.html", ctx)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/run", response_class=HTMLResponse)
|
||||||
|
async def run_invoke(
|
||||||
|
request: Request,
|
||||||
|
agent_name: str = Form(...),
|
||||||
|
input: str = Form(...),
|
||||||
|
) -> HTMLResponse:
|
||||||
|
"""Execute the agent using the orchestrator directly (same process)."""
|
||||||
|
registry = get_registry()
|
||||||
|
policy_store = get_policy_store()
|
||||||
|
orchestrator = get_orchestrator()
|
||||||
|
settings = get_settings()
|
||||||
|
|
||||||
|
try:
|
||||||
|
agent_def = registry.get_agent(agent_name)
|
||||||
|
policy_name = agent_def.guardrails[0] if agent_def.guardrails else "default"
|
||||||
|
policy = policy_store.get_policy(policy_name)
|
||||||
|
|
||||||
|
execution: AgentExecution = await orchestrator.invoke(
|
||||||
|
agent_def=agent_def,
|
||||||
|
policy=policy,
|
||||||
|
user_input=input,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Persist like the API does: index always, JSONL only for terminal runs.
|
||||||
|
_record_execution(
|
||||||
|
settings.data_dir, str(execution.trace_id), agent_def.name, agent_def.version
|
||||||
|
)
|
||||||
|
if execution.status in {"completed", "failed", "blocked_by_guardrail"}:
|
||||||
|
append_execution(settings.data_dir, execution)
|
||||||
|
for v in execution.violations:
|
||||||
|
append_violation(settings.data_dir, v)
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
return HTMLResponse(
|
||||||
|
f'<div class="text-red-400 border border-red-900 bg-red-950/40 rounded p-3 text-sm">'
|
||||||
|
f"Execution error: {str(e)[:300]}</div>",
|
||||||
|
status_code=400,
|
||||||
|
)
|
||||||
|
|
||||||
|
text_color, badge_style, status_label = _STATUS_BADGES.get(
|
||||||
|
execution.status,
|
||||||
|
(
|
||||||
|
"text-forge-steel",
|
||||||
|
"bg-forge-surface2 border-forge-iron",
|
||||||
|
execution.status.replace("_", " ").title(),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
return templates.TemplateResponse(
|
||||||
|
"run_result.html",
|
||||||
|
{
|
||||||
|
"request": request,
|
||||||
|
"execution": execution,
|
||||||
|
"status_label": status_label,
|
||||||
|
"status_text_color": text_color,
|
||||||
|
"status_badge_style": badge_style,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# --- Fragments (re-fetched on the global forja-refresh event) ---------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/fragments/agents", response_class=HTMLResponse)
|
||||||
|
async def fragment_agents(request: Request) -> HTMLResponse:
|
||||||
|
return templates.TemplateResponse("_agents.html", {"request": request, **_define_ctx()})
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/fragments/approvals", response_class=HTMLResponse)
|
||||||
|
async def fragment_approvals(request: Request) -> HTMLResponse:
|
||||||
|
return templates.TemplateResponse(
|
||||||
|
"_approvals.html", {"request": request, **(await _approvals_ctx())}
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/fragments/training", response_class=HTMLResponse)
|
||||||
|
async def fragment_training(request: Request) -> HTMLResponse:
|
||||||
|
return templates.TemplateResponse("_training.html", {"request": request, **_training_ctx()})
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/fragments/promotions", response_class=HTMLResponse)
|
||||||
|
async def fragment_promotions(request: Request) -> HTMLResponse:
|
||||||
|
return templates.TemplateResponse("_promotions.html", {"request": request, **_promotions_ctx()})
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/fragments/history", response_class=HTMLResponse)
|
||||||
|
async def fragment_history(request: Request) -> HTMLResponse:
|
||||||
|
return templates.TemplateResponse("_history.html", {"request": request, **_history_ctx()})
|
||||||
|
|
||||||
|
|
||||||
|
# --- Legacy routes from the multi-page UI → anchors on the single page ------------------
|
||||||
|
|
||||||
|
_LEGACY_ROUTES = {
|
||||||
|
"/agents": "/#define",
|
||||||
|
"/policies": "/#define",
|
||||||
|
"/run": "/#run",
|
||||||
|
"/approvals": "/#approve",
|
||||||
|
"/training": "/#train",
|
||||||
|
"/promotions": "/#promote",
|
||||||
|
"/history": "/#audit",
|
||||||
|
"/armory": "/#audit",
|
||||||
|
"/tutorial": "/",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _redirect_endpoint(target: str) -> Callable[[], Coroutine[Any, Any, RedirectResponse]]:
|
||||||
|
async def _redirect() -> RedirectResponse:
|
||||||
|
return RedirectResponse(url=target, status_code=301)
|
||||||
|
|
||||||
|
return _redirect
|
||||||
|
|
||||||
|
|
||||||
|
for _path, _target in _LEGACY_ROUTES.items():
|
||||||
|
router.add_api_route(
|
||||||
|
_path, _redirect_endpoint(_target), methods=["GET"], include_in_schema=False
|
||||||
|
)
|
||||||
|
|
||||||
|
# Old per-agent detail pages → Define stage.
|
||||||
|
router.add_api_route(
|
||||||
|
"/agents/{name}", _redirect_endpoint("/#define"), methods=["GET"], include_in_schema=False
|
||||||
|
)
|
||||||
@@ -1,31 +0,0 @@
|
|||||||
# syntax=docker/dockerfile:1.7
|
|
||||||
|
|
||||||
FROM python:3.11-slim
|
|
||||||
|
|
||||||
ENV PYTHONUNBUFFERED=1 \
|
|
||||||
PYTHONDONTWRITEBYTECODE=1 \
|
|
||||||
PIP_NO_CACHE_DIR=1
|
|
||||||
|
|
||||||
WORKDIR /app
|
|
||||||
|
|
||||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
|
||||||
curl \
|
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
|
||||||
|
|
||||||
COPY dashboard/requirements.txt /app/requirements.txt
|
|
||||||
RUN pip install -r /app/requirements.txt
|
|
||||||
|
|
||||||
COPY dashboard/src /app/src
|
|
||||||
ENV PYTHONPATH=/app/src
|
|
||||||
|
|
||||||
RUN useradd --create-home --shell /bin/bash dash && chown -R dash:dash /app
|
|
||||||
USER dash
|
|
||||||
|
|
||||||
EXPOSE 8501
|
|
||||||
|
|
||||||
HEALTHCHECK --interval=10s --timeout=3s --start-period=10s --retries=3 \
|
|
||||||
CMD curl -fsS http://localhost:8501/_stcore/health || exit 1
|
|
||||||
|
|
||||||
CMD ["streamlit", "run", "/app/src/agentforge_dashboard/app.py", \
|
|
||||||
"--server.address=0.0.0.0", "--server.port=8501", \
|
|
||||||
"--server.headless=true", "--browser.gatherUsageStats=false"]
|
|
||||||
@@ -1,4 +0,0 @@
|
|||||||
streamlit>=1.38,<2.0
|
|
||||||
httpx>=0.27,<0.28
|
|
||||||
pydantic>=2.7,<3.0
|
|
||||||
PyYAML>=6.0,<7.0
|
|
||||||
@@ -1,47 +0,0 @@
|
|||||||
"""Entry point del dashboard Streamlit con sidebar de branding y health."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import streamlit as st
|
|
||||||
|
|
||||||
from agentforge_dashboard.client import CoreClient
|
|
||||||
|
|
||||||
|
|
||||||
@st.cache_resource
|
|
||||||
def get_client() -> CoreClient:
|
|
||||||
return CoreClient()
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> None:
|
|
||||||
st.set_page_config(
|
|
||||||
page_title="AgentForge",
|
|
||||||
page_icon="🛡️",
|
|
||||||
layout="wide",
|
|
||||||
initial_sidebar_state="expanded",
|
|
||||||
)
|
|
||||||
st.sidebar.markdown("## 🛡️ AgentForge")
|
|
||||||
st.sidebar.caption("Plataforma de gobernanza de agentes IA")
|
|
||||||
client = get_client()
|
|
||||||
try:
|
|
||||||
client.health()
|
|
||||||
st.sidebar.success("Core API: OK")
|
|
||||||
except Exception as exc:
|
|
||||||
st.sidebar.error(f"Core API unreachable: {exc}")
|
|
||||||
|
|
||||||
st.title("🛡️ AgentForge")
|
|
||||||
st.markdown(
|
|
||||||
"""
|
|
||||||
Bienvenido al panel de gobernanza de agentes IA.
|
|
||||||
|
|
||||||
Usa la barra lateral para navegar:
|
|
||||||
- **Registro**: catálogo de agentes y versiones.
|
|
||||||
- **Ejecutar**: lanzar un agente con guardrails completos.
|
|
||||||
- **Aprobaciones**: ejecuciones pausadas a la espera de revisión humana.
|
|
||||||
- **Historial**: trazas, violaciones y resultados.
|
|
||||||
- **Politicas**: políticas de guardrails y diff entre versiones.
|
|
||||||
"""
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
@@ -1,71 +0,0 @@
|
|||||||
"""Cliente HTTP tipado al core. Encapsula httpx y mapea respuestas a dicts."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import os
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import httpx
|
|
||||||
|
|
||||||
|
|
||||||
class CoreClient:
|
|
||||||
"""Cliente sincrono (Streamlit es sync). 2 retries con httpx Transport."""
|
|
||||||
|
|
||||||
def __init__(self, base_url: str | None = None, timeout: float = 30.0) -> None:
|
|
||||||
self.base_url = base_url or os.getenv("AGENTFORGE_CORE_URL", "http://localhost:8000")
|
|
||||||
transport = httpx.HTTPTransport(retries=2)
|
|
||||||
self._client = httpx.Client(base_url=self.base_url, timeout=timeout, transport=transport)
|
|
||||||
|
|
||||||
def health(self) -> dict:
|
|
||||||
return self._get("/health")
|
|
||||||
|
|
||||||
def list_agents(self) -> list[dict]:
|
|
||||||
return self._get("/agents")
|
|
||||||
|
|
||||||
def get_agent(self, name: str) -> dict:
|
|
||||||
return self._get(f"/agents/{name}")
|
|
||||||
|
|
||||||
def list_versions(self, name: str) -> list[dict]:
|
|
||||||
return self._get(f"/agents/{name}/versions")
|
|
||||||
|
|
||||||
def diff_versions(self, name: str, v_from: str, v_to: str) -> dict:
|
|
||||||
return self._get(f"/agents/{name}/versions/{v_from}/diff/{v_to}")
|
|
||||||
|
|
||||||
def invoke_agent(self, name: str, body: dict) -> dict:
|
|
||||||
return self._post(f"/agents/{name}/invoke", body)
|
|
||||||
|
|
||||||
def get_execution(self, trace_id: str) -> dict:
|
|
||||||
return self._get(f"/executions/{trace_id}")
|
|
||||||
|
|
||||||
def list_executions(self) -> list[dict]:
|
|
||||||
return self._get("/executions")
|
|
||||||
|
|
||||||
def approve(self, trace_id: str, body: dict) -> dict:
|
|
||||||
return self._post(f"/executions/{trace_id}/approve", body)
|
|
||||||
|
|
||||||
def reject(self, trace_id: str, body: dict) -> dict:
|
|
||||||
return self._post(f"/executions/{trace_id}/reject", body)
|
|
||||||
|
|
||||||
def list_violations(self, **filters: Any) -> list[dict]:
|
|
||||||
params = {k: v for k, v in filters.items() if v is not None}
|
|
||||||
return self._get("/violations", params=params)
|
|
||||||
|
|
||||||
def list_policies(self) -> list[dict]:
|
|
||||||
return self._get("/policies")
|
|
||||||
|
|
||||||
def list_policy_versions(self, name: str) -> list[dict]:
|
|
||||||
return self._get(f"/policies/{name}/versions")
|
|
||||||
|
|
||||||
# ----- helpers privados -----
|
|
||||||
|
|
||||||
def _get(self, path: str, params: dict | None = None) -> Any:
|
|
||||||
r = self._client.get(path, params=params)
|
|
||||||
r.raise_for_status()
|
|
||||||
return r.json()
|
|
||||||
|
|
||||||
def _post(self, path: str, body: dict) -> Any:
|
|
||||||
r = self._client.post(path, json=body)
|
|
||||||
if r.status_code in {404, 409, 422}:
|
|
||||||
return {"error": r.json()}
|
|
||||||
r.raise_for_status()
|
|
||||||
return r.json()
|
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user