# KAB Content Ingestion Agent

Contract-first, multi-tenant ingestion for GitHub REST and Microsoft Graph SharePoint into a governed KAB-compatible knowledge store.

The package is dependency-light, utilizing hexagonal architecture ports (HTTP, SecretStore, Publisher) for deterministic testing and zero-token leakage.

---

## Architecture & Codebase Alignment

The codebase is organized into two primary layers with a unified domain alignment:

```
                            +-----------------------------------+
                            |              app/                 |
                            |  (Starlette/FastAPI Web Layer &   |
                            |  LangGraph Execution Workflows)   |
                            +-----------------+-----------------+
                                              |
                                    Imports & Delegates
                                              v
                            +-----------------------------------+
                            |          kab_ingestion/           |
                            |  (Core Domain Models, Connectors, |
                            |    Orchestrator & Governance)     |
                            +-----------------------------------+
```

### 1. Core Ingestion Domain Layer (`kab_ingestion/`)
- **[kab_ingestion/models.py](file:///home/jonathanboniface/platform-engineering/kyndryl-agent-builder/content-ingestion-agent/kab_ingestion/models.py)**: Canonical domain models (`Document`, `ACL`, `Provenance`, `SyncCursor`, `IngestionRequest`, `IngestionResult`).
- **[kab_ingestion/github.py](file:///home/jonathanboniface/platform-engineering/kyndryl-agent-builder/content-ingestion-agent/kab_ingestion/github.py)** & **[kab_ingestion/sharepoint.py](file:///home/jonathanboniface/platform-engineering/kyndryl-agent-builder/content-ingestion-agent/kab_ingestion/sharepoint.py)**: Source connectors supporting full tree syncs, incremental delta syncs, and webhook event parsing.
- **[kab_ingestion/orchestrator.py](file:///home/jonathanboniface/platform-engineering/kyndryl-agent-builder/content-ingestion-agent/kab_ingestion/orchestrator.py)**: Fault-tolerant batch sync engine with bounded retries (`max_retries=3`) and dead-lettering.
- **[kab_ingestion/governance.py](file:///home/jonathanboniface/platform-engineering/kyndryl-agent-builder/content-ingestion-agent/kab_ingestion/governance.py)**: Strict multi-tenant isolation and ACL validation.

### 2. Application & Workflow Layer (`app/`)
- **[app/agent.py](file:///home/jonathanboniface/platform-engineering/kyndryl-agent-builder/content-ingestion-agent/app/agent.py)**: Service factory `create_agent_app()` and `IngestionService`.
- **[app/card.py](file:///home/jonathanboniface/platform-engineering/kyndryl-agent-builder/content-ingestion-agent/app/card.py)**: Agent discovery card (`AGENT_CARD`).
- **[app/workflows/ingestion_workflow.py](file:///home/jonathanboniface/platform-engineering/kyndryl-agent-builder/content-ingestion-agent/app/workflows/ingestion_workflow.py)**: 4-Phase LangGraph workflow runner.
- **[app/routes.py](file:///home/jonathanboniface/platform-engineering/kyndryl-agent-builder/content-ingestion-agent/app/routes.py)**: HTTP REST routes (`/health`, `/card`, `/ingest`).

### 3. Package Re-export Alignment (`app/kab_ingestion`)
- **`app/kab_ingestion`**: Operates as a lightweight re-export alignment layer. All domain models, connectors, and ports re-export directly from the canonical `kab_ingestion` root package, guaranteeing full backward compatibility and preventing duplicate definitions.

---

## LangGraph 4-Phase Execution Contract

Every ingestion workflow implements the platform's 4-phase execution contract:

1. **Discovery** (`app/nodes/discovery_node.py`): Non-blocking check for existing resources and connector availability.
2. **Validation** (`app/nodes/validation_node.py`): Validates `IngestionRequest` format and allowed sources; fails fast on error (`has_errors: True` → `END`).
3. **Generation** (`app/nodes/generation_node.py`): Connects to GitHub/SharePoint source connectors and fetches normalized documents into `fetched_documents`.
4. **Deployment** (`app/nodes/deployment_node.py`): Enforces governance approval and tenant ACL isolation, publishing approved documents to target storage.

---

## Security & Tenant Isolation

- **Secret References (`secret_ref`)**: Tokens are referenced by secret name (e.g. `secrets/github-read`), never stored in configuration files or logs.
- **Tenant Isolation & ACLs**: Every document carries `tenant_id` and `ACL {principals, groups, visibility}`. Tenant mismatch results in automatic publish rejection.

---

## Quick Start & Testing

### Running Tests
```bash
python -m pytest -q
```

### Compiling Packages
```bash
python -m compileall kab_ingestion app
```

---

## Documentation References

- **Contract Specification**: [spec/ingestion-contract.md](file:///home/jonathanboniface/platform-engineering/kyndryl-agent-builder/content-ingestion-agent/spec/ingestion-contract.md)
- **Configuration Example**: [examples/config.yaml](file:///home/jonathanboniface/platform-engineering/kyndryl-agent-builder/content-ingestion-agent/examples/config.yaml)
- **Kubernetes Manifests**: [k8s/](file:///home/jonathanboniface/platform-engineering/kyndryl-agent-builder/content-ingestion-agent/k8s)