agents/content-ingestion-agent
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
22 Commits 1 Branch 0 Tags
main
Go to file
Clone
Open with VS Code Open with VSCodium Open with Intellij IDEA
Download ZIP Download TAR.GZ Download BUNDLE
Jonathan Boniface d8a4b6b3c1
Some checks failed
quality-gates / verify (push) Failing after 7s
Details
fix: files manually
2026-09-01 17:21:58 +01:00
.agents
fix: files manually
2026-09-01 17:21:58 +01:00
.gitea/workflows
decomposer: fix validation failure for Migrate the kab_ingestion domain models, ports, GitHub SCM connector, and SharePoint CMS connector into the repository's app/ package.; Wire the migrated ingestion components into the agent application entry point, card, settings, triggers, governance, publication, and validation layers.; Populate every remaining placeholder file with working application, package, test, dependency, and Kubernetes content.; Verify the migrated agent and completed repository end to end.
2026-09-01 14:37:28 +00:00
.github
decomposer: generate deliverable files for Define the normalized ingestion contract and pluggable source-connector interface for SCM, CMS, and other content sources.; Implement a GitHub SCM connector that conforms to the normalized ingestion contract and supports scoped full and incremental ingestion of Markdown, plain-text, and source files with webhook and revision metadata.; Implement a SharePoint CMS connector that conforms to the normalized ingestion contract and supports scoped full and incremental ingestion of PDF, DOCX, and HTML files with webhook and revision metadata.; Implement ingestion orchestration and triggers; Normalize, govern, and publish ingested content to the shared knowledge store.; Add deployment/configuration, tests, and documentation for the ingestion agent.; Validate end-to-end ingestion and downstream content availability.
2026-09-01 14:10:18 +00:00
app
fix: files manually
2026-09-01 17:21:58 +01:00
docs
fix: files manually
2026-09-01 17:21:58 +01:00
examples
decomposer: generate deliverable files for Define the normalized ingestion contract and pluggable source-connector interface for SCM, CMS, and other content sources.; Implement a GitHub SCM connector that conforms to the normalized ingestion contract and supports scoped full and incremental ingestion of Markdown, plain-text, and source files with webhook and revision metadata.; Implement a SharePoint CMS connector that conforms to the normalized ingestion contract and supports scoped full and incremental ingestion of PDF, DOCX, and HTML files with webhook and revision metadata.; Implement ingestion orchestration and triggers; Normalize, govern, and publish ingested content to the shared knowledge store.; Add deployment/configuration, tests, and documentation for the ingestion agent.; Validate end-to-end ingestion and downstream content availability.
2026-09-01 14:10:18 +00:00
generation_results
decomposer: fix validation failure for Define the normalized ingestion contract and pluggable source-connector interface for SCM, CMS, and other content sources.; Implement a GitHub SCM connector that conforms to the normalized ingestion contract and supports scoped full and incremental ingestion of Markdown, plain-text, and source files with webhook and revision metadata.; Implement a SharePoint CMS connector that conforms to the normalized ingestion contract and supports scoped full and incremental ingestion of PDF, DOCX, and HTML files with webhook and revision metadata.; Implement ingestion orchestration and triggers; Normalize, govern, and publish ingested content to the shared knowledge store.; Add deployment/configuration, tests, and documentation for the ingestion agent.; Validate end-to-end ingestion and downstream content availability.
2026-09-01 14:06:05 +00:00
k8s
decomposer: fix validation failure for Migrate the kab_ingestion domain models, ports, GitHub SCM connector, and SharePoint CMS connector into the repository's app/ package.; Wire the migrated ingestion components into the agent application entry point, card, settings, triggers, governance, publication, and validation layers.; Populate every remaining placeholder file with working application, package, test, dependency, and Kubernetes content.; Verify the migrated agent and completed repository end to end.
2026-09-01 14:40:29 +00:00
kab_ingestion
fix: files manually
2026-09-01 17:21:58 +01:00
scripts
fix: files manually
2026-09-01 17:21:58 +01:00
spec
decomposer: generate deliverable files for Define the normalized ingestion contract and pluggable source-connector interface for SCM, CMS, and other content sources.; Implement a GitHub SCM connector that conforms to the normalized ingestion contract and supports scoped full and incremental ingestion of Markdown, plain-text, and source files with webhook and revision metadata.; Implement a SharePoint CMS connector that conforms to the normalized ingestion contract and supports scoped full and incremental ingestion of PDF, DOCX, and HTML files with webhook and revision metadata.; Implement ingestion orchestration and triggers; Normalize, govern, and publish ingested content to the shared knowledge store.; Add deployment/configuration, tests, and documentation for the ingestion agent.; Validate end-to-end ingestion and downstream content availability.
2026-09-01 14:03:17 +00:00
tests
fix: files manually
2026-09-01 17:21:58 +01:00
.image-version
decomposer: scaffold content-ingestion-agent
2026-09-01 14:03:16 +00:00
Dockerfile
fix: files manually
2026-09-01 17:21:58 +01:00
generation_results.json
decomposer: fix validation failure for Define the normalized ingestion contract and pluggable source-connector interface for SCM, CMS, and other content sources.; Implement a GitHub SCM connector that conforms to the normalized ingestion contract and supports scoped full and incremental ingestion of Markdown, plain-text, and source files with webhook and revision metadata.; Implement a SharePoint CMS connector that conforms to the normalized ingestion contract and supports scoped full and incremental ingestion of PDF, DOCX, and HTML files with webhook and revision metadata.; Implement ingestion orchestration and triggers; Normalize, govern, and publish ingested content to the shared knowledge store.; Add deployment/configuration, tests, and documentation for the ingestion agent.; Validate end-to-end ingestion and downstream content availability.
2026-09-01 14:05:07 +00:00
LICENSE
decomposer: generate deliverable files for Define the normalized ingestion contract and pluggable source-connector interface for SCM, CMS, and other content sources.; Implement a GitHub SCM connector that conforms to the normalized ingestion contract and supports scoped full and incremental ingestion of Markdown, plain-text, and source files with webhook and revision metadata.; Implement a SharePoint CMS connector that conforms to the normalized ingestion contract and supports scoped full and incremental ingestion of PDF, DOCX, and HTML files with webhook and revision metadata.; Implement ingestion orchestration and triggers; Normalize, govern, and publish ingested content to the shared knowledge store.; Add deployment/configuration, tests, and documentation for the ingestion agent.; Validate end-to-end ingestion and downstream content availability.
2026-09-01 14:10:18 +00:00
pyproject.toml
fix: files manually
2026-09-01 17:21:58 +01:00
pytest.ini
decomposer: scaffold content-ingestion-agent
2026-09-01 14:03:16 +00:00
README.md
fix: files manually
2026-09-01 17:21:58 +01:00
requirements-dev.txt
decomposer: fix validation failure for Migrate the kab_ingestion domain models, ports, GitHub SCM connector, and SharePoint CMS connector into the repository's app/ package.; Wire the migrated ingestion components into the agent application entry point, card, settings, triggers, governance, publication, and validation layers.; Populate every remaining placeholder file with working application, package, test, dependency, and Kubernetes content.; Verify the migrated agent and completed repository end to end.
2026-09-01 14:40:29 +00:00
requirements.txt
decomposer: fix validation failure for Migrate the kab_ingestion domain models, ports, GitHub SCM connector, and SharePoint CMS connector into the repository's app/ package.; Wire the migrated ingestion components into the agent application entry point, card, settings, triggers, governance, publication, and validation layers.; Populate every remaining placeholder file with working application, package, test, dependency, and Kubernetes content.; Verify the migrated agent and completed repository end to end.
2026-09-01 14:40:29 +00:00

README.md

KAB Content Ingestion Agent

Contract-first, multi-tenant ingestion for GitHub REST and Microsoft Graph SharePoint into a governed KAB-compatible knowledge store.

The package is dependency-light, utilizing hexagonal architecture ports (HTTP, SecretStore, Publisher) for deterministic testing and zero-token leakage.

Architecture & Codebase Alignment

The codebase is organized into two primary layers with a unified domain alignment:

                            +-----------------------------------+
                            |              app/                 |
                            |  (Starlette/FastAPI Web Layer &   |
                            |  LangGraph Execution Workflows)   |
                            +-----------------+-----------------+
                                              |
                                    Imports & Delegates
                                              v
                            +-----------------------------------+
                            |          kab_ingestion/           |
                            |  (Core Domain Models, Connectors, |
                            |    Orchestrator & Governance)     |
                            +-----------------------------------+

1. Core Ingestion Domain Layer (kab_ingestion/)

2. Application & Workflow Layer (app/)

3. Package Re-export Alignment (app/kab_ingestion)

  • app/kab_ingestion: Operates as a lightweight re-export alignment layer. All domain models, connectors, and ports re-export directly from the canonical kab_ingestion root package, guaranteeing full backward compatibility and preventing duplicate definitions.

LangGraph 4-Phase Execution Contract

Every ingestion workflow implements the platform's 4-phase execution contract:

  1. Discovery (app/nodes/discovery_node.py): Non-blocking check for existing resources and connector availability.
  2. Validation (app/nodes/validation_node.py): Validates IngestionRequest format and allowed sources; fails fast on error (has_errors: True → END).
  3. Generation (app/nodes/generation_node.py): Connects to GitHub/SharePoint source connectors and fetches normalized documents into fetched_documents.
  4. Deployment (app/nodes/deployment_node.py): Enforces governance approval and tenant ACL isolation, publishing approved documents to target storage.

Security & Tenant Isolation

  • Secret References (secret_ref): Tokens are referenced by secret name (e.g. secrets/github-read), never stored in configuration files or logs.
  • Tenant Isolation & ACLs: Every document carries tenant_id and ACL {principals, groups, visibility}. Tenant mismatch results in automatic publish rejection.

Quick Start & Testing

Running Tests

python -m pytest -q

Compiling Packages

python -m compileall kab_ingestion app

Documentation References

Reference in New Issue View Git Blame Copy Permalink
Description
Generated deliverable: content-ingestion-agent
Readme 127 KiB
Languages
Python 98%
Dockerfile 2%