agents/content-ingestion-agent
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

decomposer: generate deliverable files for Define the normalized ingestion contract and pluggable source-connector interface for SCM, CMS, and other content sources.; Implement a GitHub SCM connector that conforms to the normalized ingestion contract and supports scoped full and incremental ingestion of Markdown, plain-text, and source files with webhook and revision metadata.; Implement a SharePoint CMS connector that conforms to the normalized ingestion contract and supports scoped full and incremental ingestion of PDF, DOCX, and HTML files with webhook and revision metadata.; Implement ingestion orchestration and triggers; Normalize, govern, and publish ingested content to the shared knowledge store.; Add deployment/configuration, tests, and documentation for the ingestion agent.; Validate end-to-end ingestion and downstream content availability.
Some checks failed
ci / test (push) Failing after 7s
Details

Browse Source
This commit is contained in:
demo-bot
2026-09-01 14:10:18 +00:00
parent 78d65944c6
commit 2627747e20
22 changed files with 266 additions and 235 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
docs/contract.md Normal file
View File

@@ -0,0 +1,16 @@
# KAB ingestion contract (v1)
## Connector interface
`full(config) -> (Document[], SyncCursor)`, `incremental(config, cursor, changes?) -> (Document[], SyncCursor)`, and `webhook(config, payload, headers) -> Change[]`. Implementations must be deterministic for the same source revision, scope, and credentials. Config has `connector_id`, `tenant_id`, secret reference, source scope, and non-secret options.
## Normalized document
Required fields: stable `id`, `tenant_id`, title, UTF-8 `content` (or extractor output), MIME type, SHA-256 `content_hash`, modified timestamp, `provenance`, `acl`, metadata, and deletion marker. Binary extraction may be delegated to a managed extractor; the connector still supplies original MIME and hash.
## Provenance and ACL
Provenance includes source type/id, canonical URL, revision/eTag, retrieval time, webhook event, and connector version. ACL includes tenant, principals, groups, and visibility. Tenant equality is mandatory; downstream search must apply ACL predicates.
## Cursor/triggers
Cursors contain connector id, mode, source revision or delta token, and update time. Full runs establish a baseline. Incremental runs advance the cursor only after publication succeeds. Triggers are `on_demand`, cron `scheduled`, or signed `webhook`; each carries `full|incremental`, idempotency key, retry policy, and dead-letter destination.
## Governance
Upserts are idempotent by document id/hash. Deletes and retention events are auditable, tenant-scoped, and must use managed secret references. No token or document content is logged.