2.1 KiB
KAB ingestion contract (v1)
Connector interface
A connector is configured with tenant_id, a source-specific scope, credential_ref, accepted MIME types, and trigger policy. It exposes full(tenant_id), incremental(tenant_id, cursor), and changes_from_webhook(payload). Both sync methods return an IngestionBatch; webhook parsing returns source changes and never retrieves content. Connectors must be deterministic for the same source revision, idempotent by document_id + content_hash, and must never log credentials or content.
Normalized document
Required fields: document_id, tenant_id, title, body, mime_type, source_path, content_hash, updated_at, acl, and provenance. Optional language, metadata, and deleted are supported. document_id is a stable namespaced source ID. Body is UTF-8 extracted text; unsupported or extraction-failed content is rejected to the batch error list.
Provenance
connector, immutable source_uri and source_id, optional revision_id, optional revision_time, retrieved_at, and optional webhook_event_id provide traceability. Source revision metadata must be retained with publication.
ACL and isolation
Every document carries tenant_id and ACL {principals, groups, public}. Publication must enforce tenant equality and downstream retrieval must filter by tenant plus principal/group intersection; public never crosses tenants. Missing ACL means deny all.
Cursors and triggers
A cursor contains connector, scope fingerprint, opaque source value, monotonic version, and update time. Full sync starts a new cursor; incremental uses the stored cursor and atomically saves the returned cursor after publication. Trigger modes are on_demand, schedule (cron/interval), and webhook; each chooses full or incremental, has retry/backoff and an idempotency key. Webhooks are authenticated, bounded, and enqueue work rather than doing network retrieval inline.
Lifecycle
Deletes are tombstone documents first, then physically deleted after retention policy. Audit events include tenant, run, source, action, result, and timestamp but no secret or body.