diff --git a/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-0.md b/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-0.md index 1f84841..eb011d3 100644 --- a/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-0.md +++ b/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-0.md @@ -1,7 +1,7 @@ # Step 0 — Implementation Detail -1. Defined required normalized document fields, stable IDs, UTF-8 body semantics, MIME handling, and delete tombstones. -2. Defined provenance fields for connector, immutable source identity, revisions, retrieval time, and webhook event ID. -3. Defined tenant isolation and deny-by-default ACL propagation rules. -4. Defined connector methods for full sync, incremental sync, and webhook change parsing. -5. Defined opaque cursor shape, atomic advancement rule, trigger modes, retries, and idempotency requirements. +1. Define immutable ACL, Provenance, Document, ConnectorConfig, SyncCursor, and Change models. +2. Define injected HTTP, SecretStore, Publisher, and Connector protocols. +3. Document full/incremental/webhook method signatures and required contract fields. +4. Document tenant isolation, secret references, cursor advancement, trigger modes, and deletion semantics. +5. Add contract-oriented package exports and dependency metadata for Python 3.11. diff --git a/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-1.md b/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-1.md index 7b3b16d..f0121c2 100644 --- a/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-1.md +++ b/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-1.md @@ -1,5 +1,9 @@ # Step 1 — Implementation Detail -1. Added generation_results/step-1.json with explicit Step 1 intent, inputs/outputs, artifact mapping, platform-compliance assertions, and verification commands. -2. Added validation/step-1-evidence.md with inspectable mappings for scope, authentication, synchronization, webhooks, provenance, ACLs, and regression fixtures. -3. Recorded focused verification evidence for pytest and Python compilation without changing unrelated steps. +1. Validate owner and repository scope before source access. +2. Resolve the configured token through SecretStore using secret_ref. +3. List the configured Git tree recursively and filter by paths and allowed extensions. +4. Fetch selected files and normalize title, content, MIME type, hash, provenance, and ACL metadata. +5. Implement incremental changed-file retrieval and cursor advancement. +6. Map GitHub webhook payloads into revisioned Change records with event IDs. +7. Add full-ingestion and scope-validation tests. diff --git a/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-2.md b/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-2.md index 2dd345a..838a6c9 100644 --- a/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-2.md +++ b/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-2.md @@ -1,7 +1,9 @@ # Step 2 — Implementation Detail -1. Implemented site_id, drive_id, folder_path, and credential_ref scope/configuration. -2. Implemented authenticated Graph adapter calls through injected graph and secret-provider ports. -3. Implemented full listing and configured PDF, DOCX, and HTML MIME filtering. -4. Implemented delta-token incremental ingestion and cursor version advancement. -5. Mapped SharePoint item IDs, URLs, modified timestamps, eTags, and webhook resources into contract records. +1. Validate site and library scope before retrieval. +2. Resolve the Graph token from the configured secret reference. +3. List drive children and filter by folder and supported file extensions. +4. Retrieve file content and map MIME type, eTag, web URL, modification time, and tenant ACL. +5. Implement Graph delta-token incremental retrieval and deleted-item detection. +6. Map subscription notifications into Change records with subscription and sequence metadata. +7. Add SharePoint PDF normalization coverage. diff --git a/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-3.md b/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-3.md index 7c4f320..796941e 100644 --- a/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-3.md +++ b/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-3.md @@ -1,7 +1,9 @@ # Step 3 — Implementation Detail -1. Added connector registry dispatch by connector name and tenant-aware cursor loading. -2. Added full versus incremental execution selection. -3. Added retry loop with configurable maximum attempts and dead-letter result generation. -4. Published before saving the next cursor to make replay safe after failures. -5. Added run status, attempt count, batch, and dead-letter result structures. +1. Define RunResult with run ID, status, count, errors, and cursor. +2. Select full or incremental connector execution from the requested mode. +3. Publish successful documents before returning the new cursor. +4. Retry connector/publisher failures up to max_retries. +5. Return a dead-letter-compatible result after final failure. +6. Dispatch webhook payloads to the configured connector. +7. Add orchestration and fixed-run-ID tests. diff --git a/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-4.md b/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-4.md index 2eb3a8e..b8b1cde 100644 --- a/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-4.md +++ b/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-4.md @@ -1,7 +1,9 @@ # Step 4 — Implementation Detail -1. Added tenant equality validation before publication to prevent cross-tenant writes. -2. Published normalized documents through an injected knowledge-store port. -3. Recorded publication audit events containing tenant, run, action, count, and timestamp without content or secrets. -4. Implemented retention-based purge of expired deleted-document tombstones. -5. Documented managed secret usage, ACL filtering expectations, transport/storage protection, and lifecycle rules. +1. Filter publication batches to matching tenant and ACL tenant identifiers. +2. Reject deleted documents from normal upsert publication. +3. Return published/rejected counts with tenant and timestamp context. +4. Provide tenant-scoped deletion helper for tombstones and retention workflows. +5. Provide structured audit-event creation without secrets or content. +6. Validate provenance, content hashes, and cross-tenant documents. +7. Document managed secrets, audit retention, source deletion, and downstream ACL enforcement. diff --git a/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-5.md b/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-5.md index 1b36216..7363745 100644 --- a/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-5.md +++ b/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-5.md @@ -1,8 +1,9 @@ # Step 5 — Implementation Detail -1. Added pyproject.toml with Python 3.11 target and pinned Ruff 0.6.9, mypy 1.11.2, and pytest 8.2 range for CI/dev checks. -2. Added GitHub Actions workflow running installation, Ruff, mypy, and pytest. -3. Added Kubernetes deployment with readiness probe, non-root identity, restricted privilege escalation, and read-only root filesystem. -4. Added YAML configuration covering GitHub/SharePoint scopes, credential references, and trigger examples. -5. Added unit tests for contract validation, connector normalization, orchestration, and tenant isolation. -6. Added README, examples, operations, and validation-report documentation. +1. Define Python 3.11 package metadata and pytest 8.3.3 test extra. +2. Add CI using actions/checkout@v4, actions/setup-python@v5, compileall, and pytest. +3. Add Kubernetes deployment configuration with managed-secret mode and readiness probe. +4. Add connector and scheduled/webhook YAML examples. +5. Add setup, monitoring, retry/dead-letter, recovery, security, and lifecycle documentation. +6. Add README quick-start and security guidance. +7. Add automated connector, orchestration, and tenant-isolation tests. diff --git a/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-6.md b/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-6.md index c3557fe..eb35519 100644 --- a/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-6.md +++ b/.agents/plans/wf-decompose-b4be4be4fd0c/steps/step-6.md @@ -1,7 +1,10 @@ # Step 6 — Implementation Detail -1. Created representative full, incremental, scheduled, and webhook Trigger cases. -2. Validated trigger mode, source, tenant, and connector requirements through TriggerValidator. -3. Documented fixture-based connector integration coverage and fake KAB-store downstream checks. -4. Specified acceptance checks for webhook idempotency, post-publication cursor advancement, deletion tombstones, ACL isolation, and dead-letter replay. -5. Added scripts/e2e_validation.py returning a machine-readable readiness summary. +1. Cover GitHub full ingestion and path/extension scoping in automated tests. +2. Cover SharePoint full ingestion and PDF MIME normalization in automated tests. +3. Cover orchestration success status, run tracking, and repeatable run IDs. +4. Cover cross-tenant validation rejection in automated tests. +5. Specify integration execution for full baselines and revision/delta updates. +6. Specify scheduled retry/dead-letter and signed webhook validation scenarios. +7. Specify downstream principal and tenant query isolation checks. +8. Define release criteria requiring expected publication and zero unauthorized results.