decomposer: generate deliverable files for Define the service contract and project architecture for the FastAPI endpoint monitoring service.; Implement the typed monitor CRUD API and concurrency-safe in-memory state according to the service design.; Implement secure on-demand endpoint checks with status updates, latency measurement, robust error handling, and redacted structured logs.; Add operational API endpoints and environment-driven runtime configuration to the monitoring service.; Create automated tests for the monitoring service.; Package the service with Docker and developer documentation.; Validate the complete project.
Some checks failed
ci / validate (push) Has been cancelled
Some checks failed
ci / validate (push) Has been cancelled
This commit is contained in:
@@ -1,43 +1,46 @@
|
||||
# Endpoint Monitor service design
|
||||
# Service design and contract
|
||||
|
||||
## Scope and lifecycle
|
||||
## Architecture
|
||||
|
||||
This repository contains one unauthenticated FastAPI service for defining endpoint monitors and running checks on demand. State is held in a lock-protected, process-local `MonitorStore`; it starts empty, is lost on restart, and is not shared between workers. Production deployments of this implementation must therefore use exactly one worker. Persistence and authentication are intentionally out of scope.
|
||||
A single FastAPI process owns a `MonitorStore` guarded by an `asyncio.Lock`. Routes call the store and an `EndpointChecker`; the checker validates every initial/redirect URL, resolves every hostname, applies a total bounded timeout, and returns a typed result. No authentication is provided. State is intentionally neither durable nor shared.
|
||||
|
||||
## Resource and routes
|
||||
## Resource
|
||||
|
||||
A monitor has a server-generated UUID, name, HTTP(S) URL, creation/update timestamps, and a `current_status`. Status is one of `unknown`, `up`, `down`, or `error`. `unknown` has never been checked; `up` means the final response was 200–399; `down` means it was 400–599; `error` represents a timeout, transport failure, redirect-policy violation, or SSRF rejection.
|
||||
A monitor has UUID `id`, unique `name`, HTTP(S) `url`, timestamps, monotonically increasing `revision`, and latest status fields. Create accepts name/url. Update is PUT-like for supplied fields. URL changes clear prior status.
|
||||
|
||||
| Method | Path | Meaning |
|
||||
|---|---|---|
|
||||
| POST | `/v1/monitors` | Create (201) |
|
||||
| GET | `/v1/monitors` | List, ordered by creation time |
|
||||
| GET | `/v1/monitors/{id}` | Retrieve |
|
||||
| PATCH | `/v1/monitors/{id}` | Partially update name/URL |
|
||||
| DELETE | `/v1/monitors/{id}` | Delete (204) |
|
||||
| POST | `/v1/monitors/{id}/check` | Run an on-demand check and atomically store it |
|
||||
| GET | `/v1/monitors/{id}/status` | Retrieve current status |
|
||||
| GET | `/healthz` | Liveness |
|
||||
| GET | `/readyz` | Readiness and storage mode |
|
||||
Status values:
|
||||
|
||||
Checks do not download response bodies. Redirects are followed explicitly up to the configured bound. Every hop is parsed, DNS-resolved, policy-checked, and connected through a resolver pinned to the approved addresses. If any answer is loopback, private, link-local, multicast, unspecified, reserved, or otherwise non-global, the hop is rejected. This conservative all-addresses rule and address pinning prevent mixed-answer and DNS-rebinding bypasses. Only HTTP and HTTPS URLs without credentials are accepted.
|
||||
- `unknown`: never checked or target changed
|
||||
- `up`: final HTTP response is 200–399
|
||||
- `down`: final HTTP response is 400–599
|
||||
- `error`: DNS, policy, timeout, redirect, or transport failure
|
||||
|
||||
The checker returns a status representation for expected outbound failures rather than turning remote endpoint behavior into a 5xx response. A concurrent monitor edit causes 409 and prevents a result for the old definition from replacing current state. Missing resources return 404. Validation returns 422. Errors use `{"error":{"code":"...","message":"..."}}` and never include outbound exception text.
|
||||
A check records UTC attempt time, elapsed milliseconds, final redacted URL, optional HTTP code, and a bounded non-secret detail. Results update the monitor under the store lock. A deletion racing with a check wins: the result is not resurrected and the check route returns 404.
|
||||
|
||||
## Concurrency and logging
|
||||
## Routes
|
||||
|
||||
Store operations and compare-and-set status recording are guarded by one `asyncio.Lock`. Reads return deep copies. A monotonically increasing internal revision makes check updates atomic with respect to edits/deletes.
|
||||
- `POST /v1/monitors` → 201
|
||||
- `GET /v1/monitors` → 200
|
||||
- `GET /v1/monitors/{id}` → 200
|
||||
- `PATCH /v1/monitors/{id}` → 200
|
||||
- `DELETE /v1/monitors/{id}` → 204
|
||||
- `POST /v1/monitors/{id}/check` → 200 result; 400 policy rejection
|
||||
- `GET /v1/monitors/{id}/status` → 200 status snapshot
|
||||
- `GET /healthz` → liveness
|
||||
- `GET /readyz` → process/store readiness
|
||||
|
||||
Application events are one-line JSON records. URLs are normalized for logs by removing user information, fragments, and all query values (`?REDACTED`). Outbound exception details are classified, not logged verbatim. API resources retain the configured URL because it is part of their explicit contract; operators should still avoid URL credentials and secrets.
|
||||
Errors use FastAPI's stable `{"detail": ...}` envelope: validation 422, not found 404, duplicate/capacity conflict 409, blocked check 400. Transport failures are check outcomes rather than API failures.
|
||||
|
||||
## Project structure
|
||||
## SSRF policy
|
||||
|
||||
- `app/main.py`: application factory, lifespan, handlers
|
||||
- `app/api.py`: REST contract
|
||||
- `app/models.py`: typed request/response models
|
||||
- `app/store.py`: concurrency-safe process-local state
|
||||
- `app/checker.py`: SSRF-safe checker and redacted event logging
|
||||
- `app/settings.py`: validated `MONITOR_` environment configuration
|
||||
- `tests/`: unit and API tests with no real outbound traffic
|
||||
- `Dockerfile`, `compose.yaml`, `pyproject.toml`: packaging and workflows
|
||||
- `docs/verification.md`: validation checklist and observed limitations
|
||||
Only `http` and `https`, explicit hostnames, and no user-info are accepted. Every DNS answer must be globally routable according to Python `ipaddress`; loopback, private, link-local, multicast, reserved, unspecified, and documentation ranges are rejected. The same validation is repeated for each redirect, redirects are followed manually, and redirect count and total request time are bounded. URL query and fragment data are omitted from logs and persisted final URLs.
|
||||
|
||||
DNS validation followed by a separate client connection has an unavoidable DNS-rebinding time-of-check/time-of-use window in this compact implementation. Production environments should additionally enforce an egress proxy/firewall that only permits public destinations or use a transport that pins validated addresses while preserving TLS SNI. The application-level checks remain defense in depth, not the sole network boundary.
|
||||
|
||||
## Logging
|
||||
|
||||
Logs are one-line JSON. Check events contain monitor ID, outcome, latency, and a redacted URL only. User-info, query strings, fragments, and response bodies are never logged. Error details are normalized and bounded.
|
||||
|
||||
## Lifecycle
|
||||
|
||||
The store is created with the app and reports ready while available. A restart loses state. Exactly one Uvicorn worker is required; horizontal scaling requires replacing the store with shared durable storage and distributed check coordination.
|
||||
|
||||
Reference in New Issue
Block a user