decomposer: generate deliverable files for Define the service contract and project architecture for the FastAPI endpoint monitoring service.; Implement the typed monitor CRUD API and concurrency-safe in-memory state according to the service design.; Implement secure on-demand endpoint checks with status updates, latency measurement, robust error handling, and redacted structured logs.; Add operational API endpoints and environment-driven runtime configuration to the monitoring service.; Create automated tests for the monitoring service.; Package the service with Docker and developer documentation.; Validate the complete project.
This commit is contained in:
86
README.md
86
README.md
@@ -1,77 +1,55 @@
|
||||
# Endpoint Monitor
|
||||
# Endpoint Monitor Service
|
||||
|
||||
A typed, unauthenticated FastAPI service that keeps endpoint monitor definitions in concurrency-safe process-local memory and checks HTTP(S) targets on demand. It measures total latency, records the final HTTP status, follows bounded redirects, emits redacted JSON events, and rejects SSRF destinations at every hop.
|
||||
A typed FastAPI API that keeps endpoint monitors in concurrency-safe, process-local memory and performs SSRF-aware checks on demand. The service has no authentication and should not be exposed directly to untrusted users.
|
||||
|
||||
## Security and operational boundaries
|
||||
## Quick start
|
||||
|
||||
The service is deliberately unauthenticated; expose it only on a trusted network. It rejects credentials in URLs and rejects every DNS result set containing a non-global address (loopback, RFC1918/private, link-local, reserved, multicast, or unspecified). Approved DNS answers are pinned into the actual connection resolver, closing the resolution/connection DNS-rebinding gap. Redirects receive the same checks. Response bodies are never consumed.
|
||||
Requires Python 3.12.
|
||||
|
||||
State is **process-local and ephemeral**. Restarting loses all monitors, and multiple workers would each have divergent state. Run exactly one worker, as the supplied commands do. Use a persistent shared store before scaling horizontally.
|
||||
|
||||
## Layout
|
||||
|
||||
See `docs/service-design.md` for the complete contract. Application modules are under `app/`, tests under `tests/`, and validation evidence/checklists in `docs/verification.md`.
|
||||
|
||||
## Local development
|
||||
|
||||
Python 3.12 is required.
|
||||
|
||||
```bash
|
||||
```sh
|
||||
python -m venv .venv
|
||||
. .venv/bin/activate
|
||||
python -m pip install -e '.[dev]'
|
||||
ruff format --check .
|
||||
ruff check .
|
||||
mypy app
|
||||
pytest
|
||||
uvicorn app.main:app --host 127.0.0.1 --port 8000 --workers 1
|
||||
./scripts/validate.sh
|
||||
uvicorn monitor_service.api:app --reload
|
||||
```
|
||||
|
||||
No test performs a real outbound request; checker transport and resolution are mocked.
|
||||
Create and check a monitor:
|
||||
|
||||
## API examples
|
||||
|
||||
```bash
|
||||
curl -s http://127.0.0.1:8000/healthz
|
||||
curl -s -X POST http://127.0.0.1:8000/v1/monitors \
|
||||
-H 'content-type: application/json' \
|
||||
-d '{"name":"example","url":"https://example.com/health"}'
|
||||
curl -s http://127.0.0.1:8000/v1/monitors
|
||||
curl -s -X POST http://127.0.0.1:8000/v1/monitors/MONITOR_UUID/check
|
||||
curl -s http://127.0.0.1:8000/v1/monitors/MONITOR_UUID/status
|
||||
curl -s -X PATCH http://127.0.0.1:8000/v1/monitors/MONITOR_UUID \
|
||||
-H 'content-type: application/json' -d '{"name":"renamed"}'
|
||||
curl -i -X DELETE http://127.0.0.1:8000/v1/monitors/MONITOR_UUID
|
||||
```sh
|
||||
curl -sS -X POST http://localhost:8000/monitors -H 'content-type: application/json' \
|
||||
-d '{"name":"Example","url":"https://example.com/?secret=not-logged"}'
|
||||
curl -sS -X POST http://localhost:8000/monitors/MONITOR_UUID/check
|
||||
curl -sS http://localhost:8000/monitors/MONITOR_UUID/status
|
||||
```
|
||||
|
||||
Interactive OpenAPI documentation is at `/docs`. `up` means 200–399, `down` means 400–599, `error` means checking could not safely produce an HTTP response, and `unknown` means never checked.
|
||||
OpenAPI is at `/docs`. Liveness and readiness are at `/healthz` and `/readyz`.
|
||||
|
||||
## Configuration
|
||||
|
||||
All settings are validated at startup.
|
||||
All settings use the `MONITOR_` prefix: `APP_NAME`, `LOG_LEVEL`, `CONNECT_TIMEOUT_SECONDS` (default 3), `READ_TIMEOUT_SECONDS` (5), `POOL_TIMEOUT_SECONDS` (3), `MAX_REDIRECTS` (5), and `USER_AGENT`. Invalid or out-of-range values fail startup. Use one worker: memory is not durable or shared among processes.
|
||||
|
||||
| Variable | Default | Bounds |
|
||||
|---|---:|---:|
|
||||
| `MONITOR_APP_NAME` | `endpoint-monitor` | string |
|
||||
| `MONITOR_LOG_LEVEL` | `INFO` | logging level |
|
||||
| `MONITOR_REQUEST_TIMEOUT_SECONDS` | `5` | >0, <=30 |
|
||||
| `MONITOR_CONNECT_TIMEOUT_SECONDS` | `2` | >0, <=10 |
|
||||
| `MONITOR_MAX_REDIRECTS` | `5` | 0–10 |
|
||||
## Security behavior
|
||||
|
||||
Every initial and redirected HTTP(S) target is resolved before a request. A hop is denied if any answer is non-global, URL credentials are denied, redirect count is bounded, and timeouts are finite. Logs and check responses remove URL query and fragment data. See `docs/service-contract.md` for status/error semantics and the documented DNS resolver/connect TOCTOU limitation.
|
||||
|
||||
## Container
|
||||
|
||||
```bash
|
||||
docker build -t endpoint-monitor:local .
|
||||
docker run --rm -p 8000:8000 endpoint-monitor:local
|
||||
# or
|
||||
docker compose up --build
|
||||
```sh
|
||||
docker build -t endpoint-monitor .
|
||||
docker run --rm -p 8000:8000 endpoint-monitor
|
||||
# or: docker compose up --build
|
||||
curl -f http://localhost:8000/healthz
|
||||
```
|
||||
|
||||
The image uses a slim runtime, a non-root user, one Uvicorn worker, and an internal health check. Smoke test it with:
|
||||
The image is two-stage, runs as a non-root user, contains a healthcheck, and deliberately starts one Uvicorn worker.
|
||||
|
||||
```bash
|
||||
curl --fail http://127.0.0.1:8000/healthz
|
||||
curl --fail http://127.0.0.1:8000/readyz
|
||||
```
|
||||
## Layout and verification
|
||||
|
||||
Logs contain JSON application events with query values replaced by `REDACTED`; Uvicorn access logging is disabled in the container so raw query strings are not reintroduced there.
|
||||
- `docs/service-contract.md`: API and architecture contract
|
||||
- `src/monitor_service/`: application, checker, state, models, settings, logging
|
||||
- `tests/`: API, concurrency, checker, SSRF redirect/DNS, failure, and redaction tests
|
||||
- `scripts/validate.sh`: the authoritative Ruff format/lint, mypy, and pytest contract
|
||||
- `Dockerfile` and `compose.yaml`: container workflows
|
||||
|
||||
Run `./scripts/validate.sh`. Container verification additionally uses the build/run/curl commands above. `VALIDATION.md` records what was and was not executed by generation; it must not be read as a substitute for fresh CI output.
|
||||
|
||||
Reference in New Issue
Block a user