decomposer: generate deliverable files for Define the service contract and project architecture for the FastAPI endpoint monitoring service.; Implement the typed monitor CRUD API and concurrency-safe in-memory state according to the service design.; Implement secure on-demand endpoint checks with status updates, latency measurement, robust error handling, and redacted structured logs.; Add operational API endpoints and environment-driven runtime configuration to the monitoring service.; Create automated tests for the monitoring service.; Package the service with Docker and developer documentation.; Validate the complete project.
Some checks are pending
ci / test (push) Has started running
Some checks are pending
ci / test (push) Has started running
This commit is contained in:
121
README.md
121
README.md
@@ -1,75 +1,72 @@
|
|||||||
# FastAPI Endpoint Monitor
|
# Endpoint Monitor
|
||||||
|
|
||||||
A typed FastAPI service for process-local endpoint monitors and secure, on-demand HTTP checks.
|
A typed FastAPI service that stores endpoint monitors in concurrency-safe, process-local memory and checks them on demand with bounded timeouts, redirect validation, DNS-based SSRF policy, latency/status updates, and redacted JSON logs. There is intentionally no authentication.
|
||||||
|
|
||||||
## Features
|
|
||||||
|
|
||||||
- CRUD monitor resources and current-status retrieval
|
|
||||||
- bounded on-demand HTTP checks with latency measurement
|
|
||||||
- DNS and redirect-hop SSRF validation (only globally routable HTTP(S) targets)
|
|
||||||
- concurrency-safe in-memory storage
|
|
||||||
- health and readiness endpoints
|
|
||||||
- JSON logs with URL user-info, query, and fragment removed
|
|
||||||
- environment-validated settings, Docker packaging, and automated tests
|
|
||||||
|
|
||||||
> State is process-local and ephemeral. Run exactly one worker. Restarts erase all monitors, and multiple replicas do not share state.
|
|
||||||
|
|
||||||
## Layout
|
## Layout
|
||||||
|
- `SPEC.md`: normative API, status, security, logging, and lifecycle contract
|
||||||
|
- `app/`: configuration, models, locked store, SSRF policy, checker, and API
|
||||||
|
- `tests/`: API/unit tests with mocked outbound HTTP and DNS
|
||||||
|
- `Dockerfile`, `compose.yaml`: single-worker container packaging
|
||||||
|
|
||||||
- `docs/service-design.md` — contract, status semantics, security design, and architecture
|
## Local development
|
||||||
- `src/endpoint_monitor/` — application source
|
Requires Python 3.12.
|
||||||
- `tests/` — API and checker tests with mocked outbound HTTP/DNS
|
|
||||||
- `VALIDATION.md` — validation commands and evidence status
|
|
||||||
|
|
||||||
## Run locally
|
```sh
|
||||||
|
|
||||||
```bash
|
|
||||||
python -m venv .venv
|
python -m venv .venv
|
||||||
. .venv/bin/activate
|
. .venv/bin/activate
|
||||||
pip install -e '.[dev]'
|
pip install -e '.[dev]'
|
||||||
uvicorn endpoint_monitor.main:app --reload
|
|
||||||
```
|
|
||||||
|
|
||||||
OpenAPI is at <http://127.0.0.1:8000/docs>.
|
|
||||||
|
|
||||||
## Examples
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl -sS http://localhost:8000/healthz
|
|
||||||
curl -sS -X POST http://localhost:8000/v1/monitors \
|
|
||||||
-H 'content-type: application/json' \
|
|
||||||
-d '{"name":"example","url":"https://example.com"}'
|
|
||||||
curl -sS -X POST http://localhost:8000/v1/monitors/MONITOR_ID/check
|
|
||||||
curl -sS http://localhost:8000/v1/monitors/MONITOR_ID/status
|
|
||||||
```
|
|
||||||
|
|
||||||
Create returns `201`; reads/checks return `200`; delete returns `204`; duplicate names return `409`; missing IDs return `404`; invalid or SSRF-blocked check targets return `400`. Transport failures are represented by a `200` check result with `status: "error"` so the latest attempt remains inspectable.
|
|
||||||
|
|
||||||
## Configuration
|
|
||||||
|
|
||||||
Settings use the `MONITOR_` prefix:
|
|
||||||
|
|
||||||
| Variable | Default | Constraint |
|
|
||||||
|---|---:|---|
|
|
||||||
| `MONITOR_REQUEST_TIMEOUT_SECONDS` | `5.0` | 0.1–30 |
|
|
||||||
| `MONITOR_MAX_REDIRECTS` | `5` | 0–10 |
|
|
||||||
| `MONITOR_MAX_MONITORS` | `1000` | 1–100000 |
|
|
||||||
| `MONITOR_LOG_LEVEL` | `INFO` | DEBUG/INFO/WARNING/ERROR/CRITICAL |
|
|
||||||
|
|
||||||
Unknown variables are ignored. Invalid known values fail application startup.
|
|
||||||
|
|
||||||
## Quality checks
|
|
||||||
|
|
||||||
```bash
|
|
||||||
ruff format --check .
|
ruff format --check .
|
||||||
ruff check .
|
ruff check .
|
||||||
mypy src
|
mypy app
|
||||||
pytest -q
|
pytest -q
|
||||||
|
uvicorn app.main:app --reload
|
||||||
docker build -t endpoint-monitor .
|
|
||||||
docker run --rm -p 8000:8000 endpoint-monitor
|
|
||||||
curl -fsS http://localhost:8000/healthz
|
|
||||||
curl -fsS http://localhost:8000/readyz
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Tests never make real outbound requests. See `VALIDATION.md` for evidence and limitations.
|
OpenAPI is at `http://localhost:8000/docs`.
|
||||||
|
|
||||||
|
## Examples
|
||||||
|
```sh
|
||||||
|
curl -s http://localhost:8000/healthz
|
||||||
|
curl -s http://localhost:8000/readyz
|
||||||
|
curl -s -X POST http://localhost:8000/monitors -H 'content-type: application/json' \
|
||||||
|
-d '{"name":"example","url":"https://example.com/?token=secret","timeout_seconds":3}'
|
||||||
|
curl -s -X POST http://localhost:8000/monitors/UUID/check
|
||||||
|
curl -s http://localhost:8000/monitors/UUID/status
|
||||||
|
```
|
||||||
|
CRUD also supports `GET /monitors`, `GET|PUT|DELETE /monitors/{id}`.
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
All settings are startup-validated. Invalid values prevent startup.
|
||||||
|
|
||||||
|
| Environment | Default | Constraint |
|
||||||
|
|---|---:|---|
|
||||||
|
| `MONITOR_REQUEST_TIMEOUT_SECONDS` | 5 | >0, <=30 |
|
||||||
|
| `MONITOR_MAX_REDIRECTS` | 5 | 0..10 |
|
||||||
|
| `MONITOR_MAX_MONITORS` | 1000 | 1..100000 |
|
||||||
|
| `MONITOR_LOG_LEVEL` | INFO | standard uppercase level |
|
||||||
|
|
||||||
|
Per-monitor timeout overrides the default. Query strings are emitted only as `?REDACTED`.
|
||||||
|
|
||||||
|
## Container
|
||||||
|
```sh
|
||||||
|
docker build -t endpoint-monitor .
|
||||||
|
docker run --rm -p 8000:8000 endpoint-monitor
|
||||||
|
# or: docker compose up --build
|
||||||
|
curl http://localhost:8000/healthz
|
||||||
|
```
|
||||||
|
The image runs as a non-root user with exactly one Uvicorn worker.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
Run the lint, format, type, and test commands above, then:
|
||||||
|
```sh
|
||||||
|
docker build -t endpoint-monitor .
|
||||||
|
docker run -d --rm --name endpoint-monitor -p 8000:8000 endpoint-monitor
|
||||||
|
curl --fail http://localhost:8000/healthz
|
||||||
|
curl --fail http://localhost:8000/readyz
|
||||||
|
docker stop endpoint-monitor
|
||||||
|
```
|
||||||
|
|
||||||
|
## Security and operational limitations
|
||||||
|
DNS and every redirect target are checked and non-global addresses are denied. This policy is defense in depth, not a substitute for egress firewalling: the standard HTTP transport performs its own DNS lookup, so hostile DNS rebinding between policy resolution and connection remains possible. Enforce outbound network policy in production.
|
||||||
|
|
||||||
|
Data is ephemeral and isolated per process. Restarting loses all monitors; multiple workers produce divergent state. There is no scheduler, persistence, cross-process readiness dependency, TLS termination, rate limiting, or authentication. Deploy one worker, place behind appropriate controls, and use persistent shared storage before scaling.
|
||||||
|
|||||||
Reference in New Issue
Block a user