decomposer: generate deliverable files for Define the service contract and project architecture for the FastAPI endpoint monitoring service.; Implement the typed monitor CRUD API and concurrency-safe in-memory state according to the service design.; Implement secure on-demand endpoint checks with status updates, latency measurement, robust error handling, and redacted structured logs.; Add operational API endpoints and environment-driven runtime configuration to the monitoring service.; Create automated tests for the monitoring service.; Package the service with Docker and developer documentation.; Validate the complete project.
This commit is contained in:
89
README.md
89
README.md
@@ -1,15 +1,22 @@
|
||||
# Endpoint Monitor
|
||||
|
||||
A typed FastAPI service that stores endpoint monitors in memory and runs secure, on-demand HTTP checks. There is no authentication; do not expose it directly to untrusted networks.
|
||||
A small, unauthenticated FastAPI service that stores endpoint monitors in memory and checks them on demand. It validates DNS and every redirect destination against an SSRF policy, measures latency, updates status atomically, and emits structured logs with URL query data removed.
|
||||
|
||||
## Important operating constraints
|
||||
|
||||
State is process-local, ephemeral, and not shared. Restarting loses all monitors. Run **one worker and one replica only**; multiple workers/replicas return inconsistent data. There is no authentication, so bind to a trusted network or put an authenticated gateway in front. DNS validation reduces SSRF risk but network-layer egress controls are still necessary against DNS rebinding and implementation defects. URL paths are logged; do not put secrets in paths.
|
||||
|
||||
## Layout
|
||||
|
||||
* `docs/SERVICE_DESIGN.md` — complete API/status/security contract
|
||||
* `app/models.py`, `app/store.py` — typed resources and locked state
|
||||
* `app/security.py`, `app/checker.py` — DNS/redirect SSRF policy and checks
|
||||
* `app/main.py` — API and operational routes
|
||||
* `tests/` — mocked API, checker, race, and redaction tests
|
||||
* `docs/VERIFICATION.md` — reproducible validation record
|
||||
```text
|
||||
app/ FastAPI routes, models, locked store, checker and SSRF policy
|
||||
tests/ API/unit tests; outbound HTTP and DNS are mocked
|
||||
SERVICE_DESIGN.md full resource, error, status, security and logging contract
|
||||
Dockerfile non-root, healthchecked, one-worker image
|
||||
compose.yaml local container workflow
|
||||
pyproject.toml package, lint, type-check and test configuration
|
||||
requirements*.txt pinned direct runtime/development dependencies
|
||||
```
|
||||
|
||||
## Local development
|
||||
|
||||
@@ -18,42 +25,76 @@ Python 3.12 is required.
|
||||
```sh
|
||||
python -m venv .venv
|
||||
. .venv/bin/activate
|
||||
pip install -e '.[dev]'
|
||||
ruff format --check .
|
||||
python -m pip install -r requirements-dev.txt
|
||||
ruff check .
|
||||
ruff format --check .
|
||||
mypy app
|
||||
pytest
|
||||
uvicorn app.main:app --reload
|
||||
```
|
||||
|
||||
Create and check a monitor:
|
||||
OpenAPI is at <http://127.0.0.1:8000/docs>.
|
||||
|
||||
## API examples
|
||||
|
||||
```sh
|
||||
curl -sS -X POST http://localhost:8000/api/v1/monitors \
|
||||
curl -s http://127.0.0.1:8000/health/ready
|
||||
curl -s -X POST http://127.0.0.1:8000/monitors \
|
||||
-H 'content-type: application/json' \
|
||||
-d '{"name":"Example","url":"https://example.com/health?token=not-logged"}'
|
||||
curl -sS -X POST http://localhost:8000/api/v1/monitors/MONITOR_UUID/check
|
||||
curl -sS http://localhost:8000/api/v1/monitors/MONITOR_UUID/status
|
||||
curl -sS http://localhost:8000/healthz
|
||||
-d '{"name":"Example","target_url":"https://example.com/"}'
|
||||
curl -s http://127.0.0.1:8000/monitors
|
||||
curl -s -X PUT http://127.0.0.1:8000/monitors/MONITOR_UUID \
|
||||
-H 'content-type: application/json' \
|
||||
-d '{"name":"Example home","target_url":"https://example.com/"}'
|
||||
curl -s -X POST http://127.0.0.1:8000/monitors/MONITOR_UUID/check
|
||||
curl -s http://127.0.0.1:8000/monitors/MONITOR_UUID/status
|
||||
curl -i -X DELETE http://127.0.0.1:8000/monitors/MONITOR_UUID
|
||||
```
|
||||
|
||||
OpenAPI is at `/docs` and `/openapi.json`.
|
||||
HTTP below 400 is `up`, HTTP 400–599 is `down`, and network/DNS/timeout failures are `error`. A policy-blocked target returns 400 and leaves the previous completed status unchanged. Missing resources return 404 and invalid request data returns 422.
|
||||
|
||||
## Configuration
|
||||
|
||||
All settings use the `MONITOR_` prefix: `APP_NAME`, `LOG_LEVEL`, `REQUEST_TIMEOUT_SECONDS` (default 5, max 30), `MAX_REDIRECTS` (default 5, max 10), and `MAX_RESPONSE_BYTES` (default 65536, max 1048576). Invalid values fail startup.
|
||||
Settings are read at startup. Invalid values prevent startup.
|
||||
|
||||
## Container
|
||||
| Variable | Default | Constraint |
|
||||
|---|---:|---|
|
||||
| `MONITOR_HOST` | `0.0.0.0` | runtime launcher binding (Compose/Docker command is fixed) |
|
||||
| `MONITOR_PORT` | `8000` | 1–65535; use with a custom launcher command |
|
||||
| `MONITOR_LOG_LEVEL` | `INFO` | DEBUG/INFO/WARNING/ERROR/CRITICAL |
|
||||
| `MONITOR_CONNECT_TIMEOUT_SECONDS` | `2` | >0, <=30 |
|
||||
| `MONITOR_READ_TIMEOUT_SECONDS` | `5` | >0, <=60 |
|
||||
| `MONITOR_WRITE_TIMEOUT_SECONDS` | `5` | >0, <=60 |
|
||||
| `MONITOR_POOL_TIMEOUT_SECONDS` | `2` | >0, <=30 |
|
||||
| `MONITOR_MAX_REDIRECTS` | `5` | 0–10 |
|
||||
| `MONITOR_MAX_CONNECTIONS` | `50` | 1–500 |
|
||||
| `MONITOR_USER_AGENT` | `endpoint-monitor/1.0` | 1–100 characters |
|
||||
|
||||
## Containers
|
||||
|
||||
```sh
|
||||
docker build -t endpoint-monitor .
|
||||
docker run --rm -p 8000:8000 endpoint-monitor
|
||||
# or
|
||||
docker build -t endpoint-monitor:local .
|
||||
docker run --rm -p 8000:8000 endpoint-monitor:local
|
||||
docker inspect --format '{{json .Config.Healthcheck}}' endpoint-monitor:local
|
||||
docker compose up --build
|
||||
```
|
||||
|
||||
The image uses an unprivileged user and one Uvicorn worker. One worker is mandatory because data is process-local.
|
||||
The image uses a pinned Python 3.12 slim base tag, installs runtime dependencies only, runs as UID/GID 10001, declares a Python-based liveness healthcheck (no curl dependency), disables access logs to avoid unredacted query strings, and explicitly starts one Uvicorn worker.
|
||||
|
||||
## Important limitations
|
||||
## Verification and current evidence
|
||||
|
||||
State is neither persistent nor shared: restart loses all monitors and multiple workers have divergent data. DNS checks occur before each hop, but the default HTTP stack resolves again while connecting. Use an egress proxy/firewall in hostile production environments to close that DNS-rebinding timing window. Logs redact URL credentials, query, and fragment; avoid secrets in path segments.
|
||||
The intended clean-room verification sequence is:
|
||||
|
||||
```sh
|
||||
python -m pip install -r requirements-dev.txt
|
||||
ruff check . && ruff format --check .
|
||||
mypy app
|
||||
pytest
|
||||
docker build -t endpoint-monitor:verify .
|
||||
docker run -d --rm --name endpoint-monitor-verify -p 18000:8000 endpoint-monitor:verify
|
||||
python -c "import urllib.request; print(urllib.request.urlopen('http://127.0.0.1:18000/health/live').read())"
|
||||
docker inspect --format '{{.State.Health.Status}}' endpoint-monitor-verify
|
||||
docker rm -f endpoint-monitor-verify
|
||||
```
|
||||
|
||||
Generation persisted all source, tests, packaging, and documentation in one repository, but the generation environment exposed no shell or Docker executor. Consequently these commands were **not executed during generation**; there is no claim of passing test/build/startup evidence. The committed tests are inspectable evidence of intended coverage, not execution evidence. Validators should run the sequence above. See `VERIFICATION.md` for the exact honest status.
|
||||
|
||||
Reference in New Issue
Block a user