decomposer: generate deliverable files for Define the service contract and project architecture for the FastAPI endpoint monitoring service.; Implement the typed monitor CRUD API and concurrency-safe in-memory state according to the service design.; Implement secure on-demand endpoint checks with status updates, latency measurement, robust error handling, and redacted structured logs.; Add operational API endpoints and environment-driven runtime configuration to the monitoring service.; Create automated tests for the monitoring service.; Package the service with Docker and developer documentation.; Validate the complete project.
Some checks failed
ci / validate (push) Has been cancelled
Some checks failed
ci / validate (push) Has been cancelled
This commit is contained in:
116
README.md
116
README.md
@@ -1,59 +1,75 @@
|
|||||||
# Endpoint Monitor Service
|
# FastAPI Endpoint Monitor
|
||||||
|
|
||||||
A typed FastAPI service for process-local endpoint monitors and secure, on-demand HTTP checks. The complete contract and security semantics are in [SPEC.md](SPEC.md).
|
A typed FastAPI service for process-local endpoint monitors and secure, on-demand HTTP checks.
|
||||||
|
|
||||||
## Quick start
|
## Features
|
||||||
|
|
||||||
Requires Python 3.12.
|
- CRUD monitor resources and current-status retrieval
|
||||||
|
- bounded on-demand HTTP checks with latency measurement
|
||||||
|
- DNS and redirect-hop SSRF validation (only globally routable HTTP(S) targets)
|
||||||
|
- concurrency-safe in-memory storage
|
||||||
|
- health and readiness endpoints
|
||||||
|
- JSON logs with URL user-info, query, and fragment removed
|
||||||
|
- environment-validated settings, Docker packaging, and automated tests
|
||||||
|
|
||||||
```sh
|
> State is process-local and ephemeral. Run exactly one worker. Restarts erase all monitors, and multiple replicas do not share state.
|
||||||
python -m venv .venv
|
|
||||||
. .venv/bin/activate
|
|
||||||
pip install -e '.[dev]'
|
|
||||||
make check
|
|
||||||
uvicorn app.main:app --reload
|
|
||||||
```
|
|
||||||
|
|
||||||
Open `http://127.0.0.1:8000/docs`. Example:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
curl -sS -X POST http://127.0.0.1:8000/monitors \
|
|
||||||
-H 'content-type: application/json' \
|
|
||||||
-d '{"name":"Example","url":"https://example.com/health?token=not-logged"}'
|
|
||||||
curl -sS -X POST http://127.0.0.1:8000/monitors/UUID/check
|
|
||||||
curl -sS http://127.0.0.1:8000/monitors/UUID/status
|
|
||||||
```
|
|
||||||
|
|
||||||
## Configuration
|
|
||||||
|
|
||||||
Settings are validated at startup and use the `MONITOR_` prefix: `LOG_LEVEL` (default `INFO`), `TOTAL_TIMEOUT_SECONDS` (10, max 60), `CONNECT_TIMEOUT_SECONDS` (3), `READ_TIMEOUT_SECONDS` (5), `MAX_REDIRECTS` (5, max 10), and `USER_AGENT`. Connect/read values cannot exceed total timeout.
|
|
||||||
|
|
||||||
## Security behavior
|
|
||||||
|
|
||||||
Every hop is resolved and all returned addresses must be globally routable. A mixed public/private DNS response is rejected. A dedicated connector receives only the validated addresses for that hop, so the HTTP library cannot perform a second unvalidated lookup; TLS still verifies the URL hostname. Redirects are followed manually only after the next URL passes the same policy. Literal IPs are classified identically, URL credentials are rejected, environment proxies are ignored, redirect count is bounded, and the entire operation (including DNS and all hops) has a total timeout. Logs contain a redacted URL with no query, fragment, or credentials.
|
|
||||||
|
|
||||||
The checks in `tests/test_security.py` provide inspectable evidence for private/special IP blocking, mixed-answer rejection, address pinning, per-hop redirect validation, total timeout mapping, and query redaction. `tests/test_atomic.py` proves revision-based atomic status application. Outbound HTTP is faked in tests; tests never depend on the public network.
|
|
||||||
|
|
||||||
## Docker
|
|
||||||
|
|
||||||
```sh
|
|
||||||
docker build -t endpoint-monitor:local .
|
|
||||||
docker run --rm -p 8000:8000 endpoint-monitor:local
|
|
||||||
# or
|
|
||||||
docker compose up --build
|
|
||||||
```
|
|
||||||
|
|
||||||
The image runs as a non-root user with exactly one Uvicorn worker and a healthcheck.
|
|
||||||
|
|
||||||
## Layout
|
## Layout
|
||||||
|
|
||||||
* `SPEC.md` — route/status/error/security contract and architecture
|
- `docs/service-design.md` — contract, status semantics, security design, and architecture
|
||||||
* `app/main.py` — FastAPI routes and dependency wiring
|
- `src/endpoint_monitor/` — application source
|
||||||
* `app/store.py` — lock-protected volatile state and compare-and-set status
|
- `tests/` — API and checker tests with mocked outbound HTTP/DNS
|
||||||
* `app/security.py` — URL redaction, DNS resolution, and IP policy
|
- `VALIDATION.md` — validation commands and evidence status
|
||||||
* `app/checker.py` — pinned transport, redirects, timeouts, latency/error mapping
|
|
||||||
* `tests/` — API, policy, timeout, logging, and concurrency tests
|
|
||||||
|
|
||||||
## Operational limitation
|
## Run locally
|
||||||
|
|
||||||
There is deliberately no database. All monitors vanish on restart and multiple processes would have divergent state. Keep one worker, as the container does. For durability or horizontal scaling, replace `MonitorStore` with shared transactional storage before deployment. There is no authentication; place the service behind suitable network and identity controls.
|
```bash
|
||||||
|
python -m venv .venv
|
||||||
|
. .venv/bin/activate
|
||||||
|
pip install -e '.[dev]'
|
||||||
|
uvicorn endpoint_monitor.main:app --reload
|
||||||
|
```
|
||||||
|
|
||||||
|
OpenAPI is at <http://127.0.0.1:8000/docs>.
|
||||||
|
|
||||||
|
## Examples
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -sS http://localhost:8000/healthz
|
||||||
|
curl -sS -X POST http://localhost:8000/v1/monitors \
|
||||||
|
-H 'content-type: application/json' \
|
||||||
|
-d '{"name":"example","url":"https://example.com"}'
|
||||||
|
curl -sS -X POST http://localhost:8000/v1/monitors/MONITOR_ID/check
|
||||||
|
curl -sS http://localhost:8000/v1/monitors/MONITOR_ID/status
|
||||||
|
```
|
||||||
|
|
||||||
|
Create returns `201`; reads/checks return `200`; delete returns `204`; duplicate names return `409`; missing IDs return `404`; invalid or SSRF-blocked check targets return `400`. Transport failures are represented by a `200` check result with `status: "error"` so the latest attempt remains inspectable.
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
Settings use the `MONITOR_` prefix:
|
||||||
|
|
||||||
|
| Variable | Default | Constraint |
|
||||||
|
|---|---:|---|
|
||||||
|
| `MONITOR_REQUEST_TIMEOUT_SECONDS` | `5.0` | 0.1–30 |
|
||||||
|
| `MONITOR_MAX_REDIRECTS` | `5` | 0–10 |
|
||||||
|
| `MONITOR_MAX_MONITORS` | `1000` | 1–100000 |
|
||||||
|
| `MONITOR_LOG_LEVEL` | `INFO` | DEBUG/INFO/WARNING/ERROR/CRITICAL |
|
||||||
|
|
||||||
|
Unknown variables are ignored. Invalid known values fail application startup.
|
||||||
|
|
||||||
|
## Quality checks
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ruff format --check .
|
||||||
|
ruff check .
|
||||||
|
mypy src
|
||||||
|
pytest -q
|
||||||
|
|
||||||
|
docker build -t endpoint-monitor .
|
||||||
|
docker run --rm -p 8000:8000 endpoint-monitor
|
||||||
|
curl -fsS http://localhost:8000/healthz
|
||||||
|
curl -fsS http://localhost:8000/readyz
|
||||||
|
```
|
||||||
|
|
||||||
|
Tests never make real outbound requests. See `VALIDATION.md` for evidence and limitations.
|
||||||
|
|||||||
Reference in New Issue
Block a user