Build Nginx live traffic monitor

This commit is contained in:
Jordan Wages 2026-09-17 17:47:16 -05:00
commit fb66a69bd7
9 changed files with 525 additions and 2 deletions

View file

@ -1,3 +1,93 @@
# nginx-realtime-traffic-dashboard
# Nginx Live Traffic Monitor
Shows realtime traffic flows for an nginx reverse proxy per server.
![Dashboard screenshot](docs/screenshot-placeholder.svg)
A small Python/FastAPI dashboard for seeing live traffic through an Nginx reverse proxy. It follows one dedicated access log, aggregates requests into bounded one-second in-memory buckets, and streams telemetry to browsers over Server-Sent Events. There is no database or persistent application telemetry.
## Requirements and installation
Python 3.10+ is required. From a checkout:
```sh
python3 -m venv .venv
. .venv/bin/activate
pip install -r requirements.txt
```
The dashboard uses Bulma and Chart.js from their public CDNs. For an offline deployment, vendor those two assets and update `app/static/index.html`.
## Nginx log format
Add the dedicated log (the application does not change Nginx configuration):
```nginx
log_format traffic '$msec $host $request_method $status '
'$request_length $bytes_sent '
'$request_time $upstream_response_time';
access_log /var/log/nginx-traffic/traffic.log traffic;
```
The follower tolerates malformed lines, `-` upstream times, multiple upstream values, temporary disappearance, and rename/replacement rotation. It starts at the end of an existing file, like `tail -F`, so old log history is not mistaken for live traffic. The application keeps only configured rolling buckets (and bounded latency samples); totals are from application startup.
## Running
```sh
python -m app.main
```
Open <http://127.0.0.1:8080>. Configuration is available through flags or environment variables:
| Setting | Flag | Environment variable | Default |
| --- | --- | --- | --- |
| Traffic log | `--log-path` | `TRAFFIC_LOG_PATH` | `/var/log/nginx-traffic/traffic.log` |
| Listen address | `--host` | `LISTEN_ADDRESS` | `127.0.0.1` |
| Listen port | `--port` | `LISTEN_PORT` | `8080` |
| Retention seconds | `--retention` | `TELEMETRY_RETENTION` | `3600` |
Example:
```sh
TRAFFIC_LOG_PATH=/var/log/nginx-traffic/traffic.log LISTEN_ADDRESS=0.0.0.0 LISTEN_PORT=8080 \
python -m app.main --retention 3600
```
The visible window can be set to 1 minute, 5 minutes, 15 minutes, or 1 hour. A new browser receives enough current history to fill its selected graph window. Host names are discovered automatically and can be selected for breakout graphs. Theme selection (system, light, dark) is stored in browser local storage.
## tmpfs and permissions
One possible tmpfs setup is:
```sh
sudo install -d -o nginx -g nginx -m 0750 /var/log/nginx-traffic
sudo mount -t tmpfs -o size=64M,mode=0750,uid=nginx,gid=nginx tmpfs /var/log/nginx-traffic
```
The service account needs search/read permission on the directory and read permission on `traffic.log`. If Nginx writes as another group, grant the monitor account group access or use an ACL, for example:
```sh
sudo setfacl -m u:nginx-monitor:rx /var/log/nginx-traffic
sudo setfacl -m u:nginx-monitor:r /var/log/nginx-traffic/traffic.log
```
## systemd
Copy the included `nginx-traffic-monitor.service` to `/etc/systemd/system/`, adjust `WorkingDirectory`, virtualenv path, and the `User`/`Group` to match the installation, then:
```sh
sudo systemctl daemon-reload
sudo systemctl enable --now nginx-traffic-monitor
sudo systemctl status nginx-traffic-monitor
```
The service should run as an unprivileged account. Put a reverse proxy or firewall in front of it if it must be accessed beyond localhost; this utility intentionally has no authentication.
## Development
Run the focused parser/aggregation tests with:
```sh
pytest -q
```
The server task follows the path by inode, drains currently available data from a renamed file, and reopens a replacement. A missing path is retried without busy polling. The publisher sends one compact snapshot per second to each SSE subscriber, and disconnecting clients are removed automatically.