Unify log import script #3
4 changed files with 36 additions and 24 deletions
|
@ -21,8 +21,8 @@ This document outlines general practices and expectations for AI agents assistin
|
||||||
source .venv/bin/activate
|
source .venv/bin/activate
|
||||||
pip install -r requirements.txt
|
pip install -r requirements.txt
|
||||||
```
|
```
|
||||||
The `init.sh` script can create this environment automatically. Always
|
The `run-import.sh` script can initialize this environment automatically.
|
||||||
activate it before running scripts or tests.
|
Always activate the virtual environment before running scripts or tests.
|
||||||
|
|
||||||
* Dependency management: Use `requirements.txt` or `pip-tools`
|
* Dependency management: Use `requirements.txt` or `pip-tools`
|
||||||
* Use standard libraries where feasible (e.g., `sqlite3`, `argparse`, `datetime`)
|
* Use standard libraries where feasible (e.g., `sqlite3`, `argparse`, `datetime`)
|
||||||
|
|
10
README.md
10
README.md
|
@ -23,3 +23,13 @@ python scripts/generate_reports.py monthly
|
||||||
```
|
```
|
||||||
|
|
||||||
Reports are written under the `output/` directory. Each command updates the corresponding `<interval>.json` file and produces an HTML dashboard using Chart.js.
|
Reports are written under the `output/` directory. Each command updates the corresponding `<interval>.json` file and produces an HTML dashboard using Chart.js.
|
||||||
|
|
||||||
|
## Importing Logs
|
||||||
|
|
||||||
|
Use the `run-import.sh` script to set up the Python environment if needed and import the latest Nginx log entries into `database/ngxstat.db`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./run-import.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
This script is suitable for cron jobs as it creates the virtual environment on first run, installs dependencies and reuses the environment on subsequent runs.
|
||||||
|
|
13
init.sh
13
init.sh
|
@ -1,13 +0,0 @@
|
||||||
#!/bin/bash
|
|
||||||
set -e
|
|
||||||
|
|
||||||
echo "[INFO] Creating virtual environment..."
|
|
||||||
python3 -m venv .venv
|
|
||||||
source .venv/bin/activate
|
|
||||||
|
|
||||||
echo "[INFO] Installing dependencies..."
|
|
||||||
pip install --upgrade pip
|
|
||||||
pip install -r requirements.txt || echo "[WARN] requirements.txt not found, skipping."
|
|
||||||
|
|
||||||
echo "[INFO] Running database setup..."
|
|
||||||
python scripts/init_db.py
|
|
|
@ -1,13 +1,28 @@
|
||||||
#!/bin/bash
|
#!/usr/bin/env bash
|
||||||
set -e
|
set -e
|
||||||
|
|
||||||
|
# Ensure virtual environment exists
|
||||||
|
if [ ! -d ".venv" ]; then
|
||||||
echo "[INFO] Creating virtual environment..."
|
echo "[INFO] Creating virtual environment..."
|
||||||
python3 -m venv .venv
|
python3 -m venv .venv
|
||||||
source .venv/bin/activate
|
source .venv/bin/activate
|
||||||
|
|
||||||
echo "[INFO] Installing dependencies..."
|
echo "[INFO] Installing dependencies..."
|
||||||
pip install --upgrade pip
|
pip install --upgrade pip
|
||||||
pip install -r requirements.txt || echo "[WARN] requirements.txt not found, skipping."
|
if [ -f requirements.txt ]; then
|
||||||
|
pip install -r requirements.txt
|
||||||
|
else
|
||||||
|
echo "[WARN] requirements.txt not found, skipping."
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
echo "[INFO] Activating virtual environment..."
|
||||||
|
source .venv/bin/activate
|
||||||
|
fi
|
||||||
|
|
||||||
echo "[INFO] Running database setup..."
|
# Run log import
|
||||||
|
echo "[INFO] Importing logs..."
|
||||||
python scripts/init_db.py
|
python scripts/init_db.py
|
||||||
|
|
||||||
|
# Deactivate to keep cron environment clean
|
||||||
|
if type deactivate >/dev/null 2>&1; then
|
||||||
|
deactivate
|
||||||
|
fi
|
||||||
|
|
Loading…
Add table
Add a link
Reference in a new issue