From 0c504a5918042f0b6c118e1edc322d27f92bb01f Mon Sep 17 00:00:00 2001 From: janis steiner Date: Wed, 6 May 2026 19:42:55 +0200 Subject: [PATCH] docs --- README.md | 245 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 245 insertions(+) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..3d579af --- /dev/null +++ b/README.md @@ -0,0 +1,245 @@ +# Jakach Logging + +A self-hosted log analysis and alerting system. Ingests logs from files, TCP/UDP syslog, and HTTP, evaluates them against configurable rules, notifies via Telegram, and provides a Bootstrap web UI for management. + +## Architecture + +``` +┌──────────────┐ ┌──────────────┐ ┌──────────────┐ +│ Log Files │ │ TCP/UDP │ │ HTTP POST │ +│ (tail -f) │ │ syslog │ │ /ingest │ +└──────┬───────┘ └──────┬───────┘ └──────┬───────┘ + │ │ │ + └─────────────────┼──────────────────┘ + │ + ┌──────────▼──────────┐ + │ Worker (PHP-CLI) │ + │ FileWatcher │ + │ SocketListener │ + └──────────┬──────────┘ + │ + ┌──────────▼──────────┐ + │ Rule Engine │ + │ preg_match rules │ + │ Rate Limiter │ + └──────────┬──────────┘ + │ + ┌──────────▼──────────┐ + │ TelegramNotifier │ + └──────────┬──────────┘ + │ + ┌──────────▼──────────┐ + │ SQLite Database │ + │ (FTS5 full-text) │ + └──────────┬──────────┘ + │ + ┌──────────▼──────────┐ + │ API (PHP-FPM) │ + │ REST + Auth │ + └──────────┬──────────┘ + │ + ┌──────────▼──────────┐ + │ Web UI (Bootstrap)│ + │ nginx :8080 │ + └─────────────────────┘ +``` + +## Quick Start + +```bash +# Clone and start +git clone jakach-logging +cd jakach-logging +docker compose build --no-cache && docker compose up -d + +# Seed default rules and syslog sources +docker compose exec api php bin/seed + +# Open the UI +open http://localhost:8080 +``` + +The first user to log in via Jakach Auth is automatically added as admin. + +## Authentication + +Uses [Jakach Login](https://github.com/jakani24/jakach-login) for authentication. Users authenticate via `auth.jakach.ch`. The first user is automatically authorized; subsequent users must have their `user_token` added in Settings > Security. + +## Log Sources (Ingestion) + +### File sources +```bash +curl -X POST http://localhost:8080/sources \ + -H 'Content-Type: application/json' \ + -d '{"name":"nginx-access","type":"file","address":"/var/log/nginx/access.log"}' +``` + +Supports glob patterns: `"/collect/*.log"` + +### TCP/UDP syslog +The worker already listens on `:9514` (TCP and UDP). Add the source via UI or: + +```bash +curl -X POST http://localhost:8080/sources \ + -H 'Content-Type: application/json' \ + -d '{"name":"syslog","type":"tcp","address":"tcp://0.0.0.0:9514"}' +``` + +Other containers can send logs via Docker syslog driver: +```yaml +services: + my-app: + logging: + driver: syslog + options: + syslog-address: "tcp://:9514" + tag: "my-app" +``` + +### HTTP POST +```bash +curl -X POST http://localhost:8080/ingest \ + -H 'Content-Type: application/json' \ + -d '{"line":"2024-01-01 ERROR: something broke","source":"my-app"}' +``` + +### Shared volume +```yaml +services: + my-app: + volumes: + - jakach-logging_log_collect:/collect +``` + +Files written to `/collect/*.log` are automatically picked up by the worker. + +## Alert Rules + +Default rules (created by `bin/seed`): + +| Rule | Pattern | Severity | +|------|---------|----------| +| PHP Fatal Error | `/PHP Fatal/i` | critical_high | +| PHP Exception | `/Uncaught (Exception\|Error)/` | critical | +| PHP Warning | `/PHP (Warning\|Notice)/i` | warning | +| HTTP 5xx | `/" (50[0-9]) /` | critical | +| HTTP 4xx | `/" (4[0-9]{2}) /` | warning_low | +| Failed Login | `/Failed (login\|password\|authentication)/i` | critical_low | +| Out of Memory | `/out of memory\|OutOfMemory/i` | emergency | +| Connection Refused | `/Connection (refused\|reset\|timed? out)/i` | warning_high | +| Disk Space | `/disk (full\|space\|usage\|low)/i` | critical_low | +| Service Started | `/service started\|daemon started\|ready to serve/i` | notice | +| Slow Query | `/slow (query\|request\|response)/i` | warning_high | + +Rules use PHP regex. Add custom rules via UI or API: +```bash +curl -X POST http://localhost:8080/rules \ + -H 'Content-Type: application/json' \ + -d '{"name":"SSH Login","pattern":"/sshd.*Accepted/i","severity":"notice"}' +``` + +Each rule can have a `rate_limit_seconds` to prevent alert storms. + +## Severity Levels + +`debug` → `info` → `notice` → `warning_low` → `warning` → `warning_high` → `error` → `critical_low` → `critical` → `critical_high` → `emergency` + +## Telegram Notifications + +Configure in Settings > Telegram Notifications: +1. Create a bot via [@BotFather](https://t.me/BotFather) on Telegram +2. Get your chat ID (send a message to your bot, visit `https://api.telegram.org/bot/getUpdates`) +3. Enter bot token and chat ID in settings, click Save, then Test + +## Alerts Workflow + +- **Open** — newly triggered, shown on dashboard +- **Acknowledged** — someone has seen it +- **Resolved** — closed, hidden from dashboard counts + +Quick actions in the table: ✓ (acknowledge), ✓✓ (resolve). Click any row for full detail. + +## API Endpoints + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/health` | No | Health check | +| POST | `/ingest` | No | Ingest a log line | +| GET | `/auth/me` | No | Current session user | +| POST | `/auth/logout` | No | Destroy session | +| GET | `/sources` | Yes | List sources | +| POST | `/sources` | Yes | Create source | +| PUT | `/sources/{id}` | Yes | Update source | +| DELETE | `/sources/{id}` | Yes | Delete source | +| GET | `/rules` | Yes | List rules | +| POST | `/rules` | Yes | Create rule | +| PUT | `/rules/{id}` | Yes | Update rule | +| DELETE | `/rules/{id}` | Yes | Delete rule | +| GET | `/alerts` | Yes | List alerts (query: `severity`, `status`, `limit`) | +| GET | `/alerts/search?q=` | Yes | Full-text search alerts | +| GET | `/alerts/counts` | Yes | Alert counts by status/severity | +| POST | `/alerts/{id}/status` | Yes | Update alert status | +| GET | `/logs/search?q=` | Yes | Search raw log entries | +| GET | `/config/allowed_tokens` | Yes | Get allowed user tokens | +| PUT | `/config/allowed_tokens` | Yes | Set allowed user tokens | +| GET | `/config/telegram` | Yes | Get Telegram config | +| PUT | `/config/telegram` | Yes | Set Telegram config | + +## Log Search + +In the Logs tab, search through all ingested log lines. Use `*` as wildcard: +- `error` — matches any line containing "error" +- `error 500` — matches lines containing both "error" AND "500" +- `*` — shows all logs + +## Development + +```bash +# Install PHP deps locally +composer install + +# Run PHP built-in server (no Docker) +composer serve +``` + +## Project Structure + +``` +├── bin/ +│ ├── consume # Worker entrypoint +│ └── seed # Seed default data +├── config/ +│ └── default.php # Default rules config +├── docker/ +│ ├── Dockerfile.api # PHP-FPM image +│ ├── Dockerfile.php # PHP-CLI image +│ ├── entrypoint-api.sh +│ └── nginx.conf +├── docker-compose.yml +├── public/ +│ ├── index.html # SPA frontend +│ ├── index.php # API entrypoint +│ └── oauth.php # Auth callback +└── src/ + ├── Api/ + │ ├── AuthMiddleware.php + │ └── Router.php + ├── Model/ + │ ├── Alert.php + │ ├── AlertSeverity.php + │ ├── AlertStatus.php + │ ├── LogSource.php + │ ├── LogSourceType.php + │ └── Rule.php + ├── Notifier/ + │ └── TelegramNotifier.php + ├── RuleEngine/ + │ └── Engine.php + ├── Storage/ + │ ├── Database.php + │ └── Repository.php + └── Worker/ + ├── FileWatcher.php + ├── Orchestrator.php + └── SocketListener.php +``` \ No newline at end of file