reforger_crawler_main/docs/admin-dashboard.md

# Admin dashboard, metrics and telemetry

The API binary serves an admin dashboard at **`/admin/ui/`** (`/admin` redirects there).
It is a React app in `web/`, built into `web/dist` and embedded with `go:embed`.

## Access

- Sign in with `ADMIN_SECRET`. The server sets an HttpOnly, `SameSite=Strict`
  session cookie (`ADMIN_SESSION_TTL_HOURS`, default 7 days).
- Rotating `ADMIN_SECRET` signs every session out.
- If `ADMIN_SECRET` is empty, the dashboard and admin API are disabled.
- Set `TRUSTED_PROXIES` behind Traefik/Dokploy. Without it, client IPs can be
  spoofed via `X-Forwarded-For`, which affects visitor counts and login throttling.

## Admin API

The JSON API lives under `/admin/api`. Scripts can call it with the
`X-ADMIN-KEY: <ADMIN_SECRET>` header instead of a cookie. Cookie requests that
change data must also send `X-Admin-CSRF: 1`.

| Area | Endpoints |
|---|---|
| Stats | `GET overview`, `GET overview/duplicates` (job state + top shared hashes), `POST overview/duplicates/refresh`, `GET traffic?from&to`, `GET top?kind=addon\|search&days` |
| Queue | `GET queue`, `POST queue/reset-stuck` |
| Addons | `GET addons?q&status&type&sort&order&limit&offset`, `GET addons/:id`, `GET addons/:id/files` |
| Addon actions | `POST addons/:id/{priority\|to-be-indexed\|reindex\|reset-indexing\|recheck\|restore}` with body `{"value": bool}`, `DELETE addons/:id`, `DELETE addons/:id/purge?confirm=<id>`, `POST bulk/addons` |
| Whitelist | `GET whitelist`, `POST whitelist` with body `{"hashes":[], "text":"", "note":""}`, `DELETE whitelist/:id`, `GET hashes/:hash` |
| Metrics | `GET metrics/catalog`, `GET metrics/query?service&name&agg&groupBy&match&from&to&step` |
| Services | `GET services`, `GET services/events`, `DELETE services/:id` |
| System | `GET system`, `POST system/maintenance` with body `{"action": "refresh-stats\|flush-metrics\|rollup\|purge-deleted-files"}` |

The legacy endpoint `POST /admin/addWhitelistedHash` (form field `hash`, `X-ADMIN-KEY`) still works.

## Metrics

- Every request is timed. Latency is recorded per route as a histogram, plus
  status class, caller group and web vs direct client.
- The handlers also emit metrics for indexer claims and submissions, job
  duration, scraper creates/updates/deletes, Discord webhooks and admin actions.
- Every minute, gauges sample the queue depth and total addon count.

Storage:

- Data is aggregated in memory per minute and flushed to the database every
  minute (`metric_series` and `metric_points` tables).
- Minute data is rolled up hourly.
- Retention is configurable: `METRICS_MINUTE_RETENTION_DAYS` (default 7) and
  `METRICS_HOUR_RETENTION_DAYS` (default 180).

Website visitors are counted as a salted hash of IP and user agent. The salt
rotates daily and only lives in memory, so the count is approximate. The most
looked-up addons and search queries are stored per day in `daily_top_items`.

Prometheus:

- Set `METRICS_TOKEN` to expose `GET /metrics` in the Prometheus text format.
- Authenticate with `Authorization: Bearer <token>`.
- This lets you add an external Prometheus/Grafana later.

## Telemetry from indexers and scrapers

Services report to `POST /back/telemetry` with `X-SECRET-KEY`. Every push is
also a heartbeat. An instance is shown as offline after three missed
`heartbeatIntervalSec`.

```json
{
  "service": "indexer", "instance": "indexer-1", "version": "1.4.0",
  "startedAt": 1727950000, "heartbeatIntervalSec": 15, "status": "ok",
  "metrics": [
    {"name": "jobs_total", "type": "counter", "value": 3, "labels": {"result": "ok"}},
    {"name": "data_window_bytes", "type": "gauge", "value": 7.1e9},
    {"name": "download_seconds", "type": "summary", "count": 3, "sum": 41.2, "min": 8.1, "max": 20.3}
  ],
  "events": [{"level": "warn", "message": "data limit reached", "fields": {"limitGB": 50}}]
}
```

Metric types:

- `counter`: the increase since the last push.
- `gauge`: the current value.
- `summary`: pre-aggregated values from the client.

Limits:

- Names must match `^[a-z_][a-z0-9_]{0,63}$`.
- At most 8 labels per metric.
- At most 1000 metrics and 200 events per push.
- At most 200 series per metric and 5000 external series in total. Anything
  over these limits is dropped and counted.

A stdlib-only Go client is in `telemetry/`, as its own module:

```go
import "gitea.tbdevent.eu/TBD/reforger_crawler_main/telemetry"

c := telemetry.New(telemetry.Config{BaseURL: apiURL, SecretKey: secret, Service: "indexer", Version: version})
c.Start(ctx)
defer c.Close(context.Background())

c.Counter("addons_indexed_total", "result", "ok").Inc()
c.Summary("download_seconds").Observe(took.Seconds())
c.Gauge("data_window_bytes").Set(float64(used))
c.Event(telemetry.Warn, "data limit reached", nil)
```

Reported metrics show up in the dashboard under **Services** and in the **Metrics explorer**.

## Development

```sh
# API on SQLite (overrides config.yaml)
DB_DRIVER=sqlite SQLITE_PATH=dev.db SECRET_KEY=dev ADMIN_SECRET=devadminsecret1234 go run .

# UI with hot reload, proxied to :8083
npm --prefix web install
npm --prefix web run dev     # http://localhost:5173/admin/ui/

# production build (needed before `go build` to embed the UI)
npm --prefix web run build
```

If the UI was not built, the binary still compiles and serves a placeholder page.