gov-sec-baseline

European government website security posture — turned into a rankable leaderboard with shareable report-card images.

What this is

The Internet Cleanup Foundation's SecurityBaseline.eu scans ~200K government and public-sector domains across 32 European countries. The data is published, but only as an interactive map — there is no comparison view, no country ranking, no API surface a journalist or sales rep can hit.

This product re-shapes that public scan output into:

• A country-vs-country leaderboard sortable by composite score, TLS validity, DNSSEC, exposed admin panels, tracking cookies.
• Drill-down pages with all ~20 metrics, ranked weak spots, and the worst-scoring domains per country.
• Type-ahead domain search across every scanned host.
• On-demand live TLS check via Qualys SSL Labs.
• Shareable SVG report cards at /share/:cc.svg (1200×630, Twitter-card optimal) — the artifact people actually post.
• Trending sidebar pulling HN stories about government security.

No accounts, no auth, no rate limit beyond a 1/min cap on the manual rescrape button.

Data sources & refresh intervals

| Source | URL | Refresh | What we use |
|--------|-----|---------|-------------|
| SecurityBaseline.eu (Internet Cleanup Foundation) | https://securitybaseline.eu/data/config/ and https://securitybaseline.eu/data/report/{layer_id}/{YYYY-MM-DD}/ | Daily at 04:00 UTC (node-cron) | Per-country scan report — TLS, DNSSEC, HSTS, tracking, banner exposure, mail TLS/SPF/DKIM/DMARC, etc. |
| Qualys SSL Labs | https://api.ssllabs.com/api/v3/analyze?host={domain} | On user demand only, cached 24h in SQLite | Live TLS grade for a domain typed into the UI |
| HN Algolia | https://hn.algolia.com/api/v1/search_by_date?query=government+security&tags=story&hitsPerPage=30 | Every 30 min (node-cron) | "Trending on HN" sidebar |

No fake data. If a metric isn't in the source, we don't invent it (e.g. IPv6 — SecurityBaseline doesn't publish it via the report endpoint, so it's dropped and its 5% weight is redistributed across the remaining metrics).

The SecurityBaseline scrape walks all 87 country/region layers in parallel (4 workers, with retries on the previous 1–7 days if today's snapshot is HTTP 500). The first boot kicks the scrape off in the background; you'll see countries populate over ~10–30 seconds.

Composite score

Weighted average on 0–100, missing metrics redistribute proportionally:

• 25% Valid TLS certificate (certificate_trust ok %)
• 20% No exposed admin/banner leakage (inverse of bannergrab issues)
• 15% DNSSEC enabled (zonemaster_dnssec ok %)
• 15% No third-party trackers (inverse of web_privacy_tracking issues)
• 10% HSTS present (http_security_header_strict_transport_security)
• 10% Modern TLS configuration (internet_nl_web_tls)
• 5% IPv6 reachable — not in the source, weight is redistributed

Grades: A ≥ 85 · B ≥ 70 · C ≥ 55 · D ≥ 40 · F < 40.

API endpoints

All under /gov-sec-baseline. No auth.

| Method | Path | Returns |
|--------|------|---------|
| GET | /health | {ok:true} |
| GET | /api/status | DB row counts, last-scrape log for each source |
| GET | /api/leaderboard | Ranked array of countries: rank, cc, name, flag, domains, score, grade, four headline metric percentages |
| GET | /api/metrics/list | The metric catalogue (key, label, weight, direction) |
| GET | /api/country/:cc | Country detail: all metrics, rank, worst-scoring 25 domains |
| GET | /api/domain/search?q= | Up to 20 matching domain hosts |
| GET | /api/domain/:host | Single domain's per-flag breakdown |
| GET | /api/tls/check?domain= | On-demand Qualys SSL Labs result (cached 24h) |
| GET | /api/trending | Last ~20 HN stories matching "government security" |
| POST | /api/admin/rescrape | Kicks an out-of-band re-scrape. Rate-limited to 1/min. Public. |
| GET | /api/admin/rescrape/status | Whether a scrape is in flight |
| GET | /share/:cc.svg | 1200×630 SVG report card for the country |
| GET | /country/:cc | HTML wrapper with OG meta pointing at /share/:cc.svg (so Twitter/HN unfurls render) |
| GET | / | SPA |

Running locally

node --version   # requires >= 22
npm install
PORT=4762 node server.js

Then open <http://localhost:4762/gov-sec-baseline/>.

On the very first launch the database is empty — the server kicks off an initial scrape of SecurityBaseline.eu and HN in the background. Reload after ~15s and the leaderboard populates. If you'd rather wait for it before showing the UI, run npm run scrape first.

Environment variables (all optional):

• PORT — defaults to 4762
• DB_PATH — override the SQLite file (default: ./data.db)
• NODE_ENV

No secrets are required. OPENROUTER_API_KEY / BRAVE_API_KEY appear in .env.example purely because the deployment vault expects every product manifest to advertise them — they are not used.

Out of scope

• No port scans of our own (legal minefield — Qualys SSL Labs is the only on-demand external call and it's their service).
• No accounts, no email, no historical time-series — single current snapshot per country.
• No non-European governments.
• No PDF/CSV export in v1.

Architecture notes

• Stack: Node ≥ 22, Express, better-sqlite3 (WAL), node-cron, helmet, compression. Vanilla JS SPA, no build step.
• SQLite is local-only; the schema lives in db.js.
• All scrapers write a row in scrape_log so the UI can show "last scrape: 3h ago — partial".
• /api/admin/rescrape is intentionally public (the SPEC explicitly disallows auth) but is rate-limited to 1 call/min via an in-memory token.

Honest known limitations

• SecurityBaseline.eu's /data/report/{id}/{date}/ endpoint returns HTTP 500 for some layer IDs on some days. We retry the prior 1, 2, 7 days; if all four dates 500 for every layer of a country, that country is skipped on this run and its previous snapshot is preserved.
• Country domain counts vary wildly because the source dataset itself does — NL covers 53 sub-domains (Brabant + Groningen province only), LT covers 131, LI covers 1. We surface the source numbers honestly rather than padding.
• The Qualys SSL Labs scan for a domain it's never seen can take 30–90 seconds the first time; subsequent calls within 24h return from our local cache.

Data: securitybaseline.eu · TLS audit: Qualys SSL Labs · Trending: HN Algolia.