← back to gallery

Patch Clock

How fast does each AI agent framework patch its CVEs? Live MTTP leaderboard.

aisecurityai-agentscvemttpghsaleaderboardlive-data
Open product ↗

patch-clock

When a CVE drops on your AI agent framework, how long do you stay exposed?

patch-clock is a live, public scoreboard that ranks AI agent frameworks (LangChain, Semantic Kernel, MCPJam Inspector, CrewAI, AutoGen, LlamaIndex, MCP SDKs, Anthropic + OpenAI SDKs, Strands, Flowise, Dify, Langflow, Haystack, Agno, Pydantic AI, AG2, Open WebUI, LiteLLM, vLLM, …) by how fast they ship a patched release after a security advisory is filed against them.

The single number on the page is MTTP — mean time to patch:

patch_lag_hours = first_patched_release_published_at − advisory_published_at

All numbers come from public, unauthenticated APIs. No mocks. No seeds. No auth.

---

Why this exists

The AI-agent-framework attack surface has gone vertical in 2026:

When a CVE drops on an agent framework the only number that matters to defenders is how long until a patched release ships. That number is not published anywhere — until now.

How it works

  1. Scan. Every 6h, for each tracked (ecosystem, package_name), we hit GitHub's global Security Advisories endpoint and upsert any new advisories. Every 4h we cross-check OSV.dev to catch entries GHSA misses.
  2. Refresh releases. Every 12h, for each tracked GitHub repo, we cache releases + tags + commit dates so we can convert a "patched version" string into a wall-clock timestamp.
  3. Resolve patch lag. Every hour, for each advisory whose patched_versions string is parseable, we look up the publish time of that version (Releases → Tags → npm registry → PyPI), compute patch_lag_hours, and flip the advisory's status to patched.
  4. Aggregate. Daily rollup computes median + mean lag per framework per day.
  5. Render. Single-page vanilla-JS dashboard reads from /patch-clock/api/*.

If an advisory's patched_versions is empty or unresolvable, it stays on the unconfirmed wall — we never invent a date. If a patched release predates the advisory (stealth-fix), we flag pre_disclosure_fix and exclude it from MTTP averages.

Data sources

| Source | URL | Frequency | What we read |
|---|---|---|---|
| GitHub Security Advisories | https://api.github.com/advisories?ecosystem={eco}&affects={pkg}&per_page=100&sort=published&direction=desc | 6h per package | ghsa_id, cve_id, published_at, withdrawn_at, severity, cvss.score, vulnerabilities[].patched_versions |
| OSV.dev | POST https://api.osv.dev/v1/query | 4h per package | aliases, affected[].ranges[].events[].fixed |
| GitHub Releases | https://api.github.com/repos/{owner}/{repo}/releases?per_page=100 | 12h per repo | tag_name, published_at |
| GitHub Tags + Commits | /repos/{owner}/{repo}/tags + /commits/{sha} | on demand | commit committer.date |
| npm registry | https://registry.npmjs.org/{pkg} | on demand | time["X.Y.Z"] |
| PyPI JSON | https://pypi.org/pypi/{pkg}/json | on demand | releases[version][0].upload_time_iso_8601 |

Unauthenticated rate limits: GitHub allows 60 req/h without a token; with GITHUB_TOKEN set in env, that becomes 5000 req/h. Both modes are supported. We self-throttle per-host with a tiny rate limiter to stay polite.

Stack

No basic auth. No API keys. All endpoints (including POST /api/refresh) are public. The refresh endpoint is rate-limited to 1 call per 5 minutes per process.

Endpoints

GET  /patch-clock/                        Dashboard SPA
GET  /patch-clock/health                  Liveness probe
GET  /patch-clock/sources                 Transparency page
GET  /patch-clock/api/stats               Top-line stats
GET  /patch-clock/api/leaderboard?window=90d|365d|all
GET  /patch-clock/api/frameworks          Same shape as leaderboard, 365d window
GET  /patch-clock/api/frameworks/:slug    Framework deep-dive
GET  /patch-clock/api/advisories?status=&framework=&severity=&q=&limit=
GET  /patch-clock/api/advisories/:ghsa_id Single advisory
GET  /patch-clock/api/unpatched           Currently-unpatched advisories
GET  /patch-clock/api/recent              Recently-resolved patches
GET  /patch-clock/api/trend?days=90&framework=
POST /patch-clock/api/refresh             Trigger out-of-band scan (1/5min)
POST /patch-clock/api/recompute           Re-derive patch lag for any newly-parseable advisory
GET  /patch-clock/api/card/:slug.svg      1200×630 framework card
GET  /patch-clock/api/badge/:slug.svg     shields.io-style badge

Running

npm install
PORT=4835 node server.js
# open http://localhost:4835/patch-clock/

Optional env: GITHUB_TOKEN (lifts rate limit), MAX_PACKAGES_PER_CYCLE (default 40), ADVISORY_BACKFILL_DAYS (default 365).

The first boot kicks off an initial scan in the background. Expect the leaderboard to populate within a few minutes.

What MTTP means

For each advisory that we can resolve a patched release date for:

Limitations

How to contribute a framework

Open a PR adding an entry to lib/tracked.js:

{
  slug: 'your-framework', name: 'Your Framework', vendor: 'Vendor Inc', repo: 'vendor/repo',
  packages: [
    { eco: 'pip', name: 'your-framework' },
    { eco: 'npm', name: '@vendor/framework' },
  ],
}

On next start the bootstrap will idempotently insert the new (ecosystem, package_name) rows into tracked_packages and the GHSA scan will pick them up.

License

MIT.