karpathy-score

Lint and rank any public repo's CLAUDE.md / AGENTS.md / SKILL.md against the Karpathy 4-rules manifesto plus a community best-practices rubric. Live GitHub data, no seeds, no auth.

URL once deployed: <https://holyai.me/karpathy-score/>

Why

In May 2026 the single hottest open-source artifact is the project-level agent instruction file. forrestchang/andrej-karpathy-skills (the 70-line distillation of Karpathy's four observations) crossed 109k stars in three months and topped GitHub Trending for 28 consecutive days.

Hundreds of thousands of repos now ship a CLAUDE.md or AGENTS.md. None of them are linted. Most are short, copy-pasted from Karpathy verbatim, or missing the load-bearing rules entirely. karpathy-score is the missing linter and the missing leaderboard.

What it does

• Crawls public GitHub for CLAUDE.md, AGENTS.md, and SKILL.md files in repos with ≥50 stars (≥20 for SKILL.md).
• Fetches the raw content from raw.githubusercontent.com.
• Scores each file 0–100 across ten axes (Plan-Before-Act, Surgical-Edits, No-Bloat, Success-Criteria, Tool/Env Specificity, Anti-Hallucination, Failure-Mode Awareness, Structure & Readability, Project-Specific, Freshness).
• Publishes a live leaderboard with filters (kind, language, score floor).
• Live linter: paste any owner/name and get a score on demand.
• Shareable SVG result card at /karpathy-score/og/<owner>/<name>.svg (1200×630).
• Distribution histogram, per-kind averages, top files per rule, weekly biggest movers.

Stack

| Layer | Pick |
|---|---|
| Runtime | Node.js ≥ 18, Express 4 |
| Storage | better-sqlite3 (WAL mode) |
| Scheduler | node-cron |
| Hardening | helmet, compression, morgan |
| UI | Vanilla JS SPA, dark theme, no framework |

Mounted under BASE_PATH = /karpathy-score. Health probe at /karpathy-score/health. No auth anywhere — every endpoint (read + write/refresh) is public.

Endpoints

GET  /karpathy-score/                          # SPA shell
GET  /karpathy-score/health                    # liveness JSON
GET  /karpathy-score/api/leaderboard           # top scored files
GET  /karpathy-score/api/repo/:owner/:name     # full score detail + hit snippets
POST /karpathy-score/api/score                 # body: {repo:"owner/name"} live lint
GET  /karpathy-score/api/stats                 # distribution + per-kind + top-by-rule
GET  /karpathy-score/api/recent                # newest scored files
GET  /karpathy-score/api/movers?window=7       # biggest score deltas over N days
POST /karpathy-score/api/refresh?job=harvest   # force a cron tick (1/min per IP)
GET  /karpathy-score/og/:owner/:name.svg       # share-card (1200x630 SVG)

Cron schedule (real, no mocks)

| Job | Cron | What |
|---|---|---|
| harvest | 0 /6 | GitHub code-search for each filename, upsert candidates |
| score-batch | /15 | Fetch + score the next 20 unscored/stale files |
| refresh-meta | 30 /6 | Re-pull repo metadata (stars, pushed_at) |
| rubric-sync | 0 4 | Re-pull the canonical Karpathy CLAUDE.md so the rubric source is transparent |

Data sources

• https://api.github.com/search/code — filtered by filename:CLAUDE.md stars:>=50 etc. Uses GITHUB_TOKEN if available (5000 req/h authenticated, 60 unauthenticated).
• https://raw.githubusercontent.com/<owner>/<repo>/<branch>/<path> — raw file content. No auth required.
• https://api.github.com/repos/<owner>/<name> — stars, pushed_at, language, default branch.
• https://api.github.com/repos/<owner>/<name>/git/trees/<branch>?recursive=1 — repo tree for project-specific cross-reference scoring.

No mock data. No seed rows. If a fetch fails, the row stays unscored and the UI hides it honestly.

Run locally

npm install
cp .env.example .env       # then add a real GITHUB_TOKEN for higher rate limits
npm start
open http://localhost:4823/karpathy-score/

To run without scheduled tasks (single-shot harvest then exit):

DISABLE_CRONS=1 npm start &
node -e "require('./fetcher').harvestAll().then(()=>require('./fetcher').scoreBatch(20))"

Env

| Var | Required | Purpose |
|---|---|---|
| PORT | yes | Listen port (default 4823) |
| NODE_ENV | no | production for combined logs |
| GITHUB_TOKEN | optional | Higher rate limits. In production injected from the RNDLAB vault. |
| LOG_LEVEL | no | reserved |
| DB_PATH | no | sqlite path (default ./data/karpathy.db) |
| DISABLE_CRONS | no | set to 1 to skip the scheduler |

License

MIT. Author: Cowork (Claude Opus 4.7).