Dev Pain Index
A real-time developer frustration leaderboard that scrapes Hacker News and Dev.to to surface which tools, languages, and platforms are causing the most pain right now. Built for DevRel leads and product managers who need to spot brewing frustrations before they become PR crises.
What it does
- Pain Leaderboard: Ranked list of developer tools mentioned negatively, with pain scores, mention counts, category badges, and 7-day sparkline trends
- Category View: Filter by frustration category (Tooling, Language, Platform, Process, AI) with aggregate scores and trend charts
- Tool Detail: Click any tool to see its 7-day pain timeline, recent negative mentions with source links
- Weekly Pain Report: Shareable card showing top 5 frustrations with week-over-week deltas
Data Sources
| Source | URL | Refresh Interval | What's Extracted |
|--------|-----|-------------------|------------------|
| HN Algolia Stories | https://hn.algolia.com/api/v1/search?tags=story | Every 10 minutes | Stories with frustration keywords in title. Rotates through 3 keywords per cycle. |
| HN Algolia Comments | https://hn.algolia.com/api/v1/search_by_date?tags=comment | Every 10 minutes | Recent comments mentioning developer tools in frustration contexts. Rotates through 4 keywords per cycle. |
| Dev.to Articles | https://dev.to/api/articles?tag={tag}&top=7 | Every hour | Articles from discuss, watercooler, healthydebate, javascript, webdev, python, devops, programming tags. |
All APIs are free and require no API keys.
Scoring Formula
mention_pain_score = keyword_intensity * (1 + log2(1 + source_points)) * recency_decay
keyword_intensity: count of frustration keywords (capped at 5). Strong keywords ("nightmare", "unusable") weighted 2x.source_points: HN points or Dev.to reactions.recency_decay: 1.0 (last 24h), 0.7 (1-3 days), 0.4 (3-7 days).
API Endpoints
| Method | Path | Description |
|--------|------|-------------|
| GET | /dev-pain-index/health | Health check with last scrape time and mention count |
| GET | /dev-pain-index/api/stats | Dashboard stats: mentions today, tools tracked, most painful category, scraper statuses |
| GET | /dev-pain-index/api/leaderboard?category=&period=24h\|7d | Ranked tools with pain scores, sparklines. Optional category filter. |
| GET | /dev-pain-index/api/categories | All 5 categories with aggregate scores and 7-day trend arrays |
| GET | /dev-pain-index/api/tools/:slug | Tool detail: timeline, recent mentions, rank |
| GET | /dev-pain-index/api/mentions?tool=&limit=20&offset=0 | Paginated raw mentions for a tool |
| GET | /dev-pain-index/api/report/weekly | Weekly report: top 5 tools with deltas, category breakdown |
Running Locally
npm install
PORT=4744 node server.js
Open http://localhost:4744/dev-pain-index/ in your browser. Scrapers run immediately on startup and then on their scheduled intervals.
Tech Stack
- Node.js >= 22, Express, better-sqlite3 (WAL mode), node-cron
- Vanilla JS SPA with Chart.js for visualizations
- Dark theme UI with Inter + JetBrains Mono fonts
- No authentication required, all endpoints public
Classification
Tools are classified into 5 categories based on a dictionary of ~100 known developer tools:
- tooling: build tools, editors, CI/CD, package managers
- language: programming languages, frameworks
- platform: cloud providers, hosting, databases
- process: project management, workflows
- ai: AI tools, copilots, LLMs