papers-pulse
Live pulse of AI research — HuggingFace Daily Papers + arXiv firehose, refreshed every 15 minutes from public unauthenticated APIs.
What it is
A self-hosted dashboard that surfaces what AI research the community is actually reading right now.
Two real, public, unauthenticated data sources are polled on a cron:
| Source | URL | Refresh |
|---|---|---|
| HuggingFace Daily Papers | https://huggingface.co/api/daily_papers?date=YYYY-MM-DD | every 15 min for today; every 6h for last 7 days (backfill) |
| HuggingFace paper detail | https://huggingface.co/api/papers/{arxivId} | every 6h for top-60 papers (snapshot deltas → Climbers) |
| arXiv firehose | http://export.arxiv.org/api/query?search_query=cat:cs.AI+OR+cat:cs.CL+OR+cat:cs.LG&sortBy=submittedDate&sortOrder=descending&max_results=200 | every 30 min |
Every metric on the page comes from one of those calls. No mock data. No seed arrays. No Math.random jitter. No "presets". If the upstream API returns nothing, the dashboard shows nothing.
Views
- Today's pulse — top 30 papers from HuggingFace's daily list, ranked by upvotes. Hero card for #1, grid for the rest.
- Recent — sortable table of the last 7/14/30 days, top by upvotes.
- Climbers — papers with the biggest upvote delta in a 6h / 24h / 3d / 7d window. Computed from the snapshot table — earliest meaningful results appear after a few cron cycles.
- arXiv firehose — newest 100 cs.AI / cs.CL / cs.LG submissions.
- Keywords — chip cloud of HF-generated
ai_keywords, weighted by upvotes. - Orgs — top labs / companies / universities by 30-day upvote sum.
- Authors — top researchers by 30-day upvote sum (nightly rollup).
- Paper detail modal — title, full abstract, AI summary, keywords, GitHub repo + stars, 24h upvote sparkline (Chart.js), arXiv & HuggingFace direct links.
Stack
- Node.js 20+ / Express 5
- better-sqlite3 (WAL mode)
- node-cron
- helmet + compression + dotenv
- Vanilla JS SPA (no React, no bundler)
- Chart.js via CDN (sparklines only)
No build step. No auth — every endpoint is public. The dashboard is meant to be inspected directly.
Run locally
npm install
cp .env.example .env
node server.js
# → http://localhost:4757/papers-pulse/
On first boot, the cron bootstrap backfills the last 7 days of HF daily and one round of the arXiv firehose. The DB lives at ./data/papers-pulse.db by default.
Endpoints
All mounted under /papers-pulse/. All public, no auth.
| Method | Path | Notes |
|---|---|---|
| GET | /health | service info, paper counts, last fetch — auth-free, returns 200 |
| GET | /api/today?date=YYYY-MM-DD&limit=30 | top papers on a given HF daily |
| GET | /api/recent?days=7&category=cs.AI&limit=50 | top papers in a window |
| GET | /api/climbers?window=24h&limit=20 | biggest upvote deltas |
| GET | /api/firehose?category=cs.CL&limit=100 | newest arXiv submissions |
| GET | /api/keywords/top?days=7&limit=30 | weighted ai_keywords |
| GET | /api/orgs/top?days=30&limit=20 | org leaderboard |
| GET | /api/authors/top?limit=30 | author leaderboard |
| GET | /api/paper/:arxivId | full detail + sparkline + daily listings |
| GET | /api/search?q=...&limit=20 | title/abstract/keyword search |
| GET | /api/stats | sitewide counters |
| GET | / | the SPA |
Why this exists
arXiv's cs.AI category alone shows ~178k total submissions and 400–600 new papers per day in May 2026. No human keeps up. HuggingFace Papers has become the de-facto community signal for "what mattered today" — but nobody surfaces its history, its climbers, its top orgs or its overlap with the raw arXiv firehose in one place.
papers-pulse does. It's the dashboard you bookmark.
License
MIT