RL Gym Scaffold

Generate, validate, and containerize Gymnasium RL training environments from plain-English task descriptions. Type what you want your agent to learn, get a complete runnable gym with env.py, train.py, requirements.txt, and Dockerfile.

How it works

1. User describes an RL task in plain English
2. LLM (via OpenRouter) generates a complete Gymnasium environment and training script
3. Server validates the generated code structurally (class inheritance, required methods, space definitions)
4. Environment is saved to a public library with a shareable permalink
5. Anyone can browse, fork, and download environments as ZIP archives

API Endpoints

| Method | Path | Description |
|--------|------|-------------|
| GET | /rl-gym-scaffold/health | Health check — returns {"ok":true} |
| POST | /rl-gym-scaffold/api/generate | Generate environment from {"description":"..."} |
| GET | /rl-gym-scaffold/api/environments | List all environments (paginated, 12/page) |
| GET | /rl-gym-scaffold/api/environments/:id | Get single environment with all files |
| GET | /rl-gym-scaffold/api/environments/:id/download | Download environment as ZIP |
| POST | /rl-gym-scaffold/api/environments/:id/fork | Fork an environment for remixing |
| GET | /rl-gym-scaffold/api/packages | Cached PyPI package versions |
| GET | /rl-gym-scaffold/api/trending | Cached trending GitHub RL repos |

Data Sources

| Source | URL | Refresh Interval |
|--------|-----|-----------------|
| PyPI Package API | https://pypi.org/pypi/{package}/json | Every 6 hours |
| GitHub Search API | https://api.github.com/search/repositories?q=gymnasium+reinforcement-learning&sort=stars | Every 24 hours |
| OpenRouter Chat API | https://openrouter.ai/api/v1/chat/completions | On-demand per generation |

### Tracked PyPI packages
gymnasium, stable-baselines3, numpy, torch, shimmy, pygame

Running locally

# Install dependencies
npm install

# Set environment variables
cp .env.example .env
# Edit .env and set OPENROUTER_API_KEY

# Start server
node server.js
# → http://localhost:4746/rl-gym-scaffold/
```

The server fetches PyPI versions and GitHub trending repos on startup. Generation requires a valid OpenRouter API key.

Stack

• Node.js >= 22, Express 5
• better-sqlite3 (WAL mode)
• node-cron for scheduled data fetches
• archiver for ZIP downloads
• Vanilla JS SPA frontend (dark theme, JetBrains Mono)

Environment variables

| Variable | Default | Description |
|----------|---------|-------------|
| PORT | 4746 | Server port |
| NODE_ENV | production | Environment |
| OPENROUTER_API_KEY | — | Required for LLM generation |