topvibecoder/eval

Overview

This skill performs a fast, parallel codebase evaluation using the Gemini CLI and a set of structured metrics. It scans a repository, runs multiple metric evaluations concurrently, aggregates results into a human-readable report, and submits scores to a ranking API to track progress over time.

How this skill works

The tool first scans the codebase and extracts a repo tree and source files with line numbers while skipping common noise folders. It then runs parallel Gemini evaluations for each selected metric, aggregates JSON outputs into a final markdown report, and visualizes scores with a terminal bar chart. After aggregation it can submit results to the TopVibeCoder ranking API and append ranking history for longitudinal tracking.

When to use it

• Quickly benchmark a project before demos or PR reviews
• Track improvements across iterations by saving ranking history
• Compare similar projects by submitting to a public ranking API
• Automate periodic quality checks in CI for early feedback
• Generate a concise report for stakeholders or onboarding

Best practices

• Exclude heavy or irrelevant directories (node_modules, .git, .gemini) to speed scans
• Limit included characters with EVAL_MAX_CHARS for very large repos
• Run evaluations in parallel but cap EVAL_PARALLEL to avoid resource contention
• Use metadata.json to provide accurate name/description for better ranking context
• Add .evals/ to .gitignore to avoid committing evaluation artifacts

Example use cases

• Run a one-minute evaluation before a demo to surface major UX or technical issues
• Add as a pre-merge check to catch regressions in prompt design or integration
• Measure impact and creativity of different prototype branches
• Generate a report to include in a project README or developer handoff
• Track product improvements month-over-month using ranking_history.jsonl