literature-engineer_skill

Overview

This skill builds a large, verifiable candidate pool of literature for evidence-first surveys. It expands multi-route recalls (offline imports, arXiv/semantic routes, and snowballing), normalizes metadata, and emits a stable, provenance-rich papers/papers_raw.jsonl ready for dedupe, ranking, and citation generation. The skill blocks rather than fabricates when the target pool or stable identifiers cannot be reached.

How this skill works

The skill ingests offline exports and optional snowball files, labels provenance per input, and merges records. If network is enabled it performs arXiv and Semantic Scholar-style retrievals and online snowballing. It canonicalizes identifiers (arXiv id, DOI, canonical URL), unions provenance across duplicates, and writes papers/papers_raw.jsonl, a human CSV, and a concise retrieval report with route counts and missing-meta stats.

When to use it

• You need to expand candidate literature to at least 1,200 records before mapping or writing.
• Stage C1 of an evidence-first survey pipeline: prepare upstream evidence and citations prior to drafting.
• You have partial offline exports and want to merge plus backfill stable IDs/provenance.
• You plan to run dedupe/rank, citation generation, or mapping and need standardized input.
• You want an auditable retrieval report showing route coverage and missing metadata.

Best practices

• Always aim for >=1200 verifiable records; do not accept fabricated entries under any circumstance.
• Provide as many offline exports as available under papers/imports/ and papers/snowball/ before online runs.
• Enable --online and --snowball when possible to backfill IDs and expand via cited-by/reference graphs.
• Review retrieval_report.md for coverage buckets and missing-meta counts; add more exports or enable network if targets aren’t met.
• Prefer exports that include arXiv IDs or DOIs (arXiv/OpenReview/ACL) to minimize missing stable identifiers.

Example use cases

• Merge multiple conference and arXiv exports into a single provenance-rich candidate pool for deduping and ranking.
• Run an offline-first pipeline in a restricted environment, then enable --online to backfill IDs and run snowballing when network is available.
• Seed a large survey with pinned canonical IDs (classic papers) plus keyword retrieval to ensure must-cite anchors.
• Produce a papers/papers_raw.jsonl that downstream citation generators and mapping tools can consume without additional normalization.