assafelovic/gpt-researcher
Overview
This skill describes GPT Researcher, an autonomous deep-research agent implemented in Python that orchestrates planning, retrieval, and report generation. It is designed to run locally or as a service, parallelize sub-tasks for speed, and produce citation-backed Markdown reports suitable for developer workflows and integrations.
How this skill works
GPT Researcher instantiates GPTResearcher which selects an agent type and role prompt, then the ResearchConductor plans sub-queries and parallelizes processing of each sub-query using retrievers and LLM providers. Contexts are aggregated, optional images are generated, and the ReportGenerator composes a final Markdown report with citations and structured sections.
When to use it
- Adding a new feature, provider, or retriever to the agent
- Integrating MCP data sources like GitHub via the MCP server
- Customizing research workflows or report formats
- Troubleshooting research pipelines, async issues, or WebSocket streaming
- Running deep recursive research (deep mode) for comprehensive analysis
Best practices
- Keep configuration keys lowercased when accessed (config.my_var) to avoid bugs
- Register new retrievers in actions/retriever.py and export them from retrievers/__init__.py
- Implement graceful degradation: catch exceptions and stream errors instead of crashing
- Use async/await everywhere for research methods and retriever search functions
- Use WebSocketHandler or similar to stream progress and logs during long runs
Example use cases
- Extend the system with a new LLM provider: add provider folder, update prompts, register in agent
- Create a custom retriever for an internal knowledge base and register it to be used by the conductor
- Integrate a GitHub MCP connector to surface repo-level context for a security audit report
- Run deep research on a technical topic to generate a recursive, citation-rich deep report
- Embed the agent in a backend service and stream intermediate outputs to a frontend via WebSocket
FAQ
Create the retriever class under retrievers/your_retriever, implement an async search() returning title/href/body items, register it in actions/retriever.py, and export it in retrievers/__init__.py.
Where are prompts and defaults configured?
Prompts live in prompts.py and default configuration values are in config/variables/default.py; note config keys are accessed lowercased in runtime.
How do I enable MCP data sources?
Pass mcp_configs and select mcp_strategy when constructing GPTResearcher; ensure the MCP server command and environment variables are correctly set for each connector.
