scientific-critical-thinking_skill

Overview

This skill evaluates scientific rigor and evidence quality to support reliable conclusions. It inspects methodology, experimental design, statistical validity, bias, confounding, logical fallacies, and overall confidence using frameworks such as GRADE and Cochrane risk-of-bias. The output is a concise, actionable critique and recommendations to improve study credibility.

How this skill works

I systematically inspect study design, measures, analysis, and reporting to identify flaws and strengths. Key steps include assessing internal/external/construct validity, checking randomization and blinding, evaluating statistical methods and power, detecting biases and confounders, and applying evidence-grading frameworks. I summarize risk-of-bias, evidence certainty, and concrete remediation steps.

When to use it

• Assessing a single research paper before citation or policy use
• Conducting systematic reviews or meta-analyses
• Evaluating clinical trial or observational study claims
• Designing or improving a research protocol
• Interpreting high-stakes findings for decision making
• Reviewing statistical analyses or replication attempts

Best practices

• Start with the research question and whether the design can answer it
• Prespecify outcomes and analysis plans; look for registration or protocols
• Prioritize validated, objective measures and standardized procedures
• Check power and sample size calculations before trusting small-sample effects
• Distinguish primary from exploratory analyses and correct for multiple comparisons
• Report effect sizes with confidence intervals and discuss practical relevance

Example use cases

• Critically reviewing a randomized trial before guideline recommendation
• Evaluating whether an observational association could be causal
• Auditing a meta-analysis for publication bias and heterogeneity issues
• Advising on study changes to reduce confounding and improve blinding
• Interpreting conflicting literature and grading overall evidence strength
• Spotting statistical or logical errors in a high-profile claim