debug-logging-assistant_skill

Overview

This skill helps add purposeful debug logging to a Python scraper that extracts emails and LinkedIn links from URLs. It focuses on improving observability without changing program behavior or control flow. The goal is concise, high-signal logs that explain decisions, inputs, and error context for faster troubleshooting.

How this skill works

Inspect the scraper code and recent failures to identify entry points, external requests, parsing branches, and error handlers. Insert debug-level logs before and after risky operations (HTTP calls, HTML parsing, regex extraction) and around branches that affect results. Ensure messages use existing log levels and safe identifiers, and validate there are no functional changes.

When to use it

• When failures are intermittent or spikes in missing extractions appear
• When adding new data sources or crawling patterns and you need visibility
• When debugging parsing edge cases for LinkedIn URL formats or malformed emails
• When optimizing retry/backoff behavior or diagnosing rate-limiting
• During code review to improve maintainability and future debugging

Best practices

• Log concise context: endpoint URL, HTTP status, response size, parsing outcome, and record identifiers (but not raw PII).
• Place logs at high-signal spots: request start/end, parse entry/exit, branch decisions, and exception handlers.
• Use consistent prefixes and levels already present in the codebase (e.g., scraper.debug, extractor.debug).
• Avoid logging full payloads or credentials; redact or summarize sensitive fields (hash counts, domain only).
• Keep the volume reasonable: prefer one clear log per logical block rather than many low-value messages.

Example use cases

• Record an HTTP request start with target URL and user-agent, then a follow-up with status code and response length.
• Log parser branch taken when encountering LinkedIn profile patterns vs search-result patterns, including matched token and outcome count.
• On email extraction failure, log snippet length, regex tried, and sample surrounding text (redacted) to reproduce issues.
• When retrying after a 429/5xx, log retry count, backoff interval, and whether the eventual request succeeded.