data-wrangler_skill

Overview

This skill provides production-grade tabular data manipulation powered by pandas and openpyxl. It exposes CLI scripts for 18 common operations plus an Excel toolkit for advanced workbook creation and formatting. Use it to inspect, clean, transform, validate, merge, pivot, and convert CSV, Excel, JSON, Parquet, and TSV files with predictable, scriptable commands.

How this skill works

Two Python CLI scripts drive the functionality: data_wrangler.py for general data operations and excel_toolkit.py for Excel-specific workflows. data_wrangler performs inspect, filter, sort, group, merge, pivot, dedupe, derive, validate, convert and other operations using pandas. excel_toolkit leverages openpyxl to create multi-sheet workbooks, apply header styles, freeze panes, set data validation/dropdowns, autosize columns, and export sheets to CSV. Both scripts accept input/output paths and options for reproducible pipelines.

When to use it

• Cleaning, filtering, sorting, aggregating, or profiling tabular files (CSV/TSV/JSON/Parquet/XLSX).
• Merging or joining multiple datasets and deduplicating records before analysis.
• Creating formatted Excel workbooks with multi-sheet layouts, freeze panes, autofilter, and validation lists.
• Converting between formats (CSV ⇄ XLSX, JSON ⇄ Parquet) in an automated pipeline.
• Adding computed columns, casting types, running SQL-style queries, or applying validation rules.
• Pivoting/unpivoting, sampling, splitting datasets by value, or exporting subsets for downstream tools.

Best practices

• Install optional dependencies (pyarrow, xlrd, fastparquet) only when you need the related formats to avoid bloat.
• Test transformations on a sample subset using sample or inspect before applying to full dataset.
• Encode validation rules as JSON to keep checks reproducible and versionable across runs.
• Pin pandas and openpyxl versions in production environments to ensure consistent behavior.
• Use the CLI options for --sheet, --nrows, --where, and --seed to make scripts deterministic and scriptable.

Example use cases

• Filter a customer CSV to active users and export a deduplicated Excel report with formatted headers.
• Merge sales and customer datasets, compute profit columns, group by region, and save a Parquet summary.
• Convert multiple CSVs into a multi-sheet XLSX report with autofilter, freeze panes, and dropdown status lists.
• Validate incoming data via a JSON rules file to produce a validation report before loading into a database.
• Pivot monthly sales into a wide table for dashboard ingestion and convert the result to JSON for an API.