profiling-tables_skill
- Python
251
GitHub Stars
1
Bundled Files
2 months ago
Catalog Refreshed
4 months ago
First Indexed
Readme & install
Copy the install command, review bundled files from the catalogue, and read any extended description pulled from the listing source.
Installation
Preview and clipboard use veilstrat where the catalogue uses aiagentskills.
npx veilstrat add skill astronomer/agents --skill profiling-tables- SKILL.md3.8 KB
Overview
This skill provides a deep data profile for a specific table to help new team members and analysts understand structure, content, quality, and key statistics. It produces metadata, column-level stats, cardinality insights, representative samples, and a concise data-quality assessment. Use it when you need a quick, actionable snapshot of any table.
How this skill works
Given a table name, the skill queries information_schema for column metadata, runs row-count and column-specific aggregations (numeric, string, timestamp) and computes cardinality and frequency distributions. It retrieves representative samples and evaluates completeness, uniqueness, freshness, validity, and consistency to produce a structured summary and recommended follow-up queries. Queries are adapted to the table's data types and size.
When to use it
- Onboarding new analysts who need to understand a table quickly
- Investigating data quality before building reports or models
- Assessing freshness and completeness after an ingestion or ETL change
- Profiling large tables to guide sampling and downstream transforms
- Validating suspected anomalies or value distributions in production data
Best practices
- Provide a fully qualified table name (database.schema.table) when possible
- Limit heavy aggregations on very large tables; use sampled queries for exploratory runs
- Run cardinality analysis on suspected categorical columns rather than every column
- Compare timestamp columns to known ingestion times to check freshness
- Store and reuse computed statistics to avoid repeated full-table scans
Example use cases
- Generate a one-page profile during sprint handoff so engineers know table shape and expected ranges
- Detect columns with unexpectedly high null rates before joining tables in a pipeline
- Identify skewed keys that cause hotspotting in downstream aggregation jobs
- Confirm that a recent schema change didn’t introduce invalid formats or duplicates
- Produce recommended investigative queries when a dashboard metric diverges
FAQ
A table name; a fully qualified name (database.schema.table) is preferred. If not fully qualified, the skill will search information_schema to locate it.
How does it handle very large tables?
It uses targeted aggregations and recommends sampling for exploratory profiling. For full scans, run during off-peak windows or persist results for reuse.
