azure-pipelines-validator_skill

Overview

This skill is a comprehensive toolkit for validating, linting, testing, and securing Azure DevOps pipeline YAML files (azure-pipelines.yml / azure-pipelines.yaml). It runs layered checks—YAML linting, syntax/schema validation, best-practices guidance, and security scanning—to produce actionable line-numbered feedback. The tool is designed for CI integration and local audits with minimal setup via an automatic virtual environment for dependencies.

How this skill works

The validator inspects pipeline YAML structure and content, validating YAML syntax and Azure Pipelines schema rules, then applies best-practice heuristics and security patterns. It reports problems with line numbers, severity (info/warning/medium/high), and remediation suggestions. You can run full validation or pick individual stages (syntax, best practices, security) via command-line flags or call the individual Python scripts directly.

When to use it

• Validating new or changed azure-pipelines.yml files before committing or merging
• Running security audits to detect hardcoded secrets, insecure scripts, or :latest container tags
• Enforcing CI/CD best practices: task version pinning, timeouts, caching, and template usage
• Debugging pipeline failures caused by invalid hierarchy, missing fields, or malformed tasks
• Optimizing pipelines for performance: caching, parallelization, and artifact retention

Best practices

• Pin task versions (avoid @0 and prefer explicit versions) and pin VM images instead of using 'latest'
• Add displayName to stages and jobs for readability and maintainability
• Use caching for package managers and add timeouts for long-running jobs
• Prefer templates to reduce duplication and use variable groups for many inline variables
• Mark sensitive variables as secret and avoid hardcoding credentials or service connection IDs

Example use cases

• Pre-merge check: run security-only mode in strict mode to block secrets from entering the repo
• CI job: install dependencies and run the full validator to surface syntax and best-practice warnings
• Migration: validate and lint a set of pipeline files when refactoring or moving to reusable templates
• Audit: run the security scan to find dangerous shell patterns (curl | bash), eval usage, or insecure TLS settings