tiktok-crawling_skill

Overview

This skill provides command-line patterns and workflows for crawling, downloading, and analyzing TikTok content using yt-dlp and ffmpeg. It focuses on reproducible outputs, metadata export, incremental archiving, and filters to target specific videos. Use it to build backups, datasets, or research-ready exports for analysis.

How this skill works

It uses yt-dlp to fetch video files and metadata from TikTok URLs, profile pages, hashtag/search queries, or sound pages. Output templates place files and .info.json metadata into organized folders; flags like --download-archive avoid re-downloading. You can simulate runs, export JSON/CSV, or pipe metadata to jq or pandas for analysis.

When to use it

• Backing up a creator profile or channel to a local archive
• Building datasets for engagement or trend analysis
• Incremental scraping as new videos are published
• Filtering downloads by date, views, duration, or title patterns
• Exporting metadata for tooling (JSON, JSONL, CSV)
• Automating scheduled scrapes via cron or CI pipelines

Best practices

• Install ffmpeg alongside yt-dlp for post-processing and format merging
• Use --download-archive to track and skip already-downloaded content
• Prefer --cookies-from-browser for authenticated or less-rate-limited access
• Use --simulate / --dump-json for testing and metadata-only exports before downloading
• Apply --sleep-interval and max-sleep-interval to reduce rate-limit risk
• Store per-video metadata (.info.json) and export JSONL/CSV for downstream analysis

Example use cases

• Download a single video: yt-dlp "https://www.tiktok.com/@handle/video/123"
• Archive an entire profile into organized folders with metadata and a download archive
• Run a scheduled script that scrapes multiple handles daily and skips existing videos
• Collect top videos for a hashtag or sound via playlist-end and filter by view_count
• Export metadata as JSONL and run jq or load into pandas for top-N, total views, or time-series analysis