worker-module-architecture_skill

Overview

This skill documents the Worker module architecture for a CI executor responsible for receiving and running pipeline tasks on build agents. It explains core responsibilities—task dispatch, plugin execution, logging, heartbeats, environment management, and artifact upload—and maps components to practical implementation patterns.

How this skill works

It inspects the end-to-end executor lifecycle: startup, environment preparation, continuous heartbeat, task claim/creation via a task factory, task execution through ITask implementations, log buffering and upload, and build finalization. The guide describes how plugin atoms are fetched, validated, executed and how outputs (variables, artifacts, reports) are handled and reported back to the control plane. It also covers resilient API client behavior and retry strategies for service interactions.

When to use it

• Design or extend a Worker/Agent that executes CI pipeline tasks
• Implement new plugin (atom) execution or add task types via TaskFactory
• Improve logging, heartbeat, or timeout handling for more robust runs
• Add or optimize artifact upload, caching, or SDK environment support
• Integrate credential parsing, workspace layout, or OS-specific script runners

Best practices

• Keep Runner responsibilities minimal: prepare workspace, start services, loop claim/execute, then cleanup
• Register task implementations centrally in TaskFactory and support plugin discovery by reflection
• Buffer and filter logs locally before batching uploads; include fold markers and severity flags
• Use heartbeat to detect cancellation and enforce job-level timeouts with a supervisor task
• Treat API calls as transient: use retries for 502/503/504 and network errors, surface meaningful error reports

Example use cases

• Add a new source-control task (e.g., GitHub Pull) by implementing ITask and registering it with TaskFactory
• Implement a custom MarketAtomTask flow that downloads, verifies SHA, injects .sdk.json and executes cross-platform scripts
• Tune LoggerService to redact secrets and decrease upload frequency while preserving real-time tailing
• Implement a job timeout monitor that triggers EngineService.timeout() and kills process trees
• Create a lightweight WORKER mode for quick, non-build jobs running on ephemeral hosts