administering-linux_skill

Overview

This skill provides practical Linux system administration for managing systemd services, processes, filesystems, networking, performance tuning, and troubleshooting on modern distributions (Ubuntu, RHEL, Debian, Fedora). It focuses on outcomes: deploy reliable services, diagnose production issues, optimize servers, and manage users and security basics. The content is hands-on with essential commands, decision frameworks, and repeatable workflows.

How this skill works

The skill inspects and guides common server tasks: creating and managing systemd units and timers, monitoring and controlling processes, analyzing logs with journalctl, and handling disk and network configuration. It provides step-by-step workflows for troubleshooting CPU/memory/disk/network bottlenecks, tuning sysctl and ulimits, and hardening SSH/firewalls. Examples and templates speed deployment and investigation in production environments.

When to use it

• Deploy or manage long-running services with systemd
• Diagnose slow or failing production servers (CPU, memory, I/O, network)
• Configure filesystems, LVM, or choose an appropriate filesystem for workload
• Harden SSH and basic firewall rules for remote access
• Tune kernel and resource limits for databases or high-load applications
• Create scheduled jobs using systemd timers or cron

Best practices

• Prefer systemd unit files and overrides in /etc/systemd/system and use daemon-reload after changes
• Collect metrics and correlate logs (journalctl) with top/htop, iostat, and ss before acting
• Use non-root service users, minimal privileges, and ReadWritePaths/PrivateTmp for service isolation
• Apply sysctl and limits via /etc/sysctl.d/ and /etc/security/limits.conf for reproducible tuning
• Keep a rescue SSH session when restarting sshd and test configuration with sshd -t
• Start with conservative changes in production and validate with monitoring and rollback plans

Example use cases

• Create a systemd service for a web application with automatic restart and minimal privileges
• Troubleshoot an application experiencing high load: identify CPU spike, check dmesg/journalctl for OOM, and isolate offending process
• Recover disk space: use df/du/ncdu to find large directories and adjust log rotation or archive old data
• Harden server access: deploy ed25519 keys, disable root login, restrict AllowUsers and adjust SSH port if required
• Schedule backups with a systemd timer and a oneshot service that runs reliably after reboots