fynd-backend-microservices_skill

Overview

This skill provides expert debugging and operational guidance for FYND’s Node.js backend running on Kubernetes, GCP, Kafka, Redis, and related services. I focus on fast triage and practical fixes for pods crashing, Kafka lag, Redis memory issues, API latency, DB migration failures, memory leaks, LLM cost spikes, and general service failures. The goal is to reduce mean time to recovery and prevent recurrence with concrete remediation and monitoring guidance.

How this skill works

I start by collecting key signals: Kubernetes pod states and logs, consumer group offsets, Redis memory usage, application traces, and database errors. I form hypotheses (OOM, poison message, slow query, leaking allocation), test them with targeted commands or small experiments, and deliver step-by-step fixes plus monitoring and resilience recommendations. Bundled scripts and checklists speed common diagnostics and ensure repeatable remediation.

When to use it

• Pod restarts, CrashLoopBackOff, or failing deployments on Kubernetes
• Sustained Kafka consumer lag, DLQ growth, or rebalancing failures
• Redis memory spikes, TTL misconfigurations, or pub/sub issues
• Unexplained API latency, high error rates, or throughput drops
• Database migration errors, schema drift, or replication problems
• Unexpected LLM cost increases or runaway token usage

Best practices

• Gather structured signals first: kubectl, consumer offsets, Redis INFO, APM traces
• Reproduce minimally and test hypotheses before wide changes
• Add graceful shutdown handlers and resource limits to pods
• Implement circuit breakers, backoff, and DLQ strategies for resilience
• Profile memory and CPU regularly to catch leaks early
• Automate alerts with meaningful thresholds and postmortem checklists

Example use cases

• Diagnose and fix a CrashLoopBackOff caused by missing graceful shutdown and OOM
• Recover consumers from persistent Kafka lag by identifying poison messages and reprocessing via DLQ
• Reduce Redis memory footprint by auditing keys, adjusting TTLs, and refining eviction policy
• Trace an API latency spike to a slow DB query and deploy a short-term cache plus optimized query
• Contain rising LLM costs by adding token limits, batching requests, and routing to cheaper models