chat-bot_skill

Overview

This skill provides a Python LLM chat interface compatible with OpenAI-style APIs, including support for local servers and streaming token responses. It includes session management for multi-turn conversations, configurable model parameters, and a ChatManager to handle multiple independent chat sessions. Use it to build chatbots that require history, streaming output, or per-session configuration.

How this skill works

Instantiate LLMChat with a configuration (base_url, api_key, model, etc.) and call ask() for single-turn queries or chat() for multi-turn conversations that preserve history. Streaming mode yields tokens incrementally for responsive UIs, while non-streaming returns the full response. ChatManager creates and tracks multiple LLMChat instances keyed by session ID, with optional per-session LLMConfig and automatic timeout-based cleanup.

When to use it

• Building chatbots that need multi-turn context and preserved conversation history.
• Implementing streaming output to display tokens as they arrive for low-latency UX.
• Managing multiple user sessions with independent state and custom prompts.
• Testing models hosted on OpenAI-compatible endpoints or local servers.
• Rapidly iterating system prompts and model parameters per session.

Best practices

• Keep system prompts focused and minimal to guide behavior without overwhelming context.
• Use streaming for interactive UIs and non-streaming for batch or logged outputs.
• Limit history size or prune old messages to stay within model token limits.
• Set sensible timeouts for ChatManager sessions to free resources for inactive users.
• Provide per-session config for roles like "assistant", "creative writer", or domain experts.

Example use cases

• Customer support chatbot that keeps conversation context per user session and streams replies to the web UI.
• Developer tool that explains code snippets with multi-turn follow-ups and example generation.
• Multi-user education assistant where each student has an isolated chat with a custom system prompt.
• Experimentation harness to compare model responses with different temperature, max_tokens, or system prompts.