local-voice_skill

Overview

This skill provides local text-to-speech (TTS) and speech-to-text (STT) running entirely on Apple Silicon using FluidAudio models and the Apple Neural Engine. It delivers sub-second synthesis and transcription offline, avoiding cloud services and recurring costs. It is designed for macOS 14+ on M1/M2/M3/M4 machines and exposes a simple local HTTP daemon for integration.

How this skill works

A Swift-based daemon loads FluidAudio CoreML models and runs inference on the Apple Neural Engine for low-latency audio processing. It exposes HTTP endpoints for synthesize, transcribe, and health checks, and uses espeak-ng as a phoneme fallback. The binary and framework run from a local install path and can be managed via a LaunchAgent for persistent availability.

When to use it

• Adding offline voice assistant capabilities on Apple Silicon Macs
• Replacing cloud TTS/STT to improve privacy and reduce cost
• Embedding low-latency voice in desktop apps or automation
• Prototyping voice features where network access is limited
• Running batch transcription locally for sensitive audio

Best practices

• Build and run natively on ARM64 macOS (do not use Rosetta)
• Install espeak-ng for phoneme fallback to improve pronunciation
• Place the framework alongside the binary and add rpath using install_name_tool
• Load the daemon via a LaunchAgent for automatic start and log collection
• Warm models with a first request at startup to avoid the initial 8–10s delay

Example use cases

• Voice assistant: route microphone audio to /transcribe and synthesize replies from /synthesize
• Local TTS for notifications and accessibility without cloud audio upload
• Real-time transcription for meetings or dictation with sub-second latency
• On-device audio generation for apps that must keep user data private
• Edge automation: generate spoken alerts from scripts using the helper wrapper