Project overview
What's Included
A component that makes it easy to integrate so players can chat with AI and affect realtime gameplay events and scenarios.
LLMAI Plugin
Connects the Unreal 5 engine to AI providers to stream text, voice and call functions at runtime.
- ULLMAIClientComponent
- ULLMAIAudioStreamComponent
- Blueprint library
- Data assets & settings system
- Debug logging system
LLMAI Terminal
Blueprint based chat UI that can either be spawned in world or part of game overlay.
- Professional chat UI
- Voice mode toggle
- Connection management
- System debug logging
LLMAI Arcade
A classic arcade game controlled by AI via function calling and voice interface.
- AI game control
- Dynamic enemy spawning
- Voice command integration
- 17+ AI functions demonstrated
LLMAI Live Link
Metahuman enhanced with AI lip sync and actions.
- MetaHuman lip-sync via LiveLink
- 16 AI-triggered gestures
- Character movement waypoints
- Jump animation state machine
Key Features
Streaming text
Real-time character-by-character AI responses for natural conversations.
Advanced voice
Low-latency bidirectional voice with automatic speech detection.
Function calling
Blueprint-defined functions let AI control your game in real time.
Multiple AI providers
OpenAI, Grok (xAI), and LocalAI on the same Realtime API protocol.
Complete demos
Terminal UI plus Arcade and LiveLink projects with full examples.
Developer friendly
Actor components, events, and thread-safe integration patterns.
Built in C++ so you don't have to (unless you want). Fully exposes a set of blueprint interfaces and templates for dynamic AI-controlled gameplay in Unreal Engine projects.
Requirements
Engine
- Unreal Engine 5.6+
- Windows 64-bit only (Windows 10/11 x64)
Development tools
- Visual Studio 2022 (required for plugin compilation)
- Windows SDK (latest version recommended)
Hardware
- 8GB RAM minimum (16GB recommended for compilation)
- GPU with DirectX 11+
AI service
- LocalAI distribution (available separately) for offline use
- Internet connection for OpenAI/Grok (not required for LocalAI)
- WebSocket support (enabled by default in UE5)
Optional
- Microphone for voice input
- Audio output for AI voice responses
Distributed under the Fab Standard License when available on Fab.