LLMAI - Project Overview

🚀 Quick Start | 🔧 Plugin Installation | 📖 Plugin Documentation | 📚 API Reference

Overview

LLMAI is a component based AI integration solution for Unreal Engine 5 that brings sophisticated conversational AI directly into your games and applications. This complete package includes:

🔌 LLMAI Plugin

Core AI integration with OpenAI Realtime API

💬 LLMAI Terminal

Interactive chat interface with text and voice communication

🎮 LLMAI_Arcade Demo

AI-controlled Rocketeer arcade game demonstrating function calling

🎭 LLMAI_LiveLink Demo

MetaHuman character with lip-sync and AI-controlled gestures

Perfect for developers wanting to integrate AI assistants, voice interfaces, or dynamic AI-controlled gameplay into their Unreal Engine projects.

Key Features

💬 Streaming Text Communication

Real-time text responses from AI with character-by-character streaming for immediate, responsive conversations that feel natural and engaging.

🎤 Advanced Voice Communication

Real-time voice conversations with AI featuring automatic speech detection, multiple voice personalities, and low-latency bidirectional audio streaming with noise reduction.

Define custom AI functions through Blueprint integration that allow AI to control game systems, spawn objects, adjust settings, and interact with your game in real-time with flexible parameter support.

🎮 Complete Demo Applications

Production-ready Terminal interface with voice controls and an interactive Rocketeer arcade game where AI plays and controls gameplay, providing comprehensive Blueprint examples and educational content.

🛠 Developer-Friendly Integration

Simple Blueprint components for any Actor with event-driven architecture, comprehensive settings, and thread-safe implementation for reliable operation in complex game scenarios.

🖥️ Platform & Requirements

Windows 64-bit support (Win10/11 x64) for Unreal Engine 5.6+ with built-in AudioCapture plugin automatically enabled and Visual Studio 2022 required for development.

🎮 What's Included

LLMAI Plugin `Plugins/LLMAI/`

The core AI integration plugin providing:

ULLMAIClientComponent - Main AI communication component (instance properties)
ULLMAIAudioStreamComponent - Advanced audio capture and streaming (properties)
Blueprint Library - Utility functions and helpers
Data Assets - Function definition and profile systems
Settings System - Global project configuration options
Debug Logging System - Quickly find the root cause of problems

LLMAI Terminal `Content/LLMAITerminal/`

A complete chat interface used by both demo projects:

Professional UI - Desktop-style chat interface
Voice Mode - Toggle between text only and voice communication
Connection Management - Easy AI service connection with model/voice selection
System Logging - Debug and monitoring log

LLMAI_Arcade Demo `Content/LLMAITerminal/RocketeerGame/`

An interactive arcade game showcasing AI function calling:

AI Game Control - AI can start/stop game, adjust difficulty
Dynamic Gameplay - AI spawns enemies, creates flight paths
Real-time Interaction - Voice commands control game systems
17+ AI Functions - Complete function calling demonstration

LLMAI_LiveLink Demo `Content/LLMAITerminal/Character/`

A MetaHuman character demonstration with lip-sync and AI control:

Lip-Sync - TTS voice output drives MetaHuman lip animations via LiveLink
16 Gestures - AI triggers expressive character gestures
Character Movement - AI moves character between waypoints
Jump Action - Full character jump with animation state machine

System Requirements

Engine Version

Unreal Engine 5.6+
Windows 64-bit only (Windows 10/11 x64 required)

Development Tools

Visual Studio 2022 (required for plugin compilation)
Windows SDK (latest version recommended)

Hardware

8GB RAM minimum (16GB recommended for compilation)
GPU with DirectX 11+ (for optimal performance)

AI Service

OpenAI API Key for cloud operation, OR

LocalAI distribution (available separately) for offline operation
Internet Connection (required for OpenAI, not for LocalAI)
WebSocket Support (enabled by default in UE5)

Optional Components

Microphone (for voice input features)
Audio Output (for AI voice responses)

🔧 Configuration

The LLMAI plugin provides comprehensive configuration options through Unreal Engine's Project Settings panel. Configuration covers OpenAI API settings, audio processing parameters, connection options, and debug features.

Plugin Installation: 🚀 Quick Start and for detailed plugin installation steps, see Plugin Installation Guide

Complete Configuration Guide: See Plugin Configuration Documentation for detailed settings explanations and recommendations.

📄 License

This project is distributed under the Fab Standard License when available on fab.com - see the Fab End User License Agreement (EULA) for complete licensing terms and conditions.