Skip to main content

Terminal demo

Terminal demo

info

The LLMAI Terminal is the shared chat interface used by both demo projects (LLMAI_Arcade and LLMAI_LiveLink). It showcases real-time text and voice communication through a clean, desktop-style interface. Used In: • LLMAI_Arcade , AI controls the Rocketeer game through function calling • LLMAI_LiveLink , AI controls a MetaHuman character with lip-sync and gestures Perfect for: Understanding AI integration, testing voice features, learning LLM capabilities, developing cutting edge LLM functionality in Unreal Engine.

LLMAI Terminal Interface Overview​

Getting Started​

Prerequisites​

Before trying this demo, you need to enable the LLMAI plugin

Required setup: Install and enable the LLMAI plugin, then configure your chosen provider (OpenAI, Grok, or LocalAI). See Getting started, Configuration, and the Demo guide for details.

Launch the Demo​

  1. Open the Level: Navigate to Content/LLMAITerminal/Levels/LVL_LLMAITerminal.umap
  2. Play the Level: Click Play in Editor or use Alt+P
  3. Connect to AI: The connection dialog will appear automatically

Configure Your Experience​

  1. Choose AI Model: Select from available models
  2. Select Voice: Pick an AI voice personality
  3. Set Instructions: Customize AI behavior or use the helpful defaults
  4. Click Connect: Wait for the "AI Session Ready" confirmation

Features Showcase​

Text Communication​

Voice Communication​

System Integration​

Interface Guide​

Connection Dialog​

The connection dialog is used to connect to your selected AI provider and configure your session:

Configuration Options​

  • Provider: Select from available providers: OpenAI, Cloud-based AI using OpenAI's Realtime API (requires API key) LocalAI, Local/offline AI processing (available separately, no API key required)
  • Model Selection: Use the model drop-down list to select from available OpenAI models
  • Voice Selection: Choose your desired voice from the Voice drop-down list
  • AI Instructions: Enter instructions to customize the AI's personality, approach, and even accent Example: "You speak in a very broad Australian accent"
  • Rocketeer Game Functions: or LiveLink Integration Functions: Use the checkbox to include game integration functions
  • Log Interface: Toggle the log display using the Log Interface toggle button at bottom-left

Connecting: Once you've selected your desired options, click the Connect button. When the connection is established, it will automatically show the chat interface.

Chat Interface Overview​

The main chat interface provides all the tools you need for AI communication:

Interface Components​

  • Chat Messages History: Displays all messages between "Player1" and "Chatbot" with full conversation context
  • Text Input Field: Enter your text message at the bottom of the chat interface Press Enter or click the Send button to send your message Messages appear in real-time with streaming responses
  • Voice Mode Toggle: Turn voice communication on and off at any time Enables microphone input and voice output through default speakers Provides natural voice conversation capabilities
  • Log Toggle: Access the Log Interface toggle button at bottom-left (also available during conversation)
  • Close Button: Located at top-right to disconnect from chat and automatically end voice mode

How to Use​

Text Chat​

  1. Type your message in the input field at the bottom
  2. Press Enter or click the Send button
  3. Watch the response - The response is streamed in real-time from the AI to the chat area
  4. Continue the conversation naturally back and forth with historical context maintained

Voice Mode​

  1. Click the Voice Mode button to enable voice mode
  2. Start speaking and your voice will be streamed to the AI
  3. AI will respond with streaming text and voice output in real-time
  4. Continue talking - you can naturally interrupt the AI at any time
  5. Click the Voice Mode button again to return to text-only mode

Log Interface​

Log Toggle: Toggle the in-game log to show LLMAI and terminal events

Demo-Specific Features​

The Terminal interface is used by both demo projects, each with different AI capabilities:

LLMAI_Arcade Demo​

Say "Launch Rocketeer" and the AI will use a function call to start the arcade game.

Refer to the Rocketeer Game Documentation for detailed information.

Enable voice mode to see real-time lip-sync on the MetaHuman character. Try commands like "Wave hello" , "Move to the left" , or "Jump!"

Refer to the Live Link demo for gesture and movement functions.

What to Try​

Basic Conversations​

  • "Hello, how are you today?" - Start with simple greetings
  • "What can you help me with?" - Discover AI capabilities
  • "Tell me about Unreal Engine" - Explore knowledge areas
  • "Can you help me with game development?" - Get technical assistance
  • "Launch Rocketeer" - Try the game integration feature

Technical Highlights​

Blueprint Architecture​

The terminal demonstrates professional Blueprint implementation patterns:

  • Component-Based Design: Clean separation of concerns
  • Event-Driven Communication: Responsive UI updates
  • Error Handling: Graceful failure recovery
  • Performance Optimization: Efficient message handling

🆘 Troubleshooting​

Connection problems​

  • API/connection error: Verify your provider API key or LocalAI endpoint in Project Settings
  • Network issues: Check internet access and firewall rules (cloud providers)
  • Model availability: Try a different model in the connection dialog

Audio issues​

  • Voice mode not working: Check Windows microphone permissions
  • No AI voice output: Verify speakers/headphones and system audio levels

Refer to the FAQ and Debug and logging for more help.

This terminal demonstration provides a complete foundation for understanding AI integration in Unreal Engine and serves as an excellent starting point for your own AI-powered applications.