Configuration

Quick start

New to LLMAI? Use Configuration quick-start for a minimal setup guide, then return here for the full reference.

The LLMAI plugin exposes the OpenAI Realtime API over WebSocket. OpenAI, Grok (xAI), and LocalAI are provider profiles on the same session model—ConnectToAI, voice, VAD, and function calling share one configuration shape. Most Project Settings are identical across profiles; only authentication and a few profile-specific fields differ.

Compare capability differences (not setup steps) on the Providers comparison table. Enable the plugin first—see Installation on your projects.

Project Settings UI labels match the tables below (Edit → Project Settings → Plugins → LLMAI).

Where configuration lives

Location	Persists?	Typical use
Project Settings (Config/DefaultGame.ini)	Yes (disk)	Team defaults, API keys in dev, model/voice lists
Plugin DefaultGame.ini	Shipped with plugin	Fab defaults; fills gaps on fresh projects
Blueprint SetDefault* / connect dialog	Session/process only (unless bPersistToProjectConfig=true)	Demo UI, runtime overrides
ConnectToAI parameters	Per connection only	Explicit session overrides
Environment variables	External to project	CI, secrets, headless builds
Console CVars (llmai.debug.*)	Runtime only	Troubleshooting verbosity

All defaults live under Edit → Project Settings → Plugins → LLMAI. The Terminal connection dialog and ConnectToAI can override provider, model, voice, and instructions for an individual session—see the Terminal demo.

Per-component voice and audio tuning (for example VoiceThreshold, mic gating) is configured on the Details panel of ULLMAIClientComponent, not in Project Settings. See Component instance properties.

Settings priority

Configuration resolves through separate tracks depending on what you are setting. Do not assume one global priority order applies to API keys, models, and debug logging.

flowchart TD
    subgraph iniLoad [INI at engine startup]
        UEAuto[UE LoadConfig]
        ProjectINI[Project DefaultGame.ini]
        PluginINI[Plugin DefaultGame.ini merge fill-empty]
        UEAuto --> ProjectINI --> PluginINI
    end

    subgraph connectModel [Model at ConnectToAI]
        ConnectArg[Connect parameter if non-empty]
        RuntimeOverlay[Runtime overlay SetDefault]
        ProjectCDO[Project Settings CDO]
        EnvModel[Environment variable]
        ConnectArg --> RuntimeOverlay --> ProjectCDO --> EnvModel
    end

    subgraph connectKey [API key at ConnectToAI]
        ProjectKey[Project Settings APIKey]
        EnvKey[Environment variable]
        CmdKey[Command line flag]
        ProjectKey --> EnvKey --> CmdKey
    end

INI and Project Settings loading (startup)

At module startup, provider defaults load in this order:

1. UE automatic LoadConfig() on ULLMAISettings
2. Manual parse of {Project}/Config/DefaultGame.ini when nested arrays fail or key fields are empty
3. Plugin INI merge — Plugins/LLMAI/Config/DefaultGame.ini is always merged into still-empty provider fields

Project Settings override plugin defaults for fields you customize. If your project INI defines OpenAI but not Grok, Grok voices and models are filled from the plugin INI automatically.

Fresh projects

If {Project}/Config/ is missing or incomplete, voice and model lists come from the plugin INI after merge. On a fresh project you should see OpenAI and Grok voice dropdowns populated without manual INI editing.

Session settings at connect (model, voice, instructions)

When ConnectToOpenAI, ConnectToLocalAI, or ConnectToGrok is called with empty model or instructions, the plugin resolves values in this order:

Priority	Source	Example
1 (highest)	Connect parameter	ConnectToAI Model pin or Terminal dialog selection
2	Runtime overlay	SetDefaultAIModel, SetDefaultInstructions, etc. (Blueprint library)
3	Project Settings	Per-provider Model / Instructions in the editor
4 (lowest)	Environment variable	LLMAI_OPENAI_MODEL, OPENAI_MODEL, etc.

Voice follows the same overlay → Project Settings path. Grok also accepts an explicit Voice connect parameter; see Grok voice lock below.

Known model lists (dropdown validation) use overlay → Project Settings via GetKnownAIModels. At connect time, comma-separated environment variables can override the list for that component only:

Provider	Environment variables
OpenAI	LLMAI_OPENAI_MODELS, OPENAI_MODELS
LocalAI	LLMAI_LOCALAI_MODELS, LOCALAI_MODELS
Grok	LLMAI_GROK_MODELS, GROK_MODELS

API keys (separate track)

API key discovery is separate from model/voice settings. The plugin uses the first non-empty key found in this order:

Provider	Project Settings field	Environment variables (in order)	Command line
OpenAI	OpenAI Provider → API Key	LLMAI_OPENAI_API_KEY, OPENAI_API_KEY	-OpenAIKey=
Grok	Grok Provider → API Key	XAI_API_KEY, LLMAI_GROK_API_KEY, GROK_API_KEY	-GrokKey= or -XAIKey=
LocalAI	LocalAI Provider → API Key (optional)	LLMAI_LOCALAI_API_KEY, LOCALAI_API_KEY	-LocalAIKey=

Security

Project Settings is recommended for local development. Use environment variables or command-line flags for CI and shipping workflows so secrets are not committed to source control. The plugin logs which source was used—it never logs key values.

Common misconception

If Project Settings contains a non-empty API key, environment variables are not used, even when set. To force env-based keys, leave the Project Settings API Key field empty.

LocalAI often runs without authentication; the Realtime Base URL is the critical connection setting. An optional API key is supported when your server requires it.

Debug settings (Project Settings vs CVars)

Layer	Behavior
Project Settings → LLMAI | Debug	Applied once at module startup
Console llmai.debug.* CVars	Override at runtime; authority after init

Project Settings field	Initializes CVar
Enable Verbose Logging	All debug levels (equivalent to llmai.debug.EnableAll)
Save Audio Debug Files	llmai.debug.SaveAudioFiles
Log WebSocket Messages	llmai.debug.LogRawWebSocketData

Console commands such as llmai.debug.EnableAll control logging verbosity at runtime. They do not replace Project Settings for provider, model, or voice configuration. See Debug and logging.

Runtime Blueprint settings (v2.2)

Blueprint functions SetDefaultAIModel, SetDefaultAIVoice, SetDefaultInstructions, SetDefaultAIProvider, SetKnownAIModels, and SetKnownAIVoices write to a process-wide runtime overlay by default:

• Affects GetDefault* reads and connect fallbacks
• Does not write to disk unless you pass bPersistToProjectConfig=true
• When bPersistToProjectConfig=true, the overlay merges into Project Settings and saves Config/DefaultGame.ini

The Terminal connect dialog uses these at runtime. Pass bPersist=true where you want demo choices to persist across editor sessions.

Use GetProviderCapabilities and IsProviderFeatureSupported to hide UI options a provider does not support—for example, OpenAI GA ignores session temperature, so a temperature slider should be disabled when OpenAI is selected. See Providers comparison.

Project Settings catalog

The following tables mirror Edit → Project Settings → Plugins → LLMAI in v2.2.

LLMAI | AI

Setting	Purpose
Default AI Provider	OpenAI, LocalAI, or Grok when connect does not specify a provider

Per-provider settings (OpenAI, LocalAI, Grok)

Each provider subsection uses the same category groups. Field names below match the editor.

Authentication

Setting	OpenAI	LocalAI	Grok
API Key	Required	Optional	Required

Models

Setting	Purpose
Model	Default Realtime model for this provider
Transcription Model	Speech-to-text model (default: whisper-1)
Known Models	List of models shown in UI dropdowns and used for validation

Voice

Setting	Purpose
Realtime Voice	Default voice ID for Realtime sessions
Known Voices	Voice entries (Name, Id, Description) for UI dropdowns
TTS Model (LocalAI only)	Optional TTS backend (chatterbox, kokoro-tts, etc.)

Instructions

Setting	Purpose
Instructions	Per-provider system prompt (multi-line). There is no global instructions field—each provider has its own.

Connection

Setting	OpenAI	LocalAI	Grok
Realtime Base URL	wss://api.openai.com/v1/realtime	ws://localhost:8080/v1/realtime	wss://api.x.ai/v1/realtime
Header Value	Deprecated (ignored for GA)	Optional	—

Audio Format

Setting	Purpose
Realtime Output Sample Rate	Target sample rate (24000 Hz recommended)
Input Audio Format	Default: pcm16
Output Audio Format	Default: pcm16
Audio Output Speed (LocalAI only)	Playback speed multiplier

Timing

Setting	Purpose
Voice Mode Start Timeout (s)	Max wait for voice mode to become ready
Playback Buffer Seconds	Audio playback buffering
Server VAD Wait Multiplier	Multiplier for server-side VAD timing
Response Coalesce Delay (s)	Delay before coalescing response chunks

Audio Streaming

Setting	Purpose
Audio Send Interval (s)	How often mic audio is sent to the server
Max Accumulated Audio Size (bytes)	Cap on buffered input audio
Max Collected Response Size (bytes)	Cap on buffered response audio

VAD (Voice Activity Detection)

Setting	Purpose
VAD Type	Semantic VAD (OpenAI only) or Server VAD
VAD Eagerness	Semantic VAD only: Low, Medium, High, Auto
VAD Threshold	Server VAD sensitivity (0.1–1.0)
VAD Prefix Padding Ms	Audio padding before speech detection
VAD Silence Duration Ms	Silence duration before turn end

Grok Features (Grok only)

Setting	Purpose
Enable Search	Grok live web/X search

LLMAI | Audio (shared)

Setting	Purpose
Default Sample Rate	WASAPI capture default (actual rate comes from device)
Default Channels	Mono (1) or stereo (2)
Default Bits Per Sample	Bit depth for capture
Default Voice Threshold	Mic activity threshold
Default Max Silence Duration	Silence before end-of-turn
Default Audio Output Directory	Where debug WAV files are saved
Recognition Gain Boost Multiplier	Input gain for speech recognition

LLMAI | Socket

Setting	Purpose
WebSocket Message Memory Limit (bytes)	Max incoming message size. Increase if you see WebSocket error 1009 (message too big). Default: 32 MB.

LLMAI | Audio Stream

Setting	Purpose
Streaming Chunk Size (bytes)	WASAPI capture chunk size
Streaming Worker Sleep (s)	Background streaming thread sleep interval

LLMAI | LiveLink

Setting	Purpose
Enable LiveLink By Default	Auto-enable LiveLink on audio stream component
Default LiveLink Subject Name	LiveLink subject for lip-sync (default: LLMAIAudio)

LLMAI | Debug

Setting	Purpose
Enable Verbose Logging	Enable detailed logging at startup (see Debug settings)
Save Audio Debug Files	Save captured audio to disk for debugging
Log WebSocket Messages	Log raw WebSocket traffic
Max Content Preview Length	Truncate long log content previews
Max Result Preview Length	Truncate long function result previews

Environment variables and CLI flags

Variable	Provider	Purpose
LLMAI_OPENAI_API_KEY, OPENAI_API_KEY	OpenAI	API key fallback
LLMAI_OPENAI_MODEL, OPENAI_MODEL	OpenAI	Default model fallback
LLMAI_OPENAI_MODELS, OPENAI_MODELS	OpenAI	Comma-separated Known Models override (per component at connect)
LLMAI_LOCALAI_API_KEY, LOCALAI_API_KEY	LocalAI	Optional API key
LLMAI_LOCALAI_MODEL, LOCALAI_MODEL	LocalAI	Default model fallback
LLMAI_LOCALAI_MODELS, LOCALAI_MODELS	LocalAI	Known Models override
XAI_API_KEY, LLMAI_GROK_API_KEY, GROK_API_KEY	Grok	API key fallback
XAI_MODEL, LLMAI_GROK_MODEL, GROK_MODEL	Grok	Default model fallback
LLMAI_GROK_MODELS, GROK_MODELS	Grok	Known Models override

Command-line flags: -OpenAIKey=, -LocalAIKey=, -GrokKey=, -XAIKey=

Upgrading after a plugin update

When you install a newer LLMAI plugin in a project that already had LLMAI configured, existing project INI values are kept. At startup the plugin merges Plugins/LLMAI/Config/DefaultGame.ini only into still-empty provider fields—it does not replace your current Model, KnownModels, or KnownVoices entries.

Recommended reset (any plugin upgrade)

1. Close Unreal Editor and back up {Project}/Config/DefaultGame.ini.
2. Remove the entire [/Script/LLMAI.LLMAISettings] block from that file (from the [/Script/LLMAI.LLMAISettings] line through the end of all LLMAI provider lines—do not delete unrelated [...] sections elsewhere in the file).
3. Install or copy the new plugin version, then open the project.
4. On first launch, LLMAI settings are filled from the shipped plugin defaults—including default Model values and KnownModels / KnownVoices lists for OpenAI and Grok.
5. Open Edit → Project Settings → Plugins → LLMAI and re-enter API keys, custom Instructions, LocalAI Realtime Base URL, and any other values you had before the reset.

What you lose when deleting the section

Deleting [/Script/LLMAI.LLMAISettings] clears all saved LLMAI Project Settings in that file—not only models. API keys stored in INI, custom instructions, debug toggles, and LocalAI URLs must be entered again after reset.

Upgrading from v2.1 to v2.2

After reset, shipped defaults include OpenAI model gpt-realtime-2 and Grok model grok-voice-think-fast-1.0, with KnownModels lists matching the v2.2 plugin INI. If you keep an old project INI without resetting, stale OpenAI preview model names may fail to connect under the GA Realtime API.

See Changelog — Upgrading from a previous version for why reset is needed and the v2.1 → v2.2 example; see Version 2.2 for full release notes.

INI customization

Project overrides live in {Project}/Config/DefaultGame.ini under [/Script/LLMAI.LLMAISettings]. Plugin defaults ship in Plugins/LLMAI/Config/DefaultGame.ini. Editing Project Settings in the editor writes the same INI section.

Minimal example (OpenAI block with correct property names):

[/Script/LLMAI.LLMAISettings]
DefaultAIProvider=OpenAI
OpenAI=(APIKey="",Model="gpt-realtime-2",RealtimeVoice="alloy",Instructions="You are a helpful assistant.",RealtimeBaseUrl="wss://api.openai.com/v1/realtime",KnownModels=("gpt-realtime-2","gpt-realtime-mini"),KnownVoices=((Name="Alloy",Id="alloy",Description="Neutral voice")))
Grok=(APIKey="",Model="grok-voice-think-fast-1.0",RealtimeVoice="Ara",RealtimeBaseUrl="wss://api.x.ai/v1/realtime",KnownModels=("grok-voice-think-fast-1.0","grok-voice-latest"),KnownVoices=((Name="Ara",Id="Ara")))

Use KnownModels and KnownVoices (not legacy names like KnownRealtimeModels). Voice entries use (Name="...",Id="...",Description="...") tuples.

Provider profiles

Configure the profile you plan to use. You only need one provider running for a given session, but you may fill in all three if you switch between them.

OpenAI

Authentication: API key required.

1. Edit → Project Settings → Plugins → LLMAI → OpenAI Provider
2. Enter API Key (or use env/CLI sources in API keys)
3. Set Model (default: gpt-realtime-2) and Realtime Voice
4. Optionally set Instructions for this provider

Default model for LLMAI v2.2 is gpt-realtime-2. OpenAI dropped preview Realtime models as of May 2026—see Providers — OpenAI.

OpenAI GA ignores session temperature in the Realtime API. Use GetProviderCapabilities if your UI exposes a temperature slider.

Grok (xAI)

Authentication: API key required (from console.x.ai).

1. Edit → Project Settings → Plugins → LLMAI → Grok Provider
2. Enter API Key (or use env/CLI sources in API keys)
3. Set Realtime Voice — Ara, Rex, Sal, Eve, or Leo
4. Optionally set Model (default: grok-voice-think-fast-1.0) and Enable Search

warning

Grok locks the voice after the first session update. You must specify voice when connecting (via the Voice parameter or Project Settings default). It defaults to Ara if omitted and cannot be changed without reconnecting.

LocalAI

Authentication: Optional API key. Requires a running LocalAI Realtime server.

1. Edit → Project Settings → Plugins → LLMAI
2. Set Default AI Provider to LocalAI (optional if you always pass provider at connect)
3. Under LocalAI Provider:
   • Realtime Base URL — e.g. ws://localhost:8080/v1/realtime (must use ws://, not http://)
   • Model — e.g. gpt-realtime
   • TTS Model — e.g. chatterbox (optional)
   • Transcription Model — e.g. whisper-1
   • Realtime Voice — TTS voice for output
   • Known Voices — define voices matching your TTS backend

Download, Docker, and server setup: LocalAI Realtime Endpoint. Return here for Unreal-side defaults after the server is running.

Where providers differ

Setup is largely shared; behavior differs at runtime. See the comparison table for full detail.

• Voice mid-session: OpenAI and LocalAI allow changing voice without reconnecting; Grok does not.
• VAD: OpenAI supports Semantic VAD and Server VAD; Grok and LocalAI use Server VAD only.
• Session temperature: OpenAI GA ignores temperature; use GetProviderCapabilities to drive UI visibility.
• Web search / MCP: Grok has native built-in search; OpenAI supports server-side MCP; LocalAI has no server-side MCP.
• LocalAI-only fields: Realtime Base URL, TTS Model, Header Value, Audio Output Speed.

Verification

1. Plugin installed

1. Create an Actor Blueprint
2. Add Component → search for LLMAI Client Component
3. If the component appears → installation succeeded
4. If not → Installation troubleshooting

2. Realtime connection

1. Add LLMAI Client Component to an Actor
2. On Begin Play, call Connect to AI (or use Blueprint equivalent)
3. Use your configured Default AI Provider, or pass provider explicitly
4. Set Model to your profile default (e.g. gpt-realtime-2 for OpenAI)
5. For Grok, always pass Voice (e.g. Ara)
6. For LocalAI, ensure the server is running before connecting
7. Play the level and check Output Log for AI Session Ready

3. Config merge (fresh or partial project)

On a project with OpenAI configured but no Grok INI block, confirm the Output Log shows Grok voices/models loaded from the plugin INI merge (e.g. Loaded PLUGIN Grok.KnownVoices: 5 entries).

4. Full setup (demos)

If you have the LLMAI demo content:

1. Open Content/LLMAITerminal/Levels/LVL_LLMAITerminal.umap
2. Play in editor and connect via the Terminal dialog
3. Chat or enable voice mode to confirm end-to-end behavior

See the Demo guide for Terminal, Rocketeer, and Live Link walkthroughs.

Troubleshooting

Start with these checks:

• Confirm the correct provider profile is selected and its auth is configured (API key or LocalAI Realtime Base URL).
• API key in env but connection fails: check whether Project Settings already has a different (or empty-looking) key—Project Settings wins when non-empty.
• Read the Output Log for connection errors; enable llmai.debug.EnableAll if you need more detail.
• Cloud providers (OpenAI, Grok): verify API key, credits, network, and firewall/WebSocket access.
• LocalAI: verify the server is running, the endpoint uses ws://, and the URL matches Project Settings.

For deeper help, use the dedicated references rather than duplicating them here:

• FAQ — API keys, models, voices, Grok voice lock, distribution
• Debug and logging — console commands and log categories
• Installation troubleshooting — provider-specific connection errors