Skip to main content

NAF API

NAF — the Native Avatar Framework — is the engine that loads, animates, and renders Genies avatars. This page is the reference for its iOS surface.

The Partner SDK ships NAF as a dependency, and GeniesPartnerSDKNAF wraps the paths most apps need: render a user's avatar, dress it, save the outfit, talk to it. Reach for this page when you want more than those conveniences give you — a custom render loop, per-material shading, hand-tuned animation, or loading content that doesn't come from a user's inventory.

Frameworks​

NAF ships as four libraries. Pick the highest-level one that fits, then drop down where you need to.

FrameworkLanguageUse it when
NAFKitUISwiftYou want a view that loads, renders, and animates an avatar end to end.
NAFKitUIToolsSwiftYou drive your own renderer but want the SDK's orbit camera, auto-framing, and debug overlays.
NAFKitObjective-CYou drive your own renderer and only want asset loading, animation, and skeleton math.
NAFKitConfigSwiftAlways. It carries the resolver config, shader presets, and texture settings.

NAFKitUI depends on NAFKitUITools and NAFKit, so adding it pulls all three. NAFKitConfig is a separate dynamic product you add alongside. A fifth, NAFKitLiveKit, is optional and only needed for live conversational avatars.

import NAFKitUI       // NAFView, NAFAvatarSession
import NAFKit // NAFAvatar, NAFContainerApi, NAFError
import NAFKitUITools // CameraControls, AvatarBounds, ToonPreset
import NAFKitConfig // NAFKitConfig

Quick start​

NAFView is a UIView that owns its own Metal pipeline, an HDR offscreen target, a tone-map pass, and an auto-framed orbit camera. Loading an avatar runs the whole pipeline — optional silhouette preview, multi-asset composition, then the avatar comes to life and the remaining LODs stream in behind it.

let view = try NAFView()
addSubview(view)

try await view.loadCampSmartAvatar("bm_smart-avatar_abc123", silhouette: true, lods: [2, 0])
view.applyDefaultToonPreset()
view.frameAvatar()

For SwiftUI, wrap it with NAFAvatarView(view:).

Going headless​

NAFAvatarSession is the same pipeline without the view. It owns loading, animation, behavior, and per-frame skeleton math; you own the scene, the camera, and the draw calls. Every NAFView has one as view.session, and the session's own API is public so you can use it alone.

let session = NAFAvatarSession()
session.onMeshesRebuilt = { meshes in myRenderer.applyMeshes(meshes) }
try await session.loadCampAvatar("bm_avatar_abc123")

See Bringing your own renderer for the full loop.

Configuration​

NAFKitConfig​

NAFKitConfig holds the bundled resolver configuration, the shader presets directory, and texture settings. Every accessor falls back to the bundled value, and each has a matching override you can set at launch.

MemberNotes
resolverConfigResolver JSON. Bundled -dev in DEBUG builds, -prod in release.
resolverConfigOverrideSet at startup to replace the bundled resolver JSON.
shaderPresetsDirectoryDirectory holding toon preset JSON.
shaderPresetsDirectoryOverridePoint at your own presets instead of the bundled shaders/.
textureSettingsURLOverrideReplace the bundled textureSettings.json.

NAFView() and NAFAvatarSession() both default their resolverConfig: argument to NAFKitConfig.resolverConfig, so the common case needs no configuration at all.

Service credentials​

NAFServiceConfiguration sets the base URL, bearer token, SDK version, and arbitrary headers for a named backend service. All four are static.

try NAFServiceConfiguration.setBaseURL(baseURL, forService: "camp")
try NAFServiceConfiguration.setBearerToken(token, forService: "camp")
try NAFServiceConfiguration.setSDKVersion("1.4.0", forService: "camp")
try NAFServiceConfiguration.setHeader("X-Request-Source", value: "ios", forService: "camp")

All four throw on an unknown service name, and passing nil clears a value. Headers set this way are applied to chat requests as well as asset resolution.

tip

If you're using the Partner SDK, GeniesAuthClient keeps NAF's token in sync for you. Set these by hand only when you're running NAF without it.

Metal device​

NAFContext.initialize(device:) sets the process-wide Metal device. NAFView and NAFAvatarSession do this for you on first use; call it yourself only if you need NAF on a specific device before either exists. NAFContext.device and NAFContext.isInitialized report the current state.

Loading avatars​

By identifier​

The identifier-based loaders take an ID and resolve everything else — the avatar document, its assets, its behavior — from the Genies backend. That backend is CAMP, the Content and Asset Management Platform: the services that store and serve avatars, wearables, and animations. The name runs through this API, in loadCamp…, NAFCampResolver, and equipCampWearable.

Resolution is per device as well as per avatar: the backend answers with the texture formats and channel counts this device can decode, so one identifier delivers different payloads to different phones. There's no variant for you to pick.

MethodLoads
loadCampAvatar(_:)An avatar as published. Also accepts a subspecies ID, loaded as a body-only avatar.
loadCampUserAvatar(_:)The same avatar with the signed-in user's saved state applied. Subspecies IDs are rejected.
loadCampSmartAvatar(_:)A smart avatar, with its behavior activated on the loaded rig.
loadCampUserSmartAvatar(_:)A smart avatar carrying the user's saved state, with behavior activated.

All four exist on both NAFView and NAFAvatarSession, and all four are async throws.

try await view.loadCampUserSmartAvatar(
smartAvatarID,
silhouette: true,
lods: [2, 0]
)

A smart avatar is an avatar with a behavior configuration attached — the thing that lets it idle naturally, react, and hold a conversation. The two smart-avatar loaders take talking:, breathing:, and emotion: asset IDs that override the clips its behavior would otherwise use; passing nil for any of them skips that override.

Load options​

Every loader shares the same options.

OptionDefaultWhat it does
silhouettefalseShow a flat low-detail stand-in while the real assets stream in.
lods[0]LOD progression. [2, 0] shows LOD 2 first, then upgrades to LOD 0 in the background.
idleNAFDefaultAssets.idleIdle clip. nil holds the bind pose.
behaviornilA NAFBehaviorConfig to activate after load. Not available on the smart-avatar loaders, which carry their own.
colorsnilAvatar-wide color overrides.
bodyColorsnilColor overrides scoped to the body.
assetColorsnilPer-asset color overrides, keyed by asset ID.
proceduralTexturesfalsetrue generates textures on the GPU as the avatar loads, which an avatar's saved wearable colors need to show. false binds the authored base color, which is what a wearable's catalog icon shows. Defaults to false from NAFKit 0.1.36.
dynamicsnilNAFDynamicsOptions. nil keeps the session's current settings.
sdkVersionnilPin the content SDK version. nil uses the service configuration.
resolvernilSupply your own NAFCampResolver.

NAFColorOverrides is [String: SIMD4<Float>] — attribute ID to RGBA.

NAFDefaultAssets carries the bundled idle, talking, and breathing clip references, each a NAFAssetRef(assetId:version:). They're vars, so you can repoint them globally.

By definition​

Before identifier-based loading existed, an avatar was described by a definition — a list of asset IDs plus shape attributes. That path is still supported.

let avatar = try NAFAvatar(fromJSON: json)
try await view.loadAvatar(avatar, mode: .behavior(silhouette: true), lods: [2, 0])

NAFAvatar holds assets: [NAFAssetEntry], shapeAttributes: [String: NSNumber], and an optional slug. NAFAssetEntry is an assetId plus a version.

NAFLoadMode picks how alive the avatar arrives:

CaseResult
.staticPoseGeometry only. The avatar holds still.
.animated(silhouette:)Idling, but nothing decides what it does next.
.behavior(silhouette:)Fully alive: idle, talking, breathing, gestures, emotions.

A second overload takes silhouette:, idle:, and behavior: separately instead of a mode. loadAvatarDocument(_:) takes an already-resolved avatar document as [String: Any], which is what you want if you resolved it yourself.

unloadAvatar() tears the current avatar down on either NAFView or NAFAvatarSession.

Resolving by hand​

NAFCampResolver is the resolution layer underneath the loaders. Use it when you want the resolved document before loading it — to cache it, inspect it, or load it later.

let resolver = NAFCampResolver()
resolver.resolveSmartAvatar(id, sdkVersion: nil) { result, error in
// result is the resolved document
}
MethodReturns
assetType(for:)NAFCampAssetType — .avatar, .wearable, .animation, .subspecies, or .unknown.
getCatalog(completion:)The full catalog.
getCatalog(type:ownership:limit:completion:)A filtered catalog page.
resolveItem(_:lods:sdkVersion:completion:)Any item, dispatched on its identifier prefix.
resolveAvatar / resolveUserAvatarAn avatar document, as published or with user state.
resolveSmartAvatar / resolveUserSmartAvatarA smart avatar document plus its behavior config.
resolveWearable / resolveSubspecies / resolveAnimationThe matching document.
resolveBehavior(_:sdkVersion:completion:)A behavior configuration.

init(callbackQueue:) moves completions off the main queue.

Wearables on their own​

To render a wearable with no avatar under it — a storefront tile, a detail view — load it standalone.

try await view.loadCampWearable(wearableID, lods: [0])
try await view.loadWearable(NAFAssetRef(assetId: id), lods: [0])
try await view.loadWearableDocument(document, assetId: id)

session.isStandaloneWearable reports whether the session is in this mode.

Wearables and composition​

What's on screen is one composed avatar, not separate models drawn on top of each other. Equipping or removing anything rebuilds it: the garment is fit to the body, meshes that would clash are resolved, and the colors, shader presets, dynamics, and layering overrides already in force are reapplied to the result. These calls are async, and the batch forms — equipAssets and replaceAll — pay for one rebuild instead of one per item.

Fitting is per avatar: a garment refits to the proportions of whoever wears it, so one asset serves bodies of different shapes. Rig recipes — the authored description of how a rig refits to a body — are reachable with setRigRecipeConfig(_:) and rigRecipeConfig().

Equip and unequip​

try await view.equipCampWearable(wearableID, lods: [0])
try await view.equip(NAFAssetRef(assetId: id), lods: [0])
view.unequip(assetId: id)

equipAssets(_:) takes a batch of NAFEquipItems — either .asset(ref, lods:colors:) or .campWearable(id, lods:colors:) — and applies them in one rebuild instead of one per item.

view.equippedAssetIds lists what's worn, in composition order. The body comes first.

Replacing​

Equipping something that conflicts with what's on already does the right thing implicitly, but when you want to be explicit about what comes off, use the replacement API. It removes and adds in a single rebuild.

try await view.replaceAssets([oldShirtID], withCampWearable: newShirtID)
try await view.replaceAssets([oldShirtID], with: nil) // remove only
try await view.replaceAll(items) // swap the whole outfit

Overloads take a NAFAssetRef, a list of NAFEquipItems, a CAMP wearable identifier, or a wearable document.

Layering​

Two things decide how worn assets sit together. Anchoring pulls a garment's mesh onto the surface beneath it; combination order decides which asset wins where they overlap. Assets author both, the backend can ship presets that override the authored values, and you can override either at runtime.

let anchors = try await session.anchorLayers()
let stiffened = try await session.setLayerStiffness(0.5, assetId: id) // false if the asset isn't anchored
_ = try await session.unsetLayerStiffness(assetId: id)

let order = try await session.combinationLayers()
_ = try await session.setCombinationOrder(3, assetId: id)
_ = try await session.unsetCombinationOrder(assetId: id)

NAFAnchorLayer reports assetId, meshId, the effective stiffness, the authoredStiffness, any presetStiffness, and whether it isOverridden. NAFCombinationLayer does the same for order.

anchoringSettings() and setAnchoringSettings(_:) read and write the global NAFAnchoringSettings — smoothingIterations, smoothingUnshrink, and skipLayering.

Colors​

Assets author color attributes — named slots a material exposes for recoloring. Setting one tints every material that declares it.

let attributes = try await session.getColorAttributes()
// attributes.global — avatar-wide attribute ids
// attributes.byAsset — attribute ids per asset id

let applied = try await session.setColor("primary", color: SIMD4<Float>(1, 0, 0, 1))
_ = try await session.setColor("primary", color: red, assetId: shirtID) // one asset only
_ = try await session.unsetColor("primary", assetId: shirtID)

All three return Bool — false means nothing declared that attribute — and take waitForRenders: to hold until the change is on screen. Colors set this way survive rebuilds — an equip, an unequip, or an LOD upgrade reapplies them.

Animation and behavior​

A loaded avatar is already alive: it idles, breathes, blinks, and looks around on its own. The APIs here are how you ask it for something more — play a gesture, hold an expression, change stance, speak — and how you read back what it's doing.

Gestures, emotions, and postures come from an animation library Genies authors and maintains. Clips are animated once and retargeted onto the Genies rig, then grouped under the tags you call by name — so the same request reads correctly on avatars built to very different proportions.

Each call below is described by what it does to the avatar. How NAF combines them internally isn't part of the API and changes between releases, so build against these. Direct animation control is the lower level, for when you're driving playback yourself.

Tags​

playGesture(tag:) and playEmotion(tag:) take a tag. A tag names a group of clips rather than a single one, and carries its own type and transition timings; an emotion tag also bounds a range of authored facial poses that intensity moves along, rather than scaling a single clip up.

Which tags an avatar supports depends on the avatar, and resolves from the backend at load rather than from the SDK — so read them instead of hardcoding a list. availableAnimationTags returns a NAFAvailableAnimationTags with gestures and emotions dictionaries: each NAFGestureAnimationTag reports tagType, specialGesture, isActive, transition times, and the animIds behind it; each NAFEmotionAnimationTag adds breathingSpeed and the faceMinAnimId / faceMaxAnimId that bound its range. view.gestureTags and view.emotionTags give the names alone.

Idle, talking, and breathing​

These three run continuously once an avatar is loaded. You choose the clip each one uses and how strongly it reads.

view.setIdle(clip)
view.setIdleWeight(1.0)

view.setTalking(clip)
view.setTalkingEnabled(true)
view.talkingWeight = 1.0

view.setBreath(clip)
view.breathWeight = 1.0
view.breathSpeedMultiplier = 1.2

isTalkingActive, currentTalkingWeight, and idleTime report the live state.

Gestures​

A gesture is a one-shot action — a wave, a nod, a shrug.

view.playGesture(tag: "wave")
view.playGesture(clip, options: NAFGestureOptions(blendInTime: 0.9, holdTime: 1.0))
view.playGUID(guid, startTime: 0)

NAFGestureOptions sets how long the gesture takes to come in and to go out, how long it holds at its peak, and isSpecialGesture. Both overloads take a completion: that fires when the gesture ends.

setAutoFidgetEnabled(_:) turns the small idle fidgets on and off; isAutoFidgetEnabled reads it.

Playing a clip you supply​

playGestureClip(_:options:completion:) plays a clip you loaded yourself as a gesture. The avatar treats it exactly like a tagged one — it takes over from a gesture already playing, and idle, talking, and expression carry on around it. What it adds is mirroring and per-blendshape facial control. NAFView and NAFAvatarSession both have it.

let options = NAFGestureClipOptions()
options.mirrorMode = .left // .right plays the authored side; .random picks one
options.defaultBlendShapeWeight = 1.0 // applies to every facial channel you don't name below
options.blendShapeWeights = ["Smile": 0.6] // names are the targets after mirroring

let accepted = view.playGestureClip(clip, options: options) {
// the gesture ended
}

The Bool says whether NAF took the request — not whether the gesture finished. On false nothing happened at all: whatever was playing keeps playing, and completion is never called. Check it, because a rejection doesn't throw.

A request is rejected when something in options is unusable — a weight outside [0, 1], a blendshape name that's empty or malformed, a weight that isn't a number, a time that isn't finite, or a mirrorMode that isn't one of the three. Naming a blendshape the avatar doesn't have is fine: that one is skipped and the rest still play.

Its timings are tighter than NAFGestureOptions by default: blendInTime and blendOutTime are 0.1 and holdTime is 0, against 0.9 / 0.9 / 1.0. An accepted gesture publishes NAFEventGestureStarted, then NAFEventGestureEnded when it finishes.

Emotions​

An emotion is an expression the avatar wears rather than an action it performs.

view.playEmotion(tag: "happy", intensity: 2)
view.setEmotion(clip, timeMin: 0, timeMax: 1)
view.setEmotionWeight(1.0, intensity: 0.5)

currentEmotion is the one in force; emotionClipInfo returns a NAFEmotionClipInfo for it.

Postures​

A posture is a whole-body stance — standing, seated, leaning. Transitions blend.

let available = view.postures
if view.transitionToPosture("seated") {
// view.isTransitioningPosture is true until it settles
}
// view.currentPosture, view.targetPosture

Where the avatar is paying attention, and what its head and eyes do about it.

view.setAttentionMode(.listening, blendTime: 0.3)
view.lookAtTarget = NAFLookAtTarget(head: point, eyes: point)
view.lookWeight = 1.0
view.gazeWeight = 1.0
view.isBlinkEnabled = true

NAFAvatarController.AttentionMode is .listening, .speaking, .bored, .thinking, .angry, or .custom. NAFLookAtTarget takes a head target and an optional separate eye target; omitting the eye target uses the head one. lookWeight and gazeWeight control how far the head and the eyes follow, independently.

On a headless session, set session.cameraPositionProvider so gaze knows where the viewer is, and session.avatarModelTransform so it knows where the avatar is. tick(deltaTime:renderer:) reads both per frame.

Body motion​

Body motion is how animated the avatar is while it talks — the sequences it uses, the beats it emphasizes, and the overall energy. It comes from a configuration document rather than from clips you play.

try view.setBodyMotionConfig(json: configJSON)
let config = try view.bodyMotionConfig()

try view.playSequence("nod", intensity: 1, speed: 1, modifier: 0)
try view.playEmphasis(0.8)
try view.stopSequences(duration: 0.2)

// view.isSequenceActive, view.sequenceWeight, view.bodyMotionActivity

bodyMotionDiagnostics() reports what's driving the body right now.

Behavior​

A smart avatar arrives with a behavior — the thing that decides when it fidgets, where it looks, how it carries itself. You can swap it after load, tune it, and read back what's active.

try await view.activateBehavior(NAFBehaviorConfig(slug: "friendly"))
try await view.setBehaviorConfigJSON(json)

NAFBehaviorConfig takes a slug, optional talking / breathing / emotion refs, useRightHanded, and either a configJSON or a behaviorId to resolve. behaviorConfig and behaviorConfigJSON read back what's active, and onBehaviorLoaded fires once it's live.

To read or patch a behavior's parameters while it runs:

let graphJSON = try view.serializeGraph()
try view.applyGraphParams(json: patchJSON)

The serialized form is the behavior's own representation, meant for tooling and debugging rather than as a format to build against.

IK and retargeting​

Retargeting fits animation authored on one skeleton onto this avatar's proportions; IK keeps hands and feet where they belong once it has.

view.isFBIKEnabled = true          // full-body IK
view.isFootIKEnabled = true // or per-foot: isFootIKEnabledLeft / Right
view.isAllRetargetEnabled = true
view.setRetargetEnabled(.layer1, enabled: false)

setRetargetEnabled(_:enabled:) turns it off for one animation slot — see Direct animation control.

IK targets​

IK targets pull a limb toward a goal you set, on top of whatever the animation is doing — a hand reaching for something, a foot planted on a step. The targets are .root, .leftHand, .rightHand, .leftFoot, and .rightFoot; check supportsIKTarget(_:) first, since it depends on the avatar's rig.

let session = view.session
guard session.supportsIKTarget(.rightHand),
let current = session.ikTargetPose(.rightHand) else { return }

let goal = NAFIKGoal()
goal.position = current.position + SIMD3<Float>(0, 0.15, 0.1) // avatar-local
goal.positionWeight = 1 // rotationWeight stays 0: the animation keeps the rotation

session.moveIKTarget(.rightHand, goal: goal, duration: 0.4) // ease there, then hold
// Later:
session.releaseIKTarget(.rightHand, duration: 0.4) // fade back to the animation

A goal's position and rotation (an XYZW quaternion) are in the avatar's local pose axes and units; .root moves the pose, not the object in your scene. positionWeight and rotationWeight blend from the animated pose, from 0 to 1; a weight of 0 leaves that part to the animation. positionWeight defaults to 1, rotationWeight to 0.

APIPurpose
setIKTarget(_:goal:)Holds the goal until you change or clear it.
moveIKTarget(_:goal:duration:)Moves there over duration seconds, then holds. A new move replaces one in progress.
releaseIKTarget(_:duration:)Fades back to the animated pose over duration seconds, then clears.
clearIKTarget(_:) / clearIKTargets()Clears at once.
ikTarget(_:)The current goal, mid-move included, or nil when none is set.
ikTargetPose(_:)Where the limb actually is now, both weights 1, or nil when the target isn't supported.
ikTargetRevision(_:)Changes with each new command; stable while a move plays.
hasActiveIKTargetsWhether any goal still needs updates.

The setters return false and leave the current goal alone for an unsupported target or an invalid goal. Moves and releases pause while IK is disabled. Requires NAFKit 0.1.38: earlier versions declare NAFIKGoal but don't export it, so creating one fails to link.

Pausing​

isAnimationPaused freezes animation while rendering continues; isRenderingPaused stops the draw loop. isAnimationActive and isAnimationAttached report whether there's anything to play.

If you loaded with .staticPose or idle: nil and want animation later, attachAnimation(dynamics:) brings the stack up after the fact.

Direct animation control​

Underneath everything above, animation plays in individually addressable slots that NAFAnimationChannel enumerates. Reach for these when you're driving playback yourself — scrubbing, syncing to timing that comes from outside NAF, or building tooling.

view.setLayerWeight(.layer1, weight: 0.5)
let weight = view.layerWeight(.layer1)
let info = view.channelInfo(.layer1) // NAFAnimationChannelInfo
view.setChannelCurrentTime(.layer1, time: 0)

NAFAnimationChannelInfo carries guid, status, duration, currentTime, speedMultiplier, isLooping, and isMirror. onChannelStateChanged fires with the channel, its old status, its new status, and the info whenever a channel transitions. channelWeights returns a NAFAnimationChannelWeights snapshot.

caution

Which slot carries which part of a performance is an implementation detail, and it changes between releases. Code written against the APIs above keeps working; code written against particular channels may not.

Dynamics​

Dynamics is physics-based secondary motion — hair, cloth, and accessories that sway and settle as the avatar moves. It's tuned per asset when the asset is authored, so the configuration that ships with a garment is the motion intended for it. NAF finds it during load, applies it, and reapplies it after every rebuild. You don't have to wire anything up.

// Turn it off for this load
try await view.loadCampAvatar(id, dynamics: .disabled)

// Or override the asset's configuration
let options = NAFDynamicsOptions(
enabled: true,
override: .json(myConfigJSON),
fallbackToAsset: true
)

NAFDynamicsOverride is .json(String), .url(URL), or .config(_:) from a dictionary. An override replaces the asset's configuration rather than adding to it; fallbackToAsset decides what happens when the override can't be used.

After load:

let state = await session.configureDynamics(options)
let state = await session.setDynamicsOverride(.url(url), fallbackToAsset: true)
let state = await session.clearDynamicsOverride()
// session.dynamicsOptions, session.dynamicsState

onDynamicsChanged fires when the active source changes.

Lower-level controls remain for hand-built setups: appendDynamicsGroup(_:), clearDynamics(), resetSimulation(), isDynamicsEnabled, dynamicsWeight, and hasActiveDynamics.

Voice​

NAFVoice drives the avatar's face from a voice service's reply, so its mouth and expression match what's being said.

let voice = NAFVoice()
try await voice.loadConfiguration(from: configURL)

if let shapes = voice.parseBlendshapes(response) {
voice.loadAnimation(on: view, frames: shapes.frames)
}
let tags = voice.parseTags(response) // emotion / posture cues with timings
let audio = voice.extractAudio(response) // base64 audio payload

Because response shapes differ between services, NAFVoiceConfiguration is field-mapped. configure(_:) takes a NAFVoiceConfigurationUpdate naming blendshapesPerFrame, frameRate, defaultGain, and the JSON keys for audio, blendshapes, tags, and tag fields.

NAFView, NAFAvatarSession, and NAFAvatarController all conform to NAFVoiceAnimationPlaying, so any of them can be the playback target. To drive the face directly:

view.setVoiceAnimation(blendshapeValues, gain: 1.0, voiceContinues: false)
view.stopVoiceAnimation()
// view.voiceAnimationWeight, view.voiceAnimationTime

Chat​

GeniesChat is the API most apps should use. NAFChatStream is the transport underneath it — reach for it when you're running NAF without the Partner SDK, or when you need control over requests, sessions, and audio buffering.

let stream = try session.makeChatPerformance()
stream.delegate = self

let request = NAFChatRequest()
request.configId = configID
request.input = "Hey"
request.mode = .voice
let generation = try stream.sendMessage(request, bearerToken: token)

makeChatPerformance() binds the stream to the loaded avatar so replies animate it. makeChatStream(baseURL:audioSink:) lets you point at a different service and take the audio yourself.

AreaAPI
SendsendMessage(_:bearerToken:), regenerate(_:bearerToken:), and sendRestMessage / regenerateRest for the non-streaming path
SessionsnewSession(configId:bearerToken:completion:), loadLatestSession(...), prepareSession(_:configId:bearerToken:completion:)
UtilitiestranscribeAudio(_:bearerToken:completion:), loadFollowupPrompts(_:bearerToken:completion:)
ControlcancelRequest(_:), stop(), invalidate(), updateAudioClock(...)
CapabilitysupportsNativeStreaming, supportsSessionManagement, supportsChatUtilities, supportsMotionChannels(_:fps:channelSet:)

NAFChatStreamDelegate reports didChangeState:, didBindResponse:, didReceiveText:, didReceiveCue:, didInterruptResponse:, and didFailWithError:. NAFChatState moves through .idle, .thinking, .buffering, .playing, .paused, .stalled, .completed, .interrupted, and .failed.

Text arrives as NAFChatTextBlocks carrying text, displayText, kind, timing, and per-character timings for karaoke-style display. Implement NAFChatAudioSink to own playback yourself.

For plain server-sent events outside chat, NAFSseStream.start(url:method:headers:body:bearerToken:onFrame:onState:) is a thin SSE client.

Rendering with NAFView​

Camera​

NAFView has a built-in orbit camera with pan and pinch gestures.

view.cameraTarget = .zero
view.cameraDistance = 3.0
view.cameraYaw = 0
view.cameraPitch = 0.15
view.cameraMinDistance = 0.3
view.cameraMaxDistance = 20
view.cameraRotateSensitivity = 0.005
view.cameraPanSensitivity = 0.003
view.cameraGesturesEnabled = true

view.frameAvatar() // fit the avatar to the viewport
view.onAutoFrame = { } // fires when framing settles

view.cameraControls exposes the underlying CameraControls if you want to drive it directly.

To place the avatar rather than the camera, set avatarPosition or avatarModelTransform.

Frame rate and scale​

PropertyDefaultNotes
preferredFramesPerSecond60Target frame rate.
renderScale1.0Render below native resolution to save power.
rendersOnDemandfalseDraw only when something changes. Call setNeedsRedraw() to request a frame.
adaptiveFrameRatefalseDrop to adaptiveIdleFPS when nothing is moving, rise to adaptiveActiveFPS when it is.
sampleCount4MSAA samples. Clamped to what the device supports.

Lighting and tone mapping​

view.lighting = SceneLighting.standard
view.lighting.ambient = AmbientLight(color: .one, intensity: 0.3)
view.lighting.directionals = [DirectionalLight(intensity: 1.5, direction: SIMD3(5, 5, 5))]

view.hdrEnabled = true
view.toneMapping = ToneMappingSettings(exposure: 1.0, mode: .acesFilmic)

ToneMapMode is .linear, .acesFilmic, .reinhard, .none, .cineon, or .agx. SceneLighting also carries a hemisphere configuration for sky/ground ambient.

Background and backdrop​

clearColor sets a solid background; setting backgroundColor to .clear makes the view transparent. EnvironmentMapLoader builds an environment texture from an HDR file.

A backdrop is a flat card behind the avatar that follows it around — useful for cutouts and share cards.

var backdrop = NAFAvatarBackdropSettings()
backdrop.enabled = true
backdrop.color = SIMD3(1, 1, 1)
backdrop.width = 3
backdrop.height = 3
backdrop.distanceBehindAvatar = 0.45
try view.setAvatarBackdrop(backdrop, texture: texture)

Color grading​

try view.setColorGradingLUT(image: lutImage, strength: 1.0)
try view.setColorGradingLUT(cubeFile: url)
try view.setColorGradingLUT(cubeText: cubeString)
view.colorGradingLUTStrength = 0.5

ColorGradingLUTLoader builds the texture by hand if you'd rather. Strip-layout images and .cube files are both accepted.

Outline​

PostOutlineSettings adds a screen-space outline pass with enabled, depthEnabled, depthThreshold, thickness, color, and the camera clip planes.

view.postOutline.enabled = true
view.postOutline.thickness = 1.5
view.postOutline.color = SIMD3(0, 0, 0)

Several avatars in one view​

let second = view.createAvatar()         // NAFViewAvatar
try await second.session.loadCampAvatar(otherID)
second.modelTransform = translation
second.isVisible = true
second.layerMask = 0b10
second.remove()

Each NAFViewAvatar has its own session, transform, visibility, debug toggles, and layer mask. view.renderLayerMask decides which layers draw.

Picking​

view.isPickingEnabled = true
view.pickPrecision = .mesh // or .bounds
view.pickScope = .mesh // or .avatar
view.onPick = { hit in
// hit?.instanceID, hit?.meshIndex, hit?.worldPoint, hit?.distance
}

.bounds tests the bounding box and is cheap; .mesh walks skinned triangles and is exact.

A translate gizmo is available with isGizmoEnabled, isGizmoVisible, and gizmoAxisLength.

Debug overlays​

showSkeleton draws the bone hierarchy, showBoundingBox draws the AABB, and rigColliderSettings draws the rig's collision shapes with enabled, xray, opacity, and selectedKey. session.rigColliders() lists them.

Managed renderer configuration​

The properties above are the convenient face of a single configuration object that the native renderer validates and applies atomically. Use configureRenderer(_:) when you want to set several things at once and know exactly what the device accepted.

let configuration = NAFRendererConfiguration(
environment: NAFRendererEnvironmentConfiguration(
backgroundMode: .solidColor,
toneMapping: .aces,
highDynamicRange: true
),
shadows: NAFRendererShadowConfiguration(enabled: true, mapResolution: 1024),
postProcess: NAFRendererPostProcessConfiguration(
enabled: true,
ssao: NAFRendererSsaoConfiguration(enabled: true, quality: .high),
bloom: NAFRendererBloomConfiguration(enabled: true, intensity: 0.4),
antiAliasing: NAFRendererAntiAliasingConfiguration(mode: .smaa)
)
)

let result = try view.configureRenderer(configuration, fallbackPolicy: .disableUnsupported)

NAFRendererConfiguration groups camera, directionalLights, environment, shadows, outline, silhouette, and postProcess. Any group you leave nil is left alone.

NAFRendererFallbackPolicy decides what happens when the device can't do what you asked: .strict rejects the whole configuration, .disableUnsupported turns off the parts it can't honour and applies the rest.

NAFRendererConfigurationResult reports status (.applied, .unchanged, .rejected), the appliedState, and diagnostics — each a severity, a NAFRendererDiagnosticCode such as .shadowsUnsupported or .shadowResolutionClamped, a path, and a message.

Ask before you configure with view.rendererCapabilities, which returns features, limits (max texture dimension, skin joints, morph targets, buffer sizes), and supported sampleCounts. view.rendererConfiguration returns the current NAFRendererSceneState.

Scene textures go in by slot:

try view.setRendererTexture(texture, for: .environment)

NAFRendererTextureSlot is .background, .environment, or .colorGradingLut.

view.rendererMetrics returns counters for publications, submissions, instance counts, and model bytes; resetRendererMetrics() zeroes them.

Shaders and toon presets​

Genies avatars are toon shaded. A toon preset is a JSON document of shader parameters, optionally scoped per mesh, per material, or per asset.

view.attachToonShader()
view.applyDefaultToonPreset()
view.apply(preset) // a ToonPreset
view.disableToonShader()
note

loadAvatar and loadUserAvatar from GeniesPartnerSDKNAF already apply the avatar's authored preset. Call applyDefaultToonPreset() yourself only on the direct loaders, or when you deliberately want the bundled look instead of the one authored for the asset.

session.sourceShaderPreset is the preset the backend shipped with the avatar; session.sourceAssetShaderPresets is the per-asset list. NAF keeps both in sync across rebuilds but leaves toon shading off until a preset is applied.

NAFShaderManager.shared is the live parameter interface. Attach a target, pick a mesh and material, then read and write uniforms:

NAFShaderManager.shared.attachAvatar(id: myID, view: view)
NAFShaderManager.shared.setActiveAvatar(id: myID)
NAFShaderManager.shared.setActiveMesh("Body", materialIndex: 0)

NAFShaderManager.shared.setToonFloat("rimPower", 2.0)
NAFShaderManager.shared.setToonVec4("baseColor", SIMD4(1, 0, 0, 1))
let value = NAFShaderManager.shared.getToonFloat("rimPower")

meshNames, materialNames, meshMaterials, and toonUniformNames tell you what's available. toonParams(forMesh:materialIndex:), surfaceParams(...), and outlineParams(forMesh:) read a whole parameter set at once. applyShaderPresetLayers(json:name:) applies a layered preset document.

ToonPresetExporter.export(name:baseMesh:snapshots:) turns the current state back into a preset document, reporting any materials that disagreed.

Bringing your own renderer​

NAF can own loading, animation, and skeleton math while your renderer owns the GPU. Conform to NAFAvatarRenderer:

public protocol NAFAvatarRenderer: AnyObject {
func updateBoneMatrices(for meshIndex: Int, matrices: NSData)
func updateMorphWeights(for meshIndex: Int, weights: NSData)
}

Then drive the session per frame:

let session = NAFAvatarSession()

session.onMeshesRebuilt = { meshes in myRenderer.applyMeshes(meshes) }
session.onSilhouette = { /* silhouette is on screen */ }
session.onAvatarReady = { lod in /* full avatar at this LOD */ }
session.onLodUpgrade = { upgrade in /* upgrade.lod, upgrade.assetIds */ }
session.onError = { error in }

session.cameraPositionProvider = { camera.eyePosition }
session.avatarModelTransform = uprightTransform

try await session.loadCampAvatar(id, silhouette: true, lods: [2, 0])

// per frame
session.tick(deltaTime: dt, renderer: myRenderer)

onMeshesRebuilt fires after every composition change — load, equip, unequip, LOD upgrade, silhouette swap — and hands you the new [NAFMeshData]. Rebuild your buffers from it. session.childMeshes is the current set, and session.meshUpdates is a Combine publisher of the same signal.

tick(deltaTime:renderer:) advances animation and uploads bone matrices and morph weights. If you'd rather split the two, update(deltaTime:) advances and uploadAnimationState(to:) uploads.

To pull the data yourself:

let bones  = session.boneMatrices(forMeshAt: index)      // Data of float4x4
let morphs = session.morphTargetWeights(forMeshAt: index)
let positions = session.bonePositions

session.boneNames, boneParentIndices, and poseJointNames describe the skeleton.

Mesh data​

NAFMeshData carries positions, normals, tangents, uvs, indices, jointIndices, jointWeights, inverseBindMatrices, jointCount, morphTargets, minBound / maxBound, and a primitives array. Each NAFMeshPrimitive has an index range, a NAFMaterial, and the sourceMeshIds / sourceAssetIds it came from — which is how you map a drawn triangle back to the wearable that contributed it.

isDirty and areMaterialsDirty tell you whether geometry or materials changed since you last read them.

NAFMeshData.makeMorphPositionDeltaBuffer(device:) and makeMorphWeightsBuffer(device:) (from NAFKitUITools) build the Metal buffers for you.

NAFKitUITools​

The pieces every custom renderer ends up rewriting:

TypeWhat it gives you
CameraControlsOrbit camera with pan, pinch, and frame(minBound:maxBound:fovY:aspect:) auto-framing. Publishes target, distance, yaw, pitch.
FrameContext / SceneMathShared FOV, perspective, and per-pass camera state.
AvatarBoundscombine(of:) for a composed entity's union AABB, transformed(min:max:by:) for coordinate fixes.
SkeletonDebugRendererBone-hierarchy line overlay, matching what NAFView.showSkeleton draws.
BoundingBoxDebugRendererAABB line overlay.
AvatarPickerray(screenPoint:viewportSize:frame:) and pick(...) against AvatarPickCandidates.
SceneInteractionControllerTap-to-pick and gizmo dragging wired to a host UIView.
TransformGizmoThe translate gizmo itself.
ColorMathrgb(fromHex:).

The built-in Metal driver​

If you want NAF to own the render loop but not the view, NAFRenderer is an MTKViewDelegate with onUpdate and onRender callbacks:

let renderer = NAFRenderer(device: nil)
renderer.configureView(mtkView)
renderer.onUpdate = { dt in session.tick(deltaTime: Float(dt), renderer: myRenderer) }
renderer.onRender = { encoder in myRenderer.render(encoder: encoder, context: frame) }

Assets and cache​

NAFContainerApi is the shared asset handle: it resolves URIs, downloads, decodes, and caches. Every session has one as session.containerApi, and you can share a single instance across sessions so they share a cache.

let api = NAFContainerApi(resolverConfig: NAFKitConfig.resolverConfig)
let session = NAFAvatarSession(containerApi: api)

Prefetching​

Warming the cache before you need an asset is the difference between a visible pop and a seamless change.

let warmed = try await api.prefetchAsset(wearableID, lods: [0])
let batch = try await api.preloadAssets(avatar, lods: [0]) // every asset an NAFAvatar names

api.prefetchAssets(requests) { progress in
// progress.completed, progress.total, progress.result
}

NAFAssetPrefetchRequest takes an id, version, lods, optional cache key, and sdkVersion. prefetchAssets reports progress as it goes; preloadAssets(_ avatar:) returns a NAFAssetPrefetchBatchResult with succeeded and failed counts and per-request results.

Animations​

let clip = try await api.loadCampAnimation(animationID)
let clip = try await api.loadAnimation(uri, version: "1")

NAFAnimationClip reports duration, track counts, sourceHeight, isLeftHanded, and — when the clip carries them — motion descriptors describing which body regions it drives and where contacts fall. skeletonNames and floatNames give the authored track names in native order — before target binding, masks, and mirroring; a clip with a nonzero track count but empty names binds by index instead. NAFAnimationClip.decode(_:) and .load(contentsOf:) read a clip from data or disk.

Cache control​

CallClears
NAFContainerApi.clearAllCaches()Everything.
clearTextureCache()Textures.
clearCombinableAssetCache()Avatar and wearable assets.
clearAnimationCache()Animation clips.
clearLoadedLods()The LOD residency table.
resetRuntime()The native runtime itself.

NAFContainerApi.assetCacheDirectory is where cached assets land.

addTexturePreference(_:hqTranscoding:fallbacks:) sets the texture format preference order, and resetTextureSettings() returns to the bundled defaults. setTargetCoordSpaceForward(_:up:right:) matches NAF's output to your engine's coordinate convention; it defaults to the space NAF renders in, forward +Z, up +Y, right -X.

Inspection and export​

let data = try view.exportGLB(format: .glb, includeTextures: true)

Build reports explain what the pipeline did to the avatar:

ReportReports
refitReportUtility meshes, cages, and how many meshes were deformed to fit the body.
anchorReportWhether anchoring ran, and how many meshes and surfaces it touched.
skeletonRootCorrectionReportWhether a root correction was applied, and to how many roots.

sourceRigRecipes() lists the rig recipes each asset shipped, rigRecipeOverrides() returns what's overriding them, and mergedRigRecipe() gives the result. utilityMeshes returns the non-rendered helper meshes.

session.baseAssetId is the body asset, session.detectedGeneration the avatar generation, and session.contentSourceState the resolved description of everything currently loaded.

Content authoring tools​

NAFContentToolsClient and the NAFContent* types are for authoring NAF assets, not consuming them — inspecting manifests, decoding packages, editing materials and extensions, previewing textures, and generating new asset revisions. NAFContentRevisionStore persists revisions locally.

Most integrations never touch this surface. Reach for it if you're building content tooling on top of NAF.

Events, errors, and diagnostics​

Errors​

NAF errors arrive as NSError in the NAFErrorDomain, with a NAFErrorCode:

RangeCodes
100sassetLoadFailed, assetNotFound, assetParseFailed, resolverNotReady
200savatarBuildFailed, skeletonSetupFailed, controllerUnavailable, builderUnavailable, colorIdMissing
300sanimationLoadFailed, animationParseFailed, animationCommandInvalid, animationCommandRejected
400sbehaviorLoadFailed, behaviorSlugMissing, behaviorConfigInvalid
500smetalDeviceUnavailable, metalQueueUnavailable, metalPipelineFailed, rendererStateRejected
600sserviceRequestFailed, serviceRequestCancelled
900snativeException, nativeUnknown

userInfo can carry NAFErrorURIKey, NAFErrorKindKey, NAFErrorTrackingIDKey, NAFErrorServiceKindKey, NAFErrorHTTPStatusKey, NAFErrorRequestIDKey, and NAFErrorTraceIDKey — the last two are what to quote when you file a bug.

Swift-level errors: NAFViewError (.metalDeviceUnavailable, .commandQueueUnavailable, .managedRendererUnavailable), NAFRendererConfigurationError, and NAFVoiceError.

Events​

NAFEventBus.shared publishes lifecycle events:

let subscription = NAFEventBus.shared.subscribe(.avatarReady) { event in
// event.userInfo
}
subscription.cancel()

NAFEventType covers asset load start/complete/fail, .avatarReady, gesture start/end, .emotionApplied, voice animation start/stop, idle entered, talking start/stop, and errors. subscribe(_:queue:handler:) picks the delivery queue.

NAFEventBridge.shared is a string-keyed bridge for events NAF doesn't model natively — subscribe(_:handler:), subscribeAll(_:), fire(_:payload:), and sendRequest(_:payload:timeout:completion:) for request/response.

LiveKit transport​

NAFKitLiveKit connects NAF to a LiveKit room for live conversational avatars.

NAFLiveKitTransport(room:agentIdentity:) carries behavior and chat messages over LiveKit data channels, on the behavior, agent.state, and lk.chat topics.

NAFLiveKitReceiverClock exposes the WebRTC playout clock so lip sync lines up with what the user actually hears, rather than with when the audio arrived. observe(track:) attaches it to a remote audio track; read(nearCaptureEpochMilliseconds:outputDelaySeconds:) returns a Sample for a given capture time.

GeniesChat in .live mode wires both of these up for you.