Creator SDK

Authoring and runtime SDK for XR medical-training simulations.

Component View: Creator SDK - Runtime SDKModules that make up Creator SDK's Runtime SDK.Creator SDK[Software System]Runtime SDK[Container: Unity / C# (.NET) — Runtime asmdefs from core, experimental, and omen packages]CoreExtraGeometric Algebra[Component: C# — GA asmdef: PGA/PgaMotor +CGA/{R410, CgaSpatialMath}] Geometric-algebra math core: PGAmotors for rigid-transforminterpolation and CGA (R4,1)multivectors for spatial queries.Hub[Component: C# — MAGES.Hub.Runtime] Wires modules together and drivestheir lifecycle.JARIA Client[Component: C# — MAGES.JARIA.Runtime] Sends conversation turns to theJARIA language model.Speech Backend[Component: C# —MAGES.Experimental.EmbodimentJARIA.MicrosoftCognitiveServices.Runtime] Pluggable speech-to-text andtext-to-speech provider.Microphone[Component: C# —MAGES.Hub.{Web,NonWeb}Microphone] Provides the audio input stream;build target selects animplementation.Character Controller[Component: C# —MAGES.CharacterController.Runtime] Drives avatar body and headanimation.Generic IK[Component: C# — MAGES.GenericIK.Runtime] Solves inverse kinematics forprocedural avatar posing.Mesh Deformations[Component: C# —MAGES.MeshDeformations.Runtime] Soft-body deformation and cuttingfor tissue simulation.Experimental SceneGraph[Component: C# —MAGES.Experimental.SceneGraph.Runtime] Extends SceneGraph with extraaction types and effects (Place / Hold/ Click / Impact / Delay).Analytics[Component: C# — MAGES.Analytics.Runtime] Captures training performancemetrics for review and grading.Data Container[Component: C# — MAGES.DataContainer.Runtime] Shared session-scoped storage forcross-module data.SceneGraph[Component: C# — MAGES.SceneGraph.Runtime] Runs the authored proceduralsimulation and broadcasts actionprogress.Interaction[Component: C# — MAGES.Interaction.Runtime] Turns hand and controllermanipulations into simulationevents.Device Manager[Component: C# — MAGES.DeviceManager.Runtime +drivers] Abstracts the input device so othermodules remain platform-agnostic.Networking[Component: C# — MAGES.Networking.Runtime] Synchronises simulation state acrossmultiplayer participants.Localization[Component: C# — MAGES.Localization.Runtime] Localises action text and UI strings.Collision Sound[Component: C# — MAGES.CollisionSound.Runtime] Plays contextual sound effects whenobjects collide.Recorder[Component: C# — MAGES.Recorder.Runtime +optional capture asmdefs] Captures full session playback forlater review.UI[Component: C# — MAGES.UIs] In-headset menus, notifications, andlicense prompts.JARIA[Component: C# —MAGES.Experimental.EmbodimentJARIA.Runtime] Drives in-scene AI characters:listens, consults the LM, speaks theresponse.GenAI Module[Component: C# — MAGES.GenerativeAI.Runtime] Runtime configuration for OMEN'sgenerative-AI services.Trainee Performs a medical-training scenario inthe simulator.ORamaVR Cloud[Software System] ORamaVR's hosted backend: identityat login.oramavr.com, AI agentplatform at api.oramavr.com,content portal atportal.oramavr.com, and analytics atanalytics.api.oramavr.com.Azure Speech[Software System] Cloud speech-to-text andtext-to-speech.Photon Cloud[Software System] Hosted relay for multiplayer statesynchronisation.Unity XR Plugins[Software System] OpenXR, Oculus XR, XRManagement, Input System, and ARFoundation — headset, controller,and hand-input backends.Unity DOTS Stack[Software System] Burst compiler, Job System,Collections, and Mathematics —high-performance compute used bythe XPBD soft-body solver.Dissonance VoIP[Software System] In-session voice-chat asset; its audiois captured into session recordings.CharacterController → SceneGraph — Contributes SetCharacterControllerStepContributes SetCharacterControllerStepMeshDeformations → SceneGraph — Contributes Suturing ActionData + effectContributes Suturing ActionData + effectExperimentalSceneGraph → SceneGraph — Contributes ActionData + effect implementorsContributes ActionData + effect implementorsExperimentalSceneGraph → Interaction — Queries grabbable / hand stateQueries grabbable / hand stateExperimentalSceneGraph → DeviceManager — Queries camera heightQueries camera heightJARIA → GeometricAlgebra — Computes user-relative spatial context (CGA points / planes)Computes user-relative spatial context (CGA points / planes)JARIA → JARIAClient — Sends EmbodiedAIRequest / JariaRequestSends EmbodiedAIRequest / JariaRequestJARIA → SpeechBackend — STT/TTS via speech handler interfacesSTT/TTS via speech handler interfacesJARIA → Microphone — Captures the microphone audio streamCaptures the microphone audio streamTrainee → JARIA — Speaks; receives voice + subtitles + animationsSpeaks; receives voice + subtitles + animationsJARIAClient → ORamaVRCloud — Sends conversation turns [HTTPS / JSON]Sends conversation turns[HTTPS / JSON]JARIA → ORamaVRCloud — Reads tenant token budget [HTTPS / JSON]Reads tenant token budget[HTTPS / JSON]Recorder → ORamaVRCloud — Uploads session recordings [HTTPS]Uploads session recordings[HTTPS]SpeechBackend → AzureSpeech — Speech-to-text + text-to-speech [Azure Speech SDK]Speech-to-text + text-to-speech[Azure Speech SDK]Networking → PhotonCloud — Joins rooms; relays state + events [Photon PUN / UDP]Joins rooms; relays state + events[Photon PUN / UDP]DeviceManager → UnityXRPlugins — Reads headset / controller / hand input through the XR driver layer [Unity.XR.Management + Unity.InputSystem]Reads headset / controller / hand input through the XR driver layer[Unity.XR.Management + Unity.InputSystem]Recorder → DissonanceVoIP — Captures in-session voice chat into recordings [Dissonance Voip capture asmdef]Captures in-session voice chat into recordings[Dissonance Voip capture asmdef]Hub → SceneGraph — Owns + ticks the module lifecycleOwns + ticks the module lifecycleHub → Interaction — Owns + ticks the module lifecycleOwns + ticks the module lifecycleHub → DeviceManager — Owns + ticks the module lifecycleOwns + ticks the module lifecycleHub → Networking — Owns + ticks the module lifecycleOwns + ticks the module lifecycleHub → Analytics — Owns + ticks the module lifecycleOwns + ticks the module lifecycleHub → DataContainer — Owns + ticks the module lifecycleOwns + ticks the module lifecycleHub → JARIA — Owns + ticks the module lifecycleOwns + ticks the module lifecycleHub → Localization — Owns + ticks the module lifecycleOwns + ticks the module lifecycleHub → CollisionSound — Owns + ticks the module lifecycleOwns + ticks the module lifecycleHub → Recorder — Owns + ticks the module lifecycleOwns + ticks the module lifecycleHub → UI — Owns + ticks the module lifecycleOwns + ticks the module lifecycleHub → GenAIModule — Owns + ticks the module lifecycleOwns + ticks the module lifecycle
Component View: Creator SDK - Runtime SDK
Modules that make up Creator SDK's Runtime SDK.
LegendUser cardSoftware systemExternalCore moduleAI moduleExtra moduleExperimentalRelationship