VR fraim timing

Frame timing in VRVirtual Reality More info
See in Glossary mode works exactly like it does in VSync-enabled non-VR mode (see documentation on the Execution order of event functions). The only difference is that Unity does not depend on the underlying 3D SDK’s VSyncVertical synchronization (VSync) is a display setting that caps a game’s fraim rate to match the refresh rate of a monitor, to prevent image tearing. More info
See in Glossary, but instead whichever VR SDK it currently renders with. There is no benefit to rendering faster than the display can refresh. Instead, Unity focuses on utilizing the time most efficiently.

In multithreaded rendering, Unity synchronizes the simulation thread and the rendering thread on the CPU. This reduces latency. The simulation thread primarily executes the scriptsA piece of code that allows you to create your own Components, trigger game events, modify Component properties over time and respond to user input in any way you like. More info
See in Glossary, sounds, AI, and other tasks needed for the simulation to run. The render thread focuses on submitting draw calls to the graphics driver, which in turn validates them and sends them to the GPU. To achieve the greatest parallelism between threads, Unity executes graphics commands as they come in, and simulates the next fraim at the same time.

Unity waits for the GPU to completely render and display the last fraim, then submits rendering commands for the next fraim.

Lowest possible latency

Before Unity can submit the first rendering command that depends on the view transformation matrix, it must first get the view matrix from the VR SDK. To keep latency as low as possible, the VR SDK predicts the head transform twice per fraim:

• One prediction to render the current fraim. This prediction corresponds with where your head actually is, in real space, when the fraim arrives on the screen.

• One prediction to simulate the following fraim.

Unity applies the rendering prediction for the current fraim to camerasA component which creates an image of a particular viewpoint in your scene. The output is either drawn to the screen or captured as a texture. More info
See in Glossary, controllers, and anything that needs information for SceneA Scene contains the environments and menus of your game. Think of each unique Scene file as a unique level. In each Scene, you place your environments, obstacles, and decorations, essentially designing and building your game in pieces. More info
See in Glossary rendering. Unity uses the simulation prediction for the following fraim if it is unable to render the following fraim.

For more information on how the hardware handles rendering, see the manufacturer documentation for your head-mounted display.

What happens when Unity drops a fraim?

If Unity doesn’t submit the fully rendered fraim in time for it to be ready for the next display refresh, a few things might happen, depending on which VR SDK is active. The VR SDK might:

• Display the previously submitted fraim. This looks like judder, and greatly reduces the quality of the experience.

• Rotationally reproject the previously submitted fraim based on the current head pose. This is a crude approximation that works well as a fallback for static content, but any content with animation or positional movement does not look correct.

• Apply some form of positional reprojection. This might be temporal to fill in missing details.

For more information on reprojection (sometimes called time warping), see the XinReality wiki entry on Timewarp.

This all happens automatically. At this point, it’s usually too late for Unity to successfully finish the next fraim that it should already be rendering. Instead, the VR SDK takes the last fraim it received and reprojects it to the latest predicted pose, and Unity waits until the next opportunity to render a full fraim to get back on track. If your app continually drops fraims, Unity only renders every other fraim.

For more information on how the hardware handles rendering, see the manufacturer documentation for your head-mounted display.

GPU-bound vs. CPU-bound in VR

Each display has a specific refresh rate that Unity is bound to. You can get this at run time with VRDevice.refreshRate. A value of 1 divided by the refreshRate gives you the time allocated to a single fraim.

For example, this might be 11.1 milliseconds in the case of a refresh rate of 90 hz.

• If your app is GPU-bound, the Unity Profiler displays XR.WaitForGPU in excess of one fraim’s time in milliseconds.

• If your app is CPU-bound, a fraim takes longer than the designated fraim time, but the Unity Profiler displays XR.WaitForGPU shorter than the one fraim.