I recently dove into 3D widgets in UE4 and ran into a few caveats/unknowns for my particular usage. Hopefully these points will save you some time in your own work.

UE4 VR Documentation & Resources

Initial Widget and Actor setup

I’ll forgo covering the basics of setting up 3D widgets as this is capably covered in Unreal documentation. Once you have an actor setup with a widget component attached to it consider the following additional notes…

  • You may not see the widget in editor or in game because the plane is not oriented the correct way. Be sure to adjust rotation of the widget so it faces the camera.
  • A widget component will default to 500x500 and everything will get crunched into the small space of this plane. You will likely want to increase the content size to match your actual widget size. For me this is often 1920x1080.

Structuring VR components on a Character

There are numerous ways you can structure components on a character to get your preferred result in game. Here are the requirements I was working towards for our project…

  • Moving the HMD maps to movement of the forward vector of the character
  • Moving the motion controllers control movement of the hands on your in name character
  • There should be an in game HUD for player health, points and/or other stats
  • When the 3D widget is showing interaction is done with the right hand only

This is the high level structure of components for the Character captured from the editor…

VR character structure

  • VRMenuWidget-Vision - Used for a UI like HUD that allows shows up in front of the player. We also use to pulse red when the players health is dangerously low.
  • VRMenuWidget - Widget used at runtime to show a widget to the player. This was mostly setup to show the pause menu to the player but could be used for other UI widgets as well.
  • VRMenuWidgetInteraction - Used to handle 3D widget interaction

Widget Component Properties

  • Set Tickable While Paused otherwise hover doesn’t work when selecting controls in VR
  • Child your menu widget to your FirstPersonCamera if you want the widget to track where the player is looking
  • Consider pausing the game before showing the menu and un-pausing once the player has taken action

Gotchas and caveats

Collisions

Be aware of collisions on the widget itself. 3D widgets do seem to require a measure of collision to work and in fact there is a Collision Preset specifically for UI that may need to be set for a UI to work. What may NOT be needed is the property Simulation Generates Hit Events. In my case I was finding having this enabled this was triggering other logic of the game. i.e. the widget is colliding with a torch/fire object, the menu widget is attached to the player character so the player character is on fire. Turning off Simulation Generates Hit Events resolved this.

Enforcing widget visibility

UMG widgets will clip the geometry inherently in the case where a 3D widget is always placed in front of the player. This creates the problem of not necessarily seeing the UI because the widget can end up embedded in a wall or other in game geometry when the player turns their head. I didn’t explore solutions to this too deeply but did find at least one basic solution to set a material to always render in front of geometry. This material can then be applied to the UMG widget and you get something that “generally” works and should allow the player to still interact with the widget.

  1. Make a copy of the Widget3DPassThrough material from your Engine directory to your project directory
  2. Open the copied material asset file look for Element0 and click Disable depth test

WidgetInteraction component debug pointer

Usable in development builds only. Won’t show up in Shipping builds but helpful for knowing where your pointer is at and if it’s properly interacting with 3D widgets.

Show/hide an in game menu

If working on a VR title consider using the controller “hamburger” button (the button with the three horizontal lines)

Oculus controller menu button

Widget Interaction control selection

Unreal translates VR motion controller button presses into effectively mouse clicks at least I seem to recall as much from documentation. As noted in documentation make use of the following blueprint nodes…

  • Press Pointer Key
  • Release Pointer Key

Be aware there are also nodes…

  • Press Key
  • Release Key

I lost some cycles making use of Press Key/Release Key and not realizing the were the wrong blueprint nodes to use.

Structuring 3D widgets in a main menu level for PC

Main menus in most games don’t allow any player movement/control past menu item selection i.e. Play, Settings, Quit, etc. UE4 Level structure allows a fairly clean way to do this.

  1. Create a new GameMode object
  2. Create a new Pawn
  3. Create a new PlayerController
  4. Open the GameMode
    1. Player Controller should be set to the new Player Controller
    2. Default Pawn Class should be set to the new Pawn class

In the level WorldSettings set GameMode override to your new GameMode

Setting Keyboard/Gamepad Focus

There are situations where setting keyboard and gamepad focus on a widget is useful and/or required. The user may also simply prefer keyboard or gamepad over mouse usage.

An initial approach to setting focus

After Event Begin Play I tried running Set Keyboard Focus. This didn’t seem to work. None of the controls in the widget seemed to capture focus. I could click on the widget which would capture focus and could then navigate between widget components. If I clicked off the widget this would remove focus and I wouldn’t be able navigate between widget controls. It seemed the initial call to Set Keyboard Focus wasn’t doing anything at all.

A second approach

Some folks in forum discussions suggested setting the keyboard focused widget every tick. Visually I found this sort of works… only the first control in the widget was focused/highlighted but I wasn’t able to switch controls in the widget. I’m not entirely certain why I wasn’t able to switch to other controls but likely there is an initialization done when setting keyboard focus that only needs to be done once.

Another forum post suggested setting a delay before setting keyboard focus. I don’t believe this type of solution is clean as it skirts the root cause of why a delay is required at all. This did work however!

Given this success I moved ahead to using a timer that constantly sets the focus to the widget at a timer rate of 0.5 seconds. The implementation is detailed below…

Blueprint for