Interaction & Locomotion
In a flat application the interface is a layer on top of the content. In VR the interface is the content — you reach into it, walk through it, and point at it with your body. This unit covers how people select, manipulate, travel, and find their way in a virtual space, and why the obvious solution is so often the wrong one.
The Core Tasks
Almost every interaction in a virtual environment decomposes into a small set of universal tasks. The value of the decomposition is practical: each task has its own literature, its own known techniques, and its own characteristic failure modes.
Selection
Specifying which object you mean. Sounds trivial; becomes hard the moment the object is far away, small, moving, or occluded by something else.
Manipulation
Changing an object's position, orientation, or scale once you have it. Six degrees of freedom at once is more than most people can control precisely.
Travel
Moving your viewpoint through the environment. The motor component of navigation, and the single biggest comfort decision in any VR project.
Wayfinding
The cognitive half of navigation: knowing where you are, where things are, and how to get back. Distinct from travel, and routinely neglected.
System control
Changing the mode or state of the application — menus, tools, settings. There is no desktop metaphor to fall back on here.
Symbolic input
Entering text, numbers, or other abstract symbols. Still genuinely unsolved in VR, and usually worth designing around rather than solving.
The central tension
Interaction techniques trade off along one axis more than any other: naturalism versus efficacy. A perfectly natural technique — reach out and grab the thing with your hand — is instantly learnable and highly present, but limits you to arm's reach and to human levels of precision. A "magic" technique — a laser pointer that reaches across the room, an arm that stretches, a miniature copy of the world you can manipulate from above — breaks realism and buys enormous capability.
Neither end is correct. The question is always which one the task actually needs.
Selection & Manipulation
The main techniques
Virtual hand natural
A one-to-one mapping: your tracked hand is a hand in the world, and you select by touching or grabbing. Maximally intuitive, excellent for embodiment, no learning curve. Limited strictly to arm's reach, and inherits every bit of your own imprecision.
Ray casting / pointing magic
A ray from hand or controller; the first object it hits is selected. Reaches anything visible, cheap to implement, and by far the most common technique in shipping VR. Its weakness is geometric: angular error at the hand becomes positional error at the target, growing linearly with distance.
Cone casting / flashlight magic
Widens the ray into a cone and disambiguates among whatever it catches — by proximity to the cone axis, or with a follow-up choice. Much more forgiving for small or distant targets, at the cost of needing a disambiguation rule.
Go-Go (arm extension) hybrid
Maps hand distance non-linearly: inside a threshold of roughly two-thirds of arm's reach the mapping is one-to-one, and beyond it the virtual arm extends far faster than the real one. Keeps the naturalism of grabbing while reaching across the room.
World-in-miniature magic
A doll's-house copy of the environment held in your hand. Manipulating a miniature moves the real thing. Superb for large-scale rearrangement and for maintaining an overview, clumsy for fine detail.
Gaze + pinch hybrid
Eye tracking selects the target, a small finger pinch confirms. Extremely low effort and fast, because the eyes are already on the target before any hand moves. Needs reliable eye tracking, and suffers when targets are close together.
Why selection at distance is hard
Hold your arm out and try to hold it perfectly still. You cannot — physiological tremor is on the order of a few tenths of a degree. At 0.5 m that is a couple of millimetres of wobble and irrelevant. At 10 m the same angular error is several centimetres, which is larger than many targets.
Two more effects make it worse in practice:
- The Heisenberg effect. The act of pressing the button to confirm a selection physically jolts the controller, moving the ray off target at the exact instant it matters. Mitigate by latching the target on button down rather than up, or by freezing the ray briefly at the moment of the press.
- Depth is not part of the ray. Ray casting picks a direction, not a distance, so it selects fine but manipulates badly — pushing an object farther away along a ray is unnatural and imprecise. Most systems switch technique after selection.
Effective positional error ≈ distance × tan(angular error)
And difficulty follows Fitts's law, with an index of difficulty of log₂(D/W + 1) for a target of width W at distance D. Doubling the distance or halving the target size costs about the same. This is why "just make the laser pointer thinner" never helps.
Demo: Selection at a Distance
Top-down view of a user and a target. Watch how a fixed amount of hand tremor turns into a growing error footprint as the target gets farther away — and how the other techniques respond.
Locomotion
How the user's viewpoint moves through a space larger than the room they are standing in. This is the decision that determines who can use your experience without feeling ill, and it is worth making before almost anything else.
The families
Physical
The user's real body does the work.
- Real walking — the gold standard for presence and spatial awareness, and essentially free of sickness. Bounded by your tracked space.
- Walking in place — marching on the spot, detected by head bob or trackers. Keeps much of the proprioceptive benefit without the floor space; feels odd and is tiring.
- Redirected walking — real walking, subtly steered. See the next section.
- Treadmills and slides — specialised hardware, effective, expensive and rarely practical outside labs and arcades.
Artificial
The world moves instead of the user.
- Teleportation — point, blink, arrive. No vection at all, so essentially no sickness; costs spatial continuity.
- Dash — a very fast but continuous hop to the target. A compromise that keeps some continuity with little exposure.
- Continuous / smooth — stick-driven walking. Most familiar to gamers, most immersive for exploration, and the largest comfort risk.
- Steering — direction set by gaze, hand, or torso. Torso-directed feels most natural; gaze-directed prevents looking around while moving.
- Vehicles — comfortable out of proportion to their motion, because a cockpit is a built-in rest frame.
The tradeoff matrix
No technique wins on every axis, which is why shipping products offer several and let the user choose.
| Technique | Comfort | Presence | Spatial awareness | Precision | Space needed |
|---|---|---|---|---|---|
| Real walking | Large | ||||
| Redirected walking | Medium–large | ||||
| Walking in place | Minimal | ||||
| Teleportation | Minimal | ||||
| Dash | Minimal | ||||
| Smooth locomotion | Minimal | ||||
| Vehicle | Minimal |
Ratings are qualitative and reflect common practice rather than a specific study. The pattern that matters is the shape: comfort and spatial awareness pull in opposite directions, and teleportation buys its comfort by paying in continuity.
Why teleportation costs spatial awareness
Continuous motion continuously updates your sense of where you are — a process called path integration that runs largely below conscious awareness. Teleporting removes the transition entirely, so there is nothing to integrate, and users routinely become disoriented after a few jumps even though each individual jump felt fine.
Mitigations: keep the destination visible before committing, preserve orientation across the jump rather than reorienting the user, use a short arc or dash instead of an instant cut, and lean harder on wayfinding aids than you would with smooth locomotion.
Demo: Locomotion Comparison
The same journey down a corridor, rendered three ways. Watch the optical flow — the amount of visual motion sweeping past you is essentially the amount of sickness risk you are taking on.
Narrows the periphery during motion.
Redirected Walking
Real walking is the best locomotion technique by every measure except one: it needs a room the size of the virtual world. Redirected walking attacks that constraint by exploiting a quirk of perception — vision dominates your sense of self-motion so strongly that it can be used to steer you without your noticing.
The gains
Rotation gain
Rotate the virtual world slightly more or less than the user's real head turn. They compensate automatically, ending up facing a different real direction than they think.
Translation gain
Scale virtual movement relative to real movement, so a short real walk covers a longer virtual distance — or the reverse.
Curvature gain
Continuously rotate the world slightly as the user walks "straight." They unconsciously correct, and their real path bends into a circle while the virtual path stays straight.
Applied below perceptual detection thresholds, none of these are noticed. Published thresholds vary with conditions, but the commonly cited approximations are that users can be rotated roughly 20% less or around 50% more than they believe, translation can be scaled by something like ±15%, and a "straight" path can be curved onto a circle of roughly 20 m radius before people catch on.
Treat those numbers as order-of-magnitude. Thresholds depend heavily on walking speed, visual richness, distraction, and whether the user is looking for the effect — and a user who has been told about redirection detects it far more readily.
When gains are not enough
- Resets. When the user approaches a physical wall anyway, interrupt: ask them to turn in place (rotating the world faster than they turn so they end up re-aimed into the open space), or use a distractor to buy the rotation.
- Impossible spaces. Overlap virtual rooms so they occupy the same physical volume. People are remarkably bad at noticing that two adjacent rooms could not geometrically fit.
- Change blindness redirection. Move a doorway or corridor while it is out of view. Users almost never notice the architecture rearranging behind them.
- Scaled environments. Shrink the virtual world, or the user, so less real walking is required.
Demo: Curvature Gain
The user believes they are walking a straight line. Increase the curvature gain and watch their real path curl to fit inside the room — until the gain exceeds the detection threshold and the illusion breaks.
Smaller radius bends the real path harder.
Wayfinding
Travel is the motor problem. Wayfinding is the cognitive one — building and maintaining a mental model of the space. A user who can move perfectly but has no idea where they are is still lost.
How spatial knowledge builds
Landmark knowledge
Recognising distinctive features. "There's the red tower." Arrives first and fastest.
Route knowledge
Sequences connecting landmarks. "Left at the tower, then straight to the bridge." Brittle — it fails as soon as you are off the route.
Survey knowledge
A map-like model supporting shortcuts and novel paths. Slowest to build, most robust once it exists, and the thing teleportation most interferes with.
Aids worth designing in
Distinctive landmarks
Visible from a distance, visually unique, and distributed. The cheapest and most effective aid there is — a corridor of identical rooms is a navigation trap.
Maps and minimaps
Decide between track-up (rotates with you, easier for immediate turns) and north-up (fixed, better for building survey knowledge). You are choosing which kind of knowledge to support.
World-in-miniature
A map you can hold, look around, and often teleport with by pointing. Doubles as a travel technique.
Trails and breadcrumbs
Showing where you have already been prevents the classic loop of re-exploring the same wing three times.
Compass and grid
A persistent global orientation reference. Cheap insurance in any environment where users teleport.
Signage and architecture
Real buildings solve wayfinding with sightlines, colour-coded zones, and consistent layout logic. Borrow all of it.
Menus & Text Entry
Where to put a menu
World-fixed
A panel anchored in the environment. Stable, easy to point at, shareable with other users — but you have to walk back to it.
Hand- or wrist-attached
A tablet in the off hand, or a watch on the wrist. Always available, naturally dismissible by dropping your arm, and it uses proprioception — you know where your own hand is without looking.
Radial / pie menus
Options arranged in a circle around the cursor. Direction is easier than position, so these are fast and become muscle memory. Limited to a handful of items per ring.
Tool belt
Items stowed at body-relative locations — hip, shoulder, chest. Highly learnable, very fast once learned, and strongly embodied. Invisible to newcomers without onboarding.
Head-locked
Welded to the view. Almost always wrong: it cannot be looked away from, it fights the user's neck, and it destroys the illusion of a place.
Direct manipulation
No menu at all — grab the tool, turn the dial, press the physical-looking button. The most VR-native answer, and the reason so many good VR apps have almost no UI.
Text entry is still bad
Every VR text entry method is substantially worse than a keyboard, and it is worth treating that as a design constraint rather than a problem to solve. Approximate rates, which vary widely by study and by user practice:
| Method | Rough rate | Notes |
|---|---|---|
| Physical keyboard via passthrough | ~40–60 WPM | By far the best option. Needs a desk and a seated setup. |
| Speech | ~100+ WPM raw | Fast but error-prone, awkward in public and around others, and poor for passwords or code. |
| Controller pointing at a virtual keyboard | ~10–15 WPM | The default. Slow but reliable and needs no extra hardware. |
| Controller "drum" typing | ~15–20 WPM | Tapping keys with two controller tips. Faster, more tiring. |
| Hand-tracked poking | ~10–13 WPM | No controller needed; suffers from no touch feedback and tracking dropouts. |
| Phone as keyboard | ~30 WPM | Pragmatic and surprisingly common in shipping apps. |
The practical lesson: design so text entry is rare. Use selection from lists, voice for search, saved profiles, QR or companion-app pairing for logins, and never require a password to be typed in a headset if you can avoid it.
Making It Feel Right
The difference between VR interaction that feels magical and VR interaction that feels like fighting a puppet is mostly in details that never appear in a feature list.
Feedback is not optional
In the real world, touching something produces instant, multi-channel confirmation. In VR your hand passes through solid objects silently. Every interaction therefore needs deliberately authored feedback, ideally on more than one channel:
- Visual — highlight on hover, change on grab, a clear held state. The minimum bar.
- Audio — a click, clunk, or snap. Astonishingly effective at selling contact, and cheap.
- Haptic — a short controller pulse on hover and a firmer one on grab. Even crude vibration substitutes convincingly for touch.
- Pseudo-haptics — using visuals to imply physical properties: a heavy object that lags your hand slightly reads as heavy, without any force feedback at all.
Passive haptics
If a virtual table is aligned with a real table, users can lean on it, and the effect on believability is out of all proportion to the effort. Physical props registered to virtual objects remain the most convincing haptics available — the limitation being that the physical world cannot change as fast as the virtual one.
Forgiving interaction beats accurate interaction
- Generous colliders. Make grabbable volumes larger than they look. Nobody has ever complained that an object was too easy to pick up.
- Snapping and pose authoring. Let the object snap to a sensible orientation in the hand rather than preserving the exact grab transform. A held pistol should look held.
- Assume tracking will drop. Hands leave the camera frustum constantly. Fail gracefully — do not drop the user's held item because a thumb was occluded.
- Respect physical limits. Do not require reaching above the head, behind the back, or to the floor. Someone using your experience is seated, or has limited mobility, or is in a small room.
- Two-handed where it is natural. Scaling by pulling apart, aiming a long object with both hands — bimanual interaction is intuitive and underused.
Accessibility
Interaction design decisions are accessibility decisions, usually without anyone noticing:
- Offer seated and standing modes, with a height calibration that is not just "stand up straight."
- Support one-handed play, and let either hand be dominant.
- Provide snap turning and teleport as first-class options, not as buried "comfort" settings.
- Never require fine motor precision for a core action; offer dwell or hold as an alternative to a fast click.
- Keep required reach inside a modest envelope, and make anything critical reachable without turning around.
- Caption audio, and do not rely on audio alone to signal something behind the user.
Self-Check
Work these out before expanding the answers.
Users keep selecting the wrong object when pointing at a control panel across the room. Name two fixes that do not involve moving the panel.
Switch from ray casting to cone casting with a disambiguation rule, so near-misses still resolve to the intended target; and latch the selection on button-down rather than button-up, to defeat the Heisenberg effect. Increasing effective target size — larger colliders than the visible geometry — is a third.
Why does teleportation reduce sickness but increase the chance users get lost?
Sickness comes largely from visual motion your vestibular system does not corroborate; teleporting shows no motion at all, so there is no conflict. But the same absence of motion removes the continuous self-motion signal your brain uses for path integration, so survey knowledge never accumulates and users become disoriented.
Your experience needs 30 m of virtual walking in a 6 × 6 m room. What are your options?
Redirected walking with curvature and rotation gains, accepting that resets will still be needed at that room size; impossible spaces or change-blindness redirection to reuse the physical volume; scaling the environment or the user down; or abandoning real walking for teleport or smooth locomotion. In practice a combination — real walking within the room, teleport between regions.
When is a "magic" technique the wrong choice even though it is more capable?
When the goal is training a real-world skill, or when presence and embodiment are the point of the experience. A surgical trainee who selects tissue with a laser pointer is learning the wrong motor task. Naturalism matters most when the virtual action is supposed to transfer to a real one.
A prototype requires the user to type a twelve-character password with a raycast keyboard. What do you tell the team?
At roughly 10–15 WPM with high error rates and no touch feedback, this will be slow and infuriating, and hiding characters removes the error correction users rely on. Design it away: pair with a companion app or QR code, use an account already signed in on the device, or authenticate on a phone. Text entry in VR should be rare by design.