VR Hardware: How a Headset Works
A VR headset is a machine for winning a race against your own nervous system. Every component exists to answer one question fast enough and sharply enough to be believed: where is your head, and what should your eyes see from there? This unit follows that signal from the tracking sensors to your retina.
The Motion-to-Photon Pipeline
Every frame you see in VR is the end of a chain that started with a sensor reading a few milliseconds ago. The whole chain is called motion-to-photon latency, and the target is roughly 20 ms or less — past that, the world feels like it is sliding around on your head, and your vestibular system starts to object.
Why prediction is mandatory
No system is fast enough to render where you are. It renders where you will be. The tracker samples your head pose, estimates velocity and acceleration, and extrapolates forward to the moment those photons will actually hit your eye. Prediction error is why fast, jerky head motion feels worse than smooth motion: the extrapolation is only as good as its assumption that you keep doing what you were doing.
Demo: Latency Budget
Adjust the frame rate and see how the budget fills up — and how far the world "slips" during a brisk head turn before the display catches up.
Re-warps the rendered frame against a fresher pose right before scanout.
Displays
A VR panel sits a few centimeters from your eye and gets magnified across your whole visual field. That magnification is unforgiving: it stretches every pixel, every gap between pixels, and every millisecond a pixel stays lit.
Panel technologies
LCD
Cheap, bright, high subpixel density, no burn-in. Backlit, so blacks are grey — a real cost in dark scenes. Slower pixel response makes low-persistence strobing harder.
Quest 2, Quest 3, Quest 3S
OLED
Per-pixel emission gives true blacks and near-instant response, ideal for the brief illumination pulses VR wants. Historically lower fill factor, so the screen-door effect shows up sooner at equal resolution.
Rift CV1, PSVR2
Micro-OLED
OLED grown on a silicon backplane: tiny, extremely dense, very bright. The enabling part for compact pancake optics and glasses-like headsets. Expensive, and small panels demand precise lens alignment.
Vision Pro, Bigscreen Beyond, Galaxy XR
Resolution, and why "4K per eye" means little
Raw pixel counts are almost meaningless in a headset, because the same pixels can be sprayed across a narrow or an enormous field of view. The number that matters is pixels per degree (PPD) — how finely the display samples one degree of your visual field.
Human vision with 20/20 acuity resolves detail at roughly 60 PPD. Most 2026 consumer headsets land between 15 and 35. This is the single biggest remaining gap between VR and looking at the real world — and it is why headsets that widen FOV without adding pixels feel blurrier, not more immersive.
Demo: Pixels Per Degree
Set a pixel count and a field of view and watch the resulting acuity — the text panel below simulates roughly what that PPD can resolve.
Refresh rate and persistence
Refresh rate is how often a new image appears; persistence is how long each image stays lit. They are separate knobs, and persistence is the one that decides whether the world smears when you turn your head.
If a pixel stays illuminated for the full frame while your eye is smoothly tracking a moving object, the image physically drags across your retina — that is motion blur, and no amount of refresh rate fixes it. So VR panels run low persistence: they flash each frame for only 1–2 ms and stay dark the rest of the time. The cost is brightness (you are throwing away most of the duty cycle) and a flicker sensitivity floor, which is part of why 72 Hz feels tiring to some people and 90 Hz+ is the modern baseline.
Demo: Persistence and Smear
Both squares move at the same speed. Your eye tracks them smoothly, so a pixel that stays lit longer paints a longer streak on your retina.
Optics
A panel 3 cm from your eye is far too close to focus on. The lens exists to move that image out to a comfortable focal distance and magnify it across your field of view — and almost every headset compromise traces back to this one component.
Fresnel vs. pancake
Fresnel
A thick lens collapsed into concentric ridges. Light, cheap, efficient — it passes most of the panel's light through, so it works with dimmer LCDs.
- Needs a long gap between panel and lens → bulky headset
- Ridges scatter bright highlights into visible god rays
- Sharp only in a narrow central sweet spot
Pancake
Folds the light path back and forth between a half-mirror and a polarizing reflector, so the optical distance fits in a fraction of the physical depth.
- Dramatically slimmer headset, better weight distribution
- Sharp edge-to-edge, much larger sweet spot, no god rays
- Throws away ~75–90% of the light → needs bright panels
This is the dependency that reshaped headset design: pancake optics only became practical once panels got bright enough to survive losing most of their light, which is why the slim-headset generation and the micro-OLED generation arrived together.
Field of view
Human binocular FOV is roughly 200–220° horizontally. Consumer headsets deliver about 90–130°, which is why VR still feels a bit like looking through a diving mask. FOV is expensive in every direction at once: wider optics are bigger and heavier, they need more pixels to hold PPD steady, and those pixels need more GPU to fill.
Demo: Field of View
Top-down view of what a given FOV actually covers, against human vision.
Fitting the lens to the face
IPD
Interpupillary distance — the gap between your pupils, typically 54–72 mm. If the lens centers do not line up with your pupils, you get eye strain, a shrunken sweet spot, and distorted scale. Adjustable IPD is the single most skipped setup step.
Eye relief
Distance from lens to eye. Too far and you lose FOV; too close and your lashes hit the lens. Glasses spacers trade FOV for clearance.
Sweet spot
The region of the lens that is actually in focus. Small sweet spots force you to aim your head rather than your eyes, which is fatiguing and unnatural.
The focus problem that optics cannot fix
The lens puts the whole image at one fixed focal distance — typically ~1.5–2 m. But stereoscopy tells your eyes to converge at whatever depth an object is supposed to be. So your eyes aim at an object 30 cm away while still focusing at 2 m. That is the vergence-accommodation conflict, and it is a major source of eye fatigue. Varifocal and light-field displays aim to solve it; nothing shipping at consumer scale has yet. (Unit 3 covers this from the perception side.)
Tracking
Tracking is what separates VR from a screen strapped to your face. The requirement is brutal: sub-millimeter, sub-degree accuracy, updated hundreds of times a second, with no drift, in an unknown room.
3DOF vs. 6DOF
3DOF — rotation only
Yaw, pitch, roll. An IMU alone can do this. You can look around, but leaning forward moves nothing — the world is painted on a sphere around you. Cheap, and immediately nauseating for anything but seated 360° video.
6DOF — rotation + position
Adds X, Y, Z translation. Now leaning, ducking, and walking all work, and objects hold their place in the room. Requires an external reference — you cannot integrate accelerometer data into position without the error exploding within seconds.
Where the reference comes from
Outside-in
External base stations sweep the room with IR lasers; photosensors on the headset time the sweeps to compute pose. Extremely precise and robust to occlusion of the headset's own view.
- Setup cost: mount and calibrate hardware in the room
- Tracking stops at the edge of the configured volume
- Still the accuracy benchmark — used in research and mocap
Inside-out (SLAM)
Cameras on the headset watch the room, identify visual features, and solve simultaneously for the map and the headset's place in it, fused with IMU data.
- Zero setup, works anywhere, enables standalone headsets
- Struggles in dark, blank, or highly repetitive rooms
- Controllers outside the camera frustum fall back to IMU dead reckoning
Tracking the rest of you
Eye tracking
IR illuminators and a camera per eye estimate gaze at ~100–120 Hz. Enables foveated rendering, automatic IPD measurement, and gaze-based selection — and doubles as uniquely sensitive biometric data.
Hand tracking
Computer vision on the headset cameras infers a skeletal hand model. No controller to pick up, but no buttons and no haptics, and it fails when hands overlap or leave frame.
Body and face
Extra trackers on waist and feet drive full-body avatars; inward-facing cameras drive facial expression. Mostly social-VR features, and the clearest privacy frontier.
Controllers & Input
The display convinces you that you are somewhere. Input decides whether you can do anything there.
Tracked controllers
The default. Buttons, triggers, and sticks give unambiguous, latency-free discrete input, plus haptics — which is why they remain better than bare hands for games. Modern designs dropped the tracking rings once headset cameras and IMU prediction got good enough.
Capacitive & finger sensing
Sensors detect fingers resting on surfaces before they press, so an avatar hand can point, wave, and grip partway. Cheap approximation of hand tracking, with buttons intact.
Hands and eyes
Look-and-pinch interaction — gaze to target, a small finger pinch to commit — turned out to be fast and low-effort, and is the primary interface on headsets aimed at productivity rather than games.
Neural wristbands
Surface EMG reads the electrical signals of intended finger movement at the wrist. Works with hands at your side and out of camera view — the leading candidate for controlling all-day AR glasses.
Haptics
Nearly all shipping haptics are vibration — convincing for impacts and textures, useless for weight or resistance. Force feedback and gloves remain lab and enterprise equipment.
Voice
Text entry in VR is genuinely bad; voice sidesteps it. Increasingly the front door to an on-device assistant rather than a command interface.
Compute & Rendering
VR's rendering bill is brutal: two eye views, at high resolution, at 90+ frames per second, with no tolerance for a dropped frame. That is roughly an order of magnitude more demanding than the same scene on a monitor.
Where the computer lives
Standalone
Mobile SoC in the headset. No cables, no PC, mass-market pricing — at the cost of a power and thermal budget measured in single-digit watts sitting against your face.
PC-tethered
A cable to a desktop GPU. Highest possible fidelity, lowest latency, worst freedom of movement. Now mostly the enthusiast and professional path.
Streamed
Render on a PC or server, encode, send over Wi-Fi, decode on the headset. Adds compression artifacts and 30–50 ms of latency, and depends entirely on your network.
The tricks that make it fit
Foveated rendering
Your acuity collapses outside the central few degrees of vision. Render full detail only where the eye is actually looking and degrade the periphery — with eye tracking this can cut shading cost by half or more, invisibly. Without eye tracking, a fixed version assumes you are looking at the center.
Reprojection / timewarp
If a frame is going to miss its deadline, take the previous frame and warp it against the latest head pose instead of dropping it. Head rotation is handled almost perfectly; translation and moving objects produce artifacts and ghosting. It is a safety net, not a substitute for hitting frame rate.
Lens distortion correction
The lens warps the image, so the renderer pre-warps it in the opposite direction — including separately per color channel, to cancel the lens's chromatic aberration. The pre-distorted frame looks grotesque on a flat screen and correct through the lens.
Single-pass stereo
The two eye views differ only by a few centimeters of offset. Submitting geometry once and producing both views in a single pass avoids paying twice for identical CPU work.
Headset Comparison
Approximate specifications for widely-used headsets. Manufacturer FOV figures are measured inconsistently and depend on face shape and eye relief, so treat them as indicative rather than exact — and note how differently the same design problem gets solved at different price points.
| Headset | Panel | Per-eye res. | Refresh | FOV (H) | ~PPD | Optics | Tracking |
|---|---|---|---|---|---|---|---|
| Meta Quest 2 | LCD | 1832 × 1920 | 72–120 Hz | ~90° | ~20 | Fresnel | Inside-out |
| Meta Quest 3 | LCD | 2064 × 2208 | 90–120 Hz | ~110° | ~19 | Pancake | Inside-out |
| Meta Quest 3S | LCD | 1832 × 1920 | 90–120 Hz | ~96° | ~19 | Fresnel | Inside-out |
| Valve Index | LCD | 1440 × 1600 | 80–144 Hz | ~130° | ~11 | Fresnel | Outside-in |
| PlayStation VR2 | OLED | 2000 × 2040 | 90–120 Hz | ~110° | ~18 | Fresnel | Inside-out + eye |
| Apple Vision Pro | Micro-OLED | ~3660 × 3200 | 90–100 Hz | ~100° | ~34 | Pancake | Inside-out + eye |
| Bigscreen Beyond 2 | Micro-OLED | 2560 × 2560 | 75–90 Hz | ~102° | ~25 | Pancake | Outside-in |
| Samsung Galaxy XR | Micro-OLED | ~3552 × 3840 | 60–90 Hz | ~105° | ~33 | Pancake | Inside-out + eye |
Read the Index row carefully: it has the widest field of view in the table and the lowest PPD, because the same modest pixel count is spread across more degrees. That is the central tradeoff of this unit in a single line.
Self-Check
Work these out before expanding the answers.
A headset doubles its per-eye resolution and doubles its field of view. What happens to perceived sharpness?
Nothing changes. PPD is pixels divided by degrees, so doubling both leaves it identical. The image covers more of your vision at the same sharpness — more immersive, no crisper.
Why does a headset flash its pixels for only 1–2 ms instead of leaving them lit?
Because your eye moves smoothly while the displayed image is held still, so a long illumination drags the image across your retina and blurs it. Short pulses keep each frame sharp. The price is lost brightness and potential flicker.
Why can't a headset with only an IMU offer 6DOF tracking?
Recovering position from an accelerometer requires integrating twice, so small biases compound quadratically — position drifts uselessly within seconds. Position needs an external reference: cameras seeing the room, or base stations seeing the headset.
Pancake lenses throw away most of the panel's light. Why did anyone adopt them?
They fold the optical path, so the headset gets dramatically thinner and better balanced, with a larger sweet spot and no god rays. Bright micro-OLED and high-brightness LCD panels made the light loss affordable.
Your frame rate drops from 90 to 45 fps but the view still responds smoothly when you turn your head. What is happening, and what will look wrong?
Reprojection is warping the last completed frame against a fresh head pose, so rotation stays responsive. What breaks is everything reprojection cannot reconstruct: moving objects judder, and edges around them ghost or smear, since the warp only knows about head motion.