Visitor Day

The process

Visitor Day was my final project for the Video Game Simulation and Production degree. It started from one question: what if the player's own home were the level? Instead of using the room as a backdrop, each chapter asks the player to do something with it: find a real bottle, bed and toilet for the alien, play air hockey on a real table, follow a sound through the house, spot a real book to escape a virtual cell, float through a zero-gravity copy of the home by grabbing its walls, and pick up two real spoons to deflect a giant parasite's lasers. Each chapter adds one new mechanic, so there is never more than one thing to learn at a time.

The core system is real-object recognition. A computer-vision model (ONNX, run through Unity Sentis) analyzes what the headset camera sees. Raw detections are noisy: one frame can mistake an object, and the same object can come back under different labels. Since every detection moves the story forward, a false positive would skip a scene. That is why a dedicated controller only confirms an object when it passes a confidence threshold and stays detected over several consecutive frames, and why each target accepts a list of equivalent labels. It all runs on the headset, so the game works offline and images of the player's home never leave the device.

At the start, the player can talk freely with Trofo, the alien, using their voice. Whisper turns speech into text, a language model running on the headset (LLMUnity / LlamaLib) writes Trofo's reply, and Piper speaks it. Open conversation is hard to fit into a scripted story, so the story itself contains it: after a few questions the translator breaks, and voiced, localized dialogue in English and Spanish takes over. The player gets a real conversation, but it can never derail the plot.

Running vision, speech recognition, a language model and speech synthesis next to hand tracking on a standalone headset was the main technical constraint. These systems only run when an event asks for them instead of every frame. Progression lives in one persistent GameFlowDirector that handles story state, language, checkpoints and scene changes, while each scene only manages its own mechanics. For the characters, I also built an in-headset tool to record hand movements and reuse them as animations, and used ARKit blendshapes for facial expressions.

The current version is the result of several rounds of UX testing, which made the instructions clearer, the interactions more responsive and the VR sections more comfortable.

Screenshots