Visitor Day
Visitor Day is a story-driven mixed-reality adventure for Meta Quest 3 and 3S built around local real-world object detection. A stranded alien appears through a portal in your home and asks for help, turning your room and familiar objects into part of an increasingly strange sci-fi story.
The experience
Real-world object detection is the central mechanic. The headset camera looks for objects such as a bottle, bed, toilet, book, and spoons, and each successful detection becomes a gameplay trigger: it advances dialogue, reveals a new objective, connects mixed and virtual spaces, or transforms a physical object into a virtual tool. The complete recognition pipeline runs locally on the headset, so these interactions do not depend on a cloud vision service.
Around that core, the adventure moves between mixed and virtual reality. Players talk naturally with Trofo, play air hockey on a physical table, follow spatial audio clues, escape a virtual prison, move through a zero-gravity version of their home, and face a final boss using the detected spoons. Hand tracking keeps interaction direct through touch, pinch, and grab gestures. The full experience is available in English and Spanish.
The process
I developed Visitor Day as my final integration project for the Video Game Simulation and Production degree. The project began with a design goal: make the player's real environment an active part of both the story and the mechanics. I structured the experience as a sequence of six situations that gradually transform a funny first contact into a surreal invasion.
Development was iterative and informed by user testing. Early tests highlighted opportunities to clarify instructions, strengthen interaction feedback, simplify navigation, and improve comfort. I addressed them with contextual prompts, audiovisual responses, checkpoints, safe recovery in the zero-gravity section, and a centralized flow that keeps transitions between scenes predictable.
Technology and challenges
The game was built in Unity 6 with URP, Meta XR SDK, MRUK, OpenXR, and the Input System. Object detection uses an ONNX computer-vision model through Unity Sentis. A dedicated controller processes label aliases, confidence thresholds, and stable detections across multiple frames before confirming an object, which helps turn noisy camera results into dependable game events.
The main technical challenge was running perception alongside local speech recognition with Whisper, an on-device language model through LLMUnity and LlamaLib, Piper text-to-speech, hand tracking, and spatial understanding on a standalone headset. Event-driven integrations and cached references keep these expensive systems from doing unnecessary work every frame. A persistent GameFlowDirector coordinates narrative state, language, checkpoints, and scene changes, while each scene owns its local mechanics.
Screenshots
