Synthetic Data for Android XR

Digital Human Imagery for ML Model Training

Over the course of four and a half years, I had the opportunity to help shape a synthetic data initiative that significantly advanced machine learning algorithms for eye tracking on the Android XR platform at Google. By blending artistic insight with hands on problem solving, I designed environments that simulate prototype hardware, developed materials for digital humans, and helped establish a scalable rendering pipeline for generating synthetic datasets.

I found particular satisfaction in collaborating closely with both artists and software engineers, translating creative goals into technical requirements, while also listening to technical feedback that guided visual improvements. This cross disciplinary approach ensured that every creative decision had a firm technical grounding.

  • Company :

    Google

  • My Role :

    Technical Artist

  • Period :

    2020 - Present

  • Link :

    Project Page

Bridging the Domain Gap & Empirical Problem Solving

A core part of my contribution was reducing the “domain gap,” or the difference between synthetic imagery and real world camera data. In addition to focusing on appearance of images, I also took an analytical route, comparing histograms, examining ML model outputs and drawing conclusions from experiments, as well as proposing iterative visual tweaks informed by direct feedback from researchers.

Whether it meant hallucinating high frequency surface details of skin or refining camera and render settings to capture subtle lens and camera sensor anomalies, I set out to strike the right balance between creative expression and data driven analysis. Through active collaboration and clear communication, I often took on a leadership role in shaping our approach to look development and render pipeline development, ensuring we pursued solutions that balanced aesthetics and efficiency.

Scalable USD Pipeline & Engineering Excellence

As our data demands grew, I played a key role in architecting a Universal Scene Description (USD) based pipeline capable of generating millions of images for training ML models.

I've been a key contributor in exploring and successfully building solutions for headless rendering of complete USD scenes. I contributed to code reviews for pipeline development and collaborated closely with software engineers to troubleshoot issues in Hydra Delegates of both Renderman and Cycles environments. This blend of engineering and artistic roles enabled me to solve problems quickly and keep the broader team moving forward.

Post Processing & Infrared Simulation

Recreating infrared imagery required a creative blend of technical experimentation and aesthetic sensitivity, especially around the eye region, where small details can make a big difference. To bridge the gap between “too perfect” 3D renders and the nuances of real IR photography, I authored shaders simulating the distinctive softness of skin captured in IR.

I designed and implemented a post processing workflow that emulated camera sensor noise, tone mapping, blooming, lens glints, and more. Much of this was prototyped in compositing software (Nuke), then I translated equivalent image operations into Python leveraging OpenCV for seamless integration into our pipeline. The result was a family of synthetic datasets that not only looked convincing but proved invaluable for advancing research on the XR platform.