Jing’s Poetic Cartography

Jing's Poetic Cartography is an immersive VR documentary that explores the diasporic experience through the sensory dimensions of cooking, memory, and oral history. The project follows Yutong Lin's connection with her aunt Jing, whom she had never met in China until after immigrating to Canada. Through the re-creation of Jing's recipe for Fen Zheng Rou 粉蒸肉 (Steamed Pork with Rice Flour), the film weaves together family stories and the bittersweet journey of immigration.

Diasporic Phenomenology

How does a recipe resemble a cartography?

Immigration is more than a journey to a foreign land; it is a diasporic experience of mediating the process of change and disorientation within our sensory registers. In this process, cooking becomes a makeshift event that soothes us, easing our perturbation. The making of homeland food dismantles a dish into ingredients, spices, and materialities, each bearing traces of migration, logistics, and translation, shaping what Yutong calls a "poetic cartography" of the diasporic experience.

How can geographies sound, feel, taste, or be dreamed of in diasporic phenomenology? In our food-sharing culture, some unspeakable emotional weight can only be expressed in the event of food. The VR experience allows viewers to witness the cooking process as if they were present, creating an intimate connection to the cultural practice and its associated memories.

While diasporic food bears immense traces of migration, logistics, and translations, it also contains the creativity of the diasporic community to utilize what is available under different circumstances. The 3D scan of the finished dish serves as both documentation and a digital artifact of cultural preservation, allowing viewers to examine the texture and form of this traditional recipe from multiple angles.

Low-Budget Volumetric Capture

Multiple point clouds from different viewpoints

As the vr technologist on this project, I developed a cost-effective multi-camera volumetric capturing and rendering workflow that made this project possible without expensive commercial solutions.

Multi-camera volumetric capturing typically costs thousands of dollars with professional systems like Depthkit and Orbbet Femto Bolt. While these commercial solutions produce cleaner results, they've unfortunately dropped support for older, more affordable cameras. This gap in accessibility prompted the development of our custom software workflow.

Our setup utilized four Intel RealSense depth cameras positioned strategically to capture the cooking process from multiple angles. This arrangement allowed us to record a more complete representation of the culinary process. These cameras are affordable and widely available in second-hand markets.

I developed a custom TouchDesigner pipeline to encode depth and color information into a false-color video format. This approach allowed us to store complex 3D data as standard video files, making the volumetric sequence editable in familiar software like Adobe Premiere. This workflow bridge significantly simplified our post-production process while preserving the spatial data integrity.

In Unity, I created custom compute shaders to decode the video-stored point cloud data and reconstruct the volumetric captures in real-time. These shaders enabled us to blend multiple camera perspectives seamlessly, creating a cohesive 3D representation that viewers could explore from any angle within the VR environment.

This custom technical pipeline provided us with unprecedented flexibility throughout the production process. By developing our own tools and workflows, we were able to maintain creative control while working within budget constraints. The resulting volumetric captures successfully preserved both the physical techniques and emotional nuances of Jing's cooking process, allowing viewers to experience this cultural practice in an immersive, spatially accurate environment.

Environment Building with AI

To create the immersive environments in the VR experience, we employed several innovative techniques that balanced technical constraints with narrative needs.

For the protagonist's homeland scenes, we used a diffusion model to generate panoramas of Chongqing and Panzhihua, China. The generated images were stitched together through a custom workflow to create seamless panoramic backgrounds. To enhance the spatial depth, we generated estimated depth maps for each panorama, which served as displacement maps for creating immersive 3D terrains that respond to viewer movement in VR.

These technical approaches go beyond traditional static skyboxes by providing enhanced parallax effects as viewers move within the VR space. The result is a more visceral, embodied experience that helps audiences connect with the emotional dimensions of Jing's story. By blending AI-generated imagery with volumetric scanning techniques, we created environments that serve not just as backdrops, but as meaningful spatial representations of the protagonist's journey across continents and through time.

Exhibition

Residency presentation at Ada X, Montréal

2024, Transmissions from Far East Nexus. Film screening at InterAccess, Toronto, ON, Canada.

2024, 시간의 메아리: Echoes Across Time. Film screening at TINC (This Is Not a Church), Seoul, South Korea.

2024, Jing's Poetic Cartography. Residency presentation at Ada X, Montréal, Canada.

Residency

2024, Artist Residency at Ada X, Montréal, Canada.

Acknowledgements

This work was generously funded by Ada X, a feminist artist-run center in Montréal, where Yutong completed a residency culminating in a public presentation of the work.

Special thanks to Jing Lin, Tyra Trono, Wenhui Zhang, Frida Chen, Zihao Li, and Professor May Chew for their contributions to this project.