Mixed Reality Light Fields
for Interactive Remote Assistance

Conference on Human Factors in Computing Systems (CHI 2020)

Peter Mohr
Graz University of Technology, VRVis GmbH
Shohei Mori
Graz University of Technology
Tobias Langlotz
University of Otago
Bruce H. Thomas
University of South Australia
Dieter Schmalstieg
Graz University of Technology
Denis Kalkofen
Graz University of Technology
Teaser: Two parties using our system in a tele-collaboration session. 
			(a) The remote user generates visual instructions on a high-quality light field representation, which has been captured and shared by the local user. Our system supports guided capturing of the light field using off-the-shelf mobile devices. Subsequently, it enables annotating the representation using simple gestures on a mobile touch screen. 
			(b) The local user follows the visual instructions in Augmented Reality.


Remote assistance represents an important use case for mixed reality. With the rise of handheld and wearable devices, remote assistance has become practical in the wild. However, spontaneous provisioning of remote assistance requires an easy, fast and robust approach for capturing and sharing of unprepared environments. In this work, we make a case for utilizing interactive light fields for remote assistance. We demonstrate the advantages of object representation using light fields over conventional geometric reconstruction. Moreover, we introduce an interaction method for quickly annotating light fields in 3D space without requiring surface geometry to anchor annotations. We present results from a user study demonstrating the effectiveness of our interaction techniques, and we provide feedback on the usability of our overall system.


Lightfield capture guidance
Lightfield annotation
Augmented Reality guidance


	author = {Mohr, Peter and Mori, Shohei and Langlotz, Tobias and Thomas, Bruce H. and Schmalstieg, Dieter and Kalkofen, Denis},
	title = {Mixed Reality Light Fields for Interactive Remote Assistance},
	year = {2020},
	isbn = {9781450367080},
	publisher = {Association for Computing Machinery},
	address = {New York, NY, USA},
	url = {https://doi.org/10.1145/3313831.3376289},
	booktitle = {Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems},
	pages = {1–12},
	numpages = {12}