Remotion Hardware

Replay with a Physical Robotic Arm

Learn more
example remotion hardware

Remotely mirror the phone interactions of your users

Remotion enables richer context sharing from remote users, especially during usability studies. Replay the behavior of people visiting your website or application, along with the screen capture. This website provides instructions for setting up and deploying Remotion Hardware, and hosts the open source software and blueprints.

What is Remotion for?

Remotion is a new way of replaying behavior of people visiting your website or application.

A remote user's hand motions can reflect contextual information about their surroundings. Imagine a scenario when a user shopping online on a mobile device proceeds to the checkout page which asks for their credit card information. Some users will put down the phone and start to look for their wallet, while others may have the cards readily available without the need to put the phone away. None of this contextual information is captured on the screen, yet the different movements resulting from how a user handles their device provide clues about the situation than what is displayed on the screen.

Remotion extracts more information about a user's performance in a remote usability testing, without the privacy or data size issues of video recordings. Remote user studies can feel as if the user is invisible but in the room manipulating the device, through a software or hardware replay. The mobile device is shown as a 3D model to simulate exactly how the device was manipulated and what was shown on the screen. You can then label parts of the interaction for attention, and frustration or confusion.


How Remotion Hardware Works

Remotion Hardware is comprised of five components:

The client software library for motion sensing on mobile devices (e.g., Android phones and Surface tablets) enables developers to collect sensing data by simply deploying an application on the participants' phone.

The screen projection component captures what the user sees on their screen and projects it to the screen in a software or hardware replay.

The lightweight server receives and organizes the data collected with Remotion's client and Remotion's screen provides an interface to control the software and hardware visualization replay.

The software visualization displays on the experimenter's screen a replay of what the user sees on their screen and the phone's movements.

Remotion's hardware (physical) replay replicates the remote user's movement and screen on a one-to-one scale. The robotic mount (with 4-axis degrees of freedom) is offered as blueprints for at-home construction.

Remotion system architecture diagram

Paper and Demo

Remotion: A Motion-Based Capture and Replay Platform of Mobile Device Interaction for Remote Usability Testing
Jing Qian, Arielle Chapin, Alexandra Papoutsaki, Fumeng Yang, Klaas Nelissen, Jeff Huang. Remotion: A Motion-Based Capture and Replay Platform of Mobile Device Interaction for Remote Usability Testing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), 2(2), Article 77.

Blueprints, Source Code and Documentation

We'd love for you to also have your own Remotion setup. Hardware models, video instructions, and source code are all open source. The files necessary for setting up Remotion are on Github.

Remotion Github repository

The installation video is also available on Youtube.

Research Team

Arielle Chapin

Alexandra Papoutsaki

Fumeng Yang

Klaas Nelissen

Funded in part by the National Science Foundation IIS-1552663