Controlled Experimentation in Naturalistic Mobile Settings

This paper represents our first foray into submissions to arXiv. arXiv, is super popular within the physics and astronomy communities and is a high quality location for open scientific information such as white papers and technical reports. Our work is the last publication from our RIAM project and can be found at: (which is a permanent identifier – think DOI).

We say that performing controlled user experiments on small devices in naturalistic mobile settings has always proved to be a difficult undertaking for many Human Factors researchers. Difficulties exist, not least, because mimicking natural small device usage suffers from a lack of unobtrusive data to guide experimental design, and then validate that the experiment is proceeding naturally.Here we use observational data to derive a set of protocols and a
simple checklist of validations which can be built into the design of any controlled experiment focused on the user interface of a small device. These, have been used within a series of experimental designs to measure the utility and application of experimental software. The key-point is the validation checks – based on the observed behaviour of 400 mobile users – to ratify that a controlled experiment is being perceived as natural by the user. While the design of the experimental route which the user follows is a major factor in the experimental setup, without check validations based on unobtrusive observed data there can be no certainty that an experiment designed to be natural is actually progressing as the design implies.

And the meta data…
Title: Controlled Experimentation in Naturalistic Mobile Settings
Authors: Simon Harper, Tianyi Chen, and Yeliz Yesilada
Categories: cs.HC
Comments: 12 pages, 3 tables
ACM-class: H.1.2; C.5.3; G.3


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s