Using Your Fingers to Think:

Interactive Exploration of Subjective Constraints

Andrew Bennett
School of Engineering & ICT
University of Tasmania
andrew.charles.bennett@gmail.com
ABSTRACT

There is a class of complex problems where solutions must

satisfy multiple subjective criteria, while meeting specific
quantifiable constraints. Route planning for leisurely travel
is an example of a problem in this class, where constraints
including total available time, transit times, and budget
constraints determine whether a potential solution is
acceptable to the prospective traveller. In this paper we
present an interface that leverages, metaphorically, the
elastic properties of a rubber band to allow playful
interaction with relevant constraints. The resulting touchbased human computer interface enables the traveller to
explore the solution space in the sense that constraints can
be played with in order to find a route that meets the
travellers subjective preferences. Formal step-by-step
evaluation with nine subjects confirms that leveraging the
rubber band metaphor is useful in constraint satisfaction
and that using the interface is intuitive since it leverages
real-world experiences.
Author Keywords

interaction, exploration, constraints, elasticity, tourism

ACM Classification Keywords

H5.m. Information interfaces and presentation (e.g., HCI):

Miscellaneous.
INTRODUCTION

Consider the following familiar 'scenario' which is loosely

based on [6]:
Consider a researcher attending a conference in an
unfamiliar city. At the end of the conference they have a few
hours to spend before catching their flight. Checking a
recommendation service on their tablet computer leads to
more places of interest to visit than they can possibly
accommodate. The question therefore is which ones to visit
within the time that is available?
The researcher funnels the location information to the
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee.
VINCI 2014, August 5-8, 2014, Sydney, Australia.
Copyright 2014 ACM 978-1-4503-2765-7 $10.00

Christopher Lueg
School of Engineering & ICT
University of Tasmania
christopher.lueg@utas.edu.au
StretchBack app to visualize the locations on a map and to
add specific constraints such as total time and preferred
transit times. The researcher is then plots a route covering
the most important locations. They want to add two more
locations, but as their path approaches the first it gets
stretched and harder to pull. The researcher realises that
theyve nearly hit their time limit and decides to choose the
second of the two locations, they think it will result in a
more pleasant end to their trip.
The scenario points out a number of constraints that
highlight that route planning for leisurely travel is an
instance of a class of complex problems where solutions
must satisfy multiple preferences while meeting specific
constraints, such as total available time, transit times, and
budget constraints.
Unlike determining optimal routes for supply chains, just to
give an example, the criteria in leisurely travel are often
subjective in nature. Such subjective criteria include, for
example, the relative importance of points of interest (e.g.,
locations that are interesting because of their natural beauty
vs points of historical significance), how much time is
required to enjoy a location and how much transit time
between points of interest is acceptable.
We are primarily interested in leveraging human perception
and the idea of "thinking with your fingers" (see also
[1][3][5]). In some situations an analytical solution might
provide a solution faster than this approach, however this
may not be preferable as it does not allow the user to learn
or explore the decision space, or to engage with subjective
aspects of the decision making process.
In order to allow for interactivity we translate constraints
into elastic properties that can be played with to figure out a
route that meets existing constraints and preferences.
Playing with elastic properties features in the touch based
computer game World of Goo (www.worldofgoo.com) and
related work has been done in constraint programming (e.g.,
[4]) but we are not aware of applications of elastic
constraints in interactive route planning.
In what follows we discuss how the metaphor of a rubber
band is used to translate the aforementioned constraints into
an interactive, touch-based human computer interface. First
we discuss the elastic properties of rubber bands that allow
users to perceive constraints. Mapping constraints onto
these properties would then allow users to have immediate

feedback about how close they are to approaching

constraints and when they have reached them. Then we
discuss how the metaphor was implemented on an iPad
using its touch based interaction model. Next we discuss
how the system was evaluated with task based scenarios.
The paper concludes with a discussion of the major
contributions as well as an outlook on further research.
MAPPING ELASTIC PROPERTIES OF REAL WORLD
RUBBER BANDS TO TOUCH BASED HUMAN
COMPUTER INTERACTION

visibility. It would be worthwhile to explore additional

modes of output for this feedback, aural, haptic, and so on.
Audial feedback was considered, but the physical band does
not exhibit any noticeable noises and the choice of sound
(such as ropes straining) would also be culturally biased.
Due to limitations of the device it was not possible to
provide haptic feedback and so tension was simulated by
making the user drag further when the tension was high,
this seemed to be both effective and increase visibility.

Elastic properties

In order to evaluate the usefulness of the rubber band

metaphor for representing constraints in a touch based
interface we identified the elastic properties of real world
rubber bands which allow users to perceive constraints. The
state of each property was derived from a physical band and
dependent on the distance from the bands stretch
equilibrium point.

(a) Unchanged

(c) Tension

(b) Attenuation

(c) Colour

Figure 2 Screenshots of each of the properties in isolation, the

circle represents the touch location.
TASK-BASED EVALUATION

Figure 1 Physical observations of an actual rubber band, note

the change in thickness and slight change of colour when
stretched.

We identified the following three elastic properties and

analysed them in regards to their usability for providing
feedback: Thickness, Colour, and Tension.
Translating elastic properties to touch-based interaction

Once we identified the relevant elastic properties of rubber

bands we looked into translating the properties such that
they can be interacted with in a touch based interaction
model as available on touch enabled tablet computers. The
resulting touch based user interface leverages the following
properties:
1.

Stretch based attenuation (i.e., changing thickness)

2.

Stretch based colour change

3.

Simulated tension (explanation below)

When each property was integrated into the system it was

deemed to have sufficient feedback to allow users to know
that they were approaching a constraint, and when they had
reached it. Having the feedback mechanism at the point
where the action took place had both advantages and
disadvantages, on the direct-touch device the hand
sometimes obscured the feedback, but this was countered
with additional feedback.
The choice of colour (red), was considered to be culturally
biased, however it is high contrast and aimed to increase

As mentioned in the introduction the system is designed as

a simple-to-use decision support tool for route planning.
The scenario driving the system is one of a tourist with a
limited amount of time who wishes to maximise the quality
and quantity of the attractions or points of interest (POI)
they are able to visit.
For the purpose of the evaluation we limited the subjectivity
of comparing POIs by giving each POI a fixed and visible
numeric value. In a real setting the subjectivity would be
important but subjectivity is not necessary to evaluate the
suitability of the metaphor, which was the primary purpose
of this implementation.
In order to evaluate the perception of the metaphors
mapping of constraints we arranged for a task-based
evaluation of the system involving nine subjects that we
recruited from the local university. The age range of 5
males and 4 females was 18-54 with 7 subjects in the 18-24
range and one each in 25-34 and 45-54. All were
experienced computer users and were using touch screens.
The evaluation protocol was approved by the University of
the Tasmania Social Sciences HREC.
The general structure of the experiment was a series of
semi-scripted tasks. Subjects were given a written scenario
and a series of tasks to perform. Following the experiments
they were also given the opportunity to provide feedback
and comments in a questionnaire.
The focus of the evaluation was on the reception of the
rubber band metaphor, not to thoroughly evaluate potential
errors and inefficiencies of the respective prototype.
Researchers had to be aware of major usability issues
though since they might have had an effect on the

perception of the metaphor. Fortunately, the experiments

did not reveal major usability issues.
The direct-touch tablet devices used for implementing the
metaphor were the Apple iPad and the Apple iPad 2,
however steps were taken to prevent device specific factors
from affecting the research. The primary user interface was
written with low level CoreGraphics APIs so as to reduce
familiar controls and biases from knowledge of the system.
The interface was deliberately designed to be game-like in
appearance to encourage playful exploration. Screen size,
resolution, touch responsiveness and processing power of
the device are still expected to effect the test.

(a) Path Manipulation

(b) Constraint
Recognition

(c) Decision Making

Table 1 shows that when focusing just on the usability

questions the median response was 6, less than 2% of the
responses away from 7, which was strongly agree. This
strongly suggests that the system was considered to be
usable by subjects, with the 95% confidence interval
spanning from 5 to 7 on the same scale. Two of subjects
explicitly described the system as fun, one
enthusiastically and verbally, while a third described it as
easy to use.
Agreement
Response

Standard
Usability

All Limit
Feedback

Limit
Approached
Feedback

Limit
Reached
Feedback

freq.

pctl.

freq.

pctl.

freq.

pctl.

freq.

pctl.

100

100

100

100

100

100

86

100

100

100

71

100

98

94

57

100

95

88

43

100

44

90

71

29

88

53

49

41

14

63

Table 1. Summary of the usability questionnaire, with

frequencies (freq.) of each response when grouped into
categories, and the percentile (pctl.) for which each response is
at least as high for that category. The response values range
from 1 to 7 (Likert scale). Underlined values highlight the
response representing approx. the 95th percentile.

(d) Visual/exploratory
decision making

(e) Long edges, visually

interleaved

(f) Large decision

space, single solution

Figure 3 Screenshots of varying topologies used in the tasks.

The elastic properties contributing to the perception of the

feedback were tested with side-by-side comparisons of each
elastic property on the two devices:
1. Thickness and Tension
2. Thickness and Colour
3. Tension and Colour
4. No options and All options
The usability questionnaire, based on [2], aims to find
usability issues with the system, it was used in this instance
to focus on issues that might effect the results. Two
questions were added to the usability questionnaire to allow
a measure of the how well the metaphor was initially
interpreted, these questions were "During each task I could
easily determine when I was approaching my fuel limit"
and "During each task I could easily determine when I
reached my fuel limit".

When comparing all of the elastic properties, as can be seen

in Table 2, it can be seen that tension contributed little to
the metaphor, having been chosen in less than 7% of cases.
As mentioned earlier, the tablet used in the experiments
does not support physical tension which means this was
simulated tension, by making the user drag further to create
the same amount of pull.
Question

Thickness

Tension

Colour

Which is better at telling you the limit is

being approached?

16

Which is better at telling you the limit

has been reached?

12

11

Which looks more like a rubber band?

13

Which feels more like a rubber band?

11

Which is more satisfying to use?

10

13

Which is more useful for solving the

task?

10

13

Table 2. Sum count of when subjects gave preference of

thickness, tension, or colour over each other in the
comparative study, when answering each of these questions.

When isolating just thickness and colour, as in Table 3,

there is a good balance between each property. Thickness
was shown to do better than colour when showing the limit

was reached. Thickness was indicated to be a much more

strongly identified as indicating a rubber band than colour,
with a 12 to 2 ratio. However colour was strongly indicated
at being better at showing the limit was being approached
with a 7 to 1 ratio.
Question

Thickness

Colour

Which is better at telling you the limit is being

approached?

Which is better at telling you the limit has been

reached?

Which looks more like a rubber band?

Which feels more like a rubber band?

Which is more satisfying to use?

Which is more useful for solving the task?

Table 3. Sum count of when test subjects gave preference of

thickness, or colour over each other in the comparative study,
when answering each of these questions.
DISCUSSION

The StretchBack system successfully derived properties of a

rubber band that can represent constraints and provided a
system on which these properties could be evaluated. The
system was built based on an iterative approach using a
physical rubber band for inspiration to solve problems early
in the translation process.
The comparative analysis of each of the elastic properties
showed that colour and tension both contributed strongly to
the perception of the metaphor and the feedback it
provided, whereas the simulation tension provided
significantly less with both perception and feed- back.
The first property, thickness, allows the user was found by
observing when a rubber band is stretched it gets thinner, by
exaggerating this effect the user can see approximately how
close they are to their constraint by the thickness of the
rubber band, and when the rubber band approaches full
stretch the thickness approaches zero. The thickness was
clearly the strongest indicator of the rubber band metaphor,
with a ratio of 12 to 2.
Combining the first and second property creates sufficient
feedback to know when a maximal constraint is being
approached and when it has been reached. Evaluation of
this feedback has shown high user satisfaction with some
subjects describing it as fun and easy to use. One user
described the tool not in terms of fun, but as pleasant,
saying that I would seek this tool out if it was available,
they continued by likening it to gamification, saying
gamification is usually arbitrary enhancements to make a
task fun, whereas this feels like a genuine UI improvement.
The third property of rubber bands that was identified was
tension which was much harder to reproduce on a directtouch display. Requiring to drag further when the tension
was higher to create the same amount of movement in the

path was easily understood by some, but made the interface

seem sluggish and provided little feedback relative to the
other elastic properties. From user comments and physical
interactions it is clear that tension is an important property
of an elastic band for representing constraints but device
limitations prevented the haptic feedback required to fully
realise that.
CONCLUSIONS

The major contribution of this research is an understanding

of how elastic properties of real world rubber bands can be
leveraged to provide feedback mechanism in human
computer interfaces. The metaphor of a rubber band has
been shown to be useful, the novelty of elastic properties
compared to many other feedback mechanisms being both
proximity to the user action and a physical basis for the
driving metaphor. Both colour and thickness changes can be
used in a simulated band to indicate constraints, and that
they each complement each other in both how they are
recognised and when they give feedback. The elastic
properties of the rubber band metaphor offer to support
subjective decision making by making users almost
implicitly aware of the constraints on their decisions.
The constraint algorithms in all tasks in this research used
linear models. We are currently investigating non-linear
models to allow users to make decisions based on less
predictable outcomes.
ACKNOWLEDGMENTS

The authors would like to thank participants for their time

and anonymous reviewers for their insightful comments.
REFERENCES

[1] Ishii, H. and Ullmer, B. Tangible bits: Towards

seamless interfaces between people, bits and atoms. Proc.
CHI 1997. 234-241. ACM.
[2] Lewis, C., Polson, P. G., Wharton, C. and Rieman, J.
Testing a walk-through methodology for theory-based
design of walk-up-and-use interfaces, Proc. CHI 1990.
235-242. ACM.
[3] North, C., Dwyer, T., Lee, B. and Fisher, D.
Understanding multi-touch manipulation for surface
computing, Proc. INTERACT 2009. IFIP.
[4] Shin, T. A Browsing Interface for Exploring Constraints
in Visualization Rules. Proc. Symposium on Human
Centric Computing Languages and Environments 2003,
108-110. IEEE.
[5]
Shneiderman,
B.
Direct
manipulation
for
comprehensible, predictable and controllable user
interfaces. Proc. Intelligent User Interfaces 1997, 33-39.
ACM.
[6] Yoon, H., Zheng, Y. and Xie, X. Social itinerary
recommendation from user-generated digital trails,
Personal and Ubiquitous Computing 2010. Springer.