1

Lego Crane User Test Report

Redhawks
ENG 111
Professor Amanda Stevens
24 October 2016
Isaac Steiger
Dilushka Balapiti Modarage
Leighton Kentwell
Maggie Mize
Julia Dempsey
Gretta Ebert

TABLE OF CONTENTS
I.
II.

Introduction
Executive Summary

III.
IV.
V.
VI.
VII.
VIII.
IX.
A.
B.
C.
D.

Methodology
Participants
Results
Test Reader Comments
Recommendations
Conclusions
Attachments
Writers Guide for Testing Drafts
Likert-Scale Test Questions
Observation Sheet for User Test.
Follow-up Interview Questions for Readers

Introduction:

The goal of this project was to provide a clear set of instructions in order to successfully
assemble a LEGO crane. To provide an effective set of instructions out team was able to utilize
images, different colors and graphics successfully. The crane design itself was broken into four
parts: the crane base, the crane arm, the crane head, and putting it all together.To test the
effectiveness of the set of instructions we also utilized a Likert scale and a target audience. This
greatly helped in understanding the weaknesses of our set of instructions and how to improve
them.
The target audience in this project were the students present in our class. But in reality, this target
audience has a broad range. This is because LEGO has a broad audience as a brand. To test the
effectiveness of our set of instructions, we used a Likert test as mentioned before. To
supplemented this we conducted follow up interviews with the target audience to understand
what we could change to provide what the audience required. Observations were key to noting
and recording what the user would need or request from us. Recording these were important in
understanding the improvements that were needed to be made.
Executive Summary:
The usability test took place in Bachelor Hall, room 252. It occurred at approximately 10:30 a.m.
on Wednesday, October 5th. The purpose of this test was to learn the efficiency of the instruction
set created to aid people in building a Lego crane. We hope to gain insight into any errors present
and to correct the issues.
There were five participants in the user test report. The length of the entire session took
approximately 30 minutes. All five users completed the pretest however, one participant did not
completed the posttest, which could alter our results. All were able to easily find the survey
provided to them.
The test was generally easy to use. Although we got good results from the rest of the tests, one of
the users declined to take the second Likert scale test. The test itself had an easy to follow
format, with minimal graphics to lessen the chance of users becoming distracted. The questions
were direct and participants completed the test promptly and with ease.
A few problems were identified during the test including:
Building the head portion of the model
Putting all of the pieces together at the end
The orientation of a photo within the instruction set was not user friendly

In the user test report, we have an introduction, an executive summary, a section describing our
methodology, a list of participants, our results, the test reader comments, the test readers
recommendations, our conclusions, and the attachments.
Body - Methodology:
Participants mainly consisted of other students in the classroom. We would ask them to work on
the questions on Likert scale and provide feedback using the scale. One of the key attributes to
the recruitment was that we wanted the participants to understand what the task at hand was.
The test session consisted of three parts. First the user would take the Likert test provided to
them and assess their capability of completing the LEGO part. After submission they would
attempt to build the part. We found that the users who attempted this did not take a long time in
completing the LEGO crane with the given set of instructions. Observations were noted from the
users while conducting the test so that any issues could be noted. After completing the LEGO
crane, the user would then proceed to complete a second Likert test to assess the before and after
experience of building the LEGO crane. The whole process took an average of about 30 minutes.
In the post test session we asked the users a series of follow-up interview questions. The followup interview questions that were asked are attached. We asked the users the interview questions
in a group setting with all of the writers facing all of the users. All of the interview questions
were answered and all of the users agrees on the answers given.
As part of our post test questions, we directly asked the test takers questions that would relate to
the experience they had while building the LEGO crane. Most of the questions were aimed to ask
what they felt when constructing the crane and how the instructions set helped them in doing so.
The post questionnaire was also followed by an observation sheet which was used to record the
responses of the test takers. The observations recorded would therefore help in improving the
instructions set as we would have the view of the test takers. Please refer to the attachments to
view the observation sheet, pre and post-test questionnaires.

Body - Participants:
The participants were 5 students from the 10:00 AM ENG 313 class and were all between 18 and
21 years old. They did the test session in class on October 5th, 2016 and the post-test session was
held in class on October 10th, 2016.
We separated the experiment into 5 tasks:
Task 1: Finding the required LEGOs
Task 2: Building the base

Task 3: Building the crane arm

Task 4: Building the crane head
Task 5: Putting all of the part together
Body - Results:
Isaac Steiger recorded the participants ability to complete the tasks without prompting.
Easy to
comprehend
Somewhat

Finding the
required
LEGOs

Neutral

Difficult to
comprehend

Very difficult
to
comprehend

Building the
base

Building the
crane arm

Building the
crane head

Putting it all
together

Somewhat
easy
comprehend

After observing the test takers, both and post assembling of the LEGO crane we observed some
errors which they came across. The errors were recorded by a team member by utilizing pre test,
post test questionnaires provided to the test takers. A key finding which we made was that the
test takers started on a fast note while slowing down as the process went on. The following
paragraph goes in depth as to the errors that were noted.
There were only two issues in regards to having an issue with the instructions, since the photos
had a different orientation, which was offsetting at first in comparison to all of our other images.
They were confused by two photos within the instruction set, yet were able to figure out the issue
on their own and completed the build promptly and correctly.
In our pre-test Likert-scale test we received mostly agrees in response to every questions but
there was some disagreement in the test reader's belief in their ability to make a lego crane. The
pretest Likert-scale test responses are shown below

In our post-test Likert-scale test, shown below, we saw a shift towards more strongly agree
responses which shows that the test readers felt more confident in their ability to build a LEGO
crane and their belief that text instructions are useful in a LEGO instructions set. This change in
responses show that our instruction set had a desired effect on the test readers.

Test Reader Comments:

The test readers commented that the pictures were the most helpful part. The combination of
before and after pictures for every step was liked. The arrows showing where the LEGOs go was
the most liked part of the instruction set.
The test readers liked the orientation of some of the pictures the least. The users believe that the
pictures of pieces getting put on the back of the LEGO crane did not help very much because the

pictures were from the front of the LEGO crane. The users also had issue with putting the crane
arm onto the crane body at the end of the instructions.
The only recommendation from our test readers was to improve the pictures that they were
having issue with. We interpreted this to mean the pictures where a piece was put onto the back
of the crane and the picture of how the crane arm attaches to the crane body at the end of the
instruction set.
Recommendations:
We recommend that pictures where a piece is being put onto the back are retaken from a better
angle. This is recommended because the biggest complaint from the test readers was how
difficult to understand these types of pictures were when shot from the front of the crane. The
second recommendation is to retake the photos showing how the crane arm attaches to the crane
body at the end of the instruction set. We recommend this because the only other complaint that
we received from the test readers was that the pictures showing the connection of the crane arm
to the crane body were confusing.
Conclusions:
In regards to the participants outcome, they were able to successfully build the Lego crane,
needing only little additional help in regards to one portion of the instruction set. After fixing the
orientation of a photo to make the instruction set more clear, we found that the colored arrows
were extremely helpful, probably more helpful than the words themselves. The data received
from our survey was helpful in understanding the skillset of our participants prior to them
building, as well as their feelings toward the instruction set following their completion. Both
were taken into consideration and helped us alter our instruction set/follow-up questions to be
more generalized and directed towards people who were not efficient in Lego building.
Attachments:
A. Writers Guide for Testing Drafts
Our communication usefulness objective is to describe how to build a LEGO crane. Our
communications persuasive objective is to get the user to feel comfortable and enjoy using
LEGOs. We need to test the the effectiveness of the picture and text instructions when helping
people to build a LEGO crane
Our target readers are the other students within our class. We are able to find test readers from
this group but only a small number, 5 or 6 students. This may mean that our test group may not
be representative of our target reader population.

We will ask our test readers to answer a Likert-scale before and after using our instruction
manual to make a Lego crane. Ina follow up session we will ask our test readers a set of
interview questions. We will provide our test readers with all of the LEGOs in the box and the
instruction set for the crane. We will conduct this test in the classroom during class time. We will
observe the test readers using our draft by standing in the room and watching them. We will learn
what the test readers are thinking by keeping a tract of any observation on an observation sheet
and asking interview questions. In our interview question we will ask them what they thought
was good,and what they thought was bad and what they think we could improve.
We will test our drafts persuasiveness by measuring their confidence in building LEGOs before
and after the testing in the Likert-scale test. We will avoid biasing the responses by not talking to
them and only observing them.
We do not believe that an informed consent form is needed for this particular task.

B. Likert-Scale Test Questions

Strongly
disagree

Disagree

Neither agree
nor disagree

Agree

Strongly Agree

I know how to
build a vehicle
out of Legos

I know how to
make a Lego
crane

I am confident
in my ability to
follow a Lego
instruction set

Text is useful
in a Lego
instruction set

C. Observation Sheet for User Test

Problem (What difficulties
did the readers have?)

Interpretation (What might

have caused the difficulties?)

Solution (What might prevent

these difficulties?)

Some of the parts look the

same and can confuse the
sequence of putting them
together

The same color of the part,

the instructions do not have
the parts numbered

Maybe having each par

numbered so that it
corresponds to a parts list that
would be provided to the user

Hard to figure out how to put

on the mounting piece for the
crane arm with the body

The image had the figure

Having maybe two sets of
flipped around which did not pictures so that you have two
match up with the instructions points of view

D. Follow-up Interview Questions for Readers

1.
2.
3.
4.
5.
6.

What did you find most challenging about the instruction set?
What part of the instruction set did you find easiest to understand?
Would you, as a user, want something else out of the product?
Did you find the instructions easy to follow?
Were the instructions clear and concise?
What would you recommend to improve our instructions?