To: Christa Teston From: Mark Johnson, Sarah Lynn, Pierce Trey, Morgan Weir, Crystal Zarate (Publisher

Team 2) Date: 3/9/012 RE: Usability Test Plan Introduction The purpose of this document is to detail the methods Publisher Team Two will use to analyze the effectiveness of two infographics regarding wildfire knowledge. The two infographics are shown below.

Figure 1. Gains Infographic.

Figure 2. Losses Infographic.

As seen in Figures 1 and 2, one infographic will present the data with a gains frame, while the second will present the same data with a losses frame1. To test the effectiveness of the individual infographics, a usability test will be designed and administered using methodological triangulation. The following document details this teams timeline, participant selection, experiment description, experimental methods, data analysis, and the individual responsibilities of each team member (team responsibilities listed in Appendix A). Timeline March 9 Usability test plan final draft due March 19 Review and revise test plan / Start testing participants March 23 Analyze test results March 26 Begin writing white paper March 28 White paper draft one due March 30 White paper due Participant Selection Ideally, participants would be selected at random across a large topographical region including metropolitan, country, and backwoods areas. Participants should include a variety of genders, ages, and backgrounds. However, the purpose of this usability test is to determine necessary adjustments to the infographic designs and experimental methods before running a large scale test. This usability test is designed for a small trial run and it should be noted that the results may be very different from those of a largescale experiment. The differences will likely arise due to participant selection, distribution, and data collection techniques. Team members will select a total of ten people, primarily acquaintances. Individual members will test two people, one for each frame of the infographic. This means the test will not be random and is unlikely to span the ideal region. Experiment Description Ultimately, the result of the usability test will be an analysis of whether gains or losses infographic framing is more successful in motivating people to take action toward wildfire control. To conduct an effective test, adjustments to infographic designs may be necessary before running a large-scale test. To assess this, smaller tests will be administered to gather data regarding how users feel the infographics can be altered and what affect they currently have on viewers. Although the initial round of testing includes a very limited number of participants, their critiques will be recorded to determine necessary adjustments before conducting a large-scale test.
1

Framing refers to the emotional tone of the message being conveyed. Framing refers to the words chosen, the images alongside them, and the colors used. A gains frame presents the message such that the benefits are listed while a losses frame shows the negative effects. Oftentimes it is unintentionally done by the messages creator and examples can be found in everyday life. For example, the dentist tells you to brush your teeth everyday to keep your mouth healthy versus the dentist tells you to brush your teeth everyday or your teeth will rot out. The first is a gains frame while the second is a losses.

Experimental Methods Participants viewing the infographic will be allowed an unlimited amount of time to study it. This decision is based on the assumption that when viewed in everyday life, viewers are not restricted on the amount of time they look at an infographic. Also, the amount of time voluntarily spent viewing the infographic may help our group determine the effectiveness and desirability of it. To conduct and analyze the usability test, methodological triangulation will be implemented. Methodological triangulation requires data collection from three independent collection platforms. The data will analyzed and the results used to determine the infographics effectiveness.This group has elected to use surveys, talk aloud protocol, and interviews as the measurement criteria. These criteria are explained in the following paragraphs. Survey The survey portion of this usability test will consist of a pre-survey and a post-survey. The pre-survey will be delivered before the subject views the infographic, and will consist of the questions administered in the post-survey plus a selection of background questions to ascertain the subjects history. The purpose of the pre-survey is to gain an understanding of the participants current knowledge about wildfires and wildfire prevention, while the post-survey serves to analyze whether or not the participants knowledge and/or opinions changed about this topic due to the infographic. The survey is the main data point of the test and a copy will be attached to this document as Appendices B and C (the pre-survey consists of B and C together, the post-survey consists of only C). Talk Aloud Protocol Another portion of the usability test will be the talk aloud protocol. In order to train the participant in the protocol, we will supply the participant with a sample infographic (Appendix D) and have them explain where their eyes go on the page, what portion attracts them the most, and the order in which they observe elements of the infographic. We will explain to them that we would like them to relay this information at the same time as viewing the sample infographic, making the talking aloud concurrent with the observing. We will then explain to the participant that this same procedure is what we would like them to do while observing the two wildfire infographics. It would be most helpful if the participants related to us anything that seemed to catch their attention most vividly. The order in which they view the elements of the infographic may also be helpful. We could have them explain how they feel about the use of texts and headings in the infographics, as well as any graphics on the page, and how either add or take away anything from the design. We will make note of the participants observations and

present the most common findings or most striking comments in the white paper following the usability tests.

Interview The interview will occur after the post-survey and will primarily focus on how the infographic affected the participant. This test platform will be used to specifically determine which elements of the infographic were the strongest and the weakest, as well as the amount of knowledge gained from the infographic. The interview is expected to provide feedback regarding how the infographic designs may be improved to increase effectiveness. The interview questions are attached to this document as Appendix E. Data Analysis For anonymity, participants will be assigned a number. This number will be written at the top of each page of paperwork (pre-/post-surveys, talk aloud and interview notes). Analysis of the data will begin by organizing the participants pre- and post-survey paperwork for the exercise. The physical paperwork will be in numerical order; however, the participants numbers will be placed in a spreadsheet (Appendix F). The spreadsheet will have columns of various major attributes from the pre-survey (such as age, or thought on a subject) under which we will put the numbers of the participants which possess applicable attributes. We will have a second section on the spreadsheet for the post-survey which will allow for comparison of changes in data between the preand post-survey. Secondly, the talk aloud data will be collected in an abstract form which allows for the recording of unique observations. Since humans think in an abstract, rather than completely logical and pragmatic manner, it would benefit the experiment to allow for an open boundary of results and scenarios. As the subject speaks, we will record what they are saying, and shall apply only the most common responses to the feedback of this experiment; the less common feedback can be added as errata later, should the time be available. The common data will be applied under three categories: graphical, textual, and miscellaneous (where miscellaneous applies to what they thought about the ideas, rather than any physical design aspect).

Lastly, the interview will require recording of abstract observations similar to the talk aloud exercise. Interviews will be recorded for accuracy of statements. Interview feedback will be sorted in a similar manner to the talk aloud exercise with only the addition of a Learned category. This category shall be similar to miscellaneous, only it will be based upon any new revelations that our participants may have during the

course of the experiment. These answers and ideas will be compared to the postsurvey and talk aloud exercises to determine the reliability of the participants answers and we can use them for good data. Through the use of methodological triagulation, we can determine similarities between groups, participant reliability, and the impact the infographics may have on them as well as how we could apply the participant feedback to improve the effectiveness of the infographic.