SANTA-MARIA-PhD - Thesis 2009 - Visual Genre Conventions and User Performance On The Web

!
!
!
!
!
!
!
!
!
!
!
!
!
!
Visual genre conventions and
user performance on the web
Thesis Submitted for the Degree of Doctor of Philosophy
Department of Typography & Graphic Communication
Luis Santa-Maria
December 2008
Declaration
I confirm that this is my own work and the use of all material other sources has been
properly and fully acknowledged.
Luis Santa Maria

December 2008
Abstract
One problem which designers are faced with is whether they should follow existing
conventions of a genre, or break with conventions, and innovate. Understanding the
relationship between conventions and user performance is essential in order for
designers to make informed decisions. Despite the fact that usability literature claims
that not conforming to genre conventions can cause serious usability problems, there
is not much empirical investigation on the topic.
Through a review of the literature, a framework was defined which established genre
conventions as having a visual, structural, linguistic and a functional component.
This thesis examines the impact that violation of the visual conventions of a digital
genre has on user performance. Web discussion forums were the category of websites
chosen to be investigated because of the likelihood that they are a truly digital genre
which is emerging. A survey was conducted to help establish the existing visual
conventions in the genre and to help define the experimental material. A series of
empirical studies were conducted to validate the experimental material and to
measure the performance of users familiar and unfamiliar with the genre when using
conforming and non-conforming web pages.
Results show that not conforming to visual conventions of a web forum genre results
in poorer performance and disorientation of participants familiar with the genre. But
results also indicate that this poorer performance and disorientation is short-lived
and participants who use a non-conforming forum over a brief period of time are
able to improve their performance and feel less disoriented. Furthermore,
participants who use the non-conforming forum page are able to maintain their
performance improvement when using another non-conforming web forum. Results
also show that performance varies depending on the conventionality of the website
and whether the user is retrieving information from the interface or the content.
Abstract
Finally, an empirical study was conducted on conforming and non-conforming

online news genre and the findings obtained in web forums were reproduced in this
other web genre.
These results suggest that although not conforming to visual conventions can cause a
decline in performance, the magnitude of the negative effect on user performance has
not been thoroughly investigated in usability literature. The fact that users can
quickly recover from problems caused by the violation of the visual conventions of a
genre brings a new perspective to the negotiation between innovation and
conventions. Furthermore, the results from the interaction between conventionality
and the type of task users perform can help inform design decisions based on a task
oriented approach. It also empowers designers to better consider when to conform
and when to violate conventions, and thus better estimate what impact their
decisions will have on the final users.
To Flavia for her love,
friendship and care.

Acknowledgements
The journey of a PhD is quite a long one and albeit solitary in many ways. In other
ways you are surrounded by people who touch your life in so many different
manners. Here I try to do justice to all these people who have in their own way
helped me through this remarkable journey:
! First and foremost I would like to thank my wife Flavia for all her love, care,
understanding, faith and motivation. Without her I would have never got
here.
! My parents, who have always given me their support and love across the
ocean and were always prompt to help with the technical issues in Brazil.
! To the Coordenação de Aperfeiçoamento de Pessoal do Ensino Superior
(CAPES) and all the people who work there for the sponsorship and the
friendly advice and help whenever I needed it.
! My supervisor Dr. Mary Dyson for walking this journey by my side. Her
professionalism, prompt feedback and critical sense were invaluable to me
and have taught me how to be an empirical researcher.
! Eric Kindel, Linda Reynolds and later Dr. Rob Waller for all their
contributions in the Research Advisory Group sessions.
! Dr. Anamaria de Moraes, who introduced me to empirical research and
whose advice made me consider doing a PhD abroad.
! Dr. Guilherme Cunha Lima and Edna Cunha Lima who encouraged me to
come to Reading to do my research.
! My good friends Dr. Rathna Ramanathan, David Lewis, Jo De Baerdemaeker,
Glenn, Dr. Heliana Pacheco and Dr. David Buss for their support,
camaraderie and great laughs.
! My friends Carlos Alberto Munoz & Vanessa Toniolo, José Eduardo &
Débora Guimarães, Matthew, Emma and Tristan Browne and Fernando and
Rília Varandas for their friendship and all the great times we had together
which kept me in touch with the ‘world outside my PhD’.
! My PhD colleagues who have gone on their own journeys and made me feel
less lonely: José Marconi, Luciane Fadel and Silvio Campello.
! Helen Apted for her kindness, cheerfulness and for brilliantly managing Old
Whiteknights House and always striving to make the life of all PhD students
who work there easier.
! Professor Stephen Taylor and Professor Pam DeNicolo for not only coming
up with the idea of the Graduate Schools, but also for making Old
Whiteknights House come true.
! My colleagues at Old Whiteknights House: Helen Ainslie, Colm MacCrossan,
Maura O’Reagan, Seonghye Moon and Nathalie Montillot for the
encouragement and silly debates which allowed me to remain sane.
! Brian Hoskins who has made adapting to life in the United Kingdom a lot
easier and has always been available to help.
! Finally, to all the people who selflessly volunteered to do my experiments,
specially the members of Webforumz.com who were very enthusiastic about
the whole idea.
Table of contents
List of figures vi!
List of tables xiii!
Chapter 1 Introduction 1!
1.1. Introduction 1!
1.2. Context 2!
1.3. Aims and scope of the thesis 3!
1.4. Methodology and structure of the thesis 4!
Chapter 2 Genres, conventions and usability 8!

2.2. Genres 8!
2.2.1. Definition 8!
2.2.2. Categorization into genres 9!
2.2.3. A genre framework from a design point of view 11!
2.2.4. Genres and technology 19!
2.2.5. The social aspect of genres – discourse communities 20!
2.2.6. The importance of visual conventions of a genre 21!
2.2.7. Section summary 22!
2.3. The relationship between conventionality and usability 22!
2.3.1. Defining Usability 23!
2.3.2. Usability and genre conventions 26!
2.3.3. Visual appearance and Usability 34!
2.4. Conclusion 38!
Chapter 3 Exploratory studies 40!

3.2. Exploratory study 1 - The effect of structured design on comprehension and structured recall
of electronic documents 41!
3.2.1. Background 41!
3.2.2. Access structures & typographic cues 42!
3.2.3. Types of knowledge 49!
3.2.4. Objectives 51!
3.2.5. Method 51!
3.2.6. Study design 54!
3.2.7. Participants 54!
3.2.8. Experimental Procedure 55!
3.2.9. Results 56!
i
3.2.10. Discussion 60!
3.2.11. Conclusion 63!
3.3. Exploratory study 2 - Visual appearance and the identification of the category of a website.
63!
3.3.1. Introduction 63!
3.3.2. Objective 66!
3.3.3. Method 66!
3.3.4. Participants 68!
3.3.5. Procedure 68!
3.3.6. Results 69!
3.4. Conclusion of exploratory studies 80!
Chapter 4 Finding visual conventions in web forums 81!

4.1.1. Web forums 81!
4.2. Survey 1 83!
4.2.2. Method 89!
4.2.3. Sample 91!
4.2.5. Results 92!
4.2.7. Most frequent groups of elements 106!
Chapter 5 Assessing the results of the survey in terms of user perceptions 109!
5.1. Experiments to test templates created based on the results of the survey 110!
5.1.1. Experiment 1 – Perceptions of conventionality of the templates based on the results
of the survey 110!
5.1.2. Experiment 2 - User perceptions of ease of use of the templates based on the survey
results 130!
5.2. Experiment 3 - Assessing the experimental material 137!
5.2.3. Method 139!
5.2.5. Results 146!
ii
5.3. Final remarks 150!
Chapter 6 Visual conventions and familiarity when searching in web forums 152!
6.1.1. Familiarity 152!
6.2. Experiment 4 – Visual conventions and searching for information in web forums 161!
6.2.2. Method 162!
6.2.4. Results 168!
Chapter 7 Visual conventions and performance over time 181!

7.1.1. Navigation and browsing strategies 181!
7.1.2. Expectations Conventions and usability 186!
7.1.3. Disorientation 192!
7.2. The pilot study 193!
7.2.2. Method 194!
7.2.4. Results 198!
7.3. The main experiment 212!
7.3.2. Method 212!
7.3.4. Results for part 1 of experiment 5 217!
7.3.5. Results for part 2 of experiment 5 227!
7.3.6. Summary of results 233!
Chapter 8 The effects of conventionality on users searching for information in an

online news pages 240!
8.2. Analysis of existing news websites 241!
8.3. Survey: Judgments of online news regarding the conventionality of the layouts 243!
iii
8.3.2. Method 243!
8.3.3. Results 246!
8.4. Experiment 6 – Effects of conventionality on searching for information on online news
websites 255!
8.4.2. Method 256!
8.4.4. Results 263!
Chapter 9 Conclusion 277!

9.2. Overview of the thesis 278!
9.3. Contributions 282!
9.3.1. Design 282!
9.3.2. Usability 285!
9.3.3. Final remarks on contributions 288!
9.4. Limitations and suggestions for future research 289!
Appendix A Texts used in the first Exploratory study 1 which examined conventional
access structures 292!
Appendix B Summary task and multiple-choice questions and internet usage

questionnaire used in the first exploratory study 295!
Appendix C Elements which helped identifying the website in the second

exploratory study 297!
Appendix D Criteria used for determining the screen position of a visual element for
the survey 300!
Appendix E Explanation of elements found in all three levels of a web forum 302!
Appendix F Tables with the symbolization mode frequencies of each element

appearing on each of the three forum levels 305!
Appendix G Tables with the configuration mode frequencies of each element

appearing on each of the three forum levels 307!
Appendix H Tables showing the survey 1 results for the most frequently occurring
groups of elements 309!
iv
Appendix I Templates used in experiments 1 and 2 to assess perceptions of
conventionality and ease of use 312!
Appendix J Results from experiment 3, the icon identification test 336!
Appendix K Results for Experiment 3 which asked participants to identify the

interactive elements of the non-conforming forum 340!
Appendix L Text introducing PDAs presented for participants of experiment 4

342!
Appendix M Tasks designed for the experiment 4 which tested the performance of
users familiar and unfamiliar with web forums on conforming and non-conforming
websites. 343!
Appendix N Tasks designed for the pilot study which was conducted in preparation
for experiment 5 344!
Appendix O Tasks used on both parts of experiment 5 345!
Appendix P Disorientation questionnaire applied on both parts of experiment 5

346!
Appendix Q Questionnaire used in Experiment 6 347!
References 348!
v
List of figures
Figure 1 – Model of the document design process as described by Waller (1987). 14!
Figure 2 - Model for the design process involving genres and changes and functional constraints
(based on the model developed by Waller (1987). 14!
Figure 3 - Design process which relies solely on the document genre and its conventions. 15!
Figure 4 - Design process which relies solely on the functional constraints of a document. 15!
Figure 5 - Design process which relies on the genre-functional constraints negotiation to achieve
the design solution. 16!
Figure 6 - Model of contextual factors which might influence the usability of a product. 25!
Figure 7 - Representation of the low space condition (Bernard et al., 2000). 44!
Figure 8 - Representation of the medium space condition (Bernard et al., 2000). 44!
Figure 9 - Representation of the high space condition (Bernard et al., 2000). 45!
Figure 10 - Enhanced page layout (left) and poor page layout (right). 48!
Figure 11- Sample of the unstructured text layout used int the experiment. 52!
Figure 12 – Sample of the structured text layout used in the experiment. 52!
Figure 13 - Mean reading time with standard error bars for the two layouts. 56!
Figure 14 - Mean multiple-choice scores for both layouts. 58!
Figure 15 – Mean multiple-choice structure question scores. 58!
Figure 16- Chart of relative frequencies of elements for webmail sites. 71!
Figure 17 – Chart of relative frequencies of elements for online news sites. 72!
Figure 18 - Chart of relative frequencies of elements for search engine sites. 73!
Figure 19– Chart of relative frequency of elements for e-commerce sites. 73!
Figure 20 – Chart of relative frequencies x elements for online banking sites. 74!
Figure 21 – Chart of relative frequencies of elements for online auction sites. 74!
Figure 22 - Relative frequency of words being used as cues by participants in every category. 76!
Figure 23 – Sample of an online auction website screen used in the experiment. 77!
Figure 24 – Sample of an e-commerce website with advert used in the experiment. 78!
Figure 25 - Sample of a search item extracted from a Swedish online news website. 87!
vi
Figure 26 - Sample of the user's view of a web page that extends outside the screen dimensions.
89!
Figure 27 - Grid used to determine the positioning of the elements on a web forum page. 90!
Figure 28 – Some elements on the first level of a web forum. 93!
Figure 29 – Some elements present in the forum index table of a web forum. 94!
Figure 30 - Sample of the topic moderator element in the forum index table. 94!
Figure 31 - Sample of the number of views in topic element on a forum index page. 95!
Figure 32 - Elements in the second level of a web forum. 96!
Figure 33 - Thread index table and its elements of a web forum. 96!
Figure 34 – Elements in a post page of a web forum. 98!
Figure 35 – Elements within the post area of a web forum. 99!
Figure 36 – most occurring Positions of the elements in the forum index page. 102!
Figure 37 – Least/non occurring positions of the elements in the forum index page. 102!
Figure 38 – Most occurring positions of the elements in the threads page. Elements in bold
italicized type appeared most frequently in two simultaneous positions. 103!
Figure 39 – Least/non occurring positions of the elements in the threads page. 103!
Figure 40 – Most occurring positions of the elements in the post page. Elements in bold italicized
type appeared most frequently in two simultaneous positions. 104!
Figure 41 – Least/non occurring positions of the elements in the post page. 104!
Figure 42 - Example of the main conforming template (left) and its three variations (right) on
symbolization mode, position and configuration mode (from top to bottom respectively) 112!
Figure 43 – Example of the main non-conforming template (left) and its three variations (right) on
symbolization mode, position and configuration mode (from top to bottom respectively) 113!
Figure 44 - Comparison between the main conforming menu element (with all attributes on their
most frequent values) and the menu variation where the configuration is at its least frequent
value. 114!
Figure 45 - Comparison between the main conforming menu element (all attributes on their most
frequent values) and the menu variation where symbolization is at its least frequent value 114!
Figure 46 - Comparison between the main non-conforming menu element (with all attributes on
their least frequent values) and the menu variation where the configuration is at its most
frequent value 115!
Figure 47 - Comparison between the main non-conforming menu element (with all attributes on
their least frequent values) and the menu variation where the symbolization is at its most
frequent value 115!
Figure 48 – Coding system used to name the forum templates. 116!
vii
Figure 49 - Thumbnails of the 24 conforming and non-conforming 117!
Figure 50 – Screenshot referring to the C11 template of the conforming group. This template has
the symbolization, configuration and position attributes at their most frequently occurring
values. 121!
Figure 51 – Screenshot referring to the N11 template of the non-conforming group. This template
has the symbolization, configuration and position attributes at their least frequently or non
occurring values. 121!
Figure 52- Sample of the non-conforming template from the threads page level which was chosen
more frequently as similar to existing web forums. This template had the elements in their most
common symbolization mode but their position and configuration in their least or non occurring
values. 124!
Figure 53 – Sample of the non-conforming template from the threads page level which was chosen
more frequently as similar to existing web forums. This template had the elements in their most
common symbolization mode but their position and configuration in their least or non occurring
values. 124!
Figure 54 - Elements present on the forum index page which also appear in the threads page. 127!
Figure 55 - Elements present on the threads page which also appear in the forum index page. 128!
Figure 56 - Elements in the post page which also appear in the forum index and threads page. 128!
Figure 57 - Sample of section 1 of the survey. 141!
Figure 58 - Sample of the non-conforming screen in section 2 of the survey. 143!
Figure 59 - Division of areas in the non-conforming forum as displayed in the third part of the
survey. 144!
Figure 60 - Sample of a hypothetical learning curve to explain the small difference in

performance betweeen experts and intermediate users found in Tabatabai and Shore (2005). 156!
Figure 61 - Sample of the index page design (level 1) of the conforming web forum. 166!
Figure 62 - Sample of the index page design (level 1) of the non-conforming web forum. 166!
Figure 63 - Formula for the data transformation and calculation of the rate of right answers per
second. 172!
Figure 64 – Overall performance (measured by the RPS rate) of familiar participants compared to
the unfamiliar participants. 172!
Figure 65 –Interaction between conventionality and task type. 174!
Figure 66 - Performance (measured by the RPS rate) on interface tasks compared to content
tasks. 174!
Figure 67 - Median value of the answer given to the question: ‘How would you compare the
layout of this web forum with other forums?’ (1=very different, 5=very similar). 175!
Figure 68 – Hypothetical illustration of a learning effect to explain the lack of difference in

performance between the familiar users tested with the conforming and the familiar users tested
with the non-conforming forum. 178!
Figure 69 - Sample of the convention-conforming website 188!
viii
Figure 70 - Sample of the non-conforming website 188!
Figure 71 - Sample of the Convention-conforming news homepage 189!
Figure 72 - Sample of the non-conforming online news homepage 190!
Figure 73 - Sample of the experimental setting used for the pilot study and experiment 5. 196!
Figure 74 - Task completion time for the non-conforming group. 198!
Figure 75 - Task completion time for the conforming group. 199!
Figure 76 - Mean time to complete each task in the non-conforming group. 200!
Figure 77 - Mean time to complete each task in the conforming group. 200!
Figure 78 - Overlapped graphs of the mean time to complete each task on the conforming and
non-conforming groups. 201!
Figure 79 – Overall mean time to complete each task. 201!
Figure 80 - Mean time to complete interface and content tasks by condition. 203!
Figure 81 - Mean number of right answers per task type from the conforming and non-conforming
groups. 204!
Figure 82 - Right answers per second (RPS) rate pert task type on the conforming and non-
conforming groups. 205!
Figure 83 - Number of total right answers over time and fit curve for the non-conforming group.
206!
Figure 84 - Number of total right answers over time and fit curve for the conforming group. 207!
Figure 85 - Mean revisits for the conforming and non-conforming groups. 208!
Figure 86- Mean time to complete each task of the non-conforming group 217!
Figure 87 - Mean time to complete each task of the conforming group 218!
Figure 88 - Number of right answers in the non-conforming group through the test. 219!
Figure 89 - Number of right answers in the conforming group through the test. 219!
Figure 90 - Right answers per second ratio formula. 220!
Figure 91 - Progression of right answers per second in the non-conforming group. 221!
Figure 92 - Progression of the RPS ratio for both groups in the first part of the test. 222!
Figure 93 – Average right answers per second rate (RPS) on content 224!
Figure 94 - Mean number of quits for each group on each type of task. 224!
Figure 95 - Mean number of right answers per group for each task type. 225!
Figure 96 - Average quits on content and interface tasks per group. 226!
ix
Figure 97 - Average time to complete each task in the conforming group. 227!
Figure 98 - Average time to complete each task in the non-conforming group. 228!
Figure 99 - Overlapped graphs of time to complete tasks on both groups. 228!
Figure 100 - Progression of right answers and correspondent fit curve for the conforming group.
229!
Figure 101 - Progression of right answers and correspondent fit curve for the non-conforming
group. 230!
Figure 102 - Average right answers per second rate (RPS) on content and interface tasks per
group in the second part. 232!
Figure 103 - Wireframes for the conforming online news pages designed for the experiment 242!
Figure 104 - Wireframes for the non-conforming online news pages designed for the experiment
242!
FIGURE 105 - PAGES OF THE CONFORMING ONLINE NEWS LAYOUT 244!
FIGURE 106 PAGES OF THE NON-CONFORMING ONLINE NEWS LAYOUT 244!
Figure 107 - Sample of a sheet used in the survey containing the two layouts 246!
Figure 108 - Number of times each visual attribute/element was mentioned as influencing 247!
Figure 109 - Number of times each visual attribute or element was mentioned as 248!
Figure 110- Common configuration of the components of a piece of news 249!
Figure 111 - Configuration of the components of a piece of news with the summary 250!
Figure 112 - Type of configuration of the elements of an article in the story page of online news
pages 250!
Figure 113 - Configurations of news pieces used in the non-conforming online news pages 251!
Figure 114 - Configuration used in the story pages of the non-conforming online news website
where elements of the article were dissociated by partially separating them. 251!
Figure 115 - Style and placement of the navigational menu considered as 252!
Figure 116 - Style and placement of the navigational menu in 253!
Figure 117 - Positioning and shapes of adverts commonly occurring on online news pages 254!
Figure 118 - Icons designed for each category and subcategory of the conforming online news site
257!
Figure 119 - Accordion menu designed for the non-conforming online news pages (a) with the
news category selected and (b) with the money category selected 258!
Figure 120 - Sample pages of the conforming news website 258!
Figure 121 - Sample pages of the non-conforming online news website 259!
x
Figure 122 - Sample of the experimental setting with the non-conforming website 260!
Figure 123 - Mean time spent on tasks per group 264!
Figure 124 - Mean time spent on tasks per group throughout the experiment 264!
Figure 125 - Mean right answers per group 265!
Figure 126 - Mean number of right answers per group throughout the experiment 266!
Figure 127 - Mean RPS per group throughout the experiment 267!
Figure 128 - Strategy 1 - Strategy used by the majority of the participants in the 269!
Figure 129 - Variation on strategy 1 that was the second most used strategy 269!
Figure 130 - Mean backtracking actions executed over time by participants 272!
Figure 131 - Template C11 of the forum index page, most frequently chosen as more like other
existing forums. 312!
Figure 132 - Template C12 of the forum index page 313!
Figure 135 - Template N11 of the forum index page 316!
Figure 138 - Template N14 of the forum index page, chosen the least number of times as more
like other web forums. 319!
Figure 139 - Template C21 of the threads page, most frequently chosen as more like other web
forums. 320!
Figure 140 - Template C22 of the threads page 321!
Figure 143 - Template N21 of the threads page 324!
Figure 145 - Template N23 of the threads page, chosen the least number of times as more like
other web forums. 326!
Figure 147 - Template C31 of the post page, most frequently chosen as more like other web
forums. 328!
Figure 148 - Template C32 of the post page 329!
xi
Figure 149 - Template C34 of the post page 331!
Figure 150 - Template N31 of the post page 332!
Figure 153 - Template N34 of the post page, chosen the least number of times as more like other
web forums. 335!
xii
List of tables
Table 1 - Conditions under which each group of participants was tested by Satzinger and Olfman
(1998). 30!
Table 2 - Participants' gender across conditions. 54!
Table 3 - Table with the correct answers for the multiple-choice tests by layout 57!
Table 4 - Comparison of overall comprehension results across conditions. 60!
Table 5 – Number of different screens captured in each category. 67!
Table 6 –Identification results on each category screen by participant per pass. 69!
Table 7 – Identification Ratio. 70!
Table 8 - A comparison between categories and their respective elements. A tick mark signifies
that the element indicated in the row has helped participants identify the respective website
category. 75!
Table 9 - No identification and misidentification per category. 76!
Table 10 – Number of participants that cited each unique element in each category. 78!
Table 11 - List of all elements found in the forum index page of web forums. The indented
elements are found within the forum index table. 92!
Table 12 - List of the elements found in the threads page of a web forum. The indented
elements are found within the threads table. 95!
Table 13 – List of the elements found in the post page of a web forum. The indented elements
are found within the threads table. 97!
Table 14 – Summary of the elements present in each of the three forum levels. 100!
Table 15- Most frequent groups of elements in each forum level (shown in columns). Names in
bold represent the elements which appear on all three forum levels and in italic the elements
which appear on two forum levels. 107!
Table 16 – Symbolization, position and configuration values for each conforming and non-
conforming screen. 118!
Table 17 – Summary of the chi-square tests on the number of times each group was picked as
more like other web forums. 123!
Table 18 – Least chosen and second least chosen conforming templates in each forum level. The
table also indicates the values of each attribute of the elements and the number of times
(frequency) each template was chosen. 125!
Table 19 – Number of times each template in the conforming group was chosen as more like
other web forum pages (results are out of a total of 48 comparisons per template per level. 126!
Table 20 – Templates most frequently and least frequently chosen as similar to other web forums
(results are out of a total of 48 comparisons per template per level). 130!
xiii
Table 21 - Summary of the chi-square tests on the number of times each group was picked as
easier to use. 133!
Table 22 - Frequencies that each template in the conforming 134!
Table 23 – Frequencies of the templates most frequently and least frequently chosen as similar to
other web forums (results are out of a total of 80 comparisons per template per level). 134!
Table 24 - Templates most frequently chosen in the conforming group. 136!
Table 25 – Templates with the most frequent (MF) and least frequent (LF) values for each
element attribute and which were used to design the conforming and non-conforming forums. 137!
Table 26 - Icons presented in the survey and the text content they were replacing. 141!
Table 27 – Correct answers for the areas highlighted in the third part of the survey. 144!
Table 28 - Most frequent icon associations and frequency (answers conforming to the meaning of
the icon are italicized). 146!
Table 29 - Grouped results for the identification of the areas of the non-conforming template. 147!
Table 30 - Distribution of groups within the familiar and unfamiliar categories and the forum
layout each group was tested on. 162!
Table 31 - Templates used to create each level of the conforming and non-conforming forums.
163!
Table 32 - How values were applied to the variables symbolization, position and configuration for
designing the forum templates. Values could be the most frequent (MF) or the least frequent
(LF). 164!
Table 33 - Questionnaire items with their respective extreme values meanings. 167!
Table 34 - Mean values and Standard Deviation of the time to complete tasks in each group and
condition with the significant differences highlighted and circled by a dotted line. The column
means shows the means of the users who used the conforming and non-conforming forums
respectively and the row means represent the means of the familiar and unfamilar users. 168!
Table 35 - Median values of the number of correct scores in each group and condition with the
significant differences highlighted and circled with a dotted line. The column medians shows the
medians of the users who used the conforming and non-conforming forums respectively and the
row medians represent the medians of the familiar and unfamilar users. 170!
Table 36 – Mean values of the RPS rates in each group and condition with the significant
differences highlighted and circled with a dotted line. The column medians shows the medians of
the users who used the conforming and non-conforming forums respectively and the row medians
represent the medians of the familiar and unfamilar users. 173!
Table 37 - Results for the two within-subjects one between-subjects factor ANOVA. 173!
Table 38 - Summary of the experimental tasks which were answered too quickly, too slowly or
showed great variability. 202!
Table 39 - One between-subjects, one within-subjects ANOVA results for time taken to complete
the tasks. 203!
Table 40 - One between-subjects, one within-subjects ANOVA on the number of right answers
between the conforming and non-conforming groups. 204!
xiv
Table 41 - One between-subjects, one within-subjects ANOVA reuslts for the right answers per
second (RPS) rate in the conforming and non-conforming groups. 205!
Table 42 - Questions in their order of appearance in the questionnaire. 215!
Table 43 - Comparison between the linear and quadratic models measures of accuracy for the
non-conforming group. 220!
Table 44 - Average right answers per second for each type of tasks on each counterbalancing
order. 223!
Table 45 - Mean RPS rate per task type per task type in the two counterbalanced situations. 231!
Table 46 - Summary of the results of each group in each part of the test. 233!
Table 47- Online news websites analysed to prepare the experimental material 241!
Table 48 - List of commonly occurring online news categories 256!
Table 49 - Tasks designed for the experiment 261!
Table 50 - Median values of the answers given in two instances 268!
Table 51 - Frequency of symbolization mode of the elements on the forum index page. 305!
Table 52 - Frequency of symbolization mode of the elements on the thread page. The indented
elements are found within the thread table. 306!
Table 53 - Frequency of symbolization mode of the elements on the post page. The indented
elements are found withing the post area. 306!
Table 54- Frequencies of configuration of elements in the forum index page. The indented
elements are found within the index table. 307!
Table 55- Frequencies of configuration of elements in the forum thread page. The indented
elements are found within the thread table. 308!
Table 56- Frequencies of configuration of elements in the forum post page. The indented
elements are found withing the post area. 308!
xv
Chapter 1
Introduction
1.1. Introduction
The Internet has seen its world usage grow more than 300% in the past eight years,
with approximately 1.46 billion people using it (Miniwatts Marketing Group, -
Internet Usage Statistics). Currently the total number of registered and active website
domains is reported to be over 100 million, and 74% of these are either commercial or
other sites operating with the .com suffix (DomainTools.com, 2008). In 2007, about
70% businesses in the United Kingdom had their own websites which were used for
selling products, giving access to catalogues or price lists, and providing access to job
vacancies and application forms (Pollard, 2008). This increase in web usage and in
businesses ‘going online’ is reflected in a higher demand for well-qualified web
designers (Burleson, 2004).
Besides the growing number of websites and the increase in businesses expanding on
the Internet, there is one other very frequent activity in which designers are involved,
that hardly gets mentioned in the statistics. Businesses are constantly updating their
websites, and some studies which monitored business websites for changes have
reported that most websites change over a one year period (Cosio & Dyson, 2002;
Leichty & Esrock, 2001) and sometimes changes are made after only one or two
months (Dontcheva, Drucker, Salesin, & Cohen, 2007; Tan, Foo, & Hui, 2001).
The majority of changes to business websites are made to the content of the page
(including text and update date), followed by changes to hyperlinks (change in the
URL of the hyperlink) and finally by changes in layout and images (Tan, et al., 2001).
Changes to the organizational structure of the websites seem to be less frequent, and
when they happen, they tend to be quite minor, e.g. inserting of removing a small
number of nodes (pages) to the website tree (Dontcheva, et al., 2007). Although
hyperlink changes may indicate modifications in the organizational structure of the
1
Introduction
website, this is not necessarily always true. For example, an e-commerce website may
stop selling one brand of nappies and start selling a new brand which can lead to a
change in the URL of the baby nappies hyperlink, but not to the organizational
structure of the website (i.e. baby nappies will still be located in the baby products
section).
Another situation which has risen with the popularization and advances in mobile
technology is the need for migrating websites to small screen devices (Albers & Kim,
2000; MacKay, Watters, & Duffy, 2004). This usually presents the designer with a lot
of difficulties, since it is extremely hard to maintain the visual appearance of a
website intact when moving to a reduced screen size. As a result, the original layout
of the page is changed, and that is said to affect the user experience (Lehtonen, et al.,
2006).
Thus, whether creating new sites or modifying existing ones, it is important that
designers be prepared to provide good and usable design solutions. To achieve this,
the designer has to juggle between different aspects of the design which sometimes
contradict each other (Borchers, 2001). Examples of the aspects involved in this
process, usually called ‘forces’, are usability, innovation and genre conventions. But
to be able to adequately judge the impact of these forces on the final product, the
designer needs to have access to information on each of them. Most of the times this
type of information is provided by research, but unfortunately, not all these aspects
have been thoroughly investigated and therefore not many sources are available for
designers to consult.
1.2. Context
Genres have been studied and discussed in various fields of knowledge such as
rhetoric, literature and folklore for many years. But it is only in the past twenty years
that the importance that genres play in the design of documents has begun to be
investigated (Bateman & Delin, 2001; Delin, Bateman, & Allen, 2002; Waller, 1987a).
Since then, some authors have highlighted the importance of the structural (Dillon,
1991; Vaughan & Dillon, 2006), linguistic (Dillon, 2000) and visual (Toms, 2001)
aspects of a document genre to its users.
Genres are bound by rules and conventions which create and maintain the
expectations of users (Vaughan & Dillon, 2006). Following these conventions is said
2
Introduction
to provide simplicity, consistency and predictability, and facilitate the use of the
document (Cappel & Huang, 2006). Some authors also warn that not conforming to
conventions makes users feel lost, confused and disoriented (Cappel & Huang, 2006)
and consequently usability will suffer (Nielsen, 1999a). The recommendations seem
to somehow link usability and genres, although the connection is not entirely clear.
The idea that not conforming to the conventions of a genre will cause usability
problems is in part based on the concept that users develop mental models of
documents which they interact with (Vaughan & Dillon, 1998).
As users become more familiarized with documents which belong to a genre, they
start to develop a mental model of the genre which is called ‘schematic knowledge’
(Vaughan & Dillon, 1998) or ‘superstructure’ (van Dijk & Kintsch, 1983). This
knowledge leads the user to have certain expectations about a document which
belongs to the genre they are familiar with. If, when interacting with the document
the user finds certain types of cues, the schema is triggered (van Dijk & Kintsch, 1983)
which can help users understand and remember the documents (Vaughan & Dillon,
1998). Some evidence exists to corroborate this theory, Dillon and Schaap (1996) and
Vaughan and Dillon (2006) have shown that not conforming to genre conventions
can hinder user performance. But other studies (Kalbach & Bosenick, 2006;
McCarthy, Sasse, & Riegelsberger, 2003) obtained results which challenge this
perception.
These conflicting results illustrate that more research is needed on genres and
usability, especially of electronic documents. Considering the enormous growth of
the web and the update frequency of websites it is clear that of all existing electronic
documents, websites are probably where this understanding is more urgently
needed.
1.3. Aims and scope of the thesis

This thesis aims at improving the understanding of how some of the forces, usability
and genre conventions more specifically, which are part of the document design
process, are related. This study has three main objectives. Firstly, guided by literature
on genre theory it attempts to identify the relationship between genres and the
design process. Then, based on usability and genre theory, it identifies a relationship
between these two forces which influences the design process. Secondly, it attempts
3
Introduction
to establish a systematic method for gathering information on conventional web

layouts. Thirdly, it investigates whether not conforming to the visual conventions of
a website genre affects user performance and disorientation.
It is important to point out that this thesis is inserted in the field of knowledge of
design which makes it an interdisciplinary work. As such it draws knowledge and
theories from the fields of linguistics, rhetoric, human-computer interaction and
psychology as well as graphic and interface design and typography.
1.4. Methodology and structure of the thesis

To achieve the aim and objectives described in the previous section, this study
follows the empirical approach which involves gathering information on phenomena
from observation or experimentation. Conclusions will be based on the analysis of
the resulting data which will include the adoption of statistical techniques.
The thesis is structured to reflect the progress of this research. As such, the next
chapter establishes the theoretical framework which will be used. This framework is
based on an overview of existing relevant literature. The subsequent chapters
describe empirical approaches taken to investigate issues relating to this framework.
In each of these chapters there is an introduction which contains a literature review
more focused on the specific issues relevant to the chapter.
Below is a more detailed description of what each chapter addresses:

Chapter 2 presents a review of the literature on genres and usability. The chapter
discusses how genres are linked with the design process and design patterns. It then
proposes the concept of genre conventions which can help describe a genre. Through
an analysis of usability theories, the chapter determines how genres relate to
usability.
Chapter 3 describes two exploratory studies conducted in the initial stages of the
research to probe different aspects of genre conventions as defined in the previous
chapter, and then determines the investigative path to be followed in the thesis. The
first exploratory study looks at how the presence of conventional text structuring
artefacts affects the comprehension and recall of a text. Results of the first exploratory
study indicate that when the content of a digital document is organized using
4
Introduction
conventional structuring artefacts, users seem to better recall the order in which the
content is organized.
The second exploratory study tests if people can identify the type of service provided
by a website based only on its visual appearance. The result of the second
exploratory study seems to indicate that users may be capable of identifying the
service provided by a website after a quick look at the visual appearance of the page.
This seems to be in accordance with the ideas that web genres are emerging (Dillon &
Gushrowski, 2000; Eriksen & Ihlström, 2000) and users can identify a document
genre by its visual appearance (Toms, 2001).
Based on the results from both exploratory studies, the research for this thesis will
focus on the visual aspects of digital genres.
From the outcomes of the two exploratory studies in chapter 3 and the literature
review in chapter 2, the web forum category was chosen to be used as the main
experimental material in this thesis. Chapter 4 explains the reasons for this choice
and presents the results of a survey which gathered the most frequently occurring
visual elements in a web discussion forum, along with their most frequent
symbolization mode (i.e. graphic, alphanumeric or mixed), position and modes of
configuration (see Twyman, 1979). The objective of this survey was to obtain
information on the conventional configuration of visual elements present on a
website to guide the preparation of the empirical material.
Chapter 5 contains three experiments which were conducted to test the results of the
survey in terms of user perception. The first two experiments described in the
chapter also tested for a correlation between perceived conventionality and ease of
use. For the first two experiments 24 web forum templates were created based on the
results of the survey. The layout of the templates ranged from one which conformed
to the conventions of the genre (based on the survey results), to one which did not
conform to the conventions (based on the survey results). The first experiment tested
the perceptions of the users on the conventionality of the templates. The second
experiment tested the perceptions of the users on the ease of use of the templates.
Based on the results of these two first experiments the templates for the conforming
and non-conforming1 web forum were chosen. The third experiment described in the
1
The term ‘non-conforming’ is used to describe the experimental material which does not conform
to the visual conventions of a web genre. Similarly the term ‘conforming’ is used to describe the
experimental material which conforms to the visual conventions of a web genre.
5
Introduction
chapter tested if participants were able to identify and recognize important aspects of
the non-conforming web forum. Results of this experiment showed which elements
and parts of the forum users had problems identifying and which they did not. This
knowledge was subsequently used to inform the development of the tasks used in
the experiments described in chapters 6 and 7.
Chapter 6 describes an experiment, which tested the time taken to complete tasks
and the number of correct answers of users familiar and unfamiliar with web forums2
when using conforming and non-conforming websites. The experiment also
investigated if the effect of conforming or not conforming to conventions on
performance is dependent on the type of task executed. Two different types of
experimental tasks were created: content-related tasks and interface-related tasks. In
the content-related tasks participants had to retrieve information from the content
whereas in the interface-related tasks participants had to retrieve the information
from the interface. Results showed that unfamiliar participants 3 had an equivalent
performance on both conforming and non-conforming websites but it also showed
no difference in performance of the familiar users in both the conforming and non-
conforming web forums. This lack of difference motivated the next experiment
(described in chapter 7), which further investigated this matter. This study also had
some methodological implications for the next experiment: it indicated the need to
improve the measuring apparatus used in the test, and also the need for
counterbalancing tasks.
Chapter 7 describes an experiment conducted to further investigate why the previous

study found no difference in performance between familiar users interacting with a
conforming and a non-conforming web forum. This experiment investigated what
happens with the performance of participants familiar with web forums over time.
2
The term web forums is used to indicate the plural of the word web forum. Although the plural of
forum is usually represented by the Latin word ‘fora’ the plural ‘forums’ is also grammatically
accepted and was chosen over the latter for being used in scientific articles (Abbasi & Chen, 2005;
Hanna & Nooy, 2003; Kazanis & Ginige, 2002) and the World Wide Web.
3
For reasons of simplification, throughout this thesis participants familiar and unfamiliar with
using web forums will be referred to as familiar participants and unfamiliar participants
respectively.
6
Introduction
Participants using the non-conforming website showed a performance improvement

throughout the experiment, while participants using the conforming site showed
virtually no performance improvement.
Chapter 8 describes the final experiment, which investigated whether the results
obtained from previous studies could be replicated for another web genre. For this
experiment the online news genre was used as experimental material. Results from
this experiment confirmed that participants using a non-conforming online news
genre improved their performance felt less disoriented as the experiment progressed.
Finally, Chapter 9 concludes this thesis by presenting an overview of the study and
discusses the practical and theoretical contributions of the work. It also points out the
limitations of the results and suggests areas for further research.
7
Chapter 2
Genres, conventions and usability
2.1. Introduction
This chapter describes the framework used in this thesis. It starts by looking at genre
theory and how that relates to the document design process. This relationship leads
to a suggestion of how genres can be linked with design patterns and points out the
importance of understanding genres. In sequence it explains the importance of
technology and discourse communities to the concept of genre. Finally, the chapter
attempts to describe the relationship between genre conventions and usability.
2.2. Genres
2.2.1. Definition
Several authors from different areas of knowledge have given various definitions of
the term genre in the past decades and this has made the concept somewhat blurred.
The etymology of the word shows that it originated from the Greek word ‘!"#$%’
which, according to the Liddell-Scott Greek-English lexicon, means ‘of a race, stock
or family’. It was later borrowed by the Romans and transformed into ‘genus’ which
means ‘of a kind’. Although both the Greek, and later Latin meanings, refer to groups
that share common traits, it is the Latin counterpart that can give some clues to the
contemporary use of the word.
Genre theory started with the study of rhetoric and according to Waller (1987a) the
idea of classifying types of discourse into groups seems to have been around since
the time of Aristotle.
The great majority of genre theories so far have been in the fields of literature,
linguistics, rhetoric and some in folklore and film, each having the concept of genre
adapted to its needs and idiosyncrasies. In an attempt to establish a broader
8
definition, Swales (1990) draws on ideas from four main fields that have developed
studies on the concept of genre: folklore, literature, linguistics and rhetoric. His
definition is indeed far reaching as it states that:
A genre comprises of a class of communicative events, the members of which share

some set of communicative purposes. These purposes are recognized by the expert
members of the parent discourse community, and thereby constitute the rationale for
the genre. This rationale shapes the schematic structure of the discourse and
influences and constrains choice of content and style (Swales, 1990, p.58).
There are many interesting concepts in Swales definition that are tightly connected
with genres and are useful as a starting point to understanding genres. If genres are
classes of communicative events, the involvement of at least two agents is necessary
for a genre to be established, and it requires that the individual on the recipient end
of the communicative process be familiarized with the structure, content and style of
the discourse.
When a group of people becomes familiarized with these aspects of a discourse

(structure, content and style) that shape a genre, they develop a symbiotic
relationship with the genre, and will simultaneously be able to decode and adapt it to
their communication needs. These groups of people are called discourse
communities by Swales (1990), and this concept will be addressed in more detail in
sub section 2.2.5. Although Swales (1990) suggests that the rationale for a genre
shapes the stucture of the discourse and influences content and style, he does not go
further into defining a way of categorizing documents into genres.
2.2.2. Categorization into genres

A concept that seems to be shared by some authors is one that associates the
categorization of discourses into genres using a framework formed by the principles
of rhetorical substance, form and rhetorical action (Miller, 1984). The concepts of
substance and form are referred to by Yates and Orlikowski (1992, 2002) as also being
characteristics of a genre. The authors explain that substance refers to the content as
well as the social circumstances expressed in the communication, and form refers to
the physical attributes and linguistic features of the communication. This expands the
framework proposed by Miller (1984) allowing for the physical and social aspects of
discourse to be considered.
9
With the advent of the Personal Computer and the Internet, a further expansion to
the conceptual model of genre was proposed to include another component besides
substance and form (Shepherd & Watters, 1999). Because users approach Internet
documents with expectations with respect to functionality as well as form and
content, Shepherd and Watters (1999) propose that a web genre (named cybergenre
by the authors) be characterized by form, content(i.e. substance) and functionality.
Considering that the focus of this thesis is on graphic design and typography, the
form component of a genre is of particular interest.
The role of the visual aspect of a document in genre categorization

Because most of the genre theorists come from backgrounds such as rhetoric,
linguistics and literature, the form component of a genre has previously been
associated with the linguistic aspects of communication rather than its visual
attributes. However, as seen earlier, some authors have mentioned the visual
appearance of a document in their description or analysis of a genre (Yates &
Orlikowski 1992, 2002) but despite that, not many authors have studied the visual
aspect of a genre or its impact on genre identification and classification.
The importance of the visual appearance of a genre has been illustrated in an

experiment conducted by Toms (2001). To try to understand the role of document
layout in genre recognition, the study compared three different versions of the same
document:
! content version, where all structure, layout and formatting were reduced to a
minimum (leaving only the words, phrases and sentences).
! form version, where structure, layout and formatting were preserved and all
characters were replaced by ‘X’s or ‘x’ and digits were replaced by ‘9’s thus
excluding the content.
! full version which had structure, layout, formatting and content intact.
Participants were presented with these versions and were asked to identify what the
page was.
As expected, the full version had the higher identification rate, which suggests that
content and form are both important to genre recognition. However, the results also
showed that participants who were familiarized with one specific genre performed
10
better at identifying it through the layout (form version) rather than just through
content (content version) (Toms, 2001). Also, when content from one literary genre
was formatted by the rules or conventions of another genre (e.g. a letter content
formatted as a dictionary), the visual aspect was the first clue people used to identify
the genre (e.g. in the example identifying a personal letter as a dictionary).
This experiment illustrates the important role that the visual aspect of a genre plays
in the identification of a genre. Allowing users to identify a document genre
accurately quickly enhances their interaction and makes the use of the document
more efficient (Toms, 2001). In spite of that, not many genre theories have considered
the visual appearance of genres in the depth which this empirical study suggests is
needed. One exception is the work of Waller (1987a) which established a framework
of genre bringing together language content and visual appearance. Waller also
proposed in his framework how genres are inserted and contribute to the design
process. This is extremely relevant to understanding that genres act as forces in the
design process.
2.2.3. A genre framework from a design point of view

The analysis of document genres from a typographical point of view and the model
of the design process of a document proposed by Waller (1987a) are both highly
relevant factors to this thesis. Although some other authors have developed Waller’s
model further (Bateman & Delin, 2001; Delin, et al., 2002) this thesis will base its
framework on the concepts of conventional genre structures (Waller, 1987a, p. 180)
and the model of the design process developed by Waller (1987a, p. 298).
Waller (1987a) proposes that any printed document can be accounted for by a
combination of three structures (Waller, 1987a, pp.179-180):
Topic structure – Typographic effects with the purpose of displaying information
about the author’s argument.
Artefact structure - The result of the medium where the document is being displayed
and its production technology.
Access structure – The features which make the document usable and accessible to
readers.
The separation of these three structures is theoretical as in practice these structures

overlap and in most genres they are so strongly linked that it can become hard to
think of an alternative way of designing the document (Waller, 1987a). These three
11
structural concepts form the basis for describing genres of documents (Waller, 1987a,
p. 180). But because of the strong links between them, and the overlapping of these
concepts, it can be quite hard to use these structures to describe the genres of
documents.
This thesis proposes another set of concepts which can be used to describe genres of
documents. These concepts represent the types of conventions which can exist within
a document genre (genre conventions). These genre conventions have their origin in
the functional constraints proposed by Waller (artefact, topic and access structures)
which state that when a document genre becomes more stable, these functional
constraints tend to become highly conventionalized. The proposed concept of genre
conventions consists of structural conventions, linguistic conventions, visual
conventions and functional conventions. Although these four types of conventions
also overlap and changes in one might affect another, it can be easier to separate the
contribution of each type of convention to the genre, and to pinpoint where
overlapping may occur. This makes them more appropriate for the work developed
here.
Structural conventions describe how information is organized within a genre of

documents.
Linguistic conventions describe the semantic, syntactical and lexical aspects of the
genre (e.g. choice of words and abbreviations).
Visual conventions describe how information is visually presented in a genre.
Differently from the functional constraints defined by Waller (1987a), the functional
conventions describe how the components of a particular genre respond when the
user interacts with them. They are directly linked with the artefact used to portray
the document.
Waller (1987a) proposed a model which allowed him to represent the role of genre in
the process of designing a document. What is not clear from Waller’s model (see
Figure 1) is whether functional constraints are part of a genre (Waller, 1987a, p. 180)
or if they are detached entities, since the author suggests that a negotiation process
between genre and functional constraints can occur during the design process.
Waller himself gives a clue as to how this apparently contradictory situation can
occur:
12
When those technologies change, it may be necessary to separate out the three
categories of functional imperatives in order to reassemble them to suit the new
constraints of the new technology (Waller, 1987a, p. 181).
In fact a change in technology or even a change of media provides new possibilities

(and limitations) for the artefact structure of a document, and a designer has then to
negotiate between the genre and the new functional constraints determined by the
technological change. Thus, even though functional constraints over time do become
part of the conventions of a genre, any changes in them may force a renegotiation to
happen. Although technological change is probably the most common change in the
functional constraints of a document, it is not by any means the only one, as topic
structures or access structures can change from existing genre conventions and
require the negotiation process between genres and functional constraints to happen.
For the work proposed here, some adaptations to Waller’s model of the design
process are needed. First, ‘genre’ in the model will be replaced by genre conventions,
since in principle, the negotiation process occurs between the conventions which
describe a genre and the functional constraints. Second, the term design task used by
Waller (shown as ‘task’ in his diagrams) will be replaced by the term ‘design
problem’ which can be considered as happening prior to when the design task is
determined since a design task is usually established to solve a problem which arises.
The belief adopted here is that the negotiating process between genre conventions
and functional constraints is a continuous process which is initiated upon detection
of a problem before a task is defined.
13
FIGURE 1 – MODEL OF THE DOCUMENT DESIGN PROCESS AS DESCRIBED BY WALLER (1987).
Genre
Design
Task Solution
Functional
Constraints
(Topic, artefact &
access structures)
FIGURE 2 - MODEL FOR THE DESIGN PROCESS INVOLVING GENRES AND CHANGES AND
FUNCTIONAL CONSTRAINTS (BASED ON THE MODEL DEVELOPED BY WALLER (1987).
Genre Conventions
(structural, linguistic, visual
and functional)
Design Design
Problem Solution
Functional
Constraints
(topic, artefact &
access structures)
According to Waller (1987a), the design process can then occur in different ways:
A) The designer can rely solely on the genre to provide a design solution (Figure 3).
This solution is extremely conventional and is likely to meet the expectations of the
users familiar with using that particular document genre. However innovation is not
likely to happen here and in case of technological evolution occurring, the document
might not be adapted to the media in which it is being portrayed.
14
FIGURE 3 - DESIGN PROCESS WHICH RELIES SOLELY ON THE DOCUMENT GENRE AND ITS CONVENTIONS.
Genre Conventions
and functional)
Design Design
Problem Solution
B) The designer considers in detail the functional constraints when generating the
document, but does not look at the existing genre conventions for it (Figure 4). As a
result, the document is likely to be innovative and well adapted to the media in
which it is conveyed but users who are accustomed to the conventions of the genre
may not know which strategy to use when interacting with the document.
FIGURE 4 - DESIGN PROCESS WHICH RELIES SOLELY ON THE FUNCTIONAL CONSTRAINTS OF A DOCUMENT.
Design Design
Problem Solution
Functional
Constraints
(topic, artefact &
access structures)
C) The designer either comes up with a conventional genre conforming solution, and
then validates it against the functional constraints, or designs through an analysis of
the functional constraints, and then checks the solution against the genre conventions
(Figure 5). These two ways of designing are healthier than the previous ones (Waller,
1987a) and here a negotiation occurs between the genre and the functional
constraints. When the design process occurs in this manner it can provide solutions
which conform to the expectations of users and are better adapted to the possibilities
of the media in which the document is portrayed. It can also allow for innovation to
occur and it is possible that this is the path followed in order for genre evolution to
happen.
15
FIGURE 5 - DESIGN PROCESS WHICH RELIES ON THE GENRE-FUNCTIONAL

CONSTRAINTS NEGOTIATION TO ACHIEVE THE DESIGN SOLUTION.
Genre Conventions
and functional)
Design Design
Problem Solution
Functional
Constraints
(topic, artefact &
access structures)
For designers to be able to make good choices in the negotiation process between the
genre and the functional constraints, it is necessary that they are aware of the
possibilities and limitations imposed by both the functional constraints and the genre
conventions. It is also important that they know what impact choosing one or the
other will have on the users of the document. Only then can the outcome of the
process provide a good design solution.
Design solutions, genres and patterns.

Good design solutions are also one of the main principles behind pattern languages
since a pattern can be thought of as a good design solution to a recurring problem.
This definition seems to indicate the existence of a relationship between genres and
design patterns.
The concept of pattern languages was developed by the architect Christopher

Alexander as a way of describing ‘problems which occur over and over again in our
environment’ (Alexander, Ishikawa, & Silverstein, 1977) and the solution to the
problem. The concept of pattern languages has been adopted not only in architecture,
but also in software engineering and Human-Computer Interaction (HCI).
In the field of HCI the concept of a pattern language has been gaining momentum
partly because HCI guidelines seem to be problematic when it comes to describing
design solutions. HCI guidelines focus mostly on creating coherent user interfaces
16
rather than on the necessary knowledge for proper UI design such as knowledge of
the user, the context and the task (Granlund, Lafrenière, & Carr, 2001). In fact, some
authors claim that HCI patterns are closer to architectural patterns than software
engineering patterns, since they are less rigid, and less detailed thus allowing more
creative freedom (Granlund, et al., 2001) and they also share Alexander’s ideas of
user-centered and participatory design (Borchers, 2001).
The main motivation for creating a pattern language in HCI is to have a way of
collecting, formalizing (Granlund, et al., 2001) and disseminating HCI design
knowledge (Griffiths, Pemberton, Borchers, & Stork, 2000). It can also provide a
common language of HCI design to anyone involved in the design, evaluation,
development and use of interactive systems (Borchers, Fincher, Griffiths, Pemberton,
& Siemon, 2001). But pattern languages are supposed to do much more than just
allow writing good design ideas so that they can be discussed (Alexander, 1999).
Pattern languages are ‘a more universal model of structuring design knowledge.
They can be used in any discipline that requires structured creative work of some
kind to be carried out’ (Borchers, 1999). This type of work is precisely that of graphic
designers, typographers and information designers.
A pattern language is a hierarchical structure comprised of several design patterns

and when it comes to interface design a pattern ‘describes possible good solutions to
a common design problem within a certain context’ (Tidwell, 1999). Although there
are slight differences in the exact components of an interface design pattern, most
authors agree that an HCI design pattern has the following components (Borchers,
1999, 2001; Granlund, et al., 2001):
! Name – This identifies the pattern and its central idea.
! Examples – Is used to clarify the problem. It shows existing situations in
which the problem can be encountered and how it has been solved in these
situations.
! Context – Describes the objective of the design, specifying the user and task
requirements
! Problem – Describes the design problem, the major design issues that the
pattern addresses.
! Forces – Describes all the factors that influence the design either directly or
indirectly. As mentioned earlier in chapter 1, section 1.3 this thesis aims to
contribute to the understanding of some of these forces.
17
! Design solution – Generalizes from the examples a proven way to balance the
forces for a given design context.
Although from the description of a pattern it is possible to notice some similarities

with genres, the relationship between the two is not entirely clear. This relationship,
however, seems quite relevant not only in the HCI domain, but in the fields of
graphic and information design. Some authors have attempted to link the concept of
genres with design patterns and pattern languages. Pemberton (2000) explains that
genres may be a useful tool for structuring design pattern sub-languages and
facilitating the perception of links between them. But the question of whether genres
are the same as pattern languages or if they overlap remains unanswered. In an
attempt to link them Åkesson (2003) points out the following similarities between
genres and pattern languages:
! Genres are formed from recurrent communication acts and design patterns
from recurrent design problems. When design problems are
communicational both concepts coincide.
! Both change over time.
Pemberton (2000) proposes that the development of design patterns for each genre, in
other words, representing the conventions of a genre through design patterns could
be a way of ‘packaging the knowledge’ of the genre conventions in a way which
would simplify the design task and allow for a better critical analysis of the
implications of genre membership.
Even though genre conventions may have their origins in design solutions based on
certain functional constraints and at a certain point they may be a good design
solution to a problem, this may not be perpetually true. It might be that any or all of
the functional constraints have changed, and the solution which the genre provides is
not a good one anymore. Thus it is not always viable to have a genre described
through pattern languages. When the functional constraints change, achieving a
good design solution requires negotiation between the genre and the functional
constraints. The design solution which will then result from this process can be used
as basis for the design pattern that will represent a better solution to the original
problem, and consequently can be incorporated into a pattern language.
This shows the necessity of having a good knowledge of the effects that the genre
conventions may have on the users of a particular document genre, since that will
18
improve design solutions and consequently allow for the development of

appropriate design patterns. This thesis will not aim at explaining the relationship
between genres and design patterns. It will focus on genre conventions so that it can
contribute to increasing the knowledge of genres and understanding the role genres
play in the design of documents.
2.2.4. Genres and technology

The influence of the medium used to convey a message on the visual appearance of
the message itself can be easily observed. Changes and development in the media
used to portray documents are usually connected with technological advances.
Technological evolution causes the media or artefact used to portray a document to
change thus allowing genres to change via modifications in the artefact structure
functional constraint.
Usually whenever a new technology of communication surfaces, the appearance of

the messages is shaped to adjust to and take advantage of this new technology. Frye
(1957) comments on the new possibilities that the printing press brought to the lyric
poetic style; he says that it developed ‘an increasing tendency to address the ear through
the eye’ (Frye, 1957, p.278). This statement clearly illustrates the new possibilities that
the printing press offered for the laying out of a lyric style text.
In a similar way, the computer and later the WWW, have again allowed for new
possibilities in communication which have resulted in the emergence of new genres
(Dillon & Gushrowski, 2000) or the adaptation of existing ones (Santini, 2005). This is
corroborated by the results of a study carried out by Yates, Orlikowski and
Rennecker (1997) which have shown that the introduction of a new communication
medium in an organization can generate variations on an existing genre or even the
emergence of new genres.
Genre variations & emergence of new genres

In the beginning of the WWW, many web pages seem to have used existing paper-
based genres instead of taking advantage of the technological changes of this new
communication medium to develop new genres or modify existing ones (Dillon &
Gushrowski, 2000). This argument apparently has some truth in it since there is
evidence that some paper-based genres seem to be adapting to the world wide web
(Eriksen & Ihlström, 2000). But also, some web genres without an equivalent in print
19
such as the personal home page (Dillon & Gushrowski, 2000) have appeared. It is
important to also have in mind that when the World Wide Web was initially
conceived, there were great restrictions in the kind of information that could be
displayed and how it could be presented.
As seen in the previous section, this transfer of previously existing genres to a new
technology is not a new phenomenon as it seems to have occurred previously when
technology provided improved or new media which could be used to convey a
document. A classical example occurred with the printing press where the first
printed books imitated a genre that was known at the time - the manuscript
illuminated book (Love, 1985). This imitation was not only visual (by copying the
manuscript letter styles and using decorated capitals), but also linguistic by using the
same written style and abbreviations that medieval scribes had used. It was not until
later that the old manuscript document genre was adapted to the new technology
and new genres were created.
This shows that in order to better understand a genre and to be able to make
appropriate design decisions between genres and functional constraints, it is also
important to have a good understanding of the technology involved with the artefact
in which the document is presented, and the technology relating to it.
2.2.5. The social aspect of genres – discourse communities

As mentioned earlier, the idea of genres is directly associated with social interaction
and this statement holds true in all fields that use the concept; in fact there is no sense
in talking about genres devoid of a social context.
Genres emerge from communicative needs of discourse communities under certain

circumstances and certain constraints. Often the abilities to identify, decode and
understand them are particular to those groups of people.
The familiarity of these discourse communities with the particularities of a genre

creates in their members expectations regarding the structural, linguistic, visual and
functional features of the same genre of documents. These particularities are
governed by the genre conventions and the expectations of members of the discourse
community are in line with them.
20
Although discourse communities are mentioned in genre literature (Østerlund, 2006;

Swales, 1990; S. Yates & Sumner, 1997), concerning the computer environment, some
have long doubted if it is possible to have social groups within this context. If this is
the case, it is highly unlikely that genres can emerge from the World Wide Web.
However, evidence shows that a very strong sense of ‘belonging to a group’ regularly
appears in members of web-based groups (Wallace, 1999). This is a strong argument
in support the idea that there are digital genres emerging in the WWW (Dillon &
Gushrowski, 2000; Ihlström & Åkesson, 2004; Shepherd & Watters, 1999).
2.2.6. The importance of visual conventions of a genre

As defined earlier, visual conventions of a genre describe how the information is
visually presented in a document from that genre. In design terms, visual
conventions are tightly linked to the formatting and layout of the document.
According to Waller (1999), the typographic genre originates from ‘design

imperatives’ which initially are genuinely functional but over time may lose their
original purpose and become signals to a genre. Analysing this concept under the
light of the typographic genre defined in Waller (1987a) and the framework
established for this thesis, the ‘design imperatives’ cited by Waller (1999) can be seen
as one of the originators (if not the only one) of visual, structural and in the case of
web genres, possibly some of the functional conventions of the genre.
As seen earlier, a series of studies carried out by Toms (2001) in genre recognition
shows that the formatting and layout of a document played an important role in
helping users identify to which genre it belongs.
Beghtol (2001) concurs with these findings by mentioning that a genre might be
identified by its layout. The author adds that, when no distinct layout clues are
available to help the user make a decision regarding the genre of the document, the
need to read the content in order to associate it to the genre emerges.
Apparently a user who is familiar with a particular genre (e.g. a member of a

discourse community) has an organized, high level mental representation of the
documents belonging to that genre (Vaughan & Dillon, 1998). Whenever a cue which
relates to the genre is identified, the user activates the relevant representation from
semantic memory (van Dijk & Kintsch, 1983) and this, according to Vaughan and
Dillon (1998) can aid recall, encoding and comprehension of documents.
21
2.2.7. Section summary

In this thesis, a genre is described by a set of genre conventions, namely: structural,
linguistic, visual and functional conventions. It is believed that the best possible
design solution to a design task comes from the negotiation between the genre
conventions and the functional constraints of a document.
Technological advances can originate new media or promote changes in existing ones
which can in turn alter the functional constraints of a document. This can not only
trigger the evolution of a genre, but also account for the emergence of new ones.
Conforming to the visual conventions of a document genre seems to facilitate

immediate identification of the genre by the members of its discourse community. It
may also trigger in these users a mental structure which may improve the recall,
encoding and comprehension of the document.
Finally, discourse communities (i.e. groups of people who share knowledge and
understanding of a genre and its conventions) are an essential requirement for a
genre to exist. These communities of users have certain expectations which are based
on the genre conventions they share, and it is when these expectations are not met
that usability problems with the document may appear. But how genre conventions
relate to usability is not entirely clear and needs to be further examined.
2.3. The relationship between conventionality and usability

This section will attempt to establish whether the conventions of a genre are in any
way related to the usability of a document belonging to that genre. It is expected that
this will help determine, for example, if not conforming to genre conventions can be
considered a usability problem, or a different type of problem altogether.
In order to establish this, existing definitions of usability and different views on how
usability can be measured will be examined. Through an analysis of the concept of
consistency, a link between genre conventions and usability will be formed. The
analyses of existing research will help define if there is a relationship between genre
conventions and usability and what this relationship might be.
22
2.3.1. Defining Usability

The term usability started to appear in Human-Computer Interaction literature in the
beginning of the 1980s. Although many attribute its creation to Nielsen (1985), the
term was being used long before by Gould and Lewis (1981), Shackel (1984) and
Eason (1993). Usability seems to have derived from the concept of “user friendly”
which was commonly associated with computers and machines in the 1970s.
According to Nielsen (1993, p. 23) the term “user friendly” was inappropriate
because it put users into one single dimension, when in reality the needs of different
users are different (Bevan, Kirakowski, & Maissel, 1991). Van Welie, van der Veer
and Eliëns (1999) present another possibility for the replacement of “user friendly”
by usability by saying that in the early 1980s the term “user friendly” started to
acquire some ‘vague and subjective connotations’.
It seems, however, that the term usability did not improve matters much, and there is
still no agreed upon definition of what usability means (Seffah & Metzker, 2004). The
term usability has been used with different meanings which makes the concept a
very confusing one (Bevan, et al., 1991).
The many different aspects of usability were examined by the MUSiC project in an
attempt to devise a consistent quantitative method of testing usability. The project
which was set in the 1990s was coordinated by Jurek Kirakowski with Nigel Bevan as
a collaborator. The objective of MUSiC was to develop a method of measuring
usability that would be broad rather than restrictive. To do this, the researchers
examined existing concepts and views on usability. According to Bevan et al. (1991)
the many definitions of usability that exist derive from a number of different views
on what usability is. Of these many definitions, Bevan et al. (1991) highlight four
views which relate to how usability should be measured:
! The product-oriented view, that usability can be measured in terms of the
ergonomic attributes of the product;
! The user-oriented view, that usability can be measured in terms of the mental
effort and attitude of the user;
! The user-performance view, that usability can be measured by examining
how the user interacts with the product, with particular emphasis on either
ease-of-use (how easy the product is to use) or acceptability (whether the
product will be used in the real world);
23
! The contextually-oriented view which claims that the usability of a product is

a function of the particular class of users being studied, the task they perform
and the environment in which they work.
According to Bevan et al. (1991), the contextually-oriented view can complement the
other three views on usability.
One definition which is mostly accepted by usability practitioners and researchers is

the one given by the standard ISO 9241-11 (ISO/IEC 9241-11, 1998, p. 2). The ISO
definition is performance-oriented, user-oriented and most importantly,
contextually-oriented and it says that usability is the
“Extent to which a product can be used by specified users to achieve specified goals
with effectiveness, efficiency and satisfaction in a specified context of use” (ISO/IEC
9241-11, 1998, p. 2).
Where:
Effectiveness is the accuracy and completeness with which users achieve specified
goals;
Efficiency is the resources expended in relation to the accuracy and completeness
with which users achieve goals;
Satisfaction is the freedom from discomfort, and positive attitudes towards the use
of the product (Bevan, et al., 1991; Bevan & MacLeod, 1994; ISO/IEC, 1998);
The context of use of a product is thought of as composed of the characteristics of the

users, the tasks they perform and the environment where the product is used (Bevan
& MacLeod, 1994).
The fact that usability is dependent on the context in which the product is used
increases the complexity of attempting to measure usability since it becomes a
mutable attribute of a product which can vary according to certain factors. This view
is shared by some authors (Bevan & MacLeod, 1994; Holscher & Strube, 2000;
Tabatabai & Shore, 2005) who claim that the characteristics of the context can be as
important for usability as the characteristics of the product. The authors add that
relevant changes to the context may change the usability of a system.
Thus, it is possible to conclude that there are two main factors which can affect the
usability of a system: the product features and context. These factors and their
24
relation to usability can be represented in a diagram (Figure 6). But it is important to

notice that the diagram below is a simplification as it does not illustrate inter-
relations which may occur between the contextual factors themselves. The
environment might influence the user or the type of task may affect users with
different characteristics in diverse ways.
FIGURE 6 - MODEL OF CONTEXTUAL FACTORS WHICH MIGHT INFLUENCE THE USABILITY OF A PRODUCT.
Contextual factors
User Task
Environment
characteristics
Usability
Product
features
Although the environment and the task are important contextual factors which
influence usability, understanding the user characteristics which affect it is the factor
which falls within the scope of this research.
User characteristics which may influence usability (such as familiarity with a genre)
can be hard to determine, and even though much research has been done on the
topic, more investigation is needed. However, results from existing research indicate
some characteristics of the user which affect user performance, and consequently can
change the usability of the product. These characteristics include user expertise
(familiarity) with a system (Scott & Schwartz, 2007), metacognitive skills
(Marchionini, 1995), age (Jenkins, Corritore, & Wiedenbeck, 2003; Wildemuth, 2004),
domain knowledge (e.g. great knowledge of a topic being searched in an interface)
(Lidwell, Holden, & Butler, 2003). Although these are only some user characteristics
which can influence the usability of a product, the user characteristic which seems to
25
be most frequently addressed on usability guidelines, heuristics and

recommendations, is the user familiarity with the system.
2.3.2. Usability and genre conventions

One fundamental design principle which has been widely adopted by usability
practitioners is that of consistency. This basically determines that related parts of a
system should be expressed and behave in similar ways (UPA, 2005).
The Usability Professional’s Association lists following appropriate

standards/conventions for a system as one of its principles of usable design (UPA,
2005). It adds that the actions, terminology and commands inside an application
should be used consistently (Nielsen, 1994). A similar recommendation is given by
Nielsen (1994) who says that ‘users should not have to wonder whether different
words, situations, or actions mean the same thing’. The author then adds that
platform conventions (i.e. the conventions of the system) should be followed.
Despite the fact that the recommendations made by the UPA and Nielsen show a
justifiable concern with consistency within a system, they do not explicitly address
another possible scenario very common in the web environment; that of different
websites which belong to the same category (e.g. yahoo mail, hotmail which belong
to the webmail category). This other scenario is addressed by Bevan (1997) who
points out that for a website to be consistent it should ‘meet user expectations by
following conventions established by other major sites’. He is supported by Jakob
Nielsen with what he describes as his ‘Law of Web User Experience’.
“Users spend most of their time on other sites. Thus, anything that is a convention
and used on the majority of other sites will be burned into the users’ brains and you
can only deviate from it on pain of major usability problems”(Nielsen, 1999a) .
From these different recommendations it becomes apparent that, at least for the
WWW, there are two ways in which a website can be consistent. One, which is
addressed in the recommendations of Nielsen (2005) and UPA (1997; 2003) is the
internal consistency which considers the consistency of the elements within the
system. The other, called here external consistency, is concerned about the
consistency between the system and other similar existing systems. In this sense, a
genre of digital documents can be thought of as being externally consistent. The
external consistency of a system is addressed in the recommendation of Bevan (1994)
and Nielsen (1989). There is yet a third type of consistency defined by Grudin (1989)
26
which relates to the correspondence between the interface elements and elements
which exist in the ‘world beyond computing’ (Nielsen, 1993). This type of
consistency, although not addressed in this thesis, will be named consistency with
the world for future reference.
The concern of usability practitioners and researchers with consistency is justified

since one of the attributes of usability as defined by Nielsen (1993) is learnability.
According to Nielsen, systems should be easy to learn and keeping internal
consistency is likely to facilitate learning. This happens because elements are kept
consistent across an interface so that the user only has to learn what they mean and
what they do once. Internal consistency is also likely to reduce the amount of
information the user has to learn because each element will be represented and will
function in the same way all across the interface.
External consistency can be thought as being related to another important attribute of

usability described in Nielsen (1993); efficiency. According to Nielsen, after users
have learned the design of the system they should perform tasks rapidly and
therefore perform more efficiently. Implicit in this statement is the idea that while
learning the design of an efficient interface, users do not perform as fast as they
could. From this follows that if an interface is consistent with another interface that
the user already knows it will cut down system learning time and draw upon user
memory thus improving the efficiency of a system and resulting in a more usable
interface.
Consistency, user expectations and conventions

Most descriptions of consistency involve following conventions in some way or
other. The internal consistency of an interface is usually achieved through a set of
rules defined by the developers of a system to which the interface belongs and also
take into account technological constraints. These rules can be thought of as the
‘system design conventions’ which developers will follow through the design
process. When external consistency is considered, the link between conventions and
consistency becomes even more evident. Usually external consistency involves
following standards or conventions dictated by other similar existing interfaces
(usually belonging to the genre of the interface being developed). The conventions,
when followed, should guarantee an interface which is consistent with the genre.
27
Regardless of whether the consistency of an interface is internal or external (or both),

after a certain time interacting with it, users are likely to develop expectations
regarding what they will find in the interface and how it functions.
User expectations
As outlined above, user expectations are formed through prior experience with an
interface (Vaughan & Dillon, 1998). The link between expectations and genres
becomes evident when one thinks that prior experiences with a particular set of
documents which share similar characteristics, generate in the user expectations of
finding the same similarities on every document which belongs to that particular
genre. This idea is echoed by Waller (1987b) who says that ‘to belong to a genre, a
document is effectively conforming to a set of rules or expectations’. Waller (1999)
also proposes a model to describe the origins of a genre which says that a genre
emerges from early design necessities which were genuinely functional and that
throughout time have lost their initial purpose. These early design necessities then
start to work as a signal to the genre of a document and trigger appropriate
expectations, interpretations and strategies amongst its users.
Although the process through which expectations are formed is quite a complex one
to describe and therefore understand, Vaughan and Dillon (1998) propose a model
which is based on cognitive psychology research on mental representations of
structure. According to the authors
‘Users begin by building up a mental model of an information space based on their

interaction with a set of its digital documents. With repeated exposure and if the set
of documents maintains some sort of regularity, the user will begin to develop a
schematic knowledge of that regularity. This highly organized, abstract knowledge
about digital documents would take the form of Kintsch’s superstructural
knowledge. ’(Vaughan & Dillon, 1998)
If the documents belonging to what the authors call an information space have
regularities, then these regularities can become conventions of that particular
information space, and the information space can be thought of as a document genre.
The superstructural knowledge (Grier, Kortum, & Miller, 2007) which the user
acquired through prior experience with a document genre, becomes the standard
upon which their expectations are built.
28
These expectations can guide user search for an element (Dillon, 2004) and when
violated, are not only immediately noticed by the user (Vaughan & Dillon, 2006) but
also seem to hinder user performance (Shneiderman & Plaisant, 2005).
Conventions
Thus, the concern with conventions expressed in usability literature is justifiable
since conforming to conventions can mean not only achieving consistency, but also
fulfilling users’ expectations. However the concept of conventions, specially genre
conventions, in usability, needs to be broken down further if one is to fully
understand the role these concepts play in usability.
As seen earlier in this chapter, genre conventions can be structural, linguistic, visual
or functional. These four types of conventions are different, and they may affect
usability differently. Some usability authors have given different recommendations
on how to create internally and externally consistent interfaces. Recommendations on
guaranteeing internal consistency do not directly mention conventions as they are
viewed more as rules which need to be followed. In that case, it is still possible to see
which type of conventions they will become by examining the recommendations.
Although structural, linguistic, visual and functional conventions seem to be

considered in usability literature, it seems that authors most often associate following
visual and linguistic conventions with usable design, than following structural
conventions.
Shneiderman and Plaisant (2005, p.74) recommend that ‘identical terminology should
be used in prompts, menus, and help screens; and consistent colour, layout,
capitalization, fonts and so on should be employed throughout [an interface]’. This
recommendation addresses the visual and linguistic aspects of internal consistency
but does not consider structural aspects of consistency.
Nielsen (1993, p. 132), recommends that ‘the same information should be presented
in the same location on all screens, and dialog boxes should be formatted in the same
way to facilitate recognition’ which also addresses the visual aspects of internal
consistency. A study which addressed all three aspects of internal consistency and
established twenty guidelines for designing consistent web interfaces was conducted
by Ozok and Salvendy (2004). When defining the concept of consistency, the authors
29
address visual (e.g. ‘Use consistent sizes, shapes, and colours for screen elements
such as menus, combo boxes, radio buttons, and check boxes’), linguistic (e.g. ‘Use
your words consistently within and across interfaces’) and to some extent structural
conventions (e.g. Use headings for different topics and sub-topics of the text’).
External consistency is addressed in the Usability Body of Knowledge (UPA, 2005)

which says that standards and conventions for a platform and suite of products
should be followed. Despite using the word conventions in their guideline, the UPA
does not specify which type of conventions these would be. However if one is to
consider these conventions as Interface standards such as the Apple Aqua Interface
Standard (available from http://developer.apple.com/documentation
/UserExperience/Conceptual/AppleHIGuidelines/OSXHIGuidelines.pdf) it is
possible to see that these cover structural, linguistic and visual aspects of consistency
as well as functional aspects of the interface in order to achieve consistency.
Despite the concern with conventions, not many investigations have been done on
how not conforming or conforming to structural, linguistic, visual and functional
conventions might affect interface usability.
One of the first studies to examine the effects of not conforming to conventions by
way of inconsistent interfaces is reported in Satzinger (1998) and Satzinger and
Olfman (1998). Their study examined four groups of participants using two different
modules belonging to the same system. The study investigated the effects of
consistency and inconsistency on user performance and satisfaction. Inconsistency
was achieved by manipulating the visual and linguistic aspects of the modules. Four
groups were tested as seen in the distribution shown in Table 1.
TABLE 1 - CONDITIONS UNDER WHICH EACH GROUP OF PARTICIPANTS WAS TESTED BY SATZINGER AND OLFMAN (1998).
Group Tested on modules with
1 Consistent visual appearance and consistent language syntax
2 Consistent visual appearance and inconsistent language syntax
3 Inconsistent visual appearance and consistent language syntax
4 Consistent visual appearance and inconsistent language syntax
Results showed that language syntax inconsistency (i.e. violation of linguistic

conventions) resulted in poorer performance of the participants. However, visual
inconsistency (i.e. violation of visual conventions) resulted in better performance.
30
The authors explain this result which contradicts usability literature by suggesting
that because participants were using both modules simultaneously, a consistent
visual appearance might have caused confusion in the participants. Thus,
participants might have had difficulty in discriminating which of the modules they
were using at one given time. Inconsistency on the other hand might have helped
participants in determining which exact module they were using each time, thus
allowing them to retrieve the appropriate schema to deal with it (Satzinger &
Olfman, 1998). The authors do point out that more investigation ‘on the benefits of
consistent visual appearance is certainly called for’ (Satizinger, 1991).
This thesis investigates the effects of not conforming to visual genre conventions on
user performance. Visual conventions are related with visual consistency since visual
genre conventions are formed by maintaining visual consistency among the
documents of a genre. Therefore, this research will certainly contribute to further
expand the knowledge of the effects of the visual appearance and consistency of an
interface on user performance.
This thesis will also look at the issue from a more web-focused point of view where
different websites are separate entities, as opposed to the software modules used by
Satzinger (1991) and Satzinger and Olfman (1998). This approach seems more
appropriate for examining genres than the approach taken by Satzinger (1991) and
Satzinger and Olfman (1998) who looked at the internal consistency/inconsistency of
a software system.
Another study which investigated violation visual conventions was conducted by

van Schaik and Ling (2001) varied the navigational position between the
conventional left side, top, right and bottom of the screen along with the background
(which could be either white or a specific shade of gray). Participants were presented
with 72 screens and before each screen they were shown a target word. When a
screen was displayed participants had to search to see if the target word was part of
the navigational menu or not. Results showed the top menu position provided a
higher ratio of correct identification, and a quicker response time when trying to
locate the word. Screen contrast showed no significant differences on correct
identifications or on response time. Although in this study the authors addressed
only the visual search process and not an interactive search (where the search is
carried out through interaction with the interface elements, i.e. clicking on links), it
31
shows that the conventional left side position is not necessarily the most visually
efficient place for the navigational menu to be put.
In terms of visual conventions on the Web, a more recent study which tested how
altering the position of the navigation menu of a website could alter the performance
of users searching for information on a website was conducted by McCarthy, Sasse
and Riegelsberger (2006). In the study, the position of the menu varied from the left
(conventional) to the top and right of the screen. The study showed that not
conforming to the conventional menu position (left) did indeed cause poorer
performance. However the authors mentioned that the performance deterioration
was short-lived. In a similar study, Kalbach and Bosenick (as suggested by
McCarthy, et al., 2003) investigated how altering the position of the navigation bar on
a website from the conventional left side to the right side of the screen affected user
performance when navigating. Although Kalbach and Bosenick (2006) did not find a
significant difference in performance between the two conditions, participants in the
right-sided menu group showed indications that they were learning because they
scanned and read the page more often than their counterparts.
Pearson and van Schaik (2003) conducted a study which looked at the effect of link
colour and positioning of the navigational menu in both a visual search task and an
interactive search task. Results for the visual search task agreed with van Schaik and
Ling (2001) since the top menu position along with the bottom position elicited better
response time than the left and right positions. There was no difference in response
time between the top and bottom positions and no difference in response time
between the left and right positions. No effect of position or link colour on correct
identification rate was found. However, in the interactive tasks the authors found
that performance was better on the left and right positioned menus than on the top
and bottom positioned ones.
All these studies have addressed the position of the navigational elements on the
page, which is one aspect of visual conventions. Another study which went further
and investigated visual and structural conventions was conducted by Vaughan and
Dillon (2006). They investigated the impact of not conforming to digital genres on
user comprehension, performance and navigation. In this study the authors used a
web-news conforming page and a non-conforming page. The non-conforming web-
news site altered structural conventions by manipulating the categories of the news
stories and violated visual conventions by altering attributes such as the position of
32
the navigation bar, colour, type size, and number of columns. The performance of the
participants who used the non-conforming site was worse than the performance of
the other group. However, the authors reported that over time (in this case after 5
days using the website) participants using the non-conforming website learned to use
the new website and improved their efficiency.
Because Vaughan and Dillon (2006) have altered simultaneously the structural and
visual conventions of a website it is impossible to separate the effects. Although there
seems to be indications from McCarthy et al. (2006) that altering visual conventions
hinders user performance, McCarthy’s study also shows users can quickly overcome
performance problems caused by violations in the layout. Thus further investigation
is needed to better understand the effect that not conforming to visual conventions
has on user performance.
Even though many usability authors seem to recommend that following conventions
is important for designing consistent sites, and they say that consistency can improve
usability, it is first necessary to establish whether one is talking of internal of external
consistency. But it is even more important to understand how conforming or not
conforming to each conventional component of a genre (structural, linguistic, visual
and functional) can impact the usability of a website.
As seen above, some experimental results seem to indicate that altering expected
position of elements (visual convention) does not seem to cause undue problems to
users as they seem to overcome any performance hindrance quite quickly. However,
the position of an element in a page is only one aspect of a set of visual conventions
which can be formed for an interface. The manner in which an interface element is
presented as well as the way in which its components are organized can also become
a convention. This issue will be addressed in more detail in Chapters 4 and 5; what is
important to note here is that changing the position of one element on a page is not
sufficient to fully understand how not conforming to visual conventions can impact
user performance.
The empirical evidence from Vaughan and Dillon (2006) only indicates that when
visual and structural conventions are violated simultaneously, users face a more
serious usability problem than when the position of only one element is altered.
However, it is hard to establish how much of the performance hindrance is caused by
the violation of the visual conventions and how much is due to the violation of the
33
structural conventions. Further complications can be added by other factors such as

the type of task which users are performing or how familiar participants are with the
conventions of the site used (in this case online news sites). It is even possible that not
conforming to visual or structural conventions have not much of an impact on user
performance but rather the combination of the two violations create a synergistic
effect which can bring performance down to a very low level.
The impact of not conforming to conventions on usability becomes extremely

relevant if the possibility that the impact of each conventional component (structural,
linguistic, visual or functional) on usability has a different magnitude. For example,
altering the conventional structure of a website can cause greater performance
hindrance than altering its visual convention.
This thesis will consider both external and internal consistency as a relevant aspect of
usability, but its main focus will be on investigating the impact of not conforming to
genre conventions (i.e. external consistency), more specifically visual conventions, on
user performance.
As visual conventions are the focus of this thesis it is essential that the relationship
between the visual appearance of a web page and its usability be clear.
2.3.3. Visual appearance and Usability
Inherent and apparent usability

Some usability researchers (Fu & Salvendy, 2002; Kurosu & Kashimura, 1995;
Tractinsky, 1997) consider that the usability of an interface has two different facets:
one, called inherent usability, determines how easy it is to use the system. The other
aspect, called apparent usability, establishes how easy to use users perceive the
interface to be.
Quantifying inherent usability may vary depending on how the researcher views
usability (see 2.3.1); while a product-oriented view will have the inherent usability
measured by how much the product conforms or not with the guidelines, a
performance-oriented view will quantify inherent usability by measuring dimensions
such as the time users take to complete a task and the number of mistakes made
while attempting to complete it. Because apparent usability is dependent on the
34
perceptions of the user, it is usually measured through techniques like questionnaires

and interviews.
Usability and aesthetics

The fields of Human-Computer Interaction and Usability have for many years
considered the visual appearance of web pages of secondary importance and
frequently recommended that usability should be put before aesthetics. But some
studies have contradicted this belief and showed that the aesthetics of an interface is
related to how easy to use users perceive it to be (Kurosu & Kashimura, 1995).
Tractinsky (1997) says that aesthetics can be more influential on how easy or difficult
users believe an interface to be than the inherent usability of the page itself.
Kurosu and Kashimura (1995) examined how apparent usability relates to the
inherent usability of a system. The authors tested 26 Automated Teller Machine
(ATM) interfaces which varied apparent usability (screen layout) and inherent
usability. To establish the levels of inherent usability of the layouts, the researchers
used a product-based approach. A session with interface designers was held where
the designers were asked to establish which factors they considered determinant in a
usable interface. The design factors determined in the session were:
! Glance sequence: top-left corner is looked at first so the main display should
be placed there.
! Familiarity: people are more familiar with telephone style keypad than
calculator style.
! Grouping: keys should be grouped according to their function.
! Operation sequence: special numeric keys should be arranged as: ten
thousands key first and the thousands key next to it. Also, the Yen key
should be placed in the lower corner of the screen.
! Hand dominance: numeric keys should be placed at the right side of the
screen.
The conformance of each interface with each factor suggested by the designers was
then measured for each layout and an inherent usability coefficient was established
for each interface. Then 248 participants were asked to judge the 26 ATM layouts in
terms of their beauty (aesthetics) and ease of use (apparent usability). Results showed
a strong correlation between the perceived beauty of the page and the apparent
usability. Surprisingly to the authors, no correlation was found between apparent
35
usability and the inherent usability of the interfaces. It is important to note, however,
that the authors adopted a product-oriented view on intrinsic usability since they
determined the inherent usability coefficient based on the attributes of the system
rather than on user performance. Another possible way to determine the inherent
usability of each interface would be to have different users perform tasks on each
interface and establish a usability coefficient based on that.
In another study which tested the robustness of Kurosu and Kashimura’s study,
Tractinsky (1997) adapted the experimental material used by the previous authors to
the Hebrew culture and replicated the experiment to some extent (Tractinsky
improved some methodological techniques). Tractinsky obtained the same
correlation between aesthetics and apparent usability as Kurosu and Kashimura did.
Another interesting find from Kurosu and Kashimura (1995) which was confirmed
by Tractinsky (2000) was that the only design strategy which showed a correlation
with the apparent usability of the interface was familiarity with the keypad. This
seems to indicate that users perceive familiar layouts as easier to use than unfamiliar
layouts.
Another study which investigated the contribution of apparent and inherent

usability to user satisfaction when using web pages was conducted by Fu and
Salvendy (2002). To vary the apparent usability of the test material the authors varied
the background of the web pages and the layout of the interfaces.
To quantify inherent usability the researchers used a mixed product and

performance-oriented view. First the authors manipulated the interfaces by making
one of them violate some usability guidelines and recommendations. For this the
authors changed the path necessary to complete the task, how the products were
classified and the type of feedback that the interface gave. Then authors measured
time and error rate for each participant performing the experimental tasks.
To establish a value for the apparent usability of the material, the authors asked a
group of participants to rate how easy to use they thought each design was, based on
the look of the design. Then, to validate the apparent usability the experimenters
showed each participant three pages considered as having high apparent usability
and three considered as having low apparent usability and asked participants to rate
them in terms of (i) liking and (ii) ease of use.
36
Similarly to the previous studies described above, their experiment found no

correlation between inherent and apparent usability. There was no significant
difference in time to complete the tasks and the error rate between the interfaces with
high and low apparent usability. But the study did find that participants who used
websites with high apparent usability were more active and put more items in the
shopping cart than those who used sites with low apparent usability.
Another interesting result comes from studies which investigated post-usage

perceptions of ease-of-use. Tractinsky et al. (2005) conducted a study to investigate if
perceptions of ease of use would be changed after participants had used the
interfaces. The results confirmed once more the correlation between aesthetics and
apparent usability before using the interface. Interestingly, post-usage perceptions of
usability remained related to the aesthetics of the page regardless of the inherent
usability of the page used. Chawda et al. (2005) confirmed this correlation between
aesthetics and usability judgements both pre- and post-use of the interface.
Testing aesthetic perceptions before and after website use van Schaik and Ling (2009)
found no positive correlation between usability (i.e. task performance and mental
workload) and aesthetic perceptions. However, the authors found a negative
correlation between aesthetic perceptions and the number of tasks participants
completed. In other words, the higher the aesthetic perceptions of a page, the fewer
number of tasks participants completed using that page.
Finally, a recent study (Michailidou, Harper, & Bechhofer, 2008) which investigated
the relation of visual complexity, familiarity and website aesthetics showed a
negative correlation between complexity and aesthetic perception (the more complex
a page is considered, the less attractive it looks). However, the study also showed
that familiarity with a web page is the most influential factor on the aesthetic
perception of the same page i.e. participants considered pages they were familiar
with as more aesthetically pleasant than pages they were not familiar with
(Michailidou, et al., 2008).
If users are familiar with a web page genre they regard pages belonging to that genre
to be more aesthetically pleasant than other pages with which they are not familiar
(Michailidou, et al., 2008). If web pages which are more aesthetically pleasant are
considered by users to be easier to use (Kurosu & Kashimura, 1995; Tractinsky, 1997),
37
then it is possible that familiar interfaces (i.e. convention-conforming interfaces)

might be perceived as easier to use than unfamiliar interfaces.
These studies show that the visual appearance of an interface is extremely important
to usability, since it not only gives users an idea of how easy or difficult using the site
might be, but it can even make users feel that an interface is easy to use regardless of
its inherent usability. Another interesting aspect which needs further investigation is
the relation between familiarity and apparent usability.
2.4. Conclusion
Although usability literature frequently mentions the importance of keeping
consistency, following conventions, and meeting user expectations, no connection
seems to have been made with the concept of genre. Even though research has
pointed out the emergence of digital genres (Eriksen & Ihlström, 2000), and the
transference and subsequent evolution of existing paper genres to the World Wide
Web environment (ISO/IEC, 1998), little is known of the effects of genre violations on
web usability. In this subsection an attempt is made to examine how genres are
linked to usability.
As seen before, according to the ISO definition of usability, for an interface to be

considered as having good usability, it needs to be efficient, effective and make its
users satisfied. In other words, in a usable system, users should accurately and
completely achieve the goals specified without using many resources. They should
also feel comfortable and show positive attitudes towards the system (Lidwell, et al.,
2003; Shneiderman & Plaisant, 2005; UPA, 2005).
As seen earlier in this chapter, the concept of consistency can be associated with
usability as when the conventions of a document genre are followed, the resulting
document is consistent with the other existing documents of the genre. This can be
thought of as external consistency. However, when it comes to internal consistency,
following genre conventions does not necessarily imply that the resulting document
will be internally consistent as documents which belong to a genre are not necessarily
internally consistent. According to the literature, both internal and external
consistency are constantly associated with good usability (Satzinger & Olfman, 1998)
and it is therefore plausible to assume that breaking apart from the conventions of a
genre will hinder the usability of a document.
38
External consistency is said to improve learnability since users who are familiar with
a certain interface can transfer their knowledge when using another interface which
is externally consistent with the one they already know. This, in turn, can affect the
effectiveness of the system, as users will be less likely to make mistakes if all
elements of the system behave as expected. It can also affect the efficiency if the
elements and the needed information are found in the expected place, and look as
expected. However, results from studies which have investigated the relations above
have either been not very clear or sometimes contradictory.
Conforming to visual conventions has been detrimental to user performance

(measured as accuracy and time to complete tasks) when compared to not
conforming to visual conventions of the modules of the system (Satzinger & Olfman,
1998). However the same study showed that following linguistic conventions in the
same system has in fact improved user performance when compared to a linguistic
convention violating interface (Satzinger & Olfman, 1998).
In two other studies, not conforming to visual conventions by altering the placement
of the navigational menu of a website either caused: a) no difference in performance
(Kalbach & Bosenick, 2006) or b) a poor performance when using the non-
conforming website (McCarthy, et al., 2003). Finally, not conforming to both
structural and visual conventions of a website seem to have been detrimental to user
performance (Vaughan & Dillon, 2006). However, the extent to which each type of
non-conformance contributed to the decline of performance could not be determined.
Thus the type of effects and relations between conforming and not conforming to
each type of genre conventions and user performance still needs to be determined if
designers are to be able to better decide whether or not to follow genre conventions.
The next chapter describes two exploratory studies which tested structural and visual
aspects of websites. These studies were the first empirical studies conducted in this
thesis and they were aimed at informing future directions of the research.
39
Chapter 3
Exploratory studies
3.1. Introduction
This chapter describes two exploratory studies which were conducted in 2004 to
investigate structural and visual aspects of websites. The idea for these studies was to
gain better comprehension about the two main aspects of genre conventions which
are directly associated with design: structural and visual conventions. The outcome
of these two studies will also provide directions for the research.
The first study examines whether reading from a website where the content is
organized by the use of conventional structures rather than an unstructured website
helps users comprehend the text, and recall procedural information in the correct
sequence. The latter can be an indication of the formation of a schematic knowledge
of the website structure by users. As seen in Chapter 2 the development of schematic
knowledge is one step in the formation of user expectations, which are directly
associated with genre conventions.
The second study investigates if there are certain ways of visually organizing
elements on a web page which can help users quickly identify the category of a
website. This can indicate that the expectations that users have about a certain
category of website are not restricted to the structural aspect of the site but also
include its visual appearance.
40
Exploratory studies
3.2. Exploratory study 1 - The effect of structured design

on comprehension and structured recall of electronic
documents
3.2.1. Background
When looking at a document genre, readers have expectations regarding not only its
discourse and writing style, but also about the organization of the information within
that document. These expectations are based on a mental schema built on previous
experiences with that particular genre. This schema, called superstructure by van
Dijk and Kintsch (1983), is responsible for giving the reader an idea of the
organization of the global text content. As seen previously in section 2.3.2 of Chapter
2, a proposal about the formation of superstructures was made by Vaughan and
Dillon (1998) where repeated exposure to documents with similar characteristics
allows users to develop a structural knowledge of the document.
When this structural knowledge is combined with spatial memory for layout it can
facilitate searching and browsing the material (Dillon, 1991). This can be explained by
the theory pointed by van Dijk and Kintsch (1983), that the relevant superstructure of
a document genre is activated in the memory of the reader as soon as a cue is found
in the text. However, the authors give no indication as to the nature of the cue (e.g.
semantic, syntactic or graphic). Many cues used to organize texts and indicate ideas
are (usually) semantic or syntactic, and therefore outside the scope of this research.
There are, however, graphic cues which are also used to organize textual content.
These cues usually consist of typographic elements and visual attributes which can
be used to link and relate ideas within the text and to indicate thought units.
Headings, subheadings and colour can be used to establish relationships between

textual information, and spacing can indicate thought units (Dickson, Simmons, &
Kameenui, 1995). These typographic elements, called access structures by
Waller(1979), are instruments that help the reader gain access to the text using both
spatial and typographic cues to group or order pieces of text (Hartley, 1987).
Many studies have investigated the effects of access structures on reading and
comprehension of texts both in print and on screen. Although this research will be
primarily focusing on reading from screen, some results of print research will also be
examined since they can provide relevant information.
41
Exploratory studies
3.2.2. Access structures & typographic cues
Colour
Colour can be used to express organization and structure within a text (Hartley,
1978). In electronic text, colour can be used to indicate categories of importance and
can be used to convey organization and structure (Hartley, 1994, p. 158). But because
there does not seem to be a range of colours which is intuitively associated with a
hierarchy of importance, using colour to convey structure can be difficult (Hartley,
1994) despite that, some authors recommend using colour to support the spatial
organization of text is recommended (Hartley, 1987, p. 68).
In this exploratory study, colour will used to support the organization of the textual
content. As recommended by Hartley (1987, p.70) colour will be used in headings to
reinforce the organization of the textual content.
Spacing
Another very important typographical feature that helps organize information on
screen is spacing, since it can reveal the text structure at a glance (Hartley, 1987, p.67).
However, inappropriate spacing can lead to confusion and hinder comprehension
(Hartley, 1987, p.67). Text spacing can be divided into four main types: inter-linear
spacing, word spacing, character spacing and margins.
The effects of some of these types of text spacing on reading performance were
examined by Chaparro, Baker, Shaikh, Hull, and Brady (2004). The authors compared
four layouts with different values for margins and inter-linear spacing. The values
ranged from ‘Margin’ (with 10mm of space surrounding the text) to ‘No margin’
(with 2mm of space surrounding the text), and ‘Optimal inter-linear spacing’ (with
5mm between lines of text) to ‘Sub-optimal’ (with 4mm between lines of text). The
typeface and font size remained constant throughout the experiment.
The results showed that text margins affect reading speed and comprehension since
participants read slower with the ‘Margin’ condition, but comprehended more than
with the ‘No-margin’ condition. Considering that if type size is kept constant within
an experiment, then the margin size bears an inverse relation to the number of
characters per line and consequently line length. Thus, the differences in line length
between the ‘Margin’ condition and the ‘No-margin’ condition in Chaparro et
al. (2004) might have accounted for the observed differences in comprehension. This
42
Exploratory studies
result seems to agree to a certain extent with the results from Dyson and Haselgrove
(2001) that readers' comprehension declined as line lengths increased.
Furthermore, Chaparro et al. (2004) found that inter-linear spacing did not seem to
influence reading performance. This could have been caused by the small differences
in inter-linear spacing between the ‘Optimal’ and ‘Sub-optimal’ conditions. Although
not particularly studying space as a conveyor of text structure, the study conducted
by Chaparro et al. (2004) indicates that space plays an important role in improving
reading comprehension.
The results from Chaparro et al. (2004) also illustrate a trade-off between reading
speed and comprehension that can occur when testing for these variables (Dyson &
Haselgrove, 2000, 2001; Muter & Kruk, 1984; Muter, Kruk, Buttigieg, & Kang, 1988).
Considering that the possibility of trade-off exists, to get a broader picture of the
performance, both comprehension and reading speed will be measured in the
following experiment.
One other study that tested how the amount of space affected the process of finding
information on screen was conducted by Bernard and Chaparro (2000). In this
experiment three mock websites with the same content but three different layouts
were examined. Each layout varied the amount of space between chunks of text: The
first layout (called low spacing - Figure 7) had only 3mm of space between columns of
text and no space between paragraphs. The second layout (called medium spacing-
Figure 8) had 9 mm between each column of text and between paragraphs, and the
third layout (called high spacing-Figure 9) had 19 mm between each column and
between paragraphs.
43
Exploratory studies
FIGURE 7 - REPRESENTATION OF THE LOW SPACE CONDITION (BERNARD ET AL., 2000).
FIGURE 8 - REPRESENTATION OF THE MEDIUM SPACE CONDITION (BERNARD ET AL., 2000).
44
Exploratory studies
FIGURE 9 - REPRESENTATION OF THE HIGH SPACE CONDITION (BERNARD ET AL., 2000).
Sixteen participants were asked to perform search tasks that were randomly assigned
to them in each layout condition. Each participant answered 5 task questions in each
condition so that they were in contact with all three layouts. After each task the
participants were asked to rank the difficulty of finding information and after
performing all the tasks they were asked to rank their preference to each layout.
No significant differences in search time and search errors were found between the
three spatial layouts. However, the fact that the content was the same in all three
layouts and each participant interacted with all three layouts could have caused a
carry-over effect to happen. In other words, the knowledge acquired by using one
layout was transferred to another.
In the preference assessment, participants responded that they did not like websites
with low levels of space and preferred sites with medium levels of space. As well as
this, despite the fact that this experiment did not separate the effects of paragraph
spacing from column spacing, the results suggest that using some space after
paragraphs seems to increase user satisfaction. This relates to the idea that the
underlying structure of a text can be made more evident to the reader by using space
consistently to separate elements of a text such as headings, paragraphs and
sentences (Hartley, 2004).
45
Exploratory studies
Considering that paragraphs are the smallest text elements in which one can express
a complete thought (Peels, Janssen, & Nawijn, 1985), and that space can be used to
indicate thought units (Dickson, Simmons, & Kameenui, 1995), it is not surprising
that separating text paragraphs with space can improve reader satisfaction
(Chaparro, et al., 2004).
Headings
The main function of headings is to group and sequence large segments of text
therefore communicating its structure. Research has shown that headings facilitate
recall of unfamiliar topics (Lorch & Lorch, 1996), and help users find and extract
information from a text (Hartley, 1987, 2004). Many experiments have been carried
out both in print and on screen to test variables relating to headings (Baker, Bernard,
& Riley, 2002; Hartley & Trueman, 1985; Lorch & Lorch, 1996; Schultz & Spyridakis,
2004; Williams & Spyridakis, 1992) ranging from their position in the text, to their
form (statements or questions) and to the task performed (Hartley & Trueman, 1985).
Regarding the use of headings on computer displays Hartley and Jonassen (1985)
mention that articles designed for screen display should be menu-driven and suggest
that a content list of headings should appear at the beginning of the document.
The influence of headings on printed text recall and summarization was evaluated by
Lorch and Lorch (1996) throughout two experiments. The first experiment verified
the effects of headings on a free recall task. After reading the texts, one condition
with headings and the other without, the participants were asked to do a written
recall test. The second experiment checked if the presence or absence of headings
would influence the inclusion of topics in a summary. After reading the two texts,
one layout condition with headings and the other without, the participants were
asked to write a 15-sentence-long draft of the summary whilst referring to the text as
much as they liked. This summary limitation was used to control variability in the
data and to make participants selective in what they included in the summary. The
results obtained showed that the presence of headings on printed text helped recall of
unfamiliar topics but not of familiar ones. The authors also found that headings and
familiarity with a topic had a great influence on whether or not that topic would be
included in a reader’s summary of the text.
46
Exploratory studies
On a study to verify which physical attributes of headings were better perceived by

readers, Williams and Spyridakis (1992) varied type size, position, underlining and
case (upper or upper-and-lower) on printed material. The results showed that
relative differences in type size provided the most discernible visual cues to
hierarchy for the readers.
Another study, by Baker, Bernard and Riley (2002) evaluated the effects of headings
and summaries on user satisfaction and task completion time when reading from
screen. This was done by creating three different online news presentation formats
with twelve articles taken from the New York Times website. The three layouts were
described by the authors as:
! Full – containing full texts of all twelve articles.
! Summary – displaying link titles of all twelve articles (headlines) plus small
abstracts.
! Links – displaying links to titles of all twelve articles (headlines) only.
All participants were asked to perform ten different search tasks within the news
articles using the three different layouts. After the tasks were completed, participants
answered a 6 point Likert scale satisfaction questionnaire comprised of the following
questions: "The layout made it easy to find information", "This site was visually pleasing",
"The arrangement of this site promotes comprehension", "I am satisfied with this site", and,
"The layout looks professional”
The authors found no significant difference in the time taken to complete the tasks in
the three layouts. However, the summary layout condition was considered the most
easy to find information in and was also the one participants preferred. The lack of
significant differences in task completion time could be explained if the headings
(headlines) contained the key words necessary to finding the information. Because all
three conditions had the headings this could have levelled all three layouts making
performance equivalent in all three versions.
The effects of headings, indentation and image placement on reading performance,

comprehension and participants’ satisfaction were measured by Chaparro, Shaikh
and Baker (2005). The authors compared the effects of an enhanced screen layout
with a poor screen layout. The enhanced layout was designed to include optimal use
of the headings, indentation and image positioning. The participants were asked to
read three passages in each of the two layouts with the order in which each layout
47
Exploratory studies
was presented chosen randomly for each participant. Each task was timed, and after
reading each passage the participants were asked to answer eight comprehension
questions on the passage. After reading the three passages from one layout each
participant completed a questionnaire on their reading satisfaction and the same
procedure was repeated for the other condition. Finally, after completing the tasks on
both layouts the participant was asked to decide which layout they preferred.
The difference between the number of words read per minute and comprehension
score on the enhanced layout compared to the poor layout was not statistically
significant and the authors concluded reading performance or comprehension was
not influenced by the quality of the page layout.
Unfortunately, the authors do not provide details on the criteria used to determine
the characteristics of a good layout, and on examining the examples of the page
layout provided in the article there is not much to differentiate both conditions
except for the heading positioning and text wrap around pictures (Figure 10). One
possibility for this lack of differences in reading performance and comprehension
across layouts may have been due to there being little differentiation between the
poor and enhanced layouts used by Chaparro et al. (2005).
FIGURE 10 - ENHANCED PAGE LAYOUT (LEFT) AND POOR PAGE LAYOUT (RIGHT).
Another possible explanation for no differences in the study may be due to how the
authors structured the text. Because in both layouts most of the structuring of the text
48
Exploratory studies
was in the same manner (i.e. contained the same access structures in the same parts
of the text) and considering that some access structures (such as headings) help users
find, extract (Hartley, 1987, 2004) and recall information (Lorch & Lorch, 1996) from
a text it is possible that this have accounted for a lack of difference in what
participants could remember.
Eveland, Cortese, Park, and Dunwoody (2004) suggest that studies which evaluate
the effects of information organization on the content of memory and not on the
associations of different bits of information in memory are limited. According to the
authors, it is necessary to extend measures of learning when looking for the effects of
changes in layout beyond recall, to include knowledge structure or organization. As
will be explained in the next subsection, knowledge of the way in which ideas, stored
in the memory are interconnected is essential to the comprehension of these ideas
and ‘problem solving depends on adequate structural knowledge of the ideas in the
domain being explored’ (Jonassen, Beissner, & Yacci, 1993). In other words, the
information stored in the memory must be adequately structured and connected for
it to be useful in problem solving.
The recall of stored information and some basic assessment of knowledge structure
will be carried out in this experiment. More specifically, the ability of readers to recall
information in the same order in which it is presented in the text will be investigated.
This is especially important when procedural information is presented, such as
assembly instructions, equipment operation instructions, software and hardware
installation procedures and even cooking recipes. This type of recall will be referred
to here as structured recall. Structured recall can also be an indicator that the structure
of the information has been learned adequately and can be recalled correctly when
needed.
3.2.3. Types of knowledge

Knowledge is usually divided into two distinct but related categories: declarative
and procedural knowledge (Jonassen, et al., 1993). Declarative knowledge, also
known as factual knowledge (Eveland, et al., 2004), represents the consciousness of a
concept, idea or object. However, the fact that people hold some declarative
knowledge does not necessarily mean that they are able to use it since declarative
knowledge does not by itself imply knowledge.
49
Exploratory studies
Procedural knowledge describes the schemata through which the declarative

knowledge is used to solve problems, form plans, and process arguments amongst
other procedures (Jonassen, et al., 1993). During the learning process, information is
stored in packets in the memory of the reader and relationships between the new
information and existing information are established. So when faced with a problem,
a person must retrieve from the declarative knowledge the relevant content
pertaining to a particular schema in order to solve that specific problem. Since these
two components of knowledge seem insufficient in explaining the whole process of
storing and retrieving information from memory, Jonassen et al. (1993) propose an
intermediate type of knowledge known as the structural knowledge, the function of
which is to mediate the relations between procedural and declarative knowledge.
Structural knowledge defines how the concepts of a particular domain are
interrelated, and it manages and defines how declarative knowledge contents are
connected in the memory.
It is likely that the superstructural schema which users of a genre possess and use
confronted with a document belonging to a genre (Dillon, 2000) is related to
structural knowledge. And because structural knowledge defines how the concepts
in memory are interrelated and connected, its existence can be evidence that not only
factual but also procedural knowledge exist.
Assessing structural knowledge

Most comprehension assessments used in reading experiments simply ask
participants to retrieve information from memory, and consider that the more correct
information a person retrieves, the more that person is thought to have learned. In
other words, these types of assessment only test declarative knowledge, and when
structural changes to the text are part of the experiment, they fail to check the impact
of these organizational changes in structural and procedural knowledge.
Multiple-choice questions that merely present exact sentences extracted from the text
and ask participants to compare it with their stored declarative knowledge are
simply measuring one aspect of knowledge, and one that does not necessarily mean
understanding. Therefore, experiments that evaluate the effects of text structure (e.g.
use of access structures versus no access structures) on comprehension should
present respondents with questions that encourage the use of both declarative and
50
Exploratory studies
procedural knowledge on their tests. This way a better evaluation of acquired

knowledge can be achieved.
3.2.4. Objectives
This experiment is designed to investigate if organizing the contents of a web page
through the use of conventional access structures can improve reading speed,
comprehension and structured recall. This could give indications as to whether
structuring web content in a conventional way can help with the generation of
mental schemas as opposed to leaving the content unstructured.
3.2.5. Method
Experimental texts
The textual content chosen for this study was extracted from a book on wine, and it
can be divided into two basic parts. The first one is procedural, and indicates the
process of winemaking from the picking up of the grapes until the fermentation of
the juice into wine. The second part, more hierarchical in nature, describes the
categories of wine, from the lowest quality up to the highest quality wines. A sample
of the text as it was structured in the book is presented in Appendix A.
Two text layouts were produced with this content for this experiment. One is an
unstructured text layout (from now on referred to as the unstructured layout – Figure
11); the other (from now on referred to as the structured layout -Figure 12) is laid out
to present information in a structured manner allowing the participant to read either
in a linear or non-linear form through the use of hyperlinks.
The structure is indicated by using headings and subheadings, different typefaces

with different type sizes. Colour is used consistently to indicate the organization of
the text, as headings and subheadings which relate to the process of winemaking are
represented in one colour (purple) and the ones which relate to the quality of the
wine in another colour (green).
Because of the left sided navigational menu, in the structured layout the left margin
in this layout is larger than in the unstructured version. Consequently line length in
the latter is longer than in the former. Line spacing, character spacing and paragraph
spacing are the same in both versions. But in the structured layout only one section
51
Exploratory studies
(or subsection) is visible on the screen at one time, whereas in the unstructured
layout the whole text is visible.
FIGURE 11- SAMPLE OF THE UNSTRUCTURED TEXT LAYOUT USED INT THE EXPERIMENT.
FIGURE 12 – SAMPLE OF THE STRUCTURED TEXT LAYOUT USED IN THE EXPERIMENT.
52
Exploratory studies
Through the use of the menu in the structured layout, the reader was allowed to
move freely through the text topics in any order.
Comprehension and Recall Assessments

Two methods for assessing comprehension were chosen for this experiment: a
summarization task and a multiple-choice test about the text.
Summary Task
The summarization task was open-ended and asked each participant to summarize
what was read during the experiment. A time limit of 5 minutes to complete the
summary was established to encourage the participants to write a short piece, be
selective (Lorch & Lorch, 1996) and to keep participants from getting too tired.
To evaluate the written summaries of the participants, three main criteria were
defined. If all three criteria were met, the participant was considered as having
achieved maximum comprehension.
! General comprehension
If the reader shows a general idea of what the text is about, it is assumed that
they acquired a general comprehension of the text. To be considered as
having grasped a broad idea of the text, the participant has to write at least
that the text is about winemaking and wine classification.
! Structured recall
If the reader can recall the sequence of events as they appear in the text, this
counts as evidence of structured recall. To be considered as having achieved
that, the participant has to write about all the following topics in this
particular order: grape crushing, fermentation, ordinary wine, vin-de-pays, good
wine, fine wine and great wine.
! Further comprehension
If the reader can recall any detail from the text, this is considered as having
obtained further comprehension of the text. Any specific detail that does not
give a general idea of the text and does not express a sequence of events as
they appeared in the text indicates that the reader has acquired extra
information from the text.
Any of the three criteria can appear independently in the summary, although it
seems odd that a participant might achieve ‘Further comprehension’ of the text without
achieving ‘General comprehension’.
53
Exploratory studies
Multiple-choice
The multiple-choice questions are designed to assess the text comprehension and
structure recall. As can be seen in Appendix B, the assessment consists of 6 questions,
and although some measured more than one particular item, they can broadly be
categorized as follows: two measured recall of specific information from the text
(questions 1 and 5), two measured the structural recall by asking the participants to
recall the order of appearance of sentences, winemaking processes or topics from the
text (questions 2 and 3) and two measured if the reader could arrive at conclusions
based on what they read in the text (questions 4 and 6). The order of questions was
randomized for each participant.
3.2.6. Study design

This exploratory study was designed to have two independent groups of
participants. Each group was tested on one version of text layout (structured or
unstructured).
3.2.7. Participants
Ten participants volunteered for this study. They were randomly allocated to one of
the two experimental layouts so that each participant read the text from one
condition only. Five participants were tested on the structured layout and five on the
unstructured layout. Four participants were women and six were men.
All participants were fluent in English: two of the participants were native English
speakers and eight had English as a second language. All the participants were
graduates and 90% used the Internet on a daily basis. It is relevant to note that after
randomization each native English speaker was assigned to one of the conditions.
Table 2 illustrates the gender distribution among conditions.
TABLE 2 - PARTICIPANTS' GENDER ACROSS CONDITIONS.
Gender
Male Female
Unstructured Condition 3 2
Structured Condition 3 2
54
Exploratory studies
Participants’ Internet Usage Profiles

An Internet usage questionnaire was applied to users before the test to assess their
level of familiarity with navigating the Net and using hypertexts.
Nine of the participants said they used the Internet on a daily basis and one used it
every other day. Only two participants used the Internet for less than one hour every
day and the majority (six) used it from 1 up to 5 hours. Two participants said that
they used the Internet on a daily basis for more than 5 hours. All the participants
used search engines and sent and received e-mails. More than half of them used bank
facilities online. The two least common activities were shopping/comparing price
information and playing games online.
3.2.8. Experimental Procedure

Participants were asked to read a printed text which described the experiment and
told to adjust the equipment (i.e. mouse, keyboard and computer screen) and the
height and position of the chair to their liking.
Participants were told to read the text at their own pace and to do it as carefully and
as accurately as possible. The importance of reading the whole text was reinforced
and participants were informed that they would be answering questions on it later. It
was also emphasized that the participants should signal to the experimenter when
they started and ended the reading task for timekeeping purposes.
Each participant was observed and notes were taken on their interactions with the
interface, such as the use of the mouse, scrolling patterns, text positioning and of any
comments they made. The participants in the structured layout group were also
observed for the reading sequence they chose.
When the reading task was completed, each participant was asked to write a short
summary about what they had just read and were given a maximum of 5 minutes to
do this. They then had to answer 6 multiple-choice questions about the text, and
answer a questionnaire about their Internet usage habits (see Appendix B).
55
Exploratory studies
3.2.9. Results
Reading time
The mean reading time for the unstructured layout was 613s and for the structured
layout was 669s. A Mann-Whitney non-parametric test adjusted for ties found that
the time to read the texts on each layout were not significantly different (W=35, n=5,
p=0.676).
FIGURE 13 - MEAN READING TIME WITH STANDARD ERROR BARS FOR THE TWO LAYOUTS.
Multiple-choice Questions
The table below displays the outcome of the multiple-choice questions for each
condition.
56
Exploratory studies
TABLE 3 - TABLE WITH THE CORRECT ANSWERS FOR THE MULTIPLE-CHOICE TESTS BY LAYOUT
(U.L. = UNSTRUCTURED LAYOUT AND S.L. = STRUCTURED LAYOUT).
correct
answers
Question U.L. S.L.
Tick the option containing countries where you still find the traditional method of
1 5 5
treading grapes by bare feet being used.
2 Tick the option which shows the right order of processes necessary to make wine. 2 5
Number the sentences according to the order in which they happen during the winemaking
3 0 2
process.
4 Why is it so important not to crush the grapes’ pips during the process of pressing them? 2 1
5 Tick the option in which all the names refer to ordinary wine. 2 4
6 Assign (T) for true or (F) for false to each of the sentences below
6a Good wine is usually an “ordinary” wine with a birth certificate 2 4

6b Usually the yeast cells can work up to a 15% concentration of alcohol 1 3
6c The world’s largest wine presses are located on small farms in Chianti. 4 3
6d Not many regions of the world have been able to produce great wine regularly 3 5
6e Fine wine is the rare and exquisite result of perfect conditions in a perfect vineyard. 3 1
Total per group 13 16
Two of the multiple-choice questions (numbers 3 and 6) allowed for more than one
answer: one asked the respondent to order the sentences and the other to establish if
a sentence was true or false. In the true or false question (number 6), each of the 5
sentences were marked individually, and in the sentence ordering question (number 3)
the criteria used to mark these questions was that participants had to get the
complete sentence order correct in order to be considered as having answered the
question correctly.
Participants who read the text through the unstructured layout achieved a mean of
4.8 out of 10 possible correct answers, whilst participants who read the text through
the structured layout had a mean of 6.2 out of 10 possible correct answers. A Mann-
Whitney test adjusted for ties revealed that this result was not significant (W=23.0,
n=5, p=0.459) and therefore there was no evidence that layout affected
comprehension. Although the difference in score is relatively high, the lack of
significance of the result might be due to the small number of participants in this
experiment, the small number of questions and high variability of the results.
57
Exploratory studies
FIGURE 14 - MEAN MULTIPLE-CHOICE SCORES FOR BOTH LAYOUTS.
Structured Recall
Two of the multiple-choice questions (2 and 3) required not only recall of the text
content but also required structured recall from the reader. Participants who read the
structured layout text got more of those questions right (Mean=1.4) than participants
who read the text from the unstructured layout (Mean=0.4). A Mann-Whitney test
adjusted for ties showed that this difference was statistically significant (W=18, n=5,
p=0.041) and therefore there is indication that the differences in structured recall
scores were due to the differences in the layouts. However, due to the small sample
size and reduced number of structural questions further investigation is required.
FIGURE 15 – MEAN MULTIPLE-CHOICE STRUCTURE QUESTION SCORES.
58
Exploratory studies
Summary Task
In the summary task, participants who used the structured layout wrote an average
of 71 words against an average of 61 for participants of the unstructured layout
group.
General comprehension
All participants in both groups showed a general comprehension of the text since all
participants wrote in the summary that the text was about winemaking and wine
classification. As a result the general comprehension score was 5 for both groups.
Structured recall
In the summary, four participants who used the structured layout described the
sequence of topics as they appeared in the text correctly.
Only one participant from the unstructured layout wrote correctly the sequence of
topics as they appeared in the text and one participant, although not mentioning
specific topic names, demonstrated having conscience of the relative position of each
topic in the text.
Further comprehension
Only two participants in the unstructured layout group included details from the text
(e.g. names of vinification processes, countries, wine classification names,
measurements or quantities) in the summary. Three participants who used the
structured layout included details from the text.
Overall comprehension (summary and multiple-choice)

All ten participants demonstrated having acquired a general idea of the text subject.
This can be concluded by the lack of significant differences in scores both in the
multiple-choice questions and the summary task.
59
Exploratory studies
TABLE 4 - COMPARISON OF OVERALL COMPREHENSION RESULTS ACROSS CONDITIONS.
Multiple-choice Summary Task

(mean scores) (out of 5)
U.L. 4.8 5
S.L. 6.2 5
Observations
Unstructured layout
One participant placed the mouse pointer at the beginning of each line that was being
read. Generally, most of the participants scrolled the text one paragraph at a time and
usually placed the paragraph they were reading at the top of the screen.
Structured layout
One participant read the text not following the order established by the navigation
menu. Although indicating the structure in the summary, the participant only got
one multiple-choice structural question correct. Two participants did not read the
first link (How Wine is Made) and one of them also missed the fourth link (What is
Great Wine).
Three participants, while reading the text, left the mouse pointer on the menu
indicating the last clicked link, and two used the mouse pointer to help read the lines
(using it as people use their fingers while reading).
3.2.10. Discussion
Comprehension
The results suggest that the difference in layout does not seem to influence reading
time. Having a structured layout does not seem to influence comprehension and
consequently declarative knowledge.
Looking at the scores from the multiple-choice questions and the summary task, it
can be observed that participants who read from the structured layout achieved
higher scores than participants from the unstructured layout group (Table 4).
However, this result was not statistically significant for the multiple-choice test and
there was no score difference in the summaries written by the participants. Although
it cannot be assumed that the use of headings and colour-coded menus to structure a
60
Exploratory studies
text had any effect on the text comprehension, the lack of significance in the results
might be due to the reduced number of participants in this exploratory study. It is
possible that a study conducted with a larger sample might reveal a significant
difference in this aspect.
Another factor which may have contributed to the lack of significant differences is
the fact that paragraph spacing was used on the unstructured layout may have given
readers some idea of the text structuring, thus influencing how they understood it
(Hartley, 2004, p.922).
Structured recall
The difference in the multiple-choice structured recall scores between the two groups
was significant. The participants who read from the structured layout answered
more structural questions correctly than the participants from the unstructured
layout group. This result was echoed in the summarization task where four
participants who read from the structured layout mentioned correctly the sequence
of topics as they appeared in the text, against only one participant who read from the
unstructured layout.
This is strong indication that the use of colour-coded headings to structure and
organize a text helped participants to recall information in the sequence it appeared
in the text.
Remarks on the methods used
Observations
Observing participants throughout the experimental tasks has revealed some

interesting patterns such as using the mouse pointer to indicate the word being read
on a line. This suggests that the readers may have had some trouble following lines
with their eyes, and probably returning to the beginning of the following line.
Another valuable observation was that two participants missed the first link to text
(represented by the heading ‘How Wine is Made’ on the menu) and one of them
missed another link to the text ‘What is Great Wine’. This might have been due to the
fact that this heading was on the top left of the page where the logo of a website is
usually located. Therefore, participants may have interpreted it as the ‘title’ or ‘name’
61
Exploratory studies
of the page. Yet another explanation is that participants perceived those links as a
section header devoid of any content used only to group the links.
Summary Task
The summarization task was an important tool not only for encouraging participants
to read the text attentively, but also for indicating differences between conditions.
However limiting the time each participant had available for this task prevented
some participants from writing more in their summaries. Perhaps combining a time
limit with the number of sentences as suggested by Lorch and Lorch(1996) might
encourage participants to write more on the summary task.
Multiple-choice questions on comprehension

Although results of the multiple-choice questions on comprehension showed no
significant result, they did encourage participants to read the text carefully. The lack
of significance was probably due to the small number of questions (only 6). In future
research, a greater number of questions will be used.
Structured recall assessment
Despite the fact that the method used to assess structure recall provided valuable
results, different and more precise techniques of tackling this particular aspect of
knowledge might be developed for future research.
Other issues
It would also have been interesting to have some reader preference test to evaluate
both conditions and to get a view of participants’ impression of each layout.
Knowledge assessment
The comprehension assessments from this experiment dealt solely with two types of
knowledge: declarative and structural. Regarding structural knowledge, only a very
simple and superficial method of structural recall test was applied. Future research
might improve on these methods of assessment of structural knowledge and
declarative knowledge.
62
Exploratory studies
3.2.11. Conclusion
Results from this exploratory study seem to indicate that using conventional access
structures such as headings, and subheadings along with colours appears to help
readers of digital documents organize and better retrieve sequential information.
This indicates that these conventional structures may help readers to store the read
information in an organized manner or, at least it may help them retrieve this
information in an orderly fashion.
Furthermore, when it comes to web genres there is another aspect of content

structuring which is worth investigating. Users who belong to the discourse
community of a particular web genre expect the content of a website from that genre
to be organized in a certain way. This involves not only using the access structures
conventional to that genre, but is also related to how the content is distributed
throughout the website.
3.3. Exploratory study 2 - Visual appearance and the

identification of the category of a website.
3.3.1. Introduction
From the previous experiment it was gathered that presenting text in a structured
layout text can improve the structure recall of the textual information by the readers.
The structuring of the text is usually achieved by the use of visual elements such as
headings, colours, paragraph spacing, typeface, type size and icons.
To understand a text, readers use three different types of cues they find in texts:
graphic, semantic and syntactic (Dillon, 2000; Loines, 1991; Stamboltzis & Pumfrey,
2000). Graphic cues (e.g. levels of headings, colours, typeface and type position)
provide the reader access to the information (Loines, 1991), pointing out what is
relevant or not. Syntactic cues define how the information in the text is related, and
semantic cues help readers recover the meaning of the text (Loines, 1991).
As mentioned in Chapter 2, whenever approaching a document, readers have some

expectations of how it will be structured and what the organization of its key
elements will be (Dillon, 2004). These user expectations are generally based on
previous experiences with certain ways of organizing and laying out a particular
type of document which over time has become standardized. These standards have
63
Exploratory studies
given rise to highly conventional document forms that are instantly recognized as
being part of a genre (Dillon & Gushrowski, 2000).
The formation of these genres according to Dillon (2002) is usually thought of as a

very slow process, taking sometimes decades for a genre to emerge. However, some
authors argue that, that at least for web genres, this process seems to occur faster
(Crowston & Williams, 2000; Dillon, 2000; Dillon & Gushrowski, 2000; Shepherd &
Watters, 1999). This idea is backed up by some research that has reported the
emergence of genres exclusive to the web such as the personal homepage (Dillon &
Gushrowski, 2000) and weblogs (Herring, Scheidt, Bonus, & Wright, 2004).
Dillon and Gushrowski (2000) point out that when it comes to digital documents,
many existing paper conventions are adopted (e.g. online newspaper and
magazines). But as digital documents are an entirely different media from paper
ones, the demands involved in accessing information on screen cannot be solved by
simply transporting a printed genre to the electronic medium (Hartley & Jonassen,
1985).
It seems that readers use their ‘knowledge of genre conventions and grammatical
rules to make predictions about the text in order to comprehend it’ (Loines, 1991). If
the layout of a text conflicts with the expectations of the reader then it will increase
their uncertainty by inhibiting the understanding of the meaning of the text (Loines,
1991).
In the digital domain, genre violation seems to leave the reader without an adequate
representation of the text, confusing them so that they rely on navigation buttons and
links to locate themselves (Dillon & Vaughan, 1997).
As many of the genres found in electronic documents seem to follow genres from
print they are regarded by some authors (Dillon & Gushrowski, 2000) as
inappropriate for the digital media. These authors believe that only when genres are
bred within the digital space they will be adequate for electronic documents. This
statement, however, is quite restrictive and it does not observe that even though
transferring a paper based genre to the digital space may result in the short term in
inadequate documents, since each media offer different functional constraints. But, if
this transference is considered only a step in the evolution of a genre towards a new
media, then it is also reasonable to think that the resulting evolved genre will be as
64
Exploratory studies
adequate to the new media as a genre bred in the digital space. In fact, this point of
view is shared by other authors who see this transference of a paper genre to the
digital domain as a step in the natural evolution of a document genre (Eriksen &
Ihlström, 2000; Santini, 2007).
In search of an emerging truly digital genre, Dillon and Gushrowski (2000)

considered the personal homepage as a strong candidate since it has no paper
equivalent with little or no possibility of acquiring conventions from print. The
authors looked at over 100 homepages and identified great similarities between
them. Sampling the pages, a list of elements was identified and their frequency of
occurrence was noted. Then eight experimental homepages were created using these
elements (the five most and least occurring plus the remainder). Participants were
then asked to rank these pages in preference as a home page design, and then to look
at the list of elements made by the authors and select the ones they thought should be
included in a good personal homepage design. Participants were free to add other
elements they felt were relevant to the list.
The results showed that users broadly agree on the contents of a personal web page
and they generally reflect the existing home page contents on the web. The
preferences and expectations across the groups of participants suggest that the
personal home page might be the first unique digital information genre. This being
true, genre emergence could be happening faster than initial studies from paper
suggested (Dillon & Gushrowski, 2000).
Some questions arise from these results obtained by Dillon and Gushrowski (2000)
given that genre emergence might be happening faster than was expected; are there
other genres on the Web? What are the visual elements that compose them?To
answer the first question, it is necessary to categorize the types of existing websites
currently present on the internet.
According to an Internet Activity Index published by the Online Publishers

Association (OPA, 2005), Internet usage can be divided into the four following
activities:
! Content - websites and Internet applications that are designed primarily to
provide news, information and entertainment. Examples of those included in
this class are CNN.com, ESPN.com, Windows Media Player and MapQuest.
65
Exploratory studies
! Communications - websites and Internet applications that are designed to

facilitate the exchange of thoughts, messages, or information directly between
individuals or groups of individuals. Examples of those included in this
category are Yahoo! Mail, AOL Instant Messenger and MSN Groups.
! Commerce - websites and Internet applications that are designed for
shopping online. Examples of those included in this category are Amazon,
eBay, Shopping.com and Dell.com.
! Search - websites and Internet applications that scan the Web to provide
prioritized results based on specific criteria from user-generated requests.
Examples of those included in this category are Google Search, MSN Search
and Yahoo! Search.
Apparently, the OPA classification seems to miss one online activity which McGhan
(2005) reports to have increased in usage in the U.S. since 2002: online banking.
Whether or not these categories have indeed already bred a digital genre is
something that remains to be evaluated.
3.3.2. Objective
The objective of this exploratory study is to investigate whether web users will be
able to determine the category of a website (based on the type of service offered) by
only examining the visual appearance of the page.
Knowing that a document which complies to a set of conventional forms can be

instantly recognized as belonging to a certain genre (Dillon & Gushrowski, 2000;
Waller, 1987b), the study will aim to find out if any of these identifying elements are
part of a conventional form, thus verifying if there are any genres in the process of
formation.
3.3.3. Method
Based on the OPA (2005) and McGann (2005) reports, six categories of online
activities were defined for this study; webmail (an Internet interface that allows users
to read and write e-mail using a Web browser), online news, search engine, e-
commerce, online banking, and an online auction.
Website pages which illustrated the service offered by the website (e.g. a balance
screen for online banking or an inbox for webmail) were captured from the Internet.
66
Exploratory studies
The total number of pages captured was 20 and they were arranged in the categories
as illustrated in Table 5. The decision to use captured screens instead of designing
them was made because the study aimed at investigating if Web users were able to
identify real-life websites and designing them could create constructs which were not
representative of existing web categories ad consequently not be properly identified.
TABLE 5 – NUMBER OF DIFFERENT SCREENS CAPTURED IN EACH CATEGORY.
Category Nº of screens
Web mail 3 screens

Online news 3 screens
Search engine 3 screens
E-commerce
DVD 3 screens
Supermarket 3 screens
Online banking 2 screens
Online auction 3 screens
The supermarket and DVD selling sites were grouped together under the e-commerce
category because both have selling products as their main objective and do this in a
similar manner. Although an online auction website also sells products, the selling
process occurs in a different manner to the e-commerce websites, thus it was placed
in a different category. The smaller number of online banking screens was due to the
difficulty of finding available bank accounts the screens of which could be captured.
To keep the participants from using the logo of the website as a cue, each screen had
its logo edited so that identification would not be possible, and references to the
name of the website written on the screen were blurred so that they could not be
read. But even with the logos removed, participants could still look for words within
the page to help them with guessing the category of the website.
Considering that the study was investigating visual appearance and not semantic or
syntactic cues the possibility of participants gathering cues from text had to be
minimized. The option of using placeholder text would mean altering all text in the
20 website screens which would be extremely time-consuming. The alternative
decision of displaying each screen for an extremely short period of time (1 second)
was made. This way, participants would not have much time to look for words and
use them as cues and if within this short time frame they showed that they had used
67
Exploratory studies
words as cues, it could be an indicator that the visual appearance was not distinctive
enough for them to define the category of the website.
To make sure that the screens were not be displayed without the participant being
ready each participant was allowed to determine when each screen was displayed by
pressing the space key on the keyboard.
The sequence in which the screens appeared was randomized but each participant
was presented with the same screen captured images.
Finally, participants were not given the categories of websites to choose from. This
allowed for more free answers to be given.
3.3.4. Participants
Five people, three female and two male, volunteered for this study. Of the five
participants two were PhD students, three had a MSc. degree, one was an
undergraduate, and the other was a secondary school student. All the participants
confirmed using the Internet at least four times a week for a minimum of one hour.
This ensured that participants had a minimum exposure to the Web so that they were
more likely to have seen the website categories used in the study beforehand.
Participants in this experiment have not taken part in the previous exploratory study
described in 3.2.
3.3.5. Procedure
The study was divided into two parts, called first and second pass. The first pass
aimed at obtaining an immediate identification which would only rely on the overall
appearance of the page without allowing participants to concentrate on detail. The
objective of the second pass was to investigate if any particular visual feature would
have more influence in the process of identification of the page.
In the first pass each participant was told that the test consisted of identifying the
type of service provided by a website by looking at an image of its page. Participants
were told that a total of twenty website screens would be displayed, one at a time, as
they pressed the space key on the computer keyboard. They were informed that each
screen would be shown for a very short period of time and that all the website logos
had been removed. After every screen, each participant was asked to write down
68
Exploratory studies
and/or comment on the type of service the website provided, and to say which
visual elements seen had that led to that conclusion.
In the second pass participants went through all the 20 screens again, but this time
they determined the length of time that each screen was displayed by pressing the
space key on the computer keyboard to change to the next one. This time participants
were allowed to examine each screen for as long as they liked. After each screen was
examined, participants were reminded what their answer in the first pass had been.
They were asked to either confirm or change the answer they had given in the first
pass.
In the case where they changed their mind, they were asked to say whether there
were any elements on the screen that made them change their opinion and, if the
answer was affirmative, they were asked what these elements were. Each session was
recorded on a Digital Voice Recorder so that every comment made by the
participants could be analysed later.
3.3.6. Results
Identification
The following table shows the results for each screen for the first and second passes:
TABLE 6 –IDENTIFICATION RESULTS ON EACH CATEGORY SCREEN BY PARTICIPANT PER PASS.
Participant 1 Participant 2 Participant 3 Participant 4 Participant 5 Total Pos. Id

1st 2nd 1st. 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd
pass pass
Web mail ! ! X ! X !
Web mail ! ! ! ! X ! 11 2
Web mail UI UI ! ! ! !
Online news ! ! ! ! !
Online news ! ! ! ! ! 14 1
Online news ! UI ! ! ! !
Search engine ! ! ! ! !
Search engine X X X ! ! ! ! 12 1
Search engine X X ! ! ! !
E-commerce (DVD) UI UI ! ! X ! X !
E-commerce (supmkt) X X UI ! ! X ! X !
E-commerce (DVD) ! ! X ! ! X !
16 8
E-commerce (supmkt) ! X ! ! ! !
E-commerce (supmkt) X X ! X ! ! X X
E-commerce (DVD) X X ! ! ! !
Online banking ! X ! ! ! !
8 2
Online banking ! ! ! ! X !
Online auction ! ! X ! X X X !
Online auction X X ! ! X X X ! 6 2
Online auction ! X X X X X X !
Total Correct Ident. 12 0 14 5 15 3 15 2 12 7 67 16
!= Correct Identification X = Incorrect Identification UI = Unidentified
69
Exploratory studies
The total number of correct identifications in the first pass for each participant and
for each category can be seen in Table 6. These figures do not allow a fair comparison
between categories since the number of screens in each category varied (e.g. e-
commerce had a total of six screens whilst online banking had only two). So, to have
a better idea of the ease of identification of each category, the positive identifications
for each category on the first pass were added, and divided by the total number of
screens displayed in each category, producing an identification ratio per category as
seen in the following page:
TABLE 7 – IDENTIFICATION RATIO.
Online Online Online search

web mail e-commerce
newspaper auction banking engine
identification ratio 0.73 0.53 0.93 0.4 0.8 0.8
The most easily identified category was the online newspaper followed by online
banking, search engines, and web mail. It seems that the hardest category to identify
was online auctions.
Most of the screens that were not identified in the first pass were identified in the
second. The category that presented the least correct identifications on the second
pass was the online auction: out of the 8 incorrect identifications on the first pass only
3 were corrected on the second pass. The only other category that presented one
incorrect identification on the second pass was the e-commerce category, where one
participant was not able to identify a screen either the first or second time.
Visual cues
Regarding visual cues helping participants to identify the categories of the websites,
observations made when the participants misidentified the website were ignored.
Although misleading visual cues may give clues to relevant aspects of other
categories, for the purposes of this study only the elements that led to correct
identifications will be focused on.
When analysing the comments on which visual cues were perceived as helpful
towards identification, the comments were grouped together according to element
similarity.
70
Exploratory studies
Finally, whenever a participant mentioned a word or a sentence from the screen that
helped guess the category of the website, it was assumed that the content was more
powerful as an indicator of the type of site than visual elements for that particular
participant. As this study focuses on visual elements, specifying which words were
read was not necessary, therefore any comments like this were counted under the
‘Words’ keyword. In Appendix C, the way in which the comments made by each
participant were grouped can be observed.
Once all the elements were sorted, the number of times each one was mentioned
showed the frequency of each element (Ef). The frequency provides a simple way of
evaluating the influence a determined element had on readers’ decisions about the
function of the website.
To analyse the frequency results for every element on each category, the frequency
values will be ordered from the most frequent element to the least frequent providing
at a glance, a general idea of the elements that were cited per category.
Webmail
The graph in Figure 16 shows that there were not great discrepancies amongst the
frequencies for each element, with ‘Checkboxes’, ‘List’ and ‘To-From columns’
(columns in a table that indicate respectively the recipient and sender of an e-mail)
being the most cited elements. Considering that the element ‘To-From columns’
indirectly implies a table, it can be said that the configuration of elements in a table
can be considered quite a relevant visual element on a webmail site.
FIGURE 16- CHART OF RELATIVE FREQUENCIES OF ELEMENTS FOR WEBMAIL SITES.
71
Exploratory studies
Online News
In this category, the visual configuration of text near a picture seems to have been the
most poignant visual cue for the participants (Figure 17). It was followed by the use
of ‘Headlines’ which are a particular element of newspapers. The least frequent
elements in the participants’ commentary were the use of colours and the current
date being displayed. These results suggest that perhaps the way of organizing
pictures and text together with headlines helps readers identify the site as online
news at a glance.
FIGURE 17 – CHART OF RELATIVE FREQUENCIES OF ELEMENTS FOR ONLINE NEWS SITES.
Search Engine
The search engine category had as the most frequent element the repetition of a name
(e.g. searched results about London showing several times the word ‘London’ on the
screen). This might be due to the fact that the predominant feature in a search engine
is words. The second most cited element was ‘Links’ (Figure 18) but this, in this case,
is directly related to ‘Words’ as they also contain words which can be used as cues.
This might indicate that search engines are still more textual than graphic interfaces.
The third most frequent element comment made by the participants (‘Layout’) does
not refer to a precise element, but to a form of arranging graphical and textual
elements on the screen. This possibly indicates that there is a layout that signals to
readers that the page they are looking at is a search engine, although it is not clear
from the data obtained how this layout is described.
72
Exploratory studies
FIGURE 18 - CHART OF RELATIVE FREQUENCIES OF ELEMENTS FOR SEARCH ENGINE SITES.
E-commerce
For this category, ’Words’ was the most frequent element cited by participants as a
cue to the function of the site (Figure 19). The second most frequent element in
participants’ comments was ’Pictures’. All the other six elements mentioned showed
frequencies well below the first two.
FIGURE 19– CHART OF RELATIVE FREQUENCY OF ELEMENTS FOR E-COMMERCE SITES.
73
Exploratory studies
Online Banking
For the online banking screens, ’Words’ was again the most frequent element and, in
this category, the frequencies of the other elements were very small when compared
to it.
FIGURE 20 – CHART OF RELATIVE FREQUENCIES X ELEMENTS FOR ONLINE BANKING SITES.
Online Auction
The graph in Figure 21 shows that for online auctions, once again the pattern of
looking for words to be used as cues to the function of the site was repeated as the
’Words’ elements were the elements most frequently mentioned by participants.
Pictures of products next to textual descriptions (‘Pictures & description’) followed
as the second most frequent element. ‘Colours’ and ‘Pictures’ were the least cited
elements.
FIGURE 21 – CHART OF RELATIVE FREQUENCIES OF ELEMENTS FOR ONLINE AUCTION SITES.
74
Exploratory studies
3.3.7. Discussion
Looking at the elements that were mentioned in each category and analysing the
Unique Elements (elements that were mentioned in only one category) and Common
Elements (elements that were mentioned in more than one category), it can be
observed that the two categories with the lowest identification ratio; online banking
and the online auction, both had few or no unique elements (see Table 8). This might
have contributed to the misidentification and to the inability to identify the category
of the website.
TABLE 8 - A COMPARISON BETWEEN CATEGORIES AND THEIR RESPECTIVE ELEMENTS. A TICK MARK SIGNIFIES THAT THE
ELEMENT INDICATED IN THE ROW HAS HELPED PARTICIPANTS IDENTIFY THE RESPECTIVE WEBSITE CATEGORY.
Categories Web Online e-commerce Search Online Online Total

mail News Engine Auction Banking
Elements
Advertisement " " 2
Checkboxes " 1
Colours " " " 3
Date " 1
Envelope Icon " 1
Headlines " 1
Layout " " 2
Links " " 2
Links and Summaries " 1
List " " 2
Ordered items " 1
Picture " " " " 4
Picture and Text " 1
Picture and " " " 3
description
Pictures and Links " 1
Price " 1
Repeated Names " 1
Shopping Cart " 1
Table " " 2
Tabs " 1
To-From Columns " 1
Underlined Text " 1
Words " " " " " 5
Unique Elements 3 4 3 3 0 1
Common Elements 3 5 5 4 4 5
Of all the categories, online banking, e-commerce and online auction showed the
greatest occurrence of words being used as cues by the participants (Figure 22). The
categories e-commerce and online auctions also presented the greatest number of
screens not identified or misidentified (Table 9) .
75
Exploratory studies
One explanation for the greater use of words for those categories when compared to
the others might be that some particular categories of websites have not yet achieved
a visual standard. This might be due to the fact that they are sites where words are
more relevant than graphic elements, or that these sites belong to new categories or,
even, these categories of websites are not as frequently used as some others.
FIGURE 22 - RELATIVE FREQUENCY OF WORDS BEING USED

AS CUES BY PARTICIPANTS IN EVERY CATEGORY.
TABLE 9 - NO IDENTIFICATION AND MISIDENTIFICATION PER CATEGORY.
No Identification Misidentification
1st pass 2nd pass 1st pass 2nd pass
Web mail 1 0 3 0
Online news 1 0 0 0
Search engine 0 0 3 0
E-commerce 3 0 10 1
Online Banking 0 0 2 0
Online Auction 0 0 8 5
The auction websites were particularly interesting as they were repeatedly

categorized as e-commerce: 7 out of the 15 instances on the first pass and 5 out of 15
instances on the second pass. Amongst them, one screen in particular (Figure 23) had
three participants believe it was an e-commerce site on the first pass and even on the
second pass these participants still thought it was e-commerce. This might be
76
Exploratory studies
explained by the similarities of purposes (both selling products) influencing the

elements used to organize and convey information in both categories. An alternative
explanation is perhaps that there are enough visual similarities between e-commerce
and auction websites to make it difficult for participants to distinguish between the
two. Either way it is possible that the e-commerce and online auction categories
belong to a same web genre and therefore not only have the similarity of purpose but
also share similar structural, linguistic, visual and functional genre conventions.
The other two participants who managed to identify this particular screen (Figure 23)
used a semantic cue, the word ‘Auctions’ to help them. These results might be
explained by this particular site possibly having some features of an e-commerce
website. It can be verified by looking at the elements mentioned in each category
(Table 8) that three of the four visual elements (75%) cited for online auctions are
common to e-commerce websites.
FIGURE 23 – SAMPLE OF AN ONLINE AUCTION WEBSITE SCREEN USED IN THE EXPERIMENT.
Another interesting phenomenon occurred on a particular e-commerce website that

displayed an advert of a ‘summer sale’; it took up about a quarter of the screen area
(Figure 24) with an illustration of a woman on a sunny beach. Three out of the five
participants mistook the website for a travel agency website. On a webmail site,
another participant saw an advert for Sky Cable TV and thought the site was of a TV
channel. The same thing happened on an e-commerce website where a participant
saw a picture of two girls and believed it was a dating site. This might indicate that
sometimes adverts can confuse website users, making them at first glance,
misinterpret the function of a site, and that pictures are an important visual cue.
77
Exploratory studies
FIGURE 24 – SAMPLE OF AN E-COMMERCE WEBSITE WITH ADVERT USED IN THE EXPERIMENT.
On the other hand, it seems that there were some visual elements and a particular
way of organizing the information in some categories, such as online news and
search engines, that might have helped people immediately identify them as such.
Looking into the Unique Elements in each category and how many times each was
cited by the participants, it is possible to see which Unique Elements were more
commonly associated with each category (Table 10).
TABLE 10 – NUMBER OF PARTICIPANTS THAT CITED EACH UNIQUE ELEMENT IN EACH CATEGORY.
Web mail Nº Online News Nº e-commerce Nº Search Engine Nº Online Nº

citing citing citing citing banking citing
Picture and Repeated
Checkbox 4 6 Shopping Cart 2 5 Tabs 1
Text Names
To-From Ordered
3 Headlines 4 Price 2 1
Columns items
Underlined Pictures and Links and
Envelope Icon 1 1 1 1
Text Links Summaries
Date 1
On webmail sites, checkboxes seemed to be the most associated with this category
whilst on online news websites, it was the way of positioning together pictures with
text along with headlines. On e-commerce sites, it was the shopping cart, in search
engines, the presentation of a name repeated several times seemed to be the most
perceived cue, and on online banking web sites, it was tabs.
78
Exploratory studies
The fact that some elements were cited in more than one category (e.g. tables, cited
on web mail and online banking) does not discard them as being identifiers of a
particular category, but it seems that the existence of additional visual cues (unique
or not) might be necessary in the process of identifying the site.
The e-commerce and online banking categories had some unique visual elements, but
these were not cited by many participants. This was observed not only on e-
commerce but also on online banking screens where the most-used cues were words
and sentences. This might indicate that, when lacking visual conventions, users tend
to look for words and sentences to use as cues on guessing a website’s function, or
that for those websites, words are more relevant than graphic elements.
3.3.8. Conclusion
Despite the small scale of this study, it seems to indicate that some visual elements
play an important role in helping participants identify website categories.
The study suggests that these elements may be part of a visual convention, thus
reinforcing findings from Dillon and Gushrowski (2000) about the emergence of
genres in some website categories.
The impact of these genres on the performance of users when searching for
information is yet to be investigated. The role played by the relative position of the
elements on screen in the identification of the website categories was not studied
here. This issue will be further investigated in Chapter 5 where the extent to which
the position of the elements on a web page can influence how conventional users
perceive that page will be examined.
Considering the comment made by Waller (1987b), that a genre conforms to a set of
rules and expectations of the reader, the fact that users were able to identify certain
categories of websites by looking at their layout and sometimes singling out some
visual elements in the process, suggests that users associate the existence of certain
visual elements and layout rules with particular websites. This generates
expectations in the users about what a certain category of website should look like,
which can be an indication that some digital genres might be emerging or even
already be established.
79
Exploratory studies
Finally, it is important to note that as seen in Chapter 1 (page 1) websites are being
constantly updated and although most changes made are to content, it is likely that
some, if not all the pages used in this experiment have changed.
3.4. Conclusion of exploratory studies

The results of both exploratory studies described here suggest interesting research
paths which are worth pursuing in this thesis. However, due to time constraints only
one of them can be investigated in this research.
Text organization through the use of access structures and its impact on readers of
both printed and electronic texts have been well researched and discussed (Dyson &
Gregory, 2002; Hartley & Jonassen, 1985; Lorch & Lorch, 1996; Schultz & Spyridakis,
2004; Waller, 1979; Williams & Spyridakis, 1992). But access structures are one aspect
of the functional constraints of a document which can belong to all three types of
genre conventions: visual, structural and linguistic. Waller (1979) illustrates this with
an example of the visual difference between an educational textbook and a novel.
The author mentions that although both books have a structure, the structure of the
educational text is made explicit by typographic (i.e. visual) signalling whereas in the
novel the structure is linguistically signalled. This typographic signalling can be
achieved via the use of access structures such as headings, summaries and lists. In
the case of headings, their position in relation to the text can become a visual
convention, the word used in the heading can become a linguistic convention, and
the place of that particular heading in the hierarchy of the document and the type of
content associated with it can become a structural convention. This example shows
how difficult it can be to link the functional constraints of a document with a
particular type of genre convention.
Taking all this into account, the research in this thesis will concentrate on visual
conventions of a genre. This will hopefully add to an understanding of genres,
conventions and functional constraints involved in the process of creating electronic
documents. Furthermore it will investigate how not conforming to the visual
conventions of a document genre will affect user performance and consequently
usability of the document.
80
Chapter 4
Finding visual conventions in web
forums
4.1. Introduction
Results of the last exploratory study (see chapter 3, section 3.3) seem to indicate that
some web page layouts are becoming easily recognized with web users being able to
identify the category of a website with a quick glance at its layout. This could mean
that visual genre conventions may be emerging in the Web.
As mentioned previously, this thesis proposes to investigate the impact on user

performance of not conforming to the visual conventions of a genre. However, to
generate the appropriate experimental material to be tested, it is necessary to
establish what is and what is not conventional in web forums.
With this in mind, a survey was conducted to look at the visual features of web
forums and gather knowledge of what a conventional web forum looks like. The
survey is described in this Chapter and its results will provide a basis for the
development of both a conforming and a non-conforming forum to be used in future
experiments.
4.1.1. Web forums

As seen in Chapter 2, genres are communicative events; in other words can only exist
when communication occurs. A genre also relies heavily on a having a discourse
community with certain communicative needs or purposes associated with it. These
two characteristics of genres indicate that in the World Wide Web, web forums are
strong candidates for an emerging digital genre and consequently for having
established visual conventions. Therefore in this thesis they will be the category of
electronic documents chosen for the experiments.
81
Finding visual conventions in web forums
Web forums are ‘ongoing conferences in which participants start topics (or threads),
post replies to each other, and read what others have said’ (Wallace, 1999, p.5).
Within forums, discussions on a particular subject are held, and after some time, web
forum users are said to develop a sense of belonging to a virtual community
(Wallace, 1999, p.56). Forums also have their own particular discourse style
conventions which even include some rules of social interaction (Fayard & DeSanctis,
2006; Fayard, DeSanctis, & Roach, 2004) and differentiates them from other styles of
online communication.
Web forums can be created using several programming languages such as php, Perl,
Java and ASP. However, php has been the original language and until today is one of
the mostly used forum programming languages due to the popularity of a web
forum creation package called phpBB (which means php Bulletin Board). The phpBB
web forum creation package is extremely efficient in terms of creating a functional
web forum, but in terms of flexibility it is quite restrictive for the average developer.
The restrictiveness of this development environment can be thought of as being a
‘design imperative’ (Waller, 1999), which is likely to have had great influence in the
visual conventions of the web forum genre. As a result, even though more modern
languages and resources are available, the conventions are still based on the design
constraints originally imposed by phpBB.
Forum structure
The vast majority of web forums have at least three distinct nested levels: (a) the first
level (index page), where a list of topics contained in the forum is presented; (b) the
second level (threads page) where a list of sub-topics (threads) from a chosen topic
are listed and (c) the third level (post page) where messages from members (posts)
are written and presented. For reasons of simplicity, in this thesis the three levels will
be named forum index page, threads page and post page respectively.
The forum index page, the first and topmost level, presents users with all the topics
that the forum contains, and allows them to choose which subject(s) most interest
them. This level, although accessible to the user, usually does not allow them to
modify any content. Generally forum moderators are responsible for indexing and
creating topics in this area.
The threads page, which is the second level, contains all the threads that are posted on
a particular topic. Threads are groups of linked messages that share a common
82
theme. The threads page allows greater freedom to the users since they can create
new threads and sometimes can even remove them.
Finally, the post page, the third and innermost level, is where users have greater
interaction with the forum. Here, users communicate by reading and writing
messages (posts) to each other and replying to other users’ previous posts. This is the
level that offers users most freedom of expression. Here they can generally,
depending on the forum interface, write, edit, quote, reply to posts and also post
images and emoticons (small symbols or icons that represent moods and feelings).
4.2. Survey 1
4.2.1. Introduction
As mentioned earlier, the aim of this survey is to look at the visual elements which
appear on web forums and see how frequently they occur. The main objective here is
to obtain a picture of what a conventional web forum looks like. This will help to
design forums that either comply with, or do not comply with the visual conventions
of the web forum genre. These forums will be used in future experiments to test the
influence of visual conventions on user performance.
Similar surveys to verify the occurrence of elements in specific web pages have been
conducted before (Cosio & Dyson, 2002; Dillon & Gushrowski, 2000; Ihlström &
Åkesson, 2004; M. R. Patel, 2003; Scheidt & Wright, 2004). Of these surveys, the ones
which are more relevant to this study will be addressed here. Dillon and
Gushrowski (2000), although mentioned in the previous chapter, have important
methodological contributions and will be analysed in more depth first. Then,
Ihlström and Åkesson (2004) and Cosio and Dyson (2002) will be analysed since they
have contributions in terms of the visual organization, positioning and visual
appearance of the elements on the pages.
To verify if the personal homepage was a candidate for an emerging digital genre,
Dillon and Gushrowski(2000) examined 100 websites for a list of occurring elements.
Elements were defined by the authors as features and components of the pages such
as email addresses, graphics and external links. The authors categorized all the
elements and counted their frequencies of occurrence. After everything was counted
the final ranking of elements was divided into three groups: the 5 most common
elements, the 5 least common elements and the remainder. From that division the
83
authors created 8 experimental pages which combined the three categories in a

systematic way to create four ‘common-element’ pages and four ‘uncommon-
element’ pages. Then printouts of the pages were shown to 57 participants who were
asked to rate them from 1-8 in terms of preference of personal homepage design.
However, the experimenters were not clear on whether they had taken any measures
to ensure that the participants judging the layouts were familiar with the personal
homepage. Considering that genres are closely related to a community of people who
use them and are not easily interpretable by someone outside this community, asking
participants who are not familiar with the personal homepage to judge it can lead to
results which might not be representative of the personal homepage genre.
After ranking the pages, participants were then shown the list of elements occurring
in personal homepages and were asked to choose any elements from the list which
they thought should be included in the personal homepage. With this, the authors
managed to verify if there was agreement between what was commonly used in
personal homepages and what users thought should be in a personal homepage. The
authors concluded that there is agreement between the elements users think should
be in personal homepage and what actual elements of the existing personal
homepage.
However, while listing the elements present in a particular category of websites and
checking the agreement of the findings with users are valid procedures, some
important aspects of the page design should also be considered. As the authors
themselves have acknowledged, the number of graphic elements present on a page
can affect its ‘look and feel’ (Dillon & Gushrowski, 2000, p. 203). However, it is not
just the number of graphic elements on a page that can influence its ‘look and feel’,
but also the visual form in which any element appears on a page can influence its
‘look and feel’. Whether elements occur in a graphic, textual or combined form
should be considered when surveying for web genres, since these factors can
influence the appearance of a page. Another aspect not considered by the authors but
which can significantly influence the layout of the page is the positioning of the
elements. The location of key features on a page can not only be learned as users
consistently return to it (Ehret, 2002), but users come to expect them to be placed in
those common positions (Bernard, 2001b).
A study which goes a step further into analyzing the visual aspect of the elements
present on web pages was conducted by Cosio and Dyson (2002). The authors looked
84
at a sample of 50 websites to establish the basic graphic components these websites

used. The authors were interested in finding which elements were more frequent and
which information structures were used on the websites examined. Their website
sample covered diverse activities: education, commerce, financial, public services,
government, cultural, sports related and personal pages. The authors restricted their
analysis to the homepage and two subsequent levels of each website. To categorize
the elements found in the websites analysed, the authors developed modes of
symbolization and configuration of graphic language proposed in Twyman (1979,
1982). The modes of symbolization are of three different types: verbal, pictorial and
schematic (Twyman, 1982). However, when it comes to defining these types
Twyman (1982) simply mentions that the ‘verbal and pictorial categories speak for
themselves’ and goes on to define the schematic category as that whose graphic
images are neither verbal nor pictorial’(Twyman, 1982). Cosio and Dyson (2002) define
the pictorial category as referring to any image, photograph or illustration and the
verbal mode of symbolization the authors define as referring to ‘any letter of the
alphabet, number or any written words (Cosio & Dyson, 2002). For the survey the
authors adopted three categories of configurations of graphic language: linear
interrupted, linear branching and matrix (see Twyman, 1979 for samples of each
mode of configuration).
After examining the websites, Cosio and Dyson (2002) present a list with the
elements found, the frequency counts and the various ways the elements appear (e.g.
the background could appear as a picture or as a solid colour). By looking at the way
the elements could appear the authors take an approach not taken by Dillon and
Gushrowski (2000) but extremely relevant considering the layout of the interface. But
as in Dillon and Gushrowski (2000) for most elements analysed the authors do not
consider their positioning on the page. The only exception is the navigation menu of
the pages analysed. For the menu, the authors consider its location as relevant and
they established its location by using a grid on the screen. But probably due to the
fact that the menu is normally visible on a page, the authors do not consider areas of
the screen which are only accessible by scrolling down the page.
Although it was not the purpose of the survey carried out by Cosio and Dyson (2002)
to obtain a ‘picture’ of one particular website genre, the method used to categorize
the symbolization of web visual elements based on Twyman (1979) is very useful.
However, as some of the elements present on a web page can have more than a single
component (e.g. a menu usually has various words representing menu items and
85
sometimes graphic icons as well), describing only the method of symbolization of the
element is not enough, it is also necessary to explain how the components of the
element are arranged within the element. For this the categories of configuration of
visual language (Twyman, 1979, p.8) used by Cosio and Dyson (2002) are extremely
useful.
As seen in Chapter 2, genre theory usually categorizes genres according to substance

and form (Yates & Orlikowski, 1992); this description is further expanded by Schmid-
Isler (2000) to encompass another component: function. But this categorization still
does not cover all visual possibilities of a genre so Ihlström and Åkesson (2004)
propose four genre concepts: content, form, functionality and positioning. With the
addition of positioning to the description of a genre (Ihlström & Åkesson, 2004), it is
possible to draw a parallel with the layout structure of a genre as defined in Bateman
and Delin (2001) and Delin et al. (2002). A genre layout structure involves the nature,
appearance and position of the communicative elements on a page (see Chapter 3).
So, bringing together the concept of genre components (Ihlström & Åkesson, 2004)
and the layout structure of a genre (Bateman & Delin, 2001; Delin, et al., 2002) the
nature and appearance of an element would relate to the form concept and position
would be linked with the positioning concept.
So, the final survey to be analysed was conducted by Ihlström and Åkesson (2004)
goes a step further than Cosio and Dyson (2002) and considers not only the visual
aspect of the elements, but also the positioning of these elements on the screen. To
establish a framework for identifying the characteristics of a genre, the authors
examined 85 front pages of Swedish online newspapers. The authors were looking
for which elements were present (content), how they were presented (form), what
functionality they required, and where the elements were located on the screen
(positioning).
To establish the location of the elements on screen the authors divided the pages into
a grid. Two features stand out when analysing the positioning system used by the
authors: first the grid used was not divided regularly (i.e. the sections were not
equally sized). The second interesting feature is that unlike what was done by Cosio
and Dyson (2002), Ihlström and Åkesson (2004) have devised a positioning system
which took into account the areas of the page which are only visible when scrolling.
The process used was the following: the whole page (not just the visible area) was
positioned under the grid, then each element was detected, positioned and then
86
related to form and functionality (Ihlström & Åkesson, 2004). The authors however
do not enter into much detail as to how elements were categorized or what on the
page was considered to be an element. This is extremely relevant since an element
could have different components which could appear in different relative positions
to each other and in different formats (e.g. pictorial, alphanumeric, etc.). For example,
when looking at a search element on a page (such as the one in Figure 25) have the
authors considered the element to be the text input box, the button next to it? Or did
they consider the search element to be a combination of the input box and the
button?
FIGURE 25 - SAMPLE OF A SEARCH ITEM EXTRACTED FROM

A SWEDISH ONLINE NEWS WEBSITE.
Similarly the authors do not explain how the criteria used for establishing the form of
the item. The example given by the authors of a ‘video item’ which could be presented
as an icon, an image or a link, is also not very clear since an icon can simultaneously
be an image and a link on a web page.
Summary
The analysis of these three surveys provide useful inputs on how to conduct a survey
to identify what is conventional in web forums. It is important to point out that the
main aim of the survey conducted here differs from all of the three surveys examined
earlier. This survey is focused on identifying what is visually conventional in web
forums and to use this knowledge to develop two web forums to be used in future
experiments.
From an analysis of the previous surveys some interesting points were noted;
since the focus of this thesis is on visual conventions the functionality aspect of
genres studied by Ihlström and Åkesson (2004) will not be considered.
Similarly to what was done by Cosio and Dyson (2002), elements will be classified
using the modes of symbolization found in Twyman (1979). However, here
Twyman (1979) will be used as a basis for the classification system as for reasons of
simplicity some of his categories will be adjusted. Considering the way elements are
87
presented in web forums, and the fact that they can have more than a single
component, the categories of symbolization will be the following:
! Graphic - the element is an image or icon either concrete or abstract (joining
pictorial and schematic representations from Twyman (1979) into one
category).
! Alphanumeric - the element is made of a combination of characters
containing the letters of the alphabet, numbers, punctuation signs and/or
other non-Latin character.
! Hybrid - the element is composed of at least one graphic and one
alphanumeric part.
When defining modes of configuration of graphic language, Twyman (1979), takes

into account how the reader uses the text. Since this thesis is concerned with the
visual conventions of the genre, rather than structural or functional, the
configuration modes adopted here will be based on some configuration modes from
Twyman (1979) but will consider only the visual characteristic of the elements
without concerning with how the reader approaches the element.
! Horizontal - when the components of an element are arranged as a horizontal
line without breaks.
! Linear interrupted - when the components are arranged in a linear fashion
but lines are artificially broken and stacked.
! Matrix - when the components are organized in a matrix fashion with rows
and columns.
The positioning of the elements on a page will be done using an asymmetric grid
considering the whole extent of the page (including the areas off-screen) as was done
by Ihlström and Åkesson (2004) and to certain extent by Cosio and Dyson (2002).
Finally, as in Dillon and Gushrowski (2000) the results from the survey will be tested
on users in order to obtain feedback on their validity.
It is also important that the survey identifies:

! The elements which are present in the forums.
! Where these elements are located on a page.
! How these elements are presented.
! How the subcomponents of the elements are arranged.
88
4.2.2. Method
The survey examines existing web forums listing the existing elements, their
symbolization modes, position and the modes of configuration. Henceforth the
symbolization modes will be referred as symbolization and modes of configuration as
just configuration for reasons of simplicity. As mentioned before, an element could be
categorized in terms of its symbolization as graphic, alphanumeric or hybrid.
The configuration of the elements will either be horizontal, linear interrupted or

matrix. For establishing the positioning of the elements the whole length and width
of the page was taken into account, not just the part which was visible on the screen.
If this was not done, the positioning of the elements would depend on the amount of
horizontal/vertical scrolling. Considering the page shown in Figure 26 it is clear by
looking at the scrollbar that there is more content below the one displayed in the
browser window. By scrolling the screen downwards all elements which were
positioned at the bottom of the screen will be moved to the top of the window, thus
altering their position.
FIGURE 26 - SAMPLE OF THE USER'S VIEW OF A WEB PAGE THAT EXTENDS OUTSIDE THE SCREEN DIMENSIONS.
If only the screen area was considered in positioning the elements, their positions
would be relative to the amount of scrolling. The approach was then to divide the
89
forum page (not the screen) into a grid with 9 quadrants (Figure 27). The sections
were given different dimensions, with the central row having the largest area since it
is usually where the largest elements in the forums are located, such as the forum
index table, threads table or the post area. Horizontally the screen was divided in
three equal parts and vertically it was divided so that the top and bottom parts
contained 20% of the height and the central part consisted of 60% of the height. The
areas were labelled: top left, top centre, top right, centre left, centre, centre right,
bottom left, bottom centre, bottom right.
FIGURE 27 - GRID USED TO DETERMINE THE POSITIONING OF THE ELEMENTS ON A WEB FORUM PAGE.
90
The criteria for determining the positioning of each element in the quadrant are
described in Appendix D.
Configuration
This criterion described how the components of some elements are arranged in
relation to each other. In this survey, each multi-component element can be classified
as horizontal, linear interrupted or matrix. When the element has only one
component it is not possible to establish a configuration for it (e.g. the forum status
indicator usually consists of a single graphic icon, so it is neither horizontal, nor
linear interrupted, nor matrix). These elements are indicated by the abbreviation
‘N.A.’ (not applicable) on the tables. An alphanumeric element was considered as
being multi-component when it had two or more words. The matrix class comprised
of elements which were neither horizontal nor vertical alone, but a mixture of both.
For example an element such as the forum index table which contains rows and
columns of pictorial and alphanumeric elements, is classified as having a matrix
configuration.
4.2.3. Sample
As mentioned before, many forums are generated using templates and forum
creating packages such as phpBB. This causes many forums to have a very similar
appearance and elements, which suggests that a small sample of pages should
represent the great majority of existing web forums. Thus a sample of 15 web forums
was chosen randomly from search engine results. For a forum to be selected it had to
contain at least an index page, a threads page and a post page. The first 15 different
web forums which appeared as the results on a search in www.google.com and had
at least all three levels were selected to be used in this survey.
4.2.4. Procedure
Initially, all occurring elements on all forums were listed. On each web forum, the
forum index page was examined first, then the threads page and finally the post
page. The elements were listed and put in a table and then the forums were re-
examined level by level, and the symbolization, position and configurations of each
element on each level were noted.
91
4.2.5. Results
Identification of the elements
Forum Index (first level)
The elements found in this level are listed in Table 11 and illustrated in Figure 28. For
a full explanation of the elements see Appendix E.
TABLE 11 - LIST OF ALL ELEMENTS FOUND IN THE FORUM INDEX PAGE OF WEB FORUMS.
THE INDENTED ELEMENTS ARE FOUND WITHIN THE FORUM INDEX TABLE.
Forum index elements

Forum index table
Topic title
Topic summary
Nº of threads
Nº of posts
Last post
Topic moderator
Nº of views
Status indicator
Topic type
Menu
Logo
Login field
Captions
Current time & date
User information area
Breadcrumbs
Forum noticeboard
Adverts
Forum search field
From all the elements detected, it seems that the forum index table is probably the
most important in this level given that it is the starting point for any user who
searches the interface. It contains several sub-elements that portray information for
the users and are relevant for this research. The following indented elements seen in
Table 11 were found in the forum index table. Some of the elements are illustrated in
Figure 29, since it was not possible to find a single web forum which contained all the
elements found. Figure 30 and Figure 31 illustrate the elements missing from Figure
28.
92
FIGURE 28 – SOME ELEMENTS ON THE FIRST LEVEL OF A WEB FORUM.
Logo Menu
Breadcrumbs
Login
Field
Forum
Noticeboard
Forum
Index Table
User info area
Captions
Current time
& date
93
FIGURE 29 – SOME ELEMENTS PRESENT IN THE FORUM INDEX TABLE OF A WEB FORUM.
Number of threads
in topic
Topic title
Number of
posts in topic
Last post
in topic
Topic
summary
Topic status
indicator
FIGURE 30 - SAMPLE OF THE TOPIC MODERATOR ELEMENT IN THE FORUM INDEX TABLE.
Topic
moderator
94
FIGURE 31 - SAMPLE OF THE NUMBER OF VIEWS IN TOPIC ELEMENT ON A FORUM INDEX PAGE.
Number
of views
in topic
Thread page (second level)

On the thread page the menu, breadcrumbs, captions, logo, current time and date as
well as the login field elements found in the forum index are still present. This, allied
to the presence of the thread table, makes the overall appearance of this page similar
to that of the first level. In the following page, Table 12 shows a list of the elements
found; the new ones are explained in Appendix E.
TABLE 12 - LIST OF THE ELEMENTS FOUND IN THE THREADS PAGE OF A WEB FORUM.
THE INDENTED ELEMENTS ARE FOUND WITHIN THE THREADS TABLE.
Thread page elements

Thread table
Thread name
Thread author
Thread status
Last Post
Number of replies
Number of views
Emoticons
Menu
New thread button
Number of thread pages
Breadcrumbs
Forum jump
Captions
Logo
Posting rules
Current time & date
Forum search
Login field
Popular topics
95
FIGURE 32 - ELEMENTS IN THE SECOND LEVEL OF A WEB FORUM.
Menu
Logo
Breadcrumbs Login
Field
New thread
button Forum thread
table
Captions
Forum
jump
Current
time & date
Posting Rules
As with the Forum index table, within the thread table there are other elements that
convey information to the users (see the indented elements on Table 12). Some of
these (
Figure 33) are similar to the ones that appear on the Forum index table, others are
different.
FIGURE 33 - THREAD INDEX TABLE AND ITS ELEMENTS OF A WEB FORUM.

Number of
Emoticons Last post views
Thread
name
Thread author
Thread status Number of replies

indicator 96
Post page (third level)
Despite containing some elements that are common to the second and first levels
(menu, breadcrumbs, logo, jump to forum, posting rules, time and date, login field,
adverts, forum search field), the third level is visually quite different from the
previous levels. In the following page is a list of the elements found on this level
(Table 13) and the new elements are explained in Appendix E. The elements are also
illustrated in Figures 34 and 35.
TABLE 13 – LIST OF THE ELEMENTS FOUND IN THE POST PAGE OF A WEB FORUM.
THE INDENTED ELEMENTS ARE FOUND WITHIN THE THREADS TABLE.
Post page elements

Post area
Time & Date of Post
Post content
Author Information
Quote button
Post title
Signature
User profile button
Private message button
Post number
Edit Post
Author’s website button
E-mail author button
Menu
Next/previous thread button
Breadcrumbs
Reply to post button
Jump to forum
Logo
Posting rules
Thread title(out post area)
Current time & date
Adverts
Login field
New post button
Search topic field
Table of contents
97
FIGURE 34 – ELEMENTS IN A POST PAGE OF A WEB FORUM.
Menu
Logo
Login
Breadcrumbs field
Post reply
button Post
area
Next/Previous
thread buttons
Forum
jump
Current time
Posting Rules & date
98
FIGURE 35 – ELEMENTS WITHIN THE POST AREA OF A WEB FORUM.
Time and date of post Author information Post number

Post content
Quote
button
Signature
A summary of which elements appear on each web forum level is shown on Table 14.
From this table it is possible to see that many elements appear across all forum levels
and this is an indicator that usually there is a fair degree of consistency across the
levels. As mentioned in Nielsen (1993, p.91) to some extent this might be due to the
standards imposed by the web forum creation engines, but still there is the case of
some elements where repetition could be solely due to a design choice.
99
TABLE 14 – SUMMARY OF THE ELEMENTS PRESENT IN EACH OF THE THREE FORUM LEVELS.
Forum index Threads page Post page

Menu " " "
Logo " " "
Login field " " "
Captions " "
Current time and date " " "
Users information area "
Breadcrumbs " " "
Forum notice board "
Adverts " "
Forum search field " " "
Topic title "
Topic summary "
Status indicator " "
Number of threads in topic "
Number of posts in topic "
Last post in topic "
Topic moderator "
Number of views in topic "
New thread button "
Number of thread pages "
Forum jump " "
Posting rules " "
Popular topics "
Thread name "
Thread author "
Last post "
Number of replies "
Number of views "
Emoticons "
Post area "
Next/previous thread button "
New post button "
Table of contents "
Time & date of post "
Post content "
Author info "
Quote button "
Post title "
Signature "
Profile button "
Private message button "
Post number "
Edit post "
Author’s website button "
Email button "
100
The next step was to find out how these elements were displayed across different
web forums. As seen before, there are three possible formats for presenting the
information. These are graphic, alphanumeric and a hybrid form which mixes
graphic and alphanumeric information into one element.
Symbolization
Most elements in all three levels of a web forum are presented in an alphanumeric
format or hybrid format (graphic and alphanumeric). In the Index page of the forum
only one element was presented in a pure graphic style and this was repeated in the
threads page of the forum. Apparently it was only in the post page that a greater
variety of elements were presented in a pure graphic style. But even this does not
happen frequently and the graphic symbolization mode of elements is quite rare.
Tables 51, 52 and 53 in Appendix F show the frequency of symbolization modes of
the elements at the first, second and third levels of the forums respectively. As in
previous tables, the indented elements are found within the topmost non-indented
element. Note that not all the elements listed appear together on the same web forum
page; hence most totals do not add up to 15.
Position
As mentioned before, the results of this survey will be used to create the conforming
and non-conforming material to be tested in future experiments. For this reason, as in
Dillon and Gushrowski (2000), the more frequently occurring and the least frequently
occurring values for the position of the elements are the most relevant and will be
examined further.
However, determining the position of the least frequently occurring elements can
present a problem. Problems can emerge when the element (which appears only a
few times) appears simultaneously in two screen positions (see criterion (d) in
Appendix D). This double position which would be the least frequently occurring
position for the element coincides with the most frequently occurring position.
Whenever this happened, a position in which the element never occurs (i.e. a non-
occurring position) was chosen as the least occurring position for that element. To
simplify the visualization of the position results, an image of the grid was created
and the name of the element was placed in the quadrant which represents either its
most occurring or least occurring position. The most occurring positions of each
element in the forum index, threads and post pages are displayed in Figures 36, 38
101
and 40 respectively. Figures 37, 39 and 41 show the least/non occurring positions for
every element in the forum index, threads and post pages respectively.
It is important to note that whenever an element is shown in bold italicized type, this
indicates that on the same level page the element appeared most frequently in two
simultaneous positions (e.g. in Figure 38, the element ‘new topic button’ appears on
the top left quadrant and the bottom left quadrant on the same page at the same
time). When an element appears the same number of times in different positions, the
name of the element was placed on all tied positions but without bold or italic. For
example, in Figure 39 the element login field appears the same number of times in
the top left and centre-top positions so it is placed on both.
FIGURE 36 – MOST OCCURRING POSITIONS OF FIGURE 37 – LEAST/NON OCCURRING POSITIONS OF THE

THE ELEMENTS IN THE FORUM INDEX PAGE. ELEMENTS IN THE FORUM INDEX PAGE.
Breadcrumbs Menu Login Field Login field Logo Menu
Logo Adverts Forum search Noticeboard Captions
Noticeboard Adverts
Time & date
Index table Index table User info area
Adverts
Captions Login field

Adverts
Time & date Time & date
User info area Forum search
102
FIGURE 38 – MOST OCCURRING POSITIONS OF THE FIGURE 39 – LEAST/NON OCCURRING POSITIONS OF

ELEMENTS IN THE THREADS PAGE. ELEMENTS IN BOLD THE ELEMENTS IN THE THREADS PAGE.
ITALICIZED TYPE APPEARED MOST FREQUENTLY IN TWO
SIMULTANEOUS POSITIONS.
New topic Menu Nº of thread New topic New topic btn Menu
button pages button
Breadcrumbs New topic btn
Breadcrumbs Forum search Login field
Logo Nº of thread
Logo Login field pages
Login field
Thread table Forum search Forum jump
New topic Time & date Nº of thread Thread table Captions

Button pages
Forum search New topic btn Forum search
Captions Forum jump
Captions Time & date
Posting rules Popular topics
Posting rules
103
FIGURE 40 – MOST OCCURRING POSITIONS OF THE FIGURE 41 – LEAST/NON OCCURRING POSITIONS OF THE
ELEMENTS IN THE POST PAGE. ELEMENTS IN BOLD ELEMENTS IN THE POST PAGE.
ITALICIZED TYPE APPEARED MOST FREQUENTLY IN TWO
SIMULTANEOUS POSITIONS.
Breadcrumbs Menu Adverts Login field Next / previous Logo

button
Reply button Adverts Login field New post btn Menu
Topic title
Logo New post btn Reply button
Breadcrumbs
Topic title Forum search New post
Logo button
Post area Post area Post area
Table of Forum jump

contents
Search field
Reply button Next/previous Forum jump Breadcrumbs Forum jump Time & date
button
Posting Rules New post btn Reply button Posting rules
Time & date
Time & date
Adverts
New post btn
Search field
104
Configuration
Most elements on web forums seem to appear in a horizontal configuration mode
and the second most frequent configuration mode seems to be the matrix mode
closely followed by the linear interrupted one.
Some elements, however, only have one component which does not allow for any
configuration mode to be established (e.g. the forum status indicator is usually a
graphic icon, so it is impossible to establish a configuration mode). The frequency
counts of the configuration modes of each element present on the web forums are
listed in the tables in Appendix G.
4.2.6. Discussion
An overall examination of the results obtained seem to indicate that most web
forums have their elements presented in a very similar manner, positioned in similar
places and configured in similar ways. This seems to suggest that web forums have
highly conventional layouts and could be strong candidates for a digital genre. Some
particularities of each of the three attributes surveyed are commented on below.
Position
At first glance, the top and bottom of the web forum levels seem to have a high
concentration of elements compared to the centre. This however is not necessarily
true since the centre areas of the forums usually contain a table which is made up of a
considerable amount of sub elements which also densely populate this area of the
screen. So in the end, the central area of the forum appears as densely populated as
the top and bottom of the screen and in some cases, even more.
Looking at the elements which are present in all three forum levels (Figure 36, Figure
38 and Figure 40) it is possible to see that these tend to be in the same position in all
levels where they appear. This to some extent contributes to the internal consistency
of the pages as mentioned before.
Symbolization mode
Considering that the sole purpose of a web forum is to be a media for discussion, it is
not surprising that the great majority of elements are presented in an alphanumeric
or a hybrid form. After all, written text is currently the best way to communicate on
the web.
105
Configuration
The configuration of most elements in a web forum appear to be either horizontal or
linear interrupted. This is not surprising considering that the symbolization mode of
most elements is alphanumeric. There is also a sense that some alphanumeric
elements are configured in horizontal format just because of their length, and were
they longer, they would be configured as linear interrupted. For this survey a
distinction was made between these two configuration modes regardless of this
issue. The reason for this decision is based on the concept that if some long
alphanumeric elements which would normally be configured as linear interrupted
were configured as horizontal, it would add to the non-conformance of the design.
The inverse situation also applies.
4.2.7. Most frequent groups of elements

The next chapter will describe two experiments which asked web forum users to
compare several templates and choose the ones they thought were more
conventional (experiment 1) and the ones they thought would be easier to use
(experiment 2). These templates ranged from highly conventional to highly
unconventional, and were created based on the results obtained from the survey.
These results indicated (a) which elements most commonly occur on web forums and
(b) the most frequently occurring and least frequently occurring values for the
symbolization, position and configuration of these elements.
Although the results of the survey indicate which elements most frequently occur on
web forums, simply using these elements to create the templates could generate
unrealistic combinations. This can make the experimental material not representative
of existing forums. Another approach (the one chosen) is to choose all the elements
which occur in a web forum and see which combination of elements happens more
frequently and use this. This approach is more likely to result in a design which more
resembles existing forums than merely picking the most frequently occurring
elements. In Appendix H a table shows all the groups of elements appearing on each
web forum examined, and the resulting patterns. The results for the most frequent
patterns for each forum level are shown here in Table 15.
106
TABLE 15- MOST FREQUENT GROUPS OF ELEMENTS IN EACH FORUM LEVEL (SHOWN IN COLUMNS). NAMES IN BOLD
REPRESENT THE ELEMENTS WHICH APPEAR ON ALL THREE FORUM LEVELS AND IN ITALIC THE ELEMENTS WHICH APPEAR ON
TWO FORUM LEVELS.
Forum index page Threads page Post page
Menu Menu Menu

Logo Logo Logo
Login Field Login Field Login Field
Current Time & Date Current Time & Date Current time & date
Breadcrumbs Breadcrumbs Breadcrumbs
Captions Captions next/previous thread button
User Info Area Forum Jump Forum jump
Forum Noticeboard Nº of Thread Pages reply to topic button
Forum index table Posting Rules Posting Rules
Forum Title New Topic Button Post area
Status Indicator Thread table Author Info
Forum Summary Thread Name Post Content
Nº of Threads in Forum Status Indicator Time & Date of Post
Nº of Posts in Forum Thread Author Post Title
Last Post Nº of Replies Quote Button
Nº of Views Signature
Last Post Profile Button
Private Message Button
email Button
Poster website button
From this table, it is possible once more to see that there is a group of elements that
appear repeatedly across the forum levels (shown in bold) and some smaller sets of
elements appearing in two levels (shown in italic). This indicates that these elements
are likely to be more relevant in terms of maintaining the visual consistency within
the website. It is possible that changes in the symbolization, configuration and
position of these elements will bring more problems to users than the same changes
in the other elements.
The most frequent groups of elements in each forum level (shown in Table 15) will be
used on both the conforming and non-conforming forums. If different elements were
used it would be difficult to establish whether variations in the perceptions of
conventionality and ease of use were due to the presence (or absence) of an element
or the variation of the attributes of the elements.
107
4.2.8. Conclusion
The survey gathered data on the elements present on web forums. The data indicates
the frequency of the type of symbolization, configuration and the positioning for
each forum element. As previously explained in the introduction to the Chapter,
these results will serve as basis for constructing the material to be used in the next
experiments. But before any material is developed, it is important to ensure that the
system used in this survey to establish the set of ‘conventional’ elements represents
what is considered by users to be a convention-conforming web forum layout.
It is also interesting to see if in terms of user perceptions ease of use and

conventionality are correlated. Considering that users may be more willing to use
web pages which they perceive as easy to use, this could give indications as to
whether conventionality may be a page attribute which can make users more willing
to use a web page.
With that in mind, three experiments described in the following Chapter will be
conducted next; the first one will ask users to choose from a series of templates
(created based on the results of the survey), those which they think look more like
web forums they have seen before; the second experiment will use the same
templates which were used in the first experiment to ask users which templates they
perceive as easier to use. The results of these first two experiments will then be used
to create the experimental material (one conforming forum and one non-conforming
forum). Finally, after the materials are created, a survey will assess some aspects of
the non-conforming interface regarding identification and user comprehension. Then
the experimental material will be used in subsequent experiments.
108
Chapter 5
Assessing the results of the survey in
terms of user perceptions
This Chapter describes three experiments which were conducted to validate the
results obtained in the survey. This validation process consisted of asking web forum
users about different aspects relating to the results obtained from the survey.
The purpose of the two initial experiments is twofold: first they will attempt to
confirm that the results obtained from the survey agree with what web forum users
perceive a conventional and easy to use web forum layout to be. Second, they will
attempt to verify if there is a relationship between what users perceive as a
conventional web page and a web page which is easy to use. This relationship is
important to establish as it can give indications as to whether conventionality and
ease of use are perceived as one thing or two separate concepts. Based on the results
of these two experiments, the conforming and non-conforming experimental material
will be created.
Finally, a third experiment, which tested if web forum users would be able to
correctly identify relevant aspects of the design of the non-conforming experimental
material which was created, is described.
109
Assessing the results of the survey in terms of user perceptions
5.1. Experiments to test templates created based on the

results of the survey
5.1.1. Experiment 1 – Perceptions of conventionality of the templates
based on the results of the survey
Introduction
This experiment consists of presenting to participants a number of forum page
templates which were created based on the results of the survey. Participants will be
asked which templates they think are most similar to pages they have seen before in
other web forums. The layout of the templates will range from conforming (most
frequent symbolization, configuration and position values for the elements) to non-
conforming (least frequent symbolization, configuration and position values for the
elements).
Objectives
The survey previously described gathered elements which were present on web
forums and their most common symbolization modes, positions and configuration
modes. From these results a set of elements was established as most common to web
forums and the most frequent values for the three attributes (symbolization, position
and configuration) were noted as likely candidates for the creation of a conforming
page layout. However, a web forum generated by these results in an ‘artificial’
manner might not properly represent the web forum genre. So, the need to obtain the
feedback of web forum users is quite important.
Also, in the second exploratory study (see Chapter 3, section 3.3.7) it was detected
that the way some elements are positioned in relation to other elements cued
participants as to the category the web page belonged to. This seems to indicate that
perhaps position or configuration of elements may have more influence on user
perceptions of conventionality.
Finally, as seen in Chapter 2, genres emerge from communication needs of discourse

communities and members of these communities are able to identify, decode and
understand them. Thus, testing the forums designed on the basis of the results of the
survey with web forum users, seemed the best way of ensuring that they are
appropriate designs and meet the expectations of users for the genre. So, this first
experiment was put in place to:
110
! Verify if the conclusions drawn from the survey regarding a conforming

forum layout are in agreement with what is perceived as convention-
conforming by web forum users.
! Find which web forum layouts users consider least conventional so that it can
be used as a template for designing the non-conforming material for future
experiments.
! See if any of the three attributes of an element (symbolization, position or

configuration) could have stronger influence on user perception of
conventionality when compared to the other attributes. If that is the case, it
can be extremely valuable for designers to know what impact changing each
of theses attributes will have in terms of user perceptions of conventionality.
The results of this experiment will be used (along with the results of the following
study) to define the templates to be used to create the experimental material for
future experiments.
Method
Material
Based on the results of the survey, 24 web forum screens were designed in a way that
each level comprised of two groups of forum templates: one group, named
conforming and the other group non-conforming. Each group had four templates per
forum level; one main template and three variations.
The main template in the conforming group was meant to represent a highly
conventional layout and it was designed with the attributes of the elements in the
most frequently occurring values. The main template in the non-conforming group
was meant to represent a non-conforming forum layout so it was designed with the
attributes of the elements in the least frequently or non-occurring values.
Because of the possibility that these main templates do not represent what users
perceive as more conventional or least conventional in a web forum, template
variations were created. The variations were intended to also help verify if any of the
three attributes would have more influence than the other two on the perception of
conventionality of web forum users. Knowing if one of the attributes has more
111
influence on user perception than the other will allow for better understanding of
how users may perceive the conventionality of web pages.
Because in this experiment three attributes of the elements are studied

(symbolization, position and configuration) there were three variations per level per
group, and on each variation template one attribute was altered. Samples of main
conforming and non-conforming templates with their variations are shown in figures
Figure 42 and Figure 43.
FIGURE 42 - EXAMPLE OF THE MAIN CONFORMING TEMPLATE (LEFT) AND ITS THREE VARIATIONS (RIGHT) ON SYMBOLIZATION
MODE, POSITION AND CONFIGURATION MODE (FROM TOP TO BOTTOM RESPECTIVELY)
112
FIGURE 43 – EXAMPLE OF THE MAIN NON-CONFORMING TEMPLATE (LEFT) AND ITS THREE VARIATIONS (RIGHT) ON
SYMBOLIZATION MODE, POSITION AND CONFIGURATION MODE (FROM TOP TO BOTTOM RESPECTIVELY)
For example, a variation of configuration on a conventional template (see Figure 44)

was designed with the symbolization and position attributes fixed at the most
frequently occurring values, and the configuration changed to the least or non-
occurring value. In the same way, a variation of symbolization was designed with the
position and configuration attributes at their most frequently occurring values and
the symbolization at its least or non-occurring value (Figure 45)
113
FIGURE 44 - COMPARISON BETWEEN THE MAIN CONFORMING MENU ELEMENT (WITH ALL ATTRIBUTES ON THEIR MOST
FREQUENT VALUES) AND THE MENU VARIATION WHERE THE CONFIGURATION IS AT ITS LEAST FREQUENT VALUE.
FIGURE 45 - COMPARISON BETWEEN THE MAIN CONFORMING MENU ELEMENT (ALL ATTRIBUTES ON THEIR MOST FREQUENT
VALUES) AND THE MENU VARIATION WHERE SYMBOLIZATION IS AT ITS LEAST FREQUENT VALUE
A variation of configuration on a non-conforming template was designed with the

symbolization and position of the elements fixed in their least frequent values and
the configuration in their most frequently occurring value (Figure 46). Similarly a
variation of the symbolization attribute on a non-conforming template was done with
the configuration and position attributes fixed on their least frequent values and the
symbolization in its most frequently occurring value (Figure 47).
The variation of an attribute value was always made to the opposite extreme value
since subtle changes (e.g. from the most frequently occurring value to the second
most occurring value) might not generate a difference big enough to be detected by
the participants.
114
FIGURE 46 - COMPARISON BETWEEN THE MAIN NON-CONFORMING MENU ELEMENT (WITH ALL ATTRIBUTES ON THEIR LEAST
FREQUENT VALUES) AND THE MENU VARIATION WHERE THE CONFIGURATION IS AT ITS MOST FREQUENT VALUE
FIGURE 47 - COMPARISON BETWEEN THE MAIN NON-CONFORMING MENU ELEMENT (WITH ALL ATTRIBUTES ON THEIR LEAST
FREQUENT VALUES) AND THE MENU VARIATION WHERE THE SYMBOLIZATION IS AT ITS MOST FREQUENT VALUE
115
To simplify the classification of the templates a coding system was put in place. It
consists of a letter and two numbers. The letter indicates whether the template is
from the conforming (C) or non-conforming group (N). The next number represents
the forum level with which that template is associated (1-3) and the last number (the
rightmost one) represent the number of the template (i.e. 1 to 4 as each forum level
had four templates). The coding system is explained in Figure 48.
FIGURE 48 – CODING SYSTEM USED TO NAME THE FORUM TEMPLATES.
The templates which were generated in order to test the results of the survey against
web forum user perceptions are illustrated on Figure 49. The templates C11, C21,
C31 are the main conforming templates with all the attributes set at their most
frequently occurring values. The templates N11, N21 and N31 are the main non-
conforming template with all three attributes set at their least frequently occurring
values.
116
FIGURE 49 - THUMBNAILS OF THE 24 CONFORMING AND NON-CONFORMING

WEB FORUM PAGES DESIGNED FOR ASSESSING THE RESULTS OF THE SURVEY
117
Table 16 shows how the three attributes (symbolization, position and configuration)
were used to design each template in each forum level in both the conforming and
non-conforming groups.
TABLE 16 – SYMBOLIZATION, POSITION AND CONFIGURATION VALUES FOR EACH

CONFORMING AND NON-CONFORMING SCREEN.
Conforming Symbolization Position Configuration Template

Forum index MF MF MF C11
MF MF LF C12
MF LF MF C13
LF MF MF C14
Threads page MF MF MF C21
MF MF LF C22
MF LF MF C23
LF MF MF C24
Post page MF MF MF C31
MF MF LF C32
MF LF MF C33
LF MF MF C34
Non-conforming Symbolization Position Configuration Template
Forum index LF LF LF N11
MF LF LF N12
LF LF MF N13
LF MF LF N14
Threads page LF LF LF N21
MF LF LF N22
LF MF LF N23
LF LF MF N24
Post page LF LF LF N31
MF LF LF N32
LF MF LF N33
LF LF MF N34
MF – Most frequently occurring value

LF – Least frequently or non-occurring value
Research has shown that repeated exposure to a stimulus increases the sense of
familiarity in users (Zhang & Ghorbani, 2004). Thus, because all participants will be
exposed to all the templates, if the same content is used over and over it is possible
that the sense of familiarity of users with the templates will increase. However, this
familiarity can be a result of repeated content exposure, rather than layout. This
improved sense of familiarity could be mistaken for conventionality by the
participants.
In an attempt to minimize the possibility of the content influencing the judgment of

participants, two different content sources are used in the templates. Both contents
used in the creation of the templates were retrieved from existing web forums.
118
Because it is the layout of the interface that is under scrutiny in this test, some aspects
of the layout which are not being investigated were held constant. The same sans-
serif typeface (Arial) was used on all templates because it is extremely common in
web forums. The same colour scheme was also adopted for all templates. These two
layout aspects were fixed at a constant value to keep them from influencing the
perception of conventionality of users.
The graphic elements used in the design of the conforming templates were obtained
from existing web forums. However, because the non-conforming templates have as
their basis the least, or non-occurring symbolization modes for each element, some
symbolization modes are hardly ever (if at all) found in existing forums. This
happens especially when the most frequently occurring symbolization mode is
alphanumeric and the non-occurring symbolization mode is graphic. Whenever this
happened (e.g. a textual element needed to be replaced by a graphic element in a
non-conforming template) the following procedures were adopted:
(1) If the textual element was not exclusive to web forums (e.g. FAQ menu
item) and in another interface, a graphic was used to represent the
element (e.g. a question mark icon used to represent FAQ), then this
graphic element was used in the forum.
(2) If (1) was not possible then an attempt was made to find an icon which
could represent the element. The initial criterion was to use a
representational image but if that was not possible then an abstract one
was used.
The aesthetic consistency (consistency of style and appearance) with the other
elements was maintained as much as possible.
As mentioned before, to achieve greater likeness to a real web experience, each screen
was designed in HTML and was displayed using a Firefox browser. This allowed
participants to use the mouse as they normally would when browsing the Internet.
However, all links and clickable buttons on the templates were disabled.
Experimental design
For the test, the templates (see Table 16) were grouped into pairs. Each pair was
composed of one template from the conforming group and one from the non-
119
conforming group. In a pair both templates being used were from the same forum
level so that two different levels were never put together.
To create a setting that would emulate as close as possible a real browsing

experience, with scroll bars and real web page dimensions, the templates were
presented to participants in a browser engine (e.g. Firefox). While participants could
only see one screen at a time, they were given free mobility to switch to and fro
between the screens as many times as needed in order to compare them.
The pairs were put together so that all templates on one level of a group were
compared with all templates on the same level of the other group. For example C11
was compared with N11, N12, N13 and N14 and the same happened for C12, C13
and C14. This resulted in a total of 48 pairwise comparisons per participant.
For each participant the order in which each screen was displayed within each pair
was randomized. For example, for one participant a pair could appear as C11, then
N13; for another participant the same pair could appear as N13 then C11. The
sequence in which the pairs appeared was also randomized so that each participant
saw the pairs in a different order. Following each pair, a white background screen
was displayed to separate the comparisons and ensure that pairs remained together.
On the top right corner of each screen, the number of the pair and screen that were
being displayed was shown. This information was displayed using very small
characters so as to interfere visually as little as possible with the layouts. It had the
purpose of both guiding the participant when choosing the screens, and the
experimenter when taking note of the choices made (some samples can be seen in
Figures 50 and 51).
120
FIGURE 50 – SCREENSHOT REFERRING TO THE C11 TEMPLATE OF THE CONFORMING GROUP. THIS TEMPLATE HAS THE
SYMBOLIZATION, CONFIGURATION AND POSITION ATTRIBUTES AT THEIR MOST FREQUENTLY OCCURRING VALUES.
FIGURE 51 – SCREENSHOT REFERRING TO THE N11 TEMPLATE OF THE NON-CONFORMING GROUP. THIS TEMPLATE HAS THE
SYMBOLIZATION, CONFIGURATION AND POSITION ATTRIBUTES AT THEIR LEAST FREQUENTLY OR NON OCCURRING VALUES.
Participants
Twelve people, all familiar with web forums, took part in the experiment.
Participants were either graduate or undergraduate students from different academic
backgrounds. All participants were fluent in English either as their first or second
121
language. Participants in this experiment have not taken part in any previous
experiments conducted for this thesis.
Procedure
Each participant was told that they were going to see 48 pairs of screens and on
seeing each pair was going to be asked to choose the screen layout which they
thought looked more like what they would see on a web forum. Participants were
also told that only the visual aspect was being considered, thus they should not pay
attention to the content. It was also reinforced that the objective was not to choose the
best design, but the design that was more similar to a design they would expect to
see on a web forum.
Before the test started, each participant listened to a brief description on how to
navigate through the interface, and was allowed to use a simulation to get used to
using the keyboard to move forward and back on the screens. For each pair of
templates, participants could use the ‘<’ and ‘>’ keys on the keyboard to switch
between the templates. On each template the participant was allowed to scroll up
and down (and left to right when necessary) as they normally would when browsing
the Web. After looking at both the templates of a pair the participant was taken to a
screen with a white background which asked the participant to tell the experimenter
which screen from the pair looked more like what the participant would expect to see
in a web forum layout. After the choice was made, the experimenter noted on a table
which screen was chosen, and by pressing the ‘>’ key the participant was taken to the
next pair.
During the test, participants were allowed to make comments on anything that they
found relevant and the experimenter wrote down the comments.
Results
Templates chosen as more similar to other web forums
Of the 576 (48 pairs seen by 12 participants) times that each pair of templates was
displayed during the experiment, the templates from the conforming group were
chosen 482 times (83.7%) over the non-conforming ones as being more similar to
other web forum layouts. In 94 (16.32%) cases the templates from the non-conforming
122
group were thought to look more like what users would expect to see on a web
forum.
Thus, overall the templates from the conforming group were perceived as more
similar to other forums than the templates belonging to the non-conforming group. A
Chi-square goodness-of-fit test was applied to the data and the results are
summarized in Table 17.
TABLE 17 – SUMMARY OF THE CHI-SQUARE TESTS ON THE NUMBER OF TIMES EACH

GROUP WAS PICKED AS MORE LIKE OTHER WEB FORUMS.
Group
Conforming Non-conforming P
Forum index (level 1) 160 32 < 0.01
Threads page (level 2) 146 46 < 0.01
Post page (level 3) 176 16 < 0.01
Total 482 94 < 0.01
Non-conforming templates frequently chosen as similar to other web forums
Another interesting result which came out of this experiment is that some non-
conforming templates were chosen as similar to other web forums a considerable
number of times. Of all the non-conforming templates, the one which was chosen the
most was N22 (chosen 20 times). This template represented a threads page layout
(second level) which had the elements presented in their most frequent value, but
their position and configuration in their least frequent or non occurring value
(Symbolization=MF, position=LF, configuration=LF). The template is shown in
Figure 52.
123
FIGURE 52- SAMPLE OF THE NON-CONFORMING TEMPLATE FROM THE THREADS PAGE LEVEL WHICH WAS CHOSEN MORE
FREQUENTLY AS SIMILAR TO EXISTING WEB FORUMS. THIS TEMPLATE HAD THE ELEMENTS IN THEIR MOST COMMON
SYMBOLIZATION MODE BUT THEIR POSITION AND CONFIGURATION IN THEIR LEAST OR NON OCCURRING VALUES.
Similarly, in the post page (third level) the most frequently chosen non-conforming
template was N32, chosen 7 times (see Figure 53) which was also designed with the
same attribute pattern (symbolization=MF; position=LF; configuration=LF).
However, in the forum index page (first level) the non-conforming template which
presented the highest frequency of times chosen (10 times) was N11 which had all
three attributes in their least occurring values (symbolization=LF, position=LF,
configuration=LF). The template is shown in Figure 51.
FIGURE 53 – SAMPLE OF THE NON-CONFORMING TEMPLATE FROM THE THREADS PAGE LEVEL WHICH WAS CHOSEN MORE
FREQUENTLY AS SIMILAR TO EXISTING WEB FORUMS. THIS TEMPLATE HAD THE ELEMENTS IN THEIR MOST COMMON
SYMBOLIZATION MODE BUT THEIR POSITION AND CONFIGURATION IN THEIR LEAST OR NON OCCURRING VALUES.
124
Conforming templates chosen the least times as similar to other forums

Looking at the symmetrical situation, at the templates of the conforming group
which were chosen the least number of times, it is possible to see that on the forum
index page, template C12 was chosen 38 times. This template had the symbolization
and position of the elements in their most frequently occurring values, and
configuration in its least occurring value. On the threads page (second level) and post
page (third level), the conforming templates which were least picked were C23 (24
times) and C33 (40 times). These two templates presented the same arrangement for
the three attributes (symbolization=MF, position=LF, configuration=MF). However,
looking at the second least chosen conforming templates, a pattern emerges: all three
templates had the symbolization of the elements in its least occurring value and the
position and configuration in their most occurring values (symbolization=LF,
position=MF, configuration=MF). It is also possible to observe from Table 18 that for
the templates in the first and third levels the difference in frequency between the
least chosen and second least chosen templates is not very big which could indicate
that there was no significant difference in perception from the perspective of the
participants.
TABLE 18 – LEAST CHOSEN AND SECOND LEAST CHOSEN CONFORMING TEMPLATES IN EACH FORUM LEVEL. THE TABLE ALSO
INDICATES THE VALUES OF EACH ATTRIBUTE OF THE ELEMENTS AND THE NUMBER OF TIMES (FREQUENCY) EACH TEMPLATE
WAS CHOSEN.
Symbolization Position Configuration Frequency

C12 MF MF LF 38
Level 1
C14 LF MF MF 40
C23 MF LF MF 24
Level 2
C24 LF MF MF 38
C33 MF LF MF 40
Level 3
C34 LF MF MF 43
Least chosen conventional template
Second least chosen conventional template
Non-conforming templates least chosen as being similar to other forums

Similarly, the non-conforming templates which were least chosen as being similar to
other web forum layouts were N14 at the index page level, and N23 at the threads
page level. Both these templates had the symbolization and configuration at their
least occurring values and the position at its most occurring value. However, the
template on the post page which was never picked (N34) had a different
arrangement of these attributes (see Table 16).
125
Looking at the number of times that each template from the conforming group was
chosen as looking more like other web forum pages (Table 19), it is possible to
observe that the frequency difference between the most often chosen template and
the second most chosen one is relatively small.
TABLE 19 – NUMBER OF TIMES EACH TEMPLATE IN THE CONFORMING GROUP

WAS CHOSEN AS MORE LIKE OTHER WEB FORUM PAGES (RESULTS ARE OUT OF A
TOTAL OF 48 COMPARISONS PER TEMPLATE PER LEVEL.

C11 41 C21 43 C31 48
C12 38 C22 41 C32 45
C13 41 C23 24 C33 40
C14 40 C24 38 C34 43
Discussion
The templates, which according to the survey results were designed to be conforming
(i.e. conforming group), were indeed chosen by participants as more like other web
forums when compared with the non-conforming group. This seems to indicate that
the method used to combine the results of the survey to create a conforming forum
interface are in accordance with what users expect a convention-conforming web
forum page to look like.
The templates chosen the least number of times as more similar to other forum pages
can be considered as samples of pages which violate the expectations of users
regarding the visual appearance of a web forum page. This can be an indication that
these pages to a certain extent, do not meet the expectations of web forum users and
thus, possibly, do not conform with the visual conventions of the web forum genre.
Considering that the template variations on the conforming layout had only one of
the three element attributes modified to its least frequent value, the fact that these
templates were picked more often as similar to other web forums than the non-
conforming template and its variations suggests that altering just one of the three
element attributes (symbolization, position or configuration) does not seem sufficient
to alter perceptions of conventionality. However, it was noted that between all
conforming variation templates, the ones where the symbolization mode was in its
least occurring values were the ones which were picked the least number of times as
similar to existing web forum pages.
126
Despite that, this change on the symbolization attribute was not enough to make
participants perceive these conforming templates as less similar to other forums than
the non-conforming screens which had the symbolization mode in its most occurring
value. The results also indicate that the perception of conventionality of the forum
index page and the threads page is affected similarly by similar changes in the
attribute of the elements. The templates chosen the least number of times for the
index page (N14), and the threads page (N23), shared the same values for the three
attributes (symbolization and configuration=least/non occurring values and
position=most occurring). But the least chosen template for the post page (N34) did
not share the same attribute configuration (see Table 16). Considering that these two
levels (index page and threads page) share a more similar layout and have a greater
number of common elements (as can be seen from Figure 54 and Figure 55), when
compared with the post page (Figure 56), it is not surprising that changes in the same
attributes might influence them in a similar manner.
FIGURE 54 - ELEMENTS PRESENT ON THE FORUM INDEX PAGE WHICH ALSO APPEAR IN THE THREADS PAGE.
Logo
Login field
Breadcrumbs
Menu
Forum index
table
Status Last post

indicator
Captions
Time & date
127
FIGURE 55 - ELEMENTS PRESENT ON THE THREADS PAGE WHICH ALSO APPEAR IN THE FORUM INDEX PAGE.
Logo
Login field
Breadcrumbs
Menu
Forum index
table
Status Last post
indicator
Captions
Time & date
FIGURE 56 - ELEMENTS IN THE POST PAGE WHICH ALSO APPEAR IN THE FORUM INDEX AND THREADS PAGE.
Logo
Breadcrumbs Login field

Menu
Time &
date
128
Furthermore, this result could be an indicator that the symbolization and

configuration of an element might have a greater influence on the perception of
conventionality than its position. But when the template which was perceived as the
least conventional in the post page (N34) is examined along with the least picked
templates of the other levels (N14 and N23), the common attribute value to all three
is the symbolization of the elements set in its least occurring value. This indicates
that even more than the configuration of the elements, the symbolization is more
likely to influence user perception of conventionality.
The indication that the symbolization mode of the elements could have more
influence on the perception of conventionality than the position or configuration of
the elements is extremely relevant, since 2/3 of the templates in the non-conforming
group have the symbolization mode in its least occurring value (see Table 16).
However, changing the symbolization mode of web forum elements to their
least/non occurring values means that some elements will be displayed on a web
forum in a purely graphic manner.
This graphic symbolization mode may be quite difficult to decode and comprehend
resulting in problems such as poorer performance, comprehension and higher
cognitive load. This may cause users to consider these forums as less easy to use than
the other ones.
The next experiment will address the impact of the symbolization mode of the
elements on perceptions of ease of use thus, results may show the need for yet
another experiment to be conducted in order to determine if participants are able to
use the elements in the least conventional template without much difficulty.
In summary, the templates which participants chose most and least frequently as
similar to other existing forums are described in Table 20, and indicated in Appendix
I. These are strong candidates to be used in the creation of the test material for future
experiments. But before it is decided which templates will be used to create the
experimental material, another experiment (described in 5.1.2) will be conducted to
test how web forum users perceive the templates in terms of ease of use. The results
of this and the following experiment will be the determinants as to which templates
will be used in the main experiments described in Chapters 6 and 7.
129
TABLE 20 – TEMPLATES MOST FREQUENTLY AND LEAST FREQUENTLY CHOSEN AS SIMILAR TO OTHER WEB FORUMS (RESULTS
ARE OUT OF A TOTAL OF 48 COMPARISONS PER TEMPLATE PER LEVEL).
FORUM INDEX THREADS PAGE POST PAGE

TIMES CHOSEN TIMES CHOSEN TIMES CHOSEN
C11 41
CONFORMING C21 43 C31 48
C13 41
NON-CONFORMING N14 6 N23 3 N34 0
5.1.2. Experiment 2 - User perceptions of ease of use of the

templates based on the survey results
Introduction
As seen in Chapter 2 (see 2.3) genre conventions and usability are linked via external
consistency. Some usability authors claim that not conforming to conventions
established by the majority of websites (i.e. external consistency) can cause major
usability problems (Nielsen, 1999a). Other authors take the contrary position that
violating some expectations of users regarding conventions does not seem to badly
affect usability (Kalbach & Bosenick, 2006; McCarthy, et al., 2003).
Furthermore, user acceptance (willingness to use) of software or interface seems to be

influenced by the perceived ease of use (Morris & Dillon, 1997). If that is the case,
understanding whether users perceive ease of use and conventionality as the same
thing may provide additional knowledge as to how not conforming to genre
conventions can affect users’ acceptance of a website.
This second experiment is a follow-up to experiment 1 and attempts to further

investigate the relationship between conventions and usability. This investigation
will focus on user perceptions of ease of use, which will be considered as indicators
of good apparent usability (Fu & Salvendy, 2002; Tractinsky, 1997). The same
templates from the previous experiment (see 5.1.1) will be used, and participants will
be asked to compare them in terms of ease of use.
130
Objectives
! Examine which of the templates created from the results of the survey and
used in the previous experiment web forum users perceive as easier to use.
! Investigate if there is a correlation between the perceptions of conventionality
and ease of use amongst web forum users.
Methods
Material
The material used for this experiment was basically the same as the one used in
experiment 1. However, unlike experiment 1, this one was conducted entirely online.
This allowed it to be completed in a shorter timeframe, and provided a more efficient
way of recruiting by focusing the recruitment advertising to web forums thus
ensuring that participants were web forum users. For the online experiment to
function it is necessary to adapt the interface so that it can work on different
computer configurations. It is also necessary to devise a way of sending the answers
to the experimenter. It is vital that the test instructions are presented to participants
as a start page for the test.
The solutions were as follows: the interface was adapted to work on the two most
popular Internet browsers: Internet explorer 6 and Mozilla Firefox. To send the
answers, the interface was encapsulated inside an online php form which would
transmit the answers to the experimenter via e-mail. Finally an instruction page was
created and placed before the test.
Experimental design
Here once more the experimental design of experiment 1 was reproduced. However,
all pair orders and the order of each template in each pair were again randomized.
Participants
Participants were recruited by posting an invitation with a link to participate in the
test on several web forums. All forums where the post was placed were written in
English. Posting in English web forums ensured that all participants who responded
to the announcement were web forum users, and had a minimum level of language
understanding to perform the test. Although less was known about the participants
in this experiment than in the previous one, the sample was probably more varied
since participants could come from any part of the world. Because the characteristic
which was more likely to influence the outcome of the test were the familiarity of
131
users with forums (and this was guaranteed by the way in which participants was
recruited) the lack of further information on participants does not constitute a major
problem. Furthermore, the more varied sample improves the generalization of the
results.
In total 20 participants volunteered to do the test and submitted their results via
email to the experimenter. The emails contained data from the test and the number of
the participant. When the number reached 20 (which was the established number of
participants for this test) the test was taken offline. Participants in this experiment
have not taken part in any previous experiments conducted for this thesis.
Procedure
The experiment instructions and how to navigate the interface were displayed to the
participants. Participants were informed that they were going to see 48 pairs of
screens and after each pair they would be asked to choose one page layout they
thought would be easier to use. At the end of the instructions participants were told
to press a key to start the test. By doing so, participants were taken to the first pair of
templates.
As in experiment 1, participants were allowed to scroll up and down (and left to right
when necessary) on each template as they normally would when browsing the Web.
After looking at both templates in a pair, a white background screen was displayed
saying that one pair had ended and asking participants to choose which of the two
templates they thought looked easier to use.
After the 48 pairs of templates were compared an email was sent to the experimenter
with the participant number and the answers. Then a screen thanking the
participants for taking part in the test was displayed.
Results
Of the 960 times (48 pairs presented to 20 participants) that each pair of templates
was displayed during the experiment, the conforming templates were chosen 822
times (85.63%) over the non-conforming ones as easier to use. The conforming
samples included not only the conforming templates where the symbolization,
configuration and position attributes assumed their most occurring values, but also
the variations around the conforming pages.
132
In 138 (14.37%) cases the non-conforming sites were thought to be easier to use than
the conforming templates. Overall the conforming templates were seen by
participants as easier to use than their non-conforming counterparts. The total
number of times the templates of each level were compared was 320 times per level.
Again looking at each level, the conforming templates were chosen more frequently
as easier to use than the non-conforming templates. In the forum index (level 1) the
conforming templates were chosen 286 times as easier to use than the non-
conforming templates. The non-conforming templates were chosen 34 times as easier
to use.
In the threads page (level 2) the conforming templates were chosen as easier to use
than their non-conforming counterparts 275 times, while the non-conforming
templates were picked only 45 times.
Finally, in the post pages, the conforming templates were again picked more
frequently as easier to use than the non-conforming template. The conforming
templates were chosen 261 times against 59 times that the non-conforming were
picked.
A Chi-square test was applied to the data and the results are summarized in Table 21.
TABLE 21 - SUMMARY OF THE CHI-SQUARE TESTS ON THE NUMBER OF TIMES EACH GROUP WAS PICKED AS EASIER TO USE.
Group
Conforming Non-conforming P
Forum index (level 1) 286 34 <0.01
Threads page (level 2) 275 45 <0.01
Post page (level 3) 261 59 <0.01
Total 822 138 <0.01
The non-conforming templates which were picked more frequently as easier to use
when compared to their conforming counterparts were N12 in the forum index level
(13 times), N22 in the threads page level (18 times), and N32 in the post page level (27
times).
The non-conforming templates which were least often chosen as easier to use when
compared to a conforming template were N14 in the forum index level (4 times), N23
in the threads page level (6 times), and N34 in the post page level (5 times).
133
When it comes to the templates in the conforming group the results are generally
much closer than the results from the non-conforming group with not much
difference between the first and second most chosen templates in each forum level.
The frequency that each template in the conforming group was chosen as the easier
to use when compared to a non-conforming template is shown in Table 22.
TABLE 22 - FREQUENCIES THAT EACH TEMPLATE IN THE CONFORMING

GROUP WAS CHOSEN AS EASIER TO USE (RESULTS ARE OUT OF A TOTAL OF
80 COMPARISONS PER TEMPLATE PER LEVEL).

C11 72 C21 74 C31 70
C12 66 C22 71 C32 65
C13 71 C23 61 C33 58
C14 76 C24 69 C34 69
The number of times the most frequently and least frequently chosen templates were
picked as easier to use is shown in Table 23.
TABLE 23 – FREQUENCIES OF THE TEMPLATES MOST FREQUENTLY AND LEAST FREQUENTLY CHOSEN AS SIMILAR TO OTHER
WEB FORUMS (RESULTS ARE OUT OF A TOTAL OF 80 COMPARISONS PER TEMPLATE PER LEVEL).
Forum index Threads page Post page

Times chosen Times chosen Times chosen
Conforming C14 76 C21 74 C31 70
Non-conforming N14 4 N23 6 N34 5
Discussion
Overall, the conforming group templates were more frequently chosen as easier to
use than the non-conforming group ones. Below a correlation test is applied to the
results of both this and the previous study. Here, it suffices to say that participants
considered the templates of the conforming group easier to use.
In this experiment, some non-conforming templates were chosen as easier to use

when compared to some conforming templates. Although the number of times these
templates were chosen is much smaller than that of the conforming templates, some
interesting observations can still be made looking at which templates were chosen by
participants.
134
All of the non-conforming templates more frequently chosen as easier to use had the
position and configuration of their elements in their least occurring values, while
their symbolization was set in its most occurring value. This finding reinforces the
idea from the previous experiment that the symbolization of the elements on a page
might be quite influential not only on the user perception of the conventionality of
the page, but also on their perceptions regarding the ease of use of that page.
The non-conforming templates which were least chosen in this experiment

(experiment 2) were the same ones that were least often chosen as similar to other
existing forums in experiment 1 (N14, N23 and N34). This seems to suggest that
participants might perceive non-conforming layouts as harder to use or vice versa.
The results from these two experiments show that there seems to be a relationship
between what web forum users perceive as conventional, and what they perceive as
easy to use in a forum interface. A Spearman-& test showed a strong correlation
(r=0.83) between the perceived conventionality and the perceived ease of use of the
templates on both experiments.
5.1.3. Conclusion
The correlation found between the two studies seems to indicate that users who are
familiar with web forums seem to find web forum interfaces which follow the visual
conventions of the genre also easier to use.
The analysis of the experimental results also seem to indicate that the conventionality
parameters gathered in the survey are a fair representation of what users expect a
web forum to look like, and what they consider as easier to use.
Future studies will consist of getting users to search for information in a conforming
forum and a non-conforming forum. To define the layout of these pages the results
from these two experiments discussed above will be taken into account.
To create a forum which violates the established visual conventions, the templates
which were chosen the least number of times as more like other web forums and least
number of times as easier to use in each study will be used. These templates (N14,
N23 and N34) were perceived by users as those least visually similar to other web
forums they previously used and also as being less easy to use than other layouts.
135
However, choosing templates to be used to create a conforming forum is not so easy.

The reason for that resides in the fact that there was no complete agreement between
the two experiments regarding the most frequently chosen templates. For the threads
page and the post page, the same templates were chosen in the first and second
experiment. But in the forum index page, there was no complete agreement: C11 and
C13 were equally considered as the templates which looked more like other web
forums, and C14 was the template users thought was easier to use (see Table 24).
However the conforming template which was the second choice for users in terms of
ease of use was C11, which was chosen 72 times. The frequency difference between
the first choice (C14) and the second choice (C11) is not large enough to suggest users
would feel that the latter would be more difficult to use than the former.
TABLE 24 - TEMPLATES MOST FREQUENTLY CHOSEN IN THE CONFORMING GROUP.
TEMPLATES FORUM INDEX THREADS PAGE POST PAGE

TIMES CHOSEN TIMES CHOSEN TIMES CHOSEN
MORE LIKE OTHER C11 41 (85.4%)

C21 43 (89.6%) C31 48 (100%)
WEB FORUMS C13 41 (85.4%)
C14 76 (95%)
EASIER TO USE C21 74 (92.5%) C31 70 (87.5%)
C11 72 (90%)
Thus for the purpose of creating a conforming web forum to be used in future
experiments it seems appropriate that C11 be used instead of the other most chosen
templates C13 and C14. One reason is that the frequency difference between C11 and
C13 in the first experiment was non-existent, and the difference between C11 and C14
in the second experiment is not big enough to justify the use of C14 instead of C11.
Furthermore, by using C11, the whole web forum would benefit from consistency
since the templates chosen for the other two levels had the same values for the three
element attributes as C11. All three templates have the symbolization, position and
configuration of their elements in their most occurring values. The templates which
were chosen the least number of times as more like other web forums and considered
the least easy to use (N14, N23 and N34) all have symbolization at its least occurring
value. Both the conforming and non-conforming templates seem to confirm that
symbolization has a considerable influence on user perceptions of conventionality
and ease of use.
The templates which will be used to design the web forums for the upcoming
experiments are shown in Table 25 and are indicated in Appendix I.
136
TABLE 25 – TEMPLATES WITH THE MOST FREQUENT (MF) AND LEAST FREQUENT (LF) VALUES FOR EACH ELEMENT
ATTRIBUTE AND WHICH WERE USED TO DESIGN THE CONFORMING AND NON-CONFORMING FORUMS.
Conforming Symbolization Position Configuration Template
Forum index MF MF MF C11
Threads page MF MF MF C21
Post page MF MF MF C31
Non-conforming Symbolization Position Configuration Template
Forum index LF MF LF N14
Threads page LF MF LF N23
Post page LF LF MF N34
5.2. Experiment 3 - Assessing the experimental material

5.2.1. Introduction
From the results of the previous experiment it is possible to see that the
symbolization mode of the elements plays an important role in user perceptions of
ease of use. The templates which will be used to create the non-conforming material
were the ones perceived least often as easy to use by participants. One of the reasons
for these templates being perceived as not as easy to use as the others could be due to
the considerable number of elements which are presented as graphic images.
There is a possibility that user perceptions may be converted into usage difficulties
when using the interface. Thus, an indication of how hard it is for users to identify
the elements which are presented in this manner is necessary. Furthermore, it is
important to test not only the ability to identify the elements but also the ability to
recognize the main interactive elements of a forum. By doing so it will be possible to
have an idea about how hard or easy it is for forum participants to use the non-
conforming interface. Extreme difficulties with interpreting the graphic elements or
the interactive elements of the non-conforming forum might indicate that
participants find it difficult to use the interface. On the other hand, if participants
have no difficulties in identifying the graphic elements or the interactive ones, this
137
might suggest that there will be no complications when using the non-conforming
forum.
Many elements which are textual in the conforming forum are represented
graphically in the non-conforming forum by the use of icons. Literature divides icons
into three categories: abstract (composed of geometric marks and shapes),
representational (simple pictures of familiar objects and operations) and semi-
abstract (formed by combining both abstract and representational images) (Blattner,
Sumikawa, & Greenberg, 1989). Representational icons are believed to be easier to
interpret than the abstract and the semi-abstract ones (Blattner, et al., 1989;
McDougall & Curry, 2004; Moyes & Jordan, 1993, p. 50). However, differences in
accuracy when identifying representational versus abstract icons tend to disappear as
the user acquires more experience with them (McDougall, Curry, & De Brujin, 1999).
This seems to indicate that, as expected, icon identification improves according to
familiarity with the icon. This familiarity can be understood as either familiarity with
the object depicted in the icon or as experience with the icon in an interface
(McDougall & Curry, 2004).
Grouping icons together (in clusters) is also believed to allow users to learn icon-
function relationships more quickly (McDougall & Curry, 2004). Clustering icons
allows inferences about categories of icon-function relationships to be made and this,
in turn, helps users interpret the meaning of the icons in the groups.
There is another important variable to icon identification which was not considered
above that is, position. It seems that when identifying an icon, users first look at its
shape and if that is not sufficient for identification, they use the position of the icon
on the interface to help identify it (Moyes, 1994). This assumes that the user is
familiar with the placement of the icon and expects to find it in a determined position
on screen. Research carried out by Moyes (1995) shows that contextual information,
especially the position on the icon on the interface, is extremely relevant in
identifying abstract icons, whereas the form of the icon is more important when users
are faced with a representational icon.
As the non-conforming interface contains both representational and abstracts icons it

is possible that the place where the icon is located might be relevant for identifying
the abstract ones. However, unlike the conforming forum, the non-conforming
(designed from the templates which participants chose least often as conventional
138
and easy to use) are not internally consistent. Thus, in the non-conforming forums
the elements are positioned in their most common place except for the post page (the
third level) of the forum, where they are placed in their least common position. So,
the lack of consistency on the positioning of the icons in the non-conforming forum
may deprive users of using this contextual cue to help identify the icons. But it is also
plausible to assume that to get to the post page (third level) of the forum participants
are likely to go through the other two levels and that by then they may have inferred
the meaning of the icons from their position on these pages.
5.2.2. Objectives
This study aims at further examining the experimental material generated based on
the templates selected from the two previous experiments. The results obtained will
serve to inform future experimental designs and the design of tasks. More
specifically, this experiment has three main objectives:
! Verify if web forum users can identify the icons which will be used to replace
the textual elements in the non-conforming forum. It will give information as
to which icons are easier/hard to identify in the non-conforming interface so
that tasks can be designed avoiding the use of elements which participants
found hard to identify.
! Obtain perceptions of the participants regarding the conventionality of a

finished page of the conforming and non-conforming forums. This will
indicate that the experimental material is adequate and will further validate
previous results.
! Investigate how well web forum users can recognize the interactive forum
elements (e.g. menu, login area, index table) in the non-conforming template.
This will give some indication as to how difficult it is for participants to
locate the information within the forum and which interactive elements are
harder to identify. Then tasks can be designed to avoid elements which are
harder to identify.
5.2.3. Method
The survey was divided into three distinct sections, each devised to assess one of the
objectives. The first section investigates the first objective; the second section
investigates the third objective and the third section, the second objective.
139
All the text in the forum templates was replaced with placeholder text (‘Lorem
ipsum’) so that participants could not use the content to help them throughout the
test.
Due to the nature of the survey and the fact that it needed participants that were
familiar with web forums, the simplest solution was to devise an online
questionnaire that people would be able to access from wherever they were. Then, by
placing links to the test on different web forums the chances that participants were
familiar with web forums would highly increase.
Section 1
On the templates chosen to generate experimental material there are thirteen
elements which are textual in the conforming forum, and graphic in the non-
conforming version. This happened because the symbolization mode of the elements
was changed from its most frequent value (conforming) to the non occurring value
(non-conforming). This means that these particular elements are not presented
graphically on web forums. In the design of the experimental material these thirteen
elements are presented as graphic icons.
The thirteen icons are never used on web forums so they were obtained from icon
databases available on the Internet. For this survey, the icons were positioned
vertically on a web page and participants were given empty fields next to each icon
to fill in what they thought the icon represented (Figure 57). Icons were displayed out
of context in order to keep participants from being exposed to the forum interface in
the first section and this could influence the last part of the test where participants
had to identify the interactive elements of the forum.
140
FIGURE 57 - SAMPLE OF SECTION 1 OF THE SURVEY.
The icons were presented to all participants in the same order. Table 26 shows the
icons that were presented to the participants and the textual content that they were
replacing.
TABLE 26 - ICONS PRESENTED IN THE SURVEY AND THE TEXT CONTENT THEY WERE REPLACING.
Icon 1 Icon 2 Icon 3 Icon 4 Icon 5 Icon 6 Icon 7
FAQ Password Register Today’s posts Members List Calendar User ID
Icon 8 Icon 9 Icon 10 Icon 11 Icon 12 Icon 13
Time and date Thread starter Last post Threads Replies Views
141
Section 2
This section investigated the second objective which is to check the perceptions of
web forum users regarding the conforming and non-conforming forum screens
created from the templates chosen in 5.1.3.
Since section was designed as a quick check on the perceptions of conventionality of

the users regarding two finished experimental screens (one conforming and one non-
conforming) only one screen of one level of each forum was tested. The level chosen
to be displayed to users was the forum index page for two reasons (a) it is the first
screen users will see when they start the test and (b) in a conforming forum the index
page and the threads page share a similar design (see page 127). So the experimental
material based on the templates C11 for the conforming forum, and N14 for the non-
conforming forum were used.
This section consisted of two separate screens; one displayed the conforming page
and the other displayed the non-conforming page (Figure 58). At the bottom of each
screen was a question which asked participants to rate how the page they were
looking at compared with web forum pages they had seen before. The rating was
done through a Likert type scale ranging from 1 to 4 in increments of 1 where the
value 1 stood for ‘Very different’ and 4 to ‘Very similar’.
142
FIGURE 58 - SAMPLE OF THE NON-CONFORMING SCREEN IN SECTION 2 OF THE SURVEY.
To keep the order in which the two templates were displayed from affecting the
results, half of the participants were shown the non-conforming screen first and the
other half were shown the conforming screen first.
Section 3
This section investigated if participants are able to identify the elements which they
would have to interact with while using the non-conforming forum page. If they can
do this without difficulty then it is likely that they will be able to know where to
retrieve the information they require when performing future tests. The ability to
identify interactive elements of the non-conforming forum can indicate the difficulty
participants might have when using the non-conforming template in comparison
with the conforming template.
143
In this section of the survey, an image of the index page of a non-conforming forum
page was displayed. The relevant elements were circled and numbered 1 to 6 (Figure
59). Below the forum image, input fields corresponding to the key areas were shown,
and a text asked participants to write what they thought these parts of the forum
were. The names of each element which was to be identified are shown in Table 27.
TABLE 27 – CORRECT ANSWERS FOR THE AREAS HIGHLIGHTED IN THE THIRD PART OF THE SURVEY.
Area 1 Login
Area 2 Menu
Area 3 Thread/Topic / Post area
Area 4 Thread/topic / post
Area 5 Key or legend
Area 6 Time and date
FIGURE 59 - DIVISION OF AREAS IN THE NON-CONFORMING FORUM AS DISPLAYED

IN THE THIRD PART OF THE SURVEY.
144
Participants
The questionnaire was made available online by postings on 8 web discussion
forums. The posts contained requests asking people to participate with a hyperlink to
the test page. All forums chosen for posting had English as their main language and
the discussion topics ranged from old books to postgraduate research. This ensured
that participants were not only web forum users but also had some fluency in English
and had different backgrounds. Most forums were not based in the UK and the ones
which were based in the United Kingdom, were not related to the University of
Reading These measures were put in place to minimize the possibility that
participants in this experiment had already participated in other experiments in this
thesis.
The answers were sent through e-mail to the experimenter. A total of 34 web forum
users participated in the survey.
5.2.4. Procedure
By clicking on the link provided on the post on the web forums, participants were
redirected to a page containing Section 1 of the survey. Here participants were asked
to consider the text elements which appear in a web forum page and associate them
with one of the icons displayed by writing the name of the element in the field next to
the icon image. Upon completion of this part of the survey participants clicked on a
‘continue’ button and were sent to a page thanking them for answering the first part
of the survey and informing them that whenever they felt ready they should click on
a link placed below the text to move on to the second part of the test.
Clicking on the link directed participants to the second part of the test. As mentioned
before, the second part consisted of a comparison between both the conforming and
non-conforming templates with existing web forum screen participants had seen
before. Each template was shown (the order depending on the counterbalancing)
together with its Likert type scale. After participants had rated the first template, the
second template was displayed with another Likert type scale. The third and final
part of the test consisted of the non-conforming template screen with 6 key areas
highlighted. Participants were then asked to fill in what they thought each area was
for in the forum. When completing this final part a screen thanking participants for
taking part in the test was shown.
145
5.2.5. Results
The average time that participants took to complete the test was 5 minutes and 39
seconds. The fastest participant took approximately 2 minutes to complete the test
and the slowest took 25 minutes.
Section 1
The results for the first part of the survey are very diverse with some icons being
easily identified and others not being identified at all. Because participants were free
to write what they wanted for every icon, the words they used to express an idea
were not always the same (e.g. identification, identity and user id were used to
describe the ‘User ID’ icon), so for the purpose of analysis words were grouped into
similar concepts and the most frequent results are shown on Table 28. A complete
list of all the entries and their frequency count can be found in Appendix J.
TABLE 28 - MOST FREQUENT ICON ASSOCIATIONS AND FREQUENCY (ANSWERS CONFORMING

TO THE MEANING OF THE ICON ARE ITALICIZED).
Icon1 Icon2 Icon3 Icon4 Icon5

1
Help 23 Password 14 Write new post 19 Calendar 1
1 Messenger
FAQ 9 Security 8 Reply 4 Dates 4
2 Member’s list
Get info 1 Lock 3 Register 0 Today’s post 5
0
Icon6 Icon7 Icon8 Icon9 Icon10

Support
Calendar 32 User profile 17 Time & date 12 9 Last post 5
VOIP
Gallery 1 User ID 5 Appointment 3 6 Next page 3
Thread
Archives 1 Video 2 Events 2 0 Go to 3
starter
Icon11 Icon12
Icon13
Thread 7 Post/Reply 7 Search 15
Cotton reel 1 Quote 3 View 2
Tailor 1 Comments 2 Zoom 2
Section 2
In this part both the conforming and non-conforming pages were assessed for their
similarity with other forum pages that participants had used before. Twenty-seven
146
participants considered the conforming page more similar to other forums than the
non-conforming one. Only one participant thought that the non-conforming page
was more like other web forums than the conforming. Six participants found no
differences between the layout of the conforming and non-conforming pages in
relation to existing web forums (i.e. attributed both pages the same value on the
scale).
Overall, the conforming page was thought of as more like other web forum pages
with a median of 3.5 points on the scale and the non-conforming templates was
thought of as different from other web forum pages with a median of 2.0 points in
the scale. A Wilcoxon signed rank test showed that the difference between the two
scores was significant (W=396.0, n=28, p<0.001).
Section 3
As happened in Section 1, in this Section participants were allowed to write as they
pleased therefore some answers expressed the same concept with different wording.
So a similar procedure as the one used to group the answers in Section 1 was used
here. The grouped results can be seen in Table 29 and a complete table with the
ungrouped results can be seen in Appendix K.
TABLE 29 - GROUPED RESULTS FOR THE IDENTIFICATION OF THE AREAS OF THE NON-CONFORMING TEMPLATE.
Area 1 (login field) Area 2 (Menu) Area 3 (Threads table)

Login 29 Menu 12 List of threads 22
Don’t know 2 Don’t know 3 Main content 3
Entry 1 Access/action 3 Don’t know 3
Search 1 User details 3 Entry box 2
User function 1 toolbar 3 Background 1
navigation 2 Conversation 1
information 2 Page number 1
links 2 Search result 1
other 4
Area 4 (Thread) Area 5 (Key/legend) Area 6 (Time and date)

Thread/topic 24 Don’t know 8 Time & date 23
Don’t know 3 Key / legend 6 Don’t know 5
link 2 Online users 5 Save 2
Thread listing 2 Change colour 3 Copyright 1
search result 1 Copyright 1 Close 1
criterion 1 Footer text 1 Login 1
Forum status 1 Stats 1
Guide to new posts 1
links 1
Meta data 1
Site details 1
Permissions 1
147
5.2.6. Discussion
Section 1 – Icons
The results from the icon identification section were not clear–cut, with some icons
being associated with the expected web forum textual elements, while others not at
all. Although participants were informed that they had to associate the icons with
web forum elements, results show that some icons were associated with certain
software (e.g. icon 5 with MSN or icon 9 with Skype) which are not part of web
forums. Although all icons came from icon libraries available online, it is possible
that their shape resembles that of the icon from the software cited. Other icons were
associated with features which are not present in web forums (e.g. icon 9 with Voice
over IP, icon 6 with gallery). It is likely that because these icons are not commonly
present in web forums that the associations which participants might have for them
are related to previous experiences with similar icons in other software or interface.
This could explain why even when asked to relate the icons to a forum, participants
kept associating them with meanings they were familiar with.
The icons that were most correctly identified were: icon 2 (Password), icon 6
(calendar), icon 8 (time & date) and icon 12 (reply). Icon 4 (Today’s Post), icon 3
(register) and 9 (thread starter) were not correctly identified by any participant. A
large number of participants incorrectly identified other icons: icon 1 (FAQ), icon 5
(member’s list), icon 7 (user ID), icon 10 (last post), and icon 11 (thread). However,
some participants did get the right meaning for these icons with the frequency of
correct textual elements being cited staying between the most and the least cited
words. This mixed result could be explained by the fact that the icons were presented
out of context so participants had trouble identifying them (Moyes, 1995).
So it is likely that when placed in context and in position within the interface,
participants will be able to better identify the icons. However, in the following
experiments tasks shall be designed to require that participants only interact with the
icons that had a high frequency of correct identification.
Section 2 – Conventionality of the templates

The conforming forum page was perceived as the one most similar to other web
forum interfaces participants have used. The fact that participants were recruited
from 8 different web forums strengthens the conviction that the conforming
templates chosen in the previous experiments faithfully represent what is perceived
148
by web forum users as a convention-conforming web forum design. The non-

conforming template was rated by most participants as quite different from the web
forum interfaces they were accustomed to seeing. This also reinforces the idea that
the templates violate some existing visual conventions for web forums and
consequently will suit the purposes of the future experiments.
Part 3 – Identification of interactive elements of the forum in the non-

conforming template
As mentioned previously, this section was intended to verify if users who were
familiar with web forums would be able to identify the interactive elements of a
forum on the non-conforming template. The results show that all interactive elements
had high rates of identification and the key/legend element, although having a high
number of people who could not identify it at all (8 participants), had 6 participants
identifying it for what it was. This poor identification rate might be explained by the
fact that in a conforming web forum, the key/legend element usually has graphic
and textual components whereas in the non-conforming it is completely textual.
Overall, identification of the interactive elements on the non-conforming page was

fairly straightforward. The fact that participants were able to identify the interactive
elements of the interface (e.g. menu) may provide some form of contextual cues
which may help them to infer meaning to the icons present in the interface (e.g. FAQ
icon, Calendar icon) more easily.
5.2.7. Conclusion
The results of this experiment (experiment 3) show a clear distinction in the
perception of the participants regarding the conventionality of the templates.
However it indicates that participants have no great difficulties in working out what
the interactive elements of a non-conforming web forum are. The icons have
presented some difficulties when it comes to relating them to textual elements
present in web forums. However, because the icons were presented out of context
and not clustered, no extra cues (such as positioning and grouping) were given to
participants to help them on the identification process.
The interactive elements on a web forum page were easily identified by the
participants. It seems that users familiar with web forums might be able to overcome
some usage difficulties in the non-conforming forum by inferring and interpreting
149
the interface based on their existing knowledge of forums. As mentioned before, it is

possible that that the icon identification problems might be minimized when these
are placed in context within the interface.
Finally, this experiment indicates that the templates selected to build the material for
the next experiments are good, valid representations of what a forum does and does
not look like. The conforming template is in conformity with what users expect a
forum page to look like. Users did not perceive the non-conforming template as
similar to other existing web forums which does indicate that, at least to some extent,
they violate what users expect to see in a web forum page layout. It also shows that
any difficulties participants may have regarding the interpretation of the icons and
recognition of the interactive elements in a web forum are not difficult to overcome.
5.3. Final remarks

Users perceive non-conforming interfaces as harder to use than ones that follow
conventions. However, results from the last experiment which asked participants to
identify the interactive elements in a non-conforming forum page indicate that this
difficulty might not be as significant as users may believe.
As seen before, there seems to be a connection between the identification of icons in

an interface and the level of familiarity with it (McDougall & Curry, 2004). Even
though literature highlights the importance of positioning to icon identification, and
the results of the first two experiments shown that symbolization of elements has an
important effect on user perceptions of how easy an interface is to use, no research
has yet fully investigated the impact which changing the position, symbolization and
configuration of interface elements has on the performance of users who are familiar
with a specific interface.
Research points out (McDougall, et al., 1999), that by using a certain interface for
some time, users will be familiarized with the icons and identification performance
will no longer suffer. It is possible that after using an interface for some time users
will become familiar with not just the icons it contains, but the whole interface.
However, it is possible that the time necessary for this familiarization to happen may
be too long.
150
The next Chapter describes an experiment which investigates this matter further, by
looking at what happens with the performance of both users who are familiar and
users who are unfamiliar with web forums when using a conforming and a non-
conforming web forum.
151
Chapter 6
Visual conventions and familiarity when
searching in web forums
6.1. Introduction
The exploratory study conducted earlier (see Chapter 3, section 3.3) showed that
some web page layouts are quickly recognized by users. Users are then able to
predict what type of content will be found within the page. As this thesis aims at
investigating the effect of conforming to and not conforming to visual conventions on
the time and accuracy to complete tasks by web users, a survey was conducted
(Chapter 4) in an attempt to establish what is conventional for a particular category
of website. The survey listed the most frequently occurring elements within a specific
category of websites: the web forums. The results of the survey were then used to
create several templates which varied their degree of conventionality. Web forum
users were then shown the templates and asked to compare these in terms of their
similarity with other existing web forums and their perceived ease of use (see
Chapter 5). The outcome of this experiment was a rating of templates ranging from
those considered more similar to existing web forums and those considered least
similar to existing web forums and those considered easy to use and those
considered not so easy to use.
This Chapter continues the study of visual conventions. It describes an experiment

designed to investigate how not conforming to visual conventions of a web forum
affects the performance of users who are familiar and users who are unfamiliar with
the conventions. The experimental material used here was based on the results
obtained in the survey and experiments as reported in Chapter 5.
6.1.1. Familiarity
The Concise Oxford English Dictionary defines familiar as something ‘well known
from long or close association’. From this definition it is possible to establish that for
152
Visual conventions and familiarity when searching in web forums
familiarity with a stimulus to be achieved it requires prior contact with the stimulus
for a certain period of time. From this definition it is also possible to directly associate
familiarity with genres. As seen in Chapter 2, section 2.2.5 genres emerge from the
communicative needs of a discourse community. This discourse community becomes
familiar with particularities of the genre and comes to expect these to be followed by
each document belonging to that genre.
But it seems that prior experience with a stimulus is not the only requirement to
breed familiarity. Although some authors do indeed point out that prior experience
with a stimulus can produce a feeling of familiarity (Whittlesea, 1993; Zhang &
Ghorbani, 2004), prior experience is not the only factor which influences the feeling
of familiarity. According to Zhang and Ghorbani(2004), the following factors also
contribute:
! Repeated exposure – the feeling of familiarity increases with repeated
exposure to a stimulus.
! Level of processing - associated with the amount of familiarity which is
obtained. The deeper the processing of the stimulus, the greater the feeling of
familiarity achieved. Deep processing (processing meaning) results in an
increase in familiarity when compared with shallow processing (processing
perceptual aspects).
! Study duration – longer times spent studying something lead to an increase
in familiarity. For web navigation purposes the authors define study duration
as the period of time a user stays navigating and using a website.
! Forgetting rate - defined by the interval of time between two consecutive
exposures to a stimulus. In terms of web navigation the exposures can be
thought of as visits to one website. The longer the time interval between
exposures , the greater the decrease in the feeling of familiarity (Zhang &
Ghorbani, 2004).
An experiment conducted by Gefen (2000), where 84 web users answered questions

on familiarity and website trust, shows that in the World Wide Web, familiarity is an
important component in building user trust towards a website. Apparently
familiarity is not only important for creating trust in users but, in fact some authors
share the opinion that web users have extreme desire for familiarity (Usborne, 2003;
Zhang & Ghorbani, 2004).
153
Familiarity, however, is a fairly broad concept and a person can be familiar with
different aspects of different things. The fact that research has shown that one kind
of familiarity can be affected by other types of familiarity (Lim, Benbasat, & Todd,
1996) makes it important to look at other types of familiarity in order to understand
how familiarity with a genre can be achieved. Thus a brief review of the types of
familiarity relevant to this study is outlined below.
Nielsen (1993, p.43) has subdivided user experience into three categories: experience
with the system, experience with computers and experience with the task domain.
Since familiarity comes from experience (Zhang & Ghorbani, 2004), after acquiring
experience with each of the aspects outlined above, it is possible to say that a user
achieves: familiarity with the system, familiarity with computers and familiarity with
the task domain.
Although this division seems quite useful, at least in terms of navigating the Web
users can be familiar with a certain task type but be unfamiliar with the topic being
addressed in the task (Capra, 2006, p. 3). A good example would be a user searching
for financial information online: the user can be extremely familiar with searching for
information online but have no familiarity with the financial topic. Yet another type
of familiarity which should be considered is familiarity with a text genre. A reader
can be familiar with a text genre such as academic papers, but be ignorant of a
particular topic addressed in one specific paper they are reading. Therefore, a more
adequate subdivision of familiarity for WWW users which covers a broader range of
research done in the area is proposed below:
! Familiarity with the topic.
! Familiarity with the text genre.
! Familiarity with the task.
! Familiarity with the interface (or system).
! Familiarity with computers.
All these types of familiarity are somehow linked with genres and in order to become
familiar with a genre a user has to achieve some degree of each type of familiarity.
However, mapping these types of familiarity to the conventions of a genre is
extremely complex since there is much overlapping and some types of familiarity are
linked with different genre conventions.
154
For the purposes of this thesis only the first four types of familiarity will be
addressed since the type of users being tested will all be familiar with computers.
Familiarity with a specific interface is sometimes referred to as user expertise and

users are usually considered novices, experts or somewhere in-between (Nielsen,
1993). Users who are new to an interface (and in a more specific case to a website) are
novices and users with intensive use of the interface are considered experts. The
problem with this classification starts when one tries to place within the novice-
expert interval users who are not novices but have not yet reached expertise status.
Those are usually referred to as intermediate users, but their degree of familiarity
could be situated closer to the novice or closer to the expert status and this could
make a difference in the outcomes of an experiment.
It seems that the distinction between the expert users and those who are not novices
but also not experienced enough to be expert can be quite hard to establish. In fact in
a study which attempted to identify success on web search tasks of experts,
intermediate users and novices using different strategies, Tabatabai and Shore (2005)
found little difference between experts and intermediate users. The authors attribute
the small difference in performance to the fact that intermediate users had Internet
search training just before the test started while experts did not. An alternative
explanation is that the learning curve for the interface in question showed a steep
incline in the initial part and carried on more horizontally from then on (Nielsen,
1993). An example of this situation is illustrated in Figure 60 where the difference in
performance (vertical axis) is quite small between intermediate and expert users
when compared to the novices.
155
FIGURE 60 - SAMPLE OF A HYPOTHETICAL LEARNING CURVE TO EXPLAIN THE SMALL

DIFFERENCE IN PERFORMANCE BETWEEEN EXPERTS AND INTERMEDIATE
USERS FOUND IN TABATABAI AND SHORE (2005).
Either way, this study illustrates the difficulty in defining where the boundary
between experts and intermediate users should be. Differences in performance
between intermediate and expert users can be quite hard to obtain. For the next
experiment in the thesis, intermediate and expert users will be grouped together as
users who are familiar with web forums. By allocating users who have used web
forums before (intermediate and experts) in one group, and users who have never
used a forum before (novices), the differences between the two groups are
maximized.
Familiarity with a topic

Research has shown that familiarity with a certain topic can increase online search
efficacy (Hirsch, 1997; Kelly & Cool, 2002) and it can also improve performance when
searching online for information on a familiar topic (Lazonder, Biemans, & Wopereis,
2000; McDonald & Stevenson, 1998; Patel, Drury, & Shalin, 1998).
When users who are familiar with a topic use a familiar interface, a synergistic effect
seem to occur and these users change their search strategy (Jenkins, et al., 2003). They
tend to explore the content of the site further, going deeper within the structure of
the website (Holscher & Strube, 2000; Jenkins, et al., 2003). This illustrates the
importance of taking familiarity with the search topic into account when designing
156
the material for the next experiment. Having a topic with a high probability of users
being familiar with can make some participants perform in an extraordinary fashion
which could compromise the validity of the results. So, topic familiarity will be
minimized by choosing a search topic participants are not familiar with. This will
keep user performance from being improved by a participant already knowing about
a topic. An extreme example of this situation would be a participant knowing the
answer to the task without having to search the forum.
Familiarity with text genre

Familiarity with a text genre helps familiar users locate themselves within the text
structure as shown by Dillon and Schaap (1996). In this study, the experimenters
asked users familiar and unfamiliar with reading scientific journals to place
paragraphs extracted from journals in the appropriate section (introduction, method,
results or discussion). All paragraphs were stripped of explicit structural cues (such
as headings) and half of them had other embedded cues (such as references to the
section it belonged to, references to literature, formulae and acronyms of tests or
research tools) removed. The authors concluded that users familiar with the text
genre rely on the semantic details to locate themselves in the information space.
Commenting on the results of this experiment, Dillon(2000) added that unfamiliar
users, however, have to depend on more explicit spatial indicators such as headings
and titles on the screen.
Similarly to scientific journals, web discussion forums do have a particular writing

style (see Chapter 5) and linguistic conventions which forum users become familiar
with. So if the results from Dillon and Schaap (1996) hold true to any hypertext and
are extendable to user interfaces, it is possible that in the next experiment, the group
of unfamiliar users (i.e. unfamiliar with the interface) might perform better in the
conforming forum than in the non-conforming one. Since users in all the groups will
be unfamiliar with the topic and unfamiliar users will not have familiarity with the
interface either, they will have to rely on explicit cues in order to locate themselves.
Because many cues which are textual in the conforming forum will be graphic- in the
non-conforming, unfamiliar users tested on the latter are likely to have their sense of
location impaired.
157
Familiarity with the task

Familiarity with the task has also been associated with improved performance
(Marchionini, 1995), and successful retrieval (Marchionini & Shneiderman, 1988)
when searching for information in an electronic environment.
Familiar tasks are more likely to be executed through automatic processing whereas
unfamiliar tasks are executed via controlled processing (Lim, et al., 1996). While
automatic processing is fast, effortless and unconscious (Logan, 1988), controlled
processing is time-consuming, effortful and performed in a step-by-step manner
(Lim, et al., 1996).
Task familiarity is achieved mainly through the frequency with which a user
performs a task (Capra, 2006, p.129). A direct implication of this is that users who are
accessing a certain website with some regularity are more likely to be familiar with
the type of tasks which are commonly executed when using this website.
This seems to suggest a connection between task familiarity and interface familiarity.
In fact, some believe that when using computer interfaces such as a website, the
presence of familiar elements may trigger automatic processing of tasks (Lim, et al.,
1996). This could mean that members of a discourse community who are familiar
with a certain web genre and more specifically its interface and tasks, when faced
with a website interface they are not familiar with (i.e. does not conform with the
genre) they will process familiar tasks in a controlled manner. In other words,
because the unfamiliar interface does not have elements which are familiar to the
user, it can not trigger the automatic processing of the tasks. As a result users would
perform tasks in a controlled manner resulting in slower performance. This concept
relates to the idea that some cues may trigger the schema that a user has for a
particular genre of documents (Vaughan & Dillon, 1998). It follows that automatic
processing may be used by the members of a discourse community when faced with
a document genre they are familiar with.
Familiarity with the interface

Research has shown that users who are familiar with an interface seem to perform
better than unfamiliar users when searching for information on that interface
(Tabatabai & Shore, 2005). One explanation for these results could be that users who
are familiar with an interface are also familiar with the tasks it involves while
158
unfamiliar users are not. So, when faced with a familiar interface (with elements they
were familiar with), they use automatic processing of the tasks while unfamiliar
users have to rely on controlled processing in order to complete the tasks.
Also, users are less likely to become confused or lost in a familiar interface than in an
unfamiliar interface (Galletta, Henry, McCoy, & Polak, 2006). This might be
explained by the idea presented in Dillon(1991), that repeated exposure to a
particular type of hypertext allows users to acquire/develop a structural model
which in turn, helps them predict with a certain degree of accuracy the location of
information and possibly their location within the hypertext. This fits with the idea
expressed earlier in Chapter 2 that not conforming to the expectations of users
regarding a genre can cause disorientation, confusion and hinder performance.
On the other hand, users who are unfamiliar with an interface, seem to not only
perform worse than users who are familiar with it, but also rely more on using the
back button when searching (Holscher & Strube, 2000). Furthermore, when
unfamiliarity with the interface is allied to unfamiliarity with the search topic, users
seem to have severe problems searching for the information they need (Holscher &
Strube, 2000).
Considering that:
! Users familiar with a certain type of website are familiar with the tasks
associated with it, while unfamiliar users are not used to these types of tasks.
! Users familiar with a certain type of website are likely to have a structural
model of the website which helps them predict the location of information.
! Even though all users (familiar and unfamiliar) will not have knowledge of
the search topic, because unfamiliar users will also not know the interface,
they are likely to have severe problems when searching for information.
It seems plausible to expect that in the experiment described in this chapter users
who are familiar with web forums will perform better than unfamiliar users on both
the conforming and non-conforming interfaces. It is also expected that familiar users
will perform better on the conforming interface than on the non-conforming one.
Finally based on the literature reviewed here there are two possibilities for the
performance of unfamiliar users: (a) they will show no difference in performance
between the two layouts or (b) they will perform worse on the non-conforming
159
layout since many of its elements are pictorial thus requiring them to be correctly
identified (see page 157).
As mentionaed earlier, it is believed that web users make mental models of the sites
they visit (Danielson, 2002a, p. 77; Vaughan & Dillon, 1998) and that this mental
model is based on the structural organization of the information on the page
(MacKay, et al., 2004). For the creation of this mental model the layout of the page
and visual and semantic cues all contribute. However, despite the layout of the page
and its visual cues contributing to the formation of the mental model of the site, the
contribution of the layout of the page to user performance once the structural mental
model of the site is created is unknown. So here it is proposed that familiarity with
the interface be split into familiarity with the interface layout, and familiarity with
the interface structure. These two types of familiarity with an interface are directly
connected with the concept of visual and structural genre conventions respectively.
Although the distinction between the two types of interface familiarity might not
seem very clear at first it is not difficult to imagine examples of websites which hold
similar structures but different layouts. For instance, websites belonging to the online
news genre tend to have their news organized in the same or very similar categories
and to share similar structures. However, their interface elements might have
different symbolization modes, positions and configurations (see Chapter 5, section
5.2.1 for more explanation on these) resulting in different layouts.
Here, these two interface characteristics will be considered separately. Although

research has pointed out that when layout and structure are changed, familiar users
reported feeling disoriented (MacKay, et al., 2004) and performance suffered
(Vaughan & Dillon, 2006) the impact of radically changing the conventional layout,
but maintaining the structure of a web site on users who are familiar and unfamiliar
with the original interface has not yet been investigated. This type of information is
extremely valuable for web designers who need to visually alter the layout of a
website and need to know how that might affect its users.
Summary
Defining familiarity seems a daunting task since there are many types of familiarity
most of which are interlinked, influencing each other and consequently user
performance. Familiarity with a task, as well as familiarity with an interface has been
reported to improve user performance. Furthermore, there seems to be a relationship
160
between the two: users who are familiar with an interface are highly likely to become
familiar with the tasks performed in that particular interface. Also, familiarity with
the search topic can improve user performance, and when combined with a familiar
interface seems to boost user performance even more. Since the focus of this next
experiment will be on visual genre conventions and familiarity with an interface
layout (i.e. the visual conventions of the genre), it seems inevitable that familiar users
will have some sort of familiarity with the tasks to be performed. However, some
tasks designed to assess user ability to retrieve information from the interface will be
included. These tasks have the purpose of testing if there is a variation in
performance when the layout of the site changes. They also have the added benefit of
not being tasks which are commonly performed on web forums. Topic familiarity
will be minimized by having a search topic which participants will not be familiar
with.
In summary, the main point this study will address is the impact on user
performance of changing the visual conventions of a page, and keeping its other
genre conventions intact. This study aims to investigate this issue by testing the
performance of users familiar and unfamiliar with web forums on both the
conforming and non-conforming web forums websites generated from the survey
and the experiments described in Chapters 4 and 5 respectively. The hypotheses here
are that (a) overall, familiar users will perform better than unfamiliar users; but (b)
within the group of users who are familiar with web forums, those using the non-
conforming forum version will perform worse than the participants using the
conforming one.
6.2. Experiment 4 – Visual conventions and searching for

information in web forums
6.2.1. Objectives
The objectives of experiment 4 are to:
! test the effect of familiarity with a web forum interface on user performance
(time to complete a task and accuracy of task answer).
! test the effect of website conventionality on user performance (time to
complete a task and accuracy of task answer).
! test if user performance varies between tasks which involve retrieving
information from content and tasks which involve retrieving information
from the interface.
161
6.2.2. Method
Participants
40 participants, 20 familiar with web forums and 20 unfamiliar with web forums
participated in this experiment. Participants who had never searched for information
and/or posted on a web forum before were considered unfamiliar with web forums.
Participants who had either searched for information and/or posted messages on
web forums were considered familiar. However, it was ensured that participants had
navigated the interface of a web forum and not simply reached the post they wanted
via and external link (e.g. a search engine). Also to minimize familiarity with the
search topic participants were not expert PDA (content of the experimental forums)
users. In fact only one participant has reported using PDAs a few times.
All of the participants were fluent in English either as a first or second language. The
academic background of participants ranged from completed GCSEs to PhD
students. Participants in this experiment have not taken part in any previous
experiments conducted for this thesis.
Experiment design
The 20 familiar and unfamiliar participants were divided into two groups of 10 each.
In total the test used four groups, two in the ‘familiar category’ and two in the
‘unfamiliar category’. In each category one group was tested on the conforming
forum and the other on the non-conforming forum (see Table 30).
TABLE 30 - DISTRIBUTION OF GROUPS WITHIN THE FAMILIAR AND

UNFAMILIAR CATEGORIES AND THE FORUM LAYOUT EACH GROUP WAS TESTED ON.
Participants Tested on
Group FC Familiar with forums Conforming forum
Group FNC Familiar with forums Non-conforming forum
Group UC Unfamiliar with forums Conforming forum
Group UNC Unfamiliar with forums Non-conforming forum
The variables that were being changed (independent variables) were the
conventionality of the web forum, the familiarity of the participants with web forums
and the type of tasks participants were performing. The variables that were being
measured (dependent variables) were time to complete the tasks, and number of
correct answers. The perception of the participants regarding some aspects of the
162
forum was also measured. Because familiar participants have had prior experience
with other forums, they were also asked to judge the conventionality of the forums in
the experiment.
Materials
Web forum designs

Two different designs were created for this experiment. The designs were based on
results of the previous experiments (experiments 1 and 2 - see Chapter 5), which
tested whether the assumptions of conventionality drawn from a survey on web
forums were in accordance with what forum users perceived as conventional and
what they perceived as easy to use.
One design consisted of a web forum which followed the templates from the survey
which was perceived as conventional by users. The other site was meant not to
conform to conventions of the genre, so its design was based on the templates from
the survey which forum users thought were the least conventional.
As seen earlier in Chapter 5, a web forum usually consists of three distinct levels: the
forum index, a threads page and a post page and for the purposes of this experiment
each experimental forum had these three levels.
The template from the experiments 1 and 2 in Chapter 5 that was most frequently
chosen as conventional and easy to on each forum level, was used to create each level
of the conforming forum. The template which was chosen as conventional and easy
to use least frequently on each forum level was used to create each level of the non-
conforming forum (see Appendix I). The templates used to make each forum level
are shown in Table 31.
TABLE 31 - TEMPLATES USED TO CREATE EACH LEVEL OF THE

CONFORMING AND NON-CONFORMING FORUMS.
Forum level Conforming Non-conforming

Index page C11 N14
Threads page C21 N23
Post page C31 N34
163
The templates chosen to be used in the design of the conforming forum: C11 (level 1),
C21 (level 2) and C31 (level 3) had the symbolization, position and configuration of
the elements fixed at the most frequent values gathered from the survey.
As explained in the previous chapter (Chapter 5), the templates used for creating the
non-conforming forum screens: N14 (level 1), N23 (level 2) and N34 (level 3), were
chosen from the templates which were least frequently picked as similar to existing
forums. Because of this, the resulting templates did not have the symbolization,
position and configuration of the elements fixed in a similar manner. While level 1
and 2 had the symbolization and the configuration of the elements fixed at the least
frequent values, and the position fixed at the most frequent value, level 3 had
symbolization and position at the least frequent value, and configuration at the most
frequent value (Table 32).
TABLE 32 - HOW VALUES WERE APPLIED TO THE VARIABLES SYMBOLIZATION, POSITION AND CONFIGURATION FOR DESIGNING
THE FORUM TEMPLATES. VALUES COULD BE THE MOST FREQUENT (MF) OR THE LEAST FREQUENT (LF).
Symbolization Position Configuration

C11 MF MF MF
C21 MF MF MF
C31 MF MF MF
N14 LF MF LF
N23 LF MF LF
N34 LF LF MF
Graphic elements
The graphic elements used (e.g. icons, buttons) are the same as those used in
experiment 3 which tested the results of the survey (see Chapter 5, section 5.2). Due
to the way the non-conforming forum was designed, elements which were presented
in a certain style in the conforming version (e.g. text), were presented differently in
the non-conforming (e.g. graphic). However, whenever the symbolization mode of
the element was the same on the conforming and non-conforming forums, the same
graphic element was used.
As explained in Chapter 5, the elements which varied their symbolization mode on

the conforming and non-conforming forums were designed keeping aesthetic
consistency (consistency of style and appearance) with the other existing elements.
164
The same colour scheme was used to design both the conforming and non-
conforming forums
Content
The experiment involves participants searching for answers within a web forum. As
mentioned in the introduction, familiarity with the search topic can help improve the
performance on search tasks. So it is important that the knowledge participants have
on the subject of the forum is minimal in order to keep them from knowing the
solution to the tasks beforehand. The content of choice was related to Personal Digital
Assistants (PDAs) since it not only provides a reasonable amount of questions to be
answered, but is also simple enough to be comprehensible without being common
knowledge.
Because the forum content is divided according to different PDA brands and/or
operating systems it is possible that most participants will not have this knowledge.
Without this, it will be quite hard for participants to grasp the organizational
structure of the website, so a brief written introduction to PDAs (see Appendix L)
was created. This introduction outlines the brands, operating systems and pictures of
the PDAs found within the forum. This introduction will not only help participants
to understand the technical terms used in the organization of the forum but will also
ensure all participants have the same basic level of knowledge about PDAs.
Both the conforming and non-conforming forum convey the same content so that any
influence that content can have on the performance of the participants is balanced
out. The content for the experimental forums was gathered from an existing web
forum. However, the textual content used in the experiment is a reduced version of
the existing one, so the breadth of the experimental forum is smaller than the real
version. This should make searching for answers not too tiring for participants.
In the following pages there are samples of the index page (first level of the forum) of
the finished conforming (Figure 61) and the non-conforming (Figure 62) designs. The
final designs consisted of the template chosen for each level (of both the conforming
and non-conforming forums) with the content included and with the colour scheme
added.
165
FIGURE 61 - SAMPLE OF THE INDEX PAGE DESIGN (LEVEL 1) OF THE CONFORMING WEB FORUM.
FIGURE 62 - SAMPLE OF THE INDEX PAGE DESIGN (LEVEL 1) OF THE NON-CONFORMING WEB FORUM.
166
Tasks
A set of five tasks (see Appendix M) was created to test the participants on the
forums. There were two main ideas behind the creation of the experimental tasks:
one was to emulate the range of possible activities which a web forum user would
normally perform while searching for information within a web forum; the other was
to assess user ability to retrieve information from the interface. So, two types of tasks
were designed: one where participants had to retrieve information from the written
content (normal web forum activity), and the other where participants had to retrieve
information from the interface of the forum (interface retrieval). Tasks were given to
participants in the same order.
Questionnaire
A questionnaire was developed to assess what participants thought of the design of
the forums and the ease/difficulty of finding information.
The questionnaire contained 3 questions: the first two were directed to all
participants and assessed what participants thought of the website they had used in
the experiment. The last question was only to be answered by participants who were
familiar with web forums, and asked them to compare the appearance of the forum
they used in the experiment with other existing forums. The possible answers to the
questions were given on a Likert scale using five steps ranging from 1 to 5. Table 33
illustrates the questions and the meaning of the extreme values of the scale in each
case. Participants answered these questions by marking the number on the scale they
thought best represented their perception.
TABLE 33 - QUESTIONNAIRE ITEMS WITH THEIR RESPECTIVE EXTREME VALUES MEANINGS.
Question 1 5
1. In your opinion, how easy or difficult was it to find information in
Very easy Very difficult
the forum?
Very well
2. Overall, how would you rate the layout of the web forum pages? Badly designed
designed
3. How would you compare the layout of this web forum with other
Very different Very similar
web forums?
6.2.3. Procedure
Each test started with the experimenter explaining to the participants what the tasks
would be. Participants were told that the experiment consisted of them looking for
167
answers to different questions (tasks) within a web forum. It was explained that the
tasks would be handed to them on a printed sheet of paper one at a time and that the
answers should be written on the same sheet.
Participants were asked to look for the answers as quickly as possible and to answer
the tasks as accurately as they could. They were told not to feel pressured as what
was being tested was the design and not them. Then the participants were given the
sheet of paper with the introduction to PDAs. After that, the participants were shown
the forum condition to which they were assigned (either conforming or non-
conforming) and given a sheet of paper containing the first task. As soon as the
participants finished reading the task and looked at the screen the time measurement
started. Time measurement stopped as soon the participants had finished writing the
answer to the task.
The participants were then asked to return to the forum index page and were given
the next task. After the participants had answered all five tasks they were presented
with the questionnaire.
6.2.4. Results
Time
A Mann-Whitney test was used on all time results to determine if differences were
significant. The results found on the analysis of the time to complete the tasks are
shown on Table 34 with the significant differences highlighted.
TABLE 34 - MEAN VALUES AND STANDARD DEVIATION OF THE TIME TO COMPLETE TASKS IN EACH GROUP AND CONDITION
WITH THE SIGNIFICANT DIFFERENCES HIGHLIGHTED AND CIRCLED BY A DOTTED LINE. THE COLUMN MEANS SHOWS THE MEANS
OF THE USERS WHO USED THE CONFORMING AND NON-CONFORMING FORUMS RESPECTIVELY AND THE ROW MEANS REPRESENT
THE MEANS OF THE FAMILIAR AND UNFAMILAR USERS.
Time to Conforming Non-conforming Row Means

complete
the tasks Mean SD Mean SD Mean SD
449.2s
Familiar 115.7 509.7s 165.3 479.4s 142.3
627.3s
Unfamiliar 627.6s 132.0 627.0s 214.5 173.3
Column 538.4s
151.6 568.4s 195.9
Means
168
Familiar and Unfamiliar users

Overall participants who were familiar with web forums took less time to answer the
tasks than participants who were not familiar with web forums (see ‘row means
column’ in Table 34). A Mann-Whitney test showed this result was significant
(W=308.0, n=20, p=0.006).
Conforming and Non-conforming forums

There was no significant difference in time to complete the tasks (W=404.0, n=10,
p=0.887) between participants who used the conforming forum and participants who
used the non-conforming forum (see the column means for the conforming and non-
conforming forums in Table 34).
Familiar-conforming and Unfamiliar-conforming
Comparing the time the familiar group took to complete the tasks on the conforming
forum with the time the unfamiliar group took to complete the tasks on the same
forum, it is possible to observe that the familiar group performed significantly faster
than the unfamiliar group (W=67.0, n=10, p<0.005).
Familiar-non-conforming and Unfamiliar-non-conforming
A Mann-Whitney non-parametric test showed no significant difference in the time

taken to complete the tasks between participants in the familiar group who used the
non-conforming forum and the participants in the unfamiliar group who used the
non-conforming forum (W=92.0, n=10, p=0.345).
Familiar-conforming and Familiar-non-conforming

A Mann-Whitney non-parametric test showed no significant difference in the time
taken to complete the tasks between participants in the familiar group who used the
conforming forum and the participants in this group who used the non-conforming
forum (W=96.0, n=10, p=0.521).
Unfamiliar-conforming and Unfamiliar-non-conforming
A Mann-Whitney test showed no significant difference in the time to complete a task

between participants in the unfamiliar group who used the conforming forum and
the participants of this group who used the non-conforming forum (W=109.0, n=10,
p=0.791).
169
Correct scores
Scores were defined according to the number of correctly answered tasks by each
group. A Mann-Whitney test was used on all the correct score results to determine if
differences were significant. Looking at the significant results shown on Table 35 it is
possible to see that the significant differences in score follow the same pattern as the
significant differences in time taken to complete the tasks.
TABLE 35 - MEDIAN VALUES OF THE NUMBER OF CORRECT SCORES IN EACH GROUP AND CONDITION WITH THE SIGNIFICANT
DIFFERENCES HIGHLIGHTED AND CIRCLED WITH A DOTTED LINE. THE COLUMN MEDIANS SHOWS THE MEDIANS OF THE USERS
WHO USED THE CONFORMING AND NON-CONFORMING FORUMS RESPECTIVELY AND THE ROW MEDIANS REPRESENT THE
MEDIANS OF THE FAMILIAR AND UNFAMILAR USERS.
Correct scores Conforming Non-conforming Row Medians
Familiar 3.0 3.0 3.0

Unfamiliar 2.0 3.0 2.4
Column Medians 3.0 3.0
Overall, the familiar group answered more tasks correctly than the unfamiliar group
(see row medians in Table 35). This difference was significant (W=483.0, n=10,
p=0.041).
Comparing the scores of participants using the same forum type across the familiar
and unfamiliar groups, it is clear that when using the conforming forum familiar
participants achieved better scores than the unfamiliar participants. A Mann-
Whitney test showed this result to be significant (W=135.0, n=10, p=0.019).
Familiar-non-conforming and Unfamiliar-non-conforming

Comparing the scores of the familiar and unfamiliar participants who used the non-
conforming forum no significant difference is evident (W=111.5, n=10, p=0.635).
Also, comparing the scores for participants within the same group who used each
version of the forums showed no significant results.
Familiar-conforming and Familiar-non-conforming
A Mann-Whitney test found no significant difference in scores between participants

in the familiar group who used the conforming forum, and participants in this group
who used the non-conforming forum (W=117.0, n=10, p=0.3847).
170
Unfamiliar-conforming and Unfamiliar-non-conforming

In the unfamiliar group a Mann-Whitney test showed no significant difference in
scores between participants who used the conforming forum and participants who
used the non-conforming forum (W=95.0, n=10, p=0.458).
Right answers per second (RPS) rate

One way of looking at the results to better assess any improvement or hindrance in
performance is to combine time taken and correct answers into one rate (Whiteside,
Jones, Levy, & Wixon, 1985). Thus, the number of correct answers and the time to
complete the tasks were combined into one measurement.
However, because of the great number of zero correct scores (participants not
answering a task correctly), in most cases combining scores and time to complete
tasks caused the contribution of the time factor to be lost.
A data transformation was then applied to the data in order to adjust the scores that
were zero to a value that would not cancel out the contribution of time. The problem
caused by this can be appreciated when considering the hypothetical example of two
participants: one that takes a long time to before quitting the task and another
participant who quickly quits the task. Even though these two participants did not
complete the task correctly, the time they took to complete the task is relevant and
can not be ruled out.
Kirk (1995) suggests four types of transformations that can be applied to the data:
square-root transformation, logarithmic transformation, reciprocal transformation
and sine transformation. Kirk also suggests a method for selecting which
transformation is more appropriate for a particular data set.
After applying the method described in Kirk (1995), the square-root transformation
came out as the most appropriate to the data since it resulted in the distribution that
was closest to the normal. To make the resulting number easier to work and
understand it was multiplied by constant value of 100. This resulting number
indicates the rate of right answers per second so that a higher value indicated better
performance. The formula used to calculate it is shown in Figure 63.
171
FIGURE 63 - FORMULA FOR THE DATA TRANSFORMATION AND

CALCULATION OF THE RATE OF RIGHT ANSWERS PER SECOND.

Familiar users showed an overall better performance than unfamiliar users (see
Figure 64) with a mean RPS rate of 1.44 (S.D.=0.55) against a mean of 0.85 (S.D.=0.62)
in the unfamiliar group. A Mann-Whitney test showed that this difference was
significant (W=522.0, n=20, p<0.03).
FIGURE 64 – OVERALL PERFORMANCE (MEASURED BY THE RPS RATE) OF FAMILIAR

PARTICIPANTS COMPARED TO THE UNFAMILIAR PARTICIPANTS.
When using the conforming layout, participants in the familiar group performed
better (Mean=1.61, S.D.=0.62) than the participants in the unfamiliar group
(Mean=0.64, S.D.=0.25). Again a Mann-Whitney test on the data revealed a significant
difference in performance between familiar and unfamiliar participants when using
the conforming forum (W=518.0, n=20, p<0.01).
Similarly to the analysis of the time to complete the tasks and number of correct
answers, no other significant results were found between:
172
(a) Familiar participants and unfamiliar participants on the non-conforming layout.

(b) Familiar participants on the conforming layout and non-conforming layout
(c) Unfamiliar participants on the conforming layout and non-conforming layout
The results are summarized in Table 36.
TABLE 36 – MEAN VALUES OF THE RPS RATES IN EACH GROUP AND CONDITION WITH THE SIGNIFICANT DIFFERENCES
HIGHLIGHTED AND CIRCLED WITH A DOTTED LINE. THE COLUMN MEDIANS SHOWS THE MEDIANS OF THE USERS WHO USED THE
CONFORMING AND NON-CONFORMING FORUMS RESPECTIVELY AND THE ROW MEDIANS REPRESENT THE MEDIANS OF THE
FAMILIAR AND UNFAMILAR USERS.
Correct scores Conforming Non-conforming Row Medians
Familiar 1.61 1.27 1.44

Unfamiliar 0.70 0.99 0.85
Column Medians 1.15 1.13
Task type
As mentioned before experimental tasks were divided in two categories: content-
related and interface-related. On content-related tasks, participants had to read
through the content of the forum posts to find the answer. On interface-related tasks
participants had to search the visual elements of the interface (e.g. looking for an icon
that indicates that a post is locked) to find the answer to the task. Of the five tasks,
three were content-related and two were interface-related.
To analyze whether familiarity and conventionality had a different effect on

performance depending on the type of task (content or interface) being performed, a
two within-subjects, one between subjects factor ANOVA was applied to the RPS
rate. From the results (Table 37) an interaction between task type and conventionality
is evident. The main effects of familiarity and task type were also detected.
TABLE 37 - RESULTS FOR THE TWO WITHIN-SUBJECTS ONE BETWEEN-SUBJECTS FACTOR ANOVA.
Source (between-subjects) SS df Mean Sq. F-ratio p

Familiarity 216.773 1 216.773 4.811 0.035
Conventionality 41.321 1 41.321 0.917 0.345
Familiarity x Conventionality 49.639 1 49.639 1.102 0.301
Error 1622.126 36 45.059
Source (within-subjects) SS df Mean Sq. F-ratio P

Task type 407.523 1 407.523 11.600 0.002
Task type x Familiarity 56.710 1 56.710 1.614 0.212
Task type x Conventionality 254.676 1 254.676 7.249 0.011
Task type x Familiarity x Conventionality 16.135 1 16.135 0.459 0.502
Error 1264.686 36 35.130
173
The graph indicates (see Figure 65) that participants using the non-conforming site
performed better than the conforming group on interface tasks, however, on content
tasks both groups showed an equivalent performance. Surprisingly, these results
suggest that the non-conforming interface resulted in better performance.
FIGURE 65 –INTERACTION BETWEEN CONVENTIONALITY AND TASK TYPE.
A main effect of task type showed performance to be overall better on interface tasks
than on content tasks (Figure 66).
FIGURE 66 - PERFORMANCE (MEASURED BY THE RPS RATE) ON

INTERFACE TASKS COMPARED TO CONTENT TASKS.
174
Questionnaire
For all the analysis of the questionnaire data the Mann-Whiney test was used.
Overall there was no difference (W=351.5, n=20, p=0.095) in perception of the
difficulty of finding information in the non-conforming forum (Median=4.00) and in
the conforming forum (Median=3.00). However there is a trend towards significance
of this result and it is possible that a larger sample size would reveal this difference
to be significant.
For the second question, the test also showed no difference in the overall perception
of participants regarding the quality of the design of the conforming (Median=3.00)
and non-conforming (Median=3.00) websites (W=405.5, n=20, p=0.908).
Familiar users
Looking at the questionnaire answers in the group of users who were familiar with
web forums no difference in perception of ease/difficulty in finding answers
between the conforming and non-conforming forum was found (W=88.5, n=10,
p=0.201). In terms of the quality of the design of the conforming and non-conforming
websites the test revealed no difference in user perception (W=105.0, n-10, p=1.000).
However, when comparing the conforming and non-conforming design to

previously visited web forums (Figure 67) participants found the layout of the
conforming forum (Median=4.00) more like other forums than the non-conforming
one (Median=3.00). This result was statistically significant (W=137.0, n=10, p<0.01).
FIGURE 67 - MEDIAN VALUE OF THE ANSWER GIVEN TO THE QUESTION: ‘HOW WOULD YOU COMPARE
THE LAYOUT OF THIS WEB FORUM WITH OTHER FORUMS?’ (1=VERY DIFFERENT, 5=VERY SIMILAR).
Participants who used Participants who used the

the conforming forum non-conforming forum
175
Unfamiliar users
In the unfamiliar group of users, similar results to the familiar group were found.
There was no difference in the perception of the ease/difficulty of finding
information between the conforming and non-conforming websites (W=92.0, n=10,
p=0.312). Similarly, no difference was found between the perceived quality of the
design of conforming pages and non-conforming ones (W=103.0, n=10, p=0.904).
Participants in this group were not asked to compare the forums with existing ones
since by being unfamiliar with them it was assumed that they had not come across
web forums and so could not have an opinion about their design.
6.2.5. Discussion
Performance
Search performance seems to improve according to familiarity with the interface
where the search is taking place, and familiarity with the type of task being executed.
Considering that the familiar group encompasses users ranging from intermediate to
experts and the unfamiliar group encompasses novice users in terms of web forum
usage, this result agrees with what was found by Holscher and Strube (2000). The
authors found that users with high Web expertise had higher success rates on the
Web search tasks when compared to novice users.
In this experiment (experiment 4), users who were familiar with web forums
completed tasks faster than users who were unfamiliar with this type of web pages.
They also obtained a better score than the unfamiliar group and a higher RPS rate
than the unfamiliar participants. This confirms hypothesis (a), which stated that
overall, familiar participants will perform better than unfamiliar ones.
Participants in the familiar group were web forum users and were familiar not only
with the interface but also with the tasks which are executed in web forums. Thus it
is no surprise that they performed better than participants who never used a web
forum before. After all, despite the interface in the non-conforming forum being
changed, the processes (e.g. decisions, paths to take) necessary to complete the tasks
remained the same as in the conforming forum. Because familiar participants knew
web forums they were familiar with the type of tasks performed in them. On the
other hand, unfamiliar participants were not used to the style of tasks which can be
performed in a web forum and might have had difficulty in knowing how to process
176
the tasks. An explanation which stems from this one is that because of familiarity
with forums and the tasks it involves (see Lim, et al., 1996), users in the familiar
group processed tasks automatically while unfamiliar participants used controlled
processing of tasks. This could have resulted in faster and more accurate task
completion for the former group.
However, a much more puzzling result was the fact that no significant difference in
performance was found between participants of the familiar group using the
conforming forum and those using the non-conforming forum. This contradicted
hypothesis (b) which predicted that familiar participants would perform better in the
conforming layout than in the non-conforming one. This also goes against the
literature (Galletta, et al., 2006; Holscher & Strube, 2000; Tabatabai & Shore, 2005)
according to which familiar participants should have performed significantly better
in the conforming version than in the non-conforming one. There could be various
explanations for the lack of significant difference in performance between familiar
users using conforming and non-conforming pages. One explanation could be that
since the experimental forums did not have a lot of breadth compared to existent web
forums, the scope of the search was reduced. Thus, participants had a reduced
number of links to search for information. As a result the familiar participants (who
were familiar with the forum structure and with the tasks performed in web forums)
found answers very quickly independently of the interface they were using.
However, there was no difference in performance between familiar and unfamiliar
participants who used the non-conforming forum. Considering that unlike the
familiar participants, the unfamiliar participants were not accustomed to the forum
structure and tasks, it would be expected that their performance would have been
worse than the performance of their counterparts. But, on the contrary, when using
the non-conforming forum there were no significant differences between the groups.
Another explanation is that a learning process could have occurred. In the case of
participants who were familiar with forums and the associated tasks, they could have
learned the non-conforming forum fairly quickly and from then on performance was
equivalent to the familiar participants using the conforming forum (see Figure 68). In
the unfamiliar group, learning could have occurred as well, but here the group was
unfamiliar with both the conforming and non-conforming interfaces, and both had to
be learned. However, the time to learn either interface would be very similar,
resulting in similar learning curves and consequently no differences in performance.
The fact that learning could have occurred does not necessarily rule out that the
177
narrowness of the site and the familiarity of some users might have influenced the
lack of differences. In fact it is possible that all these factors have influenced user
performance.
FIGURE 68 – HYPOTHETICAL ILLUSTRATION OF A LEARNING EFFECT TO EXPLAIN THE LACK OF DIFFERENCE IN PERFORMANCE
BETWEEN THE FAMILIAR USERS TESTED WITH THE CONFORMING AND THE FAMILIAR USERS TESTED WITH THE NON-
CONFORMING FORUM.
Familiar users using a

conforming forum
Familiar users using a

non-conforming forum
Task type
Another interesting result was the interaction between conventionality and task type.
Results showed that when conventionality is violated, participants tend to perform
better on interface related tasks than on content related tasks. One explanation for
this could be because interface related tasks were written with an explicit cue to
where the information was located (in this case the exact link where participants had
to click), and required participants to depend less on the layout than content tasks.
So, when visual conventions were not followed, the cues embedded in the interface
tasks could have made finding interface elements easier than finding content.
Furthermore, because task order was not randomised or counterbalanced it is very

likely that learning occurred through the experiment. Therefore further investigation
of this interaction is needed.
178
Questionnaire
The lack of difference in user perceptions of the ease/difficulty of finding
information on both versions of the forum is consistent with the absence of
differences in performance between the two conditions. This result could be due to
the fact that participants felt equally about the ease/difficulty of finding information
on both versions of the forum. An alternative explanation would be that by the end
of the test (when participants answered the questionnaire) participants who were
using the non-conforming version were more familiar with the new interface and
judged it according to their final impression and not their first.
When comparing the quality of the design of the conforming and non-conforming
forums, both the participants in the familiar and unfamiliar groups showed no
difference in perception. This could indicate that because forums were created using
the same design features (colour scheme, graphic elements or when elements
differed, the design kept the aesthetic consistency), they were quite similar visually.
This could indicate that the variations on symbolization, position and configuration
of the graphic elements on the non-conforming page were not apparent to the
participants. An alternative explanation comes from the perception of the researcher
when conducting the test. Many participants, when asked to rate the design, turned
to the researcher and asked if he had designed the screens. This could indicate that
the participants were conscious about ‘hurting the feelings’ of the experimenter and
therefore might not have expressed any negative opinion they could have had.
Finally, users who were familiar with web forums found the conforming design more
similar to other existing web forums than the non-conforming design. This result
again confirms what the studies in Chapter 5 have shown.
6.2.6. Conclusion
Although the study did not find any indication that not conforming to
conventionality can compromise user performance, the results obtained show that
participants who are familiar with web-forums performed better than unfamiliar
participants on both the conforming and non-conforming design. This suggests that
familiarity may play an important role in searching for information online, but might
also enable users to quickly learn the interface, picking up on the existing visual
elements when the expected visual conventions are not present. The interaction
between task type and conventionality is also very interesting, indicating that a
179
violation of visual conventions might have a different effect on performance,

depending on the type of task the user is performing.
However, the lack of difference in performance between familiar users who were
tested with the conforming design, and the familiar users tested with the non-
conforming design, was unexpected. This needs further investigation and another
experiment, which can look at the performance of familiar users over time, needs to
be carried out.
There are also some limitations in this experiment, which need to be resolved in the
next one. In this experiment the analysis of the effects of task type and interactions
were made more difficult because there were only 5 tasks and only 2 of them were
interface related. Therefore, any task which participants found too easy or too
difficult could bias the results.
Tasks were not designed to maximize the requirements of each type of activity (i.e.
retrieving information from content vs. retrieving information from the interface).
This is particularly true for the interface tasks. The fact that the order of the tasks was
not randomized also presented a problem since a carry-over effect could have
influenced the results. The wording on the interface tasks may have made them very
simple to answer as they indicated to the participant where to click.
The topic chosen, PDAs, also caused some problems as some participants were not
familiar with the technological terms used in the forums. Even though a basic
explanation of PDAs was given before the test started, it seems this it was not
sufficient, as participants had problems completing some of the tasks.
Finally, the number of pages of the experimental websites could have oversimplified
the tasks making it too easy for familiar users to find the answers to the tasks.
The next Chapter describes an experiment which investigates further the unexpected
lack of difference in performance of familiar users when using the conforming and
non-conforming layouts. The experiment will look at the performance of users on
both layouts over time to identify if participants using the non-conforming version
may be improving their performance as they use the website.
180
Chapter 7
Visual conventions and performance
over time
7.1. Introduction
Results from experiment 4 showed no significant differences in performance of users
familiar with web forums when presented with a website which violated their
expectations. This finding was contrary to what is suggested in the literature that not
conforming to conventions results in poor user performance (Gery, 1995; Vaughan &
Dillon, 2006; Walton & Vukovic, 2003).
This chapter describes an experiment which was devised to investigate why in the
previous experiment participants who were familiar with a certain set of visual
conventions showed no overall difference in performance, whether using a website
which violated these conventions or a website which conformed to the conventions.
Much has been researched on the expectations of users when faced with a website,
how these users browse the Web, their strategies and navigation patterns. However,
the issue of not conforming to visual conventions and the impact it has on user
performance over time has not been explored thoroughly, and the existing research
merely gives indications as to what could have caused the equivalent results. Some
relevant findings are discussed here.
7.1.1. Navigation and browsing strategies

Browsing is an information-seeking strategy which is exploratory in nature. Some
authors describe browsing as being dependent on serendipity (Marchionini &
Shneiderman, 1988).
181
Visual conventions and performance over time
There are three generally agreed strategies which can be used while browsing: (a)
search browsing, which happens when the goal is known and tends to be highly
systematic; (b) general purpose browsing – here, even though the target is known, it is
broadly defined, obliging users to probe various sources with a high probability of
having items of interest relevant to the search and (c) serendipitous browsing which is
fortuitous and has no real goal defined (Catledge & Pitkow, 1995; Cove & Walsh,
1988; Danielson, 2002b; Marchionini, 1995). Although there seems to be general
agreement regarding these three browsing strategies, there seems to be a
contradiction in the naming of the category ‘search browsing’. If all browsing
activities involve serendipity as stated by Marchionini and Shneiderman (1988), then
a systematic search with a well-defined goal in mind (search browsing) would not
qualify as browsing. A more appropriate naming for the ‘search browsing’ strategy
then is provided by Salomon (1989): goal-directed search. General purpose browsing on
the other hand, although having a goal defined, involves probing different sources
which would allow for serendipity to happen, thus qualifying it as browsing.
But are goal-directed search, general purpose browsing and serendipitous browsing used
equally when navigating through the World Wide Web? To answer this question it is
crucial to look at the activities performed while navigating through the Web.
It is not surprising to find that next to e-mailing, the primary use of the World Wide
Web is information retrieval (Broder, 2002; Gordon & Pathak, 1999; Rumpradit &
Donnell, 1999). Taking into consideration the amount of information available on the
World Wide Web, and the fact that searching for information is part of a great
number of human activities (Tabatabai & Shore, 2005), it is likely that information
retrieval will remain one of the most frequent Internet activities. This makes the three
browsing styles mentioned above very important since in one way or another they all
involve the retrieval of information. But further analysis into the serendipitous
browsing style shows that although it is extremely rewarding, it is not very efficient
for information retrieval (Stanoevska-Slabeva, Hombrecher, Handschuh, & Schmid,
1998). Because users ‘want to find the information they seek quickly and easily’
(Cappel & Huang, 2006) they will use the most efficient strategies they have
available. Therefore it is very likely that goal-directed search and general purpose
browsing are the two more common browsing styles on the web.
182
Navigation models and efficiency

When searching for information on the World Wide Web, users tend to navigate in a
hub and spoke model, where search starts at a central page (hub) and then navigates
to a new page (spoke), then returning back to the central page (Catledge & Pitkow,
1995; Tauscher & Greenberg, 1997). This browsing style is marked by intensive use of
‘backtracking’ a technique where the user returns to the starting point through the
same route they have navigated. This is achieved by clicking on the back button that
is widely available on WWW browsers. Another navigational technique which is not
as often used as backtracking, but also allows return to the central page (hub), is
called loopback (Catledge & Pitkow, 1995). Here, instead of using the back button to
return, the user clicks directly on a ‘return to home/ entry point’ link and is traversed
immediately to the desired location. However, considering the speed with which
technology evolves and the growth of the World Wide Web in the past ten years, it is
possible that the hub-and-spoke navigation model is no longer used and
backtracking and loopback are not the preferred strategies when browsing for
information on the web.
However, a recent study by Milic-Frayling et al (2004) has confirmed that the results
of previous studies are still valid. This study analysed the Web usage log of 9
participants for a period of 3 weeks. Results showed that 28% of all page visits were
visits to hub pages and 30% of all the navigation was done using the back button.
Although these browsing styles explain broadly how users navigate on the web, they
alone are not enough to explain if the navigation is being carried out in an efficient
manner. Navigation is considered efficient when a user is able to find the desired
information using the minimum necessary number of movements from link to link
(Schoon & Cafolla, 2002). Since the pilot study to be carried out will be looking at
user performance, it is important to have means of measuring the efficiency of the
navigation of the participants.
One attempt to measure the efficiency with which users navigate through a website
was proposed by Smith(1996). This measurement, called Efficiency Indicator by the
author, was found by dividing the number of nodes (information items in the
hypertext base) required to complete a task (R), by the number of different nodes
visited (D). So the more the indicator moved away from zero and towards 1, the
more efficient the search was.
183
The author also claims that the indicator informs when the user did not complete the
task. Whenever the indicator is greater than 1, it means that the task was not
completed (Smith, 1996). This affirmation is not always valid since it does not take
into account the situation where the user has visited more nodes than the minimum
required to complete the task (R), and then quits the task. This situation would have
an indicator smaller than one (R<D) but the task would not have been completed.
Unreliability regarding the incomplete tasks might prove problematic for the next
experiment to be carried out. Therefore, the efficiency indicator will be used to
quantify the efficiency of each group but not to detect whether a participant has quit
a task or not.
Another critique of the efficiency indicator (Smith, 1996) is made by Herder and Dijk
(2003). The authors claim that a study by McEneaney (2001) found that efficient
hypertext readers seem to work by going back and forward from landmark nodes at
the root of the pages and that this strategy would make efficient users look inefficient
according to Smith’s indicator standards. What Herder and Dijk (2003) fail to notice
is that the proposed efficiency indicator (Smith, 1996) takes into account the number
of different nodes visited rather than the number of nodes visited. So if readers were
to apply the browsing strategy mentioned in McEneaney (2001), the revisits would
not be counted towards the indicator.
However, Herder and Dijk(2003) question both McEneaney (2001) and Smith (1996)
methods since neither mention what pages users returned to, nor how much time
they spent on each page. Although it is important to know which pages users are
revisiting when navigating because it would help separate lost users from those
using a navigation strategy, this would not affect the efficiency indicator proposed by
Smith for the same reasons described above. However, not knowing how long users
spend on each page does affect the idea of efficiency since time is an important
indicator of efficiency.
Despite the critiques, Smith’s methods have been successfully used and endorsed by
some researchers (Larson & Czerwinski, 1998; Otter & Johnson, 2000; Selvidge,
Chaparro, & Bender, 2002) and the efficiency indicator is an important instrument in
analysing navigational efficiency when it is considered along with time.
184
Since the efficiency indicator will be adopted in this experiment, a system must be
used to allow definition of the number of steps necessary to complete a task, and
later analyse the steps participants took while doing the experiment.
One which serves the purposes of sorting the data to apply the efficiency indicator
well is a taxonomy developed by Byrne et al.(1999). By examining participants’
browsing of the web as they would in a normal work day, the authors observed six
general classes of web tasks:
! Use information – when the user attempts to use a piece of information from
the WWW (e.g. by reading, listening, copying and pasting).
! Locate on page – when the user tries to find a piece of information on the site
through visual search.
! Go to page – when the user performs an action that makes the browser
display a particular page (e.g. clicking a hyperlink, using the back button,
typing a web address).
! Provide information – when the user sends information through the web
(e.g. providing authentication information, shipping addresses).
! Configure browser – when users changed the user configurable options of
the browser to suit their needs (e.g. changing the size of a window, hiding the
toolbar).
! React to the environment – These tasks are typically when the browser
demands something from the user (e.g. a dialog box opens).
Each of these main processes is broken down into smaller sub-processes (e.g. a ‘go to
page’ action can involve clicking on a hyperlink, using the Back/Forward button,
clicking on a bookmark, etc).
This taxonomy system will be adopted in this study since it facilitates the setting of
the minimum number of pages (nodes) necessary to complete each task. It also
allows a detailed transcription of the actions of each participant including which
page they revisited and which pages were different. However, due to the kind of
tasks being created in this experiment, only 3 of the 6 classes defined by Byrne et
al.(1999) will be used. The classes used will be: (1) Use information; (2) Locate on
page and (3) Go to page.
185
7.1.2. Expectations Conventions and usability

As seen previously in chapter 2, whenever looking at a document, users have certain
expectations on how to handle it in order to extract the necessary information
(Dillon, 2004, p. 117). According to the authors, when users have some knowledge of
the document, this provides them with information on the probable structure and
organization of the elements of the document (Dillon, 2004: p.117). These
expectations are built up from long periods in contact with a particular set of
conventions that are characteristic of that type of document.
Many usability and human-factors experts warn about the problems of breaking
these conventions. It is said that not following an established set of design
conventions, thus violating the expectations of users for a particular document, is
likely to generate major usability problems (Nielsen, 1999a). The author however
gives no details as to which major usability problems are caused by these violations
of conventions.
Although the experiments conducted in Chapter 5 have shown that users perceive
conforming layouts as easier to use than non-conforming ones, the fact that
something is conventional does not necessarily mean that it is also easier to use than
something that is does not conform to conventions. Sometimes, it is even necessary to
break with standards and conventions in order to improve usability, for example
interfaces designed according to the principles of usage-centered design (Constantine
& Lockwood, 2002). Norman(2005) goes even further by stating that ‘great design
comes from breaking the rules, by ignoring the generally accepted practices’. A good
example of this can be found in an experiment conducted by Blackler et al.(2004)
which compared user performance on remote controls with different layouts. The
authors found that user performance was improved by using a specific layout which
consisted of intuitive symbols (different from the conforming ones) located in
positions different from the conforming ones.
Nielsen (1999b) also exemplifies this point well by affirming that if the navigation
menu of a web page was placed on the right side it would improve its usability
because:
(a) It would be closer to the scroll bar and that would save users time
(b) Users always look at content first (cf. DeVigal, 2003)
186
Since in cultures that read left-to-right people tend to look at the left side of the
screen first, this would leave the left side of the window for content to be placed
(Nielsen, 1999b). But Nielsen continues to reinforce the following of conventions
even ‘when a different design would be better’ (Nielsen, 1999a).
Despite this however, some other authors list other problems related to breaking
with conventions. According to Dillon and Vaughan(1997) the consequence of
breaking conventions is that it makes users feel disoriented having to rely more on
interface elements such as the browser back and forward buttons. As seen before,
using back and forward buttons on the browser can indicate that the user is lost
(which is a serious problem) but it can also be used as part of a navigation strategy
(such as hub-and-spoke model). It is also said that not conforming to conventions can
affect comprehension by making it difficult for readers to gather the textual
information into a meaningful representation of the text (Loines, 1991). Loines (1991),
however, gives no empirical evidence to support this comment. Despite that, the idea
of conforming to conventions is one widely practised and reinforced. This poses a
constant dilemma to designers – should they innovate and break new ground or
should they stick to the established set of rules and conventions that users are
accustomed to, and ensure that their expectations are met?
As literature suggests, users do have expectations on the positioning of web page

elements (Bernard, 2001a; Markum & Hall, 2003) and a web design that meets these
expectations helps users find interface elements faster (Oulasvirta, Kärkkäinen, &
Laarni, 2005) and reduce task completion time (Ehret, 2002).
As seen in Chapter 2, a study to check these statements was carried out by Kalbach
and Bosenick (2006). The study manipulated the visual convention of a website by
altering the position of the navigation bar. According to the study, if the statements
were true, user performance on websites which placed the navigation bar on the
expected left-side position (Figure 69) would be better than navigation bars placed in
an unexpected the right-side position (Figure 70).
187
FIGURE 69 - SAMPLE OF THE CONVENTION-CONFORMING WEBSITE

USED BY KALBACH AND BOSENICK (2006)
FIGURE 70 - SAMPLE OF THE NON-CONFORMING WEBSITE

USED BY KALBACH AND BOSENICK (2006)
The study found no differences in overall user performance (measured by the task
completion time) regarding the positioning of the menu on the screen. But
188
researchers found that participants who used the right-side menu spent more time
scanning the homepage on the two first tasks and that this effect diminished on the
subsequent tasks.
This lack of significant results in Kalbach and Bosenick (2006) could be explained by
another study which compared user performances on genre-conforming and non-
conforming online news pages (Vaughan & Dillon, 2006). In this study, Vaughan and
Dillon went further than Kalbach and Bosenick (2006) by not only not conforming to
the visual conventions of a website but also its structural conventions (e.g. the
conventional way of structuring the information within the website).
To develop the experimental material, the authors asked expert web news readers
and developers to come up with what a genre-conforming web page would be like.
Participants drew a sketch of the homepage, category page and story page, and then
all sketches were discussed and the agreed features were noted. Because participants
of this group had trouble coming up with non-conforming categories and design
ideas, the experimenters asked a panel of experts from computer science, cognitive
psychology and human-computer interaction what a non-conforming website would
be like. Samples of the convention-conforming and non-conforming websites used by
the authors are shown in figures 71 and 72 respectively.
FIGURE 71 - SAMPLE OF THE CONVENTION-CONFORMING NEWS HOMEPAGE

USED BY VAUGHAN AND DILLON (2006)
189
For the non-conforming sites, not only the layout of the page was changed from what
was deemed conventional, but also the categorization of the news was totally
reinvented, demanding a complete semantic rethink of the information organization
within the site by the user.
FIGURE 72 - SAMPLE OF THE NON-CONFORMING ONLINE NEWS HOMEPAGE

USED BY VAUGHAN AND DILLON (2006)
The study found that the participants who used the genre-conforming sites had a
more efficient performance on the information-seeking tasks compared to those who
used the non-conforming site. But by the fifth day they used the website, users of the
non-conforming sites had learned the new interface, increasing their efficiency while
doing the tasks.
So it is possible that the learning effect which took place in the study by Vaughan
and Dillon (2006) also happened in the study by Kalbach and Bosenick (2006).
Perhaps the fact that less page scanning was done towards the end of Kalbach and
Bosenick’s experiment was an indication that some sort of learning had happened.
This still does not explain why there was a discrepancy between the results between
of the two studies. Comparing the two studies it is clear that Vaughan and Dillon
violated the visual and structural conventions of the genre, but Kalbach and Bosenick
only altered the visual conventions of the genre. This could have made the learning
easier in the latter case consequentially speeding up the process, and making the
initial differences which might have existed, disappear.
190
People can learn to do a task by one of three ways: (a) by hearing, (b) by seeing or (c)
by doing. Out of these three ways, the most important and effective is learning by
doing (Constantine & Lockwood, 2002). Based on learning by doing, the authors
developed the Anticipatory Learning paradigm which consists of four elements:
novelty, anticipation, action and confirmation. Usually the user finds an unfamiliar
element, and guesses what the element is or how it works. Then the user tries it out
and is rewarded by finding out that the guess was correct (Constantine & Lockwood,
2002). Constantine and Lockwood (2002) mention that when all the elements
necessary for anticipatory learning to take place are present, no prolonged period of
trial-and-error is needed. With one use the user learns the interface and uses it
correctly thereafter (Constantine & Lockwood, 2002).
So in the previous study by Kalbach and Bosenick (2006), the fact that the structure of
the site was not changed might have given users a better chance to correctly
anticipate the function of a particular interface element thus reducing the learning
process. This could indicate that the violation of the visual conventions of a website
might not be as bad as currently suggested by usability experts simply because users
might pick up on the ‘new layout’ quite quickly.
In fact, some research has shown that users do learn fairly easily where elements of
the interface are positioned (Ehret, 2002; McCarthy, et al., 2003; Oulasvirta, et al.,
2005). This indicates that the difficulties caused by breaking visual conventions by
repositioning elements on a page can easily be overcome by users. The study on
interface menu positioning carried out by McCarthy, Sasse and Riegelsberger (2003)
has shown that sites that conformed to users’ expectations regarding the menu
positioning did lead to faster search times when compared to non-conforming
websites. But according to the authors this performance difference was short-lived
and when sites were revisited for the second time, there was no search time
difference.
Although there seem to be indications that not conforming to conventions might

have a negative impact on user performance some experiments seem to suggest that
this negative impact might not last for long. Furthermore, when only visual
conventions are violated it seems that this negative impact can be even more
ephemeral. But no study so far has looked at whether users can improve their
191
performance as they use a site which violates visual conventions for a certain period
of time or how long this process would take.
Also, in this experiment the effect of not conforming to conventions on disorientation

as suggested by Dillon and Vaughan (1997) will be investigated along with user
performance.
7.1.3. Disorientation
As seen before, not conforming to the expectations users have regarding a particular
genre has been reported to make users feel disoriented (Dillon & Vaughan, 1997).
Disorientation in hypertext has been defined as the ‘tendency to lose one’s sense of
location and direction in a nonlinear document’ (Conklin, 1987, p. 40). It has been
considered one of the major problems in navigating through hypertext (Conklin,
1987; Nielsen, 1990; Xu, Cockburn, & McKenzie, 2001).
Measuring disorientation, however seems to be quite difficult (Herder, 2003) and

several methods have been proposed. Basically the methods revolve around two
main ideas: one that claims a link between the actions of users when navigating and
disorientation (Rosenberg, 1996) and another which claims that user disorientation
can only be measured by asking the user about their perceived disorientation (Ahuja
& Webster, 2001).
Regarding the actions of users while navigating which could indicate disorientation,
there are page revisits and backtracking. According to a psychological model of
disorientation proposed by Yatim (2002) disoriented users face two options: (a) quit
the task altogether or (b) try to re-orient themselves. This re-orientation process can
involve using the back button (backtracking), exploring alternative routes or re-
reading the information available (Yatim, 2002). The concept that lost users try to re-
orient themselves through the use of backtracking does reinforce the use of
backtracking as a measure of disorientation. However, pressing the back button does
not always mean that the users are lost and trying to re-orient themselves. The user
could be revisiting a page to read it again (Rosenberg, 1996) which would not
necessarily mean that they are lost.
This builds up a strongest argument against using backtracking and revisits alone to
measure disorientation, since backtracking can also be used as a web navigation
strategy (Catledge & Pitkow, 1995).
192
Recently researchers have been questioning the concept of dividing these two
measurements (user action and perceived disorientation) and have actually found
correlations between them (Gwizdka & Spence, 2007; Herder, 2003; Yatim, 2002).
This correlation could indicate that although backtracking and revisits might not be a
foolproof way of measuring disorientation, they could give good indication as to
whether or not a user is lost, especially when combined with another measure.
7.2. The pilot study

Before the main experiment was executed, a pilot study was conducted in order to
investigate what happens to performance over time (i.e. whether performance
changes as participants progress on the non-conforming site) and test the experiment
design and methodology.
This pilot study compares what happens when web forum users use a non-
conforming forum with when they use a conforming forum. The hypotheses here are
that (a) participants who use the non-conforming forum will familiarize themselves
with the new interface, as happened with the studies from McCarthy et al.(2003) and
Vaughan & Dillon (2006). This familiarization will be reflected in their performance
so that the group which uses the non-conforming website will initially perform
worse than the group which uses the conforming forum, but they will show a
performance improvement throughout the test. (b) participants who use the
conforming forum will show no performance improvement throughout the
experiment.
7.2.1. Objectives
The objectives of this pilot study are:
! Check if the performance of participants using a non-conforming forum will
improve over time.
! Verify if the efficiency indicator (Smith, 1996) is suitable for use in this
experiment and test the adequacy of the taxonomy proposed by Byrne et
al.(1999) to gather data to be used in the indicator.
! Evaluate if measuring navigation corrections (e.g. using the back button or
traversing links) is a reliable way of assessing user disorientation.
193
7.2.2. Method
Experiment design
This pilot experiment used two independent groups of participants with one group
being tested on a conforming web forum design, and the other group on a non-
conforming web forum design. The variable that was being manipulated
(independent variable) was the conventionality of the web forum and the type of task
participants were answering. The variables that were being measured to look at the
effect of conventionality on performance (dependent variables) were the task
completion times, the number of right answers, the number of navigational
adjustments (revisits, backtracking and loopback) done by each participant and the
number of different pages each participant visited in order to complete the tasks.
Participants
10 participants, all graduates or postgraduates, volunteered for this pilot experiment.
They were randomly allocated to one of the experimental conditions in such a way
that 5 participants were tested with the conforming web forum and 5 participants
with the non-conforming web forum. All participants were frequent web forums
users and although their nationalities varied, they were all fluent in English. The
participants who volunteered for this experiment had not taken part in any previous
experiments in this thesis.
Materials
The web forum designs for the conforming and non-conforming conditions were
based on the templates used in the previous experiment (see Chapter 6) which
originated from the survey that looked into the web forum elements and their most
common position, configuration and symbolization values (see Chapter 5).
As seen earlier, the content used in the material for the experiment 4, described in
Chapter 6 (extracted from a PDA forum) produced some interpretation problems
since some participants were not familiar with the technical PDA terminology.
Therefore, the content used in these experimental forums was obtained from an
online book discussion forum. This choice of content was made based on the
assumption that books are a topic that the participants would be more familiar with,
consequently minimizing any comprehension difficulty due to the vocabulary used.
194
In the experiment described in Chapter 6 which tested user performance when

searching for information in a forum, some participants expressed the view that the
forum designs were not very good and needed improvement. Although the forums
were systematically designed based on the results of the survey, some minor
alterations could be done to attempt to improve the visual aspect of the designs. So
for these new designs the colour scheme was changed from shades of green to shades
of blue, maintaining the figure/background contrast and saturation values. Some of
the icons used in the previous experiment for the conforming and non-conforming
designs were also replaced by other icons commonly occurring on the Web, but with
better resolution and more aesthetically pleasing.
A screen-capture software was set to capture an image of the screen on each click the
participants made. Thus whenever a participant clicked with the mouse, a screen
would be saved on the hard drive along with the time when the click happened. To
solve the time measure at the end of the tasks, a pop-up window with a button
labelled ‘finished’ was created so that when the participants finished the task they
would click on the button signalling that they were satisfied with the answer found
and were ready to move on to the next task. This button also gave the opportunity
for the participants to quit any tasks to which they could not find the answer.
One of the difficulties in the previous experiment (see Chapter 6) for the
experimenter was stopping the measurement of time precisely when the participant
ended each task. The difficulty was due to several reasons e.g. the participants did
not state when they have finished the task, or they would say they had finished but
then change their mind; there would be a lapse between the moment the participant
verbalized finishing the task and the moment the experimenter stopped the measure.
So, in order to improve the reliability of the time measurement on each task, a very
small and discrete software stopwatch was strategically placed on the screen so that
every time a mouse click occurred and a screen was captured, the time of the click
was registered. To minimize the effect that the stopwatch may have on participants
by adding time pressure, the following measures were put in place: the stopwatch
was placed on the bottom right corner of the browser and was set to have the same
colour as the browser frame so it blended smoothly with the interface (see Figure 73).
195
FIGURE 73 - SAMPLE OF THE EXPERIMENTAL SETTING USED FOR THE PILOT STUDY AND EXPERIMENT 5.
Tasks design
To have a clearer picture of the performance changes during the experiment, this
pilot had a greater number of tasks than the previous experiment. It consisted of 10
tasks each requiring a different minimum number of steps to be taken in order for
each one to be solved. The reason for the difference in the number of steps is mainly
due to the way in which forums are structured, with a forum index page, thread
pages and post pages. Since the answers could be found in any of these three places,
the minimum number of steps to get to them would vary.
The steps were based on the taxonomy mentioned earlier, devised by Byrne et
al.(1999) which establishes the processes that are involved with a World Wide Web
task.
Since this experiment only involved searching for information, the only steps used
were: (a) locate information on page; (b) use information and (c) go to a page. Each of
these main processes was considered a step.
196
The number of steps necessary to complete a task ranged from 5 to 9 depending on

the task. Even though longer tasks may lead to more participants quitting the task
than shorter ones, it is interesting to look at how each task type and length can affect
user performance. This will allow a better selection of task style and length (number
of steps) for the main experiment, so that task styles which are too easy and the ones
which are too difficult can be detected and not used.
As in the previous experiment (see Chapter 6) there were two types of task: one
(called content tasks) required the participant to look for the answers on the written
content of the forum (e.g. ‘What is the difference between a 1st edition and a 1st trade
edition?’), the other type (called interface tasks) required that at some point in the
search for the answer participants would need to obtain some information from the
interface (e.g. ‘Which thread in the ‘Books wanted’ topic has the last post dated Nov
22?’). Extra care was taken to design interface tasks that would only require the
participants of the non-conforming condition to use the icons that were easily
recognized in the previously mentioned survey (see Chapter 5).
A total of 10 tasks, 6 content-related and 4 interface-related, were created. Due to the

icon identification limitation described above the number of interface tasks was
smaller than the content tasks. Task orders were randomized for each participant.
The task questions can be seen in Appendix N.
7.2.3. Procedure
Each participant was individually tested in a quiet room. Initially the experiment was
explained to the participant. Participants were told that the test consisted of finding
answers to questions on a web forum and it was explained how to proceed when
they had found an answer. They were told to behave as they would normally behave
when browsing the Web. They were told that, as in a real situation, they could quit a
task whenever they wanted.
When participants were ready to begin, the experimenter handed them the first task
printed on a sheet of paper. When the participants finished reading the task and
looked at the computer screen, the experimenter started the time measurement and
screen capture. Participants then searched within the forum for the answer to the task
and when they thought they had found the answer they would select it (by clicking
and dragging the mouse over it), and would click on the ‘finished’ button in the pop-
up window. By clicking on the button participants were taken back to the homepage
197
of the forum and were ready to start a new task. The experimenter wrote down the
time displayed on the stopwatch and whether the participant got the question right
(this was a safeguard for any system failure).
These procedures were repeated for every task for each participant in both groups.
7.2.4. Results
Time
The mean task completion time for the conforming group was 76.8s (SD=33) and
87.6s (SD=23.7) for the non-conforming group. A one-tailed Mann-Whitney test
found no significant difference in the times taken by each group (conforming and
non-conforming) (W=23.5, n=5 and p=0.23).
To look at how task completion time of the participants in both groups progressed
over time, the results were averaged for each group in the order each task was
answered by each participant. This was done because tasks were randomised and
participants did not receive tasks in the same order. The results for the non-
conforming group (Figure 74) and conforming group (Figure 75) are illustrated in the
graphs below.
FIGURE 74 - TASK COMPLETION TIME FOR THE NON-CONFORMING GROUP.
Non-
conforming
198
FIGURE 75 - TASK COMPLETION TIME FOR THE CONFORMING GROUP.
Conforming
On these graphs it is possible to notice that that the group which used the non-
conforming design showed a performance improvement throughout the experiment,
while the group which used the conforming design apparently had no change in
performance. A Mann-Kendall trend test indicated a significant downward time
trend on the non-conforming group (Z=-27, n=5, p<0.01). The Mann-Kendall test for
the conforming group did not reveal any significant trend (Z=-13, n=5, p=0.13).
Efficiency indicator
The efficiency indicator was calculated as proposed by Smith(1996) dividing the
number of nodes required to complete a task by the number of different nodes
visited. Both groups showed similar efficiency with the conforming group showing a
mean efficiency indicator of 0.52 (S.D.=0.187) and the non-conforming group
showing a mean value of 0.55 (S.D.=0.188). A Mann-Whitney test showed the
difference not to be significant (W=2216.0, n=5, p=0.623).
Tasks
To have an idea of the difficulty of the tasks in this experiment a graphs of the
average times to complete each task for both the non-conforming (Figure 76) and
conforming groups were plotted.
199
FIGURE 76 - MEAN TIME TO COMPLETE EACH TASK IN THE NON-CONFORMING GROUP.
Non-
conforming
FIGURE 77 - MEAN TIME TO COMPLETE EACH TASK IN THE CONFORMING GROUP.
Conforming
Looking at the graphs it is possible to notice large variabilities in both groups on

most of the tasks. Finding tasks which show different performance results between
the two groups could indicate a specific kind of tasks for which one of the layouts
would be more adequate. To verify if any of the tasks showed this effect, the two
graphs were overlapped (Figure 78). From this overlapping it is possible to see that
apparently the only task where a significant difference between the two groups
200
might show is task B. However a Mann-Whitney test on the time to complete task B
on both groups showed that even though the difference was not significant there was
a trend towards significance which indicates that with a higher sample size this
difference may become significant (W=19.0, n=5, p=0.097). Therefore it is worth
having task B in the next experiment.
FIGURE 78 - OVERLAPPED GRAPHS OF THE MEAN TIME TO COMPLETE EACH TASK

ON THE CONFORMING AND NON-CONFORMING GROUPS.
Conforming
Non-
conforming
So, to better analyse the global effect of each task the mean time to complete each task
overall was plotted (Figure 79).
FIGURE 79 – OVERALL MEAN TIME TO COMPLETE EACH TASK.
201
From the graph it is possible to see that some tasks were answered very quickly,
some took a long time to be answered and some showed high variability. These tasks
are shown in Table 38. From the graph in Figure 79 it is clear that task H was quite
difficult, taking participants longer to complete, and task I was quite easy taking little
time to be answered and presenting very little variability. Task D showed a lot of
variability among participants despite taking on average a similar time to complete
as task B. This variability indicates that some participants found this task quite easy
and others found it quite difficult. This variability could be due to the fact that not all
the participants were able to associate the term ‘Frequently Asked Questions’ with
the menu item FAQ (in the conforming design), or the question mark icon (in the
non-conforming design), thus making the task quite difficult.
Task I was not only easy for both groups but also showed virtually no variability.
This was probably due to the way in which the wording of this task was structured;
it contained all the path the participant had to navigate through to reach the page
where the answer was and also told the participant where to look for an answer.
Furthermore, the answer to task I was only one forum level away from the point
where participants started the search, so that with just one click on the right link, the
participants were already on the page where the answer was.
TABLE 38 - SUMMARY OF THE EXPERIMENTAL TASKS WHICH WERE ANSWERED

TOO QUICKLY, TOO SLOWLY OR SHOWED GREAT VARIABILITY.
Task Question
H How much is a T.S. Eliot signed 1st edition worth?
I Which thread under the “What I’m reading now” has the greatest number of views?
Find the answer for this frequently asked forum question: “How do I prevent my user name from
D
appearing in the online user’s listing”?
Which forum member wrote the last post on the thread “Trimming pages” under the “Adding value
C
to your books” topic?
B What is the difference between a 1st edition and a 1st trade edition?
Time per task type

To analyse the time taken to complete each task type a one within-subjects, one
between-subjects ANOVA was applied to the data. Results showed a main effect of
task type but no interaction of conventionality and task type (see Table 39).
202
TABLE 39 - ONE BETWEEN-SUBJECTS, ONE WITHIN-SUBJECTS

ANOVA RESULTS FOR TIME TAKEN TO COMPLETE THE TASKS.
Between subjects
Source SS df Mean Squares F-Ratio p-value
Conventionality 31,203.420 1 31,203.420 0.630 0.450
Error 396,353262 8 49,544.158
Within Subjects
Source SS df Mean Squares F-Ratio p-value
Task type 1,037,091.533 1 1,037,091.533 19.676 0.002
Task type x Conventionality 26,229.076 1 26,229.076 0.498 0.501
Error 421,659.823 8 52,707.478
Examining the graph (Figure 80) it is possible to see that interface tasks were
answered more quickly than content tasks in both the conforming and non-
conforming conditions. It is also possible to perceive that interface tasks took
virtually the same time to complete in both the conforming and the non-conforming
groups.
FIGURE 80 - MEAN TIME TO COMPLETE INTERFACE AND CONTENT TASKS BY CONDITION.
Conforming Non-conforming
Right answers per task type

A one between-subjects, one within-subjects ANOVA of the number of right answers
achieved by each group on each task type showed again a main effect of task type.
No interaction of task type and conventionality or main effect of conventionality was
detected.
203
TABLE 40 - ONE BETWEEN-SUBJECTS, ONE WITHIN-SUBJECTS ANOVA ON THE NUMBER

OF RIGHT ANSWERS BETWEEN THE CONFORMING AND NON-CONFORMING GROUPS.
Between subjects
Source SS df Mean Squares F-ratio p-value
Error 5.600 8 0.700
Within subjects
Task Type 14.450 1 14.450 28.950 0.001
Task Type x Conventionality 0.050 1 0.050 0.100 0.760
Error 4.000 8 0.500
Examining the graph for the number of correct answers per task type in both groups
it is clear that participants in both groups achieved a better overall score on content
tasks than on interface tasks.
FIGURE 81 - MEAN NUMBER OF RIGHT ANSWERS PER TASK TYPE FROM

THE CONFORMING AND NON-CONFORMING GROUPS.
Right answers per second rate (RPS) per task type

As in the experiment described in Chapter 6, the rate of right answers per second was
also examined. The rate was obtained by dividing the number of right answers by the
time to complete each task. One between-subjects, one within-subjects ANOVA
(Table 41) applied to the data revealed once more a main effect of task type on
performance.
204
TABLE 41 - ONE BETWEEN-SUBJECTS, ONE WITHIN-SUBJECTS ANOVA REUSLTS FOR THE

RIGHT ANSWERS PER SECOND (RPS) RATE IN THE CONFORMING AND NON-CONFORMING GROUPS.
Between subjects
Error 6.611 8 0.826
Within subjects
Task Type 5.842 1 5.842 7.827 0.023
Task Type x Conventionality 0.042 1 0.042 0.061 0.812
Error 5.603 8 0.700
Looking at the graph in Figure 82 it is possible to see that the overall performance
was better on interface task than on content tasks.
FIGURE 82 - RIGHT ANSWERS PER SECOND (RPS) RATE PERT TASK TYPE ON
THE CONFORMING AND NON-CONFORMING GROUPS.
Quits
Both groups had the same number of task quits throughout the experiment (5 in each
group). While the participants who used the conforming forum took an average of
227 seconds before giving up on a task, participants who used the non-conforming
version gave up on the tasks on average after 195.8 seconds. A Mann-Whitney test
showed that this difference was not statistically significant (W=32, n=5, p=0.40).
205
Right answers
Each of the 5 participants in each condition was presented with 10 tasks totalling 50
tasks executed per condition. The conforming group totalled 40 (out of 50) right
answers, while the non-conforming group totalled 43 right answers. A two-tailed
Mann-Whitney test showed no significant difference in the scores (W=23.5, n=5 and
p=0.456).
However, when the number of right answers was analyzed as participants

progressed through the test, a Mann-Kendall analysis showed no significant trend for
the conforming group (Z=0.39, n=5, p=0.34) but found a significant uptrend for the
non-conforming group (Z=2.48, n=5, p=0.006). Looking at the graph for the non-
conforming group (Figure 83) it is noticeable that after the fifth task the number of
right answers constantly remains at the value of 10 which is the maximum value
possible. Interestingly enough the same does not occur in the conforming group
(Figure 84), where the number fluctuates between 8 and 10.
FIGURE 83 - NUMBER OF TOTAL RIGHT ANSWERS OVER TIME AND FIT CURVE FOR THE NON-CONFORMING GROUP.
Non-conforming
Less conv entional
10 Fit curv e
8
Nº of right answers
1st 2nd 3rd 4th 5th 6th 7th 8th 9th 10th
Task (first to last)
206
FIGURE 84 - NUMBER OF TOTAL RIGHT ANSWERS OVER TIME AND FIT CURVE FOR THE CONFORMING GROUP.
Conforming
Navigational adjustments
The screen-capture file for one participant of the non-conforming group was
corrupted and could not be retrieved or analyzed so that the information on
navigation was based on four participants in the non-conforming group and five
participants in the conforming group.
Backtracking
In this experiment, participants tested on the conforming forum used backtracking as

their main navigational correction strategy (57 times out of 81 total corrections) while
participants tested on the non-conforming forum used backtracking only 27 times out
of a total of 71 corrections. However, the difference in the number of times
backtracking was used in each group was not significant (W=29, n=5, p=0.39)
Revisits
Each time during navigation that a participant returned to a page that they had
already been to, was considered a page revisit. Participants who used the non-
conforming forum had an average of 19.5 page revisits per participant while
participants who used the conforming forum had an average of 15.2 page revisits per
participant. A Mann-Whitney test showed, however, that this result was not
significant (W=23, n=5, p=0.71). This lack of significance might be due to the high
variability within the groups (Figure 85).
207
FIGURE 85 - MEAN REVISITS FOR THE CONFORMING AND NON-CONFORMING GROUPS.
7.2.5. Discussion
Task completion time

No significant differences have been found in the total time taken for each group to
complete the tasks. This could have happened due to the participants of the non-
conforming group having developed a performance improvement throughout the
test, as was hypothesized. Looking at the progression of the time required to
complete the tasks in the non-conforming group, a downward trend is clear. This
means that as the test progressed, participants from this group found answers for the
tasks more quickly. This seems to indicate that these participants, although familiar
with a certain set of visual conventions, when using a website which violated their
expectations they were quick to learn the new layout.
No trend was noticeable for the group which used the conforming forum. This was
expected since these participants did not have to learn any new layout and therefore
should show no improvement or worsening of performance.
Right answers
There were no significant differences in the total number of right answers in each
group. But the experiment indicated that while there was no significant trend in the
conforming group, the non-conforming group showed a slight but significant
upward trend in the number of right answers. This result allied with the downtrend
208
in time seem to suggest that the non-conforming group had an improvement in

performance throughout the test.
Task type
The results showed that in both groups participants took longer to answer content
tasks than they did to answer interface tasks. But results also show that participants
in both groups answered more content tasks correctly than they did interface tasks.
When results were combined into the RPS rate it shows that there were no difference
in the performance between the groups in each type of task. In other words, the
performance on interface tasks between the conforming and non-conforming groups
was equivalent. The same happened with the content tasks.
Overall, the RPS rate was higher on interface tasks which is expected, since finding
information within content requires that participants read or skim through content
and comprehend what they are reading, which takes longer, whereas retrieving
information from the interface is just a matter of locating it. And although the non-
conforming forum violated the visual conventions of the genre participants were
familiar with, it still conformed with the linguistic, structural and functional
conventions of the web forum genre. This may have simplified the task of retrieving
information from the interface.
Efficiency indicator
Although the efficiency indicator (Smith, 1996) is a valid measure of efficiency
(Larson & Czerwinski, 1998; Otter & Johnson, 2000; Selvidge, et al., 2002) the absence
of a significant difference in the results in this pilot study raises the question of
whether it should be applied in the main experiment. Considering that in the main
experiment backtracking, revisits and loopbacks will be measured as well as right
answers and time taken it seems there is no need to go through this analysis since
similar conclusions can be drawn from the other variables.
Tasks
From observations during the experiment, it was clear that the main reason for the
difficulty of answering task H was due to the wrong and misleading wording of the
task (‘How much is worth a T.S. Eliot signed 1st edition?’). The forum had two
209
different topics on the index page where participants looked for the answer, one
entitled ‘First editions and signed books’ – where the answer really was, and the
other entitled ‘How much is my book worth?’. Because of the wording of the
questions, participants tended to first go in the ‘How much is my book worth?’ topic
and that delayed the answer. This indicates that this task should not be used in the
main experiment. Also, greater care should be taken with the wording of the tasks so
as not to mislead the participants since doing so would add a confounding variable
to the experiment.
Task I (‘Which thread under the “What I’m reading now” has the greatest number of
views?’) was answered quite quickly and with little variability among participants
and could be explained in several ways. First, it required a small number of steps to
complete since the answer to the task was in the threads page (the second level of the
forum). It also showed the participant exactly where to go to find the answer (‘What
I’m reading now’ topic). When participants got to the second level (where the answer
was) they were faced with only 12 threads on a single web page so finding the
answer was straightforward.
Finally, the high variability of task D (‘Find the answer for this frequently asked
forum question: “How do I prevent my user name from appearing in the online user’s
listing”?’) could have different explanations; one could be that some participants did
not know that the menu item ‘FAQ’ was an acronym for ‘Frequently Asked
Questions’ (in the conforming version) or that the question mark icon stood for the
FAQ section (non-conforming version). Another explanation would be that
participants simply did not know where to find the answer to a ‘Frequently Asked
Question’.
In summary, the problems found in the design of the tasks can be categorized into
the following types:
! Difficult and misleading wording of the task which could lead participants
down the wrong path to find the answer. This adds to the difficulty of the
task.
! Tasks which easily indicate the path to the answer and where the participant
should find it on a page. This makes the task easy.
! Answers which are located quite close to the start page, requiring little
navigation. In other words, the answers are not deep enough within the web
site structure.
210
! Answers which are located in pages with too few textual elements making it
very simple for the participant to find the answer.
! Participants not being familiar enough with the terminology used in web
forums.
Backtracking and revisits

The small number of participants on each condition could have led to the lack of
significant results for the usage of backtracking or the number of page revisits on this
test. Since these two variables are considered good indicators of disorientation (Xu, et
al., 2001; Yatim, 2002) they will be considered in the next experiment. However a
perceived disorientation measurement through a questionnaire will be added to the
main experiment.
7.2.6. Conclusion
The results from this pilot indicate that participants familiar with a certain set of
visual conventions might learn a layout which violates these conventions quite
quickly. This could explain the difficulty in finding significant performance
differences when comparing user performance on conforming sites with performance
on non-conforming sites. However, further investigation with a larger sample is
needed in order to confirm these results and verify if the violation of visual
conventions has any further impact on user disorientation and performance.
The lack of significant results on the disorientation measures (backtracking and

revisits) may indicate, as suggested by Ahuja and Webster (2001), that perceived
disorientation may be a more reliable predictor of disorientation than actions
themselves. Therefore it might be a good strategy in the main experiment to measure
not only the physical aspects of disorientation, but also the perceived disorientation,
as suggested in Gwizdka and Spence (2007) and Herder (2003).
211
7.3. The main experiment

7.3.1. Objectives
Similarly to the pilot study, this experiment will attempt to verify if participants
using a non-conforming forum will have their performance improved as they
progress through the test compared to participants using a conforming one. But
based on what the pilot showed, this experiment will go further by testing whether
the participants who used a non-conforming forum will continue to show a
performance improvement when using the same non-conforming interface on
another web forum. The effect of the type of task on the performance of participants
using both conforming and non-conforming forums will also be analysed. This
experiment will also try to confirm if not conforming to conventions will result in
user disorientation.
7.3.2. Method
Design
This experiment used two independent groups of participants, with one tested using
the conforming forum design and the other using the non-conforming forum design.
As in the pilot study and the experiment described in Chapter 5, each group had to
perform a number of tasks which were divided into two types: content and interface.
The variables that were being measured in this experiment (dependent variables)
were the task completion time, the number of wrong answers, the number of
navigation adjustments done by each participant and the number of page revisits
done of participant.
This test was divided in two parts with a 5 minute interval between them. In the first
part, participants were tested on the same web forum used in the pilot study. In the
second part participants were tested on the same web forums used in the experiment
described in Chapter 6 (one conforming and one non-conforming). The group of
participants which was tested on a conforming forum in the first part of the
experiment was tested on a conforming forum in the second part as well. The group
of participants which was tested on a non-conforming forum in the first part was
tested on a non-conforming forum in the second part.
212
Participants
Twenty-eight participants, all graduates or post-graduates from different disciplines
volunteered for the experiment. Participants were randomly allocated to one of the
experimental groups in a way that each group contained 14 participants. Volunteers
for this study had not previously participated in any of the experiments conducted so
far neither in the pilot study.
All participants were web forum users having reported continuously using at least
one web forum site. Participants were fluent in English (either as first or second
language).
Materials
First part
As mentioned above, for this part the forums used were the same as those used in the
pilot study and no alterations were made to their designs.
Second part
Since participants for this experiment were entirely new to this research and did not
have any contact with previous experimental material, it was decided that the forum
used for this part would be the one used in the experiment of Chapter 6. To keep the
interface in this part consistent with the interface of the previous material the same
modifications that were considered for the first part material (see 7.2.2, materials
section) were made on this forum.
To keep consistency with the procedures that participants used in the first part, the
same pop-up windows system which allowed participants to indicate the end of a
task was added to the material for the second part.
Tasks design
From the pilot results it was possible to see that the design of the tasks would need
rethinking for the main experiment. Because the main experiment consists of two
parts it was decided that in the first part the number of tasks would be the same as in
the pilot study. The second part of the study was shorter than the first, so only six
tasks were designed for this part.
213
From the results of the pilot study, the following measures were taken when
designing the tasks for this experiment:
! the wording used was as simple as possible and whenever possible some key
words were added in the sentence to act as cues to the places where the
answers were. For example, the answer to the task ‘Identify which company
now owns the publisher called Doubleday’. (Obs.:italics added for
highlighting purposes only and were not present in the experimental task
given to participants) would be found in : ‘Help on identification >
Identification of publishers’.
! whenever possible tasks would not indicate the entire path to the answer and
tasks were designed to have answers on pages with not too little textual
elements.
! tasks would be designed whenever possible to have answers as far away as
possible from the forum start page.
Again in this study tasks were of two main types: content tasks and interface tasks.
However, in order to better analyze any possible interactions between question type
and conventionality, the number of interface and content tasks was made the same in
both parts, with 5 content and 5 interface tasks in the first part and 3 content and 3
interface tasks in the second part.
Because giving participants all tasks of one type, followed by all tasks of another
type, could introduce a confounding variable (task order), tasks were
counterbalanced in an CI-IC fashion, in other words, one participant would be given
all their tasks in this order: a content task first, followed by an interface task (CI) and
so on, while another participant would do an interface task followed by a content
task (IC) and so on. This would counterbalance any effects that starting with a
particular task type might have on the performance of participants. Also, within the
counterbalancing scheme, tasks were also randomized.
The counterbalancing scheme was maintained in the second part of the experiment
so that a participant who started with an interface task in the first part would start
with an interface task in the second part and a participant who started with a content
task in the first part would start with a content task in the second part.
214
The tasks presented to participants in both parts of the test can be seen in Appendix
O.
Disorientation questionnaire
Based on the work by Ahuja and Webster(2001) a questionnaire to assess the
perception of users regarding their disorientation when using the web forums was
created. Although the same ideas given by the authors were followed, this
questionnaire contained fewer questions than the original questionnaire designed by
Ahuja and Webster. This was because the original questionnaire had some of the
questions repeated but reverse-coded, so for reasons of simplicity and in order not to
tire the participants of the experiment excessively, only 5 of the authors’ original
questions were included in the questionnaire. The answers to the questionnaire were
arranged in a 5 point Likert scale but no numbering was placed on the scale, only
squares with the respective answer below it. Table 42 shows the questions used in the
questionnaire. For a view of the complete questionnaire used in the present study
refer to Appendix P.
TABLE 42 - QUESTIONS IN THEIR ORDER OF APPEARANCE IN THE QUESTIONNAIRE.
Question 1 I felt lost / disoriented
Question 2 I felt like I was going around in circles
Question 3 It was difficult to find a page I had previously viewed
Question 4 Learning to use the site was easy
Question 5 The site was easy to navigate
Questions 1, 2 and 3 asked participants about their perceived sense of disorientation

and questions 4 and 5 were concerned with their perceived ease of learning the
interface.
7.3.3. Procedure
Participants were tested individually and when a participant entered the room, they
were asked to sit in front of the computer and the experiment was explained. They
were told that the test would be divided into two parts with a 5-minute break
between each part. They were informed that each part consisted basically of looking
for answers to questions within a web forum and that there would be a different
215
forum for each part. It was then explained that whenever the participant found a
satisfactory answer they should select the text (with a click and drag mouse
movement) and click on the finished button that was on the pop-up window to the
left of the browser.
Participants were told to look for the answers as fast and as accurately as possible.
They were told that there were no questions with double meanings and all the
answers were on the forum. Participants were told to behave as they would in a
normal web environment, and were allowed to give up on the search for an answer
at any time they wanted. The researcher reinforced to the participant that the
experiment was concerned with testing the design of the site and not the participant
so they should not feel pressured.
When the participant was ready the web forum was presented and the paper with
the printed task was given to them. The participant would read the paper and as
soon as they finished reading and looked at the screen the stopwatch would be
started. Screen-capture software was set to capture a copy of the current screen
whenever the participant clicked the mouse.
Whenever a participant found an answer they would select it and click on the
‘finished’ button that was located on a window to the top left of the web browser.
That action would take them back to the first level of the forum and another paper
with a task would be given to them and the procedure would be repeated until all
tasks for the first part were completed. Then the participant would be requested to
answer the questionnaire on their perceptions of disorientation and ease of use of the
forum in the first part. After that the participant would take a 5 minute break while
the researcher prepared the material for the second part.
The procedures for the second part were the same as the procedures in the first part.
When the participant completed all the tasks from part two they were requested to
fill in the questionnaire on their perceptions of disorientation and ease of use of the
forum in the second part.
216
7.3.4. Results for part 1 of experiment 5
Time
The group which was tested on the conforming forum took on average 594.1 seconds
(SD=192.1) to complete the first part of the test while the group which was tested on
the non-conforming forum took on average 750.5 seconds (SD=304.4) to complete the
first part of the test. This difference was not significant (F(1,26)=2.64, p=0.116).
Looking at the time taken to complete the tasks over the course of the first part, it is
evident that both the non-conforming (Figure 86) and the conforming (Figure 87)
groups seemed to have reduced the time to complete the tasks over time. A Mann-
Kendall trend test showed a downward trend in both the non-conforming group (z=-
2,14, p=0.015) and the conforming group (z=-2.86, p=0.002).
FIGURE 86- MEAN TIME TO COMPLETE EACH TASK OF THE NON-CONFORMING GROUP
OVER THE COURSE OF THE FIRST PART OF THE EXPERIMENT AND THE CORRESPONDENT FIT CURVE.
Non-conforming
217
FIGURE 87 - MEAN TIME TO COMPLETE EACH TASK OF THE CONFORMING GROUP

OVER THE COURSE OF THE FIRST PART OF THE EXPERIMENT AND THE CORRESPONDENT FIT CURVE.
A trend curve fit test showed that the best fit for both cases was a quadratic trend
model, but despite both groups having a quadratic downtrend it is possible to tell
from the graphs that the decrease in the time taken to complete the tasks happens
faster in the non-conforming group than in the conforming. Initially participants in
the non-conforming group take much longer to complete the tasks than participants
in the other group. However, towards the end of the experiment the difference in
time taken to complete the tasks between the groups is much less.
Right answers and quits

There was no significant difference in the number of right answers in the conforming
and non-conforming group (F(1,26)=0.76, p=0.390). There was not a significant
difference in the number of times participants in each conditions quit a task
(F(1,26)=0.43, p=0.517).
Right answers over time

The number of correct answers obtained by the participants of the non-conforming
group (Figure 88) increased as the test progressed while there was no particular trend
in the conforming group (Figure 89).
218
FIGURE 88 - NUMBER OF RIGHT ANSWERS IN THE NON-CONFORMING GROUP THROUGH THE TEST.
Non-
conforming
FIGURE 89 - NUMBER OF RIGHT ANSWERS IN THE CONFORMING GROUP THROUGH THE TEST.
Conforming
A Mann-Kendall trend test showed a significant upward trend in the number of right
answers for the participants of the non-conforming group (z=1.720, n=10, p=0.042).
The test showed no upward (z=1.106, n=10, p=0.134) or downward (z=1.106, n=10,
p=0.865) trend in the number of right answers of the conforming group. The results
of a trend curve fit test on the progression of right answers on the non-conforming
219
group showed a very small difference between the fit accuracy measures on the
linear and quadratic model although the Quadratic model was the best fit.
TABLE 43 - COMPARISON BETWEEN THE LINEAR AND QUADRATIC MODELS MEASURES

OF ACCURACY FOR THE NON-CONFORMING GROUP.
Measures Quadratic Linear

MAPE (Mean Absolute Percentage Error) 24.0057 24.0871
MAD (Mean Absolute Deviation) 1.8797 1.8873
MSD (Mean Squared Deviation) 4.8829 4.9018
This means that although the equation which generates the curve which best fits the
data is quadratic, its exponential component has less influence than the linear one
meaning that the performance increased practically in a linear way. This causes the
quadratic fit curve to behave almost as a straight line as can be seen in Figure 88.
Right answers per second (RPS) rate

As it was previously done in the past two experiments the right answers per second
rate (RPS) was obtained by dividing the number of right answers by time. However,
to simplify calculations without compromising the analysis, this time the ratio was
multiplied by a constant. The formula to achieve the RPS is illustrated in Figure 90.
FIGURE 90 - RIGHT ANSWERS PER SECOND RATIO FORMULA.
A one-way ANOVA on the RPS ratio for the first part of the test showed a significant
difference (F(1,26)=5.70, df=1, p=0.025) between the conforming (M=1.30, SD=0.41) and
non-conforming group (M=0.98, SD=0.30). The participants in the non-conforming
group showed a lower mean RPS than the conforming group which indicates that in
this part of the test their performance was not as good as the other group of
participants.
220
RPS rate over time

A Mann-Kendall trend test on the right answers per second data showed no trend in
the conforming group (z= -1.25, pup= 0.895, pdown= 0.105) and an upward trend (Figure
91) in the non-conforming group (z=2.50, p=0.006).
FIGURE 91 - PROGRESSION OF RIGHT ANSWERS PER SECOND IN THE NON-CONFORMING GROUP.
Non-
conforming
A trend curve fit test showed the linear trend to be the best fit for the data.
Overlapping the RPS progression graphs of the conforming and non-conforming
groups it is possible to see that by the end of the test, the performance of the non-
conforming group was close to the performance of the conforming group (Figure 92).
221
FIGURE 92 - PROGRESSION OF THE RPS RATIO FOR BOTH GROUPS IN THE FIRST PART OF THE TEST.
Conforming
Non-conforming
Looking at the graphs of the two groups it is noticeable that towards the end of the
first part, the RPS graphs for the two groups are very close. This happens after the
point where the non-conforming trend line crosses the graph of the conforming
group. After that there are three RPS scores for each group. A post-hoc analysis on
these points showed no significant statistical difference between the last 3 RPS scores
of the two groups (F(1,26)=1.85, p=0.186). This result confirms that by the end of the
first part the performance of the two groups was equivalent.
Revisits
Participants who used the non-conforming forum performed more page revisits
(M=20.36) than the participants who used the conforming forum (M=7.93). A Mann-
Whitney test adjusted for ties on the number of page revisits carried out by each
group showed that the difference was significant (W=140.5, n=14, p=0.0043).
Backtracking
Again, participants who used the non-conforming forum used the back button more
times (M=17.36) than participants who used the conforming forum (M=5.07). A
Mann-Whitney test adjusted for ties on the number of backtracking actions carried
222
out by each group showed that the difference was significant (W=137.5, n=14,
p=0.003).
Loopback
A Mann-Whitney test adjusted for ties showed there was no significant differences in
the number of times participants used the loopback strategy to go back to a page
where they have been before (W=224.0, n=14, p=0.323).
RPS rate per task type

Analyzing the ratio of right answers per second (RPS) per task type on each of the
counterbalancing options (CI and IC) showed no carryover effect on the time to
complete tasks (Table 44).
TABLE 44 - AVERAGE RIGHT ANSWERS PER SECOND FOR EACH

TYPE OF TASKS ON EACH COUNTERBALANCING ORDER.
Content task time Interface task time
Content tasks first (CI) 0.92 2.25
Interface tasks first (IC) 0.96 2.53
An analysis of the RPS to complete each task type and the conventionality of the
forums using a one between-subject, one within-subject ANOVA showed no main
effects of task type (F(1,26)=2.73, p=0.111) or conventionality (F(1,26)=2.31, p=0.140) but
found a significant interaction between task type and conventionality (F(1,26)=46.47,
p<0.01). This type of interaction is called a disordinal interaction and in it the group
differences tend to reverse their sign at some level of the other variable (Figure 93).
As expected, in this part of experiment 5, participants who used the conforming
website performed better on interface tasks than on content tasks. With the
participants who used the non-conforming website the situation was reversed, they
performed better on content tasks than on interface tasks.
223
FIGURE 93 – AVERAGE RIGHT ANSWERS PER SECOND RATE (RPS) ON CONTENT

AND INTERFACE TASKS PER GROUP IN THE FIRST PART OF THE EXPERIMENT IN THE FIRST PART.
Number of quits per task type

Analyzing the number of quits per task type on each group with a 1-between subjects
1-within subjects ANOVA showed no main effects of task type (F(1,26)=0.00, p=1.00) or
conventionality (F(1,26)=0.432, p=0.517) but found a significant disordinal interaction
(F(1,26)=24.27, p<0.01) between the task type and the conventionality of the forum
(Figure 94).
FIGURE 94 - MEAN NUMBER OF QUITS FOR EACH GROUP ON EACH TYPE OF TASK.
224
In this test participants of the conforming group quit fewer times on interface tasks
than on content tasks, and with the participants of the non-conforming group the
situation was reversed: they quit fewer times on content tasks than on interface tasks.
This could explain the RPS rates shown before; however, further examination of the
data reveals that participants in the conforming forum quit more often on content
tasks than on interface tasks (Figure 94). This was followed by them having
completed fewer content tasks correctly when compared with interface tasks (Figure
95). This consequently led to a lower RPS rate.
FIGURE 95 - MEAN NUMBER OF RIGHT ANSWERS PER GROUP FOR EACH TASK TYPE.
But the inverse is not true for the non-conforming group. Although participants quit
more on interface tasks than content ones, they did not complete more content tasks
correctly compared with interface tasks. In fact, they still completed more interface
tasks correctly than content tasks. Therefore their higher RPS rate on content than on
interface tasks is explained by the fact that the participants who used the non-
conforming forum completed content tasks much faster than they completed
interface tasks, as can be seen in Figure 96.
225
FIGURE 96 - AVERAGE QUITS ON CONTENT AND INTERFACE TASKS PER GROUP.
Navigational adjustments per task type
Revisits
Overall participants made more page revisits in the content type tasks (M=10.75,
SD=12.21) than on interface tasks (M=3.42, SD=5.14); this result was significant
(F(1,54)=8.55, p=0.005). There was no interaction of revisits and task type, and nor was
there a main effect of conventionality.
Backtracking
Participants also used the back button more when answering content tasks (M=8.54,
SD=11.28) than when answering interface tasks (M=2.67, SD=4.21). A ANOVA
showed that this result was significant (F(1,54)=6.63, p=0.013). There was neither
interaction of backtracking and task type, and nor was there a main effect of
conventionality.
Loopback
Loopback was also used more when participants were answering content tasks
(M=1.29, SD=1.90) than when they were answering interface tasks (M=0.32, SD=0.72).
This result was significant (F(1,54)=6.29, p=0.015).
Questionnaire
In the first part participants in the conforming and non-conforming group only
differed in their perceptions of the ease of learning the website (see Question 4 on
226
Table 42). Participants who used the conforming version thought it was easy to learn
to use their site, while the other group neither agreed nor disagreed. A Mann-
Whitney test comparing the answers of the conforming (Median=4) and non-
conforming (Median=3.5) groups for this question showed that this result was
significant (W=242.5, n=14, p=0.046).
7.3.5. Results for part 2 of experiment 5
Time
A one-way ANOVA showed no difference in total time to complete the tasks for the
second part of the test (F(1,26)=0.77, p=0.388). The conforming group showed an
average of 253.8s (SD=123.2) and the non-conforming group an average of 298.4s
(SD=144.9).
The progress of time to complete the tasks showed no trend in the conforming group
(z= -0.37, p=0.354) nor in the non-conforming group (z= 0.37, p=0.646) (Figure 97 and
Figure 98).
FIGURE 97 - AVERAGE TIME TO COMPLETE EACH TASK IN THE CONFORMING GROUP.
227
FIGURE 98 - AVERAGE TIME TO COMPLETE EACH TASK IN THE NON-CONFORMING GROUP.
Overlapping the two graphs (Figure 99) it is obvious that the shapes of the graphs of
the groups are similar, which seems to suggest an equivalent performance in both
groups in the second part of the test. Another interesting feature visible in the graph
is the longer time it took both groups to complete the fifth task. Remembering that
the fifth task was not the same for all the participants, rules out the possibility of
something intrinsically more difficult about the task. One possible explanation could
be that participants were getting tired since the variability on the fifth and sixth tasks
is larger than the variability of the other tasks. Still, this does not explicate why the
variability in the sixth task is smaller than the variability of the fifth task.
FIGURE 99 - OVERLAPPED GRAPHS OF TIME TO COMPLETE TASKS ON BOTH GROUPS.
228
Right answers and quits

The difference in the number of right answers in the conforming group (M=4.57,
SD=1.95) and in the non-conforming group (M=5.43, SD=0.65) was not significant
(F(1,26)=2.44, p=0.131).
The difference between the number of times each participant quit the tasks in the
conforming (M=0.36, SD=0.63) and the non-conforming group (M=0.14, SD=0.36)
was not significant (F(1,26)= 1.21, p=0.282).
Right answers over time

A Mann-Kendall analysis of the progression of the number of right answers for both
groups showed no particular trend for the conforming group (z= -0.60, p=0.727). The
non-conforming group presented an upward trend in the number of right answers as
the test progressed (z=1.81, p=0.035). These results are illustrated in Figure 100
(conforming group) and Figure 101 (non-conforming group).
FIGURE 100 - PROGRESSION OF RIGHT ANSWERS AND CORRESPONDENT

FIT CURVE FOR THE CONFORMING GROUP.
Conforming
229
FIGURE 101 - PROGRESSION OF RIGHT ANSWERS AND CORRESPONDENT

FIT CURVE FOR THE NON-CONFORMING GROUP.
Non-conforming
Mean RPS rate

Again, to have a better idea of the performance of the participants in the second part
of the test, the right answer and time results were combined following the same
principles applied before. A one-way ANOVA found no significant difference
between the mean RPS rate (F(1,26)=0.02, p=0.882) of the conforming (M=2.16, SD=1.39)
and non-conforming group (M=2.23, SD=1.10).
RPS rate over time

A Mann-Kendall test found no trend in the conforming group (z= -0.375, pup=0.646,
pdown=0.354) and no trend in the non-conforming group (z= 0.00, pup=pdown=0.5).
Revisits
The difference in the number of page revisits by the participants in the conforming
group (M=3.00, SD=3.21) and the participants in the non-conforming group (M=8.36,
SD=12.93) was not significant (F(1,26)=2.26, p=0.144). Both groups showed a lower
mean number of page revisits than in the first part of the test, with the non-
conforming group reducing the number of page revisits quite considerably.
Backtracking
Here also the difference in the usage of backtracking by the participants in the
conforming group (M=1.79, SD=2.08) and the participants in the non-conforming
group (M=6.14, SD=8.94) was not significant (F(1,26)=3.15, p=0.088).
230
Loopback
The number of times the participants in the conforming group (M=0.64, SD=0.93)
used the loopback strategy was not significantly different (F(1,26)=0.45, p=0.509)
from the number of times the participants in the non-conforming group used it
(M=0.43, SD=0.76).
RPS rate per task type

As in the first part of the test, in the second part there was no carryover effect of task
on the right answers per second ratio (Table 45).
TABLE 45 - MEAN RPS RATE PER TASK TYPE PER TASK TYPE IN THE TWO COUNTERBALANCED SITUATIONS.
Content task RPS Interface task RPS
Content tasks first (CI) 2.87 2.62
Interface tasks first (IC) 2.57 1.40
A one-way ANOVA showed that the difference between the two interface task RPS
ratios (CI vs. IC) was not significant (F(1,11)=2.20, p=0.169).
A one between-subject, one within-subject ANOVA showed an interaction between

task type and the conventionality of the website (F(1,26)=9.71, p=0.004). This time the
interaction was ordinal (Figure 102) and the ANOVA also showed a significant main
effect of task type (F(1,26)=21.47, p<0.01). Looking at the graph, the main effect of task
type is quite clear; participants in both groups got more right answers per second on
the content tasks than on the interface tasks. The interaction shows no difference in
task type performance on a conforming page but there is a great gap in performance
between content and interface tasks when the page does not conform to conventions.
231
FIGURE 102 - AVERAGE RIGHT ANSWERS PER SECOND RATE (RPS)

ON CONTENT AND INTERFACE TASKS PER GROUP IN THE SECOND PART.
There was no difference in RPS rate of the conforming group for interface and
content tasks in the second part (F(1,26)=0.472, p=0.498).
Close examination also shows that in the second part of the test (see Figure 102), the
RPS rate of the non-conforming group on interface tasks was equivalent to the rate of
the conforming group (F(1,26)=0.65, p=0.426). Interestingly, on content tasks the non-
conforming group outperformed the conforming group (F(1,26)=4.77, p=0.038).
Number of quits per task type

There were no main effects or interaction on the number of quits per task type in
each group.
Navigational adjustments per task type
Revisits
There were no main effects or interactions of conventionality and task type on the
number of page revisits made by the two groups in the second part of the test.
Backtracking
There were no main effects or interactions of conventionality and task type on the
number of backtracks made by the two groups in the second part of the test.
Loopback
A 1-between subjects, 1-within subjects ANOVA showed a main effect of task type
(F(1,26)=6.382, p=0.018) in the number of times the loopback strategy was used by the
participants of the groups. Participants used more loopback when responding to
232
interface tasks than when responding to content tasks. There was, however no
significant interaction between conventionality and task type (F(1,26)=0.709, p=0.407)
on the use of loopback.
Questionnaire
There were no differences between groups in the feelings of disorientation and ease
of learning and navigating the web forums in the second part of the test.
However when comparing the answers given by a group in the first part with the
answers given by the same group in the second part, some interesting results came
up. A Wilcoxon signed-rank test showed that the non-conforming group felt more
lost in the first part than in the second part (Z=-2.72, n=14, p=0.006) and they felt as if
they were going round in circles more in the first part than in the second part of the
test (Z= -2.73, n=14, p=0.006). There were no differences in the perception of the non-
conforming group regarding: (a) the difficulty of finding a previously viewed page;
(b) the ease of learning the site; (c) the ease of navigating the site.
There were no differences between the answers given by the conforming group in the
first and second parts of the test.
7.3.6. Summary of results

Below is a summary of the significant results obtained from this experiment.
TABLE 46 - SUMMARY OF THE RESULTS OF EACH GROUP IN EACH PART OF THE TEST.
Exp. Part Conforming Non-conforming
Part I Downward trend Downward trend

Time to complete task
Part II No trend No trend
Part I No trend Upward trend
Number of right answers
Part II No trend Upward trend
Part I Higher Lower
RPS Rate
Part II No difference No difference
Part I No trend Upward trend
RPS Rate trend
Part II No trend No trend
Part I Fewer More
Page revisits
Part I Fewer More
Backtracking
Part I Constant Higher
Feeling of disorientation
Part II Constant Lower
233
Interactions
In the first part of the experiment, the conforming group performed better on
interface tasks than on content tasks, and the situation was reversed for the non-
conforming group where performance was better on content tasks than on interface
tasks. In the second part of the experiment the conforming group showed no
difference in performance between content and interface tasks, and the non-
conforming group maintained a better performance on content tasks than on
interface tasks. The interaction observed in the second part of the experiment was
different from the interaction in the first part of the experiment. The difference in
performance of the participants from the conforming group detected in the first part
of the experiment could be due to the forum used in the first part being larger, and
consequently having more content than the one in the second part. The same
difference in performance of the conforming group was shown in the pilot study
described in this chapter which used the same forum as in the main experiment.
However, looking at the interaction between conventionality and task type of the
experiment described in Chapter 6, the conforming group showed no difference in
performance between the task types. The web forum used in the experiment in
Chapter 6 was the same as the one used in the second part of the main study in this
chapter. Again this web forum had less content than the one used in the first part.
This suggests that the performance on task type may also be affected by the
ammount of content the website has.
In the first part of the test participants in the conforming group quit more on content
tasks than on interface tasks, but in the non-conforming group the situation was
reversed with participants quitting more on interface than on content tasks. In the
second part of the test there were no effects or interaction between task type and
conventionality.
In the first part of the test, participants made more page revisits, more use of
backtracking and of loopback when answering content tasks than when answering
interface tasks. In the second part there were no differences in page revisits or
backtracking between the two task types. However participants used the loopback
strategy more often when answering interface tasks than content tasks.
234
7.3.7. Discussion
This study has demonstrated that as the literature suggests, not conforming to visual
conventions can leave users disoriented (Dillon & Vaughan, 1997) and hinder their
performance (Nielsen, 1999a). But it has also showed that their disorientation and
hindered performance can be short-lived.
Time and right answers

There was no overall difference in time to complete the tasks nor in the number of
right answers between the groups in either part of the test.
Considering what the literature says, this result is quite unexpected. However, as will
be seen below, a decrease in time to complete tasks and increase in number of right
answers of the participants was observed in the non-conforming group throughout
the first part of the test. This could have compensated the poorer performance they
showed in the beginning of the test, making their overall time and right answers
results close enough to that of the group which used the conforming template so a
difference was not detected.
RPS rate
In the first part, performance (measured by the RPS rate) for the group using the non-
conforming website was not as good as the performance of the group using the
conforming website. But, as expected, their performance improved throughout the
first part of the test, so that by the end of the first part, participants in the non-
conforming forum had an equivalent performance to the participants of the
conforming group. This suggests that although not conforming to visual conventions
might cause usability problems thus hindering performance (Nielsen, 1999a), this
performance hindrance is short lived and users familiarize themselves with the new
interface layout quite quickly.
In the second part of the test there was no difference in the RPS rate of both groups.
This indicates that the performance of both groups were equivalent in the second
part of the test. One plausible explanation is that participants who used the non-
conforming website had familiarized themselves with the layout of the interface in
the first part of the test so that their performance was equivalent to the group which
used the conforming version in the second part.
An alternative explanation would be that because the site used in the second part had
fewer topics and fewer pages than the one in the first part, it was easier to find the
235
answers to the tasks. However, an examination of the answers given in the

questionnaire by the group who used the non-conforming website regarding how
easy it was to learn to use and navigate the forum showed that participants
attributed the same difficulty to both the first part and the second part forums. This
indicates that they did not think the site in the second part was easier to learn than
the one in the first part.
Disorientation
In the first part of the test, participants using the non-conforming site did more page
revisits and backtracking movements than the conforming group. This seems to
indicate that they needed to re-orient themselves more than the other participants
(Yatim, 2002) and could consequently be more disoriented than their counterparts.
But, according to the questionnaire answered in the first part, they have felt as lost
and disoriented as the participants in the conforming group. One explanation for this
difference between the perceived and measured disorientation of the non-
conforming group is that by the end of the test, participants were not as lost as they
were in the beginning of the test. Since the questionnaires were answered at the end
of each part, it is possible that their final disorientation state had more influence on
the answer than the initial one.
However, looking at the perceived disorientation of the non-conforming group

across the two parts of the experiment, there was an improvement in their condition;
they had felt more disoriented in the first part than in the second. These results are in
accordance with the measured disorientation of the group, since in the first part they
made more page revisits and backtracking movements than in the second part.
The perceived disorientation, ease of learning and navigation of the site by the
participants of the conforming group were not significantly different between the
first and second part of the test.
Task type
Both in the first and second parts of the test, participants who used the non-
conforming forum performed better on content tasks than on interface tasks. But for
the group of participants who used the conforming forum, the situation was
different; in the first part participants performed better on interface tasks than on
content tasks but in the second part there was no difference in performance between
content and interface tasks. So, when visual conventions are violated, users who are
familiar with the conventions perform better on search tasks which depend solely on
236
written content, than on search tasks which depend on interface information. It

makes sense to think that when visual conventions of a website are violated the
interface elements do not meet users’ expectations thus causing their performance to
suffer. If structural conventions of a website are not changed, the organization of
content in the website should remain quite the same, regardless of any violation
visual conventions.
Considering that all participants were familiar with the visual conventions of web
forums, it is not wrong to assume that finding information in the conforming
interface would be easier than finding content, which seems to support the results of
the group who used the conforming forum in the first part of the test. But their
results in the second part of the test showed no difference in performance on
interface and content tasks. One explanation could be that the content tasks in the
second part were easier than the interface tasks; however in both parts tasks were
designed based on the analysis of the results from the pilot study. The only factor
which could have caused content tasks to become easier in the second part would be
that the forum used in that part was purposely built with less content than the one
used in the first part. As a consequence, answers were not as deep in the forum in the
second part as in the first part and the pages where answers were located did not
have as many elements as the pages in the first part.
The results for the group of participants who used the non-conforming forum
showed their behaviour to be consistent across the two parts of the test. They
performed better on finding content than retrieving information from the interface.
One simple explanation would be that, once learned, the non-conforming forum was
easier to use than the conforming one. But this would probably be reflected in the
interface tasks as well as the content ones. An alternative explanation would be that
in order to adapt to the new interface, participants who used the non-conforming
forum had to make more cognitive effort in the first part than their counterparts. This
cognitive effort is reflected in the higher number of page revisits (Tauscher &
Greenberg, 1996) and disorientation that the non-conforming group showed in the
first part compared to the conforming group.
Literature has shown that the higher the cognitive effort needed to acquire some
information, the more likely the person will be to recollect the information at a later
stage (Dewhurst & Hitch, 1999). So it could be that the cognitive effort made by them
in the first part helped participants to recall the interface better than the conforming
237
group in the second part, thus improving their performance. Another explanation
which shares similar grounds with the one above is found in familiarity literature
(Yonelinas, 2002; Zhang & Ghorbani, 2004). As seen in the previous chapter, deeper
levels of processing provide users with greater familiarity (Yonelinas, 2002; Zhang &
Ghorbani, 2004) and increase recollection (Dewhurst & Hitch, 1999; Yonelinas, 2002).
In this experiment it is possible that participants who used the non-conforming
forum had to use deeper processing in the first part of the test. This could have
improved their recollection and made them achieve a high level of familiarity with
the interface which could be equivalent with the familiarity they might have had
with the conforming forum.
7.3.8. Conclusion
This study has shown that when users who are accustomed to a certain set of visual
conventions have to use a website which violates these conventions they adapt to
them quite quickly. This can explain the lack of significant results not only in the
study described in Chapter 6, but also in the study described earlier conducted by
Kalbach and Bosenick (2006). It is likely that the changes in the menu positioning
made in their study were such a small violation of visual conventions that users were
able to quickly adapt and no differences could be detected. Reinforcing this idea are
the results of a similar study conducted by McCarthy et al.(2003). The study has also
violated the conventional positioning of the navigation menu and although in this
case a difference in performance was detected, the authors have indicated
performance improved quite quickly for the users who used the non-conforming
layout.
From the results of this experiment and indications from Kalbach and Bosenick(2006)
and McCarthy et al.(2003) it can be concluded that the performance and
disorientation problems that originate from the violation of the visual conventions of
a website genre are quickly overcome. Therefore, looking back at the outcomes of the
experiment conducted by Vaughan and Dillon (2006) which violated visual and
structural conventions of a web news genre, it is possible to ponder whether
violation of any of the four types of genre convention will have the same impact on
user performance. As explained earlier, Vaughan and Dillon (2006) reported that in
their experiment the violation of genre conventions resulted in a significant
performance hindrance. However, the authors also pointed out that after 5 trials with
the violation website the participants improved their performance and the
differences between them and the participants using the convention-conforming
website disappeared.
238
Although the performance improvement of participants using the non-conforming

website shown in Vaughan and Dillon (2006) took longer to happen than the
performance improvement shown in this thesis, it reinforces the idea that users can
adapt to violations of genre conventions. The difference in time participants using a
non-conforming website took to improve performance in this thesis compared to
participants in the Vaughan and Dillon (2006) study suggests that participants may
take longer to improve their performance when structural conventions are violated in
comparison with visual conventions. Another possibility is that not conforming to
visual or structural conventions will have the same impact on user performance but
when both visual and structural conventions are violated simultaneously ( as in
Vaughan & Dillon, 2006) they produce a synergistic effect. This illustrates not only
the importance of the work developed in this thesis but also the need for more
understanding of how each type of genre convention affects the users.
Another relevant outcome of the experiment described in this chapter is the

interaction between conventionality and task type. Although this interaction was
detected earlier in Chapter 6 the results from the pilot and the main experiment
described in this chapter confirm this interaction. This interaction strengthens the
importance of considering not only the user in the design process but also the tasks
which users will be carrying out in the system.
Limitations
Although results from the experiment described here indicate that participants may
quickly develop a schema for a non-conforming website and suggest that this schema
may be carried over to other similar websites, the time interval between the two parts
of the experiment was too small. This time interval may not have been sufficient to
establish whether participants will be able to maintain the performance improvement
after a longer time without interacting with the non-conforming website. It might be
that they have acquired short term knowledge of the site which might have helped to
improve their performance. But it is possible that were they asked to use the site
again after a longer period of time, performance would again be hindered.
239
Chapter 8
The effects of conventionality on
users searching for information
in an online news pages
8.1. Introduction
A final experiment was put in place to evaluate if the results obtained for web forums
are likely to be replicated in other web genres. Mainly, the experiment attempted to
confirm the results obtained in the first part of experiment 5 (see Chapter 7) which
indicated that participants in the non-conforming group showed a performance
improvement throughout the experiment. This is important as it can indicate
whether or not the results can be extended to other web genres, adding to the
generality of the outcomes of this research.
Websites belonging to the online news category were chosen for this experiment,
because they have been consistently cited in the literature as a web genre (Åkesson,
2003; Eriksen & Ihlström, 2000; Ihlström & Åkesson, 2004; Vaughan & Dillon, 1998,
2006).
To design the material for this experiment an analysis of existing newspapers was
made. Seven mainstream newspaper websites (commonly known in Britain by the
term ‘quality newspapers’) were analysed. All seven newspapers feature in the list of
most visited United Kingdom newspaper web news sites created by the ABCe (Audit
Bureau of Circulation – new media division) and five were among the top 10 most
visited UK newspaper websites list created by Nielsen Online (available from
www.nielsen-online.com/pr/pr_071011_UK.pdf - accessed on 17/04/2009). Even
though the BBC News website does not appear on Nielsen’s nor on ABCe lists of
newspaper websites, for not having a print version, it is one of the most accessed web
news sites in the United Kingdom. For this reason it was decided that the BBC News
website should be included in the analysis. A list of the websites analysed for this
research is shown in Table 47.
240
TABLE 47- ONLINE NEWS WEBSITES ANALYSED TO PREPARE THE EXPERIMENTAL MATERIAL
Newspaper name Website URL
Financial Times www.ft.com
The Guardian www.guardian.co.uk
The Independent www.independent.co.uk
Daily Telegraph www.telegraph.co.uk
The Times www.timesonline.co.uk
BBC News www.bbc.co.uk/news
The Herald www.theherald.co.uk
The Scotsman www.scotsman.com
8.2. Analysis of existing news websites

Following the methodology applied in Survey 1 (described in Chapter 4), these online
news pages were examined for the most frequently occurring elements and the most
frequent values for the three attributes (symbolization, position and configuration).
For this analysis, wireframes of the main page, category page and story page of each
of the 8 news sites were created. Then, based on the observation of the attributes of
the most frequently recurring visual elements of these news websites, design
decisions were made regarding the layout, and wireframes for the main, category
and story page were designed for the conforming (Figure 103) and non-conforming
(Figure 104) sites.
241
FIGURE 103 - WIREFRAMES FOR THE CONFORMING ONLINE NEWS PAGES DESIGNED FOR THE EXPERIMENT
FIGURE 104 - WIREFRAMES FOR THE NON-CONFORMING ONLINE NEWS PAGES DESIGNED FOR THE EXPERIMENT
242
Once the wireframes were completed a preliminary survey was put in place to verify
with online news users that the conforming layout was indeed perceived as more
conventional.
8.3. Survey: Judgments of online news regarding the

conventionality of the layouts
8.3.1. Objectives
This survey was put in place to test:
! If the design of the conforming website was more in conformity with other
online news sites which users are accustomed to visit when compared to the non-
conforming one.
! Whether particular visual elements or visual configurations may lead
participants to perceive one layout as ‘more conforming’ than another.
! Verify if the previously found correlation between perceived conventionality and
ease of use is replicated for online news pages.
8.3.2. Method
Material
The wireframes produced in the previous stage were edited to include colour, pictures
and adverts so that they would resemble a real website as much as possible. Every
picture and advert image used in the conforming layout was also used in the non-
conforming layout. The same colour scheme was applied to both layouts. As seen earlier
(see figures 103 and 104), the text used in the templates was ‘greeked’ but the hierarchy
of information of an online news page was indicated through different type sizes and
italic types. The final conforming and non-conforming templates used in the survey are
presented in Figures 105 and 106 respectively.
243
FIGURE 105 - PAGES OF THE CONFORMING ONLINE NEWS LAYOUT
FIGURE 106 PAGES OF THE NON-CONFORMING ONLINE NEWS LAYOUT
244
Participants
Twenty participants volunteered for this survey. All participants were Internet users and
read news online at least three times a week. They were all fluent in English and used at
least one of the seven mainstream online news websites or BBC News. Participants of
this survey had not participated in any previous experiments in this thesis.
Procedure
Participants were approached and asked some questions which aimed at assessing
their online news reading profile. To avoid giving away the true purpose of the
questionnaire and risk biasing the answers, participants were told that the
questionnaire aimed at investigating their Internet usage profile. Besides asking
about their online news reading habits, the questionnaire also addressed their
emailing, online banking and online shopping profile. If the participant read online
news three times a week or more, a second set of questions were asked, if they did
not they were thanked and the survey ended.
The second set of questions in the survey aimed at obtaining judgment of online
news users regarding the conventionality of the two layouts. To facilitate
comparison, each page of the conforming layout was presented next to the equivalent
page of the non-conforming layout and both were printed one on top of the other on
a sheet of paper (Figure 107). In other words, the main pages of the conforming and
non-conforming layouts were printed together on one sheet of paper, the category
pages of both layouts were printed in another sheet and the same was done for the
story pages of both layouts. The position in which each page appeared on the paper
(top or bottom) was randomized.
Participants were shown one sheet at a time and were asked to respond which of the
two layouts they thought looked more like an online news page they have seen
before and what did they see that led them to that conclusion. Then they were asked
which of the two layouts they thought would be easier to use and why they thought
that. Participants were allowed to give more than one reason for choosing a
particular layout as more similar to other existing online news and for choosing a
particular layout as easier to use.
245
FIGURE 107 - SAMPLE OF A SHEET USED IN THE SURVEY CONTAINING THE TWO LAYOUTS
Pages were presented to the participants in the same order in which participants
would see them if they were navigating on a real website. Thus, they were shown the
main pages first, then the category pages and finally the story pages. When given the
sheet of paper with the printed pages, participants were informed at which level they
would be in an online news website. For example, when shown the sheet with the
main page, participants were informed that they should consider the layouts as the
main page of an online news website. The same was done for the category page sheet
and the story page sheet.
8.3.3. Results
Conventionality
All participants chose the conforming layout as the one more similar to other existing
online news websites for all three pages (main, category and story pages).
246
Overall, the visual elements and their attributes which seem to mostly influence
participants’ perceptions of conventionality are first the configuration of the picture,
the headline and the text on a single news piece and second the use of tabs to
represent the menu (symbolization mode) and the location of the menu on the top of
the page (see Figure 108). Interestingly enough, adverts (both their position and
shape) seem to be more influential in participants’ perceptions of conventionality
than the position of the logo on the screen, location of the ‘Other stories’ box and the
use of more than one column of text on online news pages.
FIGURE 108 - NUMBER OF TIMES EACH VISUAL ATTRIBUTE/ELEMENT WAS MENTIONED AS INFLUENCING
PARTICIPANTS' DECISION ON THE SIMILARITY OF THE PAGES WITH EXISTING ONLINE NEWS
Ease of use
All participants thought the conforming main page and category page to be easier to
use than the non-conforming alternatives. Despite the fact that 19 participants
thought the conforming story page would be easier to use, one participant thought
that the non-conforming story page would be easier to use than its conforming
counterpart. When queried why, the participant explained that because the text was
in one block, without any image or headlines interfering with it, it would probably be
easier to read than the conforming one. Here again the configuration of the news
pieces and the style and position of the menu were the most frequently chosen
elements (Figure 109).
247
FIGURE 109 - NUMBER OF TIMES EACH VISUAL ATTRIBUTE OR ELEMENT WAS MENTIONED AS
INFLUENCING PARTICIPANTS’ JUDGEMENTS ON THE EASE OF USE OF THE PAGES
Perceptions of Conventionality and perception of ease-of-use

A Spearman-! test revealed a strong correlation between perceived conventionality
and perceived ease of use (!=0.99). This result confirms that the correlation found in
web forums earlier in chapter 5 also applies for online news pages.
8.3.4. Discussion
Perceptions of conventionality
The results show that online news users agreed that the conforming layout was very
similar to existing online pages which they are accustomed to use. The fact that none
of the participants considered any of the non-conforming pages more similar to
existing online news sites indicates that the layout does seem to represent to some
degree a non-conformance with the visual conventions of the online news genre.
Most interesting was the fact that essentially (a) the configuration between the
headline, summary and photo of a news piece, (b) the position of the navigational
menu and the fact that it utilises tabs to indicate the news categories and (c) the
position of the adverts on the page seem to have been used by participants to decide
whether the website conformed to the conventions of the genre. Following these
results, further analysis of these particular elements and their attributes was done.
248
Configuration of the elements of news pieces

The standard configuration of the components of a single piece of news is to have the
headline on top of the summary text and the picture either to the left or to the right of
these elements (Figure 110).
FIGURE 110- COMMON CONFIGURATION OF THE COMPONENTS OF A PIECE OF NEWS
A common variation to this configuration happens when the summary text is

wrapped around the picture either to the left or to the right (Figure 111). Even
though this type of configuration can be found across all three types of online news
pages (main, category and story pages) it is more common in the story page due to
the larger quantity of text present.
249
FIGURE 111 - CONFIGURATION OF THE COMPONENTS OF A PIECE OF NEWS WITH THE SUMMARY
TEXT WRAPPED AROUND THE PICTURE
Yet another type of configuration common to articles in the story page of online news
websites also contains text wrapped around the picture but, in this case the headline
is located above both the text and the picture (Figure 112).
FIGURE 112 - TYPE OF CONFIGURATION OF THE ELEMENTS OF AN ARTICLE IN THE STORY PAGE OF ONLINE NEWS PAGES
250
These observations were used when generating the conforming online news website.
Also based on these observations, the configurations used for the news elements in
the non-conforming version were determined. The strategy was basically to visually
dissociate the components of the news element by either placing them in a different
configuration (Figure 113) or separating them (Figure 114) by introducing another
element (in this case an advert) between the components of the news article.
FIGURE 113 - CONFIGURATIONS OF NEWS PIECES USED IN THE NON-CONFORMING ONLINE NEWS PAGES
FIGURE 114 - CONFIGURATION USED IN THE STORY PAGES OF THE NON-CONFORMING ONLINE NEWS WEBSITE WHERE
ELEMENTS OF THE ARTICLE WERE DISSOCIATED BY PARTIALLY SEPARATING THEM.
251
Style and placement of the navigational menu

The survey results also indicated that the visual style and placement of the
navigational menu play an important role in the decision of the participants
regarding the level of conformance of the layouts presented with the online news
pages they were familiar with.
FIGURE 115 - STYLE AND PLACEMENT OF THE NAVIGATIONAL MENU CONSIDERED AS

COMMONLY OCCURRING IN ONLINE NEWS SITES
Again this result was used to inform the design of the conforming online news page
and also to determine the placement and style of the non-conforming menu (Figure
115). In the non-conforming layout the menu was placed on the right side of the
screen (Figure 116). Similarly to what was done previously for web forums, the
symbolization mode of the menu items was changed from alphanumeric to pictorial
by replacing the words with icons. Also the configuration of the menu items was
altered from horizontal to vertical.
252
FIGURE 116 - STYLE AND PLACEMENT OF THE NAVIGATIONAL MENU IN

THE NON-CONFORMING ONLINE NEWS LAYOUT
Adverts
Interestingly enough, it seems that not only it is common for adverts to be present in
online news pages but there are also certain positions where adverts can be placed
which are expected by online news users. Generally speaking on online news pages,
adverts seem to be located in two main areas of the page: the top and the right
(Figure 117). These results partly agree with those found by (Bernard, 2001a, 2001b)
that the most common position for adverts is the top of the page. However, on the
second most common position of adverts, results diverge, while Bernard found it to
be the centre bottom of the page, in the online news pages analysed, the second most
common position for adverts was the right side of the page.
The common positions found for the adverts are consistent through main, category
and story pages, though the format of the adverts may vary depending on the page
on which it is shown. Most adverts consist of long vertical or horizontal image
banners either in flash or animated GIFs. However, samples of smaller and shorter
banners are present along with some sponsored advertising presented in the form of
text and links.
253
FIGURE 117 - POSITIONING AND SHAPES OF ADVERTS COMMONLY OCCURRING ON ONLINE NEWS PAGES
Perceptions of ease of use

Generally speaking participants used the same visual elements to judge
conventionality and ease of use. The most frequently cited elements were again the
configuration of the elements of a piece of news, the placement and style of the
navigational menu and the positioning and format of the adverts. However, the
positioning of the search bar was almost as important to participants in judging ease
of use as the positioning of the adverts. This indicates that participants were taking
into account this element when judging ease of use which did not occur when they
were judging conventionality. This suggests that participants were able to separate
which elements they were using to guide their judgments of conventionality and
which ones they used for ease of use. Since the search bar has an unique function in
aiding search and is commonly used as part of a search strategy it is no wonder why
participants used it as a relevant factor in their judgments of ease of use.
254
8.3.5. Conclusion
This survey confirmed that the conforming layout did indeed conform to the
expectations of what an online news page should look like. It also provided useful
information on which visual elements and attributes mostly influence their
judgments on the conventionality and ease of use of an online news page. Knowing
the role that visual elements play on the perceptions of users is invaluable
information to designers who can use it to estimate to a certain extent, the impact of
their layouts on users. Finally, it also confirms the correlation between
conventionality and perceived ease of use found in Chapter 5, which suggests that it
may be extendable to other web genres. The implications of this correlation will be
explained in more detail in chapter 9.
8.4. Experiment 6 – Effects of conventionality on searching

for information on online news websites
8.4.1. Objectives
This experiment aims to investigate whether the results obtained in experiment 5 (see
chapter 7) can be replicated on another web genre. Experiment 5 investigated the
effect of non-conformance to the visual conventions of the web forum genre on
searching for information. Therefore this experiment will attempt to:
! See if participants using a non-conforming online news website will perform
worse than the participants using a conforming online news website.
! Check if the performance of participants using the non-conforming online
news page will improve over time.
! Assess the development of the feeling of disorientation of the participants
through the experiment.
! Assess if learning occurred amongst participants using the non-conforming
news page.
! Find whether participants using a non-conforming online news website will
adopt similar strategies to search for information as participants in the
conforming site.
255
8.4.2. Method
Material
Websites
To design the websites, the layouts presented to participants in the preliminary

survey were used as a template. To determine which categories will be used in the
experimental websites, the eight mainstream UK online news websites were
examined, their categories and subcategories were noted. The categories and
subcategories which occurred in at least four of the websites were picked to be used
in the experimental sites. These categories are listed in Table 48.
TABLE 48 - LIST OF COMMONLY OCCURRING ONLINE NEWS CATEGORIES
Category Subcategory
News UK
World
Science and Technology
Sport Football
Cricket
Motorsports
Rugby
Money Property
Investments
Saving
Insurance
Lifestyle Food & Drink
Health
The images and textual content for the conforming and non-conforming website
were obtained from the eight mainstream UK online news websites during the course
of a week (from 15th to the 21st of April 2009). Adverts were also obtained from
existing online news pages and were all animated GIFs. Clicking on adverts was
disabled to prevent participants leaving the test pages and going to other websites.
The search engine was disabled to force participants to navigate the website using
the menu and links available.
A different colour scheme to the one shown in the survey samples was used in the
final version of the material. This decision was made because the blue colour used for
256
the survey layouts was quite dark and it was believed that a lighter colour would
improve contrast and legibility.
Non-conforming pages
To represent the categories and subcategories graphically in the non-conforming
online news page several icons were designed. An attempt was made to make the
icons as clear and as representative as possible so that interpretation difficulties could
be minimized. The icons chosen are illustrated on Figure 118.
FIGURE 118 - ICONS DESIGNED FOR EACH CATEGORY AND SUBCATEGORY OF THE CONFORMING ONLINE NEWS SITE
The menu was designed in an ‘accordion’ style vertically in a way that all categories
were visible at all times but only the subcategories of the selected category were
displayed as can be seen in Figure 119. In the conforming website, the menu also
displayed all categories but only the subcategories of the selected category.
257
FIGURE 119 - ACCORDION MENU DESIGNED FOR THE NON-CONFORMING ONLINE NEWS PAGES
(A) WITH THE NEWS CATEGORY SELECTED AND (B) WITH THE MONEY CATEGORY SELECTED
a. b.
Thumbnails of some pages from the finished conforming and non-conforming

websites are shown in figures 120 and 121.
FIGURE 120 - SAMPLE PAGES OF THE CONFORMING NEWS WEBSITE
258
FIGURE 121 - SAMPLE PAGES OF THE NON-CONFORMING ONLINE NEWS WEBSITE
The same screen capture and time measuring techniques used in experiment 5 were
also used in this experiment. The pop-up window with the finished button was also
used for participants to either indicate they found the answer or for them to quit the
current task and move onto the next one.
This experimental setup ensured that at every mouse click that a participant made,
an image of the screen was saved to a file. The image contained: the browser
window, the mouse cursor (with an indication of where the click took place), the
stopwatch and the pop-up window (see Figure 122).
259
FIGURE 122 - SAMPLE OF THE EXPERIMENTAL SETTING WITH THE NON-CONFORMING WEBSITE
Tasks design
There seems to be three main ways in which users acquire information on an online
news website: one is searching for information about a well defined target (e.g.
finding out what was the score of the Arsenal vs. Manchester United football match);
another is searching for information on a target which is not totally defined (e.g.
finding out updates on the next general election); and finally the third method is the
serendipitous discovery of a topic of interest (e.g. finding out more about a new
recent scientific development in the field of genetics). These methods of acquiring
information generate in the users different styles of browsing which have been
described by Catledge and Pitkow (1995) and were mentioned in Chapter 2: search
browsing, general purpose browsing and serendipitous browsing.
Since this thesis is concerned with searching for information, for the purposes of this
experiment tasks will consist of asking participants to search for information about a
well-defined object.
Six tasks were defined with the intention of making participants look for information
on a well-defined target (Table 49). The answers to all tasks were located in the story
pages and there were no direct links to the stories in the main page. This was done to
encourage participants to use the navigational structure of the website.
260
TABLE 49 - TASKS DESIGNED FOR THE EXPERIMENT

Nº Task
1 How many properties in the UK will be repossessed in 2009?
2 What change is going to happen to Formula One next year?
3 Which country in the EU has the most comfortably sized women?
4 How much have BSkyB shares risen?
5 Which country will host the next Rugby world cup in 2011?
6 How many appearances has the football player Rafael da Silva had for the Manchester United team?
In the sixth task participants were asked to verbally talk aloud what they were
thinking and doing as they navigated through the websites. The order of tasks 1-5
was randomized per participant per group while task 6 was the final task for each
participant in both groups.
Although all tasks were designed to have participants searching for information on a
specific object, data from the first five tasks were used to assess the first two
objectives described in 8.4.1: (a) See if participants using a non-conforming online
news website will perform worse than the participants using a conforming online
news website and (b) Check if the performance of participants using the non-
conforming online news page will improve over time.
Data from the sixth task was used to assess the fifth objective described in 8.4.1: Find
whether participants using a non-conforming online news website will adopt similar
strategies to search for information as participants in the conforming site.
Disorientation questionnaire
Part of the disorientation questionnaire which was used in Experiment 5 (see

Appendix R) was also applied in this experiment. Similarly to before, participants
were asked to answer the questionnaire at two different times in the test: in the
beginning (after the first task was completed) and at the end of the five initial tasks.
The idea here was to check if perceptions of disorientation, ease of use and ease of
learning of the participants change over the course of the experiment. To assess the
fourth objective (‘Assess if learning occurred amongst participants using the non-
conforming news page’) these three questions were monitored for changes in
perceptions which also may indicate that participants have learned to use the site.
261
Participants
42 online news users participated in this experiment. All participants confirmed
reading online news at least three times a week and have not participated in the
survey or in any previous experiments in this thesis. All participants were fluent in
English and their academic background varied from completed GCSEs to PhD
degree.
Similarly to experiment 5 two independent groups of participants were used and

participants were randomly allocated to one of the two experimental groups so that
each group contained 21 online news users.
8.4.3. Procedure
Participants were tested individually on one of the two websites to which they were
randomly assigned. Participants were informed that the experiment consisted of
looking for answers to questions in an online news website. They were asked to
search for information as quickly and as accurately as possible, highlighting the piece
of text which best answered the question asked, and then clicking on a ‘finished’
button, which was located in the pop-up window to the left of the browser.
Participants were told that they should use the website as they normally would any
other in a real situation and that they were allowed to quit on a task if they wanted
(by clicking on the finished button). Finally they were informed that at two instances
during the experiment they would be asked to answer a number of questions
regarding their experience and feelings when using the website.
When the participant was ready, the experimenter handed a sheet of paper with the
first task to the participant (since task order was randomized, this varied between
tasks 1-5). When the participant finished reading the task and looked at the computer
screen, the experimenter started the stopwatch. When the participant found the
answer or quit the first task, the experimenter paused the time measure. Then the
participant was asked to answer the questions from the disorientation questionnaire.
Once that was done, a paper with the next task was handed to the participant and
once the participant stopped reading the task and looked at the screen, the
experimenter resumed the measure of time. Once the participant had either found
the answer to the task or given up searching for it the experimenter paused the
262
stopwatch and moved on to the next task. This was repeated until tasks 1-5 had all
been attempted.
After the participant finished the fifth task, they were once more asked to answer the
disorientation questionnaire. After that, the participant was told that for the next task
they would still have to search for the answer to the task on the online news page but
that this time they were required to think aloud as they searched for it. The
participant was told that their verbal protocol was going to be recorded, but the data
would remain anonymous. If the participant agreed to be recorded, the experimenter
would start the recorder and give the participant the next task. If the participant did
not agree to the recording, then the experimenter would take notes of their
verbalizations. Time measurement was resumed after the participant read the task
and looked at the screen.
These procedures were repeated for all participants in the test. It is relevant to note
that all participants agreed to be recorded therefore no notes needed to be taken.
8.4.4. Results
The results described here, unless specified otherwise, refer to the first 5 tasks
participants performed in the test. The results of the 6th task have not been included
in the performance analysis since participants were asked to talk as they went
through the process of completing the task.
Time
The group using the conforming website spent on average 90.18 seconds (S.D. =
65.59) on each task while the group which used the non-conforming website spent an
average of 135.17 seconds (S.D. = 111.65) on each task. A one-way ANOVA on the
data indicated that this difference is significant (F(1,40)=7.78, df=1, p=0.008).
263
FIGURE 123 - MEAN TIME SPENT ON TASKS PER GROUP
Analysing the changes in the time spent on tasks per group throughout the
experiment it is possible to observe that participants who used the non-conforming
news website did spend less time on each task as the test progressed. The group
which used the conforming website does seem to show a slight reduction in time
spent on tasks after the second task (see Figure 124). However, a Mann-Kendall trend
analysis in the conforming group did not indicate any trend (Z=-4, n=5, p=0.208)
while in the non-conforming group it indicated a significant down trend (Z=-8, n=5,
p=0.042).
FIGURE 124 - MEAN TIME SPENT ON TASKS PER GROUP THROUGHOUT THE EXPERIMENT
264
Right Answers
Of a total of 105 tasks, participants in the conforming group answered 84 tasks
correctly (M=0.80, S.D.=0.23) compared to 71 (M=0.68, S.D.=0.29) correctly completed
tasks in the non-conforming group (Figure 125). A one-way ANOVA showed this
difference is not significant (F(1,40)=1.98, df=1, p=0.167).
FIGURE 125 - MEAN RIGHT ANSWERS PER GROUP
Over the course of the experiment, the group using the non-conforming website
showed a gradual increase in the number of tasks answered right while the group
using the conforming website did not display any variation. A Mann-Kendall trend
analysis confirmed this by showing that the non-conforming group indeed had an
upward trend in the number of right answers (Z=8, n=5, p=0.042) while the
conforming group showed no definite trend (Z=-1, n=5, p=0.475).
265
FIGURE 126 - MEAN NUMBER OF RIGHT ANSWERS PER GROUP THROUGHOUT THE EXPERIMENT
RPS rate
The number of right answers was combined with time into the RPS rate, which has
been used previously in experiments 4 and 5 to evaluate performance. Results show
that the group using the conforming site achieved a mean RPS of 1.50 (S.D.=1.38)
while the non-conforming group achieved a mean RPS of 1.14 (S.D.=1.34). A one-way
ANOVA on the data indicates that the difference is not significant (F(1,40)=2.34, df=1,
p=0.134).
Looking at how the RPS varied throughout the experiment for each group (Figure
127) it is possible to see that the non-conforming group seems to have had a
significant performance improvement (measured by the RPS). In fact a Mann-Kendall
trend analysis shows that the non-conforming group experienced a significant
upward trend in their performance (Z=10, n=5, p<0.01).
Even though the conforming group seems to have had a performance improvement
from the 2nd to the 4th task, their performance declines from the 1st to the 2nd task and
again from the 4th to the 5th task. A Mann-Kendall trend analysis showed no
significant trend for this group (Z=4, n=5, p=0.21).
266
FIGURE 127 - MEAN RPS PER GROUP THROUGHOUT THE EXPERIMENT
Disorientation
Backtracking
Participants using the non-conforming website performed more backtracking actions

(Mean=5.76, S.D.=6.87) than participants using the conforming website (Mean=1.67,
S.D.=1.53). A one-way ANOVA on the data showed this difference to be significant
(F(1,20)=7.17, df=1, p=0.011).
Perceived disorientation
As mentioned earlier, user perceived disorientation was assessed on two instances
during the experiment: one after the 1st task and the other after the 5th task. The first
question in the questionnaire (‘Complete the sentence as appropriate: ‘I felt
lost/disorientated…’) measured the perceived disorientation of participants.
Participants had to choose between five possible answers:
1) Never
2) Sometimes
3) A few times
4) Most of the time
5) Always
The median value of question 1 for each group in each instance the test was applied
is shown in Table 50.
267
TABLE 50 - MEDIAN VALUES OF THE ANSWERS GIVEN IN TWO INSTANCES

DURING THE EXPERIMENT TO THE DISORIENTATION QUESTION
After 1st task After 5th task
Conforming 2 (Sometimes) 2 (Sometimes)
Non-conforming 4 (Most of the time) 3 (A few times)
It is clear from looking at the table that while participants who used the conforming
website showed no change in their feeling of disorientation, participants in the non-
conforming group seem to have felt less disorientated towards the end of the test
than when they started. However, the answers also indicate that even though
participants in the non-conforming group felt less disorientated towards the end of
the test, they still felt more lost than the participants in the conforming group.
Perceptions on ease of navigation
The third question in the questionnaire examined the perceptions of the participants
regarding how easy or difficult to navigate they thought the website they used was.
The question stated that ‘the site was easy to navigate’ and participants had to
express their agreement or disagreement by choosing one of five options:
1) Strongly disagree
2) Disagree
3) Neither agree nor disagree
4) Agree
5) Strongly agree.
Participants in the conforming group showed no variation throughout the
experiment in their perceived ease of use of the website on both occasions agreeing
with the statement (Median=4, Mode=4). Participants from the non-conforming
group disagreed (Median=2, Mode=2) that ‘the site was easy to navigate’ but by the
end of the 5th question they agreed (Median=4, Mode=4) with the statement.
Navigation Strategy
From the task which participants were asked to think aloud it was possible to have
an idea of the search strategies that participants were using in each website.
The majority of participants using the conforming website (57%) adopted the
following strategy to answer the 6th task (Figure 128).
268
FIGURE 128 - STRATEGY 1 - STRATEGY USED BY THE MAJORITY OF THE PARTICIPANTS IN THE
CONFORMING AND NON-CONFORMING GROUPS TO ANSWER THE 6TH TASK.
The second most adopted strategy by the participants in the conforming group
(adopted by 21% of them) was a variant of the previous one (Figure 129) where the
only difference was that instead of searching for either the name ‘Rafael’ or the word
‘Manchester United’ in the football category page, the searched only for the name
‘Rafael da Silva’
FIGURE 129 - VARIATION ON STRATEGY 1 THAT WAS THE SECOND MOST USED STRATEGY
BY PARTICIPANTS ON BOTH THE CONFORMING AND NON-CONFORMING GROUPS.
Coincidentally, the majority of participants in the non-conforming group adopted the

same strategies as participants in the conforming group. Most interesting is the
similarity in the proportions with 52% of the participants using strategy 1 (see Figure
128) and 21% of the participants using the variation where only the name ‘Rafael da
Silva’ was searched (see Figure 129).
Learning
Perceptions of ease of learning
The second question in the questionnaire addressed the perceptions of ease of

learning the website. The question asked participants to judge how strongly they
agreed or disagreed with the statement: ‘Learning to use the site was easy’. As in the
previous question, the possible answers ranged from 1 – Strongly disagree to 5 –
Strongly agree. Participants who used the conforming website showed no difference
in their perception of ease of learning over the course of the experiment agreeing on
both instances (Median=4, Mode=4) that the website ‘was easy to learn’.
269
However, the group using the non-conforming page started the test disagreeing
(Median=2, Mode=2) that ‘Learning to use the site was easy’ and ended the test
neither agreeing nor disagreeing (Median=3, Mode=4) with the statement. This result
shows an evident improvement in their perception of the ease of use of the website
which means that by the end of the test they did not think the website was as hard to
learn to use than when they started.
Furthermore, the difference between the median and the mode indicates that
although the middle value for the distribution was 3 (Median) the most frequent
choice was 4 (Mode). In fact, examining the results, by the end of the test 10
participants in the non-conforming group agreed with the statement while 5
participants neither agreed nor disagreed and 5 participants disagreed. It is plausible
to think that if participants in the non-conforming group spent a little more time
using the website they would agree that it was easy to learn to use it.
Thinking aloud
As explained earlier, in the 6th task participants were requested to think aloud as they
were searching for the answer to the question.
Analysis of the recording of the participants revealed that some participants in the
non-conforming forum indicated verbally having learned what the navigational
menu was or the meaning of the icons in the navigational menu. They made evident
in their comments that they have at some point during the experiment decoded the
symbols on the menu and learned their meaning. Some of their comments are listed
below:
! “Go to the symbol of sports on the menu, I know that is the menu now, click
there, there are four symbols the 1st is the football category.”
! “That carpet [icon], what is that? I can’t figure this one out… Oh! There is an
atom…so it must be science…still don’t get the carpet!
! “Football…I'm gonna look at the trophy thing because now I know that's
sports…”
! “Since I have been here before, I can tell that this [icon] is sports. I didn't
know that before…Anyway I can see it is sports because it has a logo I can
identify.”
Only one of the participants in the conforming group has indicated having to learn
the navigational menu by commenting “Well, now I know where the sports section
270
is…”. But the participant then explains what may be the origin of the problem by
saying: “I haven't quite got used to the menu as it is quite small.”
8.4.5. Discussion
Performance
Even though participants in the conforming online news site spent less time on each
task compared to the participants in the non-conforming site, no significant
differences were found between groups in number of right answers or in their RPS
ratio. This suggests that overall there was no significant difference in performance
between these two groups.
Examining what happened to their performance over time, it becomes evident that
while the conforming group showed no trends of improvement in any of the three
measures (time, right answers and RPS ratio), the group using the non-conforming
pages showed a significant downtrend in time, an upward trend in the number of
right answers and an upward trend in their RPS ratio. In other words, they spent less
time on each task as the experiment progressed, got more correct answers and had an
overall performance increase.
Disorientation
Backtracking
Participants in the non-conforming group performed on average around 3 ! times

more backtracking actions than participants in the conforming group. Considering
that backtracking can be an indicator of how lost a website user is this can be seen as
a predictor of how disorientated participants felt.
Looking at the development of backtracking over time (Figure 130) it is clear that the
group who used the conforming site had a small decrease in the number of
backtracking actions executed over time, while the group using the non-conforming
website increased the number of backtracking actions up to the third task and then
showed a dramatic decrease in the last two tasks.
271
FIGURE 130 - MEAN BACKTRACKING ACTIONS EXECUTED OVER TIME BY PARTICIPANTS

IN THE CONFORMING AND NON-CONFORMING GROUPS
A Mann-Kendall trend analysis revealed a significant downtrend in the conforming

group (Z=-8, n=5, p=0.042) but no significant trend in the non-conforming group
(Z=0, n=5, p=0.525). One explanation for the backtracking behaviour of the non-
conforming group could be that until the third task participants in this group were
lost and after that they strted learning the website.
Questionnaire
Examining the questionnaire, the answers given confirm the overall backtracking
results. Participants who used the conforming online news sites did not feel
extremely disorientated having mostly stated that only sometimes they felt
lost/disorientated. Furthermore, their feelings of disorientation and ‘lostness’ were
not reduced nor increased as they used the website. On the other hand, participants
in the non-conforming group started the experiment feeling lost/disorientated ‘most
of the time’ and towards the end of the experiment they were feeling
lost/disorientated only ‘a few times’. Even though their disorientation feeling eased,
they were still comparatively more disorientated than their counterparts in the
conforming group.
Perceived ease of navigation

Here again, participants from the conforming group showed no variation in their
perception of how easy to navigate the website was. However, participants in the
non-conforming group started the experiment disagreeing that the website was easy
272
to navigate and towards the end of the test they agreed that the site was easy to
navigate.
Considering that by the end of the test participants in the non-conforming group
were still feeling lost/disorientated ‘a few times’ one can not help but wonder why
also towards the end they agreed that the website was easy to navigate. One possible
explanation for this is that because their website conformed to the structural and
functional conventions of the online news genres participants quickly figured out
how to navigate through its pages without much problem.
By conforming to the structural and functional genre conventions, the website met
the expectations of users on the two aspects which can be most influential in the
process on navigating through a website. Therefore users could use the navigation
strategies and mental processes from genre conforming websites in the non-
conforming site without any difficulty. Based on this idea and the concept that some
mental processes are triggered by the presence of genre-conforming visual elements
(van Dijk & Kintsch, 1983) the following scenario could have happened.
When participants started using the website, its non-conformity with the visual
conventions of the genre did not give them any cues which signalled them to use the
strategies and mental processes associated with the online news genre. But as
participants started learning the website, they figured that the strategies and mental
processes associated with the online news genre applied to that website as well. As
they started applying those strategies and received positive confirmation navigating
the difficulty was minimized or even disappeared.
This idea is further reinforced by the fact that during the thinking aloud task
participants of both groups used the same strategies to locate the answer to the task.
The fact that participants in the conforming group were adopting it suggests that this
strategy is commonly used by online news users when facing a convention-
conforming online news page. The fact that participants in the non-conforming group
also adopted allied to the fact that they were all heavy online news users could
indicate that the same strategy they would use in a convention-conforming news
page was transferred to a website which did not conform to the visual conventions of
the genre.
273
Learning
Participants who used the conforming online news website agreed that the website
was easy to learn. But, while perceptions of ease of learning in the conforming group
remained unaltered throughout the experiment, in the non-conforming group they
changed. In the beginning of the experiment, participants of this group disagreed
that the website was easy to learn, but towards the end they neither agreed nor
disagreed that the website was easy to learn. This supports the theory that after some
time using the website participants realised that the strategies and mental processes
associated with the online news genre were applicable to the non-conforming
website. Thus learning to use the website became an easy task.
Indications that learning occurred in the non-conforming group were also seen
during the ‘think aloud’ phase where a considerable number of participants
expressed either implicitly or explicitly having learned the meaning of the icons in
the navigational menu.
8.4.6. Conclusion
Overall, results from this experiment are in accordance with the results from the ones
obtained in the first part of experiment 5 (see Chapter 7). Below a comparison of the
results in both experiments is made.
Performance
In the first part of experiment 5, the non-conforming group performed worse than
the conforming group while in experiment 6 the difference was not significant.
More interesting, however is the confirmation of the performance improvement of

participants using the non-conforming pages in experiment 6. As in the first part of
experiment 5, in experiment 6, participants using the non-conforming page showed a
significant upward trend in their performance as the test progressed. Also
confirming the results in experiment 5, the conforming group showed no significant
trend in their performance.
Disorientation
Similar to what occurred in experiment 5, in experiment 6, participants using the
non-conforming news website felt less lost and disorientated after using the website
for some time. As in experiment 5, in experiment 6 participants from the conforming
274
group did not report any changes in their feelings of disorientation and ‘lostness’
throughout the test.
Also in terms of the use of the back button, on both experiments 5 and 6 the results
were the same: the group using the non-conforming website performed significantly
more backtracking actions than the group using the conforming online news site.
Examining the graph of number of backtracking actions over time (Figure 130) it
seems that for the group using the non-conforming site, the third task represented a
turning point in the number of times the back button was used. Considering the
relation between number of backtracking actions and disorientation it is possible that
after the third task participants started feeling less lost and may have been able to
orient themselves better within the website.
Finally, while in experiment 6 participants in the non-conforming group altered their

perceptions of how easy it was to navigate the site over the course of the test, in
experiment 5 their perceptions remained unaltered. This could be explained by the
different instances where the questionnaire was applied between the two
experiments. While in experiment 6, the questionnaire was applied after participants
have finished the first task and after the 5th task in experiment 5 the questionnaire
was applied after the first part (participants had already carried out 10 tasks) then
after the second part (participants had carried out another 5 tasks).
It is likely that, in experiment 5, by the time participants had to answer the

questionnaire they had already figured out how to navigate the website and their
answers reflected that. Because in experiment 6 the questions were asked straight
after the first task had been completed the variations in their perceptions on ease of
navigating the site were more evident.
Learning
As expected, in both experiments for participants who used the conforming website
perceptions on how easy the website was to learn remained constant. In all cases they
agreed that the website was easy to learn.
With the participants in the non-conforming group perceptions of ease of learning

were not the same between the two experiments. In Experiment 5, perceptions of ease
of learning remained unchanged across the test. Participants neither agreed nor
275
disagreed that the website was easy to learn. On the other hand, in experiment 6
participants initially disagreed that the website was easy to learn, but changed their
perception towards the end of the test, neither agreeing nor disagreeing with the
statement.
It is likely that in experiment 5, the same explanation for the lack of difference in
perceptions of ease of use of the non-conforming group applies to the lack of
difference in their perceptions of ease of leaning. By the time that participants were
asked to answer the questionnaire they had already learned to use the website and
from then on their perceptions remained constant.
However, the data obtained from the ‘think aloud’ phase of the experiment described
in this chapter seems to indicate that learning has occurred since participants have
learned certain features on the non-conforming layout throughout the experiment.
Final remarks
The results of the experiment described here have satisfactorily replicated the results
found in earlier studies of the web forum genre. This strengthens the confidence that
these results can be extended to other website genres.
Results from this study on online news genre along with the previous results from
web forum genres show that users of a web genre suffer an initial decrease in
performance and an increase in disorientation when navigating through a website
which violates the visual conventions of the genre in search of information. The
results also indicate that performance improves quite rapidly and feelings of
disorientation diminish after a very short time using the website.
This study however, goes beyond that and shows that heavy users of a particular
web genre, can adopt the same strategies they would use in a genre-conforming
website in a website which violates the visual genre conventions. However,
considering that initially these users were lost and were not completing tasks
correctly, it seems that at some point during the process of using the website
participants realised that the same strategies used in a convention-conforming site
could be applied successfully. What may have caused this realisation as well as if that
realization signalled the start of the performance improvement is yet to be
investigated.
276
Chapter 9
Conclusion
9.1. Introduction
This thesis investigated the relationship between some of the forces involved in the
design process, specifically web document genres and usability. As a strategy for
studying and categorizing genres, they were viewed as a set of conventions and these
were divided into structural, linguistic, visual and functional conventions. Through a
review of genre and usability literature it established that genres are directly linked
to the usability principle of consistency. Since usability literature recommends
maintaining consistency on websites to improve usability (Nielsen, 1999a;
Shneiderman & Plaisant, 2005), conforming to the conventions of a web genre should
also improve usability of a website belonging to that particular genre. Literature
suggests that not conforming to conventions impairs usability (Nielsen, 1999a) and
makes users lost, confused and disorientated (Cappel & Huang, 2006). Research,
however, has so far provided conflicting empirical evidence on the effect on user
performance of non-conformance with conventions, with results ranging from
serious performance hindrance (Vaughan & Dillon, 2006) to no difference in
performance (Kalbach & Bosenick, 2006).
This thesis suggests that the usage of the term convention in usability literature is
quite broad, specially when web genres are involved. It proposes that before any
assumptions are made regarding conformance to conventions, it is first necessary to
establish which type of convention is being conformed to: structural, linguistic, visual
or functional. The thesis then sets out to investigate how not conforming with the
visual conventions of a website genre affects the performance and orientation of
users.
277
Conclusion
A systematic approach was chosen to investigate which visual attributes would make
up the visual conventions of the web forum genre. To do this, three attributes of the
visual elements of a web forum were scrutinized: symbolization mode, screen
position and mode of configuration. From the outcomes of this investigation, a
convention-conforming template was produced and validated by web forum users.
The approach proved successful in determining the visual attributes which make up
a convention conforming web forum layout. The principle of surveying the three
attributes of the visual elements of a web genre was then applied to determine what a
convention-conforming layout of the online news genre should look like. Again, the
approach was successful in finding a layout which conformed with what users
thought a conventional online news page should look like.
Finally, a series of experiments were put in place to test the impact of not conforming
to the visual conventions of a web genre on its users. Results from the experiments
showed that although user performance did indeed suffer in the non-conforming
website and users were disoriented, performance soon improved and disorientation
diminished. This suggests that not conforming to the visual conventions of a website
genre may not be have such dire consequences as suggested in the literature.
Considering the continuous growth of the number of websites in the World Wide
Web and the fact that many of these websites have frequent changes in layout (e.g.
the Amazon-UK website from 2000 to 2005 has made at least one significant layout
change a year to its homepage and The Independent website from 1999 to 2006 has
made substantial layout changes to their homepage at least once a year*), the results
presented here serve as empirical evidence to help designers understand, and
consequently better balance some of the forces involved in the design process such as
genre conventions and usability.
9.2. Overview of the thesis

This thesis began by establishing a framework to serve as the basis of the empirical
investigation which was conducted later. To define the framework a literature review
was made. Drawing from genre theory, pattern languages and usability, a link
between these was established. It was found that further investigation on the effects
of genre conventions was needed in order to improve the understanding of the
impact of not conforming to conventions on the user of electronic documents.
*
Source: The Wayback machine Internet archive –www.archive.org/web/web.php
278
Conclusion
Two exploratory studies were then conducted to investigate two aspects of genre
conventions which are more relevant to designers: structural and visual conventions.
Exploratory study 1 examined the effect of organizing content in an online text using
conventional access structures (such as headings and titles) and spacing, on reading
comprehension and structured recall. Results from this study indicated that the use
of conventional structures (colour-coded headings and spacing) to configure and
organise text helps participants to recall the content in an ordered manner.
Exploratory study 2 examined the effect of visual appearance on the ability to identify
the category of a website. This study showed that most participants managed to
identify the category of a website based only on its layout. It also indicated that some
visual elements seem to be associated with certain categories of websites. Even
though this experiment concentrated on website categories rather than web genres
(as some categories used in the study could belong to a single web genre) its results
are in accordance with Toms (2001) who found that users of a document genre firstly
attempt to identify the genre of a document by its visual appearance. This becomes
extremely relevant if the model proposed by Vaughan and Dillon (2006) which
suggests that when users of a certain genre of documents are in the presence of
certain visual cues associated with that genre a mental models triggered which can
help them navigate through the document and find information easily.
Based on the results of these two exploratory studies, it was decided that further
investigation on visual conventions of web genres was going to be pursued.
To investigate the effects of visual conventions of a web genre on its users, two
categories of web forums were chosen: for the initial studies web forums were picked
and for the final study online news was used. These choices were supported by the
characteristics of genres given in Swales (1990): a genre depends on communicative
events to occur and on the existence of a discourse community. Because the main
purpose of web forums is to facilitate communication between its members, it was
considered as a strong candidate for an emergent web genre. Online news websites
have on the other hand frequently been cited in literature as a web genre (Åkesson,
2003; Eriksen & Ihlström, 2000; Ihlström & Åkesson, 2004; Vaughan & Dillon, 2006)
and so were thought appropriate for the purposes of this thesis.
279
Conclusion
Survey 1 was then carried out to define which visual elements occurred more
frequently on web forums, how they were presented and located on the page and
also how their components were configured. Based on the results of this survey a set
of the most commonly occuring elements was established. Then, a series of templates
was designed ranging from the ‘most conventional’ layout (with the most frequently
occurring symbolization modes, position and configuration) to the ‘least
conventional’ layout (with the least or non occurring symbolization modes, positions
and configurations of each element).
To verify how the results from Survey 1 conform with the perceptions of web forum
users and therefore choose which templates would be used to create the material for
future experiments, three experiments were conducted. Experiment 1 assessed which
templates were perceived as more conventional by web forum users. Experiment 2
assessed which templates were perceived as easy to use by web forum users. From
the results of these two experiments a correlation between perceptions of
conventionality and perceptions of ease of use was established. From the results it
was also determined which templates would be used to design the future
experimental material.
Experiments 3 investigated aspects of the finished material. It examined how the users
identified the graphic and interactive elements of the non-conforming interface, and
assessed whether the conforming and non-conforming materials were perceived as
such by web forum users. The results showed which graphic elements were more
easily identified and which caused more problems identifying. Participants had no
difficulty in identifying the interactive elements of the non-conforming material.
Results also confirmed that the conforming material was perceived by web forum
users as being more similar to existing web forums than the non-conforming
material. These results were used to inform the design of the next experiments.
Experiment 4 investigated the effect of not conforming to visual conventions on the

performance of users who are familiar with web forums and others who are not
familiar with web forums executing different types of task. Results showed that
participants familiar with web forums performed better than participants unfamiliar
with the genre on both conforming and non-conforming layouts. Thus, no effect of
conventionality on performance was detected for either group. An interaction
between conventionality and task type was detected and it indicated that there was a
greater difference in performance between content and interface tasks for
280
Conclusion
participants in the non-conforming group than for participants in the conforming

group.
Experiment 5 further investigated the lack of difference in performance of participants

familiar with web forums on both the conforming and non-conforming layouts.
Results showed that participants who used the non-conforming web forum
improved their performance over the course of the experiment. Results also revealed
that this performance improvement was carried-over to other websites which
followed the same non-conforming principles from the website used earlier. This
may be an indication that when faced with a website which does not fully meet the
expected visual genre conventions, users quickly create a mental schema for the new
website or adapt their existing schema to fit the new visual conventions. Results also
confirmed the interaction between conventionality and task type and indicated that
perhaps the style of interaction may change as participants become more familiar
with a non-conforming web forum.
Finally, Experiment 6 investigated if the results obtained in Experiment 5 could be

extended to another web genre. In this study online news genre was investigated.
The experiment confirmed that the participants using the non-conforming website
improved their performance over the course of the experiment. Similar to Experiment
5, in this study participants also felt less disorientated over the course of the test. This
experiment also indicated that over a period of time participants in the non-
conforming group learned aspects of the interface. This can explain the gradual
increase in user performance observed on both studies. Another interesting outcome
from Experiment 6 is the fact that by the end of the test both groups of participants
were using the same strategy to search for information. This suggests that users
navigating through the non-conforming website were using the strategies they apply
for information search in a website which conforms with the genre.
The fact that results from online news tests were similar to those of web forum tests
(i.e. the same performance improvement observed on users of the non-conforming
web forum was detected on users of the non-conforming online news website)
strengthens the generality of the findings and strongly suggests that these results are
extendable to other web genres. This means that breaking away from visual genre
conventions on the Web does not cause such catastrophic usability problems as
initially thought.
281
Conclusion
9.3. Contributions
9.3.1. Design
As mentioned in the introduction, only in the past decades have genre theories
started to consider the visual aspect of document genres as an important factor to be
studied and very few have gone deeply into it (Delin, Bateman & Allen, 2002;
Bateman & Delin, 2001; Toms, 2001; Waller, 1987a). This thesis contributes to the
knowledge of document genres by dividing the conventions that make up a
document genre into structural, visual, linguistic and functional conventions, thus
allowing more focused investigations to take place. Furthermore, it focuses on the
visual conventions, bringing the contribution of the visual appearance of document
genres to centre stage.
Conventionality and perceived ease of use

By focusing on the visual conventions of genres of documents, results from this thesis
reinforce the influence of the visual appearance of websites on different types of user
perceptions as has already been suggested by some authors (Kurosu & Kashimura,
1995; Michailidou, et al., 2008; Tractinsky, 1997; Tractinsky, et al., 2000). Results from
studies conducted for this thesis have shown a strong correlation between the
perceived conventionality of a webpage and how easy to use people perceive them to
be. For two different web genres (web forums and online news) users perceived
pages conforming to the conventions of the genre as easier to use.
This result relates to a particular finding from Kurosu and Kashimura (1995) that the
familiarity of participants with the interface layout was highly correlated with
perceived usability. Participants in their experiment associated the familiar
configuration of an ATM keypad with high apparent usability. The same correlation
was detected by Tractinsky (1997). Despite the strong correlation, both authors
concentrated their analyses on the relation between perceived aesthetics
(attractiveness of the layouts) and perceived usability. By doing so, the authors failed
to acknowledge that several studies have related familiarity with perceptions of
attractiveness in people (Langlois & Roggman, 1990; Langlois, Roggman, &
Musselman, 1994; Monin, 2003) and in objects (Halberstadt & Rhodes, 2000). Recent
results from a study conducted by Michailidou et al. (2008) found a correlation
between familiarity and aesthetics (familiar web pages are perceived as being more
aesthetically pleasing than unfamiliar ones). It is then, plausible to consider that
perhaps familiarity was one important underlying factor which guided the choice of
participants on both Kurosu and Kashimura (1995) and Tractinsky et al. (1997). Since
282
Conclusion
perceptions of conformance to the conventions of a genre require familiarity with

documents of that genre, the correlation between perceived conventionality and
perceived ease of use support the results described above (Kurosu & Kashimura,
1995; Tractinsky, 2000).
Thus, considering that perceptions of ease of use can influence user attitudes towards
an interface (Morris & Dillon, 1997), and determine whether or not they will use a
website (Hasan & Ahmed, 2007), this perceptual relation between conventionality
and ease of use is an important one to be considered in the negotiation process which
happens during the design process. When designing a website which does not
conform to the visual conventions of a particular genre, designers must be aware that
users familiar with the visual conventions of that particular genre will at first glance
perceive the website as not so easy to use as another one that conforms with the
visual conventions of the genre. This can influence their decision on whether or not
to use the website.
However, results from Experiment 6 have shown that user perceptions of ease of use
of the non-conforming website did change very quickly over time with participants
thinking the website was later easier to use than they had initially thought. The
change in perceptions of ease of use may have been quick because in this experiment only the
visual conventions of the web genre has been altered and it is possible that non-conformance
with other types of web genre conventions may have a different impact on user perceptions of
ease of use. Nevertheless it is important that designers keep in mind that perceived difficulty
of use induced by non-conformance of a website with visual conventions of a genre can be
quickly reduced as users interact with the non-conforming website.
Assessing visual conventions of a genre

The general method used for designing genre-conforming website pages was
successfully tried in two distinct web genres (web forums and online news) and can
be easily applied to any design process where the need to identify what the visual
conventions of a particular document genre are. The process can be separated into
four simple stages:
1) Representative samples of documents belonging to the genre are obtained.
2) Visual elements are identified in the samples and the most frequently
occurring ones are noted.
3) The attributes for each frequent visual element on each sample are noted.
283
Conclusion
4) Based on the most common values for the most frequently occurring
elements a layout is designed.
Even though this process of analysis is a common one amongst designers and is
normally a part of the design process the work presented here contributes to the
methodology through the systematic approach of dividing the visual characteristics
of an element into three attributes (symbolization mode, position and configuration)
and examining them for each frequently occurring visual element. Another
contribution, particular to web design, is the expansion of the grid system used by
Ihlström and Åkesson (2004) for the navigational menu, to include all visual elements
of the page, including the ones located outside the browser area and not visible
without scrolling.
Furthermore, considering that it is important for a designer to investigate a genre

before starting a design task (Pemberton, 2000), this method can help to find the
visual conventions of a particular website category or genre and consequently help
with identifying and classifying existing design solutions for a particular genre so
that it can be used to help define a corpus of design patterns, and subsequently a
pattern language.
Visual attributes and perceptions of conventionality and ease of use

Even though literature suggests that website elements be placed in conventional
positions where users expect them to be (Bernard, 2001a, 2002; Markum & Hall, 2003)
results from Experiments 1 and 2 (described in chapter 5) show that in fact the
symbolization mode of the visual elements on a website is a more influent factor on
user perceptions of conventionality and ease of use than the positioning of these
elements. As such, designers should consider that altering the location of an element
from a conventional, expected place to an unexpected position on the screen might
not affect the perceptions of conventionality and ease of use as negatively as
changing perhaps a textual menu item to an icon will.
It is important to add that in some web genres, the configuration of the components
of an element may also play an important role in influencing perceptions of
conventionality. For example, in the online news genre, even though the
symbolization mode of the menu was quite important in helping participants to
judge the conventionality of the layout, the configuration of the components of a
284
Conclusion
news piece (headline, summary and picture) also played an important role in guiding
their judgments. This suggests that in other genres different attributes may play a
role that is just as important or even more important than the symbolization mode of
the elements on user perceptions of conventionality and ease of use. Further
investigation on this matter can allow for better understanding on the importance of
each element attribute in user perceptions of conventionality and ease of use.
Task type
The interaction between conventionality and task type is relevant as it indicates that
the type of task users are executing may influence the effects of conventionality on
their performance. Experiments in this thesis examined two types of tasks: retrieving
information from textual content in a website (called content tasks for simplicity) and
retrieving information from the website interface (called interface tasks). Content
tasks are one of the most common which web users consciously perform online.
Interface tasks on the other hand, are not often consciously performed but they are an
extremely common activity when navigating a website. During the navigation
process users are constantly scanning the interface for information which may aid
their search (e.g. an indicator that a new post has been placed in a thread) or give
feedback on executed actions (e.g. a change on the breadcrumbs of a website to
indicate that the user has moved from one page to another).
When designing a website therefore, designers and usability engineers must consider
the types of task which users will have to execute particularly if the website will not
conform to existing visual genre conventions. A clear example is illustrated by the
results from Experiment 5 which show that users perform worse obtaining
information from non-genre conforming interfaces than from genre conforming ones.
This is expected as, if visual conventions are altered, the interface elements may have
different screen locations, symbolization modes and configuration that can make it
difficult for users to obtain information from the interface. This can in turn hinder
user performance and make it difficult to navigate the website.
9.3.2. Usability
Genres and usability

This thesis contributes towards an understanding of the relationship between genres
and usability of web interfaces. The analysis of literature described in Chapter 2
linked genres of web pages and usability through the usability principle of external
285
Conclusion
consistencywhich is achieved when an interface is consistent with other interface

designs familiar to a user (Grudin, 1989). Therefore, genre conforming websites are
intrinsically externally consistent since they share a set of structural, linguistic, visual
and functional conventions. From this association several authors recommend that
conventions should always be followed (Bevan, 1997; Nielsen, 1999a; Shneiderman &
Plaisant, 2005; UPA, 2005) even when ‘a different design would be better if seen in
isolation’ (Nielsen, 1999b). However, as noted by van Schaik and Ling (2009) some
aspects of usability are not addressed by designs which conform to the conventions
of a genre and therefore this conformity does not guarantees that the design will be
usable.
From this stems the idea that a website which does not conform with the conventions
of a genre may not necessarily be less usable than one which does (Constantine &
Lockwood, 2002). It is true that when this happens, the usability principles of
consistency are violated, but some authors have pointed out that this is not
necessarily a bad thing and in some instances it may even be beneficial (Grudin, 1989;
Satzinger, 1991).
Structural, linguistic, visual and functional conventions and usability

By separating genre conventions into four components: structural, linguistic, visual
and functional, this thesis proposes that each of these components should be
investigated separately and whenever referring to violation of genre conventions,
literature should specify which of the components is being violated. This stems from
the idea that non-conformance with each type of convention may impact usability
differently.
This principle, allied with the outcomes from this thesis can explain the conflicting
results found between Vaughan and Dillon (2006) and the studies of Kalbach and
Bosenick (2006) and McCarthy et al. (2003). It is possible that because Vaughan and
Dillon (2006) violated both structural and visual genre conventions participants took
longer to learn to use the non-conforming website while both in Kalbach and
Bosenick and in McCarthy et al. (2003) learning happened much faster since only
visual conventions were violated (and in both cases only the positioning of the
menu).
Results from Chapters 6, 7 and 8 have confirmed that non-conformance with the
visual conventions of a genre does not seem to cause performance and disorientation
286
Conclusion
problems for very long as participants quickly learn to use the non-conforming
website and performance improves. This is illustrated by the lack of performance
differences between the users of conforming and non-conforming layouts in the
results of the experiment described in chapter 6 and the performance improvement
and decreased feeling of disorientation in users of the non-conforming layout in the
experiments in chapter 7 and 8.
This contributes to the debate on whether external consistency and conformance to

the visual conventions of a genre should be maintained at the expense of innovation
and even usability (Constantine & Lockwood, 2002). Based on the results presented
here, interface designers and systems engineers can make more informed decisions
on what is to be gained and what is to be lost on usability by following or not
conforming to the visual conventions of a genre which are determinants of external
consistency.
Task type, conventions and performance

Furthermore, user performance seems to vary depending not only on the level of
conformity of a website with the visual conventions of its genre but also on the type
of task participants are performing. As seen earlier, results from this thesis showed
an interaction between the task type and conventionality of the website participants
were using. Most interactions indicated that there was a greater difference in
performance between content and interface tasks in the non-conforming group than
there was for participants using the conforming websites. This indicates the added
importance of considering the role that each visual element on the website may have
on content retrieval tasks and also on general navigation before deciding whether to
make it conform or not with the visual conventions of the genre.
Navigation strategy
Experiment 6 showed that participants using the non-conforming website applied the
same strategy when searching for information as participants using the conforming
site. Considering that participants in the conforming group were heavy online news
users it follows that this strategy is commonly used in websites which conform with
the online news genre. However, the fact that this strategy was also adopted by
participants using the non-conforming website suggests that participants thought
that the same strategy would yield positive results in that scenario.
This reinforces the idea that users of a particular genre have schemas of that genre
stored in their memory which, upon encountering certain cues in a document, are
287
Conclusion
triggered and can help usage. It seems that participants using the non-conforming
site at some point realised that either the same, or part of the mental schema for
online news websites could be successfully applied to the non-conforming website.
Perhaps the fact that only visual conventions were altered, while structural, linguistic
and functional conventions were maintained allowed for this to happen. This may
not happen in non-conformance with other types of convention.
Contextual usability
Finally, this thesis reinforces the importance of a contextual approach to studies of
usability as suggested by some authors (Bevan, et al., 1991; Bevan & MacLeod, 1994;
Henninger, 2000; Maguire, 2001; van Schaik & Ling, 2009). A contextual approach to
usability considers that the characteristics of the users, tasks and environment are as
important as the characteristics of the product in order to determine the usability of a
system. Maguire (2001) expands the concept of ‘characteristics of the users’ to include
not only the main user of the system, but also the goals of the wider user community.
It becomes clear that the understanding of a digital genre and how it affects users
from its particular community is relevant if a contextual approach to usability is to be
pursued. It follows that, in order to evaluate the usability of a website under a
contextual approach, the non-conformance of a digital document with each type of
genre convention has to be weighed against the usability since they may impact the
performance of users from the genre’s discourse community differently.
This thesis takes a step towards improving this understanding since it has shown
that users can adapt reasonably quickly to non compliance to the visual conventions
of a digital genre.
9.3.3. Final remarks on contributions

This work has shown that the non-conformance with the visual conventions of a web
genre is not as bad as suggested in usability literature. Therefore, designers should
not be inhibited to innovate by applying design solutions that may go against
established visual genre conventions. It is important, however, that before decisions
of breaking away from visual conventions are made, the impact they will have on
user perceptions and attitudes towards the website as well their performance be
carefully considered against the improvements that the changes will bring to the
design.
288
Conclusion
Conventions are an important part of the design process, but they should never limit
or block innovation or usability. People adapt quickly to layout changes, even the
most extreme ones (the material used in this thesis is a proof of that). What needs to
be considered by designers is what cost will the non-conformance to the visual
conventions of the genre have to the user and what benefits it will bring to them after
they have adapted to the new design.
9.4. Limitations and suggestions for future research

However, this thesis only investigated the search for information when a specific
target was defined (search browsing). It may be worth pursuing in other experiments
the impact of not conforming to visual genre conventions when users are performing
other styles of browsing such as serendipitous browsing and general purpose
browsing (Catledge & Pitkow, 1995; Salomon, 1989). It is possible that the impact of
not following visual genre conventions on user performance may vary according
with the browsing strategy that users are applying. It is possible for example, that
serendipitous browsing (a common strategy used by online news readers) may suffer
by the configuration changes applied to the news elements in the non-conforming
news website. Because serendipitous browsing relies on chance discovery of a topic
of interest while browsing the website, the change in the configuration of the news
piece elements (by altering the relative position of headline, summary and picture)
may cause confusion since users may not identify these components as a single news
piece and therefore hinder the location of topics of interest.
An interesting result from Experiment 4 was the lack of performance difference

between the two groups of unfamiliar users: one using a conforming website and the
other using a non-conforming website. A further experiment could investigate if this
may be due to both interfaces taking the same time to be learned.
Experiment 6 gave some indications that online news users were adopting the same
strategies they use on conforming websites on the website which did not conform to
the visual conventions of the genre. Two interesting points of further investigation
are: 1) to evaluate the development of the strategies used by the non-conforming
group over time and 2) see if this transference of strategy happens when other genre
conventions are altered.
Evaluating how the strategies used by the group navigating a non-conforming

website evolve over time can clarify among other things: (a) whether there is a point
289
Conclusion
of realisation that the same strategies used in a genre conforming site apply or (b)
whether participants apply the same strategy from the beginning and stick to it until
by trial-and-error they learn the interface and the strategy pays off.
Investigating whether the transference of strategy happens when the website does
not conform with other types of conventions can help predict if the impact of not
conforming to certain types of genre conventions will be stronger than others. It is
possible that not conforming to structural conventions will demand a greater effort
from participants than not conforming to visual conventions, since it may require
deeper processing to adapt the existing schema or even require the generation of a
new one. This could again help explain the great impact on user performance that the
study conducted by Vaughan and Dillon (2006) had. Since in their study both visual
and structural conventions have been altered, it is possible that participant had to
build a whole new mental schema for the non-conforming website.
As seen in Chapter 7, the two parts of Experiment 5 were quite close together in time.
This interval may not have been sufficient to establish whether participants would be
able to maintain the performance improvement after a longer period of time without
interacting with the non-conforming website. Therefore a longitudinal study with a
larger time interval between trials is necessary to determine if participants can
maintain the performance improvement using the non-conforming interface. If the
performance improvement is maintained over a large time interval without practice
it could indicate that participants not only have adapted to the non-conforming
layout, but that they also have learned it and possibly either generated a new mental
schema for this new design or adapted their existing genre schema. On the other
hand, if after a large time interval without using the website user performance is
again low it would be likely that the adaptation is a temporary thing and that real
learning, and perhaps the development or adaptation of a mental schema, may only
occur after repeated exposure.
Finally, this study concentrated on the visual conventions of a web genre and
although some aspects of structural and linguistic conventions of genres have been
addressed by Vaughan and Dillon (2006) and Dillon and Schaap (1996), further
investigation is needed to determine how the non-conformance with other types of
genre conventions (structural, linguistic and functional) can affect the user of
electronic documents and more specifically web pages.
290
Conclusion
It is also important to determine the relationship between structural, linguistic,

visual and functional conventions of a genre including how much each type of genre
convention contributes to perceptions of conventionality and ease of use. This
knowledge becomes particularly important in case non-conformance to one type of
genre convention makes users perceive a web page as harder to use than non-
conformance to the other types of genre conventions. Since, as seen earlier in this
chapter, perceptions of ease of use influence user attitudes towards an interface
(Morris & Dillon, 1997), and determine whether or not users will use a website
(Hasan & Ahmed, 2007) this knowledge can be crucial for the success or failure of a
web site.
When results from Vaughan and Dillon (2006) are examined under the light of this
thesis they seem to indicate that non-conformance with some types of genre
conventions may have a greater impact on user performance than others. Therefore,
more research is required to identify how much the non-conformance to each type of
genre convention influences the performance of users.
291
Appendix A
Texts used in the first Exploratory study 1 which
examined conventional access structures
How Wine is Made

The difference between grape-juice, simple and sweet, and wine with all its qualities of
keeping, of maturing and finally of inspiring, is the process of fermentation. In
fermentation sugar is transformed into alcohol: grape-juice becomes wine.
Fermentation comes naturally to grapes. Wine did not have to wait to be invented. A
grape left alone could make wine if its skin were broken. Everything needed to make
wine is there already when the grape has ripened on the vine.
In the final stages of ripening, yeast cells appear from somewhere - it is not clear where -
on the skin of the grape and cling to it. As soon as its skin is broken they pounce on the
sugar in its juice, which is their object, and turn it into alcohol. Only a grape has enough
sugar for this to happen naturally. So-called wines are made from other fruit, but all need
extra sugar. Something like 30 per cent of the juice of a ripe grape is sugar, and this the
yeast cells set about converting.
Pressing
The way the grapes are broken and the container they ferment in are two of the most
variable things about wine. On small farms in Chianti all the grapes, black and white, are
bundled together in a large opening at the top of a barrel. They still have their stalks,
even the mud from the fields on them. They have only been slightly broken up, in the big
tub on an ox-cart into which they were gathered, by the farmer with a knotty club. He
plunges it up and down a few times to see that a fair proportion of the grapes have been
smashed, relying on the heat generated in the barrel when fermentation starts to swell
and burst the rest.
On a few farms in France, Portugal, Spain and some parts of Italy the grapes are still
trodden by bare feet, as they have been since wine was first made.
Larger wine-growers now all have presses or mechanical crushers to break the grapes,
but even these vary from old hand-operated squeezers to huge revolving horizontal
drums. The latest kind have a gigantic rubber balloon inside. When it is inflated it presses
the grapes against the slatted sides and forces out their juice without crushing their pips,
which contain unwanted and evil-tasting substances.
The biggest presses of all are those of Champagne, in which an enormous weight
descends by scarcely perceptible degrees on an enclosure containing four tons of grapes
at a time.
Usually the juice and broken grapes, or for white wine the juice alone, are pumped for
fermentation into vats or barrels, traditionally of oak but now more and more of concrete,
glass or stainless steel. Sometimes it is left to ferment where it has been crushed or
trodden.
Fermentation
The process of fermentation, which is what happens when the yeast cells get to work,
produces turbulent eruptions in the juice. Not only alcohol is made, but carbon dioxide
gas is given off in large quantities. Its bubbles keep the mass of broken grapes moving. At
292
Appendix A
the same time the heat given off by the reaction makes it warm.
Fermentation goes on until the sugar supply runs out, or until the yeast cells are
asphyxiated by the growing concentration of alcohol, whichever happens sooner. Usually
it is the sugar which gives out first. The liquid, then, instead of being a solution of sugar
and water, is a solution of alcohol in water, with the same small quantities of the acids
and oils which give it its peculiar flavour and scent. On average there will be about 10 per
cent of alcohol in red wine, a little more-11 or so-in white. There is no sugar left at all and
the resulting wine is completely dry.
If, on the other hand, the grapes were so ripe that there was still more sugar to ferment
when a concentration of 15 per cent alcohol was reached, it would be the yeast cells
which would give out first. In a solution of alcohol this strong they grow drowsy and
cease to function. They are not dead, but they are completely under the table. When this
happens there is still some sugar left unconverted into alcohol and the wine is more or
less sweet to the taste. This process is the basic one for the making of all wine and, come
to that, of any other alcoholic drink.
What is "Great" Wine?

Anyone who has high claims for his wine, then, tends to be as specific as possible about
it. He uses whatever cover the law of his country allows him-the most specific appellation
he is entitled to. He often goes further and bottles it himself, adding his personal
guarantee.
If these precautions seem over-elaborate, consider what danger there is in more

generalized naming. It is the same with food. The statement "Yorkshire pudding is
delicious" is of very limited value. The important thing is who cooks it.
Ordinary
The daily drink of real wine-drinking countries-not those for whom wine is a luxury-is
ordinary wine. In France vin ordinairehas a definite connotation. It is not a vague term
for anything which is not very exciting. Its price depends on its strength-nothing else. It is
almost always red. In Germany it is called Konsumwein, in Spain vino corriente
("current", or running, wine), in Portugal consume. We have no real term of our own for
it in English. Ordinaire is the word most commonly used. In America, particularly,
"ordinary" sounds more or less insulting.
Vin-de-pays
Vin de pays-wine of the country-is a cut above ordinaire. Though the wine in itself may
be no better it has one added dignity; you know where it comes from. Ordinaire is
completely anonymous. It is often a blend of the wine of France, Spain, North Africa,
Italy or anywhere where the price per degree of alcohol is low at the time. Vin de pays at
its worst is simply ordinaire with a birth-certificate. At its best, on the other hand, it can
be one of those excellent regional specialities that are always referred to in a patronizing
way as "little". A good hotel almost anywhere in the southern half of France should have
quite a good vin du pays (switched from de to du because it is referring to the particular
bit of pays in question). The patron gets it by knowing the grower. There is not enough to
become a widespread marketable proposition, but it is none the worse for that.
Good
With "good" wine we are on trickier ground. What does a dealer mean when he says that
a picture is good? What do you mean when you say that a dish is good? They are not
quite the same thing. "Good" must be allowed to cover all wine from just plain well-made
upwards to the top. But at the same time it has a field of its own, between ordinary and
fine. Whereas ordinary is not worth tasting with any attention, has no scent or
characteristics of its own, good wine begins to be worth thinking about. It is well made
and has its own character, its own variation on the general character of its region. In this
class come most of the wines of Bordeaux, for example, which are not Classed Growths
and, come to that, a few that are; the wines of Burgundy which are known by their
village-names alone, not the names of their individual vineyards; the blended German
293
Appendix A
wines such as Liebfraumilch; the Chiantis and Orvietos of Italy - all the wines which
carry the name of their type, when that name is one of the world's great wine-regions.
Fine
There is no general agreement on what is meant by the word "fine", often though it comes
into discussions about wine. I would narrow it down to the field above Good and below
Great. All wines which tell their full story-their vintage year, the particular vineyard and
not just the village where they were grown, often the name of the owner-on their labels
should come into this class. Most-though not all-estate-bottled wines deserve to be called
fine wines. In France the annual production of fine wines (in the broadest sense) in a
typical year is in the region of one seventh of the total (178 million Imperial gallons in a
total of 1,170 million in 1971).
Great
Great wine is a different matter. There is no vineyard in the world which always
produces great wine, however great and famous its name. Great wine is the rare and
exquisite result of perfect conditions in a perfect vineyard, perfectly handled by the
grower and carefully matured afterwards. Where the line comes between fine and great is
always a matter for debate. But the word should not be used lightly. A great wine is a
work of art, capable of providing aesthetic pleasure of the highest order to anyone who
will be attentive. As an everyday drink it is as fitting as Hamlet is for cabaret in a night-
club. Only five or six districts in the whole world have shown themselves to be capable of
producing wine like this with any regularity. They are Bordeaux, Burgundy, Champagne,
the Rhine and Moselle, the Douro river in Portugal and Jerez in Spain. At various times
great wine has been made in Hungary, Italy, South Africa, California, Australia, central
Portugal and maybe elsewhere. But it remains rare, and grows more and more expensive
as more and more people want to taste it. Even a convinced ordinary-wine drinker
should taste great wine sometimes. It is easy to get into a rut with wine and to wonder
what all the fuss is about when anyone mentions the glory of the best growths. I hope
nobody drinks them every day, because it is partly by comparison that their superlative
qualities stand out.
It is hard to put a finger on the quality of greatness in wine. To me there is always

something faintly sweet, but sweet in the sense of sweet-natured rather than sugary,
about one. There is a definite tendency to reach a quintessential taste of fresh grapes, as
even really fine cognac does although it has been distilled. But one thing they all have in
common, whether they are clarets, delicate fino sherries, champagne or hock or
burgundy: they provoke discussion. To drink a great wine alone is almost painful. There
must be somebody to share the experience with, for it is an experience, and it needs
discussing, analysing and gloating over just as a great play does; or, to put it more
precisely, a great picture, for it is not dramatic-it is beautiful.
294
Appendix B
Summary task and multiple-choice questions and
internet usage questionnaire used in the first
exploratory study
Part I – Summarization task

Summarize the text you have just read. You have 5 minutes to complete this task
Part II - Questions
1) Tick the option containing countries where you still find the traditional method of
treading grapes by bare feet being used. (You should tick only one option)
( ) Italy, Spain,Portugal, France.
( ) United Kingdom, Australia, South Africa.
( ) United States, Greece, Argentina, Chile.
( ) Greece, Australia, Brazil, Argentina.
2) Tick the option which shows right order of processes necessary to make wine.
( ) Harvesting > Juice extraction > Alcohol production.
( ) Juice extraction > Harvesting > Alcohol production.
( ) Alcohol production > Harvesting > Juice extraction.
( ) Juice extraction > Alcohol production > Harvesting.
( ) Harvesting > Alcohol production > Juice extraction.
3) Number the sentences according to the order they happen during the winemaking
process.
( ) The juice and broken grapes are pumped into vats or barrels.
( ) In fermentation sugar is transformed into alcohol: grape-juice becomes wine.
( ) The grapes are slightly broken up by the farmer with a club.
( ) Fermentation goes on until the sugar supply runs out.
( ) In the final stages of ripening, yeast cells appear on the skin of the grape and
cling to it.
4) Why is it so important not to crush the grapes’ pips during the process of pressing
them? (You should tick only one option)
( ) The carbon dioxide bubbles keep them moving impairing fermentation.
( ) They will generate heat when fermentation starts bursting the barrel.
( ) They contain unwanted evil-tasting substances.
( ) They give off acid oils that might ruin the wine.
( ) They will leave some sugar unconverted into alcohol and the wine will be sweet
to the taste.
5) Tick the option in which all the names refer to ordinary wine. (You should tick
only one option)
( ) Konsumwein, Consumo, Vino Corriente
( ) Consumo, Popular, Konsumwein
( ) Common, daily, Ordinaire
( ) Popular, daily, Konsumwein
295
Appendix B
6) Assign (T) for true or (F) for false for each of the sentences below:
( ) Good wine is usually an “ordinaire” wine with a birth certificate.
( ) Usually the yeast cells can work up to a 15% concentration of alcohol
( ) The world’s largest wine presses are located on small farms in Chianti.
( ) Not many regions of the world have been able to produce great wine regularly
( ) Fine wine is the rare and exquisite result of perfect conditions in a perfect
vineyard.
Part III – Internet usage questionnaire

1) You are: ( ) 17 – 25 years old
( ) 26 – 35 years old
( ) 36 – 45 years old
( ) 46 – 55 years old
( ) 56 – 65 years old
( ) > 65 years old
2) Academic Level: ( ) Elementary School ( ) Graduate ( ) Post Graduate
3) How frequently do you use the internet (tick the answer that applies)
( ) Daily.
( ) Every other day.
( ) At least once a week and less than every other day.
( ) At least once in a fortnight and less than once a week.
( ) At least once a month and less than once in a fortnight.
( ) Never.
4) When you are connected, how long on average do you stay connected? (tick the
answer that applies)
( ) < 1 hour
( ) 1 – 3 hours
( ) 3 – 5 hours
( ) > 5 hours
5) Which kinds of online activities you usually perform when connected? (tick all that
apply)
( ) Search for information using a search engine ( Google, Lycos, Altavista, etc).
( ) Browse the internet for fun.
( ) Compare price information using a website (NexTag, Froogle, etc).
( ) Send and receive e-mails.
( ) Play a game.
( ) Pay Bills.
( ) Bank Online.
( ) Buy something on an online auction (eBay, Ubid, etc.)
( ) Buy a product from an online store.
( ) Participate in a chat room.
( ) Listen to or download online music.
( ) Join a newsgroup.
296
Appendix C
Elements which helped identifying the website in
the second exploratory study
Comments made by participants in each category on both passes and the respective
keywords assigned. Keywords were based on the words in bold.
Category Comments Keyword

Web mail Two lists side by side similar to my e-mail list List
Sequence of topics organized as a list List
Ordered List List
to; from; size; date columns To-From Columns
'To', 'From' To-From Columns
To-From
'From', 'To', 'Date', square to check on or off
Columns/Checkbox
Small squares followed by numbers. Checkbox
Check box, same format of e-mail sites. Checkbox
Square boxes to check/uncheck Checkbox
Lines of a table Table
Table with lines with messages. Table
Use of colour to highlight items that you are supposed to read and what you have read Colours
Pictures of letters, lines of text. Envelope Icon
elements organization looked like an e-mail interface
Online
'News', 'sports', bizarre, looks like a tabloid. Words
News
'News' written. Words
The word "news" and a picture of the London bomber. Words / Picture
Picture of suicide bomber Picture
Underlined texts, pictures. Picture and Text
Headlines / Picture &
Way the pictures and text were arranged, with headlines
Text
picture of policemen and text next to it Picture and Text
A picture of British police with some text beside it. Picture and Text
Picture of London and a text. Picture and Text
Picture with text next to it. Looks like Hotmail news web site Picture and Text
Picture and Text /
A picture with text next to it. Headlines
Headlines
Presented as a list of headlines and therefore, don't look like "online news" Headlines
One big headline with a link where you can click to know more Headlines
Colours and layout of information. Colours / Layout
Layout looks like a newspaper. Layout
Has a date, the News where you can click. Date / Links
News with links to click Links
297
Appendix C

Search
the redundancy of names (Leonardo da Vinci) Repeated names
Engine
List of names displayed like a result from the search Repeated names
Items, many results. Repeated names
Same name repeated several times Repeated names
'London' displayed many times. Repeated names
list of links ad numbers accompanied by a summary Links and summaries
Many links where you can click to know more. Links
list of links on one topic Links
Picture with a name and a description highlighted Picture & description
Tourism or entertainment with the word London. Words
Items ordered on the left alternating colours. Ordered Items
Search results. Two of them separated from the rest. Layout
Search results about London similar appearance to all search engines. Layout
e-
Piece of advertisement Advertisement
commerce
Picture of book or DVD cover. E-commerce because a search engine usually doesn't Advertisement /
have adverts. Pictures
Supermarket online, it has the products displayed. Pictures
Pictures displayed as tiles on the page. Pictures
Pictures framed, small pictures that resemble movie posters Pictures
squares with pictures that resembled DVDs Pictures
DVD covers pictures Pictures
Shows products pictures that look like book covers Pictures
Picture of a movie is shown although the "Summer Holiday" ad distracted me a bit. Pictures
DVD picture, price, link 'buy', new releases. The advert mislead on the real business Pictures / Price /
of the site. Words
Word "what's new" Words
'food and drinks', 'special offers', 'fast food', 'buy one get one free'. Words
It’s written on the website "books" but there was no price. There is a shopping cart so
Words
it’s an e-commerce.
Word ‘Carrefour’, many products, search interface. Words
'Carrefour', to make your shopping list. Saw Carrefour written. Words
'Carrefour' word, type of products. Words
sentence "shopping online is easy", the section delivery is separated by a tab Words
Read the word "shopping". Words
Word 'shopping online'. Words
Sentence "shopping on line" Words
Products’ brands 'Flora', 'Sainsbury' Words
Sentence "summer sale" Words
Price of products, a line of text. Price
List / Pictures and
list of books using pictures and text
text
'War of the worlds' written next to a picture of the movie. Pictures & text
Rectangular picture that looked like a DVD or a book and next to it links Pictures and Links
Shopping cart, with a review on the film. Shopping Cart
E-commerce…there is a shopping cart…video, DVD Names of similar products (drinks)
Shopping Cart
Mentioned that it didn't look like a supermarket website.
298
Appendix C

Online
the sentence "current account card insurance" Words
Banking
words "current account" Words
Card insurance written on the middle of the screen Words
‘Internet 30hs’, ‘current account card insurance’. Words
word "account" Words
name of branch transactions were listed Words
word 'account' Words
Words
word 'security' and had some tabs on top that are typical of bank
Tabs
Words /
Bank, there is an advert of card insurance. 'Current Account' written.
Advertisement
Saw a picture of a credit card. Picture
Items ordered on table with dark/light grey colours. Table
Online
the words "express auctions" Words
Auction
Saw word 'auctions'. Words
Products, 'bids' and 'time left'. Saw someone using something similar before. Words
'bid now', 'auction' written on a tab. Words
It reads 'Price', 'offers', 'visits', 'ends on'. 'How to buy' and 'how to sell' on top. Words
'how to buy', 'how to sell', 'safe shopping' Words
It’s not very clear that it is an auction site, looks like a search engine. Small pictures
Picture & description
of products and their descriptions
small picture of products and text next to it Picture & description
The pictures are small and all of them of the same size, the text is small and looks like
Picture & description
a description of the product.
Many colours, pictures of products. Colours / Pictures
Total Frequency count per elements mentioned for each website category.
Webmail Search Engine Online news
Element Ef Element Ef Element Ef

Checkboxes 4 Repeated Names 5 Picture and Text 6
List 3 Links 2 Headlines 4
To-From Columns 3 Layout 2 Words 3
Table 2 Links and Summaries 1 Picture 3
Colours 1 Picture & description 1 Layout 2
Envelope Icon 1 Words 1 Links 2
Ordered items 1 Underlined Text 1
Colours 1
Date 1
e-commerce Online banking Online auction
Element Ef Element Ef Element Ef

Words 13 Words 9 Words 6
Pictures 9 Tabs 1 Picture & description 3
Advertisement 2 Advertisement 1 Colours 1
Price 2 Picture 1 Pictures 1
Shopping Cart 2 Table 1
List 1
Pictures & description 1
Pictures & Links 1
299
Appendix D
Criteria used for determining the screen position of
a visual element for the survey
The criteria used in the survey described in Chapter 4 to determine what the position
of a visual element of a web forum was in relation to the grid is described below:
(a) When more than 50% of the element is contained in a quadrant, its position is
assigned to that particular quadrant.
(b) When an element occupies three consecutive horizontal or vertical quadrants the
position associated with it is the centre quadrant of the respective line or column in
which it appears. This usually happens due to two factors: the size of the element
and/or its mode of configuration. This can be observed in the menu element (Figure
27) which has a horizontal configuration thus occupying the three top quadrants.
This criterion was adopted because when assembling the non-conforming version the
same element might have a different configuration. Having all three quadrants
assigned to the element could make it problematic to assign a non-conforming
position to it.
(c) Because of how the grid is constructed, when an element occupies two adjacent
horizontal quadrants these always consist of one central quadrant (top centre, left
centre, right centre bottom centre) and another outer quadrant (top left, top right,
bottom left and bottom right). In this case, the position of the element was assigned
to the outermost quadrant. This will avoid a great concentration of elements
described as in the centre of the page. In Figure 27 the element ‘Last post’ occupies
both the top row and the centre row, but since more than 50% of its area is in the
central row, it considered to be located in the central row. But within the central row,
it extends itself from the central quadrant to the centre-right quadrant occupying two
quadrants at the same time thus being noted as located at the outmost quadrant, the
centre-right quadrant.
300
Appendix D
(d) When an element appears in two or more different positions simultaneously on a

page, a separate category will be created with all occurring positions and its position
will be allocated there. As an example, suppose that one element appears
simultaneously in the top left and bottom left of a page. Instead of counting one
occurrence in the top left and one occurrence in the bottom left (which would not
properly represent how the element is displayed), a ‘top left and bottom left’
category will be created and the element will be counted there. The reasoning behind
this can be illustrated with an example: consider an element which appears 9 out of
10 times simultaneously in the top left and bottom left of the screen and one time in
the top left. If multi-positions were attributed to each single location, the most
frequent position would be considered to be the top left. But in fact, the most
frequent layout would be the one where the element appears simultaneously in the
two positions.
301
Appendix E
Explanation of elements found in all three levels of
a web forum
I) Elements found in the Forum index page (first level) of a web forum
! Forum index table – This element is made up by a group of components that

are usually placed together in a table. It occupies a large portion of the forum
area.
! Menu – This is the element that contains the links related to the user account
management and forum administration.
! Logo – The logo of the forum.
! Login field – This element is where the registered users log themselves in the
forum.
! Captions – Its function is to explain the codes used within the forum.
! Current time and date – Present the current time and date in the country
where the forum is held.
! Users information area – In this area all the information regarding users in
the forum are displayed (e.g. birthdays, new users).
! Breadcrumbs – This element works as a locator in the forum space,
illustrating to the users not only where they are but also the navigational path
that took them there.
! Forum notice board – Here the forum moderators can post announcements
and notes to all visitors of the forum.
! Adverts – These are usually links that, when clicked on, take the users to
other pages, not always related to the forum. These adverts are usually a
source of income to the forum creators.
! Forum search field – Here the user can look for a particular topic within the
forum.
Within the Forum index table, the following elements were found:
! Topic title – This element is a link to a topic of a specific subject within the
forum.
! Topic summary – A brief description of what that topic is about.
! Topic status indicator – Indicates to the user the current situation of a
particular topic (e.g. if somebody has posted something since the last visit, if
that topic allows posting or not).
! Number of threads in topic – Displays the number of threads that a
particular topic contains.
! Number of posts in topic – Shows the total number of postings that a
particular topic has.
! Last post in topic – Informs the user when the last post in a topic was made.
! Topic moderator – Indicates which user is responsible for managing a certain
topic, most of the times the forum moderators are the ones who manage all
topics in a forum but some forums give the responsibility for managing the
topic to the user who started it (see 31).
302
Appendix E
! Number of views in topic – This element shows the total number of views
that a particular topic in a forum had.
II) Elements found in the Threads page (second level) of a web forum which are not
in the forum index page.
! Thread table – Similar to the forum index table, this table contains a list of all
threads for the particular topic chosen.
! New thread button – Allows the user to create a new thread.
! Number of thread pages – This element is usually a sequence of links and it
appears when the number of threads is too big and consequently the list is
too long and has to be divided into pages. When clicked upon it takes the
user to the desired page.
! Jump to forum – This element is a list of the topics in the forum and allows
the user to quickly move from one topic to another without having to go back
to the forum index table.
! Posting rules – Establishes what the user is and is not allowed to do on that
particular level of the forum.
! Forum search – Allows the users to search for a specific topic or subject
within the whole forum.
! Popular topics – Shows which topics have been most visited and/or have
had the biggest number of posts.
Within the threads table, the following elements were found:

! Thread status indicator – Similar to the topic status indicator, this indicates
to the user the current situation of a particular thread (e.g. if somebody has
posted something since the last visit, if a thread allows posting or not).
! Thread name – It is the main title of a thread.
! Thread author – Presents the name of the author of a thread.
! Last post – This element informs the users when the last message was posted
in a particular forum. It usually contains the date and time of the post along
with the name of the person who wrote it.
! Number of replies – Indicates how many replies a particular thread has had.
! Number of views – Shows the number of views (hits) that a thread has had.
! Emoticons – These are usually small graphic images that are mainly used to
demonstrate the mood of the author of a thread.
III) Elements found in the Post page (third level) of a web forum which are not in the
forum index page nor the threads page.
! Post table – Here is where the members of the forum leave their comments
on a specific topic or discussion.
! Next/previous thread button –These buttons allow users to navigate forward
and backwards within the threads without having to return to the threads
page.
! Reply to post button – This button allows the users to reply to a specific post
from a user.
! Thread title – This consists of the title of the thread the user is actually on.
! New post button – This buttons allows users to start a new topic in a thread
without having to return to the threads page.
! Table of contents – Indicates in a list format all the postings and replies to a
post that a posting area contains.
303
Appendix E
Within the post table, the following elements were found:

! Time & date of post – Displays the time and date a particular post entered
the system.
! Post content – This is the actual content that a user has posted.
! Author info – This is and area that contains information about the author
such as the date they entered the forum, their nickname, sometimes an avatar
(picture or symbol that represents the author) or a link to further author
information.
! Quote post button – When clicked on allows the user to include a quote from
another post in his post.
! Post title – It is the title of a post, different from the topic title.
! Signature – It consists of a sentence and sometimes a picture that users insert
after their posting that works like a trademark or a signature of that
particular user.
! User profile button – This button links to the profile of a user that has posted
something in a forum.
! Private message button – This button allows users to send messages to
another user that has left a post in a forum.
! Post number – Indicates the number of a post within a thread.
! Edit post – This button is only active for the users that have posted
something to edit what they have written.
! Author’s website button – This button links to the website of the user that
has written a particular post.
! Email author button – This button allows a user to send an e-mail to another
user that has written a particular post.
304
Appendix F
Tables with the symbolization mode frequencies of
each element appearing on each of the three forum
levels
The tables illustrated in this appendix show the frequency of each symbolization
mode of the web forum visual elements listed in the survey described in Chapter 4.
TABLE 51 - FREQUENCY OF SYMBOLIZATION MODE OF THE ELEMENTS ON THE FORUM INDEX PAGE.
THE INDENTED ELEMENTS ARE FOUND WITHIN THE INDEX TABLE.
Forum index elements Graphic Alphanumeric Hybrid

Forum index table 0 1 14
Topic title 0 15 0
Topic summary 0 14 0
Nº of threads 0 13 0
Nº of posts 0 13 0
Last post 0 3 10
Topic moderator 0 5 0
Nº of views 0 1 0
Status indicator 12 0 1
Topic type 0 1 0
Menu 0 11 4
Logo 0 7 5
Login field 0 11 0
Captions 0 0 10
Current time & date 0 10 0
User information area 0 4 6
Breadcrumbs 0 2 7
Forum noticeboard 0 6 1
Adverts 0 3 3
Forum search field 0 2 1
305
Appendix G
TABLE 52 - FREQUENCY OF SYMBOLIZATION MODE OF THE ELEMENTS ON THE THREAD PAGE.

THE INDENTED ELEMENTS ARE FOUND WITHIN THE THREAD TABLE.
Thread page elements Graphic Alphanumeric Hybrid

Thread table 0 1 14
Thread name 0 8 7
Thread author 0 0 14
Thread status 10 0 2
Last Post 0 5 9
Number of replies 0 14 0
Number of views 0 13 0
Emoticons 5 0 0
Menu 0 12 3
New thread button 0 3 11
Number of thread pages 0 9 5
Breadcrumbs 0 9 4
Forum jump 0 11 1
Captions 0 0 11
Logo 0 4 7
Posting rules 0 4 6
Forum search 0 4 1
Login field 0 5 0
Popular topics 0 0 1
TABLE 53 - FREQUENCY OF SYMBOLIZATION MODE OF THE ELEMENTS ON THE POST PAGE.

THE INDENTED ELEMENTS ARE FOUND WITHING THE POST AREA.
Post page elements Graphic Alphanumeric Hybrid

Post area 0 0 15
Time & Date of Post 0 5 10
Post content 0 10 5
Author Information 0 4 11
Quote button 1 3 8
Post title 0 10 2
Signature 1 8 1
User profile button 1 2 4
Private message button 1 2 3
Post number 0 6 0
Edit Post 1 2 1
Author’s website button 0 2 1
E-mail author button 0 2 1
Menu 0 12 2
Next/previous thread button 1 11 2
Breadcrumbs 0 4 9
Reply to post button 0 3 10
Jump to forum 0 12 0
Logo 0 5 6
Posting rules 0 8 0
Thread title(out post area) 0 6 0
Adverts 0 2 4
Login field 0 5 0
New post button 0 1 4
Search topic field 0 3 1
Table of contents 0 0 1
306
Appendix G
Appendix G
Tables with the configuration mode frequencies of
each element appearing on each of the three forum
levels
Tables 54, 55 and 56 show the frequency counts of the configurations of the elements
across the web forums and the indented elements appearing on each table are the
ones contained within the tables (forum index table, threads table) or in the post area.
As mentioned earlier on Chapter 4, for the elements which had only one component
the abbreviation of the words ‘not applicable’ (N.A.) was used in the tables to
indicate that no configuration mode was established for this particular element.
TABLE 54- FREQUENCIES OF CONFIGURATION OF ELEMENTS IN THE FORUM INDEX PAGE.

THE INDENTED ELEMENTS ARE FOUND WITHIN THE INDEX TABLE.
Forum index elements Horizontal Linear interrupted Matrix

Index Table 0 0 15
Forum Title 15 0 0
Nº of Threads 13 0 0
Nº of Posts 13 0 0
Topic moderator 4 1 0
Topic summary 1 13 0
Last Post 1 12 0
Topic type 1 0 0
Nº of views N.A. N.A. N.A.
Status indicator N.A. N.A. N.A.
Menu 12 1 2
Breadcrumbs 9 0 0
Logo 7 0 5
Login field 5 1 5
Adverts 5 0 1
Captions 4 0 5
User info area 1 0 9
Forum search field 1 0 2
Forum noticeboard 0 0 7
307
Appendix G
TABLE 55- FREQUENCIES OF CONFIGURATION OF ELEMENTS IN THE FORUM THREAD PAGE.

THE INDENTED ELEMENTS ARE FOUND WITHIN THE THREAD TABLE.
Forum thread elements Horizontal Linear interrupted Matrix

Thread table N.A. N.A. N.A.
Thread author 11 3 0
Thread name 9 6 0
Status indicator N.A. N.A. N.A.
Last post 1 13 0
Nº of replies N.A. N.A. N.A.
Nº of views N.A. N.A. N.A.
Emoticons N.A. N.A. N.A.
Nº of thread pages 13 0 1
Menu 12 1 2
New topic button 12 2 0
Breadcrumbs 9 4 0
Forum jump 6 1 5
Logo 6 5 0
Forum search 2 1 2
Captions 0 1 10
Posting rules 0 9 1
Popular topics 0 0 1
Login field 0 1 4
TABLE 56- FREQUENCIES OF CONFIGURATION OF ELEMENTS IN THE FORUM POST PAGE.

THE INDENTED ELEMENTS ARE FOUND WITHING THE POST AREA.
Post area elements Horizontal Linear interrupted Matrix

Post Area 8 7 0
Time & date of post 15 0 0
Post title 12 0 0
Quote button 10 0 2
Profile button 7 0 0
Private message button 6 0 0
Post number 6 0 0
Edit post 4 0 0
Poster website button 3 0 0
email button 3 0 0
Author info 1 10 4
Signature 0 10 0
Post content 0 15 0
Next/previous thread button 13 1 0
Menu 11 1 2
Reply to topic button 11 0 0
Breadcrumbs 9 0 4
Jump to Forum 7 0 5
Thread title(out post area) 6 0 0
Logo 5 2 4
New post button 5 0 0
Adverts 3 0 3
Search topic field 3 1 0
Posting rules 0 8 0
Login field 0 1 4
Table of contents 0 0 1
308
Appendix H
Tables showing the survey 1 results for the most
frequently occurring groups of elements
Element 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Freq
Index table 15
Menu 15
Logo 12
Login Field 11
Captions 10
Current Time & Date 10
User Info Area 10
Breadcrumbs 9
Forum Index Page
Forum Noticeboard 7
Adverts 6
Forum Search Field 3
Forum Table Elements

Forum Title 15
Forum Status Indicator 14
Forum Summary 14
Nº of Threads in Forum 13
Nº of Posts in Forum 13
Last Post in Forum 13
Forum Moderator 5
Nº of Views in Forum 1
Forum Type 1
Least Most
frequent frequent
309
Appendix H
Element 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Freq
Menu 15
Thread Table 15
New Topic Button 14
Nº of Thread Pages 14
Breadcrumbs 13
Thread Status Indic. 12
Forum Jump 12
Captions 11
Logo 11
Threads page
Posting Rules 10
Current Time & Date 7
Forum Search 5
Login Field 5
Popular Topics 1
Thread table Elements

Thread Name 15
Last Post 14
Thread Author 14
Nº of Replies 14
Nº of Views 13
Emoticons 5
Least Most
frequent frequent
310
Appendix H
Element 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Freq
Post Area 15
next/prev. thread btn 15
Menu 14
Breadcrumbs 13
reply to topic button 13
Logo 11
Jump to Forum 11
Posting Rules 8
Topic title 6
time and date 6
Adverts 6
New post button 5
Login field 5
Post area
Search topic field 4

Table of contents 1
Post Area Elements

Author Info 15
Post Content 15
Time & Date of Post 15
Post Title 13
Quote Button 12
Signature 10
Profile Button 7
Post Number 6
Private Msg Button 6
Edit Post 4
email Button 3
Poster website button 3
Least Most
frequent frequent
311
Appendix I
Templates used in experiments 1 and 2
to assess perceptions of conventionality
and ease of use
FIGURE 131 - TEMPLATE C11 OF THE FORUM INDEX PAGE, MOST FREQUENTLY
CHOSEN AS MORE LIKE OTHER EXISTING FORUMS.
312
Appendix I
FIGURE 132 - TEMPLATE C12 OF THE FORUM INDEX PAGE
313
Appendix I
314
Appendix I
315
Appendix I
FIGURE 135 - TEMPLATE N11 OF THE FORUM INDEX PAGE
316
Appendix I
317
Appendix I
318
Appendix I
FIGURE 138 - TEMPLATE N14 OF THE FORUM INDEX PAGE, CHOSEN

THE LEAST NUMBER OF TIMES AS MORE LIKE OTHER WEB FORUMS.
319
Appendix I
FIGURE 139 - TEMPLATE C21 OF THE THREADS PAGE, MOST FREQUENTLY

CHOSEN AS MORE LIKE OTHER WEB FORUMS.
320
Appendix I
FIGURE 140 - TEMPLATE C22 OF THE THREADS PAGE
321
Appendix I
322
Appendix I
323
Appendix I
FIGURE 143 - TEMPLATE N21 OF THE THREADS PAGE
324
Appendix I
325
Appendix I
FIGURE 145 - TEMPLATE N23 OF THE THREADS PAGE, CHOSEN

326
Appendix I
327
Appendix I
FIGURE 147 - TEMPLATE C31 OF THE POST PAGE, MOST FREQUENTLY

CHOSEN AS MORE LIKE OTHER WEB FORUMS.
328
Appendix I
FIGURE 148 - TEMPLATE C32 OF THE POST PAGE
329
Appendix I
330
Appendix I
331
Appendix I
FIGURE 151 - TEMPLATE N31 OF THE POST PAGE
332
Appendix I
333
Appendix I
334
Appendix I
FIGURE 154 - TEMPLATE N34 OF THE POST PAGE, CHOSEN

335
Appendix J
Results from experiment 3,
the icon identification test
Icon 1 Counts Icon 2 Counts

explains something 1 Admin 2
FAQ 3 Encrypt 1
get info 1 Key 2
Help 22 Lock 1
help page 1 Lock thread 1
Question 6 Locked 1
Total 34 Login 2
Options 1
Password 12
Password protected area 1
Password/login 1
Secure payment 1
Icon 3 Counts Secure website 1
? 2 Security 6
Address 1 Start 1
Edit 4 Total 34
Entry form 1
Handwriting 1
Insert info 1
New post 3
New thread 1
Post 2
Post new message 1
Reply 3
Response 1
Signature 1
Write 6
Write message 1
Write new 1
Write new post 2
Write post 1
Writing 1
Total 34
336
Appendix J

? 9 Two people 1
Advert 1 Chat 2
Application 1 Comments 1
Birthdays 1 Contact 1
Calendar 11 Discussion 1
Date 1 Discussion forum 1
Dates 1 Forum 2
Font 1 Friend list 1
Gallery 1 Instant messenger 2
Information 1 Man handle 1
News 1 Members 2
Notes 1 Messenger 4
Pictures 1 Msn 8
Spell check 1 Msn/user details 1
Start new thread 1 People 2
Text document 1 PM 1
Total 34 Talk 1
Users 2
Total 34
Icon 6 Counts
archives 1 Icon 7 Counts
Calendar 30 ? 3
Calendar month 1 contact card 2
date 1 contact info 1
gallery 1 email 3
Total 34 identification 2
identity 2
insert image 1
member info 1
Mini-profile 1
personal page 1
profile 10
profile card 1
user id 1
vcard 2
Video 1
video conference 1
your profile 1
Total 34
337
Appendix J

appointment 2 ? 5
calendar 14 Audio 1
date 2 Call centre 1
date & time 5 Chat 1
events 2 Headset 1
meetings 1 Help 1
Schedule 1 Instant messenger 2
settings 1 Live assistance 1
Time 2 Live support 1
time & calendar 1 Online 1
time & date 2 Online help 1
zoom in 1 Person 1
Total 34 Profile 1
Skype 1
Something audible 1
Speech 1
Spoke word computer 1
Support 5
Icon 10 Counts User 2
? 5 Voice 3
attachment 1 Voice call 1
Cut 1 Voip 1
download 2 Total 34
e-blog 1
edit 1
exit 1
go to 1
go to document 2 Icon 11 Counts
incoming data 1 ? 14
insert 1 archive 1
new post 2 closed topic 1
new thread 2 cotton reel 1
Next Page 3 delete 2
post 1 forum thread 1
print 1 Ink 1
save 3 movie 1
send 1 new thread 1
send file 2 no connection 1
submit 1 Save 1
submit file 1 sewing advice 1
Total 34 tailor 1
thrash 1
thread 5
write a post 1
Total 34
338
Appendix J

? 8 ? 5
add comment 1 enlarge 1
chat 2 find 1
Comments 4 glasses 2
discuss 1 larger text 1
discussion 3 read 1
important info 1 Search 14
message 1 sight impaired 1
new post 3 accessibility option
no connection 1 text larger 1
note in text 1 view 2
Quote 2 watch thread 1
quote 1 Zoom 4
read 1 Total 34
reply 1
talk 1
thread 1
transcript 1
Total 34
339
Appendix K
Results for Experiment 3 which
asked participants to identify the
interactive elements of the
non-conforming forum
Part1 Part2 Part3 Part4 Part5 Part6

Username and
Change text
Password to Associated Threads Selected
1 font and website time
access member programs within forum thread
colour
areas
forum entry
2 search links forum link forum status time
box
3 Don’t know Don’t know Don’t know Don’t know Don’t know Don’t know
more
criterion
4 entry information, main page Don’t know put on disc
main page
dates
logon with
navigating topics of a post from
5 password and who is online time and date
options discussion user
username
forum entry
6 login toolbar topic/thread links time and date
box
7 Don’t know Don’t know Don’t know Don’t know Don’t know Don’t know
8 login menu bar main forum topic online users time and date
thread with
9 login section links list of threads key date and stats
details
functions guide to new
10 login conversation one post time and date
e.g. search posts
11 login operators page # text go to close
list of thread with
12 sign in menu bar Don’t know date stamp
messages details
access/actio
13 name/password Don’t know Don’t know Don’t know Don’t know
ns
quick user particular
14 login forum posts who is online time and date
panel post
forum
15 login faq forum font colour time
comment
login and forum current date
16 user menu forum page site details
password message link and time
login and page of thread of what you are
17 icon bar Don’t know
password postings forum allowed to do
a single
user control list of threads
18 login area thread or Don’t know time
panel o posts
post
page holding
19 login area menu area thread page key stats
threads
20 login tools messages a message meta data clock
21 login user settings main content topics links time and date
340
Appendix K
Part1 Part2 Part3 Part4 Part5 Part6

user control current date
22 login boxes all threads single thread users online
panel and time
access to
23 login section forum thread legend date
user panel
24 login menu background thread key time
login username forum most recent
25 one thread links copyright
& password information threads
List of
Menu of
26 login forms threads in Thread info Who is online Time and date
some kind
date order
contact results of a one result of
27 login to site Don’t know save
details search a search
control
system time
28 login panel topics list topic footer text
info
settings
29 user functions login forum body topic key time
change
30 sign in Don’t know forum thread time and date
colour
single piece
31 login navigation content key/legend time
of content
32 login select action all posts post copyright info time
33 login members posts thread logout login
login and main menu
34 thread list thread listing Don’t know Don’t know
password options
341
Appendix L
Text introducing PDAs presented
for participants of experiment 4
A Brief introduction to PDAs

PDAs or Personal Digital Assistants, also called handhelds or palmtops, have
definitely evolved over the years. Not only can they manage your personal
information, such as contacts, appointments, and to-do lists, today's devices can also
connect to the Internet, act as global positioning system (GPS) devices, and run
multimedia software. What's more, manufacturers have combined PDAs with cell
phones, multimedia players and other electronic gadgetry.
There are many brands of PDAs each with diverse features and caracteristics.
The two main groups of PDAs are the ones that are based on the PALM OS operating
system and those that are based on the Windows operating system (called Pocket
PCs).
Within both groups you have the standard PDAs and other PDAs that are combined
with mobile phones, these are called smartphones. The greatest examples of this
category are the Palm Treo line and the Blackberry but Panasonic, Nokia, Samsung
and other big companies also have their own smartphones.
PALM POCKET PC OTHER
Standard PDA Smartphone Standard PDA Smartphone Standard PDA Standard PDA
Tungsten E2 Treo 680 Dell Axim HP iPAQ Apple Newton Sharp Zaurus
342
Appendix M
Tasks designed for the experiment 4 which tested
the performance of users familiar and unfamiliar
with web forums on conforming and non-
conforming websites.
Tasks
1. According to the forum member hawklet00, what is needed to use a GPS

(Global Positioning System) with the Blackberry PEARL?
2. How many threads with new posts does the ‘Palm General Discussion’ topic
have?
3. Which is the oldest post under the Miscellaneous PDAs topic?
4. Which video player is suggested by the member pipesmoker to be used to play

video podcast files and display them in a full screen of a Tungsten E (PalmOS)?
5. Which is considered to be the best Newton by the forum member

stercusaccidit, the 130, the 2000 or or the 2100?
343
Appendix N
Tasks designed for the pilot study which was
conducted in preparation for experiment 5
Tasks
A) How many birthdays does the forum have on 11th of October 2007?
B) What is the difference between a 1st edition and a 1st trade edition?
C) Which forum member wrote the last post on the thread “Trimming pages” under
the “Adding value to your books” topic?
D) Find the answer for this frequently asked forum question: “How do I prevent my
user name from appearing in the online user’s listing”?
E) Which thread in the ‘Books wanted’ topic has the last post dated Nov 22?
F) What is the price of a third edition of the ‘Compendious Geographical Dictionary’?
G) In the forum topic “Help on identification”, how many viewings has the thread
“Atlas of lost worlds” had?
H) How much is worth a T.S. Eliot signed 1st edition?
I) Which thread under the “What I’m reading now” has the greatest number of views?
H) Go to the topic in the forum which has the highest number of threads and find
who published the earliest copies of ‘Le Petit Prince’
344
Appendix O
Tasks used on both
parts of experiment 5
Part I
A) How much is a 1901 American standard Bible worth?
B) Identify which company now owns the publisher called Doubleday.
C) What is the procedure to add value to a book by removing its musty odors using
kitty litter?
D) What is the difference between a first edition and a 1st trade edition?
E) What is the name of the author of the Non-fiction book ‘Nickel and Dimed: On
(Not) Getting by in America’?
F) Go to the ‘Books Wanted’ topic and find out which forum member wrote the last
post on the thread “looking for 1st ed / 1st printing of Flowers for Algernon”
G) Go to the ‘Science Fiction & Fantasy’ topic and find the title of the only thread
with new posts.
H) How many pages does the ‘General Discussion’ topic have?
I) Go to the ‘Classic literature’ topic and find who wrote the book ‘Moon and
Sixpence’
J) How many birthdays does the forum have on the calendar date of 11th of October
2007?
Part II
A) Which brand of keyboard works with the PALM Zire 31?
B) Where can you find a SWF player for the Windows Mobile/Pocket PC?
C) What can you use instead of a stylus to edit Word or .PDF documents on a
Blackberry?
D) Go to the forum topic with the highest number of threads and find out which
Newton is the best: the 130, 2000 or the 2100.
E) Go to the forum topic whose last post was done on 11-18-2005 at 6:53 pm and find
the web address where you can get the game Space Quest for the Nokia SERIES 60.
F) What is the title of the only closed thread in the ‘General Smartphone Discussion’
topic?
345
Appendix P
Disorientation questionnaire applied on
both parts of experiment 5
For each question below, mark the box that most adequately represent how felt while
doing the tasks on the web forum you just used.
1) I felt lost / disorientated
Never Sometimes A few Most of the Always

times time
2) I felt like I was going around in circles

times time
3) It was difficult to find a page that I had previously viewed

times time
4) Learning to use the site was easy
Strongly Disagree
disagree Neither Agree Strongly
agree nor agree
disagree
5) The site was easy to navigate.
Strongly Disagree
agree nor agree
disagree
346
Appendix Q
Questionnaire used in Experiment 6
1) I felt lost / disorientated

times time
2) Learning to use the site was easy
Strongly Disagree
agree nor agree
disagree
3) The site was easy to navigate.
Strongly Disagree
agree nor agree
disagree
347
References
Ahuja, J. S., & Webster, J. (2001). Perceived disorientation: an examination of a new

measure to assess web design effectiveness. Interacting with Computers, 14, 15-29.
Åkesson, M. (2003). Design patterns for the online newspaper genre - A genre analysis of 85
Swedish daily online newspapers. Götenburg University, Götenburg.
Albers, M. J., & Kim, L. (2000). User web browsing characteristics using palm handhelds for
information retrieval. Paper presented at the Proceedings of IEEE professional
communication society international professional communication conference and
Proceedings of the 18th annual ACM international conference on Computer
documentation: technology \& teamwork.
Alexander, C. (1999, Sep/Oct). The origins of pattern theory: the future of the theory, and
thegeneration of a living world. Software, IEEE, 16, 71-82.
Alexander, C., Ishikawa, S., & Silverstein, M. (1977). A Pattern Language: Towns, buildings,
construction. New York, United States: Oxfor University Press.
Baker, R., Bernard, M., & Riley, S. (2002). Reading online news: A comparison of three
presentation formats. Usability News Retrieved 06 January, 2005, from
http://psychology.wichita.edu/surl/usabilitynews/42/depth.htm
Bateman, J., & Delin, J. (2001). From genre to text critiquing in multimodal documents.
Workshop presented at the 4th International Workshop on Multidisciplinary
Approaches to Discourse: Improving text: From text structure to text type. MAD 2001,
Yttre, Belgium.
Beghtol, C. (2001). The concept of genre and its characteristics. Bulletin of the American
Society for Information Science and Technology, 27(2).
Bernard, M. (2001a). Developing schemas for the location of common web objects.
Usability News, 3(1).
Bernard, M. (2001b). User expectations for the location of web objects. Paper presented at the
CHI '01 extended abstracts on Human factors in computing systems, Seattle,
Washington.
Bernard, M. (2002). Examining User Expectations for the Location of Common E-

Commerce Web Objects. Usability News, 4(1). Retrieved from
http://www.surl.org/usabilitynews/41/web_object-ecom.asp
Bernard, M., Chaparro, B., & Thomasson, R. (2000). Finding information on the web: Does
the amount of whitespace really matter? Usability News, 2(1).
Bevan, N. (1997). Usability issues in web site design. Advances in Human Factors
Ergonomics, 21(B), 803-806.
Bevan, N., Kirakowski, J., & Maissel, J. (1991, September 1-6). What is Usability? Paper
presented at the 4th International Conference on HCI, Stuttgart, Germany.
348
Bevan, N., & MacLeod, M. (1994). Usability measurement in context. Behaviour &
Information Technology, 13(1), 132 - 145.
Blackler, A. L., Popovic, V., & Mahar, D. P. (2004). Intuitive interaction with complex
artefacts. Paper presented at the Futureground - Design research society international
conference, Melbourne, Australia.
Blattner, M., Sumikawa, D. A., & Greenberg, R. M. (1989). Earcons and icons: Their
structure and common design principles. Human Computer Interaction, 4, 11 - 44.
Borchers, J. O. (1999). Electronic Books: Definition, Genres, Interaction Design Patterns. Paper
presented at the CHI'99 International Conference on Human Factors in Computing
Systems.
Borchers, J. O. (2001). A pattern approach to interaction design AI & Society, 15(4), 359-
376.
Borchers, J. O., Fincher, S., Griffiths, R., Pemberton, L., & Siemon, E. (2001). Usability
Pattern Language: Creating A Community. AI & Society, 15(4), 377-385.
Broder, A. (2002). A taxonomy of web search. SIGIR Forum, 36(2), 3-10.
Burleson, J. (2004). Conducting the Web Designer job interview. Kittrell, NC, United States:
Rampant Tech Press.
Byrne, M. D., John, B. E., Wehrle, N. S., & Crow, D. C. (1999). The tangled web we wove: A
taskonomy of WWW use. Paper presented at the SIGCHI conference on Human factors in
computing systems: the CHI is the limit, Pittsburgh, Pennsylvania, United States.
Cappel, J. J., & Huang, Z. (2006). An analysis of the use of web design conventions in company
websites. Paper presented at the Annual conference of the International Association for
Computer Information Systems - The changing role of IS Education, Reno, Nevada,
USA.
Capra, R. G. (2006). An investigation of finding and refinding information on the Web. Virginia
Polytechnic Institute and State University, Blacksburg.
Catledge, L. D., & Pitkow, J. E. (1995). Characterizing browsing strategies in the World-
Wide Web. Computer Networks and ISDN Systems, 27(6), 1065 - 1073.
Chaparro, B., Baker, J. R., Shaikh, A. D., Hull, S., & Brady, L. (2004). Reading online text:
A comparison of four white space layouts. Usability News Retrieved 17 December, 2005,
from http://psychology.wichita.edu/surl/usabilitynews/62/whitespace.htm
Chaparro, B., Shaikh, A. D., & Baker, J. R. (2005). Reading online text with a poor layout:
Is performance worse? Usability News Retrieved 27 November, 2005, from
http://psychology.wichita.edu/surl/usabilitynews/71/page_setting.html
Chawda, B., Craft, B., Cairns, P., Rüger, S., & Heesch, D. (2005). Do “Attractive Things
Work Better”? An Exploration of Search Tool Visualisations. Paper presented at the HCI
2005 The bigger picture - 19th British HCI Group Annual Conference, Edinburgh,
Scotland, United Kingdom.
Conklin, J. (1987). Hypertext: An introduction and survey. Computer, 20(9), 17-42.
Constantine, L. L., & Lockwood, L. A. D. (2002). Instructive interaction: Making

innovative interfaces self-teaching. User Experience Magazine, 1(3).
349
Cosio, M. G. d., & Dyson, M. C. (2002). Identifying graphic conventions for genre
definition in web sites. Digital Creativity, 13(3), 165-181.
Cove, J. F., & Walsh, B. C. (1988). Online text retrieval via browsing. Information Processing
and Management: an International Journal, 24(1), 31-37.
Crowston, K., & Williams, M. (2000). Reproduced and emergent genres of communication
on the World Wide Web. The Information Society, 16(3), 201-216.
Danielson, D. R. (2002a). Transitional volatility in web navigation. Stanford university,

Stanford.
Danielson, D. R. (2002b). Web navigation and the behavioural effects of constantly visible
site maps. Interacting with Computers, 14, 601-618.
Delin, J., Bateman, J., & Allen, P. (2002). A Model of genre in document layout. Information
Design Journal, 11(1), 54 - 66.
DeVigal, A. (2003). Putting the eyetrack study to good use Retrieved 10 December, 2005,
from http://www.poynter.org/content/content_view.asp?id=38357
Dewhurst, S. A., & Hitch, G. J. (1999). Cognitive effort and recollective experience in
recognition memory. Memory, 7(2), 129 - 146.
Dickson, S., Simmons, D. C., & Kameenui, E. J. (1995). Text organization and its relation to
reading comprehension: A synthesis of the research. (Tech. Rep. No. 17). Eugene, OR:
University of Oregon, National Center to Improve the Tools of Educators.
Dillon, A. (1991). Readers' models of text structures. International Journal of man-machine

studies, 35(6), 913-925.
Dillon, A. (2000). Spatial-semantics: How users derive shape from information space.
Journal of the American Society for Information Science, 51(6), 521-528.
Dillon, A. (2002). Writing as design: hypermedia and the shape of information space. In
R. Bromme & E. Stahl (Eds.), Writing Hypertext and Learning - Conceptual and Empirical
Approaches: Elsevier.
Dillon, A. (2004). Shape: information as a structured space. In A. Dillon (Ed.), Designing

Usable Electronic Text (2 ed., pp. 214). Boca Raton, Fla. London: Routledge.
Dillon, A., & Gushrowski, B. (2000). Genres and the web: Is the personal home page the
first uniquely digital genre? Journal of the American Society for Information Science, 51(2),
202-205.
Dillon, A., & Schaap, D. (1996). Expertise and the perception of shape in information.
Journal of the American Society for Information Science, 47(10), 786-788.
Dillon, A., & Vaughan, M. (1997). It's the journey and the destination: Shape and the
emergent property of genre in evaluating digital documents. New Review of Multimedia
and Hypermedia, 3, 91-106.
DomainTools.com (2008). Domain Counts & Internet Statistics Retrieved 27 November,

2008, from http://www.domaintools.com/internet-statistics/
Dontcheva, M., Drucker, S. M., Salesin, D., & Cohen, M. F. (2007). Changes in Webpage
structure over time. Washington, United States: University of Washington.
350
Dyson, M. C., & Gregory, J. (2002). Typographic cueing on screen. Visible Language, 36(3),
326-346.
Dyson, M. C., & Haselgrove, M. (2000). The effects of reading speed and reading patterns
on the understanding of text read from screen. Journal of Research in Reading, 23(2), 210-
223.
Dyson, M. C., & Haselgrove, M. (2001). The influence of reading speed and line length on
the effectiveness of reading from screen. Journal of Human-Computer Studies, 54, 585-612.
Eason, K. D. (1984). Towards the experimental study of usability. Behaviour & Information
Technology, 3(2), 133 - 143.
Ehret, B. D. (2002). Learning where to look: location learning in graphical user interfaces. Paper
presented at the SIGCHI conference on Human factors in computing systems: Changing
our world, changing ourselves, Minneapolis, Minnesota, USA.
Eriksen, L. B., & Ihlström, C. (2000). Evolution of the web news genre - the slow move beyond
the print metaphor. Paper presented at the 33rd International Conference on System
Sciences, Hawaii.
Eveland, W. P., Cortese, J., Park, H., & Dunwoody, S. (2004). How web site organization
influences free recall, factual knowledge, and knowledge structure density. Human
Communication Research, 30(2), 208-233.
Fayard, A.-L., & DeSanctis, G. (2006). Evolution of an Online Forum for Knowledge
Management Professionals: A Language Game Analysis. Journal of Computer-Mediated
Communication, 10(4).
Fayard, A.-L., DeSanctis, G., & Roach, M. (2004). The language games of online forums. Paper
presented at the Sixty-third Annual meeting of the Academy of Management, New
Orleans, United States.
Frye, N. (1957). Anatomy of Criticism: Four Essays (10th edition ed.). Princeton: Princeton
University Press.
Fu, L., & Salvendy, G. (2002). The contribution of apparent and inherent usability to a
user's satisfaction in a searching and browsing task on the Web. Ergonomics, 45(6), 415 -
424.
Galletta, D., Henry, R. M., McCoy, S., & Polak, P. (2006). When the wait isn't so bad: the
interacting effects of website delay, familiarity and breadth. Information Systems
Research, 17(1), 20 - 37.
Gefen, D. (2000). E-commerce: the role of familiarity and trust. Omega - The International
Journal of Management Science, 28(6), 725-737.
Gery, G. (1995). Attributes and behaviors of performance-centered systems. Performance

Improvement Quarterly, 8(1), 47-93.
Gordon, M., & Pathak, P. (1999). Finding information on the World Wide Web: the
retrieval effectiveness of search engines. Information Processing and Management: an
International Journal, 35(2), 141-180.
Gould, J. D., & Lewis, C. (1985). Designing for usability: key principles and what
designers think. Communications of the ACM, 28(3), 300-311.
351
Granlund, Å., Lafrenière, D., & Carr, D. A. (2001, August 5-10). A Pattern-Supported
Approach to the User Interface Design Process. Paper presented at the HCI International
2001 9th International Conference on Human-Computer Interaction, New Orleans,
Louisiana, United States.
Grier, R. A., Kortum, P., & Miller, J. T. (2007). How users view web pages: An exploration
of cognitive and perceptual mechanisms. In P. Zaphiris & S. Kurniawan (Eds.), Human
Computer Interaction Research in Web Design and Evaluation (1 ed., pp. 22-41). London: IGI
- Idea Group Inc.
Griffiths, R., Pemberton, L., Borchers, J., & Stork, A. (2000). Pattern languages for interaction
design: building momentum. Paper presented at the CHI '00 extended abstracts on Human
factors in computing systems.
Grudin, J. (1989). The case against user interface consistency. Commun. ACM, 32(10), 1164-
1173.
Gwizdka, J., & Spence, I. (2007). Implicit measures of lostness and success in web
navigation. Interacting with computers, 19, 357-369.
Halberstadt, J., & Rhodes, G. (2000). The Attractiveness of Nonface Averages:

Implications for an Evolutionary Explanation of the Attractiveness of Average Faces.
Psychological Science, 11(4), 285-289.
Hartley, J. (1978). Designing instructional text (3rd ed.). London: Kogan Page Limited.
Hartley, J. (1987). Typography and executive control processes in reading. In B. K. Britton

& S. M. Glynn (Eds.), Executive control processes in reading (pp. 57-79). New Jersey:
Lawrence Erlbaum Associates, Inc.
Hartley, J. (1994). Designing instructional text (3rd ed.). London, United Kingdom: Kogan
Page Limited.
Hartley, J. (2004). Designing instructional and informational text. In D. H. Jonassen (Ed.),

Handbook of research for educational communications and technology (pp. 917-947). New
Jersey: Lawrence Erlbaum Associates, Inc.
Hartley, J., & Jonassen, D. H. (1985). The role of headings in printed and electronic text. In
D. H. Jonassen (Ed.), The Technology of text : principles for structuring, designing, and
displaying text (Vol. 2, pp. 237-263). Englewood Cliffs, N.J.: Educational Technology
Publications.
Hartley, J., & Trueman, M. (1985). A research strategy for text designers: The role of
headings. Instructional Science, 14(2), 99-155.
Hasan, B., & Ahmed, M. U. (2007). Effects of interface style on user perceptions and
behavioral intention to use computer systems. Computers in Human Behavior, 23(6), 3025.
Henninger, S. (2000). A methodology and tools for applying context-specific usability

guidelines to interface design. Interacting with Computers, 12(3), 225.
Herder, E. (2003). Revisitation patterns and disorientation. Paper presented at the German
Workshop on Adaptivity and User Modeling in Interactive Systems: ABIS 2003,
Karlsruhe.
Herder, E., & Dijk, B. v. (2003). From Browsing Behavior to Usability Matters. Workshop on
Human Information Processing and Web Navigation, HCI 2003 Retrieved January, 2008,
352
from
http://www.tm.tno.nl/pals/documents/from_browsing_behavior_to_usability_matte
rs.pdf
Herring, S. C., Scheidt, L. A., Bonus, S., & Wright, E. (2004). Bridging the Gap: A Genre
Analysis of Weblogs. Paper presented at the Proceedings of the 37th Annual Hawaii
International Conference on System Sciences, 2004, Hawaii, USA.
Hirsch, S. G. (1997). How do children find information on different types of tasks ?

Children's use of the Science Library Catalog. Library trends, 45(4), 725-745.
Holscher, C., & Strube, G. (2000). Web search behavior of Internet experts and newbies.
Computer Networks, 33(1-6), 337.
Ihlström, C., & Åkesson, M. (2004). Genre characteristics - a front page analysis of 85 Swedish
online newspapers. Paper presented at the 37th. Hawaii international conference on
systems science, Hawaii.
ISO/IEC 9241-11 (1998). Ergonomic requirements for office work with visual display terminals
(VDTs) - Part 11:Guidance on Usability. Geneva, Switzerland: International Organization
for Standardization.
ISO/IEC (1998). ISO 9241-11:1998 Ergonomic requirements for office work with visual display
terminals (VDTs) - Part 11:Guidance on Usability. Geneva, Switzerland: International
Organization for Standardization.
Jenkins, C., Corritore, C. L., & Wiedenbeck, S. (2003). Patterns of information seeking on
the web: A qualitative study of domain expertise and web expertise. Information
Technology and Society, 1(3), 64-89.
Jonassen, D. H., Beissner, K., & Yacci, M. (1993). Structural knowledge: Techniques for
representing, conveying and acquiring structural knowledge (1st ed.). Hillsdale, NJ:
Lawrence Erlbaum Associates, Inc.
Kalbach, J., & Bosenick, T. (2006). Web page layout: A comparison between left- and
right-justified site navigation menus. Journal of Digital Information, 4(Vol. 4). Retrieved
from http://journals.tdl.org/jodi/rt/printerFriendly/jodi-111/93
Kelly, D., & Cool, C. (2002). The Effects of Topic Familiarity on Information Search Behaviour.
Paper presented at the 2nd ACM/IEEE-CS joint conference on Digital libraries,
Portland, Oregon.
Kirk, R. E. (1995). Experimental design: Procedures for the behavioral sciences (3rd. ed.). Pacific
Grove, California: Brooks/Cole.
Kurosu, M., & Kashimura, K. (1995). Apparent usability vs. inherent usability: experimental
analysis on the determinants of the apparent usability. Paper presented at the Conference
companion on Human factors in computing systems. from
file:///E:/_UK/PhD/Artigos/printed/KUROSU%20%26%20KASHIMURA%20-
%20Apparent%20usability%20vs.%20Inherent%20usability.pdf
Langlois, J. H., & Roggman, L. A. (1990). Attractive faces are only average. Psychological
Science, 1(2), 115-121.
Langlois, J. H., Roggman, L. A., & Musselman, L. (1994). What Is Average and What Is
Not Average About Attractive Faces? Psychological Science, 5(4), 214-220.
353
Larson, K., & Czerwinski, M. (1998). Web page design: implications of memory, structure and
scent for information retrieval. Paper presented at the SIGCHI conference on Human
factors in computing systems, Los Angeles, California, United States.
Lazonder, A. W., Biemans, H. J. A., & Wopereis, I. G. J. H. (2000). Differences between

novice and experienced users in searchin information on the World Wide Web. Journal
of the American Society for Information Science, 51(6), 576-581.
Lehtonen, T., Benamar, S., Laamanen, V., Luoma, I., Ruotsalainen, O., Salonen, J., et al.
(2006). Towards user-friendly mobile browsing. Paper presented at the Proceedings of the
2nd international workshop on Advanced architectures and algorithms for internet
delivery and applications.
Leichty, G., & Esrock, S. (2001). Change and response on the corporate web site. American
Communication Journal, (5 (1)). Retrieved from
http://www.acjournal.org/holdings/vol5/iss1/articles/leichtyesrock.htm
Lidwell, W., Holden, K., & Butler, J. (2003). Universal Principles of Design. Gloucester:
Rockport Publishers, Inc.
Lim, K., Benbasat, I., & Todd, P. A. (1996). An experimental Investigation of the
interactive effects of interface style, instructions and task familiarity on user
performance. ACM TRANSACTIONS ON COMPUTER HUMAN INTERACTION, 3(1),
1-37.
Logan, G. D. (1988). Toward an instance theory of automatization. Psychological review,

95(4), 492-527.
Loines, R. (1991). Giving readers what they want: What the research tells us about readable
documents (Vol. 2). Orlando, Florida.
Lorch, R. F., Jr., & Lorch, E. P. (1996). Effects of headings on text recall and
summarization. Contemporary Educational Psychology, 21, 261-278.
Love, H. (1985). Manuscript versus Print in the Transmission of English Literature, 1600-
1700. The Bibliographical Society of Australia and New Zealand Bulletin, 9(3), 95-106.
MacKay, B., Watters, C., & Duffy, J. (2004). Web Page Transformation When Switching
Devices Lecture Notes in Computer Science (Vol. 3160/2004, pp. 228-239). Berlin: Springer.
Maguire, M. (2001). Context of Use within usability activities. International Journal of

Human-Computer Studies, 55(4), 453.
Marchionini, G. (1995). Information seeking in electronic environments (3rd edition ed.).

Cambridge: Cambridge University Press.
Marchionini, G., & Shneiderman, B. (1988). Finding facts vs. browsing knowledge in
hypertext systems. Computer, 21(1), 70-80.
Markum, J., & Hall, R. H. (2003). E-commerce web objects: Importance and expected
placement. Laboratory for Information Technology Evaluation Technical Report Retrieved 23
January, 2008, from http://campus.umr.edu/lite/tech_reports/LITE-2003-02.pdf
McCarthy, J. D., Sasse, A. M., & Riegelsberger, J. (2003, 8th - 12th September). Could I have
the Menu Please? An Eye Tracking Study of Design Conventions. Paper presented at the
HCI 2003 - Designing for Society, Bath, United Kingdom.
354
McDonald, S., & Stevenson, R. J. (1998). Navigation in hyperspace: An evaluation of the
effects of navigational tools and subject matter expertise on browsing and information
retrieval in hypertext. Interacting with Computers, 10(2), 129.
McDougall, S. J. P., & Curry, M. B. (2004, 16 - 17 September). More than just a picture: Icon
representation in context. Paper presented at the First International Workshop on Coping
with Complexity, Bath, United Kingdom.
McDougall, S. J. P., Curry, M. B., & De Brujin, O. (1999). Measuring Symbol and icon
characteristics: norms for concretness, complexity, meaningfulness, familiarity and
semantic distance for 239 symbols. Behavior research methods, instruments & computers,
31(3), 487-519.
McEneaney, J. (2001). Graphic and numerical methods to assess navigation in hypertext.

International Journal of Human-Computer Studies, 55, 761-786.
McGann, R. (2005). Online banking increased 47 percent since 2002 Retrieved 30

november, 2005, from
http://www.clickz.com/stats/sectors/finance/article.php/3481976#table
Michailidou, E., Harper, S., & Bechhofer, S. (2008, September 22-24). Visual Complexity and
Aesthetic Perception of Web Pages. Paper presented at the SIGDOC'08 - 26th ACM
International Conference on Design of Communication, Lisbon, Portugal.
Milic-Frayling, N., Jones, R., Rodden, K., Smyth, G., Blackwell, A., & Sommerer, R. (2004).
Smartback: supporting users in back navigation. Paper presented at the 13th international
conference on World Wide Web, New York, NY, USA.
Miller, C. R. (1984). Genre as a social action. Quarterly journal of speech, 70(2), 151-167.
Miniwatts Marketing Group (n.d.). Internet usage statistics - the Internet big picture
Retrieved 27 November, 2008, from http://www.internetworldstats.com/stats.htm
Monin, B. (2003). The warm glow heuristic: When liking leads to familiarity. Journal of
Personality and Social Psychology, 85(6), 1035-1045.
Morris, M. G., & Dillon, A. (1997). How user perceptions influence software use. Software,
IEEE, 14(4), 58.
Moyes, J. (1994). When users do and don't rely on icon shape. Paper presented at the
Conference on Human Factors in Computing Systems' 94, Boston, Massachusetts,
United States.
Moyes, J. (1995). Putting icons in context: the influence of contextual information on the
usability of icons. University of Glasgow, Glasgow.
Moyes, J., & Jordan, P. W. (1993). Icon design and its effect on guessability, learnability, and
experienced user performance. Paper presented at the People and Computers VIII: HCI 93,
Loughborough, United Kingdom.
Muter, P., & Kruk, R. S. (1984). Reading of continuous text on video screen. Human
Factors, 26(3), 339-345.
Muter, P., Kruk, R. S., Buttigieg, M. A., & Kang, T. J. (1988). Reader-controlled
computerized presentation of text. Human Factors, 30(4), 473-486.
355
Nielsen, J. (1990). The art of navigating through hypertext. Communications of the ACM,
33(3), 296-310.
Nielsen, J. (1993). Usability engineering. Boston ; London: Academic Press.
Nielsen, J. (1994). Ten Usability Heuristics. The Alertbox:Current issues in web usability
Retrieved 14 October, 2008, from
http://www.useit.com/papers/heuristic/heuristic_list.html
Nielsen, J. (1999a). Do interface standards stifle design creativity? The Alertbox: Current
issues in Web usability Retrieved 30 June, 2007, from
http://useit.com/alertbox/990822.html
Nielsen, J. (1999b, 2004). When bad design elements become the standard. The
Alertbox:Current issues in web usability Retrieved January, 2008, from
http://www.useit.com/alertbox/991114.html
Norman, D. (2005, July + August). Human-centered design considered harmful.

Interactions, 12, 14-19.
OPA (2005, October 2005). Internet activity index Retrieved 30 November, 2005, from
http://www.online-publishers.org/?pg=activity
Østerlund, C. S. (2006). Combining genres: How practice matters. Paper presented at the 39th
Hawaii International Conference on System Sciences, Hawaii.
Otter, M., & Johnson, H. (2000). Lost in hyperspace: metrics and mental models.
Interacting with computers, 13, 1-40.
Oulasvirta, A., Kärkkäinen, L., & Laarni, J. (2005). Expectations and memory in link
search. Computer in Human Behaviour, 21, 773-789.
Ozok, A. A., & Salvendy, G. (2004). Twenty guidelines for the design of Web-based
interfaces with consistent language. Computers in Human Behavior, 20(2), 149.
Patel, M. R. (2003). Examining web design conventions across site types. Usability News,
(5.1). Retrieved from
http://psychology.wichita.edu/surl/usabilitynews/51/guest.htm
Patel, S. C., Drury, C. G., & Shalin, V. L. (1998). Effectiveness of expert semantic
knowledge as a navigational aid within hypertext. Behaviour and Information Technology,
17(6), 313-324.
Pearson, R., & van Schaik, P. (2003). The effect of spatial layout of and link colour in web
pages on performance in a visual search task and an interactive search task. International
Journal of Human-Computer Studies, 59(3), 327.
Peels, A. J. H., Janssen, N. J. M., & Nawijn, W. (1985). Document architecture and text
formatting. ACM Trans. Inf. Syst., 3(4), 347-369.
Pemberton, L. (2000). Genre as a structuring concept for interaction design pattern

languages. HCI-SIG Workshop on patterns Retrieved 30 October, 2008, from
http://www.cmis.brighton.ac.uk/staff/lp22/genredraft.html
Pollard, M. (2008). E-commerce and ICT Activity - 2007 e-commerce Survey of Business. from
http://www.statistics.gov.uk/pdfdir/ecom1108.pdf.
356
Rosenberg, J. (1996). The structure of hypertext activity. Paper presented at the Seventh
ACM conference on hypertext, Bethseda, Maryland, United States.
Rumpradit, C., & Donnell, M. L. (1999). Navigational Cues on User Interface Design to
Produce Better Information Seeking on the World Wide Web. Paper presented at the Thirty-
second Annual Hawaii International Conference on System Sciences, Hawaii, USA.
Salomon, G. (1989). Designing casual-user hypertext: the CHI'89 InfoBooth. Paper presented
at the Proceedings of the SIGCHI conference on Human factors in computing systems:
Empowering people, Seattle, Washington, United States.
Santini, M. (2005). Genres In Formation? An Exploratory Study of Web Pages using Cluster
Analysis. Paper presented at the CLUK - 8th Annual Computational Linguistics
Research Colloquium, Manchester.
Santini, M. (2007). Characterizing Genres of Web Pages: Genre Hybridism and Individualization.
Paper presented at the HICSS 2007 - 40th Annual Hawaii International Conference on
System Sciences.
Satizinger, J. W. (1991). User interface consistency across end-user application programs: effect
on learning and satisfaction. Claremont Graduate School.
Satzinger, J. W. (1991). User interface consistency across end-user application programs: effect
on learning and satisfaction. Claremont Graduate School.
Satzinger, J. W., & Olfman, L. (1998). User interface consistency across end-user
applications: the effects on mental models. J. Manage. Inf. Syst., 14(4), 167-193.
Scheidt, L., & Wright, E. (2004). Common visual design elements of weblogs. Into the
Blogosphere: Rethoric, community and culture of weblogs Retrieved June, 2007, from
http://blog.lib.umn.edu/blogosphere/common_visual.html
Schmid-Isler, S. (2000). The language of digital genres - a semiotic investigation of style and
iconology on the World Wide Web. Paper presented at the 33rd Hawaii International
Conference on System Sciences, Maui, Hawaii.
Schoon, P., & Cafolla, R. (2002). World Wide Web hypertext linkage patterns. Journal of
Educational Multimedia and Hypermedia, 11(2), 117-139.
Schultz, L. D., & Spyridakis, J. D. (2004). The effects of heading frequency on

comprehension of online information: A study of two populations. Technical
communication, 51(4), 504-516.
Scott, B. M., & Schwartz, N. H. (2007). Navigational spatial displays: The role of
metacognition as cognitive load. Learning and instruction, 17, 89-105.
Seffah, A., & Metzker, E. (2004). The obstacles and myths of usability and software
engineering. Commun. ACM, 47(12), 71-76.
Selvidge, P. R., Chaparro, B. S., & Bender, G. T. (2002). The world wide wait: effects of
delays on user performance. International Journal of Industrial Ergonomics, 29, 15-20.
Shackel, B. (1981). The concept of usability. Paper presented at the IBM Software and
Information Usability Symposium, Poughkeepsie, New York, USA.
357
Shepherd, M., & Watters, C. (1999). The Functionality Attribute of Cybergenres. Paper
presented at the Proceedings of the Thirty-Second Annual Hawaii International
Conference on System Sciences-Volume 2 - Volume 2.
Shneiderman, B., & Plaisant, C. (2005). Designing the user interface: Strategies for effective
Human-Computer Interaction (4th. ed.): Addison-Wesley, Ltd.
Smith, P. A. (1996). Towards a practical measure of hypertext usability. Interacting with

Computers, 8(4), 365-381.
Stamboltzis, A., & Pumfrey, P. (2000). Reading across Genres: a Review of Literature.
Support for Learning, 15(2), 58-61.
Stanoevska-Slabeva, K., Hombrecher, A., Handschuh, S., & Schmid, B. (1998, 29-30
October). Efficient information retrieval: Tools for knowledge management. Paper presented
at the Proceedings of the second european conference on practical aspects of knowledge
management, Basel, Switzerland.
Swales, J. M. (1990). Genre analysis: English in academic and research settings. Cambridge:
Cambridge University Press.
Tabatabai, D., & Shore, B. M. (2005). How experts and novices search the web. Library &
Information Science Research, 27, 222-248.
Tan, B., Foo, S., & Hui, S. C. (2001). Web information monitoring: an analysis of Web page
updates. Online Information Review, 25(1), 6-19.
Tauscher, L., & Greenberg, S. (1996). Design Guidelines for Effective WWW History
Mechanisms. Paper presented at the Microsoft Workshop, Designing for the Web:
Empirical Studies.
Tauscher, L., & Greenberg, S. (1997). Revisitation patterns in World Wide Web navigation.
Paper presented at the Proceedings of the SIGCHI conference on Human factors in
computing systems, Atlanta, United States.
Tidwell, J. (1999). Common Ground: A pattern language for Human-Computer Interface

design Retrieved 18 November, 2008, from
http://www.mit.edu/~jtidwell/common_ground_onefile.html
Toms, E. G. (2001). Recognizing Digital Genres. Bulletin - American Society for Information
Science.
Tractinsky, N. (1997). Aesthetics and apparent usability: empirically assessing cultural and
methodological issues. Paper presented at the Proceedings of the SIGCHI conference on
Human factors in computing systems.
Tractinsky, N., Katz, A. S., & Ikar, D. (2000). What is beautiful is usable. Interacting with
Computers, 13(2), 127-145.
Twyman, M. (1979). A schema for the study of graphic language. In P. A. Kolers, M. E.

Wrolstad & H. Bouma (Eds.), Processing of Visible Language (Vol. 1). New York: Plenum
Press.
Twyman, M. (1982). The graphic presentation of language. Information design journal, 3(1),
2-22.
358
UPA (2005). Principles for usable design. Usability Body of Knowledge Retrieved 21
October, 2008, from http://www.usabilitybok.org/design/p287
Usborne, N. (2003). Web users crave familiarity. Excess Voice Retrieved 30 June, 2008,
from http://www.excessvoice.com/usborne-articles7.htm
van Dijk, T. A., & Kintsch, W. (1983). Strategies of discourse comprehension. New York,
United States: Academic Press, Inc.
van Schaik, P., & Ling, J. (2001). Design Parameters in Web Pages: Frame Location and
Differential Background Contrast in Visual Search Performance. International Journal of
Cognitive Ergonomics, 5(4), 459 - 471.
van Schaik, P., & Ling, J. (2009). The role of context in perceptions of the aesthetics of web
pages over time. International Journal of Human-Computer Studies, 67, 79-89.
van Welie, M., van der Veer, G., & Eliëns, A. (1999). Breaking down usability. Paper
presented at the INTERACT, 1999, Edinburgh.
Vaughan, M., & Dillon, A. (1998). The Role of Genre in Shaping our Understanding of Digital
Documents. Paper presented at the Annual Meeting of the American Society for
Information Science, Pittsburgh, PA, United States.
Vaughan, M., & Dillon, A. (2006). Why structure and genre matter for users of digital
information: A longitudinal experiment with readers of a web-based newspaper.
International Journal of Human-Computer studies, 64, 502-526.
Wallace, P. M. (1999). The Psychology of the Internet (1 ed.). New York: Cambridge
University Press.
Waller, R. (1979). Typographic access structures for educational texts. In P. A. Kolers, M.

E. Wrolstad & H. Bouma (Eds.), Processing of visible language (Vol. 1, pp. 175 - 187). New
York: Plenum Press.
Waller, R. (1987a). The typographic contribution to language: towards a model of typographic

genres and their underlying structures. University of Reading, Reading.
Waller, R. (1987b). Typography and reading strategy. In B. K. Britton & S. M. Glynn

(Eds.), Executive Control Processes in Reading (pp. 81-106). Hillsdale, New Jersey: L.
Erlbaum Association.
Waller, R. (1999). Making connections: typography, layout and language. Paper presented at
the 1999 Autumn Symposium, American Association for Artificial Intelligence,
Orlando, Florida.
Walton, M., & Vukovic, V. (2003, March and April of 2003). Cultures, literacy and the
web: Dimensions of information scent. Interactions, 10, 64-71.
Whiteside, J., Jones, S., Levy, P. S., & Wixon, D. (1985). User performance with command,
menu, and iconic interfaces. Paper presented at the Conference on Human Factors in
Computing Systems, San Francisco, California, USA.
Whittlesea, B. W. A. (1993). Illusions of familiarity. Journal of experimental psychology.

Learning, memory, and cognition, 19(6), 1235-1253.
Wildemuth, B. M. (2004). The effects of domain knowledge on search tactic formulation.

Journal of the American Society for Information Science and Technology, 55(3).
359
Williams, T. R., & Spyridakis, J. H. (1992). Visual discriminability of headings in text.
Transactions on Professional Communication, 35(2), 64-70.
Xu, G., Cockburn, A., & McKenzie, B. (2001). Lost on the web: An introduction to web
navigation research. Paper presented at the 4th New Zealand Computer Science Research
Students Conference, Christchurch, New Zealand.
Yates, J., & Orlikowski, W. J. (1992). Genres of Organizational Communication: A

Structurational Approach to Studying Communication and Media. Academy of
Management Review, 17(2), 299.
Yates, J., & Orlikowski, W. J. (2002). Genre Systems: Structuring Interaction through
Communicative Norms. The Journal of business communication, 39(1), 13.
Yates, J., Orlikowski, W. J., & Rennecker, J. (1997). Collaborative Genres for Collaboration:
Genre Systems in Digital Media. Paper presented at the Proceedings of the Thirtieth
Hawaii International Conference on System Sciences HICSS-97.
Yates, S., & Sumner, T. (1997). Digital Genres and the New Burden of Fixity. Paper presented
at the Proceedings of the 30th Hawaii International Conference on System Sciences:
Digital Documents - Volume 6.
Yatim, N. F. M. (2002). A combination measurement for studying disorientation. Paper

presented at the 35th Hawaii International Conference on System Sciences, Hawaii.
Yonelinas, A. P. (2002). The nature of recollection and familiarity: A review of 30 years of

research. Journal of memory and language, 46(3), 441-517.
Zhang, J., & Ghorbani, A. A. (2004, 13-15 October 2004). Familiarity and Trust: Measuring
Familiarity with a Web Site. Paper presented at the Proceedings of the 2nd Annual
Conference on Privacy, Trust and Security (PST'04), New Brunswick, Canada.
360

SANTA-MARIA-PhD - Thesis 2009 - Visual Genre Conventions and User Performance On The Web

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

SANTA-MARIA-PhD - Thesis 2009 - Visual Genre Conventions and User Performance On The Web

Uploaded by

Copyright:

Available Formats

!

Luis Santa Maria

Finally, an empirical study was conducted on conforming and non-conforming

friendship and care.

List of figures vi!

List of tables xiii!

Chapter 2 Genres, conventions and usability 8!

Chapter 3 Exploratory studies 40!

Chapter 4 Finding visual conventions in web forums 81!

Chapter 7 Visual conventions and performance over time 181!

Chapter 8 The effects of conventionality on users searching for information in an

Chapter 9 Conclusion 277!

Appendix B Summary task and multiple-choice questions and internet usage

Appendix C Elements which helped identifying the website in the second

Appendix F Tables with the symbolization mode frequencies of each element

Appendix G Tables with the configuration mode frequencies of each element

Appendix J Results from experiment 3, the icon identification test 336!

Appendix K Results for Experiment 3 which asked participants to identify the

Appendix L Text introducing PDAs presented for participants of experiment 4

Appendix O Tasks used on both parts of experiment 5 345!

Appendix P Disorientation questionnaire applied on both parts of experiment 5

Appendix Q Questionnaire used in Experiment 6 347!

Figure 14 - Mean multiple-choice scores for both layouts. 58!

Figure 15 – Mean multiple-choice structure question scores. 58!

Figure 28 – Some elements on the first level of a web forum. 93!

Figure 32 - Elements in the second level of a web forum. 96!

Figure 34 – Elements in a post page of a web forum. 98!

Figure 35 – Elements within the post area of a web forum. 99!

Figure 48 – Coding system used to name the forum templates. 116!

Figure 57 - Sample of section 1 of the survey. 141!

Figure 58 - Sample of the non-conforming screen in section 2 of the survey. 143!

Figure 60 - Sample of a hypothetical learning curve to explain the small difference in

Figure 65 –Interaction between conventionality and task type. 174!

Figure 68 – Hypothetical illustration of a learning effect to explain the lack of difference in

Figure 69 - Sample of the convention-conforming website 188!

Figure 71 - Sample of the Convention-conforming news homepage 189!

Figure 72 - Sample of the non-conforming online news homepage 190!

Figure 74 - Task completion time for the non-conforming group. 198!

Figure 75 - Task completion time for the conforming group. 199!

Figure 79 – Overall mean time to complete each task. 201!

Figure 90 - Right answers per second ratio formula. 220!

Figure 99 - Overlapped graphs of time to complete tasks on both groups. 228!

FIGURE 105 - PAGES OF THE CONFORMING ONLINE NEWS LAYOUT 244!

FIGURE 106 PAGES OF THE NON-CONFORMING ONLINE NEWS LAYOUT 244!

Figure 110- Common configuration of the components of a piece of news 249!

Figure 116 - Style and placement of the navigational menu in 253!

Figure 120 - Sample pages of the conforming news website 258!

Figure 123 - Mean time spent on tasks per group 264!

Figure 125 - Mean right answers per group 265!

Figure 132 - Template C12 of the forum index page 313!

Figure 133 - Template C13 of the forum index page 314!

Figure 134 - Template C14 of the forum index page 315!

Figure 135 - Template N11 of the forum index page 316!

Figure 136 - Template N12 of the forum index page 317!

Figure 137 - Template N13 of the forum index page 318!

Figure 140 - Template C22 of the threads page 321!

Figure 141 - Template C23 of the threads page 322!

Figure 142 - Template C24 of the threads page 323!

Figure 143 - Template N21 of the threads page 324!

Figure 144 - Template N22 of the threads page 325!

Figure 146 - Template N24 of the threads page 327!

Figure 148 - Template C32 of the post page 329!

Figure 150 - Template N31 of the post page 332!