You are on page 1of 33

Interactive Example Set 4

Learning Principle governing questions


Each slide has questions that allow you to assess your own understanding of the SPSS output and the principles of the course The idea is to attempt them yourself to assess your understanding and identify areas of the course where you may need to revise The answers are there if you are still stuck

Answers
The answers can be found in the notes section of the PowerPoint slide
If you are viewing this on PDF and do not have PowerPoint, you can obtain a PowerPoint viewer by typing PowerPoint viewer into Google or going to: http://www.microsoft.com/downloads/details.aspx?FamilyId=428D5727-43AB-4F24-90B7-A94784AF71A4&displaylang=en

If viewing in PDF, it is best to cover the answers with a sheet of paper and only check if you can not do them yourself
Coverage

These questions do not cover everything you need to know


Of course, Id love all the answers to be perfect first time, but if you spot what looks like an error, send me an email: jkanglim@unimelb.edu.au

Corrections?

Interactive Example Additional Practice Material: Skill Acquisition Data Set


Motivation for research
Understand relationship between abilities, prior skills and other individual differences with learning a relatively simple psychomotor task

Sample
Around 116 participants (but varies because not all participants did all tests) mix of 1st year psychology students and friends and family of researchers

Measures & Design


Demographic Information (age, gender) Background Questionnaire (need for breaks; keyboard experience, etc.) Self-report questionnaire on knowledge of text-editing Initial motivation Questionnaire Multiple choice test of knowledge of text editing Typing test Battery of nine ability tests
General cognitive; perceptual speed and reaction time

Perceptions of ability tests and task motivation questionnaire 54 trials (on a text editing task (time to complete each trial; strategies used) Retrospective perception of experiment Questionnaire
For further info, see: Anglim, J., Langan-Fox, J., & Mahdavi, N. (2005). Modeling the Relationship between Strategies, Abilities and Skilled Performance. CogSci 2005, 27th Annual Meeting of the Cognitive Science Society, July 21-23 Stresa, Italy web ref: www.psych.unito.it/csc/cogsci05/frame/poster/3/p465-anglim.pdf

Example 4.1
Research Question
A series of 11 questions were answered at the end of the experiment after the completion of the text editing task. Could they be grouped into a smaller set of meaningful constructs?

1.

Which statistical procedure could we use?

4.1 Descriptive Statistics


Descriptiv e Statistics Q1) During the task I often set a specific goal for myself Q2) I always focused my attention on the text-editing task Q3) I often worried about not being able to reach my best possible score Q4) I often forgot about trying to reach my best possible score Q5) I wondered about how my performance compared with others Q6) I constantly focused on trying to do my best Q7) I often daydreamed while doing the task Q8) I often let my mind wander while doing the task Q9) How difficult was this task? (High=Extremely Difficult) Q10) How much of your attention was required to perform this task? (High = A great deal of attention) Q11) How pressured did you feel while performing this task? (High=Extremely Pressured) Mean 4.5526 4.6316 4.4649 3.1491 4.7105 5.2982 2.8772 2.9825 3.4211 5.1930 4.1930 Std. Deviation 1.83425 1.69989 1.70984 1.57543 1.84210 1.51061 1.62470 1.64550 1.49272 1.28901 1.63413 Analysis N 114 114 114 114 114 114 114 114 114 114 114

Here we see all the 11 items. They were all asked on a 1 to 7 scale. For items 1 to 8, 7 represented strongly agree; 1 strongly disagree. For items 9 to 11, the high score is shown

1.
2.

Looking at items 1 to 8, which two variables have the lowest means and what does this suggests? Looking at items 1 to 8, which two variables have the highest means and what does this suggest?

I Extracted 3 components

4.1 Communalities
Communalities Initial 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 Extraction .730 .594 .634 .548 .453 .732 .834 .852 .649 .620 .571

Q1) During the task I often set a specific goal for myself Q2) I always focused my attention on the text-editing task Q3) I often worried about not being able to reach my best possible score Q4) I often forgot about trying to reach my best possible score Q5) I wondered about how my performance compared with others Q6) I constantly focused on trying to do my best Q7) I often daydreamed while doing the task Q8) I often let my mind wander while doing the task Q9) How difficult was this task? (High=Extremely Difficult) Q10) How much of your attention was required to perform this task? (High = A great deal of attention) Q11) How pressured did you feel while performing this task? (High=Extremely Pressured) Extraction Method: Principal Component Analysis.

1. 2. 3. 4.

What is the interpretation of the numbers in the extracted column? Which variable/s have less than half their variance explained by the extracted components? How much variance in Q1 was explained by the three components? Looking at the content of the items, which items might you expect to group together?

How many factors to extract?


Total Variance Explained Initial Eigenvalues % of Variance Cumulative % 34.387 34.387 16.474 50.862 14.768 65.630 7.253 72.883 5.621 78.504 5.395 83.898 4.882 88.780 4.099 92.879 3.052 95.931 2.835 98.766 1.234 100.000 Extraction Sums of Squared Loadings Total % of Variance Cumulative % 3.783 34.387 34.387 1.812 16.474 50.862 1.625 14.768 65.630 Rotation Sums of Total Squared 3.184a Loadings 3.029 1.834 Component 1 2 3 4 5 6 7 8 9 10 11 Total 3.783 1.812 1.625 .798 .618 .593 .537 .451 .336 .312 .136

Extraction Method: Principal Component Analysis. a. When components are correlated, sums of squared loadings cannot be added to obtain a total variance.

Theory? There was no strong theory 1. What does the Eigenvalues over 1 rule suggest? 2. What does the Scree plot suggest? 3. Whats a reasoned Decision about how many components to extract?

4.1 Variance Explained


Total Variance Explained Initial Eigenvalues % of Variance Cumulative % 34.387 34.387 16.474 50.862 14.768 65.630 7.253 72.883 5.621 78.504 5.395 83.898 4.882 88.780 4.099 92.879 3.052 95.931 2.835 98.766 1.234 100.000 Extraction Sums of Squared Loadings Total % of Variance Cumulative % 3.783 34.387 34.387 1.812 16.474 50.862 1.625 14.768 65.630 Component 1 2 3 4 5 6 7 8 9 10 11 Total 3.783 1.812 1.625 .798 .618 .593 .537 .451 .336 .312 .136

Extraction Method: Principal Component Analysis.

1. 2. 3. 4. 5. 6.

How much variance was explained by the 3 components? What is the sum of all 11 eigenvalues? What is the sum of the three extracted eigenvalues? What is 65.63% of 11? What is 34.387% of 11? What is 65.63% minus 14.768%?

4.1 Unrotated Solution


Component Matrixa Q1) During the task I often set a specific goal for myself Q2) I always focused my attention on the text-editing task Q3) I often worried about not being able to reach my best possible score Q4) I often forgot about trying to reach my best possible score Q5) I wondered about how my performance compared with others Q6) I constantly focused on trying to do my best Q7) I often daydreamed while doing the task Q8) I often let my mind wander while doing the task Q9) How difficult was this task? (High=Extremely Difficult) Q10) How much of your attention was required to perform this task? (High = A great deal of attention) Q11) How pressured did you feel while performing this task? (High=Extremely Pressured) 1 .637 .735 .625 -.728 .312 .826 -.710 -.620 -.118 .369 .313 Component 2 -.182 -.153 .288 -.125 .340 -.223 .253 .193 .769 .573 .686 3 .540 .173 .400 -.060 .490 -.009 .515 .656 -.208 -.395 -.054

1. 2. 3. 4. 5. 6.

Extraction Method: Principal Component Analysis. Which variables load a. 3 components extracted. highly on component

1? Which of them are positive and which are

negative? How would you describe someone who was high on component 1? Which variables load highly on component 2? Which of them are positive and which are negative? How would you describe someone who was high on component 2? Which variables load highly on component 3? Which of them are positive and which are negative? How would you describe someone who was high on component 3?

Total Variance Explained

4.1 Rotations & Variance Explained

Component 1 2 3 4 5 6 7 8 9 10 11

Total 3.783 1.812 1.625 .798 .618 .593 .537 .451 .336 .312 .136

Initial Eigenvalues % of Variance Cumulative % 34.387 34.387 16.474 50.862 14.768 65.630 7.253 72.883 5.621 78.504 5.395 83.898 4.882 88.780 4.099 92.879 3.052 95.931 2.835 98.766 1.234 100.000

Extraction Sums of Squared Loadings Total % of Variance Cumulative % 3.783 34.387 34.387 1.812 16.474 50.862 1.625 14.768 65.630

Rotation Sums of Total Squared 3.184a Loadings 3.029 1.834

Extraction Method: Principal Component Analysis. a. When components are correlated, sums of squared loadings cannot be added to obtain a total variance. Total Variance Explained

Component 1 2 3 4 5 6 7 8 9 10 11

Total 3.783 1.812 1.625 .798 .618 .593 .537 .451 .336 .312 .136

Initial Eigenvalues % of Variance Cumulative % 34.387 34.387 16.474 50.862 14.768 65.630 7.253 72.883 5.621 78.504 5.395 83.898 4.882 88.780 4.099 92.879 3.052 95.931 2.835 98.766 1.234 100.000

Extraction Sums of Squared Loadings Total % of Variance Cumulative % 3.783 34.387 34.387 1.812 16.474 50.862 1.625 14.768 65.630

Rotation Sums of Squared Loadings Total % of Variance Cumulative % 2.733 24.847 24.847 2.652 24.110 48.956 1.834 16.674 65.630

Extraction Method: Principal Component Analysis.

1. 2. 3. 4. 5.

Which table shows the oblique rotation and which shows the orthogonal rotation? What is the sum of eigenvalues for three components prior to rotation? What is the sum of eigenvalues after rotation for the top table? What is the sum of eigenvalues after rotation for the bottom table? Why are the two sums different?

4.1 Orthogonal (Varimax) Rotation


a Rotated Component Matrix

Q1) During the task I often set a specific goal for myself Q2) I always focused my attention on the text-editing task Q3) I often worried about not being able to reach my best possible score Q4) I often forgot about trying to reach my best possible score Q5) I wondered about how my performance compared with others Q6) I constantly focused on trying to do my best Q7) I often daydreamed while doing the task Q8) I often let my mind wander while doing the task Q9) How difficult was this task? (High=Extremely Difficult) Q10) How much of your attention was required to perform this task? (High = A great deal of attention) Q11) How pressured did you feel while performing this task? (High=Extremely Pressured) Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization. a. Rotation converged in 6 iterations.

1 .807 .628 .756 -.577 .600 .560 -.125 .030 -.140 .057 .265

Component 2 -.137 -.436 -.073 .419 .217 -.641 .905 .917 .185 -.333 -.034

3 -.245 -.093 .238 -.202 .216 -.091 -.018 -.104 .772 .712 .707

1. 2. 3. 4. 5.

6.

What is the meaning of each of the numbers in this table? What is the meaning, if you square the number in the table? What is .8072 + .1372 + .2452? Going back to the earlier slide what was the extracted communality for Q1? If we dont look at the communalities table, which three numbers are required from the above rotated component matrix to determine the communality for Q2? Without looking at the communalities table, what is the communality for Q2?

Oblique (Promax) Rotation


Pattern Matrixa Q1) During the task I often set a specific goal for myself Q2) I always focused my attention on the text-editing task Q3) I often worried about not being able to reach my best possible score Q4) I often forgot about trying to reach my best possible score Q5) I wondered about how my performance compared with others Q6) I constantly focused on trying to do my best Q7) I often daydreamed while doing the task Q8) I often let my mind wander while doing the task Q9) How difficult was this task? (High=Extremely Difficult) Q10) How much of your attention was required to perform this task? (High = A great deal of attention) Q11) How pressured did you feel while performing this task? (High=Extremely Pressured) 1 .839 .587 .789 -.531 .678 .475 .042 .212 -.126 -.014 .265 Component 2 .048 -.314 .093 .314 .364 -.546 .929 .980 .146 -.355 .009 3 -.271 -.116 .214 -.181 .199 -.113 -.007 -.099 .778 .708 .699

1. 2. 3.

2 2 Extractionis .8392Principal Component Analysis. What Method: Rotation Method: Promax with Kaiser Normalization. Why is it not the same as the communality? a. Rotation converged in 6 iterations.

+ .048 + .271 ?

Component Correlation Matrix Component 1 2 3 1 1.000 -.385 .039 2 -.385 1.000 .004 3 .039 .004 1.000

4. 5. 6.

For each of the three components say which variables load highly and what a person who was high on that component would look like? Give a clever one or two word name to each component that reflects what high scores on the variable represent Using the names you created state which two components correlated? Does this correlation make sense based on your understanding of the components?

Extraction Method: Principal Component Analysis. Rotation Method: Promax with Kaiser Normalization.

Example 4.2
Research Question
Now that we have a measure of self-reported Task Focus, we can use it as a dependent variable to think about what might predict it. The variable was created by getting the average score on the items that load on it. Is there an effect of gender and/or age group as defined by being either over or under 25 years of age on Task Focus? Do these two variables interact?
1. 2. 3. 4. Describe the three variables in terms of being binary, nominal, ordinal, interval or ratio? Which statistical procedure could we use? What would be the null hypotheses? What is your expectation?

UNIANOVA qt_fac1 BY agegroup gender /METHOD = SSTYPE(3) /INTERCEPT = INCLUDE /PLOT = PROFILE( agegroup*gender ) /PRINT = DESCRIPTIVE HOMOGENEITY /CRITERIA = ALPHA(.05) /DESIGN = agegroup gender agegroup*gender .

4.2 Descriptive Statistics


Mean 4.5938 4.6354 4.6146 4.7979 4.3750 4.5865 4.6958 4.5052 4.6005 Std. Deviation 1.18160 1.32668 1.23599 1.32332 1.14261 1.23500 1.23842 1.22510 1.22573 N 16 16 32 16 16 32 32 32 64

Descriptiv e Statistics Dependent Variable: QT: Task Focused Age Group Gender 1 25 & under 1 male 2 female Total 2 Over 25 1 male 2 female Total Total 1 male 2 female Total

1. 2. 3. 4. 5.

Looking at the graph, which gender was more task focused? Which age group was more task focused? And does there appear to be an interaction? What is the marginal mean for males? What is the marginal mean for females? What is the difference between the two? What is the approximate difference between the marginal means for males and females in terms of one of the standard deviations? Does the difference between the groups seem large? What might be deceptive about the default graph from SPSS?
Note that I filtered the cases to give equal group sizes. This would not typically be done in real research but is purely done to make it clearer as a learning tool for you.

4.2 Homogeneity of Variance


Descriptiv e Statistics Dependent Variable: QT: Task Focused Age Group Gender 1 25 & under 1 male 2 female Total 2 Over 25 1 male 2 female Total Total 1 male 2 female Total Mean 4.5938 4.6354 4.6146 4.7979 4.3750 4.5865 4.6958 4.5052 4.6005 Std. Deviation 1.18160 1.32668 1.23599 1.32332 1.14261 1.23500 1.23842 1.22510 1.22573 N 16 16 32 16 16 32 32 32 64
a Lev ene's Test of Equality of Error Variances

Dependent Variable: QT: Task Focused F .428 df1 3 df2 60 Sig. .734

Tests the null hypothesis that the error variance of the dependent variable is equal across groups. a. Design: Intercept+agegroup+gender+agegroup * gender

1. 2. 3. 4. 5. 6. 7. 8. 9. 10.

What is the assumption of homogeneity of variance? What is the relationship between the standard deviation and variance? What does the standard deviation and variance both describe about a sample? What is the standard deviation of scores on Task Focus for Males Under 25? What is the standard deviation of scores in the other three cells of the design? Are any of the standard deviations twice as big as any of the others? Which two cells have the most disparate standard deviations? Are the standard deviations in the different cells very different? What is the null hypothesis that Levenes test is testing? What interpretation do we make about the assumption based on Levenes test?

Tests of Between-Subj ects Effects Dependent Variable: QT: Task Focused

4.2 Significance Test

Source Corrected Model Intercept agegroup gender agegroup * gender Error Total Corrected Total

Type III Sum of Squares 1.457 a 1354.547 .013 .581 .863 93.195 1449.199 94.652

df 3 1 1 1 1 60 64 63

Mean Square .486 1354.547 .013 .581 .863 1.553

F .313 872.074 .008 .374 .556

Sig. .816 .000 .928 .543 .459

a. R Squared = .015 (Adjusted R Squared = -.034)

1. Write out the F statistics, degrees of freedom and p value for the three effects. 2. Whats the answer to our research questions? 3. Using your formula sheet to assist, what three values make up SS between? 4. What is SS between? 5. What is SS within? 6. What is SS between + SS within (i.e., SS Total)? 7. Which source does SS Total correspond to in the table? 8. Without looking back at earlier slides, what was the total sample size? 9. What two values were used to form the F ratio for Gender? 10.What is the critical F value from Howells Table for the hypotheses tested (=.05) 11.If all you could see was the F values, and not the significance values, how would you know that all the effects were not statistically significant? 12.What is the value for MS error? 13.What is the square root of MS error? How does this value compare to the standard deviations shown on the previous slide?

Example 4.3
Research Question
Do the three abilities (z-scores) and prior learning as defined by typing speed (words per minutes) and knowledge of text editing short cut keys (% correct) predict task performance (seconds) at the end of practice over and above initial performance (seconds)?

1. 2. 3. 4.

Describe the three variables in terms of being binary, nominal, ordinal, interval or ratio? Which statistical procedure could we use? What would be the null hypotheses? What is your expectation?

4.3 Descriptive Statistics


Descriptiv e Statistics Mean TEP: Text Editing Performance (Block 9) (Average Seconds to Complete) TEP: Text Editing Performance (Block 1) (Average Seconds to Complete) Zscore: GA: General Ability Total Zscore: PSA: Perceptual Speed Ability Total Zscore: PMA: Psychomotor Ability Total QBK: Total Knowledge of Text Editing Keys (% correct) Typing Test: Speed (Words Per Minute) 34.0112 52.7700 .0911 .1006 .0638 .4747 30.587 Std. Deviation 13.06938 14.54237 .98127 .99398 .93199 .25881 11.5023 N 99 99 99 99 99 99 99

1.

What was the average improvement in seconds from block 1 to block 9?

Note that the 54 trials were grouped into blocks made up of six trials each. Thus, initial performance is defined by block 1 (i.e., trials 1 to 6) and block 9 is made up of trials 49 to 54) Also remember that faster times imply better performance

4.3 Correlations & Multicollinearity


Correlations Pearson Correlation TEP: Text Editing Performance (Block 9) (Average Seconds to Complete) TEP: Text Editing Performance (Block 9) (Average Seconds to Complete) TEP: Text Editing Performance (Block 1) (Average Seconds to Complete) Zscore: GA: General Ability Total Zscore: PSA: Perceptual Speed Ability Total Zscore: PMA: Psychomotor Ability Total QBK: Total Knowledge of Text Editing Keys (% correct) Typing Test: Speed (Words Per Minute) 1.000 TEP: Text Editing Performance (Block 1) (Average Seconds to Complete) .725 Zscore: PMA: Psycho motor Ability Total -.571 Typing Test: Speed (Words Per Minute) -.492 Zscore: GA: General Ability Total -.312 Zscore: PSA: Perceptual Speed Ability Total -.403 QBK: Total Knowledge of Text Editing Keys (% correct) -.389

.725 -.312 -.403 -.571 -.389 -.492

1.000 -.318 -.464 -.558 -.473 -.509

-.318 1.000 .347 .216 .077 .207

-.464 .347 1.000 .425 .054 .358

-.558 .216 .425 1.000 .138 .297

-.473 .077 .054 .138 1.000 .394

-.509 .207 .358 .297 .394 1.000

1. 2. 3.
4.

Which variable has the largest correlation with block 9 text editing performance? Do all the variables have moderate or large correlations with the dependent variable? Give an interpretation to the correlation between typing speed and block 9 text editing performance Are there any very large correlations between predictors (e.g., above .7)?

4.3 Model Assessment


c Model Summary

Change Statistics Model 1 2 R R Square a .725 .525 b .771 .594 Adjusted R Square .520 .568 Std. Error of the Estimate 9.05125 8.59244 R Square Change .525 .069 F Change 107.324 3.127 df1 1 5 df2 97 92 Sig. F Change .000 .012

a. Predictors: (Constant), TEP: Text Editing Performance (Block 1) (Average Seconds to Complete) ANOVAc b. Predictors: (Constant), TEP: Text Editing Performance (Block 1) (Average Seconds1. Complete), Zscore: GA: General Ability What is the r-squared to Total, QBK: Total Sum of Knowledge of Text Editing Keys (% correct), Zscore: PSA: Perceptual Speed Ability Total, Typing Test: Speed before and after adding the Model Squares PMA: Psychomotor Ability Total df Mean Square F Sig. (Words Per Minute), Zscore: 1 Regression 8792.527 1 8792.527 107.324 .000 a ability and past experience c. Dependent Variable: TEP: Text Editing Performance (Block 9) (Average Seconds to Complete) Residual 7946.731 97 81.925 variables? Total 16739.258 98 2. What was the R-square 2 Regression 9946.903 6 1657.817 22.455 .000 b Residual 6792.356 92 73.830 change? Total 16739.258 98 3. Was the r-square change a. Predictors: (Constant), TEP: Text Editing Performance (Block 1) (Average Seconds statistically significant? to Complete) b. Predictors: (Constant), TEP: Text Editing Performance (Block 1) (Average Seconds 4. Write out the F statistic for to Complete), Zscore: GA: General Ability Total, QBK: Total Knowledge of Text the final model with the rEditing Keys (% correct), Zscore: PSA: Perceptual Speed Ability Total, Typing Test: Speed (Words Per Minute), Zscore: PMA: Psychomotor Ability Total square? c. Dependent Variable: TEP: Text Editing Performance (Block 9) (Average Seconds to 5. Whats the answer to the Complete)

research question?

Model 1

4.3 Regression Equation

Coefficientsa Unstandardized Coefficients B Std. Error -.360 3.440 .063 6.895 .091 .962 1.076 1.162 4.057 .092 .449 -.078 -.009 -.248 -.081 -.139 Standardized Coefficients Beta .725 95% Confidence Interval for B Lower Bound Upper Bound -7.188 6.468 .527 6.126 .222 -2.950 -2.251 -5.782 -12.143 -.340 .776 33.512 .584 .870 2.024 -1.167 3.970 .024

(Constant) TEP: Text Editing Performance (Block 1) .651 (Average Seconds to Complete) (Constant) 19.819 TEP: Text Editing Performance (Block 1) .403 (Average Seconds to Complete) Zscore: GA: General Ability Total -1.040 Zscore: PSA: Perceptual Speed Ability -.114 Total Zscore: PMA: Psychomotor Ability Total -3.474 QBK: Total Knowledge of Text Editing Keys -4.086 (% correct) Typing Test: Speed (Words Per Minute) -.158

t -.105 10.360 2.875 4.421 -1.082 -.106 -2.990 -1.007 -1.720

Sig. .917 .000 .005 .000 .282 .916 .004 .316 .089

Zero-

a. Dependent Variable: TEP: Text Editing Performance (Block 9) (Average Seconds to Complete)

1. 2. 3. 4. 5.

6.
7. 8. 9.

Write out the unstandardised regression equation for the final model Which regression coefficients have lower and upper bound 95% confidence intervals that are either both positive or both negative? Which regression coefficients are statistically significant? What is the degrees of freedom associated with the Block 1 text editing performance regression coefficient in model 1? What about model 2? What is the unstandardised regression coefficient for General Ability? What is the standard error? What is the regression coefficient divided by the standard error? Look up Howells table and see what critical t is associated with 92 degrees of freedom alpha = .05 (two tailed)? What is 1 - .05 in percentage terms? What is the critical t (.05 two-tailed) multiplied by the standard error for General ability? What is -1.04 plus the value you obtained in Q8? What is -1.04 minus the value in Q8?

4.3 Correlations & Multicollinearity


Coefficientsa Correlations Zero-order Partial TEP: Text Editing Performance (Block 1) (Average Seconds to Complete) TEP: Text Editing Performance (Block 1) (Average Seconds to Complete) Zscore: GA: General Ability Total Zscore: PSA: Perceptual Speed Ability Total Zscore: PMA: Psychomotor Ability Total QBK: Total Knowledge of Text Editing Keys (% correct) Typing Test: Speed (Words Per Minute) .725 .725 -.312 -.403 -.571 -.389 -.492 .725 .419 -.112 -.011 -.298 -.104 -.177 Collinearity Statistics Tolerance VIF 1.000 .428 .846 .659 .642 .683 .678 1.000 2.336 1.182 1.518 1.557 1.463 1.475 Model 1 2 Part .725 .294 -.072 -.007 -.199 -.067 -.114

a. Dependent Variable: TEP: Text Editing Performance (Block 9) (Average Seconds to Complete)

1. 2. 3. 4. 5.

Why is tolerance 1 in Model 1? Are there any concerns with multicollinearity? What is VIF for psychomotor ability? What is 1 / VIF for psychomotor ability? In Model 2 which variable has the largest semi-partial correlation? What is it squared? What would be the interpretation of it when it is squared? Which semi-partial correlations are statistically significant (hint: you may have to look at the previous slide)?

a Residuals Statistics

4.3 Casewise Diagnostics

Predicted Value Std. Predicted Value Standard Error of 1.158 Predicted Value Adjusted Predicted Value 8.2281 Residual -22.08496 Std. Residual -2.570 Stud. Residual -2.661 Deleted Residual -23.66952 Stud. Deleted Residual -2.755 Mahal. Distance .791 Cook's Distance .000 Centered Leverage Value .008

Minimum 8.8793 -2.495

Maximum 62.0571 2.784 3.974 64.8080 30.63221 3.565 3.630 31.75273 3.900 19.968 .145 .204

Mean 34.0112 .000 2.210 33.9752 .00000 .000 .002 .03602 .005 5.939 .011 .061

Std. Deviation 10.07467 1.000 .584 10.18449 8.32525 .969 1.005 8.96232 1.026 3.713 .023 .038

N 99 99 99 99 99 99 99 99 99 99 99 99

a. Dependent Variable: TEP: Text Editing Performance (Block 9) (Average Seconds to Complete)

1.

2.
3. 4. 5. 6.

What is the rule of thumb suggested by Paul for studentised residuals in large samples? Are any cases outside that rule of thumb? What is the rule of thumb suggested by Paul for leverage in large samples? Are any cases outside this rule of thumb? What is the rule of thumb suggested by Paul for identifying influential cases? Are any cases outside the rule of thumb? If cases are identified as falling outside the rule of thumb, what is the next step? What is the mean of the predicted value? Going back to descriptive statistics, what was the mean of the DV? If leverage is defined by extreme scores on predicted Y, based on the table above, what are the two possible predicted values that correspond to the case with the largest leverage value?

a Casew ise Diagnostics

4.3 Residuals
Case Number 17 Std. Residual 3.565

TEP: Text Editing Performance (Block 9) (Average Seconds to Complete) 69.60

Predicted Value 38.9671

Residual 30.63221

a. Dependent Variable: TEP: Text Editing Performance (Block 9) (Average Seconds to Complete)

1. 2. 3. 4.

Do the residuals look normally distributed? What was the observed final text edit time minus the predicted time for case 17? Is the residual positive or negative? Is the studentised residual positive or negative? Some people were faster than predicted and some people were slower than predicted. How does this relate to positive and negative residuals?

4.3 Linearity & Homogeneity of variance

1. 2. 3. 4.

Going back to the casewise diagnostics what is the mean and standard deviation of the predicted value? What would be the unstandardised predicted time at minus 1 standardised predicted values? What about plus 1 standardised predicted values Is there a difference in the spread of residuals at one versus minus one standardised predicted values? What might this mean? Is there any evidence of non-linearity?

4.3 Linearity

1.

Is there any evidence of non-linearity?

4.3 Looking at Cases

The above table shows 5 cases from the dataset and their associated values for the variables in the model and various outlier values and prediction values a Case Summaries
1 50.81 8.21 .99 .011 .06 59.03 76.91 19.3 .57 -1.47 -.66 -1.08 2 39.20 .22 .03 .000 .08 39.42 57.25 24.7 .57 1.52 1.14 -1.22 3 19.81 -4.34 -.53 .004 .09 15.47 30.00 51.0 1.00 -.13 -.76 .06 4 32.49 -7.17 -.88 .012 .09 25.32 46.69 47.6 .29 1.05 .58 -1.06 5 39.00 -.50 -.06 .000 .02 38.50 60.54 21.0 .43 .72 -.20 -.16

Unstandardized Predicted Value Unstandardized Residual Studentized Residual Cook's Distance Centered Leverage Value TEP: Text Editing Performance (Block 9) (Average Seconds to Complete) TEP: Text Editing Performance (Block 1) (Average Seconds to Complete) Typing Test: Speed (Words Per Minute) QBK: Prior Text Editing Knowledge Overall (% correct) Zscore: GA: General Ability Total Zscore: PSA: Perceptual Speed Ability Total Zscore: PMA: Psychomotor Ability Total a. Limited to first 5 cases.

Remember that the model regression equation looked like this: [Predicted Block 9] = 19.8 + .40Block1 1.0GA - .1PSA 3.5PMA 4.1QBK -.16WPM

1. 2. 3.

Confirm for a couple of cases that when you apply the model regression equation to the cases value that it corresponds to the unstandardised predicted value? Describe case 1 in words, you may wish to make reference to the means and standard deviations in the descriptive statistics Case 1 and 4 have very different profiles of scores. Compare case 4 to 1? Why do they have similar leverage values, which are clearly larger than cases 2, 3 and 5?

Example 4.4
Initial Note
Using a method that you wont learn about until 4th year psychology called cluster analysis, 3 groups of participants were created based on their ability scores. 3 Ability Groups
Lower Ability Group Higher Ability Group Lower General Ability / Higher Psychomotor Ability Group

Research Question
Do the three groups differ in terms of their overall performance on the text editing task? Overall did performance improve from block 1 (i.e., start of the task) to block 9 (i.e., end of the task) on the text editing task? Did any of the three groups improve more or less between block 1 and block 9?

1.
2. 3. 4.

Describe the three variables in terms of being binary, nominal, ordinal, interval or ratio? Which statistical procedure could we use? What would be the null hypotheses? What is your expectation?

Note that to simplify the example, I filtered some cases to give equal group sizes.
Within-Subj ects Factors Measure: MEASURE_1 block 1 2 Dependent Variable tep_mean.1 tep_mean.9
TEP: Text Editing Performance (Block 1) (Average Seconds to Complete)

4.4 Descriptives
Descriptiv e Statistics Ability Cluster 1 Lower Ability Group 2 Higher Ability Group 3 Lower General Ability / Higher Psychomotor Abilty Group Total 1 Lower Ability Group 2 Higher Ability Group 3 Lower General Ability / Higher Psychomotor Abilty Group Total Mean 63.4012 44.7313 50.0116 52.7147 44.1289 27.7679 31.9039 34.6002 Std. Deviation 13.55512 13.65499 12.97988 15.40415 14.11005 10.25845 11.85985 13.90038 N 25 25 25 75 25 25 25 75

TEP: Text Editing Performance (Block 9) (Average Seconds to Complete)

Looking at mean task performance: 1. Does it look like there is a difference between the ability groups? 2. Does it look like people improved over time? 3. Does it look like there is an interaction?

4.4 Homogeneity of covariance matrices


a Box's Test of Equality of Cov ariance Matrices

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14.

Box's M F df1 df2 Sig.

6.492 1.038 6 129201.2 .398

Note, you are not expected to Design: Intercept+abilitycluster know how to produce the Within Subjects Design: block customised output below: it shows the variance, covariance and correlation between initial and final task performance for the three groups
Ability Cluster 1 Lower Ability Group

Tests the null hypothesis that the observed covariance matrices of the dependent variables are equal across groups. a.

What does covariance mean? What is the covariance of a variable with itself? What is the relationship between correlation and covariance? What is the relationship between variance and the standard deviation? If covariance was a positive number what would that say about the two variables being covaried? And what if covariance was a negative number? What is the assumption of homogeneity of variance? What is the assumption of homogeneity of covariance? What is the null hypothesis tested with Boxs test? What are the two dependent variables in this example? Would you expect performance at block 1 (start of task) to be correlated with performance at block 9 (end of task)? Which group has the largest and which has the smallest covariance between block 1 and block 9? What about correlation? What is the variance in task performance in the lower ability group at block 1? What about block 2? Do the 6 variances look similar? Do the 3 covariances look similar? What does Boxs Matrixsuggest about the status of our assumption? Test Residual SSCP
Covariance TEP: Text Editing Performance (Block 1) (Average Seconds to Complete) 183.741 144.829 186.459 98.638 168.477 73.542 TEP: Text Editing Performance (Block 9) (Average Seconds to Complete) 144.829 199.094 98.638 105.236 73.542 140.656 Correlation TEP: Text Editing Performance (Block 1) (Average Seconds to Complete) 1.000 .757 1.000 .704 1.000 .478 TEP: Text Editing Performance (Block 9) (Average Seconds to Complete) .757 1.000 .704 1.000 .478 1.000

2 Higher Ability Group

3 Lower General Ability / Higher Psychomotor Abilty Group

TEP: Text Editing Performance (Block 1) (Average Seconds to Complete) TEP: Text Editing Performance (Block 9) (Average Seconds to Complete) TEP: Text Editing Performance (Block 1) (Average Seconds to Complete) TEP: Text Editing Performance (Block 9) (Average Seconds to Complete) TEP: Text Editing Performance (Block 1) (Average Seconds to Complete) TEP: Text Editing Performance (Block 9) (Average Seconds to Complete)

4.4 Significance Tests


1. Write out the F test and statistical significance for ability group, block and the interaction If we had three levels to our repeated measures factor, how many covariances could we have between the three levels? In this case we have two levels, how many covariances could we have? What is the assumption of compound symmetry that Philip has taught you? If there are three different covariances, it is possible for them to be different, but what if there is only 1?

Tests of Between-Subj ects Effects

Measure: MEASURE_1 Transformed Variable: Average Source Intercept abilitycluster Error Type III Sum of Squares 285896.188 8216.399 19412.152 df 1 2 72 Mean Square 285896.188 4108.200 269.613 F 1060.394 15.237 Sig. .000 .000

2.

3.

b Mauchly's Test of Sphericity

Measure: MEASURE_1 Epsilon Within Subjects Effect Mauchly's W block 1.000 Approx. Chi-Square .000 df 0 Sig. . Greenhous e-Geisser 1.000
a

4.

5.

Huynh-Feldt 1.000

Lower-bound 1.000

Tests the null hypothesis that the error covariance matrix of the orthonormalized transformed dependent variables is proportional to an identity matrix. a. May be used to adjust the degrees of freedom for the averaged tests of significance. Corrected tests are displayed in Tests the Tests of Within-Subjects Effects table. of Within-Subj ects Effects b. Measure: MEASURE_1 Design: Intercept+abilitycluster Source Within Subjects Design: block block Sphericity Assumed Greenhouse-Geisser Huynh-Feldt Lower-bound block * abilitycluster Sphericity Assumed Greenhouse-Geisser Huynh-Feldt Lower-bound Error(block) Sphericity Assumed Greenhouse-Geisser Huynh-Feldt Lower-bound

Type III Sum of Squares 12305.076 12305.076 12305.076 12305.076 33.319 33.319 33.319 33.319 4195.750 4195.750 4195.750 4195.750

df 1 1.000 1.000 1.000 2 2.000 2.000 2.000 72 72.000 72.000 72.000

Mean Square 12305.076 12305.076 12305.076 12305.076 16.659 16.659 16.659 16.659 58.274 58.274 58.274 58.274

F 211.158 211.158 211.158 211.158 .286 .286 .286 .286

Sig. .000 .000 .000 .000 .752 .752 .752 .752

4.4 SPSS Output and Split Plot ANOVA formula


1. What is a, b and n in the present experiment? 2. What does Factor A & subj w/in A correspond to in the previous SPSS Output? 3. What does Factor B, A x B, and B x Subj w/in A correspond to in the previous SPSS output? 4. Use the rules for adding and subtracting sums of squares and your knowledge from q2 and q3 to work out SS between subjects; SS within subjects; SS total; and SS cells 5. You may find it useful taking the above Sums of squares and knowledge of the design to fill out the remainder of the ANOVA table and checking your results against the previous slide Use the previous slide in conjunction with the table taken from your formula sheet below to answer the questions

4.4 Between Subjects contrasts


1. What is the sample size per group (see previous slide)? 2. How many levels were there to the within subjects variable? 3. The table on the bottom right represents the marginal means for the groups, how many observations were the means based on? 4. If we were going to perform a contrast on marginal means for the between subjects factor (Ability Cluster), and apply the formula on the top right, what would n, ms error, df error, and critical F( = .05) be? 5. How many orthogonal contrasts are possible on ability cluster? 6. Test for statistical significance whether mean task performance for the average of group 2 and 3 is different from group 1? 7. What is the only possible remaining orthogonal contrast? 8. Test the remaining contrast from Q7 for statistical significance 9. What was the Sums of Squares for Ability Cluster (see earlier slide)? 10.If the numerator df = 1, what is the relationship between SS contrast and MS contrast? 11.Whats the difference between the F contrast formula and the SS contrast formula? 12.All f ratios are the ratio between what and what? 13.What is the MS for the two contrasts you solved in Q6 and Q8? 14.What is the sum of the two SS contrasts for Q6 and Q8? 15.What is the numerator df for the effect of ability cluster? 16.What is the numerator df for any contrast? 17.What is the sum of the numerator dfs in Q6 and Q8? 18.What would the weights have been, if you had been testing for a linear trend? What about a quadratic trend? 19.What would our alpha be if we bonferonni corrected for the two contrasts performed? 20.What would critical F Scheff .05 be? Would this change the results?

Ability Cluster Measure: MEASURE_1

Ability Cluster 1 Lower Ability Group 2 Higher Ability Group 3 Lower General Ability / Higher Psychomotor Abilty Group

Mean 53.765 36.250 40.958

Fixed & Random Effects - Example


1.Imagine we were interested in the effect of a drug to reduce depression symptoms. A study was conducted at a particular research centre (Centre A) comparing a group receiving the drug with a group receiving a placebo. Assuming that the only measures obtained were Depression levels after 3 months of treatment, what is the independent and dependent variable? What inferential statistic could we use? 2.In the language of fixed and random factors, what type of variable is group (treatment vs placebo)? Why? 3.If the drug is actually effective what effect would increasing our sample size have on statistical power? 4.How would we operationalise the effectiveness of the drug? 5.Imagine now that the study is being conducted in 10 randomly selected research sites around the country, of which centre A is only one. Do you think the drugs underlying effectiveness would vary across research sites? For what reasons might it vary? 6.Would the assessed effectiveness of the drug vary based on random sampling between sites? 7.How would we assess empirically if the drugs underlying effectiveness varies across sites? 8.If we had only two sites, would we have a good estimate of how much the effectiveness varies across sites? 9.Would increasing our sample size at each site improve our understanding of the degree of variability in effectiveness across sites? 10.If you found out that the effectiveness of the drug varied substantially across sites, would you feel confident to generalise the findings from any one site to the population? What about if the effectiveness of the drug was fairly stable across sites? 11.What does the size of the site by group interaction represent in this context? 12.If there was no population interaction effect, what would MS interaction represent? 13.In the language of fixed and random effect variables, what kind of variable is site? 14.What do the rules tell us we use as our error term for site? What about group? 15.When the denominator mean square (either MS error or MS interaction) gets bigger what happens to obtained F? 16.When the denominator df (either df error or df interaction) gets bigger, what happens to critical F? 17.What happens to the interaction df when the number of sites increases? 18.If larger interaction effects indicate less certainty about the actual effect of the drug, does it make sense that we would want to incorporate that uncertainty into our significance test (i.e., our F ratio)? 19.Does it make sense that the more sites we have, the better we are able to assess our interaction effect and therefore that our interaction df will increase and power will increase?

You might also like