Principal Component Analysis

Assigned by: Dr. Sharjeel Saleem Assigned to: Waqas Nazir Roll No: 9609 MS (BA) 2nd semester

Principal Component Analysis

Principal components analysis is a method of data reduction. It reduce the large number of variables to just a few dimensions.It reduces the variables which have same impact. On the basis of correlation we classify their same impact. Suppose that you have a dozen variables that are correlated. You might use principal components analysis to reduce your 12 measures to a few principal components. In PCA there is no dependent and independent variable. To construct validity of results we use PCA.

Factor Analysis
Descriptive Statistics Mean INSTRUC WELL PREPARED INSTRUC SCHOLARLY GRASP 4.46 4.53 Std. Deviation .729 .700 Analysis N 1365 1365

INSTRUCTOR CONFIDENCE

4.45

.732

1365

INSTRUCTOR FOCUS LECTURES

4.28

.829

1365

INSTRUCTOR USES CLEAR RELEVANT EXAMPLES

4.17

.895

1365

INSTRUCTOR SENSITIVE TO STUDENTS

3.93

1.035

1365

INSTRUCTOR ALLOWS ME TO ASK QUESTIONS

4.08

.964

1365

INSTRUCTOR IS ACCESSIBLE TO STUDENTS OUTSIDE CLASS

3.78

.909

1365

INSTRUCTOR AWARE OF STUDENTS UNDERSTANDING

3.77

.984

1365

I AM SATISFIED WITH STUDENT PERFORMANCE EVALUATION

3.61

1.116

1365

COMPARED TO OTHER INSTRUCTORS, THIS INSTRUCTOR IS

3.81

.957

1365

COMPARED TO OTHER COURSES THIS COURSE WAS

3.67

.926

1365

This table shows the descriptive statistics. First Colum shows the variables. 2nd Colum shows the mean, which tell us central value of the data. 3rd Colum shows standard deviation, which shows the variance from the central value of data or mean. Last Colum shows the number of total observations used in the analysis.

INSTRUC WELL PREPARED

INSTRUC SCHOLARLY GRASP

INSTRUCTOR CONFIDENCE

INSTRUCTOR

FOCUS LECTUR

INSTRUC WELL PREPARED INSTRUC SCHOLARLY GRASP INSTRUCTOR CONFIDENCE INSTRUCTOR FOCUS LECTURES INSTRUCTOR USES CLEAR RELEVANT EXAMPLES Correlation INSTRUCTOR SENSITIVE TO STUDENTS INSTRUCTOR ALLOWS ME TO ASK QUESTIONS INSTRUCTOR IS ACCESSIBLE TO STUDENTS OUTSIDE CLASS INSTRUCTOR AWARE OF STUDENTS UNDERSTANDING

1.000 .661 .600 .566 .577

.661 1.000 .635 .500 .552

.600 .635 1.000 .505 .587

1.

.409

.433

.457

.286

.320

.359

.304

.315

.356

.476

.449

.509

I AM SATISFIED WITH STUDENT PERFORMANCE EVALUATION COMPARED TO OTHER INSTRUCTORS, THIS INSTRUCTOR IS COMPARED TO OTHER COURSES THIS COURSE WAS

.333

.333

.369

.564

.565

.582

.454

.443

.435

This table shows the correlation matrix among the variables. Correlation matrix to check the pattern of relationships. In factor analysis we have Variables that correlate fairly well, but not perfectly. Also, any variables that correlate with no others should be eliminated. Correlation matrix shows the diagonal items which are equal to 1 and off-diagonal items are not equal to zero. Means it is not an identity matrix.

KMO and Bartlett's Test Kaiser-Meyer-Olkin Measure of Sampling Adequacy. Approx. Chi-Square Bartlett's Test of Sphericity df Sig. .934 8676.712 66 .000

KMO value varies between 0 and 1, and values closer to 1 are better. Kaiser recommends a bare minimum of 0.5 and that values between 0.5 and 0.7 are mediocre, values between 0.7 and 0.8 are good, values between 0.8 and 0.9 are great and values above 0.9 are superb. In our analysis it is 0.934 which means that it is valid and superb, so we should be confident that the sample size is adequate for factor analysis Bartletts test is used to determine rather the correlation matrix is an identity matrix or not. Bartletts test is used to test the null hypothesis that the correlation matrix is an identity matrix. An identity matrix is matrix in which all of the diagonal elements are 1 and all off diagonal elements are 0. In this example Bartletts test is highly significant (p < .001) which means it is not an identity matrix. Correlation matrix shows that variables are well correlated with each other.

Communalities Initial INSTRUC WELL PREPARED INSTRUC SCHOLARLY GRASP INSTRUCTOR CONFIDENCE INSTRUCTOR FOCUS LECTURES INSTRUCTOR USES CLEAR RELEVANT EXAMPLES INSTRUCTOR SENSITIVE TO STUDENTS INSTRUCTOR ALLOWS ME TO ASK QUESTIONS INSTRUCTOR IS ACCESSIBLE TO STUDENTS OUTSIDE CLASS INSTRUCTOR AWARE OF STUDENTS UNDERSTANDING I AM SATISFIED WITH STUDENT PERFORMANCE EVALUATION COMPARED TO OTHER INSTRUCTORS, THIS INSTRUCTOR IS COMPARED TO OTHER COURSES THIS COURSE WAS Extraction Method: Principal Component Analysis. 1.000 .509 1.000 .673 1.000 .557 1.000 .601 1.000 .494 1.000 1.000 Extraction .731 .690

1.000

.652

1.000

.549

1.000

.661

1.000

.704

1.000

.658

Communalities - The proportion of each variable's variance (common variance) that can be explained by the principal components. Initial - The initial value of the communality in a principal components analysis is 1.

 Principal component analysis initial assumption that all variance is common; therefore, before extraction the communalities are all 1. Extraction - values after the extraction of variables. The values in this column indicate the proportion of each variable's variance that can be explained by the principal components. Variables with high values are well represented in the common factor space, while variables with low values are not well represented.
Total Variance Explained Component Initial Eigen values Extraction Sums of Squared Loadings Total % of Variance 1 2 3 4 5 6 7 8 9 10 11 12 6.249 1.229 .719 .613 .561 .503 .471 .389 .368 .328 .317 .252 52.076 10.246 5.992 5.109 4.676 4.192 3.927 3.240 3.066 2.735 2.645 2.096 Cumulative % 52.076 62.322 68.313 73.423 78.099 82.291 86.218 89.458 92.524 95.259 97.904 100.000 6.249 1.229 Total % of Variance 52.076 10.246 Cumulative % 52.076 62.322 4.000 3.478 Total % of Variance 33.336 28.986 Cumulative % 33.336 62.322 Rotation Sums of Squared Loadings

Extraction Method: Principal Component Analysis.

This table shows the total variance explained. Component - The initial number of factors is the same as the number of variables used in the factor analysis. Initial Eigen values are the variances explained by the principal components. we conducted our principal components analysis on the correlation matrix, the variables are

standardized, which means that the each variable has a variance of 1, and the total variance is equal to the number of variables used in the analysis, in this case. Total - This column contains the Eigen values. The first factor will always account for the most variance (and hence have the highest Eigen value), and the next factor will account for as much of the left over variance as it can, and so on. Hence, each successive factor will account for less and less variance. % of Variance - This column contains the percent of total variance accounted for by each factor. Cumulative % - This column contains the cumulative percentage of variance accounted for by the current and all preceding factors. For example, the 2nd row shows a value of 62.322. This means that the first two factors together account for 62.322% of the total variance. Extraction Sums of Squared Loadings - The number of rows in this panel of the table correspond to the number of factors retained. In our example, we used 12 variables (item13 through item24), so we have 12 components. Two components are extracted and those two components accounted for 62% of the total variance, then we would say that two dimensions in the component space account for 62% of the variance. Total - This column contains the Eigen values. We will include only those components whose Eigen value is greater than 1.The first component will always account for the most variance (and hence have the highest Eigen value), and the next component will account for as much of the left over variance as it can. There are only two components whose Eigen value is greater than 1. 1st component have Eigen value of 6.249 & 2nd component have Eigen value of 1.229. % of Variance - 1st component shows 52.076% of variance & 2nd component shows 10.246% of variance and their total or Cumulative % of variance is 62.322. First two components account for most of the variation in the data. It means that dimensionality of the problem is less than p Rotation Sums of Squared Loadings- Rotation has the effect of optimizing the factor structure and one consequence for these data is that the relative importance of the two factors is equalized.

 The values in this panel of the table represent the distribution of the variance after the varimax rotation. Varimax rotation tries to maximize the variance of each of the factors, so the total amount of variance accounted for is redistributed over the 2 extracted factors. After rotation: 1st component have Eigen value of 4.000 & 2nd component have Eigen value of 3.478. 1st component shows 33.336% of variance & 2nd component shows 28.986% of variance. Their total or Cumulative % of variance is 62.322. After rotation both variables have almost equal loadings.

Scree plot graphs the Eigen value against the component number. We take Eigen value on (Yaxis) against the factor with which it is associated (X-axis). You can see these values in the first two columns of the table immediately above. Cut-off point for selecting factors should be at the point of inflexion of this curve. 3rd factor is a point of inflexion where the slope of the line changes dramatically. We retain (or extract) only factors to the left of the point of inflexion (and do not include the factor at the point of inflexion itself). From the third component on, you can see that the line is almost flat, meaning the each successive component is accounting for smaller

and smaller amounts of the total variance. In general, we are interested in keeping only those principal components whose Eigen values are greater than 1.
Component Matrix a Component 1 COMPARED TO OTHER INSTRUCTORS, THIS INSTRUCTOR IS INSTRUCTOR USES CLEAR RELEVANT EXAMPLES INSTRUCTOR AWARE OF STUDENTS UNDERSTANDING INSTRUCTOR SENSITIVE TO STUDENTS INSTRUCTOR CONFIDENCE INSTRUC WELL PREPARED INSTRUC SCHOLARLY GRASP COMPARED TO OTHER COURSES THIS COURSE WAS INSTRUCTOR FOCUS LECTURES I AM SATISFIED WITH STUDENT PERFORMANCE EVALUATION INSTRUCTOR ALLOWS ME TO ASK QUESTIONS INSTRUCTOR IS ACCESSIBLE TO STUDENTS OUTSIDE CLASS Extraction Method: Principal Component Analysis. a. 2 components extracted. .593 .378 .641 .497 .651 .364 .685 .714 .755 .366 .763 .806 .819 2

.746 .727 .724

-.308 -.449 -.408

Component Matrix - This table contains factors loadings, which are the correlations between the variable and the component. Because these are correlations, possible values range from -1 to

+1. We used the option loading of an absolute value (.30), which tells SPSS not to include any of the correlations that are .3 or less. This makes the output easier to read by removing the low correlations that are probably not meaningful anyway. Factor - The columns under this heading are the un-rotated factors that have been extracted. Two components were extracted (the two components that had an Eigen value greater than 1). Some variables are significant on both factors and they can become the part of anyone.
Rotated Component Matrix a Component 1 INSTRUC WELL PREPARED INSTRUC SCHOLARLY GRASP INSTRUCTOR CONFIDENCE INSTRUCTOR FOCUS LECTURES INSTRUCTOR USES CLEAR RELEVANT EXAMPLES COMPARED TO OTHER INSTRUCTORS, THIS INSTRUCTOR IS COMPARED TO OTHER COURSES THIS COURSE WAS INSTRUCTOR ALLOWS ME TO ASK QUESTIONS INSTRUCTOR SENSITIVE TO STUDENTS I AM SATISFIED WITH STUDENT PERFORMANCE EVALUATION INSTRUCTOR IS ACCESSIBLE TO STUDENTS OUTSIDE CLASS INSTRUCTOR AWARE OF STUDENTS UNDERSTANDING .477 .611 .677 .707 .316 .798 .527 .481 .636 .518 .841 .811 2

.761

.697

.670

.462

.777

Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization. a. Rotation converged in 3 iterations.

Rotated Factor Matrix - This table contains the rotated factor loadings (factor pattern matrix), which represent both how the variables are weighted for each factor but also the correlation between the variables and the factor. Because these are correlations, possible values range from -1 to +1. Rotation maximizes the loading of each variable on one of the extracted factors while minimizing the loading on all other factors. This process makes it much clearer which variables relate to which factors. Before rotation 7 variables are significantly loaded on both components but after rotation 5 factors significantly load on both factors. Means it improves the loading of 2 variables on one of the extracted factors.

Component Transformation Matrix Component 1 2 1 .743 -.669 2 .669 .743

Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization.

Factor Transformation Matrix - This is the matrix by which you multiply the unrotated factor matrix to get the rotated factor matrix.

The plot above shows the items (variables) in the rotated factor space. This graph shows how the items (variables) are organized in the common factor space.