Evaluation in

Instructional Design
By: Holly Hicks, Carley Kramer & Eric Whitmer
EME 610
Spring 2016

What is Evaluation?
Evaluation is a systematic process to
determine merit, worth, value or
significance
American Evaluation Association

What are some common types of

evaluation?
Program evaluation

Product evaluation
Policy evaluation
Personnel evaluation

Self evaluation

Why do we conduct evaluations?

To demonstrate the impact or success of
instruction, program, policy, etc.
Identify areas for improvement

Zint, M. Evaluation: What is it and why do it? Retrieved from

http://meera.snre.umich.edu/evaluation-what-it-and-why-do-it.

What are the two types of evaluation

used in instructional design?
Formative

Summative

Used to improve (Markle, 1989)

Used to prove (Markle, 1989)

For revision

For validation (Baker & Alkin, 1973)

(Baker & Alkin, 1973)

For learning (Shute & Kim, 2014)

Of learning

(Shute & Kim, 2014)

What can you learn from Formative Evaluation?

How the instruction could be more effective
How the instruction could be more efficient
How the instruction could be more
interesting/motivating

How the instruction could be more usable

Tessmer, M. (1993). Planning and Conducting Formative Evaluations. London: Kogan Page.

What can you learn from Summative Evaluation?

If the instruction/program was effective

How effective the instruction/program was

Morrison, G. R., Ross, S. M., Kalman, H. K., & Kemp, J. E. (2013). Designing Effective Instruction. New Jersey: Wiley.

Evaluation Tools
Surveys
Pre/Post Tests
Observation

Focus Groups
Interviews
Morrison, G. R., Ross, S. M., Kalman, H. K., & Kemp, J. E. (2013). Designing Effective Instruction. New Jersey: Wiley.

Group Activity

Stufflebeams CIPP Evaluation Model

http://2.bp.blogspot.com/-Q9FQGPUFiV4/TvtE4bBQu5I/AAAAAAAAAB0/aJeH6WdveCw/s320/New+Picture.png

Kirkpatricks Training Evaluation Model

http://www.slideshare.net/Cardet1/mod8-ppt1-080609tmty-1

Rossis Five-Domain Evaluation Model

http://www.sajhrm.co.za/index.php/sajhrm/article/downloadSuppFile/420/761

Brinkerhoffs Success Case Method

http://3.bp.blogspot.com/-MZtW6rKO1LQ/UKQ8CV2UivI/AAAAAAAAAE0/r5vTg2TeJu8/s1600/brinkerhoff%2527s+model+of+success+case+method.gif

Pattons Utilization-Focused
Evaluation (U-FE)

http://archive.ahrq.gov/about/evaluations/mrc/fig%202-1.gif

Models, Models, Models!

Alkin, M. C. & Christie, C. A. An

Evaluation Theory Tree.
Retrieved from
http://www.corwin.com/upmdata/5074_Alkin_Chapter_2.pdf.

Problems with Evaluation

Stakeholders dont see the point
Takes too much time
Costs money
Bias (Internal vs. External)

Real Life: How do Instructional

Designers Evaluate?
Theme 1: designers
evaluate, usually
informally

Williams, D. D., South, J. B., Yanchar, S. C., Wilson, B. G., & Allen, S. (2011). How do instructional designers evaluate? A qualitative study of evaluation in practice.
Educational Technology Research and Development 59(6), 885-907.

Real Life: How do Instructional

Designers Evaluate?
Theme 2: evaluation
of instructional
design training

Williams, D. D., South, J. B., Yanchar, S. C., Wilson, B. G., & Allen, S. (2011). How do instructional designers evaluate? A qualitative study of evaluation in practice. Educational
Technology Research and Development 59(6), 885-907.

Real Life: How do Instructional

Designers Evaluate?
Theme 3: technology
use involves
evaluation

Williams, D. D., South, J. B., Yanchar, S. C., Wilson, B. G., & Allen, S. (2011). How do instructional designers evaluate? A qualitative study of evaluation in practice. Educational
Technology Research and Development 59(6), 885-907.

Real Life: How do Instructional

Designers Evaluate?
Theme 4: designers
evaluate student
learning

Williams, D. D., South, J. B., Yanchar, S. C., Wilson, B. G., & Allen, S. (2011). How do instructional designers evaluate? A qualitative study of evaluation in practice. Educational
Technology Research and Development 59(6), 885-907.

Real Life: How do Instructional

Designers Evaluate?
Theme 5: self and team evaluation

Williams, D. D., South, J. B., Yanchar, S. C., Wilson, B. G., & Allen, S. (2011). How do instructional designers evaluate? A qualitative study of evaluation in practice. Educational
Technology Research and Development 59(6), 885-907.

Real Life: How do Instructional

Designers Evaluate?
Theme 6: design decisions and theory use
involve evaluation

Williams, D. D., South, J. B., Yanchar, S. C., Wilson, B. G., & Allen, S. (2011). How do instructional designers evaluate? A qualitative study of evaluation in practice. Educational
Technology Research and Development 59(6), 885-907.

Real Life: How do Instructional

Designers Evaluate?
Theme 7: evaluating stakeholders needs
Theme 8: evaluating stakeholders priorities
Theme 9: evaluating stakeholders criteria
Williams, D. D., South, J. B., Yanchar, S. C., Wilson, B. G., & Allen, S. (2011). How do instructional designers evaluate? A qualitative study of evaluation in practice. Educational
Technology Research and Development 59(6), 885-907.

Real Life: How do Instructional

Designers Evaluate?
Theme 10: formative,
summative and
developmental
evaluation

Williams, D. D., South, J. B., Yanchar, S. C., Wilson, B. G., & Allen, S. (2011). How do instructional designers evaluate? A qualitative study of evaluation in practice. Educational
Technology Research and Development 59(6), 885-907.

Partner Activity

Ready, Set, EVALUATE!

Please answer the questions HONESTLY!

Want More Information?

American Evaluation Association
Better Evaluation
Association For Talent Development Self-Paced Course: Measuring
Learning Results
Eastern Evaluation Research Society

History of Program Evaluation

References
AEA Administrator. (2014.) American Evaluation Association. What is Evaluation? Retrieved from http://www.eval.org/p/bl/et/blogid=2.

Alkin, M. C. & Christie, C. A. An Evaluation Theory Tree. Retrieved from http://www.corwin.com/upm-data/5074_Alkin_Chapter_2.pdf.

Baker, E. L., & Alkin, U. C. (1973). Formative evaluation of instructional development. Audio Visual Communication Review, 21(4), 389-418.
Johnson, R. B., & Dick, W. (2012). Evaluation in Instructional Design: A Comparison of Evaluation Models. In R. A. Reiser & J. V. Dempsey (eds.), Trends
and Issues in Instructional Design and Technology (96-104). Boston, MA: Pearson.
Markle, S. M. (1989). The ancient history of formative evaluation. Performance and Instruction, 28(7), 27-29.
Morrison, G. R., Ross, S. M., Kalman, H. K., & Kemp, J. E. (2013). Designing Effective Instruction. New Jersey: Wiley.
Rossi, P.H., Lipsey, M. W., & Freeman, H. E. (2004). Evaluation: A Systematic Approach. Thousand Oaks, CA: Sage Publications.
Shute, V. J., & Kim, Y. J. (2014). Formative and Stealth Assessment. Handbook of Research on Educational Communications and Technology (pp.311320). New York: Springer Science+Business Media.
Tessmer, M. (1993). Planning and Conducting Formative Evaluations. London: Kogan Page.
Williams, D. D., South, J. B., Yanchar, S. C., Wilson, B. G., & Allen, S. (2011). How do instructional designers evaluate? A qualitative study of evaluation
in practice. Educational Technology Research and Development 59(6), 885-907.
Zint, M. Evaluation: What is it and why do it? Retrieved from http://meera.snre.umich.edu/evaluation-what-it-and-why-do-it.

Picture Links (in order of appearance)

http://www.30hands.net/images/main_site/RaisedHands.png

https://store.ragan.com/ProductDetails.asp?product=Y1Q4DS&cal=yes&listshow=Workshops&catid=652C00F2324445739342D4B96E80F042&grfr=Ye
s

https://pixabay.com/static/uploads/photo/2014/04/02/16/18/toolbox-306888_960_720.png

https://commons.wikimedia.org/wiki/File:Addie-evaluation.png

https://www.flickr.com/photos/notbrucelee/8616623320

https://www.flickr.com/photos/ekivemark/7843824274/in/photostream/

http://zhtech.com/

https://en.wikipedia.org/wiki/Tablet_computer

https://www.flickr.com/photos/albertogp123/5843577306

https://pixabay.com/en/people-business-corporate-workers-335298/

https://commons.wikimedia.org/wiki/File:Dick_Carey.png

http://previews.123rf.com/images/whitetag/whitetag1310/whitetag131002717/23031683-Mid-adult-man-holding-steak-Stock-Photo.jpg

https://pixabay.com/en/clipboard-green-check-list-311168/

https://www.flickr.com/photos/dmswart/6507071627

https://pixabay.com/static/uploads/photo/2015/07/09/12/44/smile-837661_960_720.jpg