Proposal for Evaluation of

Determining Instructional
Purposes Training Program
Submitted to:

Far West Laboratory for

Educational and Research
Development
Submitted by:

Western Pennsylvania
Evaluation Group (WPEG)

October 27, 2015

Introduction
October 2015, Far West Laboratory for Educational & Research Development (FWL), released
a request for proposal (RFP) for evaluation of its Determining Instructional Purposes (DIP)
training program. The following document, from Western Pennsylvania Evaluation Group
(WPEG) of Pittsburgh, Pennsylvania, is being submitted in response to the Far West Laboratory
RFP. Western Pennsylvania Evaluation Group provides professional evaluation and
consultation services for organizations with areas of expertise in planning, evaluation, data
analysis, and market research. WPEG is offering our services to assist FWLs decision-making
process with regards to determining whether or not resources should be allocated to market the
DIP training program package. Additionally, WPEG aims to provide useful information to
school administrators seeking to make decision on the purchase and use of DIP training units.

Description of Program
The Determining Instructional Purposes (DIP) training package, developed by FWL, is designed
to prepare school administrators and graduate students in educational administration in
effectively planning school programming. DIP contains three units each of which include fourto-six modules focusing on the following topics: Unit 1-Setting Goals, Unit 2- Analyzing
Problems, and Unit 3- Deriving Objectives. The units, ranging in length from 155 to 259 pages,
are intended to be self-contained allowing for use of either one unit or in combination of three
units. FWL estimates required training time for Unit 1 and 3 as 10-15 hours with Unit 2 at 12-18
hours. Materials for the units, available exclusively in print form, are available from FWL at a
cost of $8.95 per single unit or $24.95 for a set of all three units.
Training courses conducted with unit materials, as per request by program developers, should be
overseen by a coordinator that is not required to have prior knowledge of the content area but
should have gone through the units either individually or as a part of a training group. A
Coordinators handbook is included in the training package and contains directions for
coordinating all three units. This handbook is presented entirely in print form and priced at
$4.50 per copy.

Evaluation Method
The primary purpose of Western Pennsylvania Evaluation Groups (WPEG) evaluation is to
determine whether Far West Laboratory (FWL) should commit money and resources to continue
to market the Determining Instructional Purposes (DIP) training package. Overall, FWL is
interested in making decisions regarding the future use of DIP and the need to keep the package
as a part of FWL business expansion. To aid in FWLs decision-making process, WPEG intends
to evaluate the quality and effectiveness of the DIP training package in providing school
administrators and graduate students in educational administration the necessary skills to plan
effective school programs.
In order to achieve this, WPEG is proposing the use of multiple data sources to evaluate DIP
content, delivery of training, and need. Through use of both formative and summative
evaluations, WPEG is looking to assess 1) value of content and activities within the modules in

meeting instructional objectives, 2) effectiveness of different delivery modes offered, 3) level of

need for the training package. Qualitative and quantitative findings generated from the
evaluation will be reported to stakeholders at FWL and other individuals as indicated by FWL.
Facilitated Training Groups
To assess the value of content and activities within the modules and effectiveness of the different
delivery modes, four training groups will be utilized. Training groups will consist of 8-10
participants consisting of school administration graduate students from universities as well as
school administrators from schools districts located within and surrounding San Francisco,
California. Each group will be placed in one of four training workshops which are as follows: 1)
Unit 1 comprising of two, six-hour sessions, 2) Unit 2 including four, three-hour sessions, 3)
Unit 3 containing three, four-hour sessions, 4) Units 1, 2, & 3 involving five, seven-hour
sessions.
When sessions are completed, participants will complete reaction surveys regarding session
content and activities. Running this formative assessment will provide specific information
which will allow FWL to improve the training for future trainees. Observations by WPEG will
be conducted during using the Best Practices for Training Rubric and will occur during one
session per training group. The rubric will be used to assess delivery of the training and trainer
preparedness. Upon completion of each module, participant mastery of content knowledge will
be assessed through completion of a skills assessment questionnaire which will be designed with
the assistance of subject-matter experts and designers of DIP. These questionnaires will be
administered by training coordinators.
Participants and training coordinators will fill out online exit surveys at the completion of the
entire training workshop to provide feedback regarding content and activities housed within the
modules as well as delivery mode for the unit. The coordinators survey will also cover the topic
of implementation. Coordinators will also be interviewed to gain a more detailed narrative of the
DIP training package. Summative evaluations will include surveys focusing on the
implementation of DIP skills to plan effective school programs.
Costumer Analysis
Due to FWL seeking to expand the business and being in the decision-making process to
determine whether or not the DIP is worthy of monetary investment, it is important to better
understand its marketability. To do this, WPEG plans to perform a customer analysis to gauge
the level of need for the training through use of a product needs assessments. WPEG will
conduct market research and interview FWL marketing personnel on the topics of demographic
makeup, buying habits, and other characteristics of the target market. Once market research is
completed, WPEG will administer needs assessment questionnaires to school administrators as
well as other target audience members revealed by the market research. This questionnaire will
be used to identify and weigh the relative importance of factors influencing decisions to purchase
DIP and its usefulness to target consumers.

Proposed Task Schedule

Description of Task
To establish an Evaluators Program Description (EPD), WPEG will meet
with FWL stakeholders, via video conference, to review DIP program
objectives, activities, and current evaluation materials/methods.

Expected
Completion Date
February 1, 2016

WPEG will develop reaction and exit surveys (used upon completion of
each training session and at completion of entire workshop) as well as skills
assessment questionnaires (used after each module).

March 7, 2016

WPEG will create questions for the coordinators exit interviews.

March 14, 2016

WPEG will meet with FWL management via videoconference to review

March 31, 2016
surveys, questionnaires, and interview questions and complete the necessary
revisions.
WPEG will conduct market research, phone or video conference interviews
with FWL marketing personnel, and create customer needs assessment
(CNA) questionnaire.

April 4, 2016

WPEG will administer the CNA questionnaire to local school

administrators and target audience members determined by FWL marketing
personnel and market research.

May 4, 2016

Start of four training workshops.

June 13, 2016

WPEG will conduct observations of sessions using the Best Practices for
Training rubric.

June 2016

Completion of final Group 4 (Unit 1, 2, & 3) training workshop. With

groups 1, 2, & 3 finishing prior to this date.

July 18, 2016

WPEG will perform coordinator exit interviews via phone or video

conference.

July 25, 2016

WPEG will organize and analyze observation, survey, and interview data
collected during implementation of the training workshops as well as draft
the final evaluation report and submit to FWL.

August 2016

WPEG will meet with FWL stakeholders via video conference to provide
an oral presentation on their findings, answer questions, and offer
recommendations regarding next steps.

September 8, 2016

Project Personnel
Carter Barnett: Senior Evaluation Specialist
Having worked as a K-12 educator for 8 years prior to becoming interested in research and
development, Mr. Barnett has over 15 years experience conducting evaluations. Working in
partnership with state and federal agencies, non-profit organizations, universities, and K-12
educational settings he has served as Senior Evaluator for WPEG for 10 years. Mr. Barnett has
extensive experience in designing and implementing qualitative and quantitative research
methods as well as the ability to quickly and efficiently analyze content.
Heather Arenth: Research and Evaluation Assistant
Ms. Arenth, a graduate student in the Masters program in the Department of Educational
Technology at Boise State University, holds a bachelors degree in adult education and training
with a minor in statistics. She has been working with WPEG for 3 years as an Evaluation
Assistant. Ms. Arenth has broad experience designing and implementing a variety of different
surveys and questionnaires. Due to her ability to build good working relationships through
excellent communication and interpersonal skills, Ms. Arenth is now being utilized more to
conduct focus groups and interviews. Ms. Arenths background in statistics as well as her
passion to produce user friendly data products and reports has allowed WPEG to also offer her
skills to also assist the Data Analysis team. Over the past 3 years, Ms. Arenth has improved her
ability to collect, organize, analyze, and disseminate information with great detail and accuracy.
Joel Cigfox: Data Analyst
Mr. Cigfox has an extraordinary ability to turn data into information that clients are able to
insightfully use to make valuable decisions. Mr. Cigfox is new to WPEG but with his 20+ years
in the field of data analysis and an educational background including a Masters in Data Analysis
and Statistics from Carnegie Mellon University he is a welcomed addition to the WPEG team.
Over the past 20 years, Mr. Cigfox has skillfully developed and implemented data collection
systems and strategies that have helped many organizations, both for profit and non-profit,
optimize statistical efficiency and data quality. His proven work experience as a data analyst
has been displayed with his strong knowledge of and experience with reporting packages,
databases, and programming along with his strong analytical skills.
Adelyn Tyler-Paxton: Market Research Analyst
For the past 5 years, Mrs. Tyler-Paxton has served as Chief Market Research Analyst at WPEG.
She holds a Masters in marketing with a Bachelors degree statistics both of which were received
from the University of Pittsburgh. Prior to working with WPEG Mrs. Tyler-Paxton worked with
PNC Financial Services Group, Inc. for 17 years as a Marketing Manager. Her work as a
Marketing Manager allowed her to better understand business objectives and the importance of
remaining fully informed on marketing trends and competitors marketing research. Mrs. TylerPaxton duties with WPEG include interpretation of data, formulation of reports, and
consultation. Her ability to compile and analyze statistical market data and provide competitive
analyses on companies market offerings has proven extremely beneficial to WPEGs clientele.

Budget
Personnel
Professional Salaries:
Carter Barnett: 50 days at $375/day

$18,750

Heather Arenth: 45 days at $275/day

$12,375

Joel Cigfox: 15 days at $375/day

Adelyn Tyler-Paxton: 20 days at $325/day

$5,625
$6,500

TOTAL PERSONNEL

$43,250

Travel and Per Diem

One 4-day round trip: Pittsburgh to San Francisco
(includes per diem)

$850

Lodging

$600

Four day car rental for session observations

$400

Mileage estimated at an average of 75 miles/day

$170

TOTAL TRAVEL AND PER DIEM

$2,020

Supplies and Materials

Supplies and Photocopying

$650

Materials (includes 3 coordinators handbooks, 10 three unit

packages, and 30 single unit packages)
Video Conferencing Services: Join.me conference services
(includes cost for FWL and WPEG at $15 per user total
$30/month for 6 months)
TOTAL SUPPLIES AND MATERIALS
TOTAL BUDGET

$532

$180

$1,362
$46,632