You are on page 1of 120

Diploma in

Adult and Continuing Education


(DACE)

Competency Unit 4:
Plan a Training Needs Analysis
TR-HRD-401C-1

Learner Guide

2010
Version No: 1.0
Version Control Record

Version Effective Date Changes Author


1.0 1 April 2010 N.A Louise Gilbert

Copyright 2010, Institute for Adult Learning. All rights reserved. This document is provided for the
explicit use and guidance of learners of Diploma in Adult and Continuing Education (DACE) programme
with the Institute for Adult Learning (IAL). Any other use of this document or parts thereof, including
reproduction, publication, distribution, transmission, re-transmission or public showing, or storage
in a retrieval system in any form, electronic or otherwise, for purposes other than that expressly
stated above without the express permission of WDA is strictly prohibited.

For enquiries and feedback about this competency unit, please email info@ial.edu.sg

2010
Version No: 1.0
TABLE OF CONTENTS

Overview of Module 4
Performance standards
Using the Learner Guide
Self assessment

1 Scope and requirements of a TNA 11


Performance improvement
Purposes
A useful framework
Approaches
Typical TNA phases or steps

2 Prepare for a TNA 29


Client request
Project plan
Data collection types and sources of data

3 Methods to collect data 47


Qualitative data collection
Quantitative data collection

4 Methods to analyse data 63


Descriptive statistics
Findings

5 Prepare a TNA report 73


Elements of a report
Training and non training recommendations
Ethical issues and standards
Common errors and obstacles
Trends and developments

6 References 93

7 Resources 95
1. Competency Standard: Plan a Training Needs Analysis
2. Generic questions 97
3. Case Study Introduction 99
4. Data Collection Planning Guide 101
5. Learning Review 103
6. The Program Evaluation Standards A Summary 109
7. The Three Step Strategy by Charles Jennings 111
8. Post-course self assessment 115
9. TNA Competency survey 116

8 Glossary 117

9 Assessment Requirements 119

2010 Institute for Adult Learning, Singapore 3


Version No: 1.0
2010 Institute for Adult Learning, Singapore 4
Version No: 1.0
Overview
CU 4: Plan a Training Needs Analysis

Welcome

Welcome to Plan a Training Needs Analysis. It is one of the seven core modules of
the Diploma in Adult and Continuing Education program (DACE) and is a 2-credit
value module of 21 hours class contact time.

The Module is based on five key Performance Statements and Underpinning


Knowledge which are set out below.

Performance Statements

PS1
Establish the scope and requirements of a Training Needs Analysis to meet
stakeholders needs.

PS2
Prepare for Training Needs Analysis using appropriate information.

PS3
Select appropriate methods to collect data for Training Needs Analysis.

PS4
Select appropriate methods to analyse collected data

PS5
Prepare a Training Needs Analysis report

Underpinning Knowledge
1 Organisation objectives and business needs
2 Concept of performance improvement
3 Scope and process of TNA
4 Types and sources of information in organisation
5 Methods of data collection
6 Methods of data analysis
7 Ethics in the use of data
8 Trends and developments in TNA

2010 Institute for Adult Learning, Singapore 5


Version No: 1.0
Overview

Introduction

Needs analysis is a basis for training design, delivery and evaluation. It


identifies how training can help an organisation reach its goals and
objectives. Most experts agree that effective learning, training and
performance improvement initiatives should start with a needs
assessment.

This module starts by examining the scope and requirements of needs


analysis, the concept of performance improvement and the key phases of
the TNA process. Participants will work through the module topics using
a framework which aligns training needs to business, performance,
learning and learner needs to ensure both training and non training
solutions are considered.

A progressive case study starts with a client request and provides the
context for learners to apply new knowledge and skills in relation to
preparing a TNA project plan, data collection and analysis and writing a
TNA report. Ethical issues, common obstacles and trends and
developments in needs analysis are examined.

Our Approach

Reflective practice is a unique feature of the DACE programme.

As a reflective practitioner, you are expected to

1. make regular entries in your journal and/or log into other forums
to record how you think, act and feel about eg what is happening
in your modules, what topics are being covered, your opinions
and those of peers, what you have learned, how you can apply
what you have learned in your work, how your beliefs are being
challenged, your feelings about this learning journey, what has
surprised or disappointed you, helped you grow, been a barrier,
motivated you.
2. set, review personal learning goals on a continuous basis as you
progress through the program
3. assemble a portfolio of your work by eg including relevant
samples of work or evidences of your competence which reflect
your personal and professional growth

2010 Institute for Adult Learning, Singapore 6


Version No: 1.0
Overview

Making the most of this learning experience

This module has been designed using sound adult learning principles. To
gain the maximum benefit from this experience you are encouraged to

Actively participate in the learning process


Take responsibility for your own learning
Be sensitive to the learning needs of others
Keep an open mind
Maintain confidentiality for a safe learning climate
Value feedback
Review your learning
Work with each learner during the course


Summary of Assessment Requirements

Specific information is outlined at the end of this Learner Guide.

Summary of Assessment

Assessment session Duration Date

Written Assignment --- Due:

Written Assessment 90 mins

Total
90 mins

2010 Institute for Adult Learning, Singapore 7


Version No: 1.0
Using this Learner Guide
Each performance statement is set out as a specific section of this Learner Guide which
has been designed as a self-contained, competency-based learning package. Additional
materials are located in the Resources section and supplementary learning activities and
readings are included at the end of each section. Other materials will be provided during
the course to support your learning.

The Guide contains the following components:

Learning Outcome
At the beginning of each section, the specific learning
outcomes which you are expected to achieve are
outlined. It is useful for you to review your learning
against this checklist in preparation for the assessment.

Learning Activity

A Learning Activity provides an opportunity for you to apply the


new knowledge to practical situations to make the learning more
meaningful and relevant. You will be involved in small and whole
group discussions.

Reflective Point
A Reflective Point offers questions that are intended to guide
you towards drawing out your own experiences and linking
these to the learning points. Time spent on the reflection
questions will help you surface your personal values, beliefs
and assumptions about your professional practice as an Adult
Educator. You are strongly encouraged to do the exercises as
your discoveries will serve as useful contributions for the
reflective portfolio component of the assessment.

2010 Institute for Adult Learning, Singapore 8


Version No: 1.0
Pre-course SELF assessment
for Plan a Training Needs Analysis

Please rank the following learning outcomes twice on a scale of 1-5.

The first ranking is a self-assessment regarding your level of knowledge and skill
(1= I dont know very much about this subject, 5 = I am expert in this subject)

The second ranking is the importance of each learning outcome to you, based on your job
responsibilities (where 1 = not important at all, 5 = critical).
A B
Learning Outcomes My current Importance to
knowledge me
I am able to (1-5) (1-5)

PS1: scope and requirements of TNA 1 2 3 4 5 1 2 3 4 5


1. define TNA within a framework of performance
improvement
2. explain purposes of TNA
3. describe four approaches to TNA
4. outline phases or steps in a typical TNA process
PS2: prepare for TNA
5. align a training request to business, performance,
learning and learner needs
6. prepare a TNA project plan
7. plan for data collection
8. identify types of data
9. describe a range of data sources
PS3: select data collection methods
10. select appropriate qualitative data collection methods
11. select appropriate quantitative data collection methods
12. identify factors influencing selection
13. discuss guidelines for implementing data collection
PS4: select data analysis methods
14. apply basic descriptive statistics to quantitative data
15. review findings resulting from data analysis
16. explain the difference between findings and
recommendations
PS5: prepare a TNA report
17. write a TNA report containing key information
18. examine ethical issues and standards
19. discuss some common obstacles and errors
20. describe trends and developments in TNA
Totals

2010 Institute for Adult Learning, Singapore 9


Version No: 1.0
Notes

2010 Institute for Adult Learning, Singapore 10


Version No: 1.0
1 Scope and requirements of a
Training Needs Analysis

1. What is a TNA?
Scope and 2. Purpose
requirements 3. Performance improvement
of a TNA 4. Approaches
5. Phases in a TNA process

Prepare for a TNA

Data collection
methods

Data analysis
methods

Prepare a TNA
report

Most experts agree that effective learning, training and performance improvement
initiatives should start with a needs assessment. The section begins the process of
examining what a TNA is by discussing definitions, approaches and phases of a
training needs analysis within a framework of performance improvement.

Learning Outcomes

1. define TNA within a framework of performance improvement


2. explain the purposes of TNA
3. describe four approaches to TNA
4. outline phases in a typical TNA process

2010 Institute for Adult Learning, Singapore 11


Version No: 1.0
Scope and requirements of a Training Needs Analysis

The Analysis Process

The needs assessment / analysis process is one of the most critical


in training and development. Most experts agree that effective
learning, training and performance improvement initiatives should
start with a needs assessment. It is an important step in the
performance improvement business. Training needs in an
organisation must be viewed in the context of the key results that the
organisation is seeking and be part of the overall strategy for
performance improvement.

Terms used

The broadening of the scope of the analysis process from identifying .


knowledge and skills to identifying the needs of the organisation, is
reflected in the increasing use of the preferred term needs
assessment over training needs analysis. Many authors quoted in
this guide use needs assessment in their description of the process.

In this module, Training Needs Analysis (TNA) is used to mean the


same thing as needs assessment. It is used as a general term for a
planning process to collect information, analyse it and create a plan
for future action. The acronym TNA has been used by training and
learning professionals for many years and this module follows that
tradition, without assuming that solutions will be achieved only
through formal training.

The approach taken in this module, and reflected in the design,


delivery and assessment of the learning has a complete focus on
business, performance, learning and learners needs. These four
stages provide a framework within which the process of training
needs analysis can be explored.

TNA defined

There is a wide variety of opinion, and some confusion, as to what


constitutes a training needs analysis and the best ways to conduct
one. This is partly because so many methods, techniques, steps and
strategies could arguably be defined as constituting a training needs
analysis (TNA).

2010 Institute for Adult Learning, Singapore 12


Version No: 1.0
Scope and requirements of a Training Needs Analysis

TNA defined (cont)

Stone (2009: p46) says needs assessment is the most important


part of the training and performance process because it identifies
the root cause that must be addressed by a training or non-
training intervention

Brown (2002: p569) defines training needs assessment as an


ongoing process of gathering data to determine what training
needs exist so that training can be developed to help the
organisation accomplish its objectives.

Gupta et al (2007: p 14) describe a needs assessment as a


process for figuring out how to close a learning or performance
gap. It involves determining what the important needs are and
how to address them.

The CIPD (2010) refers to the process as a training and learning


needs analysis (TLNA) and describes it as a health check on the
skills, talents and capabilities of the organisation (p1) Retrieved on
7 April 2010
http://www.cipd.co.uk/subjects/lrnanddev/trainingneeds/idtlneeds.
htm

What can initiate a needs assessment?

Phillips (1999) outlines several events or situations that can initiate


a needs assessment::

New employees enter the organisation


The organisation implements new equipment, processes
and procedures
New jobs are created, or new responsibilities are developed
for current jobs
The organisation undertakes major change reforms
Management requests a needs assessment or a program
that leads to a needs assessment
Skills are upgraded, improved or planned for improvement
Significant opportunity for performance improvement exists,
or a performance problem has been identified.

2010 Institute for Adult Learning, Singapore 13


Version No: 1.0
Scope and requirements of a Training Needs Analysis

Performance Improvement

Traditionally, TNA has focused solely on identifying the needs


relevant to a planned intervention. This has meant focusing only on
data collection and analysis relevant to the content or the method of
training. The emphasis was on identifying and quantifying skills to
be learned in the training course, learning activities and learner
needs. Learning was the focus.

In todays competitive global environment, a learning focus is not


sufficient. A more comprehensive approach which goes beyond
training needs is required. The concept of performance
improvement contributes to this critical focus that training must be
aligned to business results.

Piskurich (2002) outlines some critical components of


performance improvement including:

a focus on business and organisational goals


use of a systematic approach based on desired results
focus on performance accomplishments and performance
linkage of performance analysis to the job
analysis of performance gaps
identification of interventions that will close the
performance gaps

Tobey (2005) describes three types of interventions related to


performance improvement:

1. motivational: e.g. adding incentives, rewards to influence


performers so they choose to perform to standards

2. structure/process: e.g. making changes in organisational


structure, reporting relationships, work processes and
procedures to influence job performance and results

3. knowledge and skills (training): providing knowledge and skills


to performers to improve performance and results

2010 Institute for Adult Learning, Singapore 14


Version No: 1.0
Scope and requirements of a Training Needs Analysis

A training program adds more value to the organisation when it is


focused towards performance improvement and organisational
results and when all factors that affect performance and training
effectiveness are considered.

According to Tobey (2005), the key to a successful training needs


analysis is to establish there is

a business need,
driving a performance need
identifying the specifics regarding the desired training,
and
identifying the non-training issues (that affect the
performance situation)

2010 Institute for Adult Learning, Singapore 15


Version No: 1.0
Scope and requirements of a Training Needs Analysis

Features of Needs Analysis

One way to scope the features of needs analysis is to examine the similarities
and differences of training requests that would lead to needs analysis. Gupta et
al (2007) describe the following similarities:

1. Dissatisfaction with the current situation and desire for change i.e. a gap
exists between what is and what could/should be.

2. Focus on addressing current issues or on specifying future learning or


performance needs such as
solving a current problem
avoiding a past or current problem
creating or taking advantage of a future opportunity
providing learning, development and growth

3. Imply a solution that requires training, learning, performance improvement or


a combination of these.

4. Include little evidence and few clues about whether taking the requested
action will improve learning or performance. Information about the situation
and course of action are unknown. Most requests that lead to needs
assessment include fuzzy goals, incompatible beliefs, flawed assumptions,
large leaps in logic, little diagnostic information e.g. about what caused the
current condition and what is needed to make the required changes or what
support is needed.

5. Include challenging questions, usually for which right answers cannot be


found in a book or on the internet. Requests have to be evaluated carefully.
Many factors internal to the organisation including the political context, the
culture, the organisations mission, vision, strategic goals and leadership can
impact on how the challenging questions raised by a needs assessment are
answered. Factors external to the organisation like customer expectations,
competition, legal requirements, technology and global environment also
impact on how the questions are answered.

The following differences were noted:

1. Target different levels of learning and performance eg individuals, teams,


functional units, whole organisations

2. Focus on various aspects of learning and performance


a gain in knowledge and skills for the individual, and for the
organisation e.g. databases, culture
an improvement in performance processes, accomplishments and
capacity for future performance

2010 Institute for Adult Learning, Singapore 16


Version No: 1.0
Scope and requirements of a Training Needs Analysis

Purposes of TNA

When implemented effectively, training needs assessment serves


multiple purposes according to Tobey (2005) who describes the
following as key reasons for doing a TNA.

1. It places the clients need or request in the context of the


organisations needs. Training only adds value when it is relevant
and ultimately serves a business need.

2. It validates and supports the initial issues presented by the client.


While the client knows their business, it is often difficult for the client
to know the cause or the solution for issues related to performance.
A needs analysis can reveal new information, offer a different
perspective on the issue and set the initial request in a broader
context.

3. It ensures the final training design supports employee performance


and helps the organisation meet its needs.

4. It results in recommendations regarding non-training issues that


affect achievement. Its important the training function is held
accountable for only the portion of the business and performance
needs it can affect. Identifying non-training issues adds value to the
training function.

5. It increases the chances of survival of the training function by


adding value for its results and impact.

6. It sets the foundation for evaluation.

2010 Institute for Adult Learning, Singapore 17


Version No: 1.0
Scope and requirements of a Training Needs Analysis

Benefits of TNA

Babazette (2006) describes the value of a needs assessment as


protecting the assets of an organisation and ensuring targeted
resources are used solely for the training issues. To conduct training
without having determined a legitimate training need is a waste of
valuable time and resources.

Gupta et al (2007) outline a number of specific benefits of needs


assessment as follows:

1. it provides a framework for planning and action to improve


learning, training and performance
2. It aligns resources with strategy
3. It builds relationships with those who have a stake in the
situation
4. It sets goals for future action
5. It clarifies problems or opportunities
6. It provides data for decision making
7. It identifies resources for making changes
8. It prioritizes actions
9. It determines who must be involved to maximise success
10. It builds support for learning and performance personnel
11. It engages stakeholders
12. It contributes to practical, credible, relevant and appropriate
solutions for the situation

2010 Institute for Adult Learning, Singapore 18


Version No: 1.0
Scope and requirements of a Training Needs Analysis

A Useful Framework

A Training Needs Analysis (TNA) can focus on one or more


individuals, on people within units or teams, on people across job
functions, or on an entire organisation. It can involve staff from all
levels of an organisation. It can identify organisational, operational
and individual needs.

Stages

Tobey (2005) outlines four stages which provide a useful framework


for conducting a TNA in the context of performance improvement.

Business
Performance
Learning
Learners

In responding to a training request, the training professional begins


with the business needs and context, focuses on the performance
that is required to meet the business needs within that context and
then re-focuses on the learning that must take place to support the
required performance.

The result of this approach is that the needs analysis study leads to
training recommendations and non training interventions, all of
which will support job performance and organisation goals.

The TNA approach used in this module, and reflected in the design,
delivery and evaluation of the learning has a focus on business,
performance, learning and learners.

2010 Institute for Adult Learning, Singapore 19


Version No: 1.0
Scope and requirements of a Training Needs Analysis

Approaches

Gupta et al (2007) propose four needs assessment approaches which can be used
separately or in combination:

Approach Purpose

Strategic to examine the internal and external factors that affect


performance within the context of an organisations or the
communitys business strategy
to identify the gaps between the current and desired conditions.
to identify long term solutions and solve problems that affect core
business processes.

Competency to identify the knowledge, skills, attitudes and behaviours the


-Based performer needs to excel in a job.
to measure proficiency of people, determine qualities that
distinguish average from superior performance
to develop standardised training.

Job and to gather information about the scope, responsibilities and tasks
Task for a particular job function or functions.
Analysis to guide the development of new job descriptions and the revision
of existing positions. Position descriptions help link job
requirements to training needs.

Knowledge to identify the knowledge and skills that people must possess in
and Skills order to perform effectively
to prescribe appropriate interventions to close the knowledge and
skills gaps.

Usually only those gaps caused by lack of knowledge and skills can
be improved through training. Performance deficiencies that occur as
a result of lack of motivation or organisational structure and processes
require non-training interventions.

2010 Institute for Adult Learning, Singapore 20


Version No: 1.0
Scope and requirements of a Training Needs Analysis

The TNA Process

Authors, practitioners and researchers describe the process of


needs assessment in various ways including training needs analysis,
performance analysis, gap analysis and front-end analysis. While
each of these has somewhat different processes, they have more
similarities than differences.

Approaches, strategies, methods and terms used may vary but the
intended end result of each process is about identifying performance
and learning issues and recommending feasible solutions.

In thinking through how to plan and conduct a structured needs


analysis, it is important to remember there is no single right way to
conduct a TNA. Rather there are a number of procedures that can
be followed. Typically it involves a series of steps or phases which
are ideally followed in sequence and a planning process to collect
information, analyse it and create a plan for action based on
appropriate solutions.

Three Step Process

Gupta, K. (1999) describes a three step analysis process:

1. pinpoint the problem by meeting with the client to discuss


issues and identify stakeholders

2. confirm the problem by interviewing stakeholders and


assessing the scope of the problem

3. seek solutions by identifying possible actions and gaining


agreement on an action plan which becomes the basis of a
TNA report.

Five key phases


Gupta, Sleezer and Russ-Eft (2007) outline five key phases.
1. Gather preliminary data
2. Plan
3. Perform training requirements analysis
develop tools
collect data
4. Analyse data
5. Prepare and present a report

2010 Institute for Adult Learning, Singapore 21


Version No: 1.0
Scope and requirements of a Training Needs Analysis

Seven Step Process

Tobey (2005) outlines seven steps of a TNA process as follows

Step 1 Conduct external and organisation scan


Step 2 Collect data to identify business needs
Step 3 Identify potential training intervention
Step 4 Collect data to identify performance, learning and learner needs
Step 5 Analyse data
Step 6 Deliver data analysis feedback/report
Step 7 Transition step: Begin training design

Five stage process

Cotton (2005) proposes a five stage process of needs analysis which


includes

1. Identify stakeholders
2. Determine needs
Business objectives
Performance needs
Identify the attitudes, skills and knowledge (ASK) to
support the first two needs
3. Agreed needs, critical success factors and constraints
4. Develop recommendations
5. Produce report including communication and evaluation
strategies

Whatever TNA process is followed, it is important that it focuses on


the needs of the business, performance, learning and the learners.

2010 Institute for Adult Learning, Singapore 22


Version No: 1.0
Scope and requirements of a Training Needs Analysis

Learning Activity

How would you respond to this request?

You are the in-house training officer in your organisation. You are taking a
coffee break when your department manager joins you at the table.

After a brief chat, the manager says,

Im not happy with the latest customer satisfaction results in the


call centre. Over the last three months they have steadily declined.
Im worried. The phone representatives need to be more
responsive to the customers. Would you set up a telephone skills
training session next week?

How would you respond to this request? (Write below)

2010 Institute for Adult Learning, Singapore 23


Version No: 1.0
Scope and requirements of a Training Needs Analysis

Repeat Exercise

How should you respond to this request?

You are the in-house training officer in your organisation. You are taking a coffee
break when your department manager joins you at the table.

After a brief chat, the manager says,

Im not happy with the latest customer satisfaction results in the


call centre. Over the last three months they have steadily declined.
Im worried. The phone representatives need to be more
responsive to the customers. Would you set up a telephone skills
training session next week?

How should you respond to this request? (Write below)

2010 Institute for Adult Learning, Singapore 24


Version No: 1.0
Scope and requirements of a Training Needs Analysis

A Reflective Point

Answer yes or no for each statement. Add


other statements to the list that you think reflect
your general business awareness.

Statement Yes No Steps I can take to


learn more
I can describe at least three of our competitors
and state their competitive advantage.
I am aware who customers perceive as best in
class in our industry sector.
I can describe the economic, social, political and
technological influences that affect our industry.
I am aware of future customer needs and
expectations.
I can state the financial performance of our
organisation and how this has altered over the
past three years.
I can read a profit and loss statement.

I can describe the culture of our organisation.

I can state the strengths and weaknesses of our


organisation.
I am aware of future risks facing our organisation.

I am aware of opportunities that we can take in


the future as an organisation.
I look at a range of options before making a
decision.
I make decisions in a logical fashion taking into
account cause and effect.

2010 Institute for Adult Learning, Singapore 25


Version No: 1.0
Scope and requirements of a Training Needs Analysis

A Reflective Point

1. What experiences have you had in relation to training needs analysis?

2. What was the purpose of the TNA?

3. What approach/es were used?

4. What steps were followed?

5. What was the outcome of the TNA?

6. How might the TNA have been done differently?

2010 Institute for Adult Learning, Singapore 26


Version No: 1.0
Scope and requirements of a Training Needs Analysis

Additional Reading
Linking TNA to Evaluation

Phillips (1999), Cotton (2004) and Tobey (2005) highlight a clear


relationship between needs analysis and evaluation using Kirkpatricks
four levels.

The four stages outlined by Tobey (2005) at the front-end of the


analysis have a direct link to the evaluation or back-end measurement.
During training needs analysis, measures are taken at each of the four
stages.

Needs Programme Evaluation


Assessment Objectives (Levels)
(Levels/Stages)

business needs Business impact? 4


Impact
objectives Did it a make a difference to
the organisation?

Job performance Job application? 3


Application
needs objectives Did they use it?

learning needs Knowledge and skill 2


Learning learning?
objectives
Did they learn it?

learner needs Learner reaction? 1


Satisfaction
Did they like it?
objectives

Training
recommendations

Non-training Training design


recommendations and delivery

Adapted from Phillips (1999), Tobey (2005) and Cotton (2004)

2010 Institute for Adult Learning, Singapore 27


Version No: 1.0
Scope and requirements of a Training Needs Analysis

A Reflective Point

Rate the importance of each of the TNA purposes identified by Tobey (2005), as it
applies in your organisation on a scale from low to high.

Purpose Importance in my What could you do to change/improve your


(number) organisation senior managers (clients) perception about
each purpose?

1.
low medium high
2.
low medium high
3.
low medium high
4.
low medium high
5.
low medium high
6.
low medium high

2010 Institute for Adult Learning, Singapore 28


Version No: 1.0
2 Prepare for a TNA

Scope and
requirements
of a TNA
1. the client request
Prepare for 2. the value of a project plan
a TNA 3. planning for data collection
4. types of data
5. data sources

Data collection
methods

Data analysis
methods

Prepare a
TNA Report

This section outlines the importance of aligning training needs to the business,
performance, learning and learner needs of the organisation. It examines how to
handle a client request, the value of a project plan and the steps involved in planning
for data collection. A case study guides learning.

Learning Outcomes

1. align training request to performance improvement


2. prepare a TNA project plan
3. plan for data collection
4. identify types of data
5. describe a range of data sources

2010 Institute for Adult Learning, Singapore 29


Version No: 1.0
Prepare for a TNA

The training function should consider two role responsibilities for needs
analysis. Stone (2009) outlines these as

1. the proactive role


2. the client request role

Firstly, in a proactive approach, the trainer has the responsibility to


know what is happening by consistently and constantly scanning the
organisation for indicators that business needs, performance needs or
both could be addressed by a training-related intervention. In a perfect
world, trainers would then approach the potential client/sponsor with
the information gathered and propose an intervention.

The second responsibility, the client request role is where, for


example, the department manager can call on the training
department when it has special needs.

The proactive context

In the proactive role, the trainer must stay in touch with the current
status of the organisation. Stone (2009) provides a starter list of
some of the elements to follow:

Goals and strategies of the organisation,


Current issues,
Systems and programs in place to hire and retain a competent and
satisfied workforce,
Staffing requirements and replacement plans,
Grievance and employee satisfaction issues,
Preferred management style,
Thought leaders in the organisation,
Access to results from organisation surveys e.g. exit interviews,
employee satisfaction surveys,
Competitor intelligence,
External benchmarking studies,
Best practices in industry and in functional areas
Solutions in the marketplace or used by competitors that may bring
benefits to the organisation

2010 Institute for Adult Learning, Singapore 30


Version No: 1.0
Prepare for a TNA

The client request context

The client request role is common in most organisations. When a


request is made, quick action is required to respond to client needs.

Scoping this initial phase of the needs analysis will reveal information
like the following:

Client contact information


Employee population affected
Identification of stakeholders
Timelines and constraints of the request
Key resources to be contacted for research or training
delivery assistance
Employee knowledge, skills,
Desired change
Client expectations and reporting requirements
Client interest in evaluating the project
Availability of population, job experts, and others for
extra information
Preferences, constraints

Aligning a training request to performance improvement

The focus of the initial meeting with the client should be on the results
that must be achieved by the training/learning solution, results that will
improve performance.

Stone (2009) suggests referring to the training needs analysis


processes as performance processes to keep the focus on the end
solution (performance) and not on the activities (learning) the trainer
may be asked to do.

The aim is to determine what

is driving the request,


appropriate business measures are, and
the nature of the performance gap.

2010 Institute for Adult Learning, Singapore 31


Version No: 1.0
Prepare for a TNA

Aligning a training request to performance improvement


(continued)

Questions to ask

When a client requests training, they usually provide an early signal


that suggests what is prompting the request. It is important to identify
and clarify this signal. This sets a context in which to begin the initial
line of discovery questioning.

Gupta et al (2007) have identified a number of questions to ask a


client to examine an existing performance problem.

1. What are the key business issues that must be addressed?

2. How long have the problems existed?

3. What are the consequences of not solving these problems?

4. Which business processes are affected by the problems?

5. What are the performance improvement goals?

6. What is preventing these goals from being achieved?

More Questions

Tobey (2005) has identified a series of questions to ask a client in


the initial conversation to derive key information at each level of
need of the business, performance, learning and learner.

These questions are outlined in Resource 2. We will use them to


guide us through the case study.

2010 Institute for Adult Learning, Singapore 32


Version No: 1.0
Prepare for a TNA

Learning Activity

Rank these elements

Rank the importance of each of these elements to assess training needs within an
organisation where 1 = most important and 10 = least important. Use each number
from 1 to10, once only. Write your responses in the column My ranking.

My Study Element
ranking ranking
A. Customer opinions

B. Training audits

C. Analysis of projected business or service


plans

D. Employee surveys

E. Requests from work groups

F. Senior management decisions

G. Advisory committee

H. External consultant

I. Skills inventory

J. Supervisors opinions

Based on study by Mathews,B.P., Ueno, A., Kekale, T., Repka, M., Pereira, Z., & Silva,
G. (2001). Quality Training: Needs and evaluation-findings from a European survey.
Total Quality Management,12(4), 483-490. cited in Kupta et al (2005) p 23.

2010 Institute for Adult Learning, Singapore 33


Version No: 1.0
Prepare for a TNA

Learning Activity: Case Study Part 1

Trekkers Clothing Company

TASK

1. What questions should Andy ask Tracey Tang, the Senior Production Manager to

place the training request in the context of business, performance, learning


and learner needs, and
build credibility in order to conduct a training needs analysis

2. Use Resource 2 and other information in the Learner Guide to help you.
3. Write down your suggested initial questions for client (Tracey) below.
4. Discuss in your group.
5. Select a group member to report back.

Business

Performance

(continued over)

2010 Institute for Adult Learning, Singapore 34


Version No: 1.0
Prepare for a TNA

TASK (continued)

Learning

Learner

2010 Institute for Adult Learning, Singapore 35


Version No: 1.0
Notes

2010 Institute for Adult Learning, Singapore 36


Version No: 1.0
Prepare for a TNA

Plan ahead

Spending time, money and effort on a TNA is a serious undertaking


for which resources must be available and issues clarified before the
TNA is conducted. Five important practical questions to ask before
starting a TNA are offered by the Chartered Institute of Personnel and
Development (CIPD) as follows

1. Do you have the resources to collect the data?

2. Do you have the resources to analyse the data?

3. Do you have the resources to make recommendations based on


the data?

4. Do you have the resources to plan and budget for the response to
the TNA?

5. Do you have senior level sponsorship?

TNA Proposal or Project Plan

At the preliminary stages of a TNA, after some initial data has been
gathered to get a general idea of the need, it is important to write a
plan of action or a project plan to help the analysis project stay on
track. This is a proposal for what will happen.

This document usually outlines the

purpose of the project


steps or phases
expected outcomes
(an estimate of) timelines
(an initial estimate of) costs

The plan helps to clarify responsibilitiesThe plan helps to frame an


unclear or uncertain situation, while recognising that the eventual
outcomes may be very different from what was initially anticipated.
Thats the main point of assessing the needs to eliminate guess
work about training and non training needs. So it is a working plan
that helps give a broad guide to the sponsor, the analyst and other
relevant parties of what will happen.

2010 Institute for Adult Learning, Singapore 37


Version No: 1.0
Prepare for a TNA

TNA Proposal or Project Plan (cont)

The written project plan also helps align common understandings and
expectations about the process and deal early with
misunderstandings about eg who is doing what, how it will be
conducted, purpose, timeframes etc. Once agreed to (and signed) by
relevant parties, this document serves as a communication tool and a
record of decisions made. It is not a static document and it should
reflect agreed changes as they occur. After one phase, the project
may need to take a different approach and once agreed, this should
be written into the next phase of the proposal.

Gupta et al (2007) offer the following tips for preparing a project plan

Include a Purpose section to clarify why the TNA is important


and the value of what it will achieve
Write it in business language and keep it brief and concise
Consider raising the possibility that both training and non-
training issues will be highlighted
Place the cost section at the end so that readers are familiar
with the how it will be done before seeing the budget estimates.

Sample Headings for a Project Plan for a TNA

Purpose
Phases (steps)
Expected outcomes
Timeframe
Cost estimates

2010 Institute for Adult Learning, Singapore 38


Version No: 1.0
Prepare for a TNA

Data Collection
This is the step in which data is collected for each type of need
(business, performance, learning and learners) to establish the need
for training. Most needs analysis relies on multiple-data collection
methods which provide reliable and appropriate data.

Purposes

There are multiple purposes for data collection including it

1. validates the clients original request

2. links the business need to the clients goals

3. supports or refutes hunches about what training is needed

4. identifies the gaps between actual and desired performance

Block (2000) provides a timely reminder that the purpose of data


collection in needs analysis is not research and needs assessors
should not become focused on the research process just because it
interests them. The focus of data collection is to help the client make
a decision and act, not understand. This is where analysis differs
from research.

Confidentiality and sensitivity

A general comment about confidentiality and sensitivity of the data


collected is warranted as we start this topic. During a TNA, eg
knowledge and skills gaps of individuals will be exposed, so their
needs must be respected or it is possible that senior management
are planning a significant staffing restructure, the details of which
need to be kept confidential for a time so the change process can be
managed appropriately and with the welfare of staff as a priority.

The responsibility on the analyst to maintain data confidentiality and


be sensitive to the individual is reinforced in a later section about
ethical considerations when conducting needs analysis.

2010 Institute for Adult Learning, Singapore 39


Version No: 1.0
Prepare for a TNA

Types of data

The table below from CIPD (2009) gives some general types of data that may
be collected.

Types of data What to include


Business What is required in terms of outputs, levels of customer service,
objectives interactions with other parts of the organisation. How are these
measured? What is going to change?

Technology How jobs are done now in terms of organisation and resources
and available. What technologies do people use, and how might this
organisation of change? Changes may also be planned in terms of numbers of
work people to carry out the targeted performance or in terms of the way
they are supervised or managed.

Employee Who is currently employed in the area you are analysing? Are
demographics numbers increasing or decreasing? Who is joining and leaving?
What categories of employees are included?

Education / Basic educational, vocational or academic qualifications which can


qualifications be linked to assumptions about peoples expectations of learning
provision, cognitive or thinking abilities as well as current level of
skills and knowledge

Past Previous knowledge, skills and behaviours that have been expected
experience in the past and are required currently, linked to the organisations
competence framework. Experience of past training interventions.

Job roles / Current responsibilities and the implications of planned changes on


responsibilities roles.

Current Current performance levels of individuals and teams and any areas
competence where competence needs to increase, to be measured against both
current and changed roles

Employee Whether people are full or part-time, permanent or temporary, fixed


status contract or short-term contract

Location Any issues of geographical dispersion and access to training


provision

Employee How employees feel about change? Will this affect their
attitudes and willingness to learn and acquire new skills? Does the
culture organisation want to change attitudes eg in the way
customers are treated?

2010 Institute for Adult Learning, Singapore 40


Version No: 1.0
Prepare for a TNA

Planning data collection

Tobey (2005) identifies four distinct tasks required at this point:

1. identify the questions to be answered


2. identify the data sources
3. identify potential data collection methods
4. choose the data collection method

We will now examine each of these planning steps individually.

Task 1: Identify the questions to be answered

1. identify the questions to be answered


2. identify data sources
3. identify potential data collection methods
4. choose the data collection method

Seeking information from stakeholders will clarify each type of need.

Tobey (2005) outlines some specific examples of questions to ask


including:

What problems must be resolved? To what measurable extent?


What opportunities must be capitalized on? What is the initial goal?
What strategies must be supported?
Business
What measures will indicate success?
What proportion of the problem, opportunity or strategic goal will be
attributed to the training effort?
(continued)

2010 Institute for Adult Learning, Singapore 41


Version No: 1.0
Prepare for a TNA

Planning data collection


Task 1: Identify the questions to be answered (continued)

What is a description of desired on-the-job performance?


What is a description of current on-the-job performance?

Performance What are the specific gaps between the desired and current performance?
How is on the-job performance measured?
How is on the-job performance managed and rewarded?
What tools and resources do the employees need to achieve the desired
performance?
What is a description of the work environment in which performance is
expected

What do learners know now?

What can learners do now?


Learning What skills do the learners need to be able to do differently? How well?

What knowledge do learners need to know?

What are the implications in the work environment for transfer of learning
back to the job?

What training have learners already had in this area?

How did previous training go?

What are their learning styles?

Learner What is their attitude about the job performance being targeted?

What is their attitude towards the planned training?

From what organisational levels will the learners come?

What will be the context in which they attend training (e.g. on-the-job, off-
the-job, after work, during work, with manager support or not)?

Will training attendance be mandatory or voluntary?

Tobey (2005)

2010 Institute for Adult Learning, Singapore 42


Version No: 1.0
Prepare for a TNA

Learning Activity: Case Study Part 2

Data collection questions and data sources

Andy needs to collect data from these staff.

Jenny Chan Director of Human Resources


Jerome Wong and Gary Lim Team Leaders
Michael Low Manager of Complaints Centre

He has written questions based on his initial conversation with Tracey the client.

TASK A

1. Individually decide which question belongs in which type of need.

2. Compare and discuss with your group members.

3. Categorise each question under the appropriate type of need and write out
the question in the appropriate column on the Data Collection Planning
Guide (Resource 4).

TASK B

2010 Institute for Adult Learning, Singapore 43


Version No: 1.0
Prepare for a TNA

Task 2: Identify data sources

1. identify the questions to be answered


2. identify the data sources
3. identify potential data collection methods
4. choose the data collection method

The process of identifying the best data source for each question is
the next task.

Data sources
Guiding Principles

Sources must match the purpose and needs of the data collection
process. Stone (2009) has outlined five considerations when
determining appropriate sources.
1. Credibility
Credible sources are objective and in a position to know. Credible
sources can be anyone in the organisation who can satisfy the
purpose of the TNA . When selecting credible data sources,
consider purpose, situation and constraints.
2. Convenience
Collect data from the sources at their convenience ie when they
are easily positioned to provide the data, even if this is not ideal
for the overall project time frames.
3. Cost
Recognise the economic implications of time away from work
tasks when gaining cooperation to collect data.
4. Cooperation
Plan how to gain approval to collect data and how to encourage
the willingness of the source to engage in the process.
5. Comprehensive
Use every effort to gain a complete picture to meet the purpose of
the data collection process and to maximise credibility of the
findings and recommendations.
Operational constraints can sometimes impact on what data is
collected or not. Being resourceful and flexible will assist in
managing the process.

2010 Institute for Adult Learning, Singapore 44


Version No: 1.0
Prepare for a TNA

Possible generic data sources include:

People
Clients
Executive management
Operational managers and supervisors
Employee or employee teams/groups
Advisory/reference groups/committees
High performers
Low performers
Learners
Coaches and mentors
Internal and external task experts
Internal and external subject matter experts
Customers
Training professionals
Members of professional associations or organisations
(local, regional or global)
Members of government agencies (local, national)
Members of industry associations

Documentation: Records and Reports (Extant data)

The range of data available from operational records and reports is


vast and is usually the most convenient for collecting business and
performance outcome data.

Examples include annual reports, benchmarking studies, sales


figures, complaints, grievances, quality, customer satisfaction,
customer service, productivity, HR data, job descriptions,
performance evaluations, past training feedback/evaluation
information, employee turnover, and accident and absenteeism
reports.

While such data (referred to as extant data) may be reliable and


credible, it must be remembered it was not collected with the
purpose of the needs analysis in mind. The data may be inexact
and inaccurate and not fit for purpose. An analyst may need to
infer from the data to find information/patterns and sift through the
data for relevance.

Flawed or unreliable data has little or no meaningful use when used


for decision making.
More discussion about extant data is in the next section.

2010 Institute for Adult Learning, Singapore 45


Version No: 1.0
Prepare for a TNA

A Reflective Point

Think about a client with whom you worksomeone in your organisation. (client
could mean your boss eg an in-house training manager, or a senior manager
or customer service manager within your organisation; or a client could be
someone external to your organisation)

Imagine that this client has made a training request.

Now consider these questions:

1. What is this clients perception of what training professionals do?

2. What is this clients perception of the role of training in supporting business


strategies and employee performance?

3. What questions will you ask this client to

a) Place this training request in the context of the business, performance,


learning and learner needs; and

b) Build credibility in order to conduct a TNA?

2010 Institute for Adult Learning, Singapore 46


Version No: 1.0
3 Methods to collect data

Scope and
requirements
of a TNA

Prepare for a TNA

1. qualitative data collection methods


2. quantitative data collection methods
Data collection 3. factors influencing selection
Methods 4. guidelines for implementing data collection

Data analysis
methods

Prepare a
TNA report

This section continues examining the data planning process by outlining qualitative
and quantitative data collection methods, some advantages and disadvantages for
each, factors influencing selection and guidelines for implementing the data
collection process.

Learning Outcomes

1. select appropriate qualitative data collection methods


2. select appropriate quantitative data collection methods
3. identify factors influencing selection
4. discuss guidelines for implementing the data collection

2010 Institute for Adult Learning, Singapore 47


Version No: 1.0
Methods to collect data

Task 3: Identify potential data collection


methods

1. identify the questions to be answered


2. identify the data sources
3. identify potential data collection methods
4. choose the data collection method

An overview of data collection methods

Data collection methods are either quantitative or qualitative.

Result Examples include

Qualitative soft data is more interviews


intangible, anecdotal, focus groups
personal and subjective as observation
in
opinions, attitudes,
assumptions
feelings, values,
desires

Quantitative hard data is surveys


objective and document
measurable review
whether stated in
frequency,
percentage,
proportion or time.

Some considerations when planning data collection for


needs analysis

1. get permission from the client to gain access to data sources


2. identify the most critical questions to be answered
3. determine how much data gathering will provide what data is
needed

2010 Institute for Adult Learning, Singapore 48


Version No: 1.0
Methods to collect data

Following are a few of the commonly used data collection methods


we will examine in this module. Information has been compiled from
these sources: Gupta, et al (2007), Tobey (2005), Brown (2002),
Lewis (2000), Barbazette (2006),Stone (2009), Cook (2005) and
Silberman (2006)

Qualitative Methods

A. Interviews
B. Focus groups
C. Observation

A. Interviews

An interview can be conducted in person, at work or away from it, by


phone or by computer technology and is a one-on-one discussion
which is most often used to collect qualitative data for performance
and learner needs and is also used to gather business needs
information. It can be structured or unstructured, formal or informal
(like a conversation). Exploratory interviews are useful to discover
employee opinions prior to larger scale analysis.

The element of human interaction is the greatest benefit of an in-


person interview during which non verbal cues, facial expressions
and anecdotal input add richness to the data. Barbazette (2006)
suggests interviews should last from thirty to sixty minutes and be
conducted in a quiet and private space to avoid distractions and
interruptions.

Gupta et al (2007) and Tobey (2005) outline other important practical


suggestions when interviewing:
1. plan ahead (have an agenda)
2. use a moderate tone
3. enunciate clearly
4. avoid speaking rapidly
5. keep the discussion focused
6. ask the interviewee to confirm and specify generalized
statements and assumptions
7. take notes
8. allow for pauses
9. if recording the interview, assure the interviewee that you are
doing it to identify rich anecdotal examples; that confidentiality
will be maintained and that the tape will be destroyed after the
analysis is complete
10. probe for additional responses to your questions

2010 Institute for Adult Learning, Singapore 49


Version No: 1.0
Methods to collect data

Advantages of a phone interview are to

Gather small pieces of information


Ask follow-up questions
Obtain information from many sources quickly
Obtain input from geographically disbursed individuals
Save travel costs & time

Advantages Disadvantages

rich anecdotal detail can be time consuming to


conduct and analyse
are adept at uncovering
attitudes, perceptions, causes can be difficult to analyse and
of problems and possible quantify results
solutions
skilled interviewer needed to
opportunity to discuss record responses, not interpret
complex issues that require
explanation relies on a skilled interviewer
who can generate data without
clarify quantitative data making interviewee self
conscious or suspicious
seek sensitive information
need a representative group
consistent data across
interviews can be compared can be costly
for patterns/trends
can be difficult to schedule
can be informal or formal

allows for spontaneous


feedback

flexibility to adjust plan

can explore unanticipated


issues

2010 Institute for Adult Learning, Singapore 50


Version No: 1.0
Methods to collect data

B. Focus Groups

Kreuger defines a focus group as a "carefully planned discussion


designed to obtain perceptions in a defined area of interest in a
permissive, non-threatening environment" (1988, p.18). retrieved 7
July 2009 http://www2.fhs.usyd.edu.au/arow/arer/002.htm
A focus group relies on the group to offer opinions and ideas about a
specific topic. Usually the members of the group have something in
common. Kreuger (1988) suggests that a focused interview will
include less than ten questions and often around five or six. Stewart
and Shamdasani (1990) propose that most interview guides consist of
fewer than a dozen questions.

Recommendations for group size vary. Most focus groups consist of


between 6-12 people. However, the number of participants will
depend on the objectives of the analysis. For example, smaller groups
(4-6 people) are preferable when the participants have a great deal to
share about the topic or have had intense or lengthy experiences with
the topic of discussion.
Cited in Lewis, M. (2000). Focus Group Interviews in Qualitative Research: A
Review of the Literature. Action Research E-Reports, 2. Available at:
http://www.fhs.usyd.edu.au/arow/arer/002.htm

Advantages Disadvantages

can focus on a specific time consuming and


problem, goal, task resource-intensive
useful with complex or challenging for only one
controversial issues facilitator to run
develop hypotheses for
dependent on a skilled
large group testing
facilitator to manage group
questions can be modified dynamics so all members
to explore unanticipated have a say
issues
useful detail e.g. reactions,
gain consensus
enhance buy-in
encourage listening skills
all members can be heard

2010 Institute for Adult Learning, Singapore 51


Version No: 1.0
Methods to collect data

C. Observation of Work

This data collection method involves observing employees in their


workplace. Data can be collected in a structured or unstructured
way.

In a structured observation session, decisions on what factors to


observe have been decided prior to the observation e.g. an analyst
may observe how employees welcome new guests arriving at a
hotel. In an unstructured observation, an analyst may collect and
record information relating to all hotel reception staff doing a range
of activities.

There are various techniques for observing performance; for


example, telephone monitoring, video monitoring, direct observation
using e.g. secret shoppers and monitoring through computer
software.

Advantages Disadvantages

effective way to assess tendency for performer to


needs for physical skills act differently because
they are being observed
documents performance e.g.
frequency, time intrusive

low cost generally time consuming

generate data relevant to requires highly skilled


work environment observer with process and
content knowledge
minimal interruption of work
routine

2010 Institute for Adult Learning, Singapore 52


Version No: 1.0
Methods to collect data

Quantitative Methods

A Surveys
B Document reviews

A Surveys

It is generally accepted that developing good surveys is difficult.


Kupta et al (2007) outline several key phases to follow
systematically to preparing surveys which achieve desired results.
Outlined below are the phases and considerations.

Survey development phases

Phase 1. Prepare
Establish the goals and outcomes; become familiar with the
situation and typical expressions eg translate technical jargon into
simpler language; observe people at work; review written reports;
decide whether an existing or commercially available survey is
appropriate or a new survey is needed.

Phase 2. Design
Consider the number of people involved. For large numbers use
mostly close-ended questions that can be recorded and analysed
easily by computer.
Consider how the data will be collected. Will it be collected on paper
or via an organisations intranet or internet? A web search of the
term free online survey will produce a number of free tools and
services for creating surveys and collecting survey data.
Consider how the data will be analysed. Responses to online
surveys are often analysed with easy-to-use web-based software.
Paper responses can be analysed by hand counting answers or
surveys can be electronically scanned.

2010 Institute for Adult Learning, Singapore 53


Version No: 1.0
Methods to collect data

Phase 3. Develop Questions

Sequence questions. Start with simple and interesting questions;


present sensitive questions later; place demographic items at the
end; avoid labeling or clustering similar items to minimise the effect
of the rating of one item affecting the other.

Limit the amount of information gathered in one question. Ask one


question for each piece of information.

Avoid complexity. Frame questions so they can be easily


answered; avoid questions that require complex calculations.

Word questions carefully. Use simple words; avoid leading


questions; avoid negatively phrased questions

Determine the type of questions

Open-ended questions and Fill-in-the-blanks are easy to ask but


require more time and effort to analyse.

Closed-ended questions allow respondents to choose from options


and are easy to analyse but do not allow the respondent to
elaborate. Multiple choice, two choice and scales of various kinds
can be written.

The Likert-type scales are examples of types of scales commonly


used. Survey items that ask respondents to rank or rate values or
attitudes use Likert-type scales. Generally it is considered best
practice to assign the most positive value at the higher end of the
scale.

Types of rating scales described by Barbazette (2006) include


quality, quantity, speed, sequence and perception.

For example, an excellence rating scale may include:


1. excellent
2. good
3. fair
4. poor

2010 Institute for Adult Learning, Singapore 54


Version No: 1.0
Methods to collect data

An agreement rating scale may include


1. strongly agree
2. agree
3. neutral
4. disagree
5. strongly disagree

A quantity rating scale may be


1= step not performed
2= step is done once
3= step is done required number of times
4= step is done more often than required by the standard

An ease rating scale may be


1. extremely easy
2. somewhat easy
3. somewhat difficult
4. extremely difficult

Phase 4. Write Instructions


Write explicit instructions stating how the respondent should
complete the form. State the amount of time needed, to whom it
should be returned and by when.

Phase 5. Write The Cover Letter


A cover letter explains why the person is being asked to
complete the survey and how they will benefit from it. To
improve the response rate, ask a senior staff member to
endorse his support for the survey by signing it. Confidentiality
status should be included.

Phase 6. Pilot-Test
It is wise to trial the survey to identify problems in construction
or physical layout. Other issues such as whether the
instructions are clear and concise, whether respondents
understand the questions easily, whether respondents answer
the questions easily, whether respondents have enough space
to record their comments and whether the questions elicit the
responses that address the survey purpose will be clarified. If
the responses do not really address the initial purpose of the
survey, it is best to revise and pilot-test again.
Phase 7. Conduct the survey and follow-up
The survey is ready to launch. Monitor incoming responses
frequently. Non-respondents are a common result. Follow up by
sending reminders.

2010 Institute for Adult Learning, Singapore 55


Version No: 1.0
Methods to collect data

Surveys

Advantages Disadvantages

inexpensive time consuming to develop

easy to tally results challenging to construct


questions well
easy to participate
reliability and validity issues
quick results
choosing appropriate answer
frequencies easy to understand scale is critical

soft data questions can yield qualitative respondents can skew results
data, which is tallied as quantitative e.g. by not reading questions

can use a variety of questions formats poor return rate means sample
not large enough
can reach large numbers in short time
little provision for free response
can respond without fear of retribution
lacks detail

2010 Institute for Adult Learning, Singapore 56


Version No: 1.0
Methods to collect data

B Document Reviews

There is a wide variety of information sources that can be used in a


needs analysis such as business plans, mission statements, job
descriptions, sales figures, policy manuals, customer-service records,
personnel records, budgets, financial statements, audits, staffing
statistics, accident reports, waste reports, production rates, repairs,
photographs, competency frameworks, benchmarking reports, annual
reports, performance reviews and
training evaluation forms.

The benefit of collecting such data is increased when these data are
collected in conjunction with another data-gathering method.

Gupta et al (2007) make the following suggestions when using data


from documents:

1. Be clear about the type of information being sought before


undertaking an extensive search of records.
2. Seek permission prior to using information.
3. Look for trends and patterns in the data

Advantages Disadvantages

hard data and objective usually collected for purposes


evidence other than TNA, so training
issues must be inferred
can examine trends over
time must sift through information to
separate it from unrelated data
does not involve individual
employee confidentiality no control over methodology
issues because data is used to collect data
used in aggregate form
reflects past situations rather
gives clues to trouble than current
spots
need skilled data analyst

often do not indicate cause of


problems or solutions

2010 Institute for Adult Learning, Singapore 57


Version No: 1.0
Methods to collect data

Learning Activity: Case Study Part 3 n

Potential data collection methods

Now Andy must identify potential data collection methods.

TASK

1. List potential data collection methods on the Data Collection Planning


Guide. Refer to your Learner Guide and the previous activity to help you.
.

2. Compare your list with others in the group.

2010 Institute for Adult Learning, Singapore 58


Version No: 1.0
Methods to collect data

Task 4: Choose data collection method/s

1 identify the questions to be answered


2 identify the data sources
3 identify potential data collection methods
4 choose the data collection method/s

It is widely agreed it is advisable to choose both quantitative and


qualitative data collection methods so the data sets complement each
other. For example, use surveys to gather facts and use follow up
interviews to delve into why people answered questions they way
they did.

Choose methods that will provide the best data with optimal
expenditure of resources: consider time, financial costs,
essentialness, data source availability, logistics and assessor skill
level.

Tobey (2005) has identified seven key areas to consider when


choosing data collection methods and lists relevant questions for
each.

Time What is the timeframe for the data collection process?


needed What is the timeframe for the training delivery?
How much time does each method take to implement?

Other How much time will be needed from other employees in


resources e.g. data collection and as respondents?
needed
What other resources will be needed eg equipment,
administrative support, back up for disrupted work
processes?

Other What are the costs to buy equipment, software,


costs assessment tools?

Are there any fees for external services e.g. for conducting
interviews, transcribing tapes, doing statistical analysis?

(continued over)

2010 Institute for Adult Learning, Singapore 59


Version No: 1.0
Methods to collect data

Essential This concept relates to the uniqueness of the data


-ness collection method and its relevance to the requirement

Is a specific data collection method one of only a few (or


only one) that can obtain a needed data set?

Could other data collection methods yield the same data?

Availability How accessible are the people providing the data?


of Data
Sources Who can help gain access?

Which sources will take more time to gain access?

When is the best time to access sources?

Logistics Where are the sources located?

Is travel involved?

Which essential methods require travel? Are there


alternatives?

Is there technology eg email, videoconferencing to assist?

Are there appropriate facilities eg interview rooms, in the


organisation?

Needs Does assessor have more expertise and experience in


assessor one or two methods?
skill level
Can time and resources be saved if assessor chooses a
known and practiced method over an unknown method, if
both are appropriate?

2010 Institute for Adult Learning, Singapore 60


Version No: 1.0
Methods to collect data

Implementation guidelines for data collection

These suggestions or tips for the needs assessor will help make
the implementation process more effective and efficient.

Final check Double check chosen methods to minimise time


and resource and maximise value

Develop a plan To self monitor purpose and timelines

Have a Plan B Be flexible and ready to adapt any aspect

Update client The client must approve plan


Update and report back regularly: it builds
credibility and gains client support

Do not interpret Avoid adding personal views

Stay neutral Do not impose or test own preconceived ideas;


no leading questions

Use document Existing records may contain inaccurate


data carefully information

Seek help from To minimise bias, have others conduct some


others to achieve interviews or examine documents
reliability
Liaise with client to decide how, what and how
Plan how to much data will be shared with respondents;
disseminate data ensure feedback is provided at some level

Peruse data as it Look for significant information


is received Adjust strategies if required to test validity

May suggest you can stop if trend is clear and


Repetitive data further data collection will present the same
information

Efficiency demands that if you require information


Multiple needs for different needs from the same source, it can
from same source be administered at the same time

2010 Institute for Adult Learning, Singapore 61


Version No: 1.0
Methods to collect data

Learning Activity: Case Study Part 4

Data collection methods

Andys challenge was to decide which methods will provide the most useful data
and which methods will provide for multiple needs.

TASK

1. Compare your selections with Andys choices.

2. Discuss what you think influenced his decisions to make the selections he
did.

What tips and cautions would you offer Andy to help him implement the data
collection process efficiently and effectively? List your responses below.

2010 Institute for Adult Learning, Singapore 62


Version No: 1.0
4 Methods to analyse data

Scope and
requirements
of a TNA

Prepare for a TNA

Data collection
Methods

Data analysis 1. basic descriptive statistics


methods 2. findings and recommendations

Prepare a
TNA report

This section describes how to apply basic descriptive statistics to


quantitative data, review findings and recognise the difference between findings
and recommendations.

Learning Outcomes

1. apply basic statistical methods to quantitative data


2. review findings from data analysis
3. explain the difference between findings and recommendations

2010 Institute for Adult Learning, Singapore 63


Version No: 1.0
Methods to analyse data

Data analysis adds value in two ways; firstly by developing a


current picture of whats going on and secondly by translating that
data into action items or recommendations.

Findings and Recommendations

The first task in data analysis is to classify and summarise key


findings. Findings are the facts. They are unrelated to organisational
context or needs assessors biases. They are the pure facts
indicated by the data e.g. 75% of ex-call centre telephone operators
said job stress was the main reason they left the job.

Findings dont indicate why or why not, what should be changed or


even if the data is true. Continued analysis will reveal more specific
information that will be used to identify recommendations.

Recommendations are contextual within the organisation and


situation as well as with other data. They are developed as a result
of examining the current picture and translating it into action items.

Types of Data

There are two kinds of data: quantitative (numerical) and qualitative


(all other data).

Quantitative data

Examples of quantitative data are sales figures, absenteeism


numbers, turnover figures, survey, staffing statistics.

It is easier and quicker to gather, score and analyse quantitative


data and it can be presented and summarised in tables, charts or
graphs.

Qualitative data

In comparison, qualitative data which is usually narrative in form is


more difficult and time consuming to collect and analyse but it offers
a rich source of information.

2010 Institute for Adult Learning, Singapore 64


Version No: 1.0
Methods to analyse data

Examples of qualitative data include stories, anecdotes, photographs,


survey essay questions and videos. Analysing these kinds of data
involves identifying such things as patterns, themes, trends, main
ideas and issues.

Software (such as NVIVO and Atlas) will improve the ease, accuracy
and speed of qualitative data analysis.

There are sometimes opportunities to combine both qualitative and


quantitative data analysis to determine more robust patterns and
themes. For example, how many employees said X or Y on the
survey and how many said the same things during focus groups?
Questions to consider include
Is there a consistent match?
What common threads exist between and across the
responses?
Is there a predictable pattern across both the focus group
responses and the survey?

Strategies for analysing data

Gupta et al (2007) describe some strategies for ensuring success


analysing data (p93)

1. Limit the responsibility for analysis to one or two individuals

2. Review the data for discrepancies, deviations and


irregularities

3. Present irregularities in a separate section

4. Update client on discrepancies

5. Omit irrelevant data that does not relate to the analysis and its
objectives

6. List the frequency of responses for each category

7. If faced with conflicting data, seek the opinion of an expert

2010 Institute for Adult Learning, Singapore 65


Version No: 1.0
Methods to analyse data

A framework for considering what the data means


The four stages of needs i.e. business, performance, learning and learner,
also provide a framework in which to examine what kinds of findings
should result from the data analysis.

Business needs - These findings relate to

1. importance of the business goal in the business plan


2. strength of the link between business issues and proposed training
intervention/s
3. how much of the business goals should be achieved by the training
intervention
4. how accurately the clients original presenting problem is reflected in
the business indicators
5. any other relevant business indicators

Clients should not be given the impression that training can achieve the
entire business goal. There are too many other factors that can have an
impact. It is feasible for the client and training professional to negotiate
the percentage of change the training alone could achieve.

Specific Desired Work Performance - These findings relate to

1. results expected from job performers


2. performance standards
3. background information about job environment, tools and processes
used when employees perform the targeted skills
4. organisational factors (non-training factors) other than knowledge or
skill deficiencies that affect performance
5. indicators regarding transfer strategies that might be required in the
training design

Training Design Information - These findings relate to

1. learning objectives
2. indicators for learning activities that will replicate the job environment
3. background information to replicate the job environment for skill
practices
4. test items for knowledge assessments
5. process and product checklists for skill practice assessments
6. how learners value (or not) the skills to be taught
7. other attitudinal information about the learners

Training Delivery Information - These findings relate to


1. physical environment indicators
2. facilitation process standards

2010 Institute for Adult Learning, Singapore 66


Version No: 1.0
Methods to analyse data

Descriptive Statistics

The way meaning is attributed to statistical data is through the use of


descriptive statistics. Descriptive statistics are used to describe the
basic features of the data. They provide simple summaries about the
sample and the measures. Together with simple graphics analysis,
they form the basis of almost every quantitative analysis of data.
Following are descriptions of basic descriptive statistical methods.

interval A scale in which the difference between two values


scale measured on the scale has the same meaning e.g. a
survey in which answers are on a 5 point scale of 1 to 5

nominal A type of scale with a limited number of possible


scale outcomes that cannot be placed in any order
representing their relative value e.g. age groups

frequency number of times a value appears in a data set

percentile A descriptive scale that demonstrates how a score or


measure compares to other measures in the same data
set

mode most frequently occurring value in set of scores

median score found at the exact middle of set of values

mean value computed by adding up all the values and dividing


by the number of values; also the average

reliability extent to which a test or measuring procedure yields the


same results on a repeated trial; degree of consistency

validity The extent to which a test or measuring procedure


measures what it purports to measure

significance A test for determining the probability that a given result


did not occur just by chance.

Statistical software for analysing quantitative data such as SAS


(www.sas.com) and SPSS (www.spss.com) make these calculations
easy, but it is possible to perform them manually.

2010 Institute for Adult Learning, Singapore 67


Version No: 1.0
Methods to analyse data

Learning Activity

Descriptive statistics

A knowledge test with 100 items was administered to 20 participants. The possible
score range for the test was 0 (no correct answers) to 100 (all correct answers).

Following are the test scores.

Work with a partner to answer the questions.

A B C D E F G H I J
Score Score Score Score Score Score Score Score Score Score
98 100 87 63 60 79 87 52 84 91

K L M N O P Q R S T
Score Score Score Score Score Score Score Score Score Score
98 89 87 72 75 59 68 93 80 68

1. What is the frequency distribution for these test scores?

2. What is the mean for these scores?

3. What is the median of these scores?

4. What is the mode of these scores?

2010 Institute for Adult Learning, Singapore 68


Version No: 1.0
Methods to analyse data

Ethical use of data


Ethical
Statistical results can be presented and interpreted in more than one considerations
way to achieve different goals. It is a fine line between influence and are discussed
in more detail
manipulation in the world of statistical analysis. Where that line is
in Section 6.
depends on the situation, the client and the inherent risk for the
training professional in making flawed data interpretations.

Most decisions in a needs analysis involve value judgments and value


judgments often lead to ethical issues. Most needs analysis are by
nature political because they involve support for or challenges to those
who are in power. Ethics, in either situation are likely to be important.

When statistical analysis provides indicators that arent strong, it may


be of value to seek additional corroborating data from other sources
and methods to support or refute the statistical analysis.

Learning Activity

What are the findings?

Consider the following example

Exit interviews from employees who left organisation in last 12 months were analysed.
The number of ex-employees represented in the data was 40.
Of these 40 ex-employees, 18 (45% of total respondents) said that low salary was a
major reason for their leaving.
Out of those 18, 10 (56% of the 18; 25% of total respondents) said that lack of
training and development opportunities was a second major reason for leaving.

What are the findings?

2010 Institute for Adult Learning, Singapore 69


Version No: 1.0
Methods to analyse data

Learning Activity: Case Study Part 5

Findings

TASK A

1. Use the framework on page 66 to help you review Andys findings


(Handout)

2. Write any comments below eg

which findings did you expect?

which findings surprised you?

which findings are most/least critical?

comment on the impact of the TNA if business needs were not


considered when conducting the TNA.

2010 Institute for Adult Learning, Singapore 70


Version No: 1.0
Methods to analyse data

Learning Activity: Case Study Part 5

Findings

TASK B

Look at Andys findings and complete the following

1. Tick () which findings the client was correct about (Col 3)

2. Indicate with an arrow () each finding that was new to the client (Col 4)

3. Place a T if a training issue or a NT if a non-training issue (Col 5)

4. Record the data source (Col 6)

Record your thoughts about the importance of identifying non training


issues.

2010 Institute for Adult Learning, Singapore 71


Version No: 1.0
Notes

2010 Institute for Adult Learning, Singapore 72


Version No: 1.0
5 Prepare a TNA report

Scope and
requirements of a
TNA

Prepare for
a TNA

Data collection
methods

Data analysis
methods

1. elements of a report
Prepare a 2. ethical issues and standards
TNA report 3. common errors and obstacles
4. trends and developments

The final phase in a TNA is preparing a formal report to present the feedback, including
findings and recommendations to the client. The purpose of the report is to persuade
the client to approve the training intervention design and delivery as recommended;
and to take ownership of the non-training issues and recommendations. This section
outlines the key elements of a report, ethical considerations and common errors and
obstacles faced by needs analysts. Trends and developments are summarised
described briefly.

Learning Outcomes

1. write a TNA report containing key information


2. examine ethical issues and standards
3. discuss some common errors and obstacles
4. describe trends in needs analysis

2010 Institute for Adult Learning, Singapore 73


Version No: 1.0
Prepare a TNA report

Elements of a TNA Report

The final phase in a TNA is preparing (and presenting) a formal report


to present the feedback including findings and recommendations to
the client. The purpose of the report is to persuade the client to
approve the training intervention design and delivery as
recommended; and to take ownership of the non-training issues and
recommendations.

Critical success factors, other resource constraints and an


implementation plan must be considered in the feedback process.

Like the needs assessment process itself, there are many


appropriate ways in which to prepare and present a TNA report.
Gupta et al (2007) propose final reports contain the following
elements:
Executive summary of goals, process, findings and
recommendations (typically one page)
Goals or objectives
Overview of data collection methods
Main Findings
Recommendations
Appendix (including supporting instruments and data)

When preparing reports, include pie charts, graphs and diagrams as


appropriate to highlight key findings.

The Victorian Dairy Manufacturing Sector has produced a TNA Final


Report, prepared by Kris Wilson in September 2006
Retrieved 11 August 2009 from
http://www.dairysafe.vic.gov.au/pdf/Training_Needs_Report_22Sept
06.pdf
which contains the following content headings
1. Executive Summary
2. Background
3. Aims & Objectives
4. Methodology
5. Target Audience and Responses
6. Questionnaire Findings
7. Focus Group Sessions
8. Project Conclusions and Recommendations
9. Appendix
10. References

2010 Institute for Adult Learning, Singapore 74


Version No: 1.0
Prepare a TNA report

Barbazette (2006) proposes a Ten-Part Training Plan to report the


results of the analyses and to meet projected training needs for a
group of employees or for a period of time e.g. for a twelve month
period. Here is a Training Plan Template.

1. Define the issues clearly.


2. Reaffirm the outcome, results or objectives of the plan.
3. Clearly state the performance deficiency and its causes
4. Identify/establish performance standards
5. Identify target population
6. Establish evaluation criteria
7. Describe proposed intervention
8. Estimate the cost of the plan
9. Build a partnership with management/client
10. Schedule according to business demands

Phillips (1999) recommends presenting the needs by priority with


the most critical needs listed first and the least critical listed last.
The priority is based on either the cost of satisfying the need or the
critical importance of the need to the organisations success.

Many experts suggest that the need analyst negotiate having a


face-to-face feedback session to present the results to the client
and discuss them. This provides a forum that enables the
stakeholders to own the results.

Key factors to consider to convince the client to approve the


training plan are:

1. include a level of detail that matches the client groups


expectations
2. be well equipped to answer questions
3. organise the information in such a way as to build evidence so
that the answers logically present themselves to the client
4. structure the meeting to move from the known and easy to the
unknown and more difficult
5. demonstrate an ability to link disparate points of information to
indicate a clear link between the business issues and
challenges
6. support recommendations with citations from authorities in the
field and your knowledge of industry best practices.
7. communicate confidence
8. speak to clients in their own language (avoid needs analyst
jargon)

2010 Institute for Adult Learning, Singapore 75


Version No: 1.0
Prepare a TNA report

Non Training Issues/Needs

Training is not always the right solution to a problem in the


organisation. In the context of the bigger picture of the business
problem, opportunity or strategy, training itself only supports job
performance that is related to knowledge and skill deficiencies. Lack
of job performance (and the ensuing lack of business results) is
usually related to multiple factors including.

1. lack of any feedback


2. lack of feedback about performance quality
3. unclear performance expectations they dont know what is
expected
4. lack of tools, resources, equipment, space, authority
5. skills mismatch between employer ability and job requirement
6. lack of policies/procedures to support desired performance
7. ineffective or inefficient work processes
8. lack of management support for skill transfer back to the job
9. lack of incentives/rewards for producing desired performance
10. lack of negative consequences for undesirable performance
11. uncertainty about job security
12. lack of motivation to perform (not lack of skill)
13. changes in customer needs and interests
14. external pressures such as competition, market influences,
current events
15. regulations that prevent people from performing in the way to
achieve business results

adapted from Silberman (2006), Tobey (2005) and Mager (1997)

2010 Institute for Adult Learning, Singapore 76


Version No: 1.0
Prepare a TNA report

Making training recommendations

The needs assessor will make recommendations based on the


findings. In Section five, we described findings as the facts: the
pure facts indicated by the data. Identifying findings and
developing recommendations are two separate thought processes.

Four factors help determine how to turn a finding into a


recommendation. Consider

1. how this finding relates in context with other training and


non-training findings
2. the training professionals role and level in the organisation
3. perception of the training function within the organisation
4. cost of implementing a recommendation compared to how
much benefit will be derived from it

Usually training recommendations include the following areas

1. Learning objectives for the proposed course


2. Course content emphasis/focus
3. Activities to focus on particular skills and breadth/depth
required
4. Types of learning activities and training materials (visual,
verbal, manuals, workbooks, notebooks, learning logs,
reference, handouts)
5. Delivery methods (self study, small group, classroom, e-
learning, learning lab, distance)
6. Types of learning environment, pre and post work,
prerequisites, training schedule
7. Learner profile (background, experience, expertise)

The nature of training recommendations provides a clear example of


how closely TNA is related to training design and how the line
between them can become blurred. The entire practice of
implementing training from start to finish is an iterative and
interdependent process.

2010 Institute for Adult Learning, Singapore 77


Version No: 1.0
Prepare a TNA report

Making non-training recommendations

In organisations where training is viewed as critical for helping the


company meet its strategic objectives, it is imperative to identify
non-training issues.

Tobey (2005) highlights several reasons including:

1. It gives the highest likelihood that job performance goals (and


business goals) will be achieved

2. The training effort has an improved chance of succeeding

3. The focus remains on the big picture not just on meeting a


training need

4. The role of the trainer is broadened to that of a collaborative


internal consultant

5. The credibility of the training role in the organisation improves

6. If the client chooses not to implement the non-training


recommendations, and subsequently the business and
performance goals are not met, a record of non-training issues
exists to refute any suggestion that the training was the sole
cause of the failure

2010 Institute for Adult Learning, Singapore 78


Version No: 1.0
Prepare a TNA report

Critical success factors and resource constraints

As the gap between current and desired state becomes clearer, a


number of critical success factors will emerge along with a number
of constraints.

A critical success factor (CSF) is something critical to project


success or failure.
A constraint is something which may limit the options available in
the achievement of success.

CSFs are the essential areas of activity that must be performed


well so you are able to achieve the objectives or goals for your
project. One way to determine critical success factors is to
consider the project has been successfully completed and ask
what things that, if they either had or had not happened would
have prevented a successful outcome. These are the true
CSFs.

Identifying CSFs helps create a common point of reference to


help direct and measure the success of the project. All team
members know exactly what is important. This helps people
perform their own work in the right context and towards the
same overall aims.

Senior management involvement/support is usually always a


critical success factor in the success of a work project, and it is
no different for a TNA project. Experts suggest senior
management must take the responsibility of leading from the
front through involvement such as reviewing project milestones
or contributing ideas eg to the selection of teams.

According to Cotton (2004), constraints tend to fall into the


following categories

people (too few, too many, wrong people, will people be


available when we need them? Do they have appropriate
skills? How steep is the learning curve for them?)

project overload (too many planned or concurrent projects)

2010 Institute for Adult Learning, Singapore 79


Version No: 1.0
Prepare a TNA report

budget (lack of funds, impact of stakeholders on funding,


unsubstantiated case to spend the funds, are
recommended solutions affordable? Are the
recommendations essential or nice-to-have?)

timing (scheduling clashes, too little time, bad timing, can


we deliver the solutions by the required date?, who sets the
deadlines?, are the deadlines realistic?, do we understand
the time implications of the solutions we are
recommending?)

geography (location of workers home, office, field, do we


need to use external facilities or can we use our own in-
house training centres?)

IT (new and old not yet integrated, expensive, not yet ready,
cost of data migration)

Future work (how will the training solution be delivered to


new hires who join after the delivery? Do we have staff to
deliver? Are we recommending solutions with a limited
shelf-life? Is there sufficient in-house delivery capability to
continue to deliver the solutions after key training staff has
left?)

Cotton (2004) suggests that as each constraint is discovered, it


should be documented, assessed as to how it should be managed
and whether it is a critical success factor and compiled into a risk
register. A risk register is a list of risks, each of which is detailed
with how it is to be managed, by whom, when and what effect each
may have on the overall project.
Cotton says there are four things to remember about the risk
register
1. Produce it
2. Maintain it
3. Use it
4. Communicate it (constantly)

He adds it is imperative to include the CSFs and constraints in the


TNA report.

2010 Institute for Adult Learning, Singapore 80


Version No: 1.0
Prepare a TNA report

Learning Activity: Case Study Part 6

Training Recommendations

TASK A

Based on these findings, what should Andy recommend to the client?

List your training recommendations below. Use the suggested headings in the LG
to help you.

Discuss with your group.

2010 Institute for Adult Learning, Singapore 81


Version No: 1.0
Prepare a TNA report

Learning Activity: Case Study Part 6

Non-Training Recommendations

TASK B

Based on these findings, what should Andy recommend to the client?

List your non training recommendations below.

Discuss with your group.

2010 Institute for Adult Learning, Singapore 82


Version No: 1.0
Ethical considerations

Ethical Considerations

In Section 5, ethical issues in relation to data use were briefly


mentioned. In this section we will examine in more detail ethical issues
which assessors may confront when conducting a needs analysis. In a
recent development, professional organisations have produced
guidelines for ethical standards and ethical practices which provide
direction when faced with difficult ethical issues.

Personal and professional values

How well do you know your personal and professional values? When
dealing with ethical issues, it helps to know this as it will affect how you
will act in specific situations.

An ethical issue is more likely to arise if a value judgment is involved.


In needs analysis, most decisions involve value judgments. There are
a number of areas including confidentiality, conflict of interest and use
of data where value judgments are made. A needs assessment is built
on the trust of all involved. If the trust is lost, the project will suffer.

How an assessor will act in these different situations is often


dependent on his/her own set of personal and professional values and
beliefs. Personal integrity and an ability to negotiate skillfully with e.g.
clients, respondents and colleagues will impact on the success of the
analysis and the follow-up phase of implementing recommendations.

It is sometimes difficult to sort out the ethical issues in a situation and


to identify appropriate responses to them. The recent development of
standards helps this process.
Standards/Guidelines
The following professional organisations have developed guidelines or
standards which may help identify, navigate and negotiate ethical
issues.

The Joint Committee on Standards for Educational Evaluation


published an updated edition of standards in 1994. The standards
revolve around four areas

Utility
Feasibility
Propriety
accuracy
A summary of these standards is outlined in Resource 6.
Accessed on 28 July 2009 from
http://www.eval.org/EvaluationDocuments/progeval.html

2010 Institute for Adult Learning, Singapore 83


Version No: 1.0
Ethical considerations

The International Society for Performance Improvement, Code of


Ethics is based on the following six principles and is intended to
promote ethical practice in the profession.

Source: www.ispi.org accessed on 28 July 2009

The Code of Ethics is

1. Add Value

Conduct yourself, and manage your projects and their results, in ways that
add value for your clients, their customers, and the global environment.

2. Validated Practice

Make use of and promote validated practices in performance technology


strategies and standards.

3. Collaboration

Work collaboratively with clients and users, functioning as a trustworthy


strategic partner.

4. Continuous Improvement

Continually improve your proficiency in the field of performance


technology.

5. Integrity

Be honest and truthful in representations to your client, colleagues, and


others with whom you may come in contact with while practicing
performance technology.

6. Uphold Confidentiality

Maintain client confidentiality, not allowing for any conflict of interest that
would benefit yourself or others.

2010 Institute for Adult Learning, Singapore 84


Version No: 1.0
Ethical Considerations

Members of the American Evaluation Association developed its


own set of five Guiding Principles for Evaluators, revised in 2004
which are

1. Systematic inquiry

Educators should conduct systematic, data-based inquiries

2. Competence

Evaluators provide competent performance to stakeholders

3. Integrity and honesty

Evaluators display honesty and integrity in their own behaviour,


and attempt to ensure the honesty and integrity of the entire
evaluation process.

4. Respect for people

Evaluators respect the security, dignity and self-worth of


respondents, program participants, clients and other evaluation
stakeholders.

5. Responsibilities for the general and public welfare

Evaluators articulate and take into account the diversity of general


and public interests and values that may be related to the
evaluation

extract accessed on 28 July 2009 from


http://www.eval.org/Publications/GuidingPrinciples.asp

Politics and ethics

Most needs analyses are by nature political because they involve


either support for those in power or challenges to those in power.
Ethics will feature in either situation. Additional reading on the
politics of TNA is included later in this section.

2010 Institute for Adult Learning, Singapore 85


Version No: 1.0
Ethical Considerations

Learning Activity

Evidence of improper conduct?

Consider the types of evidence that you would look for in order to determine whether
a training needs analysis has been improperly conducted.

2010 Institute for Adult Learning, Singapore 86


Version No: 1.0
Common errors and obstacles

Common errors and obstacles in needs analysis

Key questions to ask

Before committing to undertake a needs analysis, the assessor


should consider several factors which will impact on its success.

Gupta et al (2007) offer the following questions for the needs analyst
to ask:

1 Can the project use the analysts knowledge, skills and abilities?
2 Does the client seek information for making decisions and taking
action, or is the needs assessment window dressing for decisions
that have already been made?
3 Does the decision maker have the power to take action based on
needs analysis information?
4 Do all those who are involved support change?
5 Do the client and project stakeholders act ethically?
6 Are sufficient resources available to complete the project?

If any answers to the above questions indicate there are issues that
will impede progress, agreement has to be reached on how to deal
with the issues before proceeding with the TNA. If timelines are too
short, funding is inadequate, staff involved are behaving unethically
or there is a high probability of no change as a result of the TNA,
then if possible avoid conducting a TNA.

Common errors have been identified by several experts and include:

1 Insufficient data collection


2 Treating presenting problems only
3 Applying no tools or the wrong tools
4 Trying a quick fix
5 Applying the wrong fix
6 Giving feedback in the wrong language
7 Assuming one problem/one solution
8 Failing to identify non-training issues
9 Failing to educate clients regarding non-training issues

2010 Institute for Adult Learning, Singapore 87


Version No: 1.0
Common errors and obstacles

Twelve common potential needs analysis obstacles identified by Gupta et al (2007)


include

1 Lack of consensus of goals

2 Senior management supports the effort, but line managers or supervisors do not

3 Lack of a sponsor with authority

4 Too much background information to review in a reasonable time

5 Conflict about the types of data that must be collected

6 Bias in favour of a particular instrument

7 Resistance to questioning from middle management or special interest groups


(such as line staff or union groups)

8 Lack of access to those working night shifts or people assigned to restricted work
areas

9 Disagreement among team members about the method to use to implement the
assessment

10 Lack of willingness on the part of people to change when implementing a new


system or technology

11 Attitude of Who has the time to complete another survey?

12 Lack of buy-in to needs analysis results (especially negative or controversial


findings)

2010 Institute for Adult Learning, Singapore 88


Version No: 1.0
Common errors and obstacles

Four red flags

Any of these situations should lead to some type of needs assessment.


Sometimes, an organisation will omit or inadequately conduct the needs
assessment process, especially if management requests it or other
competitors are implementing it. Phillips (1999) adds that when any of
the following situations drive the development of a new program, there
should be concern as they may not be based on an appropriate
foundation.

His four red flag situations are:

1. management requests it
2. most admired companies are pursuing it
3. the program is a general trend in all organisations
4. the training and development staff thinks the program is
needed.

According to Phillips (1999) any one of these situations could translate


into problems for the program if a needs assessment is not conducted.

2010 Institute for Adult Learning, Singapore 89


Version No: 1.0
Common errors and obstacles

Additional reading
The Politics of Needs Analysis
All organisations are political entities. Politics are largely endemic to
every business/organisation. Political intelligence in the context of
change is about politics at work. It is about working with integrity towards
the common good of the organisation.
The needs assessor will benefit by being aware of the role of politics at
work when planning and conducting a TNA. Political intelligence can be
used in a positive way during change. Being politically aware means
being able to recognise the power bases and sources of influence they
possess.
Cook (2005) defines political intelligence as involving:

being aware of power bases

understanding sources of power

recognising levers of influence during change

developing strategies for influence

gaining buy-in from stakeholders

Organisational politics can have a negative impact as well as a positive


effect during change. Symptoms of negative politics at work include
behaviours that demonstrate resistance to change such as open
opposition to the change, self-interest, lack of cooperation, alliance
forming, blaming others, blocking proposals, back-stabbing, unspoken
resentment, building power bases, inward focus, denying change is
occurring and lack of motivation.

In contrast, the positive impact of politics at work can create e.g. a


shared understanding of the need for change, a common sense of
purpose, recognition of the difficulties of change, disagreements aired
openly, debate which makes for better decisions, feedback and
recognition of the impact of change and change agents who influence
others positively.

2010 Institute for Adult Learning, Singapore 90


Version No: 1.0
Common errors and obstacles

Cook (2005) suggests the need to watch out for two sets of behaviours
by people involved in the TNA.
1. their attitude towards the project be it positive or negative.
2. their drive towards activity be it active or inactive.
The following diagram by Cotton (2004) illustrates the drive to action
and positivity.
+
YES people Change
Champions
A I would..
T I could.. I will
T I can
I
T
U Change
Victims Terrorists
D
E I wont
I cant It wont
It cant
_
INACTION ACTION

Characteristics
Change champions have a positive attitude to change and are action
oriented.
Change terrorists are characterized by a negative attitude and high
levels of activity which focus on the negative.
Yes people are positive about change but do not follow through with
action.
Victims have a negative attitude towards projects and change but lack
drive and disengage from change.

Being aware of how people are likely to react during a TNA will help
prepare a needs assessor with how to deal with them.

2010 Institute for Adult Learning, Singapore 91


Version No: 1.0
Trends and developments

Trends in Needs Assessment

The landscape for TNA and needs assessment in general has changed
greatly in the last ten years (Gupta 2007). Professionals who currently
conduct needs assessments work in challenging times. In todays global
fast-paced world, a TNA needs to be agile, fast and responsive.

Learning and performance improvement has become more and more


important for individuals, teams and organisations. The stakes for getting it
right are higher than ever before.

Strategic alignment within organisations and between organisations, locally


and globally and data-based decision making and efficiency are critical.
Some key trends are outlined below.

1. Performance Improvement

Training is a valuable performance improvement tool that builds employee


proficiencies. Management wants to know that money spent on training and
development of the workforce is necessary and cost effective. Training
initiatives must be linked to organisational objectives before performance
changes take place. The concept of Performance Improvement was
discussed in detail in the first section.

Jennings (2009) challenges Learning and Development professionals to


think creatively about business problems to find solutions that are innovative
and cost-effective. He reinforces the need for needs assessors to focus on Resource 7
performance and productivity and offers a Three Step Strategy. (See
Resource 7)

2. Ethical Guidelines

The availability of relevant ethical guidelines by professional associations to


promote ethical practice has increased in recent years. These are discussed
earlier in this section.

3. Advancing technology

Technology plays a larger role in obtaining, managing and analysing data


within organisations in a timely manner and also provides new ways to
collaborate. Automated needs analysis processes offer new ways to collect
and share information via e.g. online surveys, emails, blogging and online
document searches. Organisational decisions, made quickly and
communicated with a few key strokes can have immediate local and global
impact.

2010 Institute for Adult Learning, Singapore 92


Version No: 1.0
6
Argyris, C.
References

1. Barbazette, J. (2006). Training Needs Assessment - Methods, Tools and


Techniques. San Francisco: Pfeiffer
2. Brown, J. (2002).Training Needs Assessment: A Must for Developing an
Effective Training Program. Public Personnel Management. Winter 2002,
Vol 31 Issue 4
3. Cook, S. (2005). Learning Needs Analysis: 12 Part Series. Training Journal
January December. UK: Fenman
4. Cotton, D. (2004). Essentials of Training Design. A Training Journal
Supplement. UK: Fenman
5. Gupta, K. (1999). A Practical Guide to Needs Assessment, Jossey-Bass,
6. California.
7. Gupta, K., Sleezer, C. M. and Russ-Eft, D. (2007). A Practical Guide to
Needs Assessment, Second Edition. San Francisco: Pfeiffer
8. Jennings, C. (2009) The Three Step Strategy. Posted in Strategy,
Leadership, The Training Cycle on 18/06/09 retrieved 14 August 2009 from
http://www.trainingzone.co.uk/topic/strategy/training-not-always-answer-1
9. Mager, R. F. and Pipe, P. (1997). Analysing Performance Problems: or You
really oughta wanna.(3rd ed.). Belmont, CA: Pitman Management and
Training.
10. Moskowitz, M. (2008) A Practical Guide to Training and Development. San
Francisco: Pfeiffer
11. Noe, R. A. (2006). Employee Training and Development, Fourth Edition.
McGraw Hill International Edition.
12. Phillips, J.J. (1999). HRD Trends Worldwide: Shared solutions to compete
in a global economy.
13. Silberman, M. and Auerbach, C. (2006). Active Training A Handbook of
Techniques, Designs, Case Examples and Tips. San Francisco: Pfeiffer
14. Stone, R. D. (2009). Aligning Training for Results. San Francisco: Pfeiffer
15. The American Evaluator Association. (2004) Guiding Principles for
Evaluators retrieved on 28 July 2009 from
http://www.eval.org/Publications/GuidingPrinciplesPrintable.asp
16. The Joint Committee on Standards for Evaluation. (1994). The Program
Evaluation Standards retrieved on 28 July 2009 from
http://www.eval.org/EvaluationDocuments/progeval.html

2010 Institute for Adult Learning, Singapore 93


Version No: 1.0
References

17. The International Society for Performance Improvement, Code of Ethics


(undated) retrieved on 28 July 2009 from
http://www.ispi.org/content.aspx?id=418&terms=code+of+ethics
18. Tobey, D. (2005). Needs Assessment Basics. USA: American Society for
Training & Development.

19. Wilson, K. (2006). TNA Final Report. The Victorian Dairy Manufacturing
Sector. Retrieved 11 August 2009 from
http://www.dairysafe.vic.gov.au/index.php?option=com_content&view=article&id=5
7&Itemid=173

20. Zemke, R. and Kramlinger, T. (1982). Figuring Things Out: A Trainers


Guide to Needs and Task Analysis. Reading, MA: Addison-Wesley

2010 Institute for Adult Learning, Singapore 94


Version No: 1.0
7 Resources

1. Competency Standard for Plan a Training Needs Analysis, TR-HRD-


401C-0 Singapore Workforce Skills Qualification

2. Examples of generic questions

3. Introduction to Case Study

4. Data Collection Planning Guide

5. Review of Learning

6. Joint Committee on Standards for Educational Evaluation, Program


Evaluation Standards

7. The Three Step Strategy by Charles Jennings

8. Post Course Self Assessment

9. TNA Competency Survey

2010 Institute for Adult Learning, Singapore 95


Version No: 1.0
Resource 1

Competency Standard for Plan a Training Needs Analysis,


TR-HRD-401C-0 Singapore Workforce Skills Qualification

2010 Institute for Adult Learning, Singapore 96


Version No: 1.0
Resource 2 Some generic examples of questions to ask your client in an initial
conversation that sets a context of performance improvement.
Client = person requesting training eg line manager, HR manager, senior manager or external person

Business 1. What current business needs or strategies are being affected or caused by the presenting
needs problem?

2. What business problems exist? (look for measures such as amount of increase or
decrease in business indicators, including turnover, productivity, quality, complaints,
waste, customer satisfaction: If not known, it is critical to find out)

3. What is going on in the external environment that relates to the presenting problems (eg
competition, market changes, government regulations)?

4. What other data exists from within your business that may provide information regarding
this business need eg sales figures, quality, productivity, HR information, benchmarking,

5. What changes are you seeking to achieve with this training intervention? What measures
will tell you it has been successful?

6. What business opportunities are inherent in this business need (eg new products or
markets)?

7. What business strategies are you seeking to support with this requested training
intervention?

8. Whats happening in your business that shouldnt be happening?

9. Whats not happening that should be happening?


Performance 1. What results should employees be achieving?
needs
2. What should people be doing differently?

3. What should they stop/start/keep doing?

4. What does perfect performance look like?

5. What does current performance look like?

6. What else might be getting in the way of employees performing as they should, other than
lack of knowledge and skills (non-training issues)?

7. What will be the nature of management support for job application and practice after
training?
Learning 1. What knowledge and skills do you think the targeted employees need to learn to perform
needs the way they should?

2. How important is each knowledge and skill you have listed?

3. How well should they be performing the skills by the end of the training?

Learner 1. What are the targeted learners backgrounds and experience in the subject matter?
needs
2. What are their learning styles?

3. What is their job environment like? (fast-paced, routine, stressful)

4. What are the expectations regarding when and how they will attend the training (after
work, during work, paid, unpaid)?
Adapted from Tobey (2005)

2010 Institute for Adult Learning, Singapore 97


Version No: 1.0
Leave blank

2010 Institute for Adult Learning, Singapore 98


Version No: 1.0
Case Study: Part 1 Resource 3

Introducing Trekkers Adventure Clothing Company


Trekkers is a manufacturing and retail company producing and selling adventure
sportswear. The company has several production factories throughout Asia. Each
factory produces a different line of adventure and sportswear clothing. Retail stores have
been set up in resort areas in Vietnam and Thailand and there is a busy online sales
business based in Hong Kong.

Each factory is run by its own production team. (A team leader coordinates and assigns
work to the team members). The team leaders run their production lines and are
responsible for hiring, training and performance management of their teams. All team
members have been trained in each aspect of the production line.

Andy Tan is the Senior Training Officer at the Jurong West factory in Singapore. He has
been told that

Trekkers is starting a new line of cycling clothing called UrbanTraveller,


The new line will be manufactured in the Jurong West factory.
It will include mens, womens and childrens cycling jerseys, shorts, shoes,
helmets, jackets, gloves, sunglasses and backpacks.

Tracey Tang is the senior production manager and part of the senior management team.
She has asked Andy to attend a meeting to discuss an upcoming training need.

At the meeting, Tracey gives Andy the following information:

The Urban Traveller clothing line will need three new production lines which are
scheduled to start up in 6 months.

Some operations will be similar to existing ones in the factory and a few will be
new processes.

Fifty (50) new employees will need to be hired. Some new employees will work in
the new line and some in the existing line. Some current employees will move to
Urban Traveller line teams and others will remain on existing line teams.

Hiring for the 50 new employees will start in three months. An HR rep will sit on
the panel too.

A Team leader will chair the interview panel that is hiring new staff.

Previous hiring efforts have resulted in some problem employees.

Tracey wants Andy to design and deliver a job interviewing skills training session for the
team leaders by the time the hiring phase starts.

Adapted from Tobey (2005)

2010 Institute for Adult Learning, Singapore 99


Version No: 1.0
Leave blank

2010 Institute for Adult Learning, Singapore 100


Version No: 1.0
Data Collection Planning Guide
TNA Project: Interviewing Skills for Team Leaders, Jurong West Factory, Trekkers Adventure Clothing Co

Type of Need Questions to be asked Data Source Potential Data Collection Data Collection
Methods Method

2010 Institute for Adult Learning, Singapore 101


Resource 4
Version No: 1.0
2010 Institute for Adult Learning, Singapore 102
Version No: 1.0
Learning Review

1) What topic/area was covered?

2) What happened?

3) What did I learn?

4) How will I apply that learning in my workplace?

Resource 5

2010 Institute for Adult Learning, Singapore 103


Version No: 1.0
Learning Review

5) What topic/area was covered?

6) What happened?

7) What did I learn?

8) How will I apply that learning in my workplace?

Resource 5

2010 Institute for Adult Learning, Singapore 104


Version No: 1.0
Learning Review

1. What topic/area was covered?

2. What happened?

3. What did I learn?

4. How will I apply that learning in my workplace?

Resource 5

2010 Institute for Adult Learning, Singapore 105


Version No: 1.0
Learning Review

1. What topic/area was covered?

2. What happened?

3. What did I learn?

4. How will I apply that learning in my workplace?

Resource 5

2010 Institute for Adult Learning, Singapore 106


Version No: 1.0
Learning Review

1. What topic/area was covered?

2. What happened?

3. What did I learn?

4. How will I apply that learning in my workplace?

Resource 5

2010 Institute for Adult Learning, Singapore 107


Version No: 1.0
Learning Review

1. What topic/area was covered?

2. What happened?

3. What did I learn?

4. How will I apply that learning in my workplace?

Resource 5

2010 Institute for Adult Learning, Singapore 108


Version No: 1.0
The Program Evaluation Standards A Summary Resource 6
Utility Standards
The utility standards are intended to ensure that an evaluation will serve the information needs of intended
users.

U1 Stakeholder IdentificationPersons involved in or affected by the evaluation should be identified, so that


their needs can be addressed.

U2 Evaluator CredibilityThe persons conducting the evaluation should be both trustworthy and competent to
perform the evaluation, so that the evaluation findings achieve maximum credibility and acceptance.

U3 Information Scope and SelectionInformation collected should be broadly selected to address pertinent
questions about the program and be responsive to the needs and interests of clients and other specified
stakeholders.

U4 Values IdentificationThe perspectives, procedures, and rationale used to interpret the findings should be
carefully described, so that the bases for value judgments are clear.

U5 Report ClarityEvaluation reports should clearly describe the program being evaluated, including its
context, and the purposes, procedures, and findings of the evaluation, so that essential information is
provided and easily understood.

U6 Report Timeliness and DisseminationSignificant interim findings and evaluation reports should be
disseminated to intended users, so that they can be used in a timely fashion.

U7 Evaluation ImpactEvaluations should be planned, conducted, and reported in ways that encourage
follow-through by stakeholders, so that the likelihood that the evaluation will be used is increased.

Feasibility Standards
The feasibility standards are intended to ensure that an evaluation will be realistic, prudent, diplomatic, and
frugal.

F1 Practical ProceduresThe evaluation procedures should be practical, to keep disruption to a minimum


while needed information is obtained.

F2 Political ViabilityThe evaluation should be planned and conducted with anticipation of the different
positions of various interest groups, so that their cooperation may be obtained, and so that possible attempts
by any of these groups to curtail evaluation operations or to bias or misapply the results can be averted or
counteracted.

F3 Cost EffectivenessThe evaluation should be efficient and produce information of sufficient value, so that
the resources expended can be justified.

Propriety Standards
The propriety standards are intended to ensure that an evaluation will be conducted legally, ethically, and
with due regard for the welfare of those involved in the evaluation, as well as those affected by its results.

P1 Service OrientationEvaluations should be designed to assist organisations to address and effectively


serve the needs of the full range of targeted participants.

P2 Formal AgreementsObligations of the formal parties to an evaluation (what is to be done, how, by whom,
when) should be agreed to in writing, so that these parties are obligated to adhere to all conditions of the
agreement or formally to renegotiate it.

P3 Rights of Human SubjectsEvaluations should be designed and conducted to respect and protect the
rights and welfare of human subjects.

2010 Institute for Adult Learning, Singapore 109


Version No: 1.0
The Program Evaluation Standards A Summary (continued) Resource 6
P4 Human InteractionsEvaluators should respect human dignity and worth in their interactions with other
persons associated with an evaluation, so that participants are not threatened or harmed.

P5 Complete and Fair AssessmentThe evaluation should be complete and fair in its examination and
recording of strengths and weaknesses of the program being evaluated, so that strengths can be built upon
and problem areas addressed.

P6 Disclosure of FindingsThe formal parties to an evaluation should ensure that the full set of evaluation
findings along with pertinent limitations are made accessible to the persons affected by the evaluation, and
any others with expressed legal rights to receive the results.

P7 Conflict of InterestConflict of interest should be dealt with openly and honestly, so that it does not
compromise the evaluation processes and results.

P8 Fiscal ResponsibilityThe evaluators allocation and expenditure of resources should reflect sound
accountability procedures and otherwise be prudent and ethically responsible, so that expenditures are
accounted for and appropriate.

Accuracy Standards
The accuracy standards are intended to ensure that an evaluation will reveal and convey technically
adequate information about the features that determine worth or merit of the program being evaluated.

A1 Program DocumentationThe program being evaluated should be described and documented clearly and
accurately, so that the program is clearly identified.

A2 Context AnalysisThe context in which the program exists should be examined in enough detail, so that
its likely influences on the program can be identified.

A3 Described Purposes and ProceduresThe purposes and procedures of the evaluation should be
monitored and described in enough detail, so that they can be identified and assessed.

A4 Defensible Information SourcesThe sources of information used in a program evaluation should be


described in enough detail, so that the adequacy of the information can be assessed.

A5 Valid InformationThe information gathering procedures should be chosen or developed and then
implemented so that they will assure that the interpretation arrived at is valid for the intended use.

A6 Reliable InformationThe information gathering procedures should be chosen or developed and then
implemented so that they will assure that the information obtained is sufficiently reliable for the intended use.

A7 Systematic InformationThe information collected, processed, and reported in an evaluation should be


systematically reviewed and any errors found should be corrected.

A8 Analysis of Quantitative InformationQuantitative information in an evaluation should be appropriately and


systematically analysed so that evaluation questions are effectively answered.

A9 Analysis of Qualitative InformationQualitative information in an evaluation should be appropriately and


systematically analysed so that evaluation questions are effectively answered.

A10 Justified ConclusionsThe conclusions reached in an evaluation should be explicitly justified, so that
stakeholders can assess them.

A11 Impartial ReportingReporting procedures should guard against distortion caused by personal feelings
and biases of any party to the evaluation, so that evaluation reports fairly reflect the evaluation findings.

A12 MetaevaluationThe evaluation itself should be formatively and summatively evaluated against these
and other pertinent standards, so that its conduct is appropriately guided and, on completion, stakeholders
can closely examine its strengths and weaknesses.

Accessed on 28 July 2009 from http://www.eval.org/EvaluationDocuments/progeval.html


2010 Institute for Adult Learning, Singapore 110
Version No: 1.0
2010 Institute for Adult Learning, Singapore 111
Version No: 1.0
The Three Step Strategy Resource 7
Charles Jennings

L&D leaders need to think creatively about business problems to find solutions that arent
always down the same, well-worn track. Charles Jennings offers three steps to help take
advantage of the current economic climate.

There has never been a better time than now for training and development managers to
review their modus operandi for delivering services to their organisations. They should be
taking the time to explore how they can improve both the efficiency and effectiveness of
their teams efforts.

You may ask why now? Well, the answer is that we know during times of change
whether in changing economic conditions, during times of merger or acquisition, or when
management upheavals occur organisations tend to be more open to altering existing
practices or adopting new approaches than they are when the times are good and
organisational life is stable.

Great opportunities
When things become unstuck it is worth listening to the advice of Warren Buffett, a
master of taking advantage in unsettled economic times. Buffett says: Great opportunities
come around when excellent companies are surrounded by unusual circumstances. And
its not just companies but all organisations, whether theyre driven by the need to make
a profit or the need to provide a service.

Training Needs Analyses (TNAs) have done as much as anything else to build and
maintain a gap between training and development professionals and their
business-focused customers.

So, what big changes can training and development managers make in these times that
will result in a positive impact on the value they add to their organisations? To answer this
question it is worth reflecting on what their organisations expect their training and
development departments to do.

For a start, they need to understand that most business managers are not particularly
interested in training and learning per se. Some may be, but they tend to be the exception
rather than the rule. During my career I have only known a very few line managers who
were keen to engage in discussions about instructional design or the advances in our
understanding of pedagogy. As professionals, those of us working in training and
development need to know about these things and behind closed doors we expect to
discuss them.

However, theyre just not topics with which to engage a business manager. That said,
most managers will be extremely interested in the output of the learning process in the
improved performance and productivity of individuals or groups within their remit, and in
the impact of training and development on business results. If they are senior leaders they
will also be passionate about what the training and development department is doing for
the performance of the organisation as a whole.

2010 Institute for Adult Learning, Singapore 112


Version No: 1.0
The Three Step Strategy (continued) Resource 7
Step one
To take advantage of the current climate, the first thing that training and development
managers need to do is to stop talking learning and start talking business. Training Needs
Analyses (TNAs) have done as much as anything else to build and maintain a gap
between training and development professionals and their business-focused customers.
The TNA process has an implicit assumption that the solution to the business problem is
going to involve training and, usually, that the training will be structured into some form of
course or programme. We need to step out of this noose.

Training and development managers need to analyse problems that line managers bring
to them differently to the way theyve done so with the TNA model. They need to get to the
root cause of the problem in an open and consultative way and work with the problem
owner to agree the best solutions without thinking about training as the final end point.
In my experience the best solutions sometimes involve learning interventions, sometimes
even courses and programmes, but just as often the presenting problems are best
addressed by changes in working practices, new processes, or even changes in the way
the manager manages!

The answer is to think performance and productivity not training.

Step two
The second step managers can take to improve the business impact of training and
development is to ensure that training and development accountabilities and resources
are fully aligned with business strategy. Many organisations flip-flop the organisational
structure for their training/L&D functions from centralized to devolved models and back
again on a regular cycle, always hoping that the organisational change will result in
greater business impact.

However, track records tell us that neither model provides the silver bullet alone. In fact,
organisations in the USA spend about $100 billion on training each year (and have done
so annually for almost the past 20 years) and still only about 10% of that training and
development investment results in transfer to improvements in the workplace.

Training and development departments (need) to become leaders rather than


followers, consultants rather than fulfilment services.

They need to develop high level performance consulting skills and move away from being
order takers. Some organisations with centralized T&D do well, some not so well. The
same goes for organisations where training and development is devolved into business
units. So the answer doesnt lie in a defined organisational model.

2010 Institute for Adult Learning, Singapore 113


Version No: 1.0
The Three Step Strategy (continued) Resource 7

Where the answer does lie, however, is in the development of a robust governance model
for training and development. Every organisation should make sure that it has a
meaningful governance board, or set of governance boards, that set and align the overall
training and development strategy (for the T&D managers to execute), and ensure that
training and development departments are business-driven, agile, innovative, efficient and
effective. The majority of members of the governance boards should be senior business
managers. The boards certainly should not be HR/T&D talking shops.

Step three
The third strategy training and development managers need to adopt is for their
departments to become leaders rather than followers, consultants rather than fulfilment
services. They need to develop high level performance consulting skills and move away
from being order-takers. To be effective they also need to understand whats available to
them in todays toolbox of approaches and, particularly, technologies that can be brought
to bear to build employee capability and help solve identified business problems.

Then, once they have determined whats possible and appropriate, they need to be pro-
active in designing and deploying innovative solutions. Here I would specifically stress the
need for training and development managers to understand the importance that workplace
learning and just-in-time learning (including performance support and business process
guidance) play in improving employee performance, and also the importance of the social
aspect of learning of learning through others and collaboration, and of the many
learning technologies that, given the right circumstances, have the potential to support
workplace and social learning as well as formal learning.

Apart from addressing the issues I have already discussed, training and development
managers need above all to ensure that their own teams are competent, capable and
have the professional skills to deliver a service that provides their organisation with a
training and development service that is a strategic business tool.

To do this, each and every member of their teams needs, at least, to be a competent
specialist in the areas of:
a) adult learning;
b) performance consultancy;
c) learning technologies.

Without these capabilities, any training and development manager will struggle to deliver
real value to their organisation.

Charles Jennings is the former chief learning officer for Reuters and now runs his own
specialist learning and performance consultancy practice, Duntroon Associates.

Accessed from http://www.trainingzone.co.uk/topic/strategy/training-not-always-answer-1


on 14 August 2009.

2010 Institute for Adult Learning, Singapore 114


Version No: 1.0
leave blank

2010 Institute for Adult Learning, Singapore 115


Version No: 1.0
Post-course SELF assessment
for Plan a Training Needs Analysis

Please rank the following learning outcomes twice on a scale of 1-5.

The first ranking is a self-assessment regarding your level of knowledge and skill
(1= I dont know very much about this subject, 5 = I am expert in this subject)

The second ranking is the importance of each learning outcome to you, based on your job
responsibilities (where 1 = not important at all, 5 = critical).
A B
Learning Outcomes My current Importance to
knowledge me
I am able to (1-5) (1-5)

PS1: scope and requirements of TNA 1 2 3 4 5 1 2 3 4 5


21. define TNA within a framework of performance
improvement
22. explain purposes of TNA
23. describe four approaches to TNA
24. outline phases or steps in a typical TNA process
PS2: prepare for TNA
25. align a training request to business, performance,
learning and learner needs
26. prepare a TNA project plan
27. plan for data collection
28. identify types of data
29. describe a range of data sources
PS3: select data collection methods
30. select appropriate qualitative data collection methods
31. select appropriate quantitative data collection methods
32. identify factors influencing selection
33. discuss guidelines for implementing data collection
PS4: select data analysis methods
34. apply basic descriptive statistics to quantitative data
35. review findings resulting from data analysis
36. explain the difference between findings and
recommendations
PS5: prepare a TNA report
37. write a TNA report containing key information
38. examine ethical issues and standards
39. discuss some common obstacles
40. describe trends and developments in TNA
Totals
Resource 8

2010 Institute for Adult Learning, Singapore 116


Version No: 1.0
Survey: Competencies required to implement an effective needs analysis

In Part 1, please rate the importance of the following competencies when implementing an
effective needs analysis, using the key below. Tick the box of your choice where
1 = strongly disagree 2 = disagree 3 = somewhat agree 4 = agree 5 = strongly agree
NA = not applicable

In Part 2, rate your own knowledge and skills and abilities in relation to implementing an
effective needs analysis (where 1 = I dont know much about this to 5 = I am highly skilled & capable)
Part 1 Part 2
Needs assessors/analysts must have 1 2 3 4 5 N 1 2 3 4 5 N
A A

1. Knowledge about the work environment

2. Knowledge about performance improvement

3. Skill in consulting, problem-solving, negotiating


and advocating for change

4. Skill in collecting data using such methods as


survey, interviews and observation

5. Skill in analysing qualitative and quantitative


data

6. Skill in exchanging information verbally, in


writing and through presentations

7. Ability to collaborate effectively with the client


and the project stakeholders

8. Knowledge and skills in applying relevant


needs assessment ideas and models

9. Ability to manage the needs assessment


project effectively

10. Ability to negotiate outcomes that maximise


performance and learning

11. Knowledge of ethical standards and skills in


following these standards
Total

SOURCE:

Resource 9

2010 Institute for Adult Learning, Singapore 117


Version No: 1.0
8 Glossary

Assumed skills and Assumed skills and knowledge are skills and knowledge that learners should
knowledge have and are assumed to have before they attend the training programme.
They are outlined in the competency standards.

Competency A competency is a measurable set of knowledge, skills and attitudes that a


person needs to perform a task effectively.

Competency A competency category refers to the broad area or function in which these
category competencies are mainly found.

Competency A competency descriptor gives an overview of what the competency standard


descriptor covers as well as the context in which the skills, knowledge and attitudes
would be useful. It is outlined in the competency standards.

Competency Competency elements describe the specific skills, knowledge and/or


elements attributes required for the particular competency category.

Competency level The competency level reflects the level of complexity and depth of learning
required by the competencies in the standard. It is outlined in the
competency standards.

Competency profile The competency profile typically refers to a group of competencies that are
required for a job or a role.

Continuing Continuing Education and Training (CET) refers to educational programmes


Education and for adults, usually at the post-secondary level and offered as part-time or
Training (CET) short courses in occupational subject areas. Also see Pre-employment
Training (PET).

Competency The Competency Standards is a document that refers to the skills, knowledge
Standards (CS) and attitudes needed to perform a job task and describe the acceptable levels
of performance.

Competency unit A competency unit describes a particular work role, duty or function, which
forms the smallest group of skills, knowledge and abilities set able to be
recognised separately for certification.

Continuing Educational programmes for adults, usually at the post-secondary level and
Education and offered as part-time or short courses in occupational subject areas.
Training (CET)
Credit A unit of measure assigned to courses or course of equivalent learning.

Curriculum, The CTAG is a document that provides training and assessment advice to
Training and achieve effective training and assessment.
Assessment Guide
(CTAG)
Continued over

2010 Institute for Adult Learning, Singapore 118


Version No: 1.0
Glossary

Dimensions of The dimensions of competency covers all aspects of work performance.


competency The five dimensions of competency are:
- Task skills
- Task management skills
- Contingency management skills
- Job and role management skills.
- Transfer skills

Evidence sources This section gives examples of tasks, observations, documents etc that
can be used as evidence for assessing the particular competency
element. It is outlined in the competency standards.

Generic skills Generic skills are transferable and applicable to all industries.

Industry skills Industry skills refers to know-how that is applicable to a particular


industry or industry cluster.

Learning outcomes The skills and knowledge a learner should be able to demonstrate as a
result of having undertaken training and/or assessment. It must be
measurable, clear and observable.

Occupational skills Occupational skills are job specific skills set that are needed to perform
tasks for a specific sector of an industry.

Performance Performance criteria refer to the standards of performance or behaviours


criteria required in order to demonstrate competency.

Pre-employment Pre-employment Training (PET) refers to educational programmes that


Training (PET) prepare learners for entry into the workforce. This includes secondary,
pre-university, polytechnic and university education.

Qualifications Qualifications are formal certification issued by a relevant approved


body, in recognition that a person has achieved learning outcomes
identified by the industry.

Range and context Range and context provides the type of contexts to which the
performance criteria apply. They cover items that are achievable or to be
performed across by workers, such as types of equipment, products and
services, types of customers. It is outlined in the competency standards.

Recognition of Recognition of Prior Learning acknowledges a persons skills and


Prior Learning knowledge acquired through previous training, work or life experience.
(RPL) This may be used to grant status or credit in a subject or module.

2010 Institute for Adult Learning, Singapore 119


Version No: 1.0
9 The Assessment Requirements
CU 4: Plan a Training Needs Analysis

Candidates will be assessed on their competency in CU4 Plan a Training Needs Analysis in
accordance with the following specifications:

Written Assignment
Q 1 (1-2 pages)
Prepare a Trekkers TNA project plan that outlines what will happen in the TNA. Use preliminary
information provided by Tracey (the client). Include

1 the purposes of the Trekkers TNA project


2 a description of the TNA phases to be conducted,
3 an estimate of the timelines for the conduct of relevant phases of the TNA

Q 2 (1500 words)
Write a Training Needs Analysis report for Trekkers stakeholders using the data, discussions
and findings from the Trekkers case study. Your report must include

1. Main data sources and type of data gathered


2. Data collection methods selected and why
3. Main findings of training and non training issues
4. Training and non training recommendations
5. Critical success factors and resource constraints
6. Implementation plan (timelines) for training recommendations

Q3
During the TNA process at Trekkers, one of the team leaders guesses the name of a team
member who made negative comments in a confidential interview. The team leader plans to
correct the persons misperceptions.

A Explain whether this is an ethical issue. If so, how will you deal with this issue?
B Describe two ethical principles and relevant professional standards that will inform
your decisions and actions?

Written Assessment
The candidate will complete a written test. This is an open-book assessment. Candidates will be
required to answer questions about
responding to a client request for training
reporting data using basic descriptive statistics
trends in needs analysis

.....end....

2010 Institute for Adult Learning, Singapore 120


Version No: 1.0

You might also like