You are on page 1of 38

PROJECT RISK MANAGEMENT AUDIT

WORK PROGRAM
TABLE OF CONTENTS
Introduction ...............................................................................................................................................................3
Using this Work Program .......................................................................................................................................3
Background Information.........................................................................................................................................3
What is a project? .............................................................................................................................................3
What is project management? ..........................................................................................................................3
What is a program? ..........................................................................................................................................4
What is program management? .......................................................................................................................4
What is the relationship between program management and project management? ......................................5
What is a program management office? ...........................................................................................................5
What are some of the factors that influence projects? .....................................................................................5
What is a project lifecycle? ...............................................................................................................................6
What is a program lifecycle?.............................................................................................................................6
Program Management Office (PMO) .......................................................................................................................8
Risk Management ..................................................................................................................................................8
Change Control ......................................................................................................................................................8
Communication ................................................................................................................................................... 10
Issue Tracking .................................................................................................................................................... 12
Cost Management .............................................................................................................................................. 13
Schedule Management ....................................................................................................................................... 14
Quality Management........................................................................................................................................... 16
Resource Management ...................................................................................................................................... 18
Financial and Benefits Realization ..................................................................................................................... 19
Project Lifecycle Management (PLM) .................................................................................................................. 20
Key Risk Considerations..................................................................................................................................... 20
Planning and Initiation ........................................................................................................................................ 20
Requirements Analysis ....................................................................................................................................... 22
Design ................................................................................................................................................................. 23
Development ....................................................................................................................................................... 26
Testing ................................................................................................................................................................ 28
Implementation and Roll-Out .............................................................................................................................. 34
Post-Implementation ........................................................................................................................................... 38

2 Source: www.knowledgeleader.com
INTRODUCTION
Using this Work Program
This work program is intended to provide Client XYZ’s internal audit team with guidance and direction when
executing audits of projects and/or program management office activities. Please understand that this work program
is not a prescriptive list of work step requirements; rather, it is a menu of options from which to choose controls and
work steps based on the scope (and risk) of a particular audit. The content of the work program should be
customized and/or expanded to directly align with the objectives of your audit. While most of the items outlined
herein can be leveraged across a variety of both business and IT projects, there are certain work steps that directly
align with IT projects (these are denoted with the “for IT projects” designation).

Background Information
To effectively use this work program, the auditor does not need to be a project management expert; a foundational
understanding of core project management principles should be sufficient. This section of the work program is
intended to provide our audit team with an overview of some basic background and concepts associated with
projects and programs, and the dependencies/relationships between them. Much of this content has been excerpted
from standards and guidelines published by the Project Management Institute (PMI), which is a globally recognized
authority on project management principles and leading practices.

PMI is the world’s leading not-for-profit membership association for the project management profession, with
more than half-a-million members and credential holders in more than 185 countries. Its worldwide advocacy for
project management is supported by its globally recognized standards and credentials, extensive research
program, and professional development opportunities.

What is a project?

A project is a temporary endeavor undertaken to create a unique product, service or result. The temporary nature
of projects indicates a definite beginning and end. The end is reached when the project’s objectives have been
achieved, when the project is terminated because its objectives will not or cannot be met, or when the need for the
project no longer exists. Temporary does not necessarily mean short in duration. A project typically addresses the
tactical execution steps/activities that support people, processes and/or technology.

What is project management?

Project management is the application of knowledge, skills, tools and techniques to project activities to meet the
project requirements. Project management is accomplished through the appropriate application and integration of
logically grouped project management processes across five key process areas (described below).

3 Source: www.knowledgeleader.com
Initiation • Develop Project Charter • Develop Preliminary Project Scope Statement

• Develop Project • Activity Resource • Risk Management


Management Plan Estimating Planning Risk
Identification
• Scope Planning • Activity Duration
Estimating • Qualitative Risk Analysis
• Scope Definition
• Schedule Development • Quantitative Risk
• Create Work Breakdown
Analysis
Structure • Cost Estimating
Planning • Risk Response Planning
• Activity Definition • Cost Budgeting
• Plan Purchases and
• Activity Sequencing • Quality Planning
Acquisition
• Human Resources
• Plan Contracting
Planning
• Communications
Planning

• Direct and Manage • Acquire Project Team • Request Seller


Project Execution Responses
Executing • Develop Project Team
• Perform Quality • Select Sellers
• Information Distribution
Assurance

• Monitor and Control • Schedule Control • Manage Stakeholders


Project Work
• Cost Control • Risk Monitoring and
Monitoring • Integrated Change Control
and • Perform Quality Control
Control
Controlling • Contract Administration
• Manage Project Team
• Scope Verification
• Performance Reporting
• Scope Control

Closing • Close Project • Contract Closure

Managing a project typically includes identifying requirements and addressing the various needs, concerns and
expectations of the stakeholders as the project is planned and carried out. Further, it is essential to balance
competing project constraints, which may include scope, quality, schedule, budget, resources and risk. The
relationship among these factors is such that if any one factor changes, at least one other factor is likely to be
affected. For example, if the schedule is shortened, often the budget needs to be increased to add additional
resources to complete the same amount of work in less time. If a budget increase is not possible, the scope or
quality may be reduced to deliver a product in less time for the same budget.

What is a program?
A program is defined as a group of related projects managed in a coordinated way to obtain benefits and control
not available from managing them individually. Programs may include elements of related work outside the scope
of the discrete projects in the program. Programs, like projects, are a means of achieving organizational goals and
objectives, often in the context of a strategic plan. A project may or may not be a part of a program but a program
will always have projects.

What is program management?


Program management is defined as the centralized coordinated management of a program to achieve the program’s
strategic objectives and benefits. It involves aligning multiple projects to achieve the program goals and allows for
optimized or integrated cost, schedule and effort. Projects within a program are related through the common

4 Source: www.knowledgeleader.com
outcome or collective capability. Project management focuses on the interdependencies and helps to determine the
optimal approach for managing them. Actions related to these interdependencies may include:
• Resolving resource constraints and/or conflicts that affect multiple projects within the program
• Aligning organizational/strategic direction that affects project and program goals and objectives
• Resolving issues and change management within a shared governance structure

What is the relationship between program management and project management?


Essential program management responsibilities
include the identification, monitoring and control
of interdependencies between projects, dealing
with the escalated issues among projects that
comprise the program, and tracking the
contribution of each project and the non-project
work to the consolidated program benefits. The
interactions between a program and its
components tend to be iterative and cyclical. The
graphic at right shows the interaction of
information flow between program management
and project management.

What is a program management office?

The program management office (PMO) is a crucial portion of the program’s infrastructure. The PMO supports the
program manager with the management of multiple, unrelated projects. While there are many varieties of PMOs
within organizations, for the purposes of this work program, the PMO provides support by:
• Defining the program management processes that will be followed
• Managing schedules and budgets at the program level
• Defining the quality standards for the program and for the program’s components
• Providing methodology and documentation standards
• Providing centralized support for managing changes and tracking risks and issues

In addition, for long, risky or complex programs, the program management office may provide additional support in
the areas of managing personnel resources, managing contracts and procurements, legal support, and other
support as required.

What are some of the factors that influence projects?

Environmental factors refer to both internal and external environmental factors that surround or influence a project’s
success. These factors may come from any or all of the business units/entities involved in the project. Environmental
factors may enhance or constrain project management options and may have a positive or negative influence on
the outcome. These factors may include:
• Organizational culture, structure and processes
• Government or industry standards
• Infrastructure
• Existing human resources
• Marketplace conditions

5 Source: www.knowledgeleader.com
Projects and project management take place in an environment that is broader than that of the project itself.
Understanding this broader context helps ensure that work is carried out in alignment with the goals of the company
and managed in accordance with the established practice methodologies of the organization.

What is a project lifecycle?

A project lifecycle is a collection of generally sequential and sometimes overlapping project phases whose name
and number are determined by the management and control needs of the organization or organizations involved in
the project, the nature of the project itself and its area of application. While every project has a definite start and a
definite end, the specific deliverables and activities that take place in between will vary greatly with the project. The
lifecycle provides the basic framework for managing the project, regardless of the specific work involved. No matter
how large or small, simple or complex, all projects can be mapped to the following structure:
• Starting the project
• Organizing and preparing
• Carrying out the project work
• Closing the project

What is a program lifecycle?

Programs, just as projects, have an initiation effort, a development effort and an end. The details within those three
spans are dependent on the type of program. The program begins either when funding is approved or when the
program manager is assigned. The program is ended by the steering committee when all components within the
program have successfully produced their deliverable and that final product is either delivered to the customer or
transitioned into an operations phase. The type of program being managed may influence a program lifecycle;
however, the major lifecycle phases and their deliverables remain similar. Five main phases are identified in a
program lifecycle:
• Pre-program preparations
• Program initiation
• Program setup
• Deliver of program benefits
• Program closure

The components within the program can begin at any time after the program begins and generally end before the
program itself ends. The product of these components is integrated into the final product being developed by the
program (as shown in the graphic below).

6 Source: www.knowledgeleader.com
7 Source: www.knowledgeleader.com
PROGRAM MANAGEMENT OFFICE (PMO)

Operating
Expected Design Effectiveness
Objective/Control Effectiveness Work
Practices Work Steps
Steps

Risk Management

Processes are in place • Identify, document • Hold discussion with • For a sample of
to identify, assess, and and escalate PMO management to programs or projects
mitigate risks. identified gain an understanding within a program, obtain
project/program of the risk management and review the
Risk: risks. process. Evaluate the associated risk log(s) to
level of participation in verify that risks are
Without a robust risk • Assess the impact of
the development and identified, captured,
management and identified risk on
the ongoing updating of tracked, and ultimately
mitigation process, critical milestones
the risks. Obtain and mitigated.
there is greater and overall success.
review associated
probability that a risk • For the sample selected
• Alternatives are policies and procedures.
may negatively impact above, confirm that
developed to Confirm that policies
scope, budget, quality, relevant risks were
mitigate program reference risk
or timing. communicated to
and/or project risks. prioritization and
management. As
communication
• Document risk necessary, review key
requirements.
issues and escalate artifacts and meeting
to senior • Confirm that risk minutes to confirm that
management as management efforts are management
appropriate. designed to address acknowledges and
both project-level risks accepts the risks or has
• Develop and
and company risks plans to mitigate the
document risk
associated with risks to an acceptable
mitigation activities implementing the level.
(including owners program/project.
and target
Further, confirm that risk
remediation dates)
management activities
to reduce impact of
consider both internal
identified risks. and external factors.
• Appropriate tools are • Verify that the PMO is
in place to facilitate
staffed by resources at
risk analysis and
an appropriate level
mitigation plans.
(i.e., they are able to
• The program enforce accountability).
management office
is officially
recognized, staffed
and supported by
senior management.

Change Control

Scope is clearly defined • A description or • Hold discussion with • For a sample of


and recognized. charter is created PMO management to programs or projects
detailing pertinent gain an understanding within a program, obtain
Risk: information on the and review the detailed
program and/or description or charter.

8 Source: www.knowledgeleader.com
Operating
Expected Design Effectiveness
Objective/Control Effectiveness Work
Practices Work Steps
Steps
The absence of a project (e.g., scope, of the project change Confirm materials
centrally defined, objectives, roles and control process. adequately support
documented and responsibilities, project objectives. As
• Verify whether or not a
maintained program critical processes, necessary, verify that
detailed description or
and/or project scope risks). roles/responsibilities and
charter is required.
increases the risk that decision-making
• The organization as
the actual scope of • Determine if change authority are clearly
a whole is aware of
work performed is control practices have defined.
and recognizes the
inconsistent with been designed to
effort. • Validate that key
management (and end- address program-level
documents sampled
user) intentions or • A scope/statement changes, project-level
above were reviewed
needs. of work has been changes and/or activity-
and approved by
defined and level changes.
management.
documented to
• Confirm that decision-
outline expectations,
making authority is
cost and terms of
clearly defined and
the project.
communicated.
• Decision-making
authority has been
outlined and
communicated to
stakeholders (e.g.,
creation of a RACI
chart).

Scope and change • An overall change • Hold discussion with • For a sample of
management policies, process is in place PMO management to changes to the program
processes, and that ensures all gain an understanding or changes to projects
procedures are in changes (whether to of the program and/or within a program
place. scope, schedule, project change process (current or completed),
resources, cost, (including any tools obtain and review
Note: change
quality, risk or used to facilitate the change request forms
management in the
contract process). and confirm that
context of a program
administration) are documentation clearly
and/or project may refer • Verify that the change
properly handled outlines the
to scope, schedule, process covers changes
and reflected in the requirements and
resources, cost, quality, to scope, schedule and
project plan and impacts of the change.
risk or contract cost (at a minimum).
program and/or
administration. • For the sample selected
project • Verity that scope
above, confirm that
documentation. changes are required to
Risk: management approval is
be approved by
• Changes are evidenced and centrally
If changes are not appropriate levels.
coordinated across retained.
accurately and
all areas of the
completely identified, • Confirm that relevant
program and/or
assessed, prioritized, plans/documentation
project and the
and resolved, the have been updated to
organization.
solution delivered may reflect the change
not meet the needs of • Scope changes are details sampled above.
the user community or reviewed and
may adversely impact approved by
appropriate

9 Source: www.knowledgeleader.com
Operating
Expected Design Effectiveness
Objective/Control Effectiveness Work
Practices Work Steps
Steps
the timing, cost or individuals or
functionality delivered. committees.
• Appropriate tools are
in place to facilitate
the overall change
control processes.

Communication

Identify information • Stakeholders and • Hold discussion with • Obtain and review
needs of the their PMO management to required
stakeholders, roles/requirements determine the level of communications for a
determine a suitable have been identified. upward and downward sample of programs or
means of meeting communication. projects within a
• All audiences have
those needs and deliver program (e.g., status
been identified and • Determine if a formal
the required report).
assessed relative to communication plan is
information.
their level of impact established. • For the sample selected
from the effort and above, confirm that
Risk: • Gain an understanding
anticipated communication
of the process in place
The absence of a resistance. plans/documents are
to communicate
formal communication updated and distributed
• A communication program and/or project
plan increases the risk when project changes
plan outlines how, status, changes and
that key project are made. Verify that
what, when and by completion with
messages (such as communication
whom stakeholders. Verify
status, deliverables, messages appropriately
communications and that communication
results and necessary describe the change
messages will be activities are designed
approvals) will not be requirements, impact,
delivered to the to be iterative and
obtained by the etc.
stakeholders and updated/refreshed as
appropriate individuals
intended audiences. changes are made. • For the sample selected
in a timely manner,
above, confirm that
leading to less than • The communication
necessary approvals
optimal decisions. plan is actively
have been obtained.
managed to ensure
that all stakeholders • As necessary, interview
(project team and key stakeholders to
external) are assess the quality and
effectively effectiveness of
communicated to communication
relative to their activities.
anticipated level of
impact and
resistance.
• Periodic progress
reports allow the
organization to
review the progress
of the program
and/or project and to
provide input based
on the information
provided that may

10 Source: www.knowledgeleader.com
Operating
Expected Design Effectiveness
Objective/Control Effectiveness Work
Practices Work Steps
Steps
be valuable to the
project. Progress
reports for senior
management and
project personnel
are regularly
produced and
distributed, and
content accurately
depicts the current-
state environment
(i.e., progress
reports provide a
holistic, unfiltered
view).
• Team leads should
be involved in the
defining and
documenting of
internal and external
communication
objectives as they
relate to their
respective
businesses.
• A formal approval or
closure process for a
phase of the
program and/or
project allows the
team to demonstrate
what has been
accomplished and
the business users
to agree that what
was created is what
they requested.

Program office • Progress meetings • Hold discussion with • Obtain and review
facilitates update and should be conducted PMO management to status meeting
status meetings to team regularly for teams, determine the process documentation (e.g.,
members. business personnel by which status is minutes, status reports,
and executive level communicated for team etc.) for a sample of
Risk: management. The members and current programs or
scope of these stakeholders projects within a
The absence of an meetings should program.
effective status meeting include milestone
increases the likelihood • Participate in a sample
tracking, key dates
that dependencies, of progress meetings to
and relevant
issue/risk indicators or assess the quality of the
projections.
other aspects of the discussion. Confirm that
program and/or project the following items are

11 Source: www.knowledgeleader.com
Operating
Expected Design Effectiveness
Objective/Control Effectiveness Work
Practices Work Steps
Steps
will go undetected by discussed (at a
the management office. minimum):
As a result, the − Project
appropriate action may timeline/upcoming
not be taken, potentially milestones
impacting time, cost or
resources. − Accomplishments
− Challenges/issues
− Risks
− Scope changes (if
necessary)
− Budget analysis
− Deliverables
− Communication plan
updates

Issue Tracking

Issues are captured, • There is a repository • Hold discussion with • For a sample of current
tracked and maintained for outstanding PMO management to programs or projects
in a central repository. program and/or gain an understanding within a program, obtain
project issues. of the process and tools a sample of issue
Issues status is
used for tracking issues, reports and verify that all
updated regularly. • The central facility
including logging issues, relevant information is
has the appropriate
updating issues and the being communicated
controls to allow only
closure process. within the reports and to
relevant team
Risk: appropriate project
members to update • Determine if the status
stakeholders.
The absence of a information and the of issues is periodically
formal issue information is reviewed with project • For the sample selected
management process, updated on a management and above, confirm that
consistently performed periodic basis. steering committee issues are periodically
and enforced by members. updated and tracked to
• The status of issues
program management, gauge status.
should be generated • Determine which
increases the risk that a
on a periodic basis members of • For the sample selected
critical path issue may
to allow individuals management are above, verify that
go unidentified or
in the organization to included on typical issue management approves
unresolved, potentially
review the accuracy tracking distributions. the issue listing.
restricting the team’s of the information on
ability to deliver • Confirm that access to
the issues.
required functionality on relevant issue
time and within budget • Reports on issues documentation is
or per requirements. are created restricted based on
periodically. business need.
• Reports are
distributed to
appropriate
personnel.

Issues are being acted • Issues are resolved • Hold discussion with • Obtain a listing of issues
on in the priority based on order of PMO management to that were escalated from

12 Source: www.knowledgeleader.com
Operating
Expected Design Effectiveness
Objective/Control Effectiveness Work
Practices Work Steps
Steps
established and priority and time gain an understanding a sample of current
reported at periodic sensitivity. of the issue prioritization programs or projects
intervals. and escalation process. within a program. Verify
• An escalation
that issues were
Unresolved issues are process is in place • Confirm the design of
assigned a priority and
escalated as defined. that allows issues to the issue incorporates
followed the defined
be handled in an requirements/guidelines
escalation process
Risk: efficient manner and for identifying issues,
(according to the
in accordance with opening issues, tracking
Failure to prioritize, assigned priority).
defined risk issues, monitoring
escalate and respond tolerances. issues and closing • Confirm that issues
to identified issues in a
issues. sampled above were
timely manner may • Unresolved issues
included in relevant
negatively impact follow the escalation • Understand
communications.
progress. Persistent, process. management’s process
unresolved issues may to identify trends and • Confirm that issue
• Processes are in
be detrimental to metrics associated with trends and metrics are
place to evaluate
program and/or project identified issues. communicated to key
issues to identify
success. stakeholders and
trends, metrics, etc.
management.
Results of this
analysis are shared
with key
stakeholders.

Cost Management

Costs are identified, • A detailed, end-to- • Hold discussion with • For a sample of current
managed, monitored end financial cost PMO management to programs or projects
and appropriately estimate, including gain an understanding within a program, obtain
accounted for all aspects of the of the process of and review
throughout the life of program and/or managing costs. documentation
the program and/or project exists encompassing financial
• Understand the tools
project. (outsides services, estimates and budget
used to report, monitor
hardware, software, reporting. Determine if
Risk: and analyze budget and
internal personnel the budget information is
cost information.
The absence of costs). distributed to the
detailed tracking of • Verify that the design of appropriate project
• A detailed report
budget versus actual cost-management stakeholders.
showing budgeted
costs may increase the activities includes a
versus actual • For the sample selected
risk that costs are not requirement for
expenses tracked by above, confirm that
properly controlled or comparing budgeted
milestones and changes are
that the program and/or costs and actual costs.
deliverables per the appropriately reflected in
project will not be
project plan. • Understand the the budget (leverage
completed on time and
mechanism for sample testing from
within budget. • Changes to scope
communicating cost earlier in this work
are reflected in
Without tracking information to program, as necessary).
updated financial
financial progress stakeholders. Determine
estimations. • For the sample selected
against time, there is an if thresholds have been
above, verify that
increased risk that the • A process for defined as a means for
communication activities
budget will run out communicating alerting management to
are aligned with
before the scheduled status of budget certain cost-
requirements (i.e.,
completion date. versus actual (based management scenarios.
stakeholders should be

13 Source: www.knowledgeleader.com
Operating
Expected Design Effectiveness
Objective/Control Effectiveness Work
Practices Work Steps
Steps
Without tracking on time and alerted if program and/or
financial progress milestones) to key project financials deviate
against milestones, program and/or beyond acceptable
there is an increased project owners and thresholds).
risk that the costs stakeholders.
(based on time) will be
• An attention to the
misleading if the
financial reporting
deliverables scheduled
implications and
for completion during
emphasis on
that time are
ensuring correct
incomplete.
identification and
segmentation of
capital and expense
costs.
• Costs (time and
expenses) should be
managed and
monitored at the
team, project and
program level.
• Appropriate tools are
in place to report,
monitor and analyze
budget on an
ongoing basis.
• Mechanisms are in
place to alert key
stakeholders if
program and/or
project financials
deviate beyond
acceptable
thresholds.

Schedule Management

A detailed, integrated • A detailed project • Hold discussion with • For a sample of current
project plan has been plan, which PMO management to programs or projects
developed and is used documents: gain an understanding within a program, obtain
to monitor and manage − Project tasks of the process for the plan and verify that
progress. developing project tasks, major milestones,
− Major milestones plans. % complete, estimate to
Risk: − % completion complete, target dates,
• Determine if a formal
− Estimate to key risks and resources
Without an integrated, baseline is established
completion are identified. In
effort-driven project to measure progress.
addition, verify that
plan, a holistic view of − Target dates • Understand the team’s program and/or project
the program and/or
− Key risks (to the process for monitoring assumptions are
project, considering all documented.
project and the and managing the
dependencies and
business) project plan.
resource needs, cannot

14 Source: www.knowledgeleader.com
Operating
Expected Design Effectiveness
Objective/Control Effectiveness Work
Practices Work Steps
Steps
be attained. As a − Updated scope • For the sample selected
result, milestones (or − Resources above, validate that
delivery dates) may project plans/tasks are
continue to be missed • Program and/or regularly updated to
or extended. project assumptions reflect changes.
are documented.
Without a standard • Confirm that the sample
methodology for • A baseline to project plans (from
delivering the project measure progress above) have been
and for managing the has been approved.
program (i.e., project established.
planning and tracking • Appropriate tools are
methodologies), project in place to create,
plans may be monitor and manage
developed based upon the project plan.
an inconsistent
understanding of tasks
required to implement,
potentially
compromising the
accuracy and
completeness of the
timeline.

Tasks have been • Scheduling should • Hold discussion with • Obtain and review the
identified, scheduled, review all PMO management to schedules for a sample
and are being components of a gain an understanding of current programs or
monitored on an program and/or of the project-scheduling projects within a
ongoing basis. project to ensure process. program and determine
that it has addressed if all necessary and
• Determine how the team
Risk: the necessary relevant information is
monitors the project plan
resources (time and included.
Without an integrated, and milestones and
money) to complete
effort-driven project reports information to • For the sample selected
the project as
plan, a holistic view of management. above, verify that the
outlined. Milestones
the program and/or Understand the process schedule has been
are used to judge
project, considering all to analyze potential approved by
the progress of the
dependencies and changes and their management and that
project.
resources needs, impact on the schedule. plans are in place to
cannot be attained. As • Tasks are clear, periodically review the
a result, milestones (or actionable and schedule.
delivery dates) may assigned to
• For the sample selected
continue to be missed appropriate
above, confirm that
or extended. resources.
tasks are clearly
Without a consistently • Project plans should outlined and assigned to
followed standard have tasks and appropriate resources.
methodology for milestones that are
• If the samples selected
delivering the project obtainable based on
above include any
and for managing the the requirements of
scope changes, confirm
program (i.e., project the program and/or
that the program and/or
planning and tracking project.
project schedule was
methodologies), project
• Plan scheduling is appropriately updated.
plans may be
reviewed

15 Source: www.knowledgeleader.com
Operating
Expected Design Effectiveness
Objective/Control Effectiveness Work
Practices Work Steps
Steps
developed based upon continuously and
an inconsistent updating the
understanding of tasks information is a
required to implement, priority.
potentially
• The schedule is
compromising the
reviewed on a
accuracy and
periodic basis to
completeness of the
determine if any
timeline.
changes need to be
made or if other
actions need to be
taken.

Quality Management

Objective evaluation of • Develop and • Obtain and review all • For a sample of
progress and quality of implement quality relevant documented programs or projects
work. control methods, policies and procedures within a program, obtain
policies and for quality control. evidence to support the
Risk: procedures. Confirm that quality PMO’s quality review
control requirements process.
Without a robust quality • Utilize quality control
align with organizational
control function, there is processes, • For the sample selected
requirements.
greater risk that the procedures and above, confirm that
system will not function templates to • Hold discussion with standard templates have
accurately, completely facilitate PMO management to been used (if required).
or in a timely manner, standardization. determine the level in
or that the system will which the quality control
• Provide customer
not be available during policies and procedures
satisfaction through
defined business hours. are being followed.
active practice of
Impacts of unreliable quality management.
systems can be defined
in terms of system or
business impacts.
System impacts of
unreliable systems may
include:
• System/enhanceme
nt does not meet
defined business
needs
• Unintended failures
throughout system
• System rework
needed
• System does not
integrate with other
business systems

16 Source: www.knowledgeleader.com
Operating
Expected Design Effectiveness
Objective/Control Effectiveness Work
Practices Work Steps
Steps
Business impacts of
unreliable systems may
include:
• ROI not realized as
solution fails to meet
business needs
• Loss of business
advantage to
competitors who are
successful (e.g.,
time to market)
• Loss of revenue as a
result of system
downtime
• Reduced customer
satisfaction
• Increased costs as a
result of system
rework

The program office • The program office • Hold discussion with • Obtain and review all
provides standard controls and PMO management to standard templates and
guidelines for all standardizes the gain an understanding forms. Confirm that
programs and/or methodology and all of the standard program and/or project
projects. related guidelines for projects. methodology
documentation. documentation/template
• Understand the program
Risk: s have been approved
• IT senior office’s role in
by management.
The absence of a management and maintaining the central
standard methodology program office repository for all • For a sample of current
or failure to mandate management should standards and programs or projects
use of defined define and methodology-related within a program, obtain
methodology, whether document business documentation and and review all standard
program management goals, objectives forms. documentation to
or development life- and business ensure that it was
cycle, increases the risk priorities. completed and complies
that procedures are not with defined processes.
• Program office
executed accurately,
should maintain the
completely or in a
central repository for
timely manner,
all standards and
potentially resulting in
methodology-related
the delivery of a
documentation and
solution that fails to
forms
meet the business
needs of the • Methodology usage
organization. instructions and
training manuals
The absence of an
should be developed
executed defined
and used as part of
methodology directly
training programs.
impacts the

17 Source: www.knowledgeleader.com
Operating
Expected Design Effectiveness
Objective/Control Effectiveness Work
Practices Work Steps
Steps
effectiveness of task
planning and
management, resource
assignment and
tracking, and milestone
planning. Without a
methodology, the tasks
required to achieve the
objectives cannot be
fully known.

Resource Management

Roles, responsibilities, • Formal roles for the • Hold discussion with • For a sample of current
and reporting organization are PMO management to programs or projects
relationships have been developed and gain an understanding within a program, obtain
defined and documented to of formal project roles, and review project
communicated. ensure the team has responsibilities, documentation defining
a complete reporting relationships the program and/or
Properly skilled staff
understanding of and staff performance project structure and
have been assigned
what is expected. reviews. roles and
and retained.
responsibilities. Confirm
• Organizational • Obtain and review
the current-state
Risk: structure (including relevant policy and
structure mirrors the
roles and procedure
Without a clear planned structure.
responsibilities) has documentation related
understanding of been communicated to formal roles, • Verify that staff
resource needs, the to the team and responsibilities and performance reviews are
program managers
management. reporting relationships. completed.
cannot be certain that
they have the right • Staffing • Understand the
resources identified requirements have incentives in place to
and/or in the right place been developed. motivate teams to
to deliver the solution. If complete work within the
• Management
the right resource does agreed-upon timeframe.
understands the
not exist or is not Confirm these incentives
resource
properly assigned, the are aligned with
requirements of the
accuracy, program and/or project
program and/or
completeness and objectives.
project.
timeliness of the
program and/or project • Any constraints on
may be compromised. personnel are
identified and
If team members are
addressed so that
not motivated to
the program and/or
achieve defined
project can be
milestones, there is a
completed as
risk that the effort
planned and
required to achieve
budgeted. As
those milestones will
necessary, defined
not be exerted.
escalation plans are
used to address

18 Source: www.knowledgeleader.com
Operating
Expected Design Effectiveness
Objective/Control Effectiveness Work
Practices Work Steps
Steps
troublesome
constraints.
• Task assignments
are appropriate to
the skills of the team
members.
• The team is
motivated to
complete activities
and the program
and/or project within
the agreed-upon
timeframe.
• Staff performance
reviews are
conducted on a
regular basis and
linked to
performance
management
metrics.

Financial and Benefits Realization

Anticipated benefits • A cost/benefit • Hold discussion with • For a sample of current


(financial and analysis should be PMO management to programs or projects
operational) are conducted to gain an understanding within a program, obtain
evaluated, determine of the benefits and review relevant
documented, operational benefits, realization process. documentation outlining
measurable, and anticipated costs the benefits realization
• Understand the roles
agreed upon by and return on process.
and responsibilities
stakeholders. investment
associated with • For the sample selected
timetable.
reviewing benefits above, confirm that
Risk:
• All stakeholders realization (i.e., scope/timing changes
The absence of a well- (project team determine which are reflected in benefits
defined and managed members as well as resource(s) are documentation (if
benefits realization business unit involved). appropriate).
process increases the representatives)
• For the sample selected
risk that stakeholders should be involved
above, verify that
will not receive the in the creation,
benefits documentation
benefits anticipated evaluation and
has been communicated
from the effort. monitoring of
to stakeholders and
benefits.
approved by
• Changes in scope or management.
timing are reflected
in the documentation
of anticipated
benefits.

19 Source: www.knowledgeleader.com
PROJECT LIFECYCLE MANAGEMENT (PLM)
Key Risk Considerations
• A PLM process is not documented and individuals are not aware of the process, its requirements, or when to
use it.
• Required outputs for various PLM phases are not created/completed, stored in the designated repository, or
version-controlled, ultimately leading to confusion among the project team and potential wasted time, effort and
an increased cost to the organization.
• Projects may not appropriately consider security and controls, thereby leading to design issues and/or cost
over-runs to correct issues identified after go-live.
• Project changes are not reviewed and approved prior to being made.
• Time and effort is spent on a project that has not been approved or defined.
• Success factors have not been defined for a project or project-related activities.
• Source code or software components are not developed according to company standards or adequately tested
prior to implementation in production (specific to IT projects).
• Source code or software components are not adequately planned and approved prior to implementation in
production (specific to IT projects).
• End-users and support personnel are not adequately trained for new or enhanced functionality implemented
into production.
• Projects are delivered to the business with problems or defects or do not meet requirements.

Operating
Design Effectiveness
Objective/Control Expected Practices Effectiveness Work
Work Steps
Steps

Planning and Initiation

A requirements • Business and project • Hold discussion with • Obtain and review
specification is formally management should business and project policies and
developed. define and document management teams to procedures relating to
a high-level determine if a requirements
specification requirements specifications to
document. specification is determine if they
developed for all contain:
• Senior management
projects. − Objective
should be presented
the final document • Discuss the process − Scope
and informed of key for communicating
− Outline of
project dates, project requirements to
modules/functions
personnel and budget key stakeholders.
requirements. − Input/output
− Process/interface
− Date/creator/version
number
− Logic described in
flowcharts and
decision tables
• For a sample of
projects, obtain and
review documentation

20 Source: www.knowledgeleader.com
Operating
Design Effectiveness
Objective/Control Expected Practices Effectiveness Work
Work Steps
Steps
that measures
projected project
specifications against
the established policies
and procedures.
• For the sample
selected above,
confirm that senior
management received
and reviewed relevant
documentation.

A formal development • IT, business and • Hold discussion with • Obtain and review the
methodology has been program office management to gain project management
defined and is utilized by management should an understanding of implementation
development teams for define and document the development policies and
all development projects a company-wide methodology and its procedures and other
or activities. standard project usage in the related material (i.e.,
management and environment. usage instructions and
implementation training manuals) to
• Hold discussion with
methodology. ensure the
management to
documentation
• Usage instructions determine whether
contains relevant and
and training manuals training manuals
necessary information.
should be developed and/or other training
Consider
and used as part of materials are required
benchmarking the
providing orientation for projects.
methodology against
and reference
established industry
materials for the
frameworks (if
standard
applicable).
methodology.
• For a sample of
projects, confirm that
training materials
sufficiently address the
development
methodology and
project-specific
requirements. Further,
verify that training was
provided to necessary
end users and other
interested parties.

Development plans are • Senior management • Hold discussion with • For a sample of
formally approved and should be presented management to gain projects, obtain
sponsored by with all project an understanding of documentation of
management. initiation collateral for senior management management review
review and approval. project approval, and determine the
sponsorship and frequency of the
• Senior management
oversight reviews and the depth
should take an active
responsibilities. of participation.
role and provide

21 Source: www.knowledgeleader.com
Operating
Design Effectiveness
Objective/Control Expected Practices Effectiveness Work
Work Steps
Steps
ongoing sponsorship • For the sample
and monitoring selected above, verify
oversight throughout that formal procedures
the project lifecycle. are in place for
management signoff
and confirm that
necessary approvals
were obtained.

Requirements Analysis

User involvement and • IT, business and • Hold discussion with • For a sample of
buy-in program office management to projects, obtain the
management should determine if there is a standard
define and document process in place to documentation related
shared vision of facilitate to requirements
project goals and communication gathering, business
objectives. between IT, the unit testing and
business unit and the approvals for each key
• User /business
PMO regarding project project phase.
personnel should be
goals, objectives and Evaluate the materials
involved in all
decision making. for sufficiency and
functional decision
completeness.
making processes
and actively • For the sample
participate in testing. selected above,
confirm that necessary
• Business
approvals were
management should
obtained for each key
approve all project
project phase.
decisions and each
key project phase.

Resource requirements • IT and project • Hold discussion with • For a sample of


management should management to gain projects, determine
develop a resource an understanding of whether the project
requirements matrix, how project teams team has performed a
defining projected determine the time and cost benefit analysis of
costs, personnel and effort involved in each each of the
skill set requirements, of the aspects of the components of the
and acceptable project. project by obtaining
resource variances. resource requirements
documentation.
• The business
considers detailed • For the sample
customer selected above,
requirements and IT assess the adequacy
focuses on non- of the resource
functional requirements matrix.
requirements
• For the sample
(including SLAs,
selected above, verify
OLAs, etc.).
that the business
appropriately
considered detailed

22 Source: www.knowledgeleader.com
Operating
Design Effectiveness
Objective/Control Expected Practices Effectiveness Work
Work Steps
Steps
customer requirements
and IT focused on non-
functional
requirements
(including SLAs, OLAs,
etc.).

Design

Defining a system • Management should • Hold discussion with • Obtain and review
architecture (for IT use the project’s management to standard policies and
projects) technical and determine whether procedures detailing
feasibility studies to project technical and the following project
develop a detailed feasibility studies must system requirements:
system architecture be developed as a tool − Hardware
document. for management to use
in creating a detailed − System software
system architecture − System utilities
document. − Application software
− Functional
limitations
• For a sample of
projects, determine if
the systems
architecture has been
designed to meet
performance
requirements.
• For a sample of
projects, determine if
test design
considerations have
been documented.

Managing functionality • Business • Hold discussion with • For a sample of


specification management should management to projects, obtain and
develop detailed determine the process review business and
business and by which business functional
functional management develops specifications to
specifications. specifications. confirm:
− Adequate
specifications are
provided
− Realistic budget
and personnel
resources are
available
− Business
requirements and

23 Source: www.knowledgeleader.com
Operating
Design Effectiveness
Objective/Control Expected Practices Effectiveness Work
Work Steps
Steps
process flow are
satisfied
− IT technology,
architecture and
planned/implement
ed environment
adequately satisfy
the business need
• For the sample
selected above,
determine if test design
considerations have
been documented.

Managing database • IT management • Hold discussion with • If applicable, for a


design (for IT projects) should develop a management to sample of projects,
database design to determine the process obtain and review
address the agreed- by which IT database design to
upon functionality management confirm:
considers project and − Functionality is
• IT personnel follow
company requirements addressed
company-wide design
when creating system
and development − Growth/modification
databases.
standards. flexibility
• Hold discussion with
• Database specialists • Review project
management to gain
should be involved in documents to confirm
an understanding of
the database design. usage of:
the requirements for
engaging database − Company-specific
specialists. design standards
− Design policies and
procedures
− Process flow
analysis
− Development
methodology
• For the sample
selected above,
determine whether test
design considerations
have been
documented
specifically for
databases.
• For the sample
selected above,
determine whether
database specialists
have been involved in
the database design.

24 Source: www.knowledgeleader.com
Operating
Design Effectiveness
Objective/Control Expected Practices Effectiveness Work
Work Steps
Steps

Managing interfaces (for • IT and business • Hold discussion with • For a sample of
IT projects) management should management to gain projects, review project
identify, document and understanding of documentation to
and assess the process in which IT confirm all business
implementation and the business units interfaces have been
alternatives to satisfy determine interfaces identified and that test
all interfaces for projects. plans have been
requirements. established.
• For a sample of
projects, review the
controls designed to
validate the information
being sent and
received is validated
and accurate.

Managing security (for IT • IT and business • Hold discussion with • Obtain and review
projects) management should management to security policies and
define and document determine the process procedures.
security and access by which projects
• For a sample of
control requirements. adhere to security
projects, obtain and
requirements.
• Internal audit review documentation
personnel should • Hold a discussion with showing security
review and approve management to testing evidence and
the proposed security understand internal management approval
model. audit’s role in of project specification
evaluating the adherence to standard
• Automated tools
proposed security security practices.
should be
model. If internal audit
implemented to • For the sample
is not involved,
control the security selected above,
determine whether a
environment and confirm that internal
different (but similar)
access. audit (or an equivalent
group is involved.
compliance group) was
involved in evaluating
the security model.

Managing documentation • Program office/project • Hold a discussion with • Obtain and review
management should management to gain documentation
define company-wide an understanding of detailing adopted
documentation the process to define policy and procedure
standards for system, company-wide standards for system
user and training documentation documentation
documentation standards for system, standards and user
user and training training.
• Project management
documentation.
should develop a • For a sample of
review process to • Hold a discussion with projects, review
enforce usage of management to project-related
documentation determine whether documentation and
standards documentation confirm it adheres to
standards are adopted standards.

25 Source: www.knowledgeleader.com
Operating
Design Effectiveness
Objective/Control Expected Practices Effectiveness Work
Work Steps
Steps
requirements and
enforced.

Managing training • Management should • Hold discussion with • Obtain and review
requirements define a training management to training documentation
framework, including determine the process covering
model and of system training for planned/implemented
expectations. personnel. project training
courses, a list of IT and
• Personnel targeted for
business personnel
training and teaching
who have attended or
should be identified
are planning to attend
and informed.
training, the viability
• Feedback on training and applicability of
materials should be planned training
sought from training courses, and post
attendees. training assessment of
materials by previous
training attendees.
• Determine if
mechanisms are
planned/implemented
to capture, assess and
address feedback of
training attendees.

Development

Utilization of coding • IT management • Hold discussion with • Obtain and review


standards (for IT should develop management to gain coding standards
projects) coding standards to an understanding of documents detailing:
be followed by all the process by which − Program naming
project teams. the team determines conventions
coding standards.
• IT management − Documentation
should develop standards
training materials and
− Coding policies and
usage instruction to
procedures
ease the usage and
enforcement of − QA review
adopted standards. procedures
• IT management − Coding standards
should develop a violation policies
coding review team and procedures
charged with review − Acceptable
and enforcement of minimums and
the coding standards. violation resolution
• Obtain and review
training policies and
documents detailing
usage guidelines and

26 Source: www.knowledgeleader.com
Operating
Design Effectiveness
Objective/Control Expected Practices Effectiveness Work
Work Steps
Steps
personnel
expectations.
• For a sample of
projects, confirm that
coding standards
comply with company
standards.

Managing testing • Management should • Interview IT and • Obtain and review


considerations develop a standard business management testing methodology
company-wide testing to determine standard documents including:
methodology. testing requirements − Policies
and expectations.
• Management should − Procedures
develop a QA review
− Sample test plans
team charged with
developing and − Required
executing review and documentation
enforcement − Signoff procedures
procedures for the
− Test environment
testing activities.
requirements
• IT and business − Test results &
management should expectations
jointly develop (including a
specific testing designed defect
requirements for each management
project. process and
scorecard)
• For a sample of current
projects, obtain and
review project testing
documents to
determine if adopted
testing standards are
being followed.

Creation of system • IT management • Hold discussion with IT • Obtain and review


documentation (for IT should develop a management to standards defining
projects) standard company- determine system system documentation
wide system documentation requirements including:
documentation requirements and − Policies &
standard. expectations. procedures
• IT and project • Hold discussion with − Sample
management should management to gain documentation
enforce an understanding of
− Signoff/approval
documentation the process to enforce
forms
standards. documentation
standards. − Environmental or
configuration
change

27 Source: www.knowledgeleader.com
Operating
Design Effectiveness
Objective/Control Expected Practices Effectiveness Work
Work Steps
Steps
request/approval
forms
− Budget
request/approval
forms
• For a sample of current
projects, determine if
system documentation
has been delivered or
is expected as part of
the development
phase.

Managing data • IT and business • Hold discussion with • For a sample of current
conversion (for IT personnel should management to gain projects, confirm that
projects) review existing an understanding of IT and business
systems and current the system data personnel have
interfaces and define conversion process reviewed old
data conversion and its requirements. application routines,
requirements. data files, interfaces
• Hold discussion with
and custom programs
• IT management management to
and made
should define and understand the
determination as to
document standard process to purge
applicability for the
conversion load redundant data.
new system
modules to be used in
processing all data • For the sample
conversion. selected above, obtain
meeting minutes
• IT management
and/or management
should ensure the
reports addressing the
business takes the
review and approval of:
opportunity to purge
redundant data. − System functionality
− Inbound/outbound
interfaces
− Internal/external
interfaces
− Data conversion
requirements

Testing

Managing processing • Testing execution • Hold discussion with • For a sample of current
and data integrity (for IT should confirm management to gain projects, obtain and
projects) system functionality, an understanding of review system test
data conversion and the testing process. plans, confirming that
interfaces process Determine whether the they include:
were completed design of the testing − Specific facets to be
without errors. process includes tested
requirements to
evaluate system

28 Source: www.knowledgeleader.com
Operating
Design Effectiveness
Objective/Control Expected Practices Effectiveness Work
Work Steps
Steps
• Testing should functionality, data − Test setup
confirm data is conversions and requirements
accepted and interfaces. − Detailed step-by-
processed and that step plan
expected results were
received. − Expected results
− Acceptable
• Testing activities are
return/error codes
independently
executed and • Review test results and
resources are confirm that test plan
appropriately was followed and
segregated to ensure acceptable return/error
integrity of results. codes were received
Access to sensitive for each phase of
code is restricted testing. Verify that the
based on business level of testing
need. documentation is
commensurate with the
complexity of the
change.
• Obtain and review
testing documentation
confirming all
processes and key
business flows
achieved successful
execution.
• Review test results and
confirm:
− Input data integrity
was tested and
verified at pre-input
and as a result of
the input process
− In-stream data
integrity testing was
conducted and
expected results
were achieved

Managing system • Testing should • Hold discussion with • For a sample of current
reliability (for IT projects) confirm that system management to gain projects, obtain and
functions reliably. an understanding of review relevant testing
the testing process. documentation to
• Testing should
Determine whether the verify:
confirm that the
design of the testing − Systems function
system environment
process includes reliably
does not experience
requirements to
unexpected outages − Stability of system
evaluate system
or failures. environment

29 Source: www.knowledgeleader.com
Operating
Design Effectiveness
Objective/Control Expected Practices Effectiveness Work
Work Steps
Steps
• Testing should availability and − Appropriate system
confirm that system reliability. up-time availability
up-time availability
meets business and
IT management’s
expectations.

Managing test • IT management • Hold discussion with • Obtain and review


environments (for IT should develop a management to policies and
projects) company-wide testing understand company- procedures relating to
environment wide testing company-wide testing
standard. environment environment
standards. Confirm standards.
• IT and project
that the design of the
management should • For a sample of current
testing process
jointly develop projects, obtain and
accounts for the need
specific environment review the environment
to segregate
requirements for each requirements for each
production and non-
project. project and
production
documentation
• Senior management environments.
denoting management
should be presented
approval.
with environment
costs and approval
obtained in advance.
• IT management
should ensure that the
test environment is
adequate to support
the full test program.
• IT management
should ensure that the
test environment is
effectively segregated
from the production
environment.

Implementing structured Unit: • Hold discussion with • For a sample of current


test phases (for IT management to gain projects, obtain and
• Testing should
projects) an understanding of review the business
confirm success of
the various testing unit testing
individual program
phases and associated documentation to
testing.
steps (including ensure:
• Testing should documentation and − Success of process
confirm success of approval flow
process flow system requirements).
test. − Approval of each
key testing process
• IT and business
− Resolution for each
management should
failed testing point
provide written signoff
and approval of each • For a sample of current
key testing process. projects, obtain and

30 Source: www.knowledgeleader.com
Operating
Design Effectiveness
Objective/Control Expected Practices Effectiveness Work
Work Steps
Steps
• IT management review the
should confirm that system/integration
test failure points testing documentation
have been resolved to ensure:
and re-tested and the − Success of
entire process functional point-to-
documented. point and end-to-
System/Integration: end system
integration testing
• Testing should
confirm success of − Business unit and
functional point-to- IT documented
point and end-to-end approval of testing
system integration process
test. − Resolution for each
failed testing point
• IT and business
management should • For a sample of current
provide written signoff projects, obtain and
and approval of each review the business
key testing process. unit acceptance testing
documentation to
• IT management
ensure:
should confirm that
test failure points − Business unit
have been resolved involvement
and re-tested and the − Acceptance of
entire process resolutions to
documented. identified testing
Acceptance: issues
− System results and
• Business users must
environmental
be involved in
performance
acceptance testing.
− Business unit and
• Testing should IT management
confirm success of approval of each
user acceptance key testing process
shakedown and
resolution of identified • For a sample of current
issues. projects, obtain and
review the load/stress
• Testing should testing documentation
confirm acceptable to ensure:
system results and
environment − System function
performance. reliability
− System
• IT and business
environmental
management should
stability
provide written signoff
and approval of each − Confirmation of
key testing process. system up-time
availability
• IT management
should confirm that

31 Source: www.knowledgeleader.com
Operating
Design Effectiveness
Objective/Control Expected Practices Effectiveness Work
Work Steps
Steps
test failure points − Acceptable volume
have been resolved variation impact to
and re-tested and the system
entire process performance
documented.
• For a sample of current
Load/Stress: projects, obtain and
review the security
• Testing should
testing documentation
confirm that system
to ensure:
functions reliably.
− IT and business unit
• Testing should security testing
confirm that the
system environment − Independent
does not experience system security
unexpected outages analysis
or failures. − Internal audit
approval of security
• Testing should
standard adherence
confirm that system
up-time availability − Business unit, IT
meets business and and internal audit
IT management’s approval of security
expectations. testing procedures
• Testing should − Acceptance of
confirm that volume resolutions to
variation does not identified testing
adversely impact issues
system performance • For a sample of current
or throughput. projects, obtain and
Security: review the regression
testing documentation
• IT and business to ensure:
personnel should
− Success of code
conduct security
modifications on
testing.
system functionality
• Independent system − IT and business unit
security analysis approval of each
should be conducted. key testing process
• Internal audit should − Acceptance of
confirm success of resolutions to
security testing and identified testing
adequacy of security issues
measures
implemented during Note: if any of the testing
the user acceptance guidance steps above
shakedown. cannot be completed
because the sample
• IT, business and project did not require a
internal audit certain type of testing,
management should simply indicate as such in
provide written signoff the work papers.

32 Source: www.knowledgeleader.com
Operating
Design Effectiveness
Objective/Control Expected Practices Effectiveness Work
Work Steps
Steps
and approval of
testing.
• IT management
should confirm that
test failure points
have been resolved
and re-tested and the
entire process
documented.
Regression:
• Regression testing
should confirm
success of code
modifications without
impact on other
system functionality.
• IT and business
management should
provide written signoff
and approval of each
key testing process.
• IT management
should confirm that
test failure points
have been resolved
and re-tested and the
entire process
documented.

Obtaining signoff and • All levels of • Hold discussion with • Obtain and review
approval management must management to management signoff
ensure that written understand the for the following:
approval is provided approval requirements − Policies
from project associated with project
participants. activities. − Procedures
− Documentation
requirements
• For a sample of current
projects, compare
adopted signoff
standards to project
activities undertaken
• For a sample of current
projects, obtain and
confirm receipt of
signoff documentation
for:
− Program
development

33 Source: www.knowledgeleader.com
Operating
Design Effectiveness
Objective/Control Expected Practices Effectiveness Work
Work Steps
Steps
− Unit testing
− System testing
− Integration testing
− User acceptance
testing
− Functionality
processing
− Change
management
− Senior/executive
management

Implementation and Roll-Out

Developing an • IT and business • Hold discussion with • Obtain and review


implementation strategy management should management to documents which
jointly review the determine how IT and identify the positive
factors and determine the business units and negative impacts
which strategy is most define and agree on for proposed
beneficial. desired implementation implementation
approaches. approaches:
• Project management
Determine whether − Immediate
should develop a
implementation plans
detailed − Phased
and/or back-out
implementation plan
(rollback) plans are − Parallel
with tasks, dates,
required. − Pilot
deliverables,
personnel • For a sample of current
assignments and projects, confirm that
costs. detailed implantation
• The project team plans were developed
defines rollback in accordance with
criteria that can be company
used on an as- requirements.
needed basis during • For the sample
the deployment selected above, verify
process. Rollback that rollback plans
plans are developed were developed and
and confirmed with approved. If rollback
key stakeholders. plans were used,
confirm sufficiency of
supporting
documentation.

34 Source: www.knowledgeleader.com
Operating
Design Effectiveness
Objective/Control Expected Practices Effectiveness Work
Work Steps
Steps

Managing • Assessment • Hold discussion with • For a sample of current


organization/process checklists should be management to projects, obtain and
readiness developed to ensure understand the use of review:
that pre- assessment checklists − Pre-implementation
implementation for pre-implementation readiness checklists
activities are activities.
understood, − Requirements for
• Hold discussion with business/IT
monitored and
management to organizational or
successfully
determine the process business process
completed.
used to monitor and changes
approve pre-
− Requirements for
implementation
system environment
activities.
upgrades and
installation activities
(as necessary)
• For the sample
selected above,
confirm:
− User acceptance
testing completed
successfully and
testing issues
resolved
− Training materials
have been
distributed and all
personnel have
attended mandatory
training sessions
− Database design
has been approved
and pre-loaded per
specification (as
necessary)
− Successful
completion of all
data conversion
processes and
conversion issue
resolution (as
necessary)
− Senior
management has
approved all
organization and
business process
changes
− Organizational
changes have been

35 Source: www.knowledgeleader.com
Operating
Design Effectiveness
Objective/Control Expected Practices Effectiveness Work
Work Steps
Steps
presented,
reviewed, approved
and implemented
− Business processes
and daily workflow
changes have been
presented,
reviewed, approved
and implemented

Managing production • IT and business • Hold discussion • Obtain and review


support management should management to production support
jointly develop understand agreed documents detailing:
procedures to provide upon standards, − Standards
system support policies and
including issue procedures for − Policies
logging, tracking and providing post- − Procedures
resolution. implementation − Required
support.
• Production support documentation
management should • Hold discussion with − QA review and
develop personnel management to approval
roles and understand how
− Migration to
responsibilities. production support
roles and production
• Senior management − Task prioritization
responsibilities are
should review and
defined and assigned. − Resource
approve the cost and
personnel metrics assignment
required to provide • For a sample of current
adequate production projects, review
support. production support
documentation and
verify that items listed
above were properly
completed and
sufficiently detailed.

Maintenance • IT and business • Hold discussion with • Obtain and review


management should management to maintenance support
jointly develop understand agreed documents detailing:
procedures to provide upon standards,

36 Source: www.knowledgeleader.com
Operating
Design Effectiveness
Objective/Control Expected Practices Effectiveness Work
Work Steps
Steps
service allowing for policies and − Standards
enhancements, procedures for − Policies
modifications and providing post-
problem support and implementation − Procedures
resolution. maintenance and − Required
modification support of documentation
• IT management
application/environmen − QA review and
should develop
t. approval
personnel roles and
responsibilities. • Hold discussion with − Migration to
management to production
• Senior management
understand how
should review and
maintenance support − Task prioritization
approve the cost and − Resource
roles and
personnel metrics assignment
responsibilities are
required.
defined and assigned.
• For a sample of current
projects, review
maintenance support
documentation and
verify that items listed
above were properly
completed and
sufficiently detailed.

Documentation • Project management • Hold discussion with • Obtain and review


must ensure that management to gain adopted
adequate and an understanding of documentation
thorough system and the system and user standards.
user documentation is documentation
• For a sample of current
available and centrally requirements.
projects, confirm
maintained.
documentation has
been developed for:
− System
configuration
− Database design
− Application
environment
− User configuration
− System support
requirements
• For the sample
selected above,
confirm the
documentation is
sufficiently detailed
and centrally retained.

Training • Project management • Hold discussion with • Review documentation


should define training management to and meeting minutes
requirements. determine expectations delineating training

37 Source: www.knowledgeleader.com
Operating
Design Effectiveness
Objective/Control Expected Practices Effectiveness Work
Work Steps
Steps
• Project management and requirements of requirements, target
or central training user and system attendees and
function should training. proposed training
conduct training approach and
sessions for system schedule.
and user personnel.
• For a sample of current
• A long-term projects, obtain and
commitment should review training
ensure that training is materials/manuals and
updated and provided confirm they
regularly. adequately address
training goals and
expectations.

Post-Implementation

Monitoring user • A post- • Hold discussion with • For a sample of current


satisfaction implementation management to gain or recently completed
review should be an understanding of projects, verify a post-
conducted to assess the post- implementation review
the user satisfaction implementation review was conducted.
with the process. Understand Further, confirm:
implementation in a the requirements for − Post-
number of key areas. communicating results implementation user
The frequency and to management. satisfaction surveys
scope of the post- were conducted
implementation
reviews is based on − Survey results are
the size and used to report
complexity of the project success to
project. management
− Unsatisfactory
• A presentation should
areas of survey are
be provided to senior
used to recommend
management outlining
and implement
the general user
immediate
acceptance, areas of
improvements
concern and
recommendations for
addressing these
concerns.

38 Source: www.knowledgeleader.com

You might also like