Strictly Private & Confidential

INTERNAL AUDIT REPORT

Ashfield Computer Training

Ref.: SPE121/JR/2008

To: Ciaran Conlon Director, Training Services,

Anna Kelly Director Curriculum & Quality Assurance

CC: Patricia Curtin ADG, Corporate Policy & Standards,

Ray Kelly Regional Director - North East
Eamonn McFadden Manager • Financial Standards
Jim Wadden Regional Director • Dublin

4 February 2009
 1. Executive Summary

The audit was initiated in response to a request from Training Services in the North East region
which raised concerns regarding aspects of the assessment process on contracts run by
Ashfield Computer Training (ACT). Internal Audit examined some of the assessments
highlighted by the region as v/ell as assessments completed on contracts run by ACT in other
FAS regions.

The key issue of the audit was to determine the integrity of the assessment process in these
contracts and by extension the integrity of the certification processed by FAS and issued by the
various certifying bodies such as FETAC and City & Guilds.

Before detailing the main conclusions of the audit, two important issues arose that Internal
Audit was not in a position to examine in detail but need to be highlighted and which warrant
further consideration and action by FAS. The first of these matters was that Curriculum
Sections within FAS training centres adopted different approaches to the verification of
assessments earned out by contracted trainers. The Curriculum Section in Dundalk supervised
independent verifications of contracted courses, in much the same way as they did for in-centre
courses. On the other hand, Baldoyle Training Centre did not carry out these independent
verifications. Such a divergence of approach between training centres is a matter for concern in
the context of FAS' role in regard to the certifying bodies.

The second matter that arose during the audit was the question of shortcomings in the set-up
of contracted training courses which involve a significant amount of IT training and certification.
The issues that arose in relation to the ACT contracts included outdated and unreliable
computers, the lack of adequate numbers of printers or networked printers during assessments
and computer disks being re-used or unavailable. These issues were raised both by trainees
interviewed by Internal Audit and by ACT tutors, by way of explanations as to why certain
assessment tasks such as printing and saving files to disk did not happen within the allotted
assessment timeframe. While contracted training locations are often by their nature temporary
and not as well equipped as permanent locations such as FAS training centres, the issues
raised in regard to ACT contracts suggest that they fall below a minimum acceptable standard
in relation to the provision of equipment. The matter was not examined in detail but needs to be
examined in the wider context of how contracted training is delivered generally.

In regard to the key issue of the audit; the integrity of the assessment process in the sample of
ACT contracts, Internal Audit examined a total of 7 contracts. It is important to place the
Internal Audit findings in an overall context. In all the cases examined Internal Audit found that
the assessment event did take place and in the case of 3 contracts Internal Audit found that the
sampled assessments had been processed correctly by ACT. The issues identified with the
remaining 4 contracts sampled by Internal Audit varied in significance. However, the findings
point to a number of instances of the manipulation of assessment material in contracts
delivered by ACT on behalf of FAS. The manipulation of material initially came to light on a
CAD contract run by ACT in the North East Region. ACT accepted that the assessment
material had become "unreliable" through the actions of the tutor on that contract This was
accepted in writing by ACT in April 2007.

In the view of Internal Audit, two further ACT contracts completed in the North East Region
during 2007 also showed evidence that assessment material had been manipulated. It is not
clear to Internal Audit the extent to which, if any, learners had any involvement in manipulating

r» . . i . r 1 A
 assessment material. However, the responsibility for ensuring the integrity of the assessments
as they occurred on the ground lay primarily with ACT and their tutor. In the light of ACT's
acceptance that assessment material had been manipulated on the CAD contract in the North
East there was an obvious onus on ACT to ensure that the same did not occur on the
subsequent contracts; this in the view of Internal Audit was not done.

Issues raised by Internal Audit in relation to the two contracts in Dublin North (contracts 0826
and 1240) were likewise referred to ACT for consideration. ACT was able to provide
satisfactory explanations to some of the concerns raised by Internal Audit

In both of these contracts Internal Audit identified weaknesses in the way that multi-choice
assessments were completed. In both contracts the supplied answer sheets were not used and
answers were written in pencil on the question sheet. In the case of the Word Processing 2
module for contract 1240 there was a very high number of errors (40) in the marking of the
assessment The explanation supplied by ACT and the tutor was that 'on reflection, there
seems to have been a systems failure in relation to the exam correction processHowever,
when Internal Audit applied the appropriate City & Guilds marking template to this assessment,
the correct outcome would have been that 6 learners would have passed the assessment
while 5 others should have been marked as having failed (at the time all 11 learners were
marked as having passed the assessment). In the view of Internal Audit the marking was
probably conducted with a view to ensuring that all the learners passed.

In regard to other modules on both the Dublin North contracts Internal Audit identified a number
of issues including missing assessment material (both disks and printed material), inconsistent
dates and times on assessment material and multiple copies of assessment files. In a number
of instances ACT stated that they had approval from the Curriculum Department in Baldoyle to
save assessments to disk rather than print material as required by the City & Guilds
assessment guidelines. Internal Audit was unable to independently confirm this with FAS
Baldoyle. In the case of contract 0826 Internal Audit was unable to find any supporting
assessment material (either on disk or printed) for a significant part of one of the assessments.
The explanation provided by ACT was that there had been "big problems with the disks for this
assignment". In the view of Internal Audit the obvious answer to technical problems that might
have occurred on the day of an assessment would be to postpone the assessments to a later
date and sort out these problems. The absence of any assessment material for these
assessments and the ACT explanations were less than satisfactory.

In relation to contract 1240, Internal Audit concluded that the tutor's actions had assisted some
learners in achieving a pass mark that they would otherv/ise not have achieved and that this
assistance took place after the assessment was completed.

In overall terms the assessment material examined for the four contracts has shown a number
of examples where Internal Audit is of the opinion that the assessment material had been
manipulated and the certification process has been undermined. It was not possible for Internal
Audit to fully determine the motivation of tutors working for ACT but there does appear to be
evidence, in a number of instances, of tutors "helping" learners to achieve the required grades
in the way that assessments were managed and marked. In the North East Region ACT had
accepted that one of their tutors had manipulated assessment material on the CAD contract.
Following on from that acceptance it would have been expected that ACT would expend a
significant effort in ensuring that all the assessments for their subsequent contracts in 2007
would be conducted with integrity and meet the clear standards set down by the certifying
bodies. Internal Audit is of the view that the material highlighted in this report shows that this
did not happen.

Internal Audit considers that a number of the findings of this report to be significant and
recommends that FAS considers the matters that have been raised in this report.

Internal Audit would like to acknowledge the assistance of management and staff in both the
Dublin and North East regions for their assistance in completing this audit

Joe Roe Patrick Kivlehan

Manager IT Audit/Support Director Corporate
- Internal Audit Governance/Internal
Audit

r\ - A r -i R\
2. Background

In January 2008 Internal Audit was contacted by FAS Training Services in the North East
Region in regard to apparent irregularities on two training contracts run by the Ashfield
Computer Training (ACT) within the region. The apparent irregularities highlighted by the region
related to assessments completed by the trainer on the following contracts;

• Contract 0769 - CAD - 24 January 2006 to 1 December 2006

• Contract 1032 - Office Administration - 9 October 2006 to 20 June 2007

In the case of the CAD course a number of matters of concern were identified by the region in
relation to the course assessments. Computer disks appeared to have been accessed after the
date of the assessments and test data manipulated. When interviewed by FAS management in
March 2007, the tutor used by ACT to deliver this course accepted that he had manipulated the
assessment material. On 25 April 2007 ACT wrote to Training Services in the North East and
accepted that the results of the CAD assessments were8unreliable" and added that the
company was 'totally unaware of the events in this case until brought to our attention by you,
and we have taken the necessary steps to avoid any recurrence. We also no longer use the
services of the person involved. In the latter part of 2007 the Curriculum Section in Dundalk
Training Centre then completed a certification verification of a second ACT contract within the
Region (contract 1032) and the issues this raised led, in turn, to the contact with Internal Audit.

Internal Audit commenced an examination of the assessment material for contract 1032 in late
February 2008. The initial examination of this material confirmed a number of concerns
regarding the assessments for this contract and it was decided that the assessments for a
sample of other sample contracts run by ACT would also be examined. The sampled contracts
included a number in the Dublin Regions as well as a further contract in the North East region.

In response to the issues identified in contracts in the North East Region, Training Services in
Dundalk with the assistance of Curriculum & Quality Assurance has applied a significant
amount of resources in meeting with trainees and tutors, organising and providing re-training,
liaising with the certifying bodies and ensuring that where necessary learners completed
assessments

3. Scope

The audit examined sample assessments for the following contracts;

• Dublin North I Contract 08261DN430 Computer Applications and Office Skills -

June 2006 to February 2007
• Dublin North I Contract 12401DN430 Computer Applications and Office Skills -
May 2007 to December 2007
• Dublin North I Contract 13371 Computer Applications and Office Skills - June
2007 to March 2008
• Dublin South West I Contract 11301 Manual & Computerised Accounts - March
2007 to June 2007
• Dublin South West I Contract 13251 Manual & Computerised Accounts -
September 2007 to November 2007
• North East I Contract 10321 Office Administration - October 2006 to June 2007
 North East I Contract 12271 Office Administration • April 2007 to December 2007

For the sampled assessments from the above contracts, the audit compared the written,
printed and saved data (disk) from the assessment material against the relevant requirements
specified by either FETAC or City & Guilds. Areas of concern that arose from this comparison
were raised with ACT. An initial meeting was held between Internal Audit and the Director of
Academic Affairs with ACT. This meeting took place in late May 2008. Further meetings then
took place in Baldoyle and Dundalk Training Centres where the actual assessment material
from four of the above contracts was reviewed in detail. Present at these reviews, which took
place on 18 and 27 June 2008, were ACT's Director of Academic Affairs and an ACT tutor who
had worked one of the four contracts. Internal Audit was assisted at these meetings by a
manager from the FAS Curriculum & Quality Assurance Department Arising from these
meetings on the assessments for the four contracts, Internal Audit wrote in early July 2008 to
Ashfield Computer Training and supplied the company with a detailed schedule of the issues
that had been raised at the meetings and for which Internal Audit sought further comment I
clarification. ACT responded to Internal Audit on 18 August 2008.

Internal Audit also met with trainees from one of the courses: Contact 1032, in early July and
late August 2008 to discuss details of the assessment processes. Internal Audit again met with
representatives of ACT in September 2008 and arising from these meetings supplied ACT with
an updated schedule of the queries raised previously. In this updated schedule Internal Audit
either noted the responses made by ACT and in a small number of instances accepted the
explanations provided by ACT. The company was also asked whether they wished to add any
further comments. ACT responded by email to Internal Audit on 24 September 2008. The
company did not add any further comments but referred to a Microsoft technical article on the
significance of changes to file dates and times.

4. Findings

Internal Audit examined a number of assessments from the seven sampled contracts
completed by Ashfield Computer Training (ACT). For three of the contracts examined Internal
Audit found that the assessment material met the City & Guilds or FETAC requirements and no
issues were therefore raised with ACT. In the case of the remaining four contracts, Internal
Audit identified a number of matters in relation to some or all of the modules examined and
these were brought to the attention of ACT. The findings section details the issues raised by
Internal Audit, the responses from ACT and any conclusions made.

During the examination of the assessment material a number of other issues were highlighted
to Internal Audit regarding the running of contracted Training courses and the process of
certification between contractors, FAS and the certifying bodies. While these items are not
explored in detail within this report they are important matter that the organisation as a whole
needs to examine.

4.1 Contracted Training and assessment verification.

The audit highlighted the different approaches taken by Curriculum sections in different FAS
regions to the question of independent verification of contracted training assessments. The
issues in relation to ACT arose originally from verifications completed in the North East Region
 under the auspices of the Curriculum section in Dundalk Training Centre. When Internal Audit
examined the assessment material held by FAS for the ACT contracts in Dublin North and
Dublin South I West, it became apparent that neither region had undertaken verifications of the
assessments completed as part of the external training contracts. The checks carried out in
Dublin North and Dublin South I West centred on ensuring that all the necessary assessment
material was returned and that the overall certification assessment results agreed with those for
the individual modules. These constituted completeness checks rather than verifications.

4.2 Contracted Training equipment resources

A feature of the work conducted during the audit was the number of times the issue of
inadequate computer equipment was raised. This applied to both the learners interviewed for
one of the ACT contracts and to the explanations provided by ACT to explain issues which
were queried by Internal Audit. These issues included

• Old computers which sometimes did not allow assessment material to be saved
properly.
• Typical setups where only one printer was available for the whole class and which
were not on any network, thus forcing assessments to be printed off one at a time.
• A lack of computer disks leading to the constant re-use of older and sometimes
unreliable disks.

While the issue of obsolete or inadequate equipment was not examined as part of the audit, it
v/ould be important that this matter be followed up through Training Services.

4.3 Specific Contracts and Modules examined.

In broad terms, the issues raised with ACT fell under a number of headings;

• Issues where material that was deemed to be mandatory by the certifying body for a
learner's assessment to be passed was absent from the assessment material.
• Issues where a number of learners' assessment material contained the same pattern
of errors.
• Issues where the dates of computer files on disks used as part of learners
assessments were inconsistent with the dates and times of when the assessments
were held.
• Issues where dates and times recorded on physical assessment material appeared to
differ within individual learner's assessments.
• Issues where handwriting on disk labels used as part of assessments were not those
of learners.
• Issues where multiple answer sheets were marked incorrectly.

The significance of computer file dates and times

The issue of differences between computer file creation and modification dates and times arose
in a number of the contracts examined in this audit. Normally computer file creation and
modification dates operate in a predictable manner. Many City & Guilds assessments require
 the learner to create and name a new computer file (Word document Excel spreadsheet, etc).
When the file is first saved, the "date created" is established based on the time of the computer
clock. This "date created" is normally the earliest date and time and any subsequent changes
and saves to the document alter the "date modified" rather than the "date created". It is
generally unusuai therefore to have a date modified for a file which is before a date created.
However, this does occur where a file is copied from one location to another and more
particularly from one device to another device (e.g. from disk to disk). In all cases however, the
date modified is the last date that the file was actually updated and saved by a user and is
never altered by subsequent copying.

4.3.1 North East I Contract 1032 / Office Administration - October 2006 to June 2007

This contract was run in Ballivor, Co Meath and was part of the regional response to the
closure of the N.E.C. computer plant. Internal Audit raised a number of issues with ACT in
regard to the following City & Guilds modules for this contract;

• Presentation Graphics (7262-01-006)

• Word processing 2 (7262-02-022)
• Spreadsheets 2 (7262-02-023)
• Integrated Applications (7262-02-029)

4.3.1.1 Contract 1032 - Presentation Graphics (7262-01-006)

Three learners on this contract had 2 essential files on disk with a creation date of 15 February
2007 (H0LIDAYBILBA01 and H0LIDAYBILBA02), while a third essential file
(H0LIDAYBILBA03) had both a creation and modification date of 27 February 2007. This
essential file had both creation and modification dates 12 days after the assessment date and 7
days after their assessments were marked on 19 February 2007.These learners were awarded
distinctions by the tutor for this assessment

The response from ACT and the tutor was that"due to the number of times that the disks were
accessed by Ash field, initially to correct the exams, the number of times the disks were
assessed by Ashfield/ FAS in verification, the incorrectly formatted date on a machine and the
inconsistency of floppy disks could lead to these errors' (Response 1)

Four other learners were missing an essential file from both their printouts and from their
computer disks (H0LIDAYBILBA03) and were awarded distinctions.

The response from the ACT and the tutor was thatBall mandatory and optional elements were
present when correcting exams; no bias was shown to any candidate' (Response 2)

4.3.1.2 Contract 1032 - Word processing 2 (7262-02-022)

The date that the assessment took place was 5 March 2007. For one learner, the printed
assessment material for this module was also dated 5 March 2007 (as would be expected), the
assessments were marked by the tutor on 13 March 2007 and the learner was deemed to have
achieved a distinction mark. However, data files created as part of the assessment showed
 both creation and modification dates of 5 April 2007, over 3 weeks later. ACT responded - see
Response 1 above.

Six learners were missing two essential files from their disks (PETLET) and (PETMERGE), one
learner was missing one essential file (PETMERGE) and another was missing an essential
printout (NEWSLETTER). These learners were marked as having achieved a distinction mark
in the assessment. ACT responded - see Response 1 above.

The NEWSLETTER printout for 6 learners (an essential point) was found to be identical and
contained the same 3 mistakes., the exact same layout and the same choice of clip-art. Three
of the files had a creation date of 28 August 2008: approximately two months after the contract
was finished. All 6 learners were awarded distinctions. ACT responded - see Response 1 and
also"due to teaching method used (mock examples of tests provided to practice) that lead to
students following a system they were comfortable with - similar mistakes were common",

4.3.1.3 Contract 1032 - Spreadsheets 2 (7262-02-023)

The date of this assessment was 8 March 2007. Many of the essential files on learners disks
also show a modified date of 8 March 2007 (as expected) but a creation date for the files of 4
April 2007, approximately 2 weeks after the assessments were graded by the tutor on 20
March 2007. All the disk labels for this module were completed in the handwriting of the tutor.

One learner screenshot for this exercise were created on 4 April 2007 but modified on 7 March
(a day before the assessment). The learner was awarded a distinction by the tutor.

Another learner awarded a distinction for this module on 20 March 2007 had essential
assessment files modified on 21 March 2007 and created on 30 March 2007.

In the case of a third learner's assessment printouts displayed large differences in the times;
14.09 (2.09 pm) on one sheet and 02.47 (2.47 am) on a second. While in the case of a fourth
learner there were also large variations in times on printouts that matched neither other
printouts nor the times on disks; 1.06 am and 1.54 pm.

Three of these learners were awarded a distinction while the fourth received a credit. ACT
responded -"see Response 1" above.

Eleven learners failed to print two forms onto a single portrait sheet (an essential point)

Two learners failed to print PRINTOUT1 in landscape (essential point) and one learner did not
have produce PRINTOUT1 (an essential point). ACT responded - "see Response 2" above.

4.3.1.4 Contract 1032 • Integrated Applications (7262-02-029)

The date of this assessment was 13 March 2007. Many of the essential files on learners disks
also show a modified date of 13 March 2007 (as expected) but a creation date for the files of 3
April 2007. approximately 4 days after the assessments were graded by the tutor on 30 March
2007.
 One learner's files for this assessment were created on 3 April 2007 but modified on 12 April
2007. Dates on printouts for the assessment corresponded to the date the assessment look
place; 13 March 2007.

Another two learners had files on disks that were both created and modified on 3 April 2007.

The labels for all of the learners' disks had their names entered in the handwriting of the tutor.
When interviewed by Internal Audit, the learner's maintained that they had always written their
names on their disk labels themselves and that they had never been supplied with disks with
labels completed by the tutor. ACT responded -"see Response V above.

One learner's printed assessment material as well as the material on disk appeared to be
copies of another learners material (including the name of the second candidate).

Almost all learners' assessment material contained a copy of the same incorrect pie-chart for
this module. The learners were asked in the assessment to produce a pie-chart based on the
combined sales for 4 salesmen for the first quarter of the year. The pie-chart submitted in each
case was based on that for the same individual salesman only. The same incorrect pie-chart
appeared many times in learner's assessment material, often up to 8 times per learner. All
learners were assessed as having attained a distinction for this assessment. ACT responded -
"I can t explain how this would happen, no bias was shown to any candidate'

4.3.1.5 Contract 1032 • conclusions

Missing essential elements

The audit confirmed that there were a number missing essential elements from the assessment
material for the modules examined in this contract. There was material missing from both the
printed assessment material and from disk. In the view of Internal Audit, ACT was unable to
provide a reasonable explanation as to how learners had passed these assessments in the
absence of such essential material (indeed, in many cases learners were awarded distinctions
for modules where essential elements were missing).

Inconsistent dates and times on contract 1032

In the case of the three learners highlighted in the Presentation Graphics module above, it is
not clear why two essential files should have creation dates of 15 February 2007 and the third
essential file have both a creation date and a modification date of 27 February 2007,12 days
later and 8 days after the assessments were graded by the tutor. The most likely explanation is
that someone accessed the disk at the later date and created the third file. In such a case the
integrity of the assessment process would seem to have been undermined.

In the case of the Spreadsheets 2 assessment, all learners' disks had labels completed in the
hand of the tutor and files on those disks had a creation date of 4 April 2007, almost one month
after the assessment took place. It is likely that these files were copied onto the disks at the
latter date. As mentioned above, the learner's maintained that they only ever used disks with
labels completed by themselves. The explanation provided by ACT was that the disks were
accessed on a number of occasions, that machine dates were possibly incorrect and that there
was generally an Inconsistency of floppy disks" do not explain what occurred. This is
particularly the case where there were unexplained differences between the times appearing
 on the same learners printed assessment material and where, in two cases, both the date
created and date modified characteristic of the assessment files were later than both the
assessment date and the date the assessment material was graded by the tutor.

Likewise, the explanations provided by ACT were insufficient to explain the differences in dates
and times for files on disks in the other two modules examined; Word Processing 2 and
Integrated Applications.

One explanation suggested by ACT was that the dates and times on the learner's computers
were not correct at the time of the assessments and that this therefore lead to incorrect dates
and times being applied to the learner's files created as part of the assessments. When
interviewed, the learners stated that they were conscious of ensuring that computer clock dates
and times were correct, that they were aware, during the course, of these values having been
correct and that the tutor had given the group a briefing on how to ensure that computer dates
and times were maintained accurately. Perhaps more significantly is the likelihood that if the
learners' computer dates and times were incorrect then it would be so consistently. In other
words, that certain learner's file dates and times would consistently be incorrect across the
range of assessment they completed. This was not the case, and learners with inconsistent
dates and times for one assessment often had correct dates and times for files relating to the
other assessments.

Finally, no explanation was provided by ACT as to why the date created for the NEWSLETTER
document in the Word Processing 2 module was 28 August 2007; two months after the course
finished. As mentioned above, 6 learners had the same pattern of errors in their supplied
version of this particular file. While all of the assessment material had been delivered to
Training Services in Dundalk soon after the contract finished in late June 2007, all the
assessment material was given back to ACT on August 2007 in order to allow them to address
shortcomings in the assessment material.

Consistent error patterns in assessments

The two most significant instances of the same errors repeated in learner's assignments
occurred in the Word Processing 2 and Integrated Applications modules. In the case of the
NEWSLETTER document produced in the Word assessment there were 6 candidates whose
documents were identical and contained a series of errors that suggest the document was
copied. In the Integrated Applications the consistent use of the same incorrect pie-chart also
suggested that the assessment material had been manipulated. The fact that all disks labels for
both these assessments were completed by the tutor and that the files on the disks had
creation dates subsequent to both the assessment dates and the date the assessments were
marked, points to culpability on the part of the tutor rather than the learners.

In the view of Internal Audit some of the assessment material for this contract was manipulated
to ensure that learners who had not achieved a sufficient level in the assessment were deemed
to have passed. This view is based on the absence of mandatory elements from sampled
assessments, unexplained date and time differences within assessment material and the
presence of consistent errors in assessments which could not have arisen randomly.

T» .
 4.3.2 North East / Contract 12271 Office Administration - April 2007 to December 2007

This contract was run in Navan, Co Meath and the ACT tutor used was the same as that for the
contract 1032 in Ballivor. Internal Audit raised issues in regard to the following City & Guilds
modules for this contract;

• Email 1 (7262-01-007)
• Word processing 2 (7262-02-022)
• Integrated Applications (7262-02-029)

4.3.2.1 Contract 1227 - Email 1 (7262-01-007)

One learner was missing two essential elements of the assessment (PRINTOUT1 and
PRINTOUT2). Two other learners were each missing one essential element for this
assessment. Three further learners were missing a number of essential elements from the
assessment and this was described by the tutor as being "due to a technical fault problem". All
6 learners were awarded distinctions. While the assessment took place on 21 September 2007
and was marked on 24 September, each of these three learners had blanks printouts dated 27
September 2007 included with their assessment material. It is not clear why these learners
were not required to re-sit the assessment once the technical fault was overcome.

For one learner two disks were supplied for this assessment one in the learners' handwriting
and another in the tutors handwriting. The first disk contained no files whole the second
contained files with a creation and modification dates of 25 September 2007 - 4 days after the
assessment was completed and a day after the assessment was marked by the tutor, The
learner was awarded a distinction. The possibility that an incorrect date setting on the learners
computer may have caused this difference can be largely discounted as the dates on the
printed assessment material for this learner, which are also sourced from the computer s date
and time setting, were correctly dated as 21 September 2007 (the date of the assessment)

The assessment required that the tutor send an initial email to each of the learners email
addresses and that the learners in turn respond to the tutor via email. As stated above, the
assessment took place on 21 September 2007 and the learners should have been allocated
two hours to complete the assessment. However the tutor sent the email to each of the
learners email addresses two days prior to the assessment on 19 September 2007. This
appeared at odds with restricting the assessment to the specified two hour timescale. ACT or
the tutor did not supply an explanation for this approach.

4.3.2.2 Contract 1227 - Word processing 2 (7262-02-022)

As was the case for contract 1032, some but not all of the assessments for this contract had
disks with labels completed in the handwriting of the tutor. In the case of the Word Processing
2 module the assessment took place on 20 June 2007 and the learners' files had a modified
date of the 20 June, suggesting that was when the material was last accessed. The date
created for the learners material was 21 August 2007 which was 2 months after the
assessment and the tutor completed the disk labels. The explanation provided by ACT for tutor
having completed the labels himself was that "in some instances the tutor prepared the disks
(labels and description) due to the fact some students were incapable of following basic pre-
 exam instructions and keeping to exam timing guidelines". No explanation was provided to
explain the need to copy files to these disks some two months after the assessment date.

4.3.2.3 Contract 1227 - Integrated Applications (7262-02-029)

The assessment for the Integrated Applications module took place on 6 July 2007.

All learners for this assessment had four mandatory files on their disks. Each of these files had
a creation date of 3 July 2007 and a date modified of 6 July. In addition, all saved screen shots
on disk also had the same dates created and modified as the four files. This suggested that the
files were created 3 days prior to the assessment and accessed, and updated, on the day of
the assessment.

For two of the learners the automatic date fields from their spreadsheet printouts displayed a
date of 17 September 2007, over 3 months after the assessment took place. The date of 17
September 2007 appears on 3-4 separate pages of the printed material of their assessments,
while the automatic date fields printed on some of the other pages of the same assessment
were the same as the sate of the assessment; 6 July 2007. However, like all the other learners,
the dates of the files on their disks showed a creation date of 3 July 2007 and a modified date
of 6 July. For some reason the assessments for this module were not marked until 14
September 2007

This assessment required that learners produce a pie-chart based on the total sales figures for
4 salesmen for the first quarter of the year. Ten of the candidates had the same incorrect
version of a pie-chart as part of their assessment. The values in this common pie-chart were
incorrectly based on the unit price of four items for sale. Each learner had used the incorrect
pie-chart a number of times in their assessment The pie-chart is an essential element with the
assessment All ten learners were awarded distinctions when the tutor marked the
assessments.

In response, ACT and the tutor stated that adue to teaching method used (mock examples of
tests provided to practice) this lead to the students following a system they were comfortable
with - similar mistakes were common. Due to the large number of people who made the similar
mistake it is possible the tutor followed the mistake as correct production of the pier-charf.

4.3.2.4 Contract 1227 • conclusions

Missing essential elements

The audit confirmed that there were a number missing essential elements from the assessment
material for the modules examined in this contract. There was material missing from both the
printed assessment material and from disk. In the view of Internal Audit, ACT were unable to
provide a reasonable explanation as to how learner's had passed these assessments in the
absence of such essential material.

Inconsistent dates and times

ACT and the tutor provided little by way of explanation for the differences between dates
recorded on files and printed assessment material highlighted above in each of the sampled
modules. Despite this, it is feasible that for some reason, a tutor may have copied all of the
learners' file from disks or a drive to floppy disks many weeks after the assessments (thus
ensuring that disks had file creation dates well after last modified dates). However, the dates
for the Integrated Applications assessment files strongly suggest that these were all created a
number of days in advance of the assessment While the assessment had an allocated time of
5 hours and could potentially be split over two or more days, the fact that the file creation dates,
modification dates and screenshots for all parts of the assessment were consistent would not
support this.

Likewise, the date and time differences on the E-mail module tend to suggest that some
assessment material was created both after the dates of the assessment and after the date the
assessments were graded by the tutor.

Consistent error patterns in assessments.

In the view of Internal Audit, ACT and the course tutor failed to give a reasonable explanation
as to why the same incorrect pie-chart appeared in ten of the learners' assessments for the
Integrated Applications module. Their explanation centred on possible confusion, initially by the
learners and then by the tutor, between mock tests or assessments and the official tests.

The learners were provided with sales figures of four salesmen for four separate products over
a number of quarterly sales periods. They were asked to develop a pie-chart based on the
sales total for the first quarter of the year. However, the pie chart produced by the ten learners
in question was based on the unit price of each of the four product lines sold. A pie-chart based
on product unit price is quite different from one based on sales totals for the first quarter and
while it is possible that one or two of the group might make such an error in an assessment, it
is highly unlikely than ten learners would independently make this error. In addition, the
incorrect pie-chart appeared a number of times in each of the learners' assessment material;
on average, the incorrect pie-chart was used 4 times by learners in their assessments.

In the view of Internal Audit, some of the assessment material for this contract was manipulated
to ensure that learners who had not achieved a sufficient level in the assessment were deemed
to have passed. This view is based on the absence of mandatory elements from sampled
assessments, unexplained date and time differences within assessment material and the
presence of consistent errors in assessments which could not have arisen randomly.

In addition to the verbal and written responses received from ACT and the tutors, Internal Audit
made a number of requests for a meeting with the person who was the tutor on both contract
1032 and contract 1227. The tutor's employment with ACT was terminated in early September
2008 and when contacted subsequently by Internal Audit, he stated that he was not in a
position to meet with Internal Audit

II 5 1 Irt
 4.3.3.2 Contract 0826 • Spreadsheets 2 (7261-422)

As with the Database 2 module above, the Spreadsheets 2 module also contained a multi-
choice assessment. The finding was again that the learners did not use the supplied answer
sheet and completed their answers in pencil.

The practical assessments completed for this module were PA01, PA02 and PA06. The PA01
assessment was structured in three parts. The first part involved spreadsheet design The
second and third parts of the PA01 assessment involved the creation of the actual spreadsheet
based on the design and the candidates were requested to save data files to disk and to have
printed out material based on different aspects of the saved spreadsheets.

However, Internal Audit was not able to find any printed material for parts two and three of the
assessment nor was it possible to locate any disks for the PA01 assessment. The word "pass"
was handwritten on the majority of the candidates question sheets which accompanied the part
1 material. In response, ACT and the tutor stated that there were "big problems with the disks
for this assignment These may have been done on another disk'. A subsequent examination
of all of the disks for the contract conducted jointly between ACT and FAS Internal Audit, did
not locate the files for parts two and three of the assessment.

The other two assessments completed for this module; PA02 and PA06 were likewise at
variance with the City & Guilds guidelines in that there was no printed material produced as
part of the assessments. ACT and the tutor stated that this material was u not printed but saved
to disk as agreed with FAS Curriculum Baldoyle". Internal Audit was unable to confirm whether
there was agreement on this matter between ACT and FAS Baldoyle,

4.3.3.3 Contract 0826 - conclusions

Multi-choice questions

For both the Database 2 and Spreadsheet 2 modules the designated answer sheets were not
used for the multi-choice assessments. While there was no evidence that the assessments
were conducted in a less than fair manner, the use of pencil rather than ink for the assessment
was not in accordance with the City & Guilds guidelines and was less than satisfactory.

Missing mandatory assessment material

The missing assessment material for the PA01 assessment in the Spreadsheets 2 module is
problematic. It was not possible to locate either the printed assessment matenal for parts two
and three of this assessment or any computer disks or files. Without this material being present
candidates should have not been deemed to have passed the assessment If there was a
technical problem with computer disks on the day of the assessment (as stated by ACT) then
the problems should have been addressed and the assessment completed properly at a later
stage.

Internal Audit has been unable to confirm with FAS Baldoyle, that in the case of this contract
agreement had been reached to save files to disk in the Spreadsheets 2 module, rather than
printing out as well as saving to disk, as the City & Guilds guidelines required at the time.

r»
4.3.3 Dublin North I Contract 08261 Computer Apps and Office Skills - June 2006 to
February 2007

This contract was run in Mulhuddart Co Dublin and Internal Audit raised issues in regard to the
following City & Guilds modules for this contract;

• Database 2 (7261-421)
• Spreadsheets 2 (7261-422)

4.3.3.1 Contract 0826 • Databases 2 (7261-421)

In relation to the Database 2 module, Internal Audit raised a number of queries regarding
mandatory City & Guilds assessment requirements which did not appear to accompany the
learners' material. According to the learner's instructions and tutors marking guidelines for
assessment PA01, a number of files must be saved to disk (NEWARD, HOSP2 and PA01), as
well as mandatory printouts such as

• List of patients with sore throats

• List of patients born in 1940's
• List of National Health patients
• Screen input form.

ACT and the tutor responded that the City & Guilds requirements "are not specific to Microsoft
Access, so the interpretation of the tutor was to create these items as tables in the same
database'. Likewise, rather than printing out lists the tutor asked the learners to save queries
on the database. ACT and the tutor stated that both these actions were done with the
agreement of the Curriculum Department in Baldoyle Training Centre. Internal Audit
subsequently confirmed that the material queried was in fact saved to the learners' databases
for each of the learners involved.

Unlike the 7262 City & Guilds e-Quals certification which was used in contracts 1032 and 1227
above, the older 7261 qualification used in this contract and contract 1240 below, used multi-
choice questions and a standardised answer sheet. Learners were provided with a question
relating to the module and four possible answers. The assessment instructions requested the
learners to use the answer sheet provided and to tick the appropriate selection against the
relevant question using black or blue ink.

In relation to the Database 2 module for contract 0826, none of the learners used the standard
answer sheet provided. Instead they used a pencil to indicate their selection on the question
sheet which was handed back to the tutor. In response to why the City & Guilds guidelines
were not used, ACT and the tutor stated that "as the exams were being corrected manually the
tutor felt that it was not necessary for the students to complete the answer sheet as this is
normally used for scanning purposes in correction and is not always provided by FAS with the
exams".
 4.3.4 Dublin North I Contract 12401 Computer Apps and Office Skills - May 2007 to
December 2007

This contract was also run in Mulhuddart, Co Dublin and Internal Audit raised issues in regard
to the following City & Guilds modules for this contract;

• Word Processing 2 (7261-420)

• Spreadsheets 2 (7261-422)

4.3.4.1 Contract 1240 • Word Processing 2 (7261-420)

The multi-choice assessment for the Word Processing module was examined as part of the
audit According to the original marking conducted by the tutor all 11 learners were deemed to
have passed the assessment However on examination it appeared that there were 40
incorrect markings made across the assessment The majority of the incorrect marking
occurred when incorrect answers were marked as correct but there were also a number of
other cases where correct answers supplied by the learner were marked as incorrect When
the City & Guilds marking template was applied correctly to the assessment material for this
contract 6 learners would have passed the assessment and 5 failed. The explanation supplied
by ACT and the tutor in this case was that aon reflection, there seems to have been a systems
failure in relation to the exam correction process

The PA03 assessment for this module occurred on 6 September 2007. The disk labels for the
PA03 assessment were all completed in the handwriting of the tutor. The tutor stated to Internal
Audit that this v/as done to save time.

In general the files for the PA03 assessment were created the day after the assessment (7
September 2007) and the tutor confirmed that the files had been saved to the trainee's hard
disks on the day of the assessment and copied to floppy disks the following day. One of the
learner's assessment disks was unusual in that it contained folders with the assessment
material for four other learners. In addition, each of these four learners had their own individual
disks as part of the assessment material. ACT and the tutor explained that "those four students
had initial problems saving to disk so as a back up I saved twice and forgot to delete" and the
learner whose name was on the diskhad missed nearly all of the module and I did not think he
would be able to complete the exam ...so had no disk for him...and the only disk I had
available was the backup one. I should have deleted the backups of the other students but was
underpressure somewhat then to include him".

The explanation provided by ACT and the tutor failed to fully explain this unusual situation. It is
suggested that the learner who arrived unexpectedly for the assessment used a disk that
already had been used to backup files created by four other learners. As these backup files
v/ere created by the learners during the various parts of the assessment, this suggested that
the unexpected learner completed the assessment after the four learners. This is not supported
by the fact that the learner's signature is one of the first on the official test signature sheet and
that the creation times of his files are similar to the other four learners. In addition, one of the
four learners whose assessment material is on the disk has files with both a date created and
date modified of 7 September 2007 despite also signing the official test signature sheet on 6
September 2007.
The PA05 assessment for the Word Processing 2 module was completed on the morning of 6
October 2007. The learners1 mandatory files., created as part of the assessment had times
modified of the morning (am) and times created of the afternoon (pm). Again the handwnting on
the disk labels was the same. The tutor confirmed that she had completed the labels to speed
up the process and had copied the files to disk in the afternoon, hence explaining the creation
time of the afternoon on 6 October 2007. However, for 7 of the learners, one mandatory file
(BROWSING) had a modification time of the late afternoon of 6 October 2007. A modification
date and time suggested that the file was last saved in the afternoon rather than in the morning.
Four of the times saved were after 4 pm when it is unlikely any learners were present.

4.3.4.2 Contract 1240 • Spreadsheets 2 (7261-422)

The date of the assessment for Spreadsheets 2 assessment was 6 November 2008. However,
the learners' files showed a variety of dates created and modified between the 6,7 and 8
November 2008. The explanation provided by ACT and the tutor was that "there was one inkjet
pnnter in the classroom 16 students. If the exam was before lunch I let the students print out
after lunch as this took 2-3 hours to complete...if the exam was in the afternoon I let the
students print out the next day". The files produced as part of the assessments could have
been accessed for printing at a later time without changing the date modified - by saving the
file, but this was not done.

4.3.4.3 Contract 1240 • conclusions

Multi-choice questions

No explanation, other than a reference to a "systems failure", was available for why the marking
of the multi-choice assessment for Word Processing 2 resulted in 5 learners being marked as
having passed, when a correct application of the marking would have failed them. The possible
explanation that the person marking the assessment had used an incorrect marking template is
unlikely given that the same answers to certain questions were marked as both correct and
incorrect for different learners. The net effect of the marking that occurred was that all 11
candidates were deemed to have passed. In the view of Internal Audit the marking of this
assessment was conducted in a manner that was less than satisfactory.

Dates for assessment files

While ACT and the tutor did provide explanations that explained apparent differences between
the dates and time of assessment files, not all matters have been explained to the satisfaction
of Internal Audit.

The explanation for the instance where one learner's disk also contained the assessment
material for four other learners in the Word Processing 2 module is lacking on a couple of
points. The dates and times of assessment files for the learner for whom a disk had to be
found, do not support the explanation that four other learners assessment material had been
backed up previously before he received the disk. In addition, his signature on the assessment
attendance sheet is earlier than some of the four learners on the disk and one of those had
dates for files the following day after the assessment.
The findings in relation to the PA06 file (BROWSING) suggest that this file was updated well
after the assessment was completed on 6 October 2007 for 7 of the learners and, in 4 cases, it
appeared that the file was last updated after the learners had left for the day (after 4pm).

in summary, it is the view of internal Audit that efforts were made on the part of the tutor to
ensure that learners, who had not achieved a sufficient level in some assessments, were
assisted in achieving a pass level after the assessment was completed.