Document Control Revision History Date Version Description Author 9/30/09 0.01 Initial draft Pete Peterson/Tad McDonald 10/20/09 0.07 Incorporate QA feedback Pete Peterson/Tad McDonald 10/23/2009 0.10 Incorporate Workshop feedback Pete Peterson/Tad McDonald 11/4/2009 0.11 Incorporate Initial Deliverable Review Comments Pete Peterson/Tad McDonald 11/5/2009 1.00 Final Version Pete Peterson/Tad McDonald
Quality Review History Date Reviewer Comments 10/19/2009 Lori Nolen Comments provided on printed copy 11/3/2009 Audit Staff Comments provided on printed copy 11/5/2009 Denise Birch Deliverable Approved
Page ii
Table of Contents
0 PURPOSE AND OBJECTIVES OF THE AUDIT CORE PROCESS ............................. 1 0.1 BEGINNING AND ENDING POINTS ..................................................................................................................... 2 0.2 AUDIT DIAGRAM .............................................................................................................................................. 3 0.3 AUDIT ACTIVITIES ........................................................................................................................................... 4 1 PURPOSE AND OBJECTIVES OF THE AUDIT BUSINESS ACCURACY MEASUREMENT (BAM) SUB-PROCESS ............................................................................... 6 1.1 BEGINNING AND ENDING POINTS ..................................................................................................................... 6 1.2 AUDIT BUSINESS ACCURACY MEASUREMENT (BAM) DIAGRAM ................................................................. 7 1.3 AUDIT - BAM ACTIVITIES ............................................................................................................................. 10 1.4 AUDIT BAM PROCESS ATTRIBUTES ............................................................................................................ 12 1.4.1 Elapsed Time ....................................................................................................................................... 12 1.4.2 Paper Documentation ........................................................................................................................... 12 1.4.3 Business System Data Entry & Data Transfer ..................................................................................... 15 1.4.4 Communication .................................................................................................................................... 16 1.4.5 Opportunities ....................................................................................................................................... 18 2 PURPOSE AND OBJECTIVES OF THE AUDIT - BTQ QUARTERLY REPORTING SUB-PROCESS ........................................................................................................................... 20 2.1 BEGINNING AND ENDING POINTS ................................................................................................................... 20 2.2 AUDIT - BENEFIT TIMELINESS QUALITY (BTQ) QUARTERLY REPORTING DIAGRAM .................................. 21 2.3 AUDIT BTQ QUARTERLY REPORTING ACTIVITIES ................................................................................... 23 2.4 AUDIT BTQ QUARTERLY REPORTING PROCESS ATTRIBUTES .................................................................. 24 2.4.1 Elapsed Time ....................................................................................................................................... 24 2.4.2 Paper Documentation ........................................................................................................................... 25 2.4.3 Business System Data Entry & Data Transfer ..................................................................................... 26 2.4.4 Communication .................................................................................................................................... 27 2.4.5 Opportunities ....................................................................................................................................... 27 3 PURPOSE AND OBJECTIVES OF THE AUDIT BTQ NON-MONETARY REVIEWS SUB-PROCESS ....................................................................................................... 29 3.1 BEGINNING AND ENDING POINTS ................................................................................................................... 29 3.2 AUDIT BTQ NON-MONETARY REVIEWS DIAGRAM .................................................................................. 30 3.3 AUDIT BTQ NON-MONETARY REVIEWS ACTIVITIES ............................................................................... 31 3.4 AUDIT BTQ NON-MONETARY REVIEWS PROCESS ATTRIBUTES .............................................................. 32 3.4.1 Elapsed Time ....................................................................................................................................... 32 3.4.2 Paper Documentation ........................................................................................................................... 32 3.4.3 Business System Data Entry & Data Transfer ..................................................................................... 33 3.4.4 Communication .................................................................................................................................... 34 3.4.5 Opportunities ....................................................................................................................................... 35 4 PURPOSE AND OBJECTIVES OF THE AUDIT BTQ DATA VALIDATION SUB-PROCESS ........................................................................................................................... 36
Page iii
4.1 BEGINNING AND ENDING POINTS ................................................................................................................... 36 4.2 AUDIT BTQ DATA VALIDATION DIAGRAM .............................................................................................. 37 4.3 AUDIT BTQ DATA VALIDATION ACTIVITIES ............................................................................................ 38 4.4 AUDIT BTQ DATA VALIDATION PROCESS ATTRIBUTES ........................................................................... 39 4.4.1 Elapsed Time ....................................................................................................................................... 39 4.4.2 Paper Documentation ........................................................................................................................... 40 4.4.3 Business System Data Entry & Data Transfer ..................................................................................... 40 4.4.4 Communication .................................................................................................................................... 41 4.4.5 Opportunities ....................................................................................................................................... 41
Page 1
0 PURPOSE AND OBJECTIVES OF THE AUDIT CORE PROCESS The Audit core process is comprised of four distinct sub-processes: Benefit Accuracy Measurement (BAM), Benefit Timeliness and Quality (BTQ) Quarterly Reporting, BTQ Non-Monetary Reviews, and BTQ Data Validation. Benefit Accuracy Measurement (BAM) BAM is a federally mandated program providing the basis for assessing the accuracy of Unemployment Insurance payments made to claimants and the accuracy of benefits denied to claimants. BAM is also a diagnostic tool for Federal and State workforce agency staff members to assist in error identification and causal analysis. BAM supports the Agencys desire to implement corrective actions and to track the effectiveness of those remedial actions. The objectives of the Benefit Accuracy Measurement program are to: Assess the accuracy of Unemployment Insurance payments Assess improvements in program accuracy and integrity Encourage more efficient administration of the Unemployment Insurance program A weekly, representative sample of UI payments and disqualifying ineligibility determinations are drawn (from the U.S. Department of Labor UI State Menu system), investigated, and audited intensively to determine whether they were properly administered to claimants and whether the claimants were properly paid or appropriately denied. Up to ten cases each for paid and denial determinations are randomly selected for audit. The normal annual caseload is 480 paid claims and 450 denied determinations. The specific number of required audits may be changed by the U.S. Department of Labor depending upon prevailing conditions (i.e., during periods of high unemployment, the Federal government may reduce the number of audits to allow an agency to re-assign human resources to other tasks such as claim adjudication). Benefit Timeliness Quality (BTQ) Quarterly Reporting The determination of a claimants eligibility for unemployment insurance (UI) benefits is a critical UI program function. When issues arise that may affect a claimants past, present, or future benefits, the adjudicator is responsible for determining the claimants eligibility for those benefits. Such determinations may also affect an employers liability for benefit charges, depending on the type of issues adjudicated. The adjudicators work impacts the rights of both claimants and employers. The BTQ unit is responsible to report quarterly to the U.S. Department of Labor (DOL) for nonmonetary claims. Benefit Timeliness Quality (BTQ) Non-Monetary Reviews UC has instituted an internal quality review process under the auspices of the BTQ unit to perform periodic assessments of adjudicators in each Adjudication Hub. The information gathered through the reviews provides insight to existing or potential issues that may be causing difficulties in particular adjudication areas (e.g., separation issues). Managers can use that information to take corrective action which may include arranging for remedial training, modifying policies and procedures, and/or suggesting changes to the automated system(s) to address less than satisfactory performance. Benefit Timeliness Quality (BTQ) Data Validation The comprehensive Federal data validation program is necessary to ensure the accuracy of Unemployment Insurance Required Reports (UIRR) data. UIRR data are used for economic statistics, to allocate UI administrative funding based on state workload, to measure state claimant eligibility criteria
Page 2
and performance in providing benefits, and to account for fund utilization. It is therefore essential that states report UIRR data accurately and uniformly. States report to DOL on a monthly and quarterly basis under the Unemployment Insurance Required Reports system. States are required to validate reported data every third year, except for data elements used to calculate Government Performance and Results Act (GPRA) measures. GPRA data are validated annually. The validation year will coincide with the State Quality Service Plan (SQSP) performance year. Validation of any reporting period during the twelve months beginning April 1 and ending March 31 will be considered part of the validation year. The SQSP is the vehicle through which states submit plans to implement validation or to revalidate failed items. GPRA and SQSP reporting is covered in a separate process, Federal Reporting. Note: Many units within AWI perform audit functions to monitor their performance. The processes described here address specifically BAM and BTQ and are not meant to encompass other audit functions within AWI. The following sections describe the activities associated with the Audit functions in UC. 0.1 BEGINNING AND ENDING POINTS BAM Beginning Point: A list of completed claims is downloaded from the U.S. Department of Labor (DOL) State Menu system. Ending Point: The BAM investigators case file is sent to imaging or a storage facility.
BTQ Quarterly Reporting Beginning Point: The UC Benefits system generates a random list of claims for review on the first workday of each quarter. Ending Point: The BTQ investigators case file is copied to a local, shared network drive.
BTQ Non-Monetary Reviews Beginning Point: An Adjudication Hub is selected for review. Ending Point: The results of the claims reviews are recorded in an internal reporting document.
BTQ Data Validations Beginning Point: A list of necessary populations to be validated is received from the U.S. Department of Labor (DOL) State Menu system. Ending Point: The results of the validations are submitted via the U.S. Department of Labor (DOL) State Menu system.
Page 3
0.2 AUDIT DIAGRAM
BAM Unit Benefit Accuracy Measurement BTQ Unit Benefit Timeliness & Quality Process Start + 11.01 BAM Weekly Reviews Process Ends Process Start Process Start Process Start (Data Validation) + 11.02 Quarterly Reports Process Ends + 11.03 Non-Monetary Reviews Process Ends + 11.04 Data Validation Process Ends
Exhibit 1 Audit Process Flow
Page 4
0.3 AUDIT ACTIVITIES Exhibit 2 Audit Activities lists the activities that make up the Audit Core Process. The following is a description and definition of each of the columns in this table. Index A number for use in referencing the activity Activity Label A short name for that activity that is used to represent it in the detailed workflow diagram. Activity Description The full-text description of that activity Actors The job or role title of the individuals having a primary role for that activity
Index Activity Label Activity Description Actors 11.01 BAM Weekly Reviews Each week (Monday mornings), a supervisor in the BAM unit logs on to the State Menu system (U.S. DOL application) and downloads cases for investigation/audit.
Investigators within the BAM Unit audit their assigned cases for both Paid and Denied claims (they have a maximum of 120 days to complete their investigations). The investigator enters the results in the U.S. DOL State Menu system.
Once an investigation/audit is complete, the results are returned to the BAM unit supervisor for review. The supervisor reviews the entire case file and the entries in the State Menu system, approves (or coordinates corrective action with investigators), and closes the case(s). Benefit Accuracy Measurement Unit 11.02 Quarterly Reports A U.S. DOL algorithm is coded within the UC Benefits system to randomly select, on the first day of each quarter, non-monetary cases (50 separation, 50 non-separation) for BTQ reporting. The audit reports are due by the 20 th of the following month.
(Note: The Quarterly Report has not been run for quite a while due to the re-assignment of BTQ personnel to support the increased UC workload.)
Reviewers in the BTQ unit examine the cases assigned to them by a BTQ supervisor, score each according to established guidelines, and return the results to the unit supervisor for additional processing.
The results of the BTQ audits are recorded in the U.S. DOL State Menu system. Benefit Timeliness & Quality Unit 11.03 Non-Monetary Reviews The BTQ unit conducts periodic audits of the Adjudication Hubs. A selection of claims (10 cases per adjudicator) is randomly extracted from the UC Benefits system for a given date range and assigned to BTQ examiners for review.
Examiners review cases assigned to them and score each according to established guidelines.
A BTQ unit supervisor compiles the results of the reviews and communicates the results to the Adjudication Hub supervisors for further action. Benefit Timeliness & Quality Unit
Page 5
Index Activity Label Activity Description Actors 11.04 Data Validation A BTQ supervisor accesses the U.S. DOL State Menu system and downloads validation data requirements established by the U.S. Department of Labor. Validation and reporting requirements are divided into populations by DOL and sent to all states in June or July. The agencys previous performance (Pass/Fail criteria) influences the validation requirements for content (which populations to validate) and frequency (usually, a passing score relieves the agency of reporting for a given population for three years while a failing score necessitates corrective action, re- validation, and reporting on a more frequent basis).
The results of the BTQ audits are recorded in the U.S. DOL State Menu system. Benefit Timeliness & Quality Unit
Exhibit 2 Audit Activities
Page 6
1 PURPOSE AND OBJECTIVES OF THE AUDIT BUSINESS ACCURACY MEASUREMENT (BAM) SUB-PROCESS The Audit core process is comprised of four distinct sub-processes: Benefit Accuracy Measurement (BAM), Benefit Timeliness and Quality (BTQ) - Quarterly Reporting, BTQ Non-Monetary Reviews, and BTQ Data Validation. Benefit Accuracy Measurement (BAM) BAM is a federally mandated program providing the basis for assessing the accuracy of Unemployment Insurance payments made to claimants and the accuracy of benefits denied to claimants. BAM is also a diagnostic tool for Federal and State workforce agency staff members to assist in error identification and causal analysis. BAM supports the Agencys desire to implement corrective actions and to track the effectiveness of those remedial actions. The objectives of the Benefit Accuracy Measurement program are to: Assess the accuracy of Unemployment Insurance payments Assess improvements in program accuracy and integrity Encourage more efficient administration of the Unemployment Insurance program A weekly, representative sample of UI payments and disqualifying ineligibility determinations are drawn (from the U.S. Department of Labor UI State Menu system), investigated, and audited intensively to determine whether they were properly administered to claimants and whether the claimants were properly paid or appropriately denied. Up to ten cases each for paid and denial determinations are randomly selected for audit. The normal annual caseload is 480 paid claims and 450 denied determinations. The specific number of required audits may be changed by the U.S. Department of Labor depending upon prevailing conditions (i.e., during periods of high unemployment, the Federal government may reduce the number of audits to allow an agency to re-assign human resources to other tasks such as claim adjudication). Note: Many units within AWI perform audit functions to monitor their performance. The processes described here address specifically BAM and are not meant to encompass other audit functions within AWI. The following sections describe the activities associated with the Audit - BAM functions in UC. 1.1 BEGINNING AND ENDING POINTS BAM
Beginning Point: A list of completed claims is downloaded from the U.S. Department of Labor (DOL) State Menu system.
Ending Point: The BAM investigators case file is sent to imaging or a storage facility.
Page 7
1.2 AUDIT BUSINESS ACCURACY MEASUREMENT (BAM) DIAGRAM Supervisor 11.01.01 Download Cases Investigator US Department of Labor State Menu System 11.01.02 Assign Investigator State Menu System 11.01.04 Generate Code Sheets Winscape 11.01.06 Update Investigator Open Case Document Word 11.01.07 Save Open Case Document to Investigator folder Local S: Drive 11.01.05 Move Case files to Investigator folder Local S: Drive 11.01.08 Request Weekly Certifications UC 11.01.03 Run Script UI Sun (Putty) Print Job Request Process Start Continued on Following Page
Page 8
Employer Investigator Supervisor 11.01.09 Retrieve Open Case Assignments Local S: Drive 11.01.11 Retrieve Pre-Investigation Documents (Claimant) UC, Imaging, SunTax, Employ Florida 11.01.13 Assemble Investigation Case Folder 11.01.15 Investigate Case Word 11.01.10 Print Code Sheets 11.01.14 Send Employer Wage Verification Form & Claimant Questionnaire Claimant Mail/Email/Fax Wage Verification Form Mail/Fax/Email Questionnaire Phone/Mail/Fax/Email Phone/Mail/Fax/Email Other AWI Units If Determination Error Request Redetermination If Possible Fraud Notify Benefit Payment Control Notify Appeals Clerks Office If Under Appeal 11.01.16 Inform Other Areas BD15V Weekly Certifications 11.01.12 Retrieve Pre-Investigation Documents (Employer) SunTax State Menu System Continued from Previous Page Continued on Following Page
Page 9
Supervisor Investigator 11.01.21 Review Case 11.01.22 Make Corrections 11.01.17 Complete Code Sheet 11.01.18 Update Code Sheet Data 11.01.20 Return Case Folder State Menu System 11.01.19 Close Case State Menu System 11.01.23 Close Case State Menu System 11.01.25 Store File (3 Years) If Paid Case 11.01.26 Send File to Imaging If Denied Case Process Ends Incorrect Continued from Previous Page Investigator If Improper Payment 11.01.24 Review Case Interoffice Memo Interoffice Memo
Exhibit 3 Audit - BAM Process Flow
Page 10
1.3 AUDIT - BAM ACTIVITIES Exhibit 4 Audit - BAM Activities lists the activities that make up the Audit BAM sub-process. The following is a description and definition of each of the columns in this table. Index A number for use in referencing the activity Activity Label A short name for that activity that is used to represent it in the detailed workflow diagram. Activity Description The full-text description of that activity Actors The job or role title of the individuals having a primary role for that activity
Index Activity Label Activity Description Actors 11.01.01 Download Cases The U.S. Department of Labor (DOL) provides a random selection of Paid (PCA) and Denied (DCA) claims for state audit. Each week (Monday mornings), a supervisor in the BAM unit logs on to the State Menu system (U.S. DOL application) and downloads cases for investigation/audit. (Note: the cases are automatically converted.) US Department of Labor (DOL) Supervisor 11.01.02 Assign Investigator The supervisor selects Case Assignment from the State Menu selection screen and assigns each case to a specific investigator (investigators are identified with a unique number). The process is completed for Paid (PCA) cases and repeated for Denied (DCA) cases. Supervisor 11.01.03 Run Script The supervisor logs onto the uisfl2 (UI Sun) system and runs a preloaded script to prepare the coding sheets to be used for the case reviews. Supervisor 11.01.04 Generate Code Sheets Each investigation has its own code sheet to record the results of the examination. The BAM supervisor generates the code sheets using the Winscape application. Supervisor 11.01.05 Move Case files to Investigator folder Each investigator is assigned a folder on the network S: drive. The coding sheets for the cases assigned to each investigator are placed in their S: drive folder. Supervisor
11.01.06 Update Investigator Open Case Document The supervisor maintains a Word document containing the case assignments for each investigator. Supervisor 11.01.07 Save Open Case Document to Investigator Folder The Word document created by the Supervisor is placed in each investigators S: drive folder. Supervisor 11.01.08 Request Weekly Certifications For Paid (PCA) cases, the Supervisor runs a print request to create Weekly Certifications from the UC Benefits system. Supervisor 11.01.09 Retrieve Open Case Assignments The investigator accesses the network S: drive folder to locate the cases assigned (Word document placed there by the Supervisor). Investigator 11.01.10 Print Code Sheets The code sheets are printed from the State Menu system. Investigator
11.01.11 Retrieve Pre- Investigation Documents (Claimant) Using a variety of systems at the investigators disposal (UC, Imaging, SunTax, Employ Florida), the investigator gathers information to support the case examination. In addition, the printed output from the BD15V transaction is returned to the investigator (the print job runs overnight). Investigator 11.01.12 Retrieve Pre- Investigation Documents (Employer) The investigator retrieves employer-related information from the SunTax system. Investigator
Page 11
Index Activity Label Activity Description Actors 11.01.13 Assemble Investigation Case Folder The investigator follows a common case folder assembly process to standardize the content and order of documents in the case investigation folder (one folder is prepared for each case). Investigator 11.01.14 Send Employer Wage Verification Form & Claimant Questionnaire The investigator sends a questionnaire to claimants, asking them to supply information about their particular claim. At the same time, employers are asked to verify the information about wages using the Wage Verification form.
(Note: Claimants are informed the failure of responding to the questionnaire may result in the suspension of benefit payments (if any are being currently paid). If the Claimant does not respond, further payments, if any, are suspended. Once a response is provided, payments, if any, are resumed. No such leverage is available regarding employers. Investigators 11.01.15 Investigate Case Investigators review all the information at their disposal to investigate the cases assigned to them. They may have to communicate with claimants and employers on an ad-hoc basis to clarify existing information or to gather additional information. Investigator 11.01.16 Inform Other Areas If, during the course of an investigation, it is found that a determination error may have been made, investigators may request redetermination from the Wage Determination unit. If the case being investigated is under appeal, the Clerks Office in Appeals is notified. If the possibility of fraud exists, the Benefit Payment Control (BPC) unit is informed. The results of the redetermination and/or fraud investigation are returned to the Investigator. Investigator 11.01.17 Complete Code Sheet For each assigned case, the investigator completes a separate code sheet that represents the results of the investigation. Investigator 11.01.18 Update Code Sheet Data The investigator transfers the information from the paper code sheet to the State Menu system. Investigator 11.01.19 Close Case After the code sheet data is input to the State Menu system, the investigator closes the case and prepares the case folder to be given back to the BAM supervisor. Investigator 11.01.20 Return Case Folder After review for completeness and accuracy, the investigator returns the case folder(s) to the BAM supervisor. Investigator 11.01.21 Review Case The Supervisor reviews the case information and, in an iterative process, directs the investigator to make necessary corrections to the case. Supervisor 11.01.22 Make Corrections When notified by the BAM supervisor, the investigator makes modifications to the case. Investigator 11.01.23 Close Case Once the case investigation is complete to the satisfaction of the BAM supervisor, the case is closed in the State Menu system. If an Improper Payment has been made, the Supervisor will prepare a memorandum for the Administrators review. Supervisor 11.01.24 Review Case The Administrator will review the case. If the improper payment was made in violation of law or policy, a memorandum will be prepared summarizing the results of the investigation and sent to the Adjudication Hub Supervisor and/or AWI Management for appropriate action. Administrator 11.01.25 Store File (3 Years) Completed cases are archived. Paid cases are retained for three years in hard copy. Supervisor
Page 12
Index Activity Label Activity Description Actors 11.01.26 Send File to Imaging Denied payment cases are sent to imaging for capture and retention. Supervisor Exhibit 4 Audit - BAM Activities
1.4 AUDIT BAM PROCESS ATTRIBUTES The following sections define and describe sub-process specifics for elapsed time, paper documentation, business system use, communications, and process improvements. The information is arranged in this way to support analysis of these activities, both individually and collectively. 1.4.1 Elapsed Time Exhibit 5 Audit - BAM Activity Elapsed Time lists the time required to complete each of the activities that make up the Audit BAM sub-process. The times are estimated except where noted. The following is a description and definition of each of the columns in this table. Index A number for use in referencing the activity. Activity Label A short name for that activity that is used to represent it in the detailed workflow diagram. Process Time The hands-on time it takes to complete the activity (does not include mail, transport, or time spent waiting for another activity to complete). Cycle Time The total elapsed time from the beginning of the activity until it is complete, inclusive of all delays from transport, mail, other processes, and work load. Time Goals / Limits Any time limits or goals, from any source internal or external pertaining to this activity. Limit/ Goal Source The source of the limit or goal listed above (Florida Statutes, AWI policy, etc.) Speed Bumps Specific, recurring difficulties that slow the progress of this activity.
Index Activity Label Process Time Cycle Time Limit or Goal Time Limit or Goal Source Speed Bumps 11.01 BAM 4-6 hours per file 120 Days All investigations must be completed within 120 days. ETA 395 Handbook published by Department of Labor. Gathering documentation from multiple systems. Obtaining responses from Claimant & Employer. 11.01.11 Retrieve Pre- Investigation Documents Overnight Overnight Official reproductions of certain documents must be requested and printed during an overnight process. AWI Staff The overnight print process, while generally cumbersome, does not create a significant delay in the BAM process. Exhibit 5 Audit - BAM Activity Elapsed Time 1.4.2 Paper Documentation Exhibit 6 Audit - BAM Paper Documentation lists the paper documents required to complete each of the activities that make up the Audit BAM sub-process. The reference column refers to the originator and title of policy, procedure, or instruction that contains the form or specifies the content of the document. More than one document may be required to carry out a single activity, so each is listed. The following is a description and definition of each of the columns in this table.
Page 13
Index A number for use in referencing the activity. Activity Label A short name for that activity that is used to represent it in the detailed workflow diagram. Document Title Short title, form number or other designation for the document. Document Description General description of the document contents and purpose. Reference or Source Originator and title of the source instruction or policy.
Index Activity Label Document Title Document Description Reference or Source 11.01.06 Update Investigator Open Case Document Investigator Open Case Document Word Document used to track all open cases for each Investigator BAM Supervisor 11.01.08 Request Weekly Certifications BD15V Weekly Claim Certification Report (for Key Week). Printed from the UC System. AWI Investigator 11.01.10 Print Code Sheets Code Sheet Printout from the State Menu System. Completed by hand. (4 Versions Paid Claims, Separation Denial, Monetary Denial, Non-Separation Denial) AWI Investigator 11.01.11 Retrieve Pre- Investigation Documents (Claimant) BI08 UC/SunTax Wage Crossmatch. Printed from the UC System. AWI Investigator 11.01.11 Retrieve Pre- Investigation Documents (Claimant) BD11 Wage Transcript & Determination. Printed from the UC System. AWI Investigator 11.01.11 Retrieve Pre- Investigation Documents (Claimant) BH01 Claim History. Printed from the UC System. AWI Investigator 11.01.11 Retrieve Pre- Investigation Documents (Claimant) BF01 Overpayment History. Printed from the UC System. AWI Investigator 11.01.11 Retrieve Pre- Investigation Documents (Claimant) BQ01 Determination History. Printed from the UC System. AWI Investigator 11.01.11 Retrieve Pre- Investigation Documents (Claimant) Claim Form Printed from the EIS (Imaging System) AWI Investigator 11.01.11 Retrieve Pre- Investigation Documents (Claimant) Wage Items Screen Print Printed from the Department of Revenue SunTax System. AWI Investigator 11.01.11 Retrieve Pre- Investigation Documents (Claimant) SIDI Out of State Wages Summary Report. Printed from the UC System. AWI Investigator
Page 14
Index Activity Label Document Title Document Description Reference or Source 11.01.11 Retrieve Pre- Investigation Documents (Claimant) IBIQ Out of State Wages Detail Report. Printed from the UC System. AWI Investigator 11.01.11 Retrieve Pre- Investigation Documents (Claimant) Job Registration History Printed from the EmployFlorida System. AWI Investigator 11.01.12 Retrieve Pre- Investigation Documents (Employer) Employer Report Retrieved from the Department of Revenue SunTax System. Screen print of the Detail 1 and Detail 2 tabs. AWI Investigator 11.01.13 Assemble Investigation Case Folder Investigation Activity Log Blank template (Word). Completed by hand. AWI Investigator 11.01.13 Assemble Investigation Case Folder Case Checklist Blank template (Word). Completed by hand. AWI Investigator 11.01.14 Send Wage Verification Form & Claimant Questionnaires Claimant Questionnaire Document used to gather information from the Claimant to be used during the Audit Investigation (4 Versions - Paid Claims, Separation Denial, Monetary Denial, Non-Separation Denial). Attached to a Cover Letter generated from a Word Template. AWI Investigator 11.01.14 Send Wage Verification Form & Claimant Questionnaires Information for the Claimant on the Unemployment Compensation Benefit Accuracy Measurement Program (LES UCR Bulletin 1 (Rev. 6/00) Pamphlet inserted with the Claimant Questionnaire containing information regarding the BAM program. AWI Investigator 11.01.14 Send Wage Verification Form & Claimant Questionnaires Wage Verification Form Document used to gather information from the Employer to be used during the Audit Investigation. Consists of two parts - Claimant Wage Verification, Quarterly Earnings Amounts. Attached to a Cover Letter generated from a Word Template. AWI Investigator 11.01.15 Investigate Case School Verification Form Document sent to the School that Claimant was attending during key week. Attached to a Cover Letter generated from a Word Template. AWI Investigator 11.01.15 Investigate Case (Private) Employment Agency Verification Document sent to a private employment agency to validate Claimant's job search history. Attached to a Cover Letter generated from a Word Template. AWI Investigator 11.01.15 Investigate Case Work Search Verification Regular Employer Document sent to a potential employer to validate Claimant's job search history. Attached to a Cover Letter generated from a Word Template. AWI Investigator 11.01.15 Investigate Case Work Search Verification - Labor Organization Document sent to a union or labor organization to validate Claimant's job search history. Attached to a Cover Letter generated from a Word Template. AWI Investigator
Page 15
Index Activity Label Document Title Document Description Reference or Source 11.01.16 Inform Other Areas Memo Request for Monetary Redetermination Word Template. Completed to request a redetermination based on the results of the investigation. (Note: The results of the redetermination are returned to the BAM Investigator) AWI Investigator 11.01.17 Complete Code Sheet Summary of Investigation Document used to present summary investigation information to BAM Supervisor. Investigation (4 Versions - Paid Claims, Separation Denial, Monetary Denial, Non-Monetary Non-Separation Denial). AWI Investigator 11.01.18 Update Code Sheet Data Data Collection Instrument (DCI) Report Printout from the State Menu System after data entry is completed. (4 Versions Paid Claims, Separation Denial, Monetary Denial, Non- Separation Denial) AWI Investigator Exhibit 6 Audit - BAM Paper Documentation 1.4.3 Business System Data Entry & Data Transfer Exhibit 7 Audit - BAM Data Entry & Data Transfer lists the business systems that are employed directly (touched) or indirectly (data transfer) in the course of the Audit BAM sub-process. The following is a description and definition of each of the columns in this table. Index A number for use in referencing the activity. Activity Label A short name for that activity that is used to represent it in the detailed workflow diagram. Business System The name or designation of the system touched (if any). Purpose or Intent The business objective accomplished by using the system or systems. Mode Choose one; entry is made by a person, transfers are from system to system.
Index Activity Label Business System Program or Intent Mode 11.01.03 Run Script UI Sun System (Putty) Command Line Program - Converts downloaded cases to a format to be used for Winscape Transfer 11.01.04 Generate Code Sheets Winscape Command Line Program - Creates Code Sheets for all selected cases for review. Output is a series of Word documents Transfer 11.01.08 Request Weekly Certifications UC BD15V - By SSN/Date Weekly Claim Certification Report (for Key Work) Inquiry 11.01.11 Retrieve Pre- Investigation Documents (Claimant) UC Retrieve by SSN: (BI08) UC/SunTax Wage Crossmatch (BD11) Wage Transcript & Determination (BH01) Claim History (BF01) Overpayment History (BQ01) Determination History Inquiry 11.01.11 Retrieve Pre- Investigation Documents (Claimant) EIS (Imaging) Input SSN on the Search Screen. Results are displayed and the Original Claim Form is printed. Inquiry
Page 16
Index Activity Label Business System Program or Intent Mode 11.01.11 Retrieve Pre- Investigation Documents (Claimant) SunTax Input SSN on the Search Screen. Select and print the Wage Items Screen (By Employer for Base Period & Following) Inquiry 11.01.11 Retrieve Pre- Investigation Documents (Claimant) UC Retrieve by SSN (SIDI) Out of State Wages (Search) If results are found, input SSN and State to retrieve: (IBIQ) Out of State Wages (Detail) Inquiry 11.01.11 Retrieve Pre- Investigation Documents (Claimant) EmployFlorida Input SSN on the Assist Screen. Print the Job Registration History Inquiry 11.01.12 Retrieve Pre- Investigation Documents (Employer) SunTax Input the Employer # on the Search Screen. Print the: Detail 1 Tab (Basic Demographics) Detail 2 Tab (Tax Rate/NAICS Code) Inquiry Exhibit 7 Audit - BAM Data Entry & Data Transfer 1.4.4 Communication Exhibit 8 Audit - BAM lists the messages, such as the US mail, emails, phone calls and any other correspondence, regularly used in the normal course of the Audit BAM sub-process. Messages are sent to convey required documentation, inform recipients about the status of an activity, to trigger the start or end of other activities, to grant or withdraw authorization, etc. This section only concerns messages sent during normal (as intended) operations. There may be more than one message associated with any activity in this process. The following is a description and definition of each of the columns in this table. Index A number for use in referencing the activity. Activity Label A short name for that activity that is used to represent it in the detailed workflow diagram. Message Type US mail, email, phone call, etc. Message Sender The organization or role of the message sender. Message Purpose The intended purpose for sending the message. Message Recipient The organization or role of the intended recipient. Occasion Event or action that triggers the requirement to send the message
Index Activity Label Message Type Sender Message Purpose Message Recipient Occasion 11.01.14 Send Wage Verification Form & Claimant Questionnaires Mail Investigator Claimant Questionnaire Claimant During the Investigation into a determination. 11.01.14 Send Wage Verification Form & Claimant Questionnaires Mail/Fax Investigator Wage Verification Form Employer During the Investigation into a determination. 11.01.15 Investigate Case Phone/Mail/Fa x/Email Investigator To gather additional information as necessary. Claimant/Emplo yer During the Investigation into a determination.
Page 17
Index Activity Label Message Type Sender Message Purpose Message Recipient Occasion 11.01.15 Investigate Case Mail/Fax Investigator To gather additional information as necessary. School During the Investigation into a determination. 11.01.15 Investigate Case Mail/Fax Investigator To gather additional information as necessary. Private Job Search Firm During the Investigation into a determination. 11.01.15 Investigate Case Mail/Fax Investigator To gather additional information as necessary. Potential Employer During the Investigation into a determination. 11.01.15 Investigate Case Mail/Fax Investigator To gather additional information as necessary. Labor Organization During the Investigation into a determination. 11.01.16 Inform Other Areas (as necessary) Email/Phone/ Interoffice Investigator Request Redetermination (Note: The results of the redetermination are returned to the BAM Investigator) Adjudication Hub If the file appears to contain a determination error. 11.01.16 Inform Other Areas (as necessary) Email/Phone/ Interoffice Investigator Notify Appeals of a Redetermination Appeals Unit If the file appears to contain a determination error. 11.01.16 Inform Other Areas (as necessary) Email/Phone/ Interoffice Investigator Notify BPC of possible fraud for further investigation. (Note: The results of the investigation are returned to the BAM Investigator) BPC Unit If the file appears to evidence of fraudulent activity. 11.01.23 Close Case Interoffice Supervisor Notify the Administrator of a possible improper payment made in violation of law or policy. Administrator When a possible improper payment is detected.
Page 18
Index Activity Label Message Type Sender Message Purpose Message Recipient Occasion 11.01.24 Review Case Interoffice Administrator Notify the Adjudication Hub Supervisor and/or AWI Management of the improper payment for appropriate corrective action. Adjudication Hub Supervisor/ AWI Management When an improper payment is detected in violation of law or policy. Exhibit 8 Audit - BAM Communication 1.4.5 Opportunities The purpose of this section is to capture improvement suggestions that may have a positive effect on technical, functional, or physical components of the Audit - BAM activities. Over the course of our interviews and observations, we identified a number of possible process improvement opportunities (long and short term). The following tables contain descriptions (Problem Statement) of what may be negatively affecting current operations and corresponding potential solutions (Recommendation) to address the issues. In some cases, additional analysis is required to understand fully the impact of the problems and/or recommendations. The following is a description and definition of each of the columns in this table. Origin The area of the improvement such as: Central Office, Hub, etc. Problem Statement Description of the problem or opportunity identified. Recommendation Description of the proposed recommendation, including expected result. Value Added Description of the expected value received if the recommendation to the problem statement is identified.
Short Term Opportunities for Improvement Origin Problem Statement Recommendation Value Added 11.01.11 11.01.12 There are a large number of reports that are gathered as part of the BAM process. For the most part, each report must be requested individually. Provide a single transaction for investigators to automatically generate the UC reports required to support audit activities. Significant time savings. 11.01.14 There appears to be a problem getting employers to respond to inquiries in a timely fashion which results in delays for the investigation process.
Identifying better/easier methods (e.g., online) for employer feedback may help to simplify the process for employers.
Needs further investigation. Quicker employer responses will decrease the time necessary to complete audit activities.
Page 19
11.01.25 Paid case investigation results are not imaged like the Denied cases.
Paid cases are a lot larger (many more pages of documents because Paid cases involve claimants and employers; denied cases usually only involve claimants) and there arent time or resources available to prepare the case folders for imaging. We believe further investigation is required to develop workable solutions leading to the capture of Paid cases in the imaging system. Capturing images of the Paid cases would lead to relieve the need to maintain paper archives and could provide a faster, easier method for retrieval. Exhibit 9 Short Term Opportunities for Improvement
Long Term Opportunities for Improvement Origin Problem Statement Recommendation Value Added
Exhibit 10 Long Term Opportunities for Improvement
Page 20
2 PURPOSE AND OBJECTIVES OF THE AUDIT - BTQ QUARTERLY REPORTING SUB-PROCESS The Audit core process is comprised of four distinct sub-processes: Benefit Accuracy Measurement (BAM), Benefit Timeliness and Quality (BTQ) Quarterly Reporting, BTQ Non-Monetary Reviews, and BTQ Data Validation. The following section describes the BTQ Quarterly Reporting related activities. Benefit Timeliness Quality (BTQ) Quarterly Reporting The determination of a claimants eligibility for unemployment insurance (UI) benefits is a critical UI program function. When issues arise that may affect a claimants past, present, or future benefits, the adjudicator is responsible for determining the claimants eligibility for those benefits. Such determinations may also affect an employers liability for benefit charges, depending on the type of issues adjudicated. The adjudicators work impacts the rights of both claimants and employers. The BTQ unit is responsible to report quarterly to the U.S. Department of Labor (DOL) for nonmonetary claims. Note: Many units within AWI perform audit functions to monitor their performance. The processes described here address specifically BAM and BTQ and are not meant to encompass other audit functions within AWI. The following sections describe the activities associated with the Audit - BTQ functions in UC. 2.1 BEGINNING AND ENDING POINTS BTQ Quarterly Reporting
Beginning Point: The UC Benefits system generates a random list of claims for review on the first workday of each quarter.
Ending Point: The BTQ investigators case file is copied to a local, shared network drive.
Page 21
2.2 AUDIT - BENEFIT TIMELINESS QUALITY (BTQ) QUARTERLY REPORTING DIAGRAM Supervisor Reviewer AWI Print Shop 11.02.05 Insert Data Collection Instrument If First Review 11.02.06 Review & Score File 11.02.01 Select SSNs for Review 11.02.02 Assign Reviewer 11.02.03 Setup Folder & Label Word Interoffice 11.02.04 Retrieve Pre-Review Documents UC, Imaging, FLUID, AENR If Second Review UC Adjudication Hub If Determination Error Request Redetermination Process Start Continued on Following Page
Page 22
Supervisor Reviewer Adjudication Hub 11.02.11 Complete Internal Report 11.02.09 Complete ETA 9056 Reviewers Agree? State Menu System No 11.02.07 Insert Data Collection Instrument 11.02.08 Review & Score File Yes If Failing Score Interoffice to Hub Supervisor 11.02.10 Archive File Excel Process Ends Continued from Previous Page
Exhibit 11 Audit - BTQ - Quarterly Reporting Process Flow
Page 23
2.3 AUDIT BTQ QUARTERLY REPORTING ACTIVITIES Exhibit 12 Audit BTQ Quarterly Reporting Activities lists the activities that make up the Audit BTQ Quarterly Reporting sub-process. The following is a description and definition of each of the columns in this table. Index A number for use in referencing the activity Activity Label A short name for that activity that is used to represent it in the detailed workflow diagram. Activity Description The full-text description of that activity Actors The job or role title of the individuals having a primary role for that activity
Index Activity Label Activity Description Actors 11.02.01 Select SSNs for Review The UC Benefits system has coded within it a U.S. Department of Labor (DOL) algorithm which automatically selects random claims for review (50 Separations, 50 Non-Separations). For quarterly reporting, the process is executed on the first workday of each quarter (January, April, July, and October) and creates printouts for each claim.
UCB45 Wage Determination(s) BH01 History
The output is routed to the BTQ Supervisor via interoffice mail and the Quarterly Reports are due by the 20 th day of the month following the quarter (e.g., Quarterly Report review reviews for Jan-Mar are run on April 1 and the results are due back to DOL by May 20). AWI Print Shop 11.02.02 Assign Reviewer The BTQ supervisor assigns a reviewer for each case. The cases are separated by type (50 Separations and 50 Non-Separations). The supervisor gives 50 of one type to one reviewer and 50 of the other type to a different reviewer. Later, in this process, the reviewers exchange cases and re-score each others work. Supervisor 11.02.03 Setup Folder & Label The reviewer sets up a separate folder for each assigned case and labels the folder. Reviewer 11.02.04 Retrieve Pre- Review Documents The reviewer queries multiple systems to retrieve the information necessary to complete the assessment. Reviewer 11.02.05 Insert Data Collection Instrument A blank data collection instrument (scoring sheet) is inserted into the case folder by each reviewer. Reviewer 11.02.06 Review & Score File The results of each review are scored in accordance with the ET Handbook 301 (the scoring is indicated on the data collection instrument for each scoring category).
Once all 50 assigned cases of one type are reviewed by one examiner, the reviewers exchange the folders and score the cases again. If there are any disagreements in the scoring, the cases are submitted to a supervisor to be scored a third time.
If it appears a determination error was committed and is in violation of law and policy, the case is referred to the Adjudication Hub for redetermination. Reviewer
Page 24
Index Activity Label Activity Description Actors 11.02.07 Insert Data Collection Instrument If the initial reviewers do not agree about the score for a given case, the case is scored by the BTQ supervisor using the same process.
A blank data collection instrument (scoring sheet) is inserted into the case folder. Supervisor 11.02.08 Review & Score File The case is reviewed and scored in accordance with the ET Handbook 301 as indicated on the data collection instrument. Supervisor 11.02.09 Complete ETA 9056 Once all cases are scored, the results are input to the State Menu system by completing the ETA 9056.
If a case receives a failing score, a memo and a copy of the case is sent to the Adjudication Hub supervisor to whom the initial case adjudicator reports. Reviewer 11.02.10 Archive File The review files are stored in local archives for two years. Reviewer
11.02.11 Complete Internal Report An internal report (created in Excel), containing the results of the reviews, is stored on the network Program Evaluation drive. Supervisor Exhibit 12 Audit BTQ Quarterly Reporting Activities
2.4 AUDIT BTQ QUARTERLY REPORTING PROCESS ATTRIBUTES The following sections define and describe sub-process specifics for elapsed time, paper documentation, business system use, communications, and process improvements. The information is arranged in this way to support analysis of these activities, both individually and collectively. 2.4.1 Elapsed Time Exhibit 13 Audit BTQ Quarterly Reporting Elapsed Time lists the time required to complete each of the activities that make up the Audit BTQ Quarterly Reporting sub-process. The times are estimated except where noted. The following is a description and definition of each of the columns in this table. Index A number for use in referencing the activity. Activity Label A short name for that activity that is used to represent it in the detailed workflow diagram. Process Time The hands-on time it takes to complete the activity (does not include mail, transport, or time spent waiting for another activity to complete). Cycle Time The total elapsed time from the beginning of the activity until it is complete, inclusive of all delays from transport, mail, other processes, and work load. Time Goals / Limits Any time limits or goals, from any source internal or external pertaining to this activity. Limit/ Goal Source The source of the limit or goal listed above (Florida Statutes, AWI policy, etc.) Speed Bumps Specific, recurring difficulties that slow the progress of this activity.
Page 25
Index Activity Label Process Time Cycle Time Limit or Goal Time Limit or Goal Source Speed Bumps 11.02 Quarterly Reports 1.5-2 hrs per file. 50 days Process Begins on the first working day of Quarter. Results due on the 20 th of the month following. ETA 301 Handbook published by Department of Labor. Gathering information from multiple systems. Information occasionally not present in system requiring manual intervention. Exhibit 13 Audit BTQ Quarterly Reporting Elapsed Time 2.4.2 Paper Documentation Exhibit 14 Audit BTQ Paper Documentation lists the paper documents required to complete each of the activities that make up the Audit BTQ Quarterly Reporting sub-process. The reference column refers to the originator and title of policy, procedure, or instruction that contains the form or specifies the content of the document. More than one document may be required to carry out a single activity, so each is listed. The following is a description and definition of each of the columns in this table. Index A number for use in referencing the activity. Activity Label A short name for that activity that is used to represent it in the detailed workflow diagram. Document Title Short title, form number or other designation for the document. Document Description General description of the document contents and purpose. Reference or Source Originator and title of the source instruction or policy.
Index Activity Label Document Title Document Description Reference or Source 11.02.01 Select SSNs for Review UBMSEPBSA20 Report produced from the UC system identifying the 50 Separation Issues scheduled for review. Includes official reproductions of the UCB-45 for each issue. AWI Reviewer 11.02.01 Select SSNs for Review UBMSEP2390 Report produced from the UC system containing the BH01 (Claims History) for each issue identified in the report above. AWI Reviewer 11.02.01 Select SSNs for Review UBMNSEPBSA20 Report produced from the UC system identifying the 50 Non-Separation Issues scheduled for review. Includes official reproductions of the UCB-45 for each issue. AWI Reviewer 11.02.01 Select SSNs for Review UBMNSEP2390 Report produced from the UC system containing the BH01 (Claims History) for each issue identified in the report above. AWI Reviewer 11.02.03 Setup Folder & Label Label Word Template Label used to identify review file. AWI Reviewer 11.02.04 Retrieve Pre- Review Documents BQ01 Determination History. Printed from the UC System. AWI Reviewer 11.02.04 Retrieve Pre- Review Documents Automated Non- Monetary Worksheet Adjudication Worksheet retrieved from the EIS (Imaging) System. Accompanied by any other documents imaged during the Adjudication Process. AWI Reviewer
Page 26
Index Activity Label Document Title Document Description Reference or Source 11.02.05 Insert Data Collection Instrument Blank Word Template Document used to record information about file and to record Reviewers comments and score. Completed by hand. AWI Reviewer 11.02.07 Insert Data Collection Instrument Blank Word Template Document used to record information about file and to record Reviewers comments and score. Completed by hand. AWI Reviewer 11.02.09 Complete ETA 9056 ETA 9056 Nonmonetary Determinations Quality Report (OMB 1205-0359) Printout from the DOL State Menu System showing the results of the review process. Attached to the case file. AWI Reviewer 11.02.11 Complete Internal Report Summary Report Summary report of all 100 issues reviewed as part of the cycle. Entered into Excel and stored on a central share drive. AWI Reviewer Exhibit 14 Audit BTQ Paper Documentation 2.4.3 Business System Data Entry & Data Transfer Exhibit 15 Audit BTQ Quarterly Reporting Data Entry & Data Transfer lists the business systems that are employed directly (touched) or indirectly (data transfer) in the course of the Audit BTQ Quarterly Reporting sub-process. The following is a description and definition of each of the columns in this table. Index A number for use in referencing the activity. Activity Label A short name for that activity that is used to represent it in the detailed workflow diagram. Business System The name or designation of the system touched (if any). Purpose or Intent The business objective accomplished by using the system or systems. Mode Choose one; entry is made by a person, transfers are from system to system.
Index Activity Label Business System Program or Intent Mode 11.02.01 Select SSNs for Review UC Selects 100 issues (50 Separation, 50 Non- Separation Claims) for review. Also triggers the printout of the UBMSEPBSA20, UBMSEP1430, UBMNSEPBSA20, and UBMNSEP1430 reports. Inquiry 11.02.04 Retrieve Pre- Review Documents UC Retrieve By SSN (BQ01) Determination History Inquiry 11.02.04 Retrieve Pre- Review Documents EIS (Imaging) Input SSN. Select and print the Adjudication Fact Finding File. If the Fact Finding File does not contain the UCB-412, the UCB-412 should be printed as well. Inquiry 11.02.04 Retrieve Pre- Review Documents FLUID Input SSN on the Search Screen. Results are displayed and the appropriate Claim is selected and printed. Used only if the Fact Finding File does not already contain a copy. Inquiry
Page 27
Index Activity Label Business System Program or Intent Mode 11.02.04 Retrieve Pre- Review Documents AENR Input SSN on the Search Screen. Results are displayed and the appropriate Employer Verification is selected and printed. Used only if the Fact Finding File does not already contain a copy. Inquiry Exhibit 15 Audit BTQ Quarterly Reporting Data Entry & Data Transfer 2.4.4 Communication Exhibit 16 Audit BTQ Quarterly Reporting Communication lists the messages, such as the US mail, emails, phone calls and any other correspondence, regularly used in the normal course of the Audit BTQ Quarterly Reporting sub-process. Messages are sent to convey required documentation, inform recipients about the status of an activity, to trigger the start or end of other activities, to grant or withdraw authorization, etc. This section only concerns messages sent during normal (as intended) operations. There may be more than one message associated with any activity in this process. The following is a description and definition of each of the columns in this table. Index A number for use in referencing the activity. Activity Label A short name for that activity that is used to represent it in the detailed workflow diagram. Message Type US mail, email, phone call, etc. Message Sender The organization or role of the message sender. Message Purpose The intended purpose for sending the message. Message Recipient The organization or role of the intended recipient. Occasion Event or action that triggers the requirement to send the message
Index Activity Label Message Type Sender Message Purpose Message Recipient Occasion 11.02.01 Select SSNs for Review Interoffice AWI Print Shop Provide copies of UBMSEPBSA20, UBMSEP1430, UBMNSEPBSA2 0, and UBMNSEP1430 reports. Supervisor First working day of each Quarter. Starts the review process. 11.02.06 Review & Score File Interoffice Reviewer Request redetermination of a particular claim based on facts uncovered during the review. Adjudication Hub When errors are found in the initial claim adjudication. 11.02.09 Complete ETA 9056 Interoffice Reviewer Inform Adjudicator Supervisor of a failing score (Training opportunity) Adjudication Hub If a case file has a failing score. Exhibit 16 Audit BTQ Quarterly Reporting Communication 2.4.5 Opportunities The purpose of this section is to capture improvement suggestions that may have a positive effect on
Page 28
technical, functional, or physical components of the Audit BTQ Quarterly reporting activities. Over the course of our interviews and observations, we identified a number of possible process improvement opportunities (long and short term). The following tables contain descriptions (Problem Statement) of what may be negatively affecting current operations and corresponding potential solutions (Recommendation) to address the issues. In some cases, additional analysis is required to understand fully the impact of the problems and/or recommendations. The following is a description and definition of each of the columns in this table.
Origin The area of the improvement such as: Central Office, Hub, etc. Problem Statement Description of the problem or opportunity identified. Recommendation Description of the proposed recommendation, including expected result. Value Added Description of the expected value received if the recommendation to the problem statement is identified.
Short Term Opportunities for Improvement Origin Problem Statement Recommendation Value Added 11.02.01 Quarterly audits are not being run at this time (due to workload constraints) but the mainframe reports are still being created (1,000+ pages). Ideally, an option should be provided to turn off the reports unless they are needed once reporting commences in the future. Paper savings as well as a reduction in overall workload (printing, dissemination, shredding). Exhibit 17 Short Term Opportunities for Improvement
Long Term Opportunities for Improvement Origin Problem Statement Recommendation Value Added The UC Benefits system has coded within it a U.S. Department of Labor (DOL) algorithm which automatically selects random claims for review (50 Separations, 50 Non-Separations). The algorithm in the future system should be administrator controlled to obviate the need to modify code within the system. Ideally, the interface between the DOLs State Menu system and the new UC system will be automated and preclude the need for UC staff to perform the manual download/upload processes. No need for IT resources to implement code changes in response to USDOL audit requirements. Exhibit 18 Long Term Opportunities for Improvement
Page 29
3 PURPOSE AND OBJECTIVES OF THE AUDIT BTQ NON- MONETARY REVIEWS SUB-PROCESS The Audit core process is comprised of four distinct sub-processes: Benefit Accuracy Measurement (BAM), Benefit Timeliness and Quality (BTQ) Quarterly Reporting, BTQ Non-Monetary Reviews, and BTQ Data Validation. The following sections describe the BTQ Non-Monetary related activities. Benefit Timeliness Quality (BTQ) Non-Monetary Reviews UC has instituted an internal quality review process under the auspices of the BTQ unit to perform periodic assessments of adjudicators in each Adjudication Hub. The information gathered through the reviews provides insight to existing or potential issues that may be causing difficulties in particular adjudication areas (e.g., separation issues). Managers can use that information to take corrective action which may include arranging for remedial training, modifying policies and procedures, and/or suggesting changes to the automated system(s) to address less than satisfactory performance. Note: Many units within AWI perform audit functions to monitor their performance. The processes described here address specifically BAM and BTQ and are not meant to encompass other audit functions within AWI. The following sections describe the activities associated with the Audit - BTQ Non-Monetary Reviews functions in UC. 3.1 BEGINNING AND ENDING POINTS BTQ
Beginning Point: An Adjudication Hub is selected for review.
Ending Point: The results of the claims reviews are recorded in an internal reporting document.
Page 30
3.2 AUDIT BTQ NON-MONETARY REVIEWS DIAGRAM Supervisor Reviewer Adjudicator Supervisor 11.03.05 Insert Data Collection Instrument 11.03.06 Review & Score File 11.03.01 Run Process for Selected Time Period 11.03.02 Assign Reviewer 11.03.04 Retrieve Pre-Review Documents UC, Imaging, FLUID, AENR UC Process Start 11.03.07 Complete Summary Scoring Report for Adjudicator Excel 11.03.03 Select 10 SSNs for Review For Each SSN For Each Adjudicator 11.03.08 Complete Summary Scoring Report for Hub Process Ends Word/Excel Interoffice Hub Score Report
Exhibit 19 3.2 Audit BTQ Non-Monetary Reviews Process Flow
Page 31
3.3 AUDIT BTQ NON-MONETARY REVIEWS ACTIVITIES Exhibit 20 Audit BTQ Non-Monetary Reviews Activities lists the activities that make up the Audit BTQ Non-Monetary Reviews sub-process. The following is a description and definition of each of the columns in this table. Index A number for use in referencing the activity Activity Label A short name for that activity that is used to represent it in the detailed workflow diagram. Activity Description The full-text description of that activity Actors The job or role title of the individuals having a primary role for that activity
Index Activity Label Activity Description Actors 11.03.01 Run Process for Selected Time Period The BTQ unit conducts periodic audits of the Adjudication Hubs. The supervisor begins the process by requesting a SAR Report of all closed cases for a specified time period (usually a week) for a particular adjudicator. This process is repeated for each adjudicator to be audited.
Note: This process is very similar to the Quarterly Review outlined above and is used to support internal quality reviews. Supervisor 11.03.02 Assign Reviewer The BTQ supervisor assigns reviewers to complete the audits. Supervisor 11.03.03 Select 10 SSNs for Review The reviewer selects 10 social security numbers (determinations for particular claimants) from the SAR Report and goes through an iterative process to review all 10 cases.
Note: The reviewer attempts to select more Separation cases than Non-Separation cases (Separation cases, in general, have a higher error rate). Reviewer 11.03.04 Retrieve Pre- Review Documents The reviewer queries numerous systems to gather the information necessary to complete the examination. Reviewer 11.03.05 Insert Data Collection Instrument The information is written into the data collection instrument. Reviewer 11.03.06 Review & Score File The collection instrument is reviewed and scored. At the end of the process, a copy of the scoring sheet, for each adjudicator, will be provided to the Adjudication Hub supervisor. Reviewer 11.03.07 Complete Summary Scoring Report for Adjudicator The Reviewer creates a summary report (Excel file) for each adjudicator and stores the results on the Program Eval Drive (a designated network drive). At the end of the process, the reports will also be provided to the Adjudicators supervisor. Reviewer 11.03.08 Complete Summary Scoring Report for Hub The supervisor prepares a summary report for the entire Adjudication Hub (summary scores for each adjudicator and overall statistics for the Hub) and a cover letter (memorandum) and mails the package to the Adjudication Hub supervisor along with the individual scoring sheets and summary scoring reports for each adjudicator.
The supervisor also updates the state-wide adjudication hub review results.
All reports are stored on the Program Eval drive. Supervisor Exhibit 20 Audit BTQ Non-Monetary Reviews Activities
Page 32
3.4 AUDIT BTQ NON-MONETARY REVIEWS PROCESS ATTRIBUTES The following sections define and describe sub-process specifics for elapsed time, paper documentation, business system use, communications, and process improvements. The information is arranged in this way to support analysis of these activities, both individually and collectively. 3.4.1 Elapsed Time Exhibit 21 Audit BTQ Non-Monetary Reviews Activity Elapsed Time lists the time required to complete each of the activities that make up the Audit BTQ Non-Monetary Reviews sub-process. The times are estimated except where noted. The following is a description and definition of each of the columns in this table. Index A number for use in referencing the activity. Activity Label A short name for that activity that is used to represent it in the detailed workflow diagram. Process Time The hands-on time it takes to complete the activity (does not include mail, transport, or time spent waiting for another activity to complete). Cycle Time The total elapsed time from the beginning of the activity until it is complete, inclusive of all delays from transport, mail, other processes, and work load. Time Goals / Limits Any time limits or goals, from any source internal or external pertaining to this activity. Limit/ Goal Source The source of the limit or goal listed above (Florida Statutes, AWI policy, etc.) Speed Bumps Specific, recurring difficulties that slow the progress of this activity.
Index Activity Label Process Time Cycle Time Limit or Goal Time Limit or Goal Source Speed Bumps 11.03 Non-Monetary Reviews 1 1.5 hrs per file. Varies based on size of Hub. Policy is for one to two Hubs to be evaluated each Quarter. Internal Policy Gathering information from multiple systems. Information occasionally not present in system requiring manual intervention. Exhibit 21 Audit BTQ Non-Monetary Reviews Activity Elapsed Time 3.4.2 Paper Documentation Exhibit 22 Audit BTQ Non-Monetary Reviews Paper Documentation lists the paper documents required to complete each of the activities that make up Audit BTQ Non-Monetary Reviews sub- process. The reference column refers to the originator and title of policy, procedure, or instruction that contains the form or specifies the content of the document. More than one document may be required to carry out a single activity, so each is listed. The following is a description and definition of each of the columns in this table. Index A number for use in referencing the activity. Activity Label A short name for that activity that is used to represent it in the detailed workflow diagram. Document Title Short title, form number or other designation for the document. Document Description General description of the document contents and purpose. Reference or Source Originator and title of the source instruction or policy.
Page 33
Index Activity Label Document Title Document Description Reference or Source 11.03.01 Run Process for Selected Time Period UBDNONMW1420 List of closed adjudications within a selected time period for a given adjudicator. BTQ Supervisor 11.03.04 Select SSNs for Review BH01 Claims History. Printed from the UC System. AWI Reviewer 11.03.04 Select SSNs for Review UCB45 Claim Determination. Printed from the UC System. AWI Reviewer 11.03.04 Retrieve Pre- Review Documents BQ01 Determination History. Printed from the UC System. AWI Reviewer 11.03.04 Retrieve Pre- Review Documents Automated Non- Monetary Worksheet Adjudication Worksheet retrieved from the EIS (Imaging) System. Accompanied by any other documents imaged during the Adjudication Process. AWI Reviewer 11.03.05 Insert Data Collection Instrument Blank Word Template Document used to record information about file and to record Reviewer's comments and score. Completed by hand. AWI Reviewer 11.03.07 Complete Summary Scoring Report for Adjudicator Excel Template Report of all cases reviewed for a particular adjudicator with score and comments. AWI Reviewer 11.03.08 Complete Summary Scoring Report for Hub (incl. Memo) & State Excel Chart Template Report of all cases reviewed for a Hub. BTQ Supervisor 11.03.08 Complete Summary Scoring Report for Hub (incl. Memo) & State Word Template Memo used as a cover sheet for the Adjudication Hub Scoring Report BTQ Supervisor 11.03.08 Complete Summary Scoring Report for Hub (incl. Memo) & State Excel Chart Template Report of all cases reviewed for the State BTQ Supervisor Exhibit 22 Audit BTQ Non-Monetary Reviews Paper Documentation 3.4.3 Business System Data Entry & Data Transfer Exhibit 23 Audit BTQ Non-Monetary Reviews Data Entry & Data Transfer lists the business systems that are employed directly (touched) or indirectly (data transfer) in the course of the Audit BTQ Non-Monetary Reviews sub-process. The following is a description and definition of each of the columns in this table. Index A number for use in referencing the activity. Activity Label A short name for that activity that is used to represent it in the detailed workflow diagram. Business System The name or designation of the system touched (if any). Purpose or Intent The business objective accomplished by using the system or systems. Mode Choose one; entry is made by a person, transfers are from system to system.
Page 34
Index Activity Label Business System Program or Intent Mode 11.03.01 Run Process for Selected Time Period UC Input an Adjudicator Name and a Time period: (UBDNONMW1420) List of closed adjudications within the input time period for the adjudicator. BTQ Supervisor 11.03.04 Retrieve Pre- Review Documents UC Retrieve by SSN: (BQ01) Determination History (UCB45) Wage Determinations (BH01) Claim History Inquiry 11.03.04 Retrieve Pre- Review Documents EIS (Imaging) Input SSN. Select and print the Adjudication Fact Finding File. If the Fact Finding File does not contain the UCB-412, the UCB-412 should be printed as well. Inquiry 11.03.04 Retrieve Pre- Review Documents FLUID Input SSN on the Search Screen. Results are displayed and the appropriate Claim is selected and printed. Used only if the Fact Finding File does not already contain a copy. Inquiry 11.03.04 Retrieve Pre- Review Documents AENR Input SSN on the Search Screen. Results are displayed and the appropriate Employer Verification is selected and printed. Used only if the Fact Finding File does not already contain a copy. Inquiry Exhibit 23 Audit BTQ Non-Monetary Reviews Data Entry & Data Transfer 3.4.4 Communication Exhibit 24 Audit BTQ Non-Monetary Reviews Communication lists the messages, such as the US mail, emails, phone calls and any other correspondence, regularly used in the normal course of the Audit BTQ Non-Monetary Reviews sub-process. Messages are sent to convey required documentation, inform recipients about the status of an activity, to trigger the start or end of other activities, to grant or withdraw authorization, etc. This section only concerns messages sent during normal (as intended) operations. There may be more than one message associated with any activity in this process. The following is a description and definition of each of the columns in this table. Index A number for use in referencing the activity. Activity Label A short name for that activity that is used to represent it in the detailed workflow diagram. Message Type US mail, email, phone call, etc. Message Sender The organization or role of the message sender. Message Purpose The intended purpose for sending the message. Message Recipient The organization or role of the intended recipient. Occasion Event or action that triggers the requirement to send the message
Index Activity Label Message Type Sender Message Purpose Message Recipient Occasion 11.03.08 Complete Summary Scoring Report for Hub (incl. Memo) & State Interoffice BTQ Supervisor Provide Adjudicator Supervisor with all scoring documentation. Adjudication Supervisor After any Hub review has been completed. Exhibit 24 Audit BTQ Non-Monetary Reviews Communication
Page 35
3.4.5 Opportunities The purpose of this section is to capture improvement suggestions that may have a positive effect on technical, functional, or physical components of the Audit BTQ Non-Monetary Review activities. Over the course of our interviews and observations, we identified a number of possible process improvement opportunities (long and short term). The following tables contain descriptions (Problem Statement) of what may be negatively affecting current operations and corresponding potential solutions (Recommendation) to address the issues. In some cases, additional analysis is required to understand fully the impact of the problems and/or recommendations. The following is a description and definition of each of the columns in this table.
Origin The area of the improvement such as: Central Office, Hub, etc. Problem Statement Description of the problem or opportunity identified. Recommendation Description of the proposed recommendation, including expected result. Value Added Description of the expected value received if the recommendation to the problem statement is identified.
Short Term Opportunities for Improvement Origin Problem Statement Recommendation Value Added General The Quarterly Reporting requirements (BTQ) have been suspended by USDOL in light of prevailing unemployment conditions across the nation (except reviews required to support the annual Tripartite Review). However, UC has also suspended the internal Quality reviews for adjudicators due to resources (staff has been reassigned) and cost (travel).
We are led to believe internal quality audits (adjudicators in each hub) have not been completed for two years. We recommend this quality management process be re- started. Corrective action on the front end of the adjudication process should lead to cost containment throughout the remaining processes in UC.
Note: the cost to fix a problem later in the UC process is not quantifiable based upon information at our current disposal. Exhibit 25 Short Term Opportunities for Improvement
Long Term Opportunities for Improvement Origin Problem Statement Recommendation Value Added
Exhibit 26 Long Term Opportunities for Improvement
Page 36
4 PURPOSE AND OBJECTIVES OF THE AUDIT BTQ DATA VALIDATION SUB-PROCESS The Audit core process is comprised of four distinct sub-processes: Benefit Accuracy Measurement (BAM), Benefit Timeliness and Quality (BTQ) Quarterly Reporting, BTQ Non-Monetary Reviews, and BTQ Data Validation. The following section describes the BTQ Data Validation related activities. Benefit Timeliness Quality (BTQ) Data Validation The comprehensive Federal data validation program is necessary to ensure the accuracy of Unemployment Insurance Required Reports (UIRR) data. UIRR data are used for economic statistics, to allocate UI administrative funding based on state workload, to measure state claimant eligibility criteria and performance in providing benefits, and to account for fund utilization. It is therefore essential that states report UIRR data accurately and uniformly. States report to DOL on a monthly and quarterly basis under the Unemployment Insurance Required Reports system.
States are required to validate reported data every third year, except for data elements used to calculate Government Performance and Results Act (GPRA) measures. GPRA data are validated annually. The validation year will coincide with the State Quality Service Plan (SQSP) performance year. Validation of any reporting period during the twelve months beginning April 1 and ending March 31 will be considered part of the validation year. The SQSP is the vehicle through which states submit plans to implement validation or to revalidate failed items. GPRA and SQSP reporting is covered in a separate process, Federal Reporting.
Note: Many units within AWI perform audit functions to monitor their performance. The processes described here address specifically BAM and BTQ and are not meant to encompass other audit functions within AWI. The following sections describe the activities associated with the Audit BTQ Data Validation functions in UC. 4.1 BEGINNING AND ENDING POINTS
BTQ
Beginning Point: A list of necessary validation is received from the U.S. Department of Labor (DOL) State Menu system.
Ending Point: The results of the validations are submitted via the U.S. Department of Labor (DOL) State Menu system.
Page 37
4.2 AUDIT BTQ DATA VALIDATION DIAGRAM Supervisor Validator 11.04.07 Enter Results 11.04.03 Select Population Data 11.04.04 Create Validation Sheet 11.04.05 Assign Validator 11.04.06 Validate Per 361 Handbook UC, Imaging, SunTax, FLUID, BOSS, Appeals Population List from DOL State Menu System State Menu System/ Excel Excel 11.04.08 Enter Results State Menu System Fail 11.04.09 Submit Pass For Each Population Process Ends State Quality Service Plan (SQSP) Coordinator If Systemic Error Fail 11.04.10 Prepare & Submit Report Mail US Dept. of Labor 11.04.01 Run Report Validation State Menu System Pass Fail IT Staff 11.04.02 Correct System Error Exhibit 27 Audit BTQ Data Validation Process Flow
Page 38
4.3 AUDIT BTQ DATA VALIDATION ACTIVITIES Exhibit 28 Audit BTQ Data Validation Activities lists the activities that make up the Audit BTQ Data Validation sub-process. The following is a description and definition of each of the columns in this table. Index A number for use in referencing the activity Activity Label A short name for that activity that is used to represent it in the detailed workflow diagram. Activity Description The full-text description of that activity Actors The job or role title of the individuals having a primary role for that activity
Index Activity Label Activity Description Actors 11.04.01 Run Report Validation The U.S. Department of Labor (DOL) requires validation of UC data on a periodic basis. The DOL application, State Menu system, automatically selects and schedules reviews for given UC populations
Note: Populations are Benefit transaction types (e.g., Weeks Claimed, Payments, Appeals Filed) the DOL assigns for data collection and reporting (the BTQ supervisor generates a schedule of the populations to be validated from the State Menu system).
The BTQ supervisor (or designated Benefit Service Specialist III) runs a validation process in the State Menu System. Failures in validation are reported to the IT Staff for correction of the upload files.
In general, data validations for each population are scheduled by DOL every three years. DOL will schedule specific populations for data validation on a more frequent basis if they fail. Some populations (e.g., Payments, Overpayments) are validated every year. Supervisor 11.04.02 Correct System Error Any failures are reported to the AWI IT staff for correction (if a population fails, it means the mainframe UC Benefits system is not properly transmitting data to DOL). The IT staff corrects the errors and the BTQ supervisor re-runs the validation report. The correction process is repeated until all populations pass the validation criteria. IT Staff 11.04.03 Select Population Data The data populations due for validation are selected from the State Menu. Supervisor 11.04.04 Create Validation Sheet The State Menu system displays the validations for a given population in a format similar to Microsoft Excel. The supervisor cuts and pastes the State Menu screen data into an Excel spreadsheet to facilitate the data validation process. Supervisor 11.04.05 Assign Validator A validator is assigned to review the information for a given population. Supervisor 11.04.06 Validate Per 361 Handbook The ETA 361 handbook outlines the process to be used to validate data for the numerous population types. Validators use the handbook as a guide to complete their review process. Validators may access various systems (e.g., UC, Imaging, SunTax, BOSS, FLUID, Appeals) to gather the information required to support the data validation process. Validator
Page 39
Index Activity Label Activity Description Actors 11.04.07 Enter Results The results of the validations (PASS/FAIL) are entered on the Excel worksheet and the worksheets are returned to the supervisor for input to the State Menu system. Validator 11.04.08 Enter Results The BTQ supervisor manually transfers the Excel spreadsheet validation results into the DOL State Menu system.
If a given population fails, the supervisor attempts to identify the reason for failure, either systemic or random.
Systemic errors tend to be constant and fall into one of three categories:
1) too many transactions (over counts) 2) too few transactions (undercounts) 3) transactions which are misclassified.
Systemic human errors occur when staff are using incorrect definitions or procedures. For example, a reporting unit may establish its own definition for a data element that conflicts with the federal definition deliberately or inadvertently. Systemic errors are the most serious because they occur repeatedly. They are also the easiest to detect and correct. Systemic errors do not need to be assessed very frequently, and each system error only needs to be corrected once. A one-time adjustment in a retrieval code or calculation specification or staff retraining on a corrected definition or procedure will usually correct systemic errors.
Random errors tend to be variable, and are always caused by human judgment. They fall into one of three categories:
1) input errors 2) judgment errors (as in nonmonetary determinations, status determinations, and appeals) 3) improper state definitions or procedures.
If it appears the failure is random, the supervisor selects a new population and revalidates. If the failure appears to be systemic, the SQSP Coordinator is notified and the coordinator prepares and mails an explanation to DOL, corrective action is taken, and the population will be validated on a subsequent cycle. Supervisor 11.04.09 Submit The results are submitted to DOL (both Pass and Fail). Supervisor 11.04.10 Prepare & Submit Report If the BTQ supervisor discovers a systemic problem, the SQSP Coordinator is notified and the coordinator prepares and mails an explanation to DOL. State Quality Service Plan (SQSP) Coordinator Exhibit 28 Audit BTQ Data Validation Activities
4.4 AUDIT BTQ DATA VALIDATION PROCESS ATTRIBUTES The following sections define and describe sub-process specifics for elapsed time, paper documentation, business system use, communications, and process improvements. The information is arranged in this way to support analysis of these activities, both individually and collectively. 4.4.1 Elapsed Time Exhibit 29 Audit BTQ Data Validation Activity Elapsed Time lists the time required to complete
Page 40
each of the activities that make up the Audit BTQ Data Validation sub-process. The times are estimated except where noted. The following is a description and definition of each of the columns in this table. Index A number for use in referencing the activity. Activity Label A short name for that activity that is used to represent it in the detailed workflow diagram. Process Time The hands-on time it takes to complete the activity (does not include mail, transport, or time spent waiting for another activity to complete). Cycle Time The total elapsed time from the beginning of the activity until it is complete, inclusive of all delays from transport, mail, other processes, and work load. Time Goals / Limits Any time limits or goals, from any source internal or external pertaining to this activity. Limit/ Goal Source The source of the limit or goal listed above (Florida Statutes, AWI policy, etc.) Speed Bumps Specific, recurring difficulties that slow the progress of this activity.
Index Activity Label Process Time Cycle Time Limit or Goal Time Limit or Goal Source Speed Bumps 11.04 Data Validation 80 120 hours per population. 11 months List of validations is made available in June of each year. All validations must be completed by May 20 of the following year. ETA 361 Handbook published by Department of Labor.
Exhibit 29 Audit BTQ Data Validation Activity Elapsed Time 4.4.2 Paper Documentation Exhibit 30 Audit BTQ Data Validation Paper Documentation lists the paper documents required to complete each of the activities that make up Audit BTQ Data Validation sub-process. The reference column refers to the originator and title of policy, procedure, or instruction that contains the form or specifies the content of the document. More than one document may be required to carry out a single activity, so each is listed. The following is a description and definition of each of the columns in this table. Index A number for use in referencing the activity. Activity Label A short name for that activity that is used to represent it in the detailed workflow diagram. Document Title Short title, form number or other designation for the document. Document Description General description of the document contents and purpose. Reference or Source Originator and title of the source instruction or policy.
Index Activity Label Document Title Document Description Reference or Source 11.04.04 Create Validation Sheet Excel File Copy of the State Menu System population. Used for hand comments during the validation process BTQ Supervisor Exhibit 30 Audit BTQ Data Validation Paper Documentation 4.4.3 Business System Data Entry & Data Transfer Exhibit 31 Audit BTQ Data Validation Data Entry & Data Transfer lists the business systems that are employed directly (touched) or indirectly (data transfer) in the course of the Audit BTQ Data Validation sub-process. The following is a description and definition of each of the columns in this table.
Page 41
Index A number for use in referencing the activity. Activity Label A short name for that activity that is used to represent it in the detailed workflow diagram. Business System The name or designation of the system touched (if any). Purpose or Intent The business objective accomplished by using the system or systems. Mode Choose one; entry is made by a person, transfers are from system to system.
Index Activity Label Business System Program or Intent Mode 11.04.06 Validate Per 361 Handbook UC, EIS, SunTax, FLUID, BOSS, Appeals, etc. The populations to be validated vary greatly and may require information from almost any system in use. Inquiry Exhibit 31 Audit BTQ Data Validation Data Entry & Data Transfer 4.4.4 Communication Exhibit 32 Audit BTQ Data Validation Communication lists the messages, such as the US mail, emails, phone calls and any other correspondence, regularly used in the normal course of the Audit BTQ Data Validation sub-process. Messages are sent to convey required documentation, inform recipients about the status of an activity, to trigger the start or end of other activities, to grant or withdraw authorization, etc. This section only concerns messages sent during normal (as intended) operations. There may be more than one message associated with any activity in this process. The following is a description and definition of each of the columns in this table. Index A number for use in referencing the activity. Activity Label A short name for that activity that is used to represent it in the detailed workflow diagram. Message Type US mail, email, phone call, etc. Message Sender The organization or role of the message sender. Message Purpose The intended purpose for sending the message. Message Recipient The organization or role of the intended recipient. Occasion Event or action that triggers the requirement to send the message
Index Activity Label Message Type Sender Message Purpose Message Recipient Occasion 11.04.10 Prepare & Submit Report Mail State Quality Service Plan (SQSP) Coordinator Provide an explanation to the Department of Labor concerning the systemic failure of one or more populations during the Data Validation process. US Department of Labor After a population has been deemed to contain a systemic error which will prevent its passing the Data Validation process. Exhibit 32 Audit BTQ Data Validation Communication 4.4.5 Opportunities The purpose of this section is to capture improvement suggestions that may have a positive effect on technical, functional, or physical components of the Audit BTQ Data Validation activities. Over the course of our interviews and observations, we identified a number of possible process improvement opportunities (long and short term). The following tables contain descriptions (Problem Statement) of what may be negatively affecting current operations and corresponding potential solutions (Recommendation) to address the issues. In some cases, additional analysis is required to understand
Page 42
fully the impact of the problems and/or recommendations. The following is a description and definition of each of the columns in this table.
Origin The area of the improvement such as: Central Office, Hub, etc. Problem Statement Description of the problem or opportunity identified. Recommendation Description of the proposed recommendation, including expected result. Value Added Description of the expected value received if the recommendation to the problem statement is identified.
Short Term Opportunities for Improvement Origin Problem Statement Recommendation Value Added
Exhibit 33 Short Term Opportunities for Improvement
Long Term Opportunities for Improvement Origin Problem Statement Recommendation Value Added
Exhibit 34 Long Term Opportunities for Improvement