Audit of AANDC Support to the Independent Assessment Process

Date: November 2014
Project No. 02-14

PDF Version (160 Kb, 28 Pages)

 

Table of Contents

Acronyms

AANDC

Aboriginal Affairs and Northern Development Canada

DOJ

Department of Justice

HRWSB

Human Resources and Workplace Services Branch

IAP

Independent Assessment Process

IRS

Indian Residential Schools

IRSAS

Indian Residential Schools Adjudication Secretariat

IRSSA

Indian Residential Schools Settlement Agreement

LIA

Litigation Information Analyst

NRA

National Research and Analysis

PWGSC

Public Works and Government Services Canada

RIA

Resolution and Individual Affairs Sector

RM

Resolution Manager

SAO

Settlement Agreement Operations Branch

WMB Workforce Management Board
 

Executive Summary

Background

An Audit of AANDC Support to the Independent Assessment Process was included in Aboriginal Affairs and Northern Development Canada's 2014-2015 to 2016-2017 Risk-Based Audit Plan, approved by the Deputy Minister on February 6, 2014. The audit was identified as a high priority because of the financial obligation it represents to the Department and the risks involved if delivery of a high volume of claims is not met resulting in non-compliance with the Indian Residential Schools Settlement Agreement.

The Independent Assessment Process (IAP) is a claimant-centred, non-adversarial, out of court process for the resolution of claims of sexual abuse, serious physical abuse, and other wrongful acts suffered at Indian Residential Schools (IRS). The Indian Residential Schools Settlement Agreement (IRSSA) is the largest class action settlement in Canadian history and is expected to bring a fair and lasting resolution of the legacy of Indian Residential Schools.

With respect to the IAP, the Settlement Agreement Operations Branch (SAO), within AANDC's Resolution and Individual Affairs Sector (RIA), is responsible for representing Canada at hearings, negotiating settlements without a hearing, researching and paying the settlements. The Indian Residential Schools Adjudication Secretariat (IRSAS or Secretariat) is an independent, arms-length entity which falls under AANDC's departmental umbrella. The Secretariat's mandate is to implement and administer the Independent Assessment Process under the direction of the Chief Adjudicator in an independent, objective and impartial manner.

Audit Objective and Scope

The objective of the audit was to assess the effectiveness and efficiency of the management control framework in place at IRSAS and AANDC to provide support to the IAP. More specifically, the audit assessed whether:

  • Effective management controls are in place to help ensure the efficient and effective delivery of support services to the IAP provided by IRSAS and AANDC; and
  • Controls are in place to help ensure efficient and effective governance and management of the wind down of IRSAS and AANDC while providing ongoing support services to the IAP.

The scope of the audit included responsibilities of AANDCFootnote 1  and the IRSAS to support the IAP. The audit scope covered the period April 1, 2013 to May 31, 2014 and included an assessment of management practices and controls in place to support the IAP. The scope of the audit was developed to respect the IAP and the independence of IRSAS from AANDC/Canada - a defendant in the IAP. As such, the audit did not include; the aspects of the IAP that are administered under the direction of the Chief Adjudicator; any assessment of decisions made by the independent adjudicators; and IRSAS management controls related to the processing of IAP claims.

Statement of Conformance

This audit conforms to the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program.

Strength

IRSAS developed a resource planning tool to complete the annual operational planning of staffing and financial requirements to support the IAP. The tool is used to identify the planned activities, workloads and results. Against each of the planned activities, each individual Unit within the IRSAS forecasts the staffing required and related salary budget and the operating and maintenance budget. Further details allocate the individual staff to the particular strategic objective that will be supported by individual activities. This approach was considered by the audit to be a good practice and highlighted by management as a new tool developed and used for planning for 2014-15 and subsequent fiscal periods.

Conclusion

Generally, the audit found that control practices, related to AANDC's and IRSAS's support of the IAP, are adequate and effective. Some opportunities for improvement were noted to strengthen management practices in the following areas: consistent understanding of roles and responsibilities to support HR staffing transactions; risk management; and consistency of claims processing.

Recommendations

The audit team identified areas where control practices and processes could be improved, resulting in the following recommendations:

  1. The audit recommends the Assistant Deputy Minister of the Resolution and Individual Affairs Sector implement a formal risk management framework for the Settlement Agreement Operations Branch. This could be integrated into the annual Settlement Agreement Operations planning process to identify, assess and monitor the most significant risks that may preclude or limit the achievement of effective support of the Independent Assessment Process.
  2. The audit recommends the Assistant Deputy Minister of the Resolution and Individual Affairs Sector leverage the results of the "value stream mapping" process in order to find opportunities to harmonize IAP processes and procedures between groups within the Settlement Agreement Operations Branch, including the identification of best practices which should be implemented consistently across the Branch.
  3. The audit recommends the Executive Director of the Indian Residential Schools Adjudication Secretariat continue to evolve the risk management process through the provision of risk management training, and include formal risk identification and assessment processes at the unit level. This can be effectively integrated with the annual planning process to allocate resources, monitored to regularly respond to changes in risks, and used to support decision making.
  4. The audit recommends that the Director General of HRWSB, Executive Director of IRSAS, and Assistant Deputy Minister of RIA negotiate and implement a formal agreement, such as a Memorandum of Understanding, with regard to HR support services provided by HRWSB. This agreement would clarify the roles and responsibilities and service standards that govern HR staffing, and clarify IAP specific staffing flexibilities.

Management Response

Management is in agreement with the findings, has accepted the recommendations included in the report, and has developed a management action plan to address them. The management action plan has been integrated into this report.

 

1. Background

1.1 Introduction

An Audit of AANDC Support to the Independent Assessment Process was included in Aboriginal Affairs and Northern Development Canada's 2014-2015 to 2016-2017 Risk-Based Audit Plan, approved by the Deputy Minister on February 6, 2014. The audit was identified as a high priority because of the financial obligation it represents to the Department and the risks involved if delivery of a high volume of claims is not met, resulting in non-compliance with the Indian Residential Schools Settlement Agreement.

1.2 Independent Assessment Process

The Independent Assessment Process (IAP) is a claimant-centred, non-adversarial, out of court process for the resolution of claims of sexual abuse, serious physical abuse, and other wrongful acts suffered at Indian Residential Schools (IRS). The Indian Residential Schools Settlement Agreement (IRSSA) is the largest class action settlement in Canadian history and is expected to bring a fair and lasting resolution of the legacy of Indian Residential Schools. The IAP is one element of the IRSSA, which also includes the Common Experience Payment (CEP), Truth and Reconciliation Commission, Commemoration, and measures to support healing such as the Indian Residential Schools Resolution Health Support Program and an endowment to the Aboriginal Healing Foundation. Former students of residential schools who experienced sexual or serious physical abuse may be eligible for compensation through the IAP.

Relative to the IAP, the Settlement Agreement Operations Branch (SAO), within AANDC's Resolution and Individual Affairs Sector (RIA), is responsible for representing Canada at hearings, negotiating settlements without a hearing, researching and paying the settlements. SAO also leads requests for funding to support implementation of IAP, represents Canada on the Oversight Committee and its Technical Sub-Committee, manages instructions to Canada's counsel on IAP litigation issues. SAO consists of Resolution East (Ottawa) and Resolution West (Vancouver) which support IAP, and National Research and Analysis (NRA) which support IAP (including IRSAS), as well as other IRSSA obligations.

The Indian Residential Schools Adjudication Secretariat (IRSAS or Secretariat) is an independent, arms-length entity which falls under AANDC's departmental umbrella. The Secretariat's mandate is to implement and administer the Independent Assessment Process under the direction of the Chief Adjudicator in an independent, objective and impartial manner. The role of the Secretariat staff is to develop and manage strategies and operations of the IAP, including intake, screening, hearing management, and client relations management, as well as the formulation of operational policy and the financial management control framework and the management of the human resources, information management, procurement, contracting and materiel management functions In addition, the Secretariat is responsible for ensuring a claimant-centred approach, providing one-on-one support to claims who are representing themselves, outreach, and a Group IAP contribution funding program. The Chief Adjudicator and Adjudicators, who are on contract, independently assess claims of former IRS students and render decisions. 

The IAP came into effect at the time of implementation of the IRS Settlement Agreement which began on September 19, 2007. Under the terms of the Settlement Agreement, the deadline for submitting IAP applications was September 19, 2012. Subject to court approval, IAP applications will not be accepted if they are submitted past this deadline, as stated in the Agreement.

Governance and oversight of the IAP is provided by an "all parties" Oversight Committee which is set out in the Settlement Agreement (Schedule D - Section III. r.) The Settlement Agreement states the Committee shall be "comprised of an independent chair and eight other members, two reflecting the interests of each of the following constituencies: former students; plaintiff's counsel; church entities; and government".

 

2. Audit Objective and Scope

2.1 Audit Objective

The objective of the audit was to assess the effectiveness and efficiency of the management control framework in place at IRSAS and AANDC to provide support to the IAP. More specifically, the audit assessed whether:

  • Effective management controls are in place to help ensure the efficient and effective delivery of support services to the IAP provided by IRSAS and AANDC; and
  • Controls are in place to help ensure efficient and effective governance and management of the wind down of IRSAS and AANDC while providing ongoing support services to the IAP.

2.2 Audit Scope

The scope of the audit included responsibilities of AANDCFootnote 2 and the IRSAS to support the IAP.  The audit scope covered the period April 1, 2013 to May 31, 2014 and included an assessment of management practices and controls in place to support the IAP. The scope of the audit was developed to respect the IAP and the independence of IRSAS from AANDC/Canada - a defendant in the IAP. As such, the audit did not include; the aspects of the IAP that are administered under the direction of the Chief Adjudicator; any assessment of decisions made by the independent adjudicators; and IRSAS management controls related to the processing of IAP claims. The audit scope excluded an assessment of claim files through the IRSAS claims management process. This decision was based on discussion with IRSAS management and the Legal Services Advisor for the Audit and Evaluation Sector. The determination was that the majority of these policies and procedures fell under the direction of the Chief Adjudicator, and not appropriate for AANDC (a party to the IAP) to audit.

 

3. Approach and Methodology

The audit was conducted in accordance with the requirements of the Treasury Board Policy on Internal Auditand followed the Internal Auditing Standards for the Government of Canada. The audit examined sufficient, relevant evidence and obtained sufficient information to provide a reasonable level of assurance in support of the audit conclusion.

The audit plan was developed using a risk-based approach through which key risks facing AANDC and IRSAS were identified through interviews with management and documentation review. Key risks were then linked to the core governance, risk management and control policies, processes and practices in place within AANDC and IRSAS that are designed to mitigate the risks identified. This information was used to determine the objectives, scope, and criteria for the audit.

Audit criteria define the standards against which the AANDC and IRSAS's applicable governance, risk management and control policies, processes and practices were assessed. They are expressed in terms of reasonable expectations for these processes and practices to achieve expected results and outcomes. The audit objective was linked to audit criteria (Appendix A) developed in alignment with Treasury Board of Canada Secretariat's Audit Criteria related to the Management Accountability Framework: A Tool for Internal Auditors (March 2011).

 

4. Conclusion

Generally, the audit found that control practices, related to AANDC's and IRSAS's support of the IAP, are adequate and effective. Some opportunities for improvement were noted to strengthen management practices in the following areas: consistent understanding of roles and responsibilities to support HR staffing transactions; risk management; and consistency of claims processing.

 

5. Audit Findings and Recommendations

Based on a combination of evidence gathered through the examination of documentation and systems, analysis, and interviews, each audit criterion was assessed by the audit team and a conclusion for each audit criterion was determined. Where a significant difference between the audit criterion and the observed practice was found, the risk of the gap was evaluated in the development of the conclusion and to document recommendations for improvement.

Audit results are presented separately for AANDC (SAO), IRSAS, and AANDC-Corporate Services, and categorized by the audit criteria. 

5.1 AANDC – Settlement Agreement Operations

This section provides a summary of audits results, findings, and recommendations specific to the Settlement Operations Branch within AANDC.

5.1.1 Governance, Oversight, and Strategic Direction

The audit expected the establishment of effective governance mechanisms and oversight bodies with clearly communicated mandates, and that management has clearly defined and communicated the organizations strategic direction, which is aligned to the IAP objectives.

The audit noted that AANDC participates in the "all parties" IAP Oversight Committee (as described in section 1.2).

AANDC established an Indian Residential Schools Settlement Agreement Operations Committee with draft terms of reference (January 10, 2010) which stated the scope of the committee "is to provide governance on all administrative issues and risks to the IRS Settlement Agreement". The Committee is chaired by the Director General of SAO, and membership includes representatives from AANDC, IRSAS, Department of Justice (DOJ), the Court Monitor (Crawford Class Action), and Service Canada.

SAO management noted the use of the IAP Claims Management Committee, which is co-chaired by the Director General of SAO and the Director of Legal Services, to instruct AANDC and DOJ activities and conduct as Canada's representatives to the IAP.

In support of communicating its strategic direction, SAO management complete business plans as part of the broader RIA Sector plan annually. SAO has established key activities including: providing all mandatory documents; attending hearings; conducting negotiated settlements; and processing IAP payments. Annual resource planning (staffing and financial) considers the annual number of planned hearings as communicated by IRSAS (e.g. 4,500 hearings were planned for 2013/14). 

The audit concluded that governance mechanisms and oversight bodies are in place and adequate.

5.1.2 Accountability

The audit expected that management has clearly established and communicated expected organizational and individual accountability for services in support of the IAP, including responsibilities and performance expectations to which managers and supervisors are held accountable. These expectations should be formally defined and clearly communicated, and job descriptions and/or performance agreements should in place and maintained.

The audit found that employee performance management plans are prepared by staff members at the beginning of the year to set annual goals against which they will be evaluated. These annual goal are developed with guidance provided by management. The audit reviewed guidance provided for the Litigation Information Analyst and Resolution Manager positions. The guidance includes the departmental strategic objective with example work objectives and performance indicators to establish individual goals aligned to the SAO Branch objectives as reported in the annual RIA business plan. 

The audit concluded that accountability practices are adequate and effective.

5.1.3 Results and Performance

The audit expected that Management has identified and monitors appropriate performance measures that are aligned to planned objectives in support of the IAP.

The audit noted the reporting of results and performance in support of the IAP provided in the IAP Dashboard report which includes all key steps in the IAP from claim admission, through document collection from claimant and Canada, negotiated settlement, hearings offered and held, decision and ultimately payment of the claim. This report is a collaboration between SAO, IRSAS and Crawford (Court Monitor), who have responsibilities to provide the reporting for the IAP steps that they support.

SAO also reports on a quarterly basis, the progress made for the key activities in support of the IAP. The key activities reported include: providing Canada's mandatory documentation, locating and contacting alleged perpetrators, case management, attending hearings, negotiating settlements and processing IAP payments. In addition, the Department reports on key IAP activities to Parliamentarians through the Departmental Performance Report.

The audit concluded that management's identification and monitoring of performance measures was adequate and effective.

5.1.4 People – Training and Tools

The audit expected management would have appropriate training, tools, resources and guidance available to staff responsible for providing support services to the IAP

Management provides technical and supportive training for staff. Technical training is directed to the processes in place to support the IAP (including manuals and IAP training modules) as well as orientation to the Department as needed for hires new to the federal government or AANDC. The AANDC Learning Roadmap provides a number of orientation training courses (mandatory and recommended) that are provided by AANDC and by the Canada School of Public Service. Management noted that the approach to technical training has been developed based on the staff level through a combination of hands on learning, job shadowing, observing hearings and quality control checks. The audit noted the following examples of specific training approaches:

  • Clerk level (e.g. CR4) – job shadowing is a primary method used and it is supported by an employee manual, job description and initial information orientation to the IAP.
  • Litigation Information Analyst (e.g. EC 3) – Resolution Managers provide initial training combined with an experienced LIA who is dedicated to a new LIA to provide coaching and support.
  • Resolution Manager (e.g. EC 5) – training for this position is the most formalized as these individuals represent Canada during the hearing. Training includes mandatory IAP 101 and 201 training modules as well as a three to four month period during which the trainee RM is observed during live hearings and provided feedback and coaching.
  • Training and tools are available to staff for use of information systems, such as SADRE and GroupWise. Management noted that a protocol is in place to migrate IAP claim specific documentation captured in GroupWise into SADRE in preparation of the departmental email system transition to Microsoft Outlook. General IAP records are to migrate to CIDM.

Additional available tools include Sharepoint and knowledge sharing sessions. Sharepoint is a knowledge management tool use by the Department of Justice to which SAO has access to as a reference source. Monthly conference calls are arranged (Canada's Representatives Call) to provide a forum in which DOJ and SAO Resolution Managers discuss best practices and knowledge sharing. Management noted there is also a committee responsible (Cross Team Operations) to review the processes and tools.

Supportive training provided includes wellness training that has been made available to staff to help address vicarious trauma that may occur from exposure to details of claimant abuse allegations. The training has been in place since 2008 and was revised in 2014 to focus on resilience.

The audit concluded that training and tools are in place and are adequate.

5.1.5 IAP Completion Planning

The audit expected that management has clearly established and communicated plans to provide support services to the IAP over the wind down period. This would include the identification of organizational goals, priorities, identification of risks, required systems and staff capacity to provide support to completing the IAP over the wind down period.

Planning for the wind down of the IAP was noted by the following:

  • As part of the ongoing development of a transition plan, the SAO Branch recently completed a value stream mapping exercise as part of its transition planning process to wind down over the time period 2014-15 to 2019-2020. The Branch has also initiated a Human Resources Management Process as part of the transition process that will look to align resources to evolving organizational needs.
  • Management noted work is underway to prepare a proposal to provide funding for the period 2015/16 to 2019/2020; and
  • Quarterly reporting 2013-14 included commitment to contribute to the development of a completion strategy produced by the Secretariat and reviewed by the audit. Management noted this contribution encompassed an articulation of SAO's role and general branch commitments.

The audit concluded that IAP completion planning is in place and is adequate.

5.1.6 Risk Management

The audit expected that there are adequate risk management mechanisms in place that are appropriately linked to the IAP objectives. This includes the identification, assessment for likelihood and impact, and ongoing monitoring of risks that may impact or preclude the achievement of SAO objectives.

Settlement Agreement Operations Branch is part of the Residential and Individual Affairs Sector. As such the SAO planning and reporting is included in the broader RIA Business Plan and Quarterly Reporting. The audit noted the identification of risks and mitigation strategies in the RIA Business Plan and Quarterly Reports. The audit did, however, note that a formal and comprehensive risk management framework was not in place in the SAO Branch to manage the support to the IAP.

Risk management is an important element of any large project such as the IAP and can be an effective tool to support setting priorities and resource allocation decisions in the achievement of strategic objectives and mandate.  Risk management would include environmental scanning both internal and external to identify risks other risks that could impact the SAO ability to provide the necessary support to the IAP. An example of risk identification was noted in a recent business process mapping exercise, "the worst case scenario for SAO is increasing work load while it loses its specialized expertise and its operating systems begin to fail".    Other potential risks that management identified include the risk that litigation arises from the IRSSA, including the IAP or Article 12Footnote 3 requests, resulting in additional efforts to provide support beyond the targeted wind-down period. Existing risk identification practices could be augmented to identify other key risks that could impact SAO and its ability to support the IAP to completion. The list of risks should be assessed for "likelihood and impact", with the most significant risks being monitored over time. This process could be included as a component of the annual business planning process.

Recommendation:

1. The audit recommends the Assistant Deputy Minister of the Resolution and Individual Affairs Sector implement a formal risk management framework for the Settlement Agreement Operations Branch. This could be integrated into the annual Settlement Agreement Operations planning process to identify, assess and monitor the most significant risks that may preclude or limit the achievement of effective support of the Independent Assessment Process.

5.1.7 Process Stewardship

The audit expected that processes and procedures are established and formally documented to provide clear guidance and stewardship in the execution of the operational procedures in place to ensure Canada's responsibilities are met throughout the claims process from admission to payment.

Test of Claims Processing

In order to test the effectiveness of SAO's key claim management process controls, the audit selected a sample of twenty (20) claims to review key milestones and approvals. Four of the samples selected were negotiated settlements (Negotiated Settlement Process) which are negotiations directly between the claimant, claimant counsel and Canada (represented by SAO or DOJ). The sample of claims were split evenly between the Resolution West group in Vancouver, and the Resolution East group in Ottawa. Based on the testing results, the audit found that generally key controls were operating as intended. Claims were assigned to appropriate staff, review and approvals were conducted as claims moved through various stages of the process, key decision and analysis documents were retained and available, all payments were appropriately approved (Section 34 of FAA) and accurate based on the adjudicated or negotiated compensation amount.  

The claim testing also looked at various service standards in place within SAO to manage their processes. Some of these service standards are set out in the IRSSA, some were established between SAO and IRSAS (through discussions at Oversight Committee), with others set as internal standards. The following is a summary of the service standard test results:

  • Canada's mandatory documents are to be provided to the IRSAS within 165 days: the audit found three instances where the mandatory documentation was not provided within the 165 day services standard. In each of these three instances the cause of the delay was noted in SADRE to be issues related to the person of interest (POI) research.
  • Canada has 30 days from receipt of decision to request a review. Resolution managers at SAO conduct a review of the decision and make a recommendation to have the decision reviewed or to accept the decision as rendered. The audit noted that all claims tested were reviewed by Canada's representative (SAO or DOJ) to assess if a decision review was to be recommended or not.
  • The claim payment service standard followed by SAO is to issue payment within 50 days of receiving the decision from IRSAS. AANDC's target for this service standard is for payment to be issued within 50 days 80% of the time.  The audit noted four instances (out of 20) where payment was processed well after the 50 days service standard (a week or more).

Through the review of claims processing procedures, the audit noted the approaches and procedures followed across SAO to support the IAP (i.e. level of documentation, process followed and organization structure in place) are not consistent between Resolutions East and West. Examples of differences noted include:

  • Use by Resolution West of a Excel based claim tracking document ("Tracker"), in addition to SADRE (claim tracking software used by SAO and IRSAS), to monitor the progress of claims.. Resolution East relies on SADRE and the GroupWise email system to manage claim files.
  • Alignment of Litigation Information Analysts and Resolution Managers – LIA and RM are teamed together in Resolution West whereas LIAs are pooled and support RMs as needed in Resolution East.
  • Differences in approach to assigning tasks – for example assigning Canada's representative (i.e. a Resolution Manager from SAO or from the Department of Justice) to Claimant hearings is done by a clerk (staffing level - CR4) in the West, and by resolution manager (staffing level - PM6) in the East.
  • Data management and operational process controls such as review of mandatory documentation and approvals in Resolution West included systems (SADRE and Tracker) as well as a claimant and payment file that included routing slips, checklists and documented approvals with signature/initial and date. Claim documentation in Resolution East included use of SADRE but did not include any paper file or checklists.
  • Resolution East Front End Shared Services sends a research request to a generic email account, after which a Research Coordinator from NRA-East assigns the research to be completed by contracted consultant. Resolution West Front End Shared Services provides request directly to a Senior Research Analyst (NRA-West) who assigns research to a contracted researcher.

As the IAP is completed and the work volume and claims profile changes over time, there is a risk that differences in processes between Resolution East and West impact the efficiency and effectiveness of the support provided by SAO to the IAP. It will be increasingly important to have staff that can support the national process regardless of whether they are located in the East or West. SAO undertook a process mapping exercise which focused on "testing the stability and efficiency of the business processes required to conclude the IAP". Objectives included harmonizing systems and process improvements. The exercise identified a number of recommendations for SOA management and staff consideration.

Recommendation:

2. The audit recommends the Assistant Deputy Minister of the Resolution and Individual Affairs Sector leverage the results of the "value stream mapping" process in order to find opportunities to harmonize IAP processes and procedures between groups within the Settlement Agreement Operations Branch, including the identification of best practices which should be implemented consistently across the Branch.

5.2 Indian Residential Schools Adjudication Secretariat

This section provides a summary of audits results, findings, and recommendations specific to the Indian Residential Schools Adjudication Secretariat.

5.2.1 Strategic Direction

The audit expected that defined and communicated strategic direction is in place and aligned to the organization's mandate of supporting the IAP, along with clear and communicated operational plans with objectives aimed at achieving the strategic objectives.

The audit noted that IRSAS completed a strategic plan that was approved by the Executive Committee on March 22, 2013. Seven strategic objectives were identified:

  1. Process claims in a timely manner;
  2. Ensuring a claimant-centred approach
  3. Manage resources economically, efficiently and effectively;
  4. Manage information effectively;
  5. Provide support to the Chief Adjudicator;
  6. Promote a healthy work environment/organizational wellness; and
  7. Completion of the IAP.

These objectives are reported in the IRSAS annual business plan and quarterly reporting. The strategic objectives were noted to be used to develop unit business plans.

IRSAS developed a resource planning tool to complete the annual operational planning of staffing and financial requirements to support the IAP. The tool is used to identify the planned activities, workloads and results. Against each of the planned activities, each individual Unit within the IRSAS forecasts the staffing required and related salary budget and the operating and maintenance budget. Further details allocate the individual staff to the particular strategic objective that will be supported by the individual activities. This approach was considered by the audit to be a good practice and highlighted by management as a new tool developed and used for planning for fiscal 2014-15 and subsequent fiscal periods.

The audit concluded that strategic and operational planning was adequate and effective.

5.2.2 Accountability

The audit expected that management has clearly established and communicated expected accountability for services in support of the IAP. Accountabilities include responsibilities and performance expectations to which managers and supervisors are to be held accountable. These are to be formally defined and clearly communicated and job descriptions and/or performance agreements should in place and maintained.

Interviews with management noted employee performance management plans are prepared by individual staff members at the beginning of the year to set annual goals against which they will be evaluated. IRSAS provided example work objectives provided to staff as example goals to consider and include, as applicable, in individual employee management plans. The audit reviewed a sample of employee performance management plans including Adjudication Manager, Admission Analyst, Scheduler and Case Management Manager. Work objective goals were found to be aligned to the applicable Unit and IRSAS business plan objectives and demonstrated communication and understanding of, and accountabilities for, goals aligned to strategic objective key activities.

The audit concluded that accountability mechanisms were adequate and effective.

5.2.3 Process Stewardship

The audit expected to find that operational procedures are documented and clear guidance and supporting tools are in place to ensure support services to the IAP are executed effectively. The audit also expected to see process improvements to the IAP support services over time.

IRSAS support services to the IAP are documented and provide guidance in the operational processes and procedures.  Examples noted included:

  • The IAP process is flowcharted to provide a high level illustration of the steps involve in the process and is used as part of the on-boarding process for newly hired staff;
  • Client Services Unit has produced desk guides to support staff responsible to provide assistance to Self-Represented Claimants;
  • Chief Adjudicators Office has prepared a CAO Control Framework document that provides CAO staff with the process to be followed for: budgeting, business plans, orientation, employee performance, training, and IAP procedures.
  • Decisions Unit has developed a quality control framework that is used to monitor and manage quality of processing by staff. Supervisors in the Unit follow the process outlined to select samples to verify the completeness and accuracy of processing and document handling completed by staff.

The audit noted that IRSAS prepared a report that highlighted numerous process improvements for the IAP that have been identified and implemented since inception. The Report on Process Improvements for the IAP, March 2014 identified a list of ninety two specific process improvement changes impacting claims resolution, addressing priorities of the Settlement Agreement, administration and processing of claims, and adjudication.

The audit concluded that process stewardship support and process improvement mechanisms were in place and adequate.

5.2.4 People – Training and Tools

The audit expected management would have appropriate training, tools, resources and guidance available to staff responsible for providing support services to the IAP.

IRSAS uses a combination of the AANDC Learning Roadmap for training staff who are new to government as well as training that is focused specifically on the processes in place at IRSAS. The Learning Roadmap outlines required training for new public service employees offered by AANDC and the Canada School of Public Service.

The IRSAS HR Strategy 2014-17 included guidance for all staff to follow in preparing their learning plans. Learning plans are prepared by staff and a list of training requests is reviewed by management and the audit reviewed an example employee learning plan for 2014-15. Requests for training are provided by staff and compiled into a single list for management to review and approve. The audit observed the approved training for 2014-15.

Supportive training provided includes wellness training that has been made available to staff to help address vicarious trauma that may occur from exposure to details of claimant abuse allegations. This training is often delivered in coordination with SAO Branch.

The audit concluded that training and tools were in place and were adequate.

5.2.5 IAP Completion Planning

The audit expected that management has clearly established and communicated plans to provide support services to the IAP over the wind down period. This would include the identification of organizational goals, priorities, identification of risks, required systems and staff capacity to provide support to completing the IAP over the wind down period.

IRSAS has prepared and submitted a completion strategy and plan for resolving the remaining IAP caseload, which was provided to the Supervising Courts in December 2013. The strategy included: analysis of the claims admitted, progress made in resolving claim; opportunities to improve the rate of claims resolution; projections to resolve the claims inventory; and risk identification, assessment and mitigations.

IRSAS has also prepared detailed staffing projections integrated into a Completion Action Plan that was presented to the AANDC Workforce Management Board in May 2014. Detail staffing projections were included to present the expected staffing, both indeterminate and term, required over the IAP wind down period.

The audit concluded that IAP completion planning was in place and adequate.

5.2.6 Results and Performance

The audit expected that Management has identified and monitors appropriate performance measures that are aligned to planned objectives in support of the IAP.

The audit noted the reporting of results and performance in support of the IAP provided in the IAP Dashboard report which includes all key steps in the IAP from claim admission, through document collection from claimant and Canada, negotiated settlement, hearings offered and held, decision and ultimately payment of the claim. This report is a collaboration between SAO, IRSAS and Crawford, who have responsibilities to provide the reporting for the IAP steps that they support.

A key management reporting tool is the Quarterly Report which provides a summary of progress against the strategic priorities. Key activities are reported against each of the IRSAS strategic objectives, as on target or behind target based on performance measures and milestones. IRSAS quarterly reporting is done through the AANDC's reporting process, independently to the Treasury Board Secretariat, as well as to the Oversight Committee and the Courts.

Priorities are also reported and included in the executive summary section of the Quarterly Report. Management confirmed that priorities are special projects that are completed within the current year whereas key activities are ongoing across fiscal periods. The priorities reported in the 2013-14 Quarterly Report are identical to those reported in the 2013-16 Strategic Plan and were aligned to the strategic objectives. Some of the priorities for 2014-15 were revised and some remained the same to the prior year.

5.2.7 Risk Management

The audit expected that there are adequate risk management mechanisms in place that are appropriately linked to the IAP objectives. This includes the identification, assessment for likelihood and impact, and ongoing monitoring of risks that may impact or preclude the achievement of IRSAS objectives.

IRSAS has evolved its approach to risk management since the previous audit was conducted and now includes the preparation of a corporate risk profile that includes roles and responsibilities for the Executive Team, Risk Leads, Managers, and the Integration Planning Unit. The risks noted are: Aboriginal relationship; implementation; information management; legal; resource alignment; and capacity and capabilities. Each risk is individually reported with their mitigation strategies and risk leads responsible.

The audit noted good alignment of the Corporate Profile risks and those noted in the Completion Strategy. In reviewing a quarterly report for March 31, 2014, the audit team noted that procurement (risk of not having contract in place) was reported as a high risk, but this risk was not included in risk statements in the Corporate Risk Profile (although procurement  was identified as a "risk driver" of the Resource Alignment risk area). Given the level of impact, there may be an opportunity to communicate procurement risks with a higher profile in the Corporate Risk Profile. 

Documentation reviewed and interviews with management noted that risk management could be further evolved by including managers and supervisors as part of the risk management process. This would include increasing the breadth of the environmental scan for risk identification and assessment across Units to improve the completeness of identified measured risks. This could be integrated with the annual planning process currently in place. Unit objectives would be assessed for risks that could impact their achievement. The unit level risks should be assessed for "likelihood and impact" to identify the most significant risks that require mitigations to be in place. The allocation of resources should be considered in relation to the controls necessary to mitigate these risks. The top risks should be monitored during the year on a quarterly basis.

Recommendation:

3. The audit recommends the Executive Director of the Indian Residential Schools Adjudication Secretariat continue to evolve the risk management process through the provision of risk management training, and include formal risk identification and assessment processes at the unit level. This can be effectively integrated with the annual planning process to allocate resources, monitored to regularly respond to changes in risks, and used to support decision making.

5.3 AANDC – Corporate Services

The audit expected effective consultation and cooperative relationships between IRSAS, SAO and third party service providers, for example claimant counsel, and other federal government departments including corporate service within AANDC.

5.3.1 AANDC Corporate Services (Procurement)

The audit expected to see effective consultation and cooperative relationships between IRSAS, SAO and AANDC CFO Sector relative to procurement.

AANDC CFO Sector provides procurement services support to both SAO and IRSAS. Planning work completed by the audit noted that procurement services were not considered to be a significant risk area from the perspective of SAO Branch, but was a risk area for IRSAS.

IRSAS identified procurement support as a key challenge for IRSAS in providing support to the IAP. The IRSAS Strategic Plan 2013-16 (issued March 22, 2013) noted that "despite having strategies, procedures, and plans in place, we continue to address a set of long-standing corporate challenges that are beyond the Secretariat's control but require collaboration across various sectors of AANDC if progress is to be realized. Management has placed renewed emphasis on human resources and procurement to ensure that the required capacity and services are in place". 

IRSAS requires external assistance in support of the IAP from adjudicators, to conduct the hearings and issue decisions, and from lawyers, to provide independent counsel for the Chief Adjudicator and the Secretariat. AANDC Procurement was responsible for providing assistance to the IRSAS to establish the required contracts to have the services in place. However it was recognized by IRSAS and AANDC Procurement that the authority limits for the Adjudicator contracts and legal counsel was not sufficient at AANDCFootnote 4. As a result the procurement support for these contracts requiring renewal and for new external legal counsel contracts was transferred to Public Works and Government Services Canada (PWGSC). IRSAS management indicated that the AANDC procurement group has been very supportive and helpful in assisting IRSAS engage with PWGSC procurement services. The audit did not assess the services being provided by PWGSC, as it is outside of this audit.

5.3.2 AANDC Corporate Services (Human Resources)

The audit expected to see effective consultation and cooperative relationships between IRSAS, SAO and AANDC's Human Resources and Workplace Services Branch (HRWSB) with regards to the provision of Human Resource services.

The Indian Residential Schools Settlement Agreement stipulates that the Government of Canada is to provide sufficient resources to ensure that IAP claims are processed. This means that both SAO and IRSAS are made up of federal employees, with the delegated authority for staffing flowing from the Deputy Minister of AANDC.   Therefore all staffing activities must follow Public Service Commission and departmental processes and procedures, with AANDC's HRWSB providing Human Resource support and advice to both SAO and IRSAS

The audit planning work completed noted that HR staffing has been a challenge for both SAO and for IRSAS. Staffing has been a consistent and significant risk in particular for IRSAS, who have historically experienced high levels of staff vacancy rates, as high as 30%.  

The IAP is a process that must be completed in a timely manner and accordance with the Settlement Agreement. The Completion Strategy submitted to the Supervising Courts December 2013, projects that IAP claims admitted will be resolved by 2017-18 with appropriate infrastructure remaining in place to carry out the functions assigned by the Settlement Agreement. 

The fact that both SAO and IRSAS are sunsetting organizations that will wind down as the IAP is completed creates a challenge to attract and retain sufficient staff to meet the supporting services required of the IAP. A key reason for staff turnover within both organizations is staff leaving for permanent jobs (i.e. indeterminate) across the federal government. Interviews noted that this tends to occur more often in centres where there is a large federal government presence with greater opportunities to transfer to another organization, such as in the National Capital Region. Many positions within IRSAS and SAO are staffed with term positions, with a "sunsetting" clause, meaning staff will need to find new employment once the IAP is wound down and their term expires. This creates an incentive for term staff to be constantly looking for other employment with more job security. Compounding these organizational challenges is the nature of the work which can expose staff to details of claimant abuse allegations, which can be very troubling and can create a more difficult working environment than experienced elsewhere.

These HR risks also have significant risks and implications for the Government of Canada and for AANDC in particular. These implications can be contradictory or "at odds". For example Canada must provide sufficient human resource support to the IAP to ensure that all claims get processed, otherwise risking the perception that Canada is not meeting their end of the Settlement Agreement. This must be balanced with the fact that all permanent employees remaining from SAO and IRSAS once the IAP winds down must be absorbed or re-deployed by AANDC, at a time when the Department is looking to streamline its workforce in order to manage within current budgets.

The audit noted that IRSAS and SAO have been engaging with AANDC HRWSB to identify opportunities to alleviate some HR pressures and streamline hiring processes. Most recently, between May to June 2014, IRSAS sought, and received, a number of flexibilities from AANDC's Workplace Management Board (WMB) that would help streamline staffing processes and give the Secretariat more staffing options. These include, for example, approving a number of critical indeterminate positions that can continue to be staffed in that manner, and the ability to launch new staffing processes before the applicable pool is depleted. Also, SAO and IRSAS have both been given an option of extending the expiry of staffing pools for term and indeterminate positions to 2 years, from the standard 1 year. The audit team encourages continued identification and implementation by HRWSB, SAO, and IRSAS, of flexibilities, strategies and initiatives that aim to streamline staffing processes in order to help alleviate some of the HR challenges faced by SAO and IRSAS, while remaining in line with the hiring principles of the Public Service Employment Act.

The audit also noted that SAO, IRSAS and HRWSB are experiencing some difficulties managing these staffing challenges cooperatively. For example, the audit noted delays in the implementation of staffing flexibilities provided to IRSAS by the WMB due to confusion and a delay in agreement on the nature of the decision. It is essential that once staffing initiatives are agreed upon, that they get implemented efficiently and effectively to avoid further staffing delays.  

The audit observed that there is no formal agreement, such as a Memorandum of Understanding, between these parties that establishes and clarifies the roles and responsibilities, as well as service standards, related to managing HR staffing processes and the provision or HR advice and guidance. Management from HRWSB, IRSAS, and SAO indicated that a formal agreement such as this would be beneficial. Ideally this agreement would articulate the expectations, roles and responsibilities, and service standards of managing HR processes, as well as detail the specific staffing flexibilities and process deviations available to IRSAS and SAO hiring managers. The objective of the agreement would be to clarify expectations around the provision of HR services by HRWSB and requirements of managers at IRSAS and SAO, as well as to help ensure the accurate and timely implementation of staffing process flexibilities in place to support IRSAS and SAO.

Recommendation:

4. The audit recommends that the Director General of HRWSB, Executive Director of IRSAS, and Assistant Deputy Minister of RIA negotiate and implement a formal agreement, such as a Memorandum of Understanding, with regard to HR support services provided by HRWSB. This agreement would clarify the roles and responsibilities and service standards that govern HR staffing, and clarify IAP specific staffing flexibilities. 

 

6. Management Action Plan

Recommendations Management Response / Actions Responsible Manager (Title) Planned Implementation Date (Month &Year)
1. The audit recommends the Assistant Deputy Minister of the Resolution and Individual Affairs Sector implement a formal risk management framework for the Settlement Agreement Operations Branch. This could be integrated into the annual Settlement Agreement Operations planning process to identify, assess and monitor the most significant risks that may preclude or limit the achievement of effective support of the Independent Assessment Process. The Resolution and Individual Affairs Sector will develop and implement a formal risk management framework for the Settlement Agreement Operations Branch as it relates to the Independent Assessment Process by March 31, 2015. The Strategic Integrated Management and Accountability Branch and the Settlement Agreement Operations Branch will co-lead this process with support from the Policy and Strategic Direction Sector. Assistant Deputy Minister, RIA March 2015
2. The audit recommends the Assistant Deputy Minister of the Resolution and Individual Affairs Sector leverage the results of the "value stream mapping" process in order to find opportunities to harmonize IAP processes and procedures between groups within the Settlement Agreement Operations Branch, including the identification of best practices which should be implemented consistently across Settlement Agreement Operations. The Settlement Agreement Operations Branch received a detailed report on September 18, 2014 outlining the findings of the "value stream mapping" process exercise. The report was translated and shared with all Resolution East and West staff. The Branch will identify all adaptive measures to the Assistant Deputy Minister by December 31, 2014 with measures to be implemented by the Branch by March 31, 2015. Assistant Deputy Minister, RIA March 2015
3. The audit recommends the Executive Director of the Indian Residential Schools Adjudication Secretariat continue to evolve the risk management process through the provision of risk management training, and include formal risk identification and assessment processes at the unit level. This can be effectively integrated with the annual planning process to allocate resources, monitored to regularly respond to changes in risks, and used to support decision making. As noted in the audit findings, the IRSAS has improved its' risk management process since the previous audit. The current system involves participation of unit managers, allowing for the risk identification across the organization, as well as the roll up and drill down of risk analysis and mitigation. This process provides for a complete listing of risks by risk type, which are then validated and approved by the IRSAS Executive Team, and posted on the intranet for staff's knowledge and reference.

As recommended in the audit, the IRSAS will continue to further evolve our risk management process. Risk management training will be provided to unit managers during the 3rd Quarter of 2014-15. This will include a tool for unit managers to allow them to lead risk management discussions with their staff towards a consistent mechanism across the IRSAS for risk management of all activities. This information will be utilized as enhance risk assessment during the annual planning process.
Executive Director, IRSAS December 2014

Status: This action has been met.
Update/Rationale:
As of 31/12/2014:

Before convening our work on updating the Risk Profile for 2015-16, we asked each manager to dedicate time to discuss risk with their individual units. More specifically, we wanted to give all Adjudication Secretariat staff the opportunity tell us what they think the risks are, now and over the coming year, that may  have an effect on our seven risks areas.

In order to facilitate their discussions, we provided each manager with a brief presentation with speaking points on risk management at the Adjudication Secretariat as well as the 2014-15 risk descriptions to achieving our objectives. Managers used this presentation as a tool to discuss and analyze risk to the Adjudication Secretariat with their staff before the risk sessions of December, 2014.

AES: Implementation complete. Recommendation to close. Closed.
4. The audit recommends that the Director General of HRWSB, Executive Director of IRSAS, and Assistant Deputy Minister of RIA negotiate and implement a formal agreement, such as a Memorandum of Understanding, with regard to HR support services provided by HRWSB. This agreement would clarify the roles and responsibilities and service standards that govern HR staffing, and clarify IAP specific staffing flexibilities. IRSAS and RIA agree with the recommendation and will collaborate with HRWSB to implement a Memorandum of Understanding (MOU) to confirm the roles, dedicated human resource supports, service standards and to set clear expectations regarding human resource management moving forward.

HRWSB, RIA and IRSAS will each identify a lead representative to collaborate on the terms of a MOU to be implemented by March 31, 2015.
Director General, HRWSB

Executive Director, IRSAS

Assistant Deputy Minister, RIA
March 2015
 

Appendix A: Audit Criteria

The audit objectives were linked to audit criteria developed in alignment with Treasury Board of Canada Secretariat's Audit Criteria related to the Management Accountability Framework: A Tool for Internal Auditors (March 2011).

Audit Objective #1: Assess the extent to which management controls in place help ensure the efficient and effective delivery of support services to the IAP are provided by IRSAS and AANDC
1.1 Governance and oversight bodies exist to support the efficient and effective delivery of required services to the IAP.

Management conducts strategic planning to establish objectives that are used to inform annual planning.
1.2 There are adequate risk management mechanisms that are appropriately linked to the IAP objectives.
1.3 Management has clearly established and communicated expected accountability for services in support of the IAP.
1.4 Process stewardship support services are documented to provide clear guidance and stewardship in the operational procedures on the steps taken throughout the process from admission to payment of claims.

Management effectively collaborates with partners and 3rd party service providers in the achievement of IAP objectives.
1.5 Management has identified & monitors appropriate results and performance measures that are aligned to planned objectives in support of the IAP.
1.6 Management has appropriate training and support service improvement tools and processes in place to implement and continually improve support to the IAP.
Audit Objective #2: Assess the extent to which controls are in place to help ensure efficient and effective governance and management of the wind down of IRSAS and AANDC while providing ongoing support services to the IAP
2.1 Senior management has clearly established and communicated plans to provide support services to the IAP over the wind down period.
 
 

Appendix B: Relevant Policies/Directives

The following authoritative sources were examined and used as a basis for this audit:

Guiding Agreement:

Legislation:

Planning, Monitoring and Reporting Tools:

 
 
Date modified: