ARCHIVED - Post-Implementation Audit of the First Nations and Inuit Transfer Payment (FNITP) System

Archived information

This Web page has been archived on the Web. Archived information is provided for reference, research or record keeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

Date: September 2010
Project Code: #08-24

PDF Version (154 Kb, 29 Pages)

 

 

Table of Contents

 

 

List of acronyms

The following is a list of acronyms used in this document.

AES
Audit and Evaluation Sector

CEAP
Canada's Economic Action Plan

CIDM
Comprehensive Integrated Document Management

COBIT
Control Objectives for Information and related Technology

FAA
Financial Administration Act

FNITP [Note 1]
First Nations and Inuit Transfer Payment

Gs&Cs
Grants and Contributions

GAAS
Generally Accepted Auditing Standards

Indian and Northern Affairs Canada

ITDM
IT Dependent Manual

MOU
Memorandum of Understanding

OASIS
Oracle Application Software Implementation Strategy

PKI
Public Key Infrastructure

PMBOK
Project Management Body of Knowledge

PMI
Project Management Institute

TPMS
Transfer Payments Management System

 

 

Executive Summary

Background

The First Nations and Inuit Transfer Payment (FNITP) system is the enterprise system in place since 2006 to manage more than $6 billion entrusted annually to Indian and Northern Affairs Canada (INAC) [Note 3] for the management of grants, contributions and other transfer payments.

The audit was conducted from November 2009 to August 2010. The work was conducted in a sample of five regions and INAC's headquarters in Gatineau, QC.

FNITP is not a stand-alone system, as interfaces exist to other departmental systems such as Oracle Application Software Implementation Strategy (OASIS) and Comprehensive Integrated Document Management (CIDM). These interfacing systems were not assessed as part of the audit. However, reports from these systems were used to validate the performance of control activities related to sections 32, 34 and 33 of the Financial Administration Act (FAA).

Objective and scope

The objective of the audit was to determine if the FNITP system supports an effective and efficient process to manage INAC's grants and contributions.

The scope of the audit included review of the following:

Methodology

The audit was conducted in three distinct phases: planning, conduct and reporting. A risk-based audit program was developed during the planning phase which focused on aspects of the FNITP system that impact program delivery and operations. The audit fieldwork was carried out during the conduct phase at headquarters in Gatineau, Quebec and in regional offices in Ontario, Saskatchewan, Alberta, British Columbia, and Northwest Territories. The audit program was structured to include documentation reviews, observations, interviews, and testing.

Findings and conclusions

The consensus of the interviewees was that the FNITP system is a significant improvement from the previous system known as the Transfer Payments Management System (TPMS). Throughout the audit process a number of key strengths were observed with the FNITP system. The top three strengths observed for the system were:

The audit identified the following findings:

The following root-causes of the performance issues were:

The evidence gathered suggests that there has been, and continues to be, a lack of sufficient training and support resources to maintain a Gs&Cs system that will effectively support funding to First Nations, Inuit, Métis and Northerners.

Furthermore, the lack of dedicated training and support resources to communicate the potential benefits of the system has resulted in under-utilization of the system by recipients.

In addition to the shortcomings related to resourcing, the lack of a policy framework and defined system ownership has also caused significant issues within the Department. The lack of a defined process to request, approve and implement system changes has resulted in changes being implemented without consultation or approval from the Transfer Payments Center of Expertise.

Recommendations

 

 

1. Background

INAC is one of the federal government departments responsible for meeting the Government of Canada's obligations and commitments to First Nations, Inuit and Métis, and for fulfilling the federal government's constitutional responsibilities in the North. The 2010-2011 main estimates total more that $7.2 billion of budgetary expenditures, with $6 billion earmarked for grants and contributions (Gs&Cs).

The Department supports Aboriginal people (First Nations, Inuit and Métis) and Northerners in their efforts to:

INAC delivers the vast majority of its programs and services in partnership with First Nations and federal-provincial or federal-territorial agreements.  INAC's significant investment in Information Technology supports key strategic INAC programs and operational business activities within the Department.

A Preliminary Survey of IM/IT Applications dated April 2008 was conducted to identify the applications with issues and risks that would require further attention by the Audit and Evaluation Sector (AES). This preliminary survey of IM/IT applications identified the First Nations and Inuit Transfer Payment (FNITP) application as a high-risk system.

FNITP was designed to replace the Transfer Payment Management System (TPMS). FNITP is the enterprise system in place since 2006 to manage INAC's grants, contributions and other transfer payments.

 

 

2. Objective and Scope

2.1 Objective

The objective of the audit was to determine if FNITP supports an effective and efficient process to manage INAC's grants and contributions.

2.2 Scope

The scope of the audit included review of the following:

For greater clarity, potential risks not assessed as part of this audit are the following:

 

 

3. Statement of Assurance

Sufficient work was performed and the necessary evidence was gathered to support the findings, recommendations and conclusions contained in this report. The work was conducted according to a risk-based audit program developed collaboratively with INAC management.

The risk-based audit program was based on Control Objectives for Information and related Technology, version 4.1 (COBIT 4.1) and the Project Management Institute's Project Management Body of Knowledge, version 4 (PMI PMBOK 4). The audit was executed in conformity with the Internal Auditing Standards for the Government of Canada. It does not constitute an audit or review in accordance with any Generally Accepted Auditing Standards (GAAS).

In addition to the Internal Auditing Standards for the Government of Canada, the audit procedures were aligned with Treasury Board's Policy on Internal Audit and related policy instruments as well as the Institute of Internal Auditors' (IIA) International Standards for the Professional Practice of Internal Auditing.

 

 

4. Methodology

4.1 Timeline

The audit was conducted in three distinct phases:

4.2 Areas of focus

The following are areas that were assessed for the FNITP application and the FNITP business process transformation program:

4.3 Audit approach

The following approach was based on the status of FNITP at the time of the audit:

The audit fieldwork was conducted at headquarters and in a sample of regions as follows:

Regions were selected based on the following criteria:

We used a risk-based approach to scope the audit and focus on areas of greatest risk to the Department. The audit program was structured to include documentation reviews, observations, interviews, and testing.

 

 

5. Understanding the Business Context of FNITP

FNITP is an integral tool in facilitating the management of grants and contributions at INAC. The FNITP system provides functionality to:

The goals of the FNITP system are to:

FNITP assists INAC in applying funding management practices in accordance with the Management Accountability Framework and the Treasury Board's Policy on Transfer Payments.

FNITP is a centralized departmental system that interfaces to other departmental systems. It is used by both internal and external stakeholders. FNITP is supported by the Integrated Performance and Corporate Systems Directorate and guided by the Transfer Payments Center of Expertise.

 

 

6. System Strengths

The consensus of the interviewees was that the FNITP system is a significant improvement from the previous system known as the Transfer Payments Management System (TPMS).

Throughout the audit, it was observed that the FNITP system:

 

 

7. Findings

The audit findings are outlined according to four main areas of examination as follows:

These areas are described in the following sections.

7.1 Program review

7.1.1 Adoption of the system by recipient users

A major driver for implementation of the system was to allow recipients to have access to their Gs&Cs information from within their community. The original FNITP Business Case listed the following objectives:

Some of the benefits for recipients of FNITP include the ability to:

The audit measured the adoption of the system by recipient users by calculating the percentage of users who appeared to be actively using the system. This was termed the "take-up rate". The recipient user take-up rate is the percentage of total recipients logging in to the FNITP system. Interviews and testing in the regions confirmed that the take-up rate for recipient users of the system was low. Figure 1 shows the results of the testing in the five regions visited. Take-up rates were calculated based on the following FNITP reports:

It is possible for a recipient user to have several active accounts. For purposes of testing a recipient user, it was considered valid, if at a minimum, the recipient user had at least one active account with a reported login date. Any login date was considered valid (login dates can extend back to system inception in 2006). For each region, the total number of active recipients (recipients with active funding arrangements in 2009-2010) was divided by the total number of recipient users to determine the take-up rate.

Recipient User take-up rates
(based on FNITP produced report)

Figure 1 - Recipient user take-up rates

This graph illustrates the percentage of recipients in each region who appeared to be actively using the FNITP system.

Interviewees explained that the low take-up rate was a result of a lack of training and support provided to recipient users. Interviews confirmed that in all regions recipient users did not receive adequate support tailored to their individual needs. Regional interviewees indicated that the national help desk model did not provide adequate support required to meet the needs of the regional recipient users.

Interviewees stated that there were not enough dedicated trainers to accommodate the needs of recipients. Rather than having a trainer available at the national level, interviewees suggested that additional regionally dedicated resources be obtained to deliver training to recipients.

Interviewees explained that in some instances recipients did not trust FNITP's ability to provide recipient reports to the appropriate INAC staff. This lack of trust results in recipients not only uploading recipient reports to FNITP, but also submitting copies of the same report by mail and/or fax and/or dropping off in person. Their expectation is that the more copies provided, and communication media used, the more likely the recipient report would be received and approved. However, interviewees noted that multiple submissions did not necessarily ensure a greater likelihood that the recipient report would be received and approved. Interviewees attributed the occurrence of these events to the lack of communication from INAC to recipients of the optimum method of recipient report submission.

7.1.2 Training and support

Training for recipients, Funding/Field Services Officers, Financial Arrangements and Quality Assurance Officers, and Accounting Officers is critical to FNITP data integrity and functionality. Interviewees stated that not enough training was available to support:

During the audit we identified discrepancies among regions in the following areas:

Furthermore, the training and support provided at the national level did not support the day to day operational enquiries encountered in the regions. The Department had not defined service standards for internal and external users of FNITP.

Although most regions informally identified an FNITP functional expert, interviewees stated their strong desire for additional dedicated FNITP functional experts to serve both internal and external users in the regions.

Finding 1: A lack of training and support available to regional offices and recipients has resulted in limited adoption of the FNITP system by recipients. Due to the lack of training and support, some regions no longer encourage the adoption of FNITP by their recipients.

Without sufficient training and support to meet the needs of staff and recipients, acceptance of the system will continue to be affected. Interviewees stated that many users, particularly the recipient users, are unaware of the many benefits of the system.

Recommendation 1: The CFO, in collaboration with the ADM s responsible for regional operations and staff, should develop and implement a program that ensures sufficient resources are allocated to providing training and support to recipient users. In conjunction with this training and support program, regional representatives across the country should champion the adoption of FNITP by their recipients.

7.1.3 System governance

Establishing alignment between the functional and technical areas of an organization is crucial to successful system governance. During the audit, we observed gaps in system governance as detailed below.

System change management

Alignment between functional and technical areas for change management purposes is vital in supporting operational processes effectively and in accordance with policy. Interviewees stated that the process to perform system changes remains undefined. They explained that changes were completed without consultation or approval from the functional group (INAC's Transfer Payments Center of Expertise). At the end portion of the conduct phase of the audit, we received the draft Directive for the Submission and Management of Change Requests. Although the directive has not been formally approved, it represents a step in a positive direction for the Department in implementing a policy framework to manage system changes.

Interviews with regional users identified the FNITP Change Management Governing Group as the authority responsible for the discussion and review of the changes to the system; however, interviewees noted that the mandate of this group remains unclear. Particularly, it is unclear whether this group was granted the authority to approve the implementation of changes to the system. As a result of this, changes were implemented into the system without the consent of key operational staff.

Ownership

In addition to the system change management issues resulting from the lack of a policy framework, we also observed confusion related to system ownership. Interviewees stated that both system and business ownership are associated with the Integrated Performance and Corporate Systems Directorate within the Information Management Branch.

Finding 2: A policy framework has not been established to include appropriate support from the Transfer Payments Center of Expertise. This resulted in issues related to system change management and ownership.



Recommendation 2: The CFO should ensure that the Transfer Payments Centre of Expertise and the Integrated Performance and Corporate Systems Directorate finalize a policy framework and departmental directive to address issues related to system change management and ownership.

7.1.4 Flow-through/Interdepartmental coordination

During the audit we observed flow-through funding. According to the interviews three types of flow-through funding exist:

Flow-through funding involves a transfer of goods between budgetary appropriations (between departments). Adherence to Treasury Board's Policy on Interdepartmental Charging and Transfers between Appropriations was required when executing flow-through payments. This Policyrequires a written financial arrangement between the department supplying goods and services and the department receiving goods and services. This arrangement provides the basis for charging and recovering for the goods or services transferred between appropriations.

Responsible managers of stakeholder organizations negotiate the inclusion of the following within the financial arrangement:

No evidence was provided to demonstrate that an agreement and/or Memorandum of Understanding (MOU) existed between INAC and other parties (i.e. other departments and other levels of government) providing funding.

In one case, a recipient funding agreement provided funding from INAC and Health Canada to a recipient First Nation. According to this agreement,

"The Minister (of Health) reserves the right to audit or cause to have audited the accounts and records of the Council… to ensure compliance with the terms and conditions of the Health Canada Schedule."

This agreement also indicated that

"INAC may withhold from any payment due under this Health Canada Schedule or any subsequent agreement between the Minister (of Health) or INAC and the Council."

However, the agreement did not clearly indicate how notification of a payment withholding would be communicated from Health Canada to INAC and under what circumstances the recovery would be executed.

Furthermore, execution of payment withholdings on the behalf of other parties poses a reputational risk to INAC. The recipient cannot easily distinguish the payment withholdings between INAC and other parties involved.

Finding 3: In the cases where more than one department or other levels of government are signing the funding agreements with First Nations, no evidence was available for review showing Memorandums of Understanding (MOU) exist. MOUs should detail how departments or other levels of government notify INAC of payment withholdings and under what circumstances these payment withholdings should be executed.



Recommendations 3: The CFO should implement the requirement to finalize MOUs in the cases where more than one department or other levels of government are signing the funding agreements with First Nations. The MOUs should clearly indicate how notifications of payment withholdings will be communicated from departments or other levels of government providing funding to INAC and under what circumstances these payment withholdings will be executed.

7.2 Operational business process

7.2.1 System support of grants and contributions process

The FNITP system assumes a streamlined national process for managing funding arrangements with First Nations and other funding recipients; however, regions have unique business requirements and distinct business processes that are not being supported by the system. Regional interviews identified several instances where the system was not supporting the regional process including:

The way some regions managed their capital projects was not aligned with workflow provided by FNITP. These regions have implemented a blanket override function that would allow for recognition of more than one arrangement per recipient.

Delegated authorities for the approval of FAA section 32, 34 and 33 were not configured directly into FNITP. Authority codes defined by the Financial Signing Authority Manual were not linked to activities in FNITP.

Finding 4:  FNITP is a central system supporting a decentralized process. Regional business processes and system capabilities do not fully align to effectively support the delivery of grants and contributions across all regions and programs.



Recommendations 4: The CFO should lead a cross functional team responsible for assessing regional processing requirements and establishing a procedure to incorporate them into FNITP. The team should be comprised of members from the Transfer Payments Centre of Expertise and the Integrated Performance and Corporate Systems Directorate with input from regions and programs.

7.2.2 Application controls

The Financial Administration Act (FAA) and Management Control Framework provided procedures for the internal control of funds allocated to departments and agencies that are important in maintaining accountability and integrity. During the testing, we identified gaps in the way headquarters, the regions and the FNITP system applied these procedures:

FAA Section 32
FAA Section 34
FAA Section 33
FAA Delegated authorities
Approval/Acceptance
Overrides
Funding arrangements

Finding 5: The INAC Gs&Cs process and FNITP system were not designed to fully utilize the capabilities of an automated system that supports the Management Control Framework, the Financial Administration Act (sections 32, 34 and 33) and Treasury Board's Policy on Transfer Payments.

Previously identified findings in 2007 System Under Development Audit:



Recommendation 5: The CFO should refine INAC's Financial Management Manual for Transfer Payments to include a clear definition of FAA sections 32, 34 and 33 controls. The definition should outline how the controls will be captured and represented in FNITP. System application controls should be modified to align with the refinements to the Financial Management Manual.

7.3 Reporting

The FNITP system plays a key role in providing a departmental overview of Gs&Cs information. INAC provides more than $6 billion worth of grants, contributions and other transfer payments for which it requires accurate and relevant reporting capabilities.

Throughout the audit, we observed that some regions were using other mechanisms to manage information to conduct their programs. Regionally-developed systems and spreadsheets were being used for tracking and reporting purposes. Listed below are examples of regionally-developed systems and spreadsheets:

Finding 6: FNITP does not provide certain types of reports that would be relevant to regional users. This capability is critical to the objective of providing a centralized repository for Gs&Cs data.



Recommendation 6: The CFO should lead a cross functional team responsible for determining whether FNITP is meeting regional reporting needs. The team should be comprised of members from the Transfer Payments Centre of Expertise and the Integrated Performance and Corporate Systems Directorate with input from regions and programs.

The use of regionally-developed systems and spreadsheets undermine the integrity of the data in the FNITP system and adds to operational workload. Below are ways in which it negatively impacted the Department:

Management decisions

A lack of relevant reports impaired management in making decisions about programs. Interviewees stated that regional management had formally requested new reports and enhancements to existing reports; however, we were informed that these requests were never processed. As described by interviewees, there came a point where so many enhancements were being requested that the helpdesk could not track them all. At some point, helpdesk staff disregarded all outstanding requests and began to only process new requests. As a result, many of the requests originally made were not implemented.

Duplication of effort

Significant effort is required to maintain multiple reporting systems used to track the same data. Interviewees explained that they have implemented regionally-developed systems and spreadsheets to overcome the reporting shortcomings of the FNITP system.

Availability of information

Many of the regionally-developed systems and spreadsheets are located on regional servers and in some cases on local hard drives. Having multiple regionally-developed systems and spreadsheets can be the cause of information not being available when required and increases the risk of information loss due to system failures.

 

 

8. Value Added Considerations

The following are considerations to take into account that may add value to the FNITP system:

 

 

9. Conclusion

The objective of the audit was to determine if the FNITP system supports an effective and efficient process to manage INAC's grants and contributions. The audit also assessed the degree to which the key program areas are documented for ongoing operations and whether the current FNITP production system has sufficient activity (workflow) controls, application controls, IT dependent manual controls and IT general controls to maintain data integrity, information privacy and information security.

The following root-causes of the performance issues were:

We conclude that there has been, and continues to be, a lack of sufficient training and support resources to maintain a Gs&Cs system that will effectively support funding to First Nations, Inuit, Métis and Northerners.

Furthermore, the lack of dedicated training and support resources to communicate the potential benefits of the system has resulted in under-utilization of the system by recipients.

In addition to the shortcomings related to resourcing, the lack of a policy framework and defined system ownership has also caused significant issues within the Department. The lack of a defined process to request, approve and implement system changes has resulted in changes being implemented without consultation or approval from the Transfer Payments Center of Expertise.

Summary

Progress was observed to suggest that the Department has the capability to maintain a national, integrated Gs&Cs system. However, audit findings are indicative of insufficient resource levels for program delivery and business operations which are inhibiting outreach and communication to recipients, in addition to causing departmental inefficiencies.

 

 

10. Management Action Plan

Recommendations Management Response / Actions Responsible Manager (Title) Planned Implementation Date
1. The CFO, in collaboration with the ADM s responsible for regional operations and staff, should develop and implement a program that ensures sufficient resources are allocated to providing training and support to recipient users. In conjunction with this training and support program, regional representatives across the country should champion the adoption of FNITP by their recipients. 1) The CIO is currently conducting an organizational review for the FNITP support model. The review includes the transfer of responsibilities for the support and training of employees and recipients from the CIO to Corporate Accounting and Material Management Branch (CAMMB). The transition plan will identify required resources (support and training) requirements, a proposed support model and the associated cost requirements.

The implementation of the model will require management approval before proceeding, i.e.: approval of model, funding and staffing.

2) Subsequent to the transfer of FNITP business functions from CIO to CAMMB:

  • Establish a national training working group to identify training gaps/requirements (both INAC and Recipient lens)
  • Development of an integrated Transfer payment training strategy that considers a phased implementation based on priority and risk needs.
  • Development and rollout of training materials, approaches (i.e.: blended learning, distance learning tools), national training calendar etc.
  • Pilot and QA materials and approaches
  • Deliver training and evaluate results
  • Implement ongoing ever greening of training plan and materials
Chief Information Officer (CIO) and Director General (DG), Corporate Accounting and Material Management Branch (CAMMB)

Chief Financial Officer (CFO) in collaboration with the ADM s responsible for regional operations and staff.

April 2011
2. The CFO should ensure that the Transfer Payments Centre of Expertise and the Integrated Performance and Corporate Systems Directorate finalize a policy framework and departmental directive to address issues related to system change management and ownership. Finalize development of directive for the submission and management of change requests based on ITIL methodology. Chief Information Officer (CIO) and Director General (DG), Corporate Accounting and Material Management Branch (CAMMB) December 2010
Policy framework and communications plan to be developed upon completion of transfer of FNITP business support model from CIO to CAMMB. April 2011
The communication plan will be developed with support from CFO branch communications officer. June 2012
Establish a governance structure with national representation for the implementation of policy framework based on a system critical perspective that focuses on People, Processes and Internal

Systems ownership and roles and responsibilities are addressed by means of an approved RACI matrix which forms part of action #1 above.

March 2011
3. The CFO should implement the requirement to finalize MOUs in the cases where more than one department or other levels of government are signing the funding agreements with First Nations. The MOUs should clearly indicate how notifications of payment withholdings will be communicated from departments or other levels of government providing funding to INAC and under what circumstances these payment withholdings will be executed. 1) CAMMB will coordinate with Planning and Resource Management Branch the review of the processes related to interdepartmental transfers to ensure MOUs are reflective of requirements as prescribed by the Treasury Board Policy on Interdepartmental Charging and Transfers Between Appropriations. More specifically, the review will focus on how notifications of payment withholdings should be executed.

2) An inventory will be taken of current signed MOUs and work with the Program Financial Management Advisors and the Program Managers of the MOUs to determine the best strategy to ensure compliance with the Policy on Interdepartmental Charging and Transfers Between Appropriations. Examples of strategies to include: amendments to MOUs where feasible and practical; written agreement between departments on notification mechanisms for payment withholdings and the requirement of Finance officers of both departments confirming before cheque issuance.

Director General (DG), Corporate Accounting and Material Management Branch (CAMMB) April 2011
4. The CFO should lead a cross functional team responsible for assessing regional processing requirements and establishing a procedure to incorporate them into FNITP. The team should be comprised of members from the Transfer Payments Centre of Expertise and the Integrated Performance and Corporate Systems Directorate with input from regions and programs. Conduct a fit / gap analysis which will identify the sectors/Regions business requirements.

Consultation and establishment of requirements prioritization and development of an implementation plan in a phased approach.

Chief Information Officer (CIO) and Director General (DG), Corporate Accounting and Material Management Branch (CAMMB) December 2011
5. The CFO should refine INAC's Financial Management Manual for Transfer Payments to include a clear definition of FAA sections 32, 34 and 33 controls. The definition should outline how the controls will be captured and represented in FNITP. System application controls should be modified to align with the refinements to the Financial Management Manual. Review and update the FNITP system controls support the Management Control Framework, the Financial Administration Act (Sections 32 and 34) based on identified deficiencies and/or gaps, and update the Financial management Manual for Transfer Payment to include a clear definition of FAA Sections 32, 33, 34 controls.

Develop a prioritization of change /enhancements required for FNITP. With implementation being based on availability of resources of the new support model.

Chief Information Officer (CIO) and Director General (DG), Corporate Accounting and Material Management Branch (CAMMB) December 2011
6. The CFO should lead a cross functional team responsible for determining whether FNITP is meeting regional reporting needs. The team should be comprised of members from the Transfer Payments Centre of Expertise and the Integrated Performance and Corporate Systems Directorate with input from regions and programs. Conduct a fit / gap analysis to identify the Sector/Regions' reporting requirements. The reporting requirements will be prioritized in consultation with the Sector and Regional stakeholders and taking into consideration the departmental initiative on Reduction of Reporting Burden. A plan will be developed to implement report development and/or enhancements in a phased approach, with ongoing review validation and updating. Chief Information Officer (CIO) and Director General (DG), Corporate Accounting and Material Management Branch (CAMMB) December 2011
 

 

Footnotes

  1. FNITP is also referred to as "the system" (return to source paragraph)
  2. INAC is also referred to as "the Department" (return to source paragraph)
  3. INAC is also referred to as the "Department" in this report. (return to source paragraph)
  4. Funding provided by INAC to a Tribal Council for distribution to its members. (return to source paragraph)
 
 
Date modified: