ARCHIVED - Evaluation of the Intervention Policy

Notice

This website will change as a result of the dissolution of Indigenous and Northern Affairs Canada, and the creation of Indigenous Services Canada and the eventual creation of Crown-Indigenous Relations and Northern Affairs Canada. During this transformation, you may also wish to consult the updated Indigenous and Northern Affairs home page.

Archived information

This Web page has been archived on the Web. Archived information is provided for reference, research or record keeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

Date: November 2010
Project Number: 09082

PDF Version (488 Kb, 101 Pages)

 

 

Table of Contents




List of Acronyms

AFOA
Aboriginal Financial Officers Association

AIDS
Acquired Immune Deficiency Syndrome

CFA
Comprehensive Funding Arrangement

CFNFA
Canada-First Nation Funding Arrangement

CFO
Chief Financial Officer

CM
Co-Managed Intervention

CWB
Community Well-Being Index

EPMRB
Evaluation, Performance Measurement and Review Branch

FASD
Fetal Alcohol Spectrum Disorders

FN
First Nation

FNFMB
First Nations Financial Management Board

FNITP
First Nations and Inuit Transfer Payment System

FSO
Funding Service Officer

GST
Goods and Services Tax

HC
Health Canada

HIV
Human Immunodeficiency Virus

HQ
Headquarters

HRSDC
Human Resource and Skills Development Canada

HST
Harmonized Sales Tax

INAC
Indian and Northern Affairs Canada

IP
Intervention Policy (2006)

IRS
Inuit Relations Secretariat

KI
Key Informant

MLG
Ministerial Loan Guarantee

NWT
Northwest Territories

OAG
Office of Auditor General

OSR
Own Source Revenue

P&ID
Professional and Institutional Development

PTP
Policy on Transfer Payments

RMP
Remedial Management Plan or Recipient-Managed Intervention

TB
Tuberculosis

TBS
Treasury Board Secretariat

TPM
Third Party Managed Intervention or Third Party Managers

Return to Table of Contents 






Note

Please note the expression "First Nations under Intervention Policy" or "First Nations under a specific level of intervention" applies to intervention in relation to the funding received by First Nations and organizations from INAC.

Return to Table of Contents 






Executive Summary

Introduction

Indian and Northern Affairs Canada's (INAC) current Intervention Policy was adopted in 2006 and is designed to ensure the ongoing delivery of programs and services, and to maintain accountability while defaults under funding agreements are addressed by recipients (i.e. First Nations, Aboriginal Organizations). Over the long term, application of the policy should lead to improved performance by recipients and a reduction in the number of First Nations that have seen their funding put under intervention policy[Note 1] and in the duration of such intervention.

There are four triggers for intervention: the terms and conditions of the funding agreement are not met by the recipient; the recipient's auditor gives a denial of opinion or an adverse opinion; the recipient has incurred a cumulative operating deficit equivalent to eight percent or more of its annual operating revenues; and the health, safety or welfare of First Nation members is being compromised. There are three main levels of intervention: recipient-managed, co-managed and third party managed.

The objectives of this evaluation are to assess the relevance, success, cost-effectiveness, design and delivery of the Intervention Policy, and to explore improvements to the policy and alternatives to it. At the time of the evaluation, INAC was working on a new Intervention Policy (tentatively named the Default Prevention and Management Policy). This new policy is seeking to be more aligned with INAC's new orientation and tools regarding capacity development with a focus on prevention and ongoing sustainability. This evaluation is also expected to inform the development of the new policy.

Methodology

Six lines of evidence informed the evaluation:

  1. A literature review.

  2. A document and file review.

  3. A data analysis of INAC's performance and financial data.

  4. Key informant interviews with INAC regional and Headquarter officials, other key federal departments and key people from other organizations; as well as a select number of First Nations and other Indian-administered organizations, third party managers, co-managers and funding services officers for the case studies. A total of 85 interviews were conducted with individuals or groups of individuals by phone, teleconference or during site visits for the case studies.

  5. Case studies of 24 First Nations and three other Indian-administered organizations. The case studies were designed to be broadly representative of the statistics on intervention as well as other variables, such as regions, remoteness, size and community well-being. They included successful and less successful recipients.

  6. Subject expert panel to review the evaluation approach and preliminary findings and conclusions.

The evaluation was conducted from January to June 2010 and included site visits to five regions, video conferences with two regions, and telephone interviews with two regions. The evaluation report was revised by the Evaluation, Performance Measurement and Review Branch (EPMRB) based on the findings of research conducted by the consulting firm Institute On Governance and EPMRB.

Evaluation Findings

The Intervention Policy Continues to be Relevant:
The evaluation demonstrates that the Intervention Policy remains relevant and there is a continued need to protect funding for programs and services, to ensure the delivery of essential services, and to ensure accountability for how federal monies are spent. There is also a continued need to build the capacity of recipients to manage and administer funds. The Intervention Policy is consistent with federal government and INAC plans and priorities, and the federal government's roles and responsibilities. INAC's Intervention Policy has been used as the basis for the intervention policies of Health Canada (HC) and the First Nations Financial Management Board (FNFMB); and INAC, HC and FNFMB are working together to coordinate changes to the policy.

The Intervention Policy Has Had Mixed Success in Achieving its Objectives:
There has been a modest reduction in the number of recipients under intervention, particularly in the last few years. There has also been a reduction in the number of recipients with a cumulative operating deficit of eight percent or higher. The Intervention Policy has been successful in ensuring the ongoing delivery of essential services, maintaining the health and safety of band members, and ensuring the external accountability of recipients to INAC.

On the other hand, the level of intervention has not decreased – in fact, an increasing proportion of First Nations are under co-management and a decreasing proportion of First Nations are under recipient managed intervention over the past ten years. Also, a number of First Nations that have been under some form of intervention policy for a long period of time – for example, 42 percent of First Nations under some form of intervention as of March 31, 2010, had been under intervention for ten or more years.

Any changes in the incidence, level and duration of intervention cannot necessarily be attributed solely to the implementation of the policy since other factors, such as access to own source revenue, economic performance in certain sectors, or the supply of credit can also have an impact. The major deficiency under the policy identified by the evaluation is capacity building.

A Number of Factors Affect the Success of the Intervention Policy:
There are a number of underlying factors related to governance, management, community engagement and funding practices that affect the success of the Intervention Policy. There are also a number of contextual factors related to community endowments (natural resources, geographic location, economic opportunities and access to own source revenue, the strength of the private sector, population size and growth); community member endowments (education, health, cultural identity); and social cohesion and security that could affect the success of the policy. Many of these factors are outside of the control of the Intervention Policy.

The key success factors identified by the evaluation are: willing and committed leadership in recipients, stable and competent management, community engagement, positive external relationships, and access to significant own source revenue.

The Intervention Policy is not Cost-Effective:
Considerable time and effort is required from INAC and recipients to implement the Intervention Policy. The cost of co-managers and third party managers affects the availability of band support funding for governance and administration in recipients. Third party managers are not able to use surpluses to pay off debt. Any improvements in the results of intervention as well as other interventions outside of the policy to address willingness, capacity and low own source revenue should increase cost-effectiveness.

The Intervention Policy is not Well Designed to Achieve all of its Objectives:
The major gaps in the design of the Intervention Policy identified by the evaluation are: a capacity development framework, prevention strategies, and common assessment tools. Some key informants suggested that INAC provide additional funding for co-managers and third party managers in order to increase the willingness of recipients, enhance recipients' capacity, and improve INAC's ability to monitor. Some key informants also suggested that alternative debt retirement tools are needed for recipients with little access to own source revenue.

INAC's Intervention Policy is Similar to Other Intervention Approaches:
INAC's Intervention Policy is similar to provincial government approaches to their subnational entities (municipalities, school boards and hospitals) and to other countries' approach to Aboriginal recipients in terms of objectives, triggers, levels of intervention and remedies, and oversight and monitoring. The key differences are that provincial governments are better able to control the debt of their subnational entities under a legislative framework, and that some of the other approaches provide more capacity building support through a broader range of capacity building mechanisms.

The Intervention Policy is Being Implemented as Planned, With Some Gaps in Monitoring and Support:
The Intervention Policy provides a common framework for the objectives, triggers, assessment, and levels of intervention, and INAC's regional offices are adhering to the required procedures. There are, however, variations within and between regions, primarily in terms of the capacity of recipients and INAC funding service officers. Key informants suggested that capacity be strengthened with training and other tools related to the Intervention Policy. Other suggested improvements are: increased monitoring by INAC of the performance of co-managers and third party managers; changes to the information collected in First Nations and Inuit Transfer Payment (FNITP); and more formalized processes for communication and sharing of information with other government departments.

Suggested Improvements and Alternatives

The main suggested improvements to the Intervention Policy are:

Other alternatives were suggested to deal with issues that are outside of the influence of the Intervention Policy but have an effect on its success. These issues include: increased leadership stability and community engagement, improved funding practices or a modified funding regime, and alternative debt management approaches.

 It is recommended that INAC:

  1. Implement prevention and early detection strategies to prevent First Nations going into intervention status or escalating to a more serious level of intervention. Activities should include:

    1. Better identification of financial and governance capacity gaps and needs (linked to the general assessment);

    2. Better and broader identification of triggers for third party management and co-management;

    3. Identification of incentives for third party managers to build First Nations capacity, to be written into agreements with third party managers;

    4. Development of questions to assess properly key success factors to analyze trends in relation to escalation and de-escalation (consider undertaking community surveys to better assess community capacity factors for success); and

    5. Improve communication and coordination at the national and regional level with other federal government who deliver services and programs to First Nations.

  2. It is recommended that:

    1. Third party managers be prequalified and assessed against performance criteria;

    2. INAC audit co-management and third party management arrangement on a risk basis; and

    3. Revise the third party management agreement to request participation of third party managers in evaluations as well as in audits.

  3. The proposed new Default Prevention and Management Policy should address the design and delivery gaps of the current Intervention policy by:

    1. Implementing national tools and formalized processes in the assessment of First Nations;

    2. Clarifying FSO roles and responsibilities, developing job descriptions that identify competencies and knowledge needed, and identifying training to meet requirements; and

    3. Clearly communicate the new policy and assessment processes to stakeholders and First Nations.

  4. Develop and implement better monitoring and reporting systems that involve:

    1. A performance measurement strategy to allow for meaningful reporting through quarterly progress reports and the Departmental Performance Report. Indicators such as duration, incidence and level of intervention, and level of implementation of Remedial Management Plan could be considered;

    2. A cost tracking system at the Headquarters and regional level to capture cost data in order to measure cost effectiveness and inform future decisions; and

    3. Re-design the intervention policy module in FNITP that will involve a revision of the inputs, processing and output (reports) of the system at the regional and national level in order to make it more user-friendly, reliable and to inform performance measurement strategy to be developed in line with the new policy.
Return to Table of Contents 






Management Response / Action Plan

Project Title: Evaluation of the Intervention Policy
Project #: 09082

RecommendationsActionsResponsible Manager (Title / Sector)Planned Implementation and Completion Dates
1. Implement prevention and early detection strategies to prevent First Nations going into intervention status or escalating to a more serious level of intervention. Activities should include:

a. Better identification of financial and governance capacity gaps and needs (linked to the General Assessment);
a. The roll out of the General Assessment (GA) tool got underway in October 2010 and will be fully implemented by April 1st, 2011. The GA will provide information as to the risk level of recipients and will also serve to identify recipients' capacity gaps. The GA will be used as a source of information to prevent and address defaults under a new Default Prevention Management Policy. CFO/RO - Director of Operations and Implementation and Senior Director of Transfer Payments Centre of Expertise From April, 2011
Regional Operations Sector is currently in the process of developing two more effective approaches to capacity development programming within INAC and will develop strategies to better respond to gaps identified in the GA. RO - Director General of Governance Branch and Director of Sustainable Communities 2011-12
b. Better and broader identification of triggers for third party management and co-management; b. The new Default Assessment Tool will add consistency to the decision to appoint a Third-Party Funding Agreement Manager (TPFAM). This tool will be based on concrete evidence provided from a number of sources such as field visits, the GA and other available information. CFO - Senior Director of Transfer Payments Centre of Expertise 2011-12
c. Identification of incentives for third party managers to build First Nations capacity, to be written into agreements with third party managers; c. It is often found that the relationship between the TPFAM and the recipient is not amenable to building trust and capacity. As a result, incentives are not written into the funding agreements. However, upon re-opening the MERX process identifying a new set of TPFAMs, the need to include incentives and milestones for capacity development will be evaluated. CFO - Senior Director of Transfer Payments Centre of Expertise 2011-12
d. Development of questions to assess properly key success factors to analyze trends in relation to escalation and de-escalation (consider undertaking community surveys to better assess community capacity factors for success); and d. The Policy on Transfer Payments identified key capacity requirements for success (governance, organizational capacity, mature processes / procedures, accountability mechanisms, and financial health). The Department is developing assessment tools to examine recent strengths and weaknesses related to these capacity requirements. The General Assessment and Readiness Assessment are key tools under implementation and development now. The processes and structures around these tools involve purposeful, focused community engagement. In addition, the Department is examining options for more comprehensive approaches to supporting community development and community planning, which will further strengthen our understanding of community key success factors.

The results of the GA will be reviewed annually to monitor escalation or de-escalation of all communities/recipients in various areas and key factors to generate escalation will be explored.
CFO – Senior Director of Transfer Payments Centre of Expertise, RO – DG OPS Branch and DG Governance Branch. 2011-12
e. Improve communication and coordination at the national and regional level with other federal government who deliver services and programs to First Nations. e. Work with DOJ and other federal departments to facilitate information sharing between federal departments where there is a common recipient e.g. working with Health Canada on a protocol and common agreement clauses and look for opportunities for harmonization in regards to community development.

INAC also co-ordinates an ADM Network on Aboriginal Affairs that meets regularly to improve communication and coordination. Regional managers participate in Federal Regional Councils.
RO / CFO - Director of Operations and Implementation and Senior Director of Transfer Payments Centre of Expertise Ongoing
2. It is recommended that:

a. Third party managers be prequalified and assessed against performance criteria;
a. The current third party managers' prequalified list was established in FY 2009-10 by means of a MERX competitive process. The list as well as Framework Agreements with third party managers are valid for a period of three years and can be amended for a period up to five years. CFO - Senior Director of Transfer Payments Centre of Expertise Completed
The current Intervention policy requires that a performance review be conducted to assess the performance of the third party manager in meeting the requirements of the third party management agreement. CFO is currently reviewing the Policy and will ensure performance review requirements continue to be addressed in the new Default Prevention Management Policy by means of improved monitoring tools. 2011-12
b. INAC audit co-management and third party management arrangement on a risk basis; and b. CFO will ensure that its revised directive on third party management requires sectors to perform audits of co-management and third party management agreements on a risk basis. CFO - Senior Director of Transfer Payments Centre of Expertise 2011-12
c. Revise the third party management agreement to request participation of third party managers in evaluations as well as in audits. c. The current third party framework agreements already include an audit clause that requires third party managers to provide all necessary assistance to auditors. CFO will ensure that new framework agreements include a requirement for participation of third party managers in evaluation as well. CFO - Senior Director of Transfer Payments Centre of Expertise Completed
3. The proposed new Default Prevention and Management Policy should address the design and delivery gaps of the current Intervention policy by:

a. Implementing national tools and formalized processes in the assessment of First Nations;
a. Policies/directives for default management and the GA as well as tools supporting consistency and implementation will be formalized through a national process in First Nation Inuit Transfer Payment. RO / CFO - Director of Operations and Implementation and Senior Director of Transfer Payments Centre of Expertise April 2011
b. Clarifying FSO roles and responsibilities, developing job descriptions that identify competencies and knowledge needed, and identifying training to meet requirements; and b. Roles and responsibilities will be clearly identified in the policy/directives document. CFO - Senior Director of Transfer Payments Centre of Expertise April 2011
FSO roles are currently being reviewed to identify requirements for strategic alignment with core functions. A report has already been tabled and is currently being considered in term of realigning resources to operational priorities. RO - Deputy Minister's Special Representative on Reduced Reporting April 2011
c. Clearly communicate the new policy and assessment processes to stakeholders and First Nations. c. Engagement sessions on the new Default Prevention and Management Policy were delivered over summer 2010. Policy-directives document will be reviewed based on feedback from those sessions. Training-info sessions will be provided to INAC staff before and after April 2011 implementation. INAC will liaise with AFOA in order to update Aboriginal training material provided by AFOA to Aboriginal recipients. CFO - Senior Director of Transfer Payments Centre of Expertise July 2011
4. Develop and implement better monitoring and reporting systems that involve:

a. A performance measurement strategy to allow for meaningful reporting through quarterly progress reports and the Departmental Performance Report. Indicators such as duration, incidence and level of intervention, and level of implementation of Remedial Management Plan could be considered;
a. CFO will report quarterly to the Financial Management Committee Senior Management meeting, on the intervention status, based on clear indicators. National oversight will also be managed annually by Operations Committee to review all communities with default management activity. RO / CFO - Director of Operations and Implementation and Senior Director of Transfer Payments Centre of Expertise April 2011
b. A cost tracking system at the headquarters and regional level to capture cost data in order to measure cost effectiveness and inform future decisions; and b. Cost of 3rd party managers will be tracked in FNITP and oversight reports will be regularly produced. CIO / CFO - CIO, Manager IPCSD and Senior Director of Transfer Payments Centre of Expertise 2011-12
A one time assessment of costs other than 3rd party management will be performed after two years of application of the new policy to measure the cost effectiveness. RO / CFO - Director of Operations and Implementation and Senior Director of Transfer Payments Centre of Expertise 2013-14
c. Re-design the intervention policy module in FNITP that will involve a revision of the inputs, processing and output (reports) of the system at the regional and national level in order to make it more user-friendly, reliable and to inform performance measurement strategy to be developed in line with the new policy. c. The FNITP intervention module will be redesigned to align with the new Default Prevention and Management Policy and its underlying directives. CIO / CFO - CIO, Manager IPCSD and Senior Director of Transfer Payments Centre of Expertise 2011-12

The Management Response / Action Plan for the Evaluation of the Intervention Policy were approved by the Evaluation, Performance Measurement and Review Committee on November 18, 2010.

Return to Table of Contents 






1. Introduction

1.1 Overview

The purpose of the evaluation is to assess the relevance, design, delivery, success and cost-effectiveness of Indian and Northern Affairs Canada's (INAC) Intervention Policy and to look at best practices, improvements and possible alternatives. The evaluation report provides information on the policy, the methodology that was used in the evaluation, the findings and conclusions for each of the evaluation questions, and overall conclusions and recommendations.

1.2 Policy Profile

1.2.1 Background

INAC's Intervention Policy is an internal financial management policy that has evolved over time to deal with various issues such as indebtedness, remedial management plans, and third party management. The current policy was developed partly in response to a 2003 audit by the Auditor General of Canada of the implementation of third party management; and partly, as a result of a comprehensive legal review to reduce Crown exposure to liabilities.

The policy was introduced in December 2006 and came into effect on April 1, 2007. It is issued under the authority of INAC's Chief Financial Officer. It consolidates all previously existing intervention practices under a single policy.

At the same time as the Intervention Policy was adopted, a Companion Initiative to develop the capacity of recipients under Intervention Policy was introduced. The objective of the Companion Initiative was "to support short term, high impact initiatives that will make a measurable difference."[Note 2] It was funded for 2006/07 and 2007/08 under the authority of the Indian and Inuit Management Development transfer payment authority after which time it was to be included in a renewed authority.[Note 3]

At the time of the evaluation, INAC was working on a new Intervention Policy (tentatively named the Default Prevention and Management Policy) as some elements were considered not addressed by the 2006 Intervention Policy. This new policy is seeking to be more aligned with INAC's new orientation and tools regarding capacity development with a focus on prevention and ongoing sustainability. To assist the Department in the development of this new policy, the Department seeking for the advice of experts in financial management, including advice from the First Nations Financial Management Board, the Aboriginal Financial Officers Association and the Assembly of First Nations through consultation and engagement session. Engagement sessions with stakeholders and regional staff will be held over the summer 2010 with the goal of having a new policy in place for the beginning of fiscal year 2011-2012. This evaluation is expected to inform the development of the new policy.

1.2.2 Policy Objectives and Expected Outcomes

The purpose of the Intervention Policy is to permit the delivery of programs and services and maintain accountability while problem situations are being addressed; and to put the onus on the recipient to correct the problem situations. The policy is also designed to support timely intervention and consistency in regional operations; to facilitate ongoing monitoring of intervention; and to improve the effectiveness of intervention. The aim of the policy is to encourage the recipient in default to enhance their capacity to provide programs and services, and to provide for an exit strategy where a lesser or no form of intervention is required.[Note 5]

1.2.3 Policy Framework

The Intervention Policy sets out a framework for intervention by the Minister in the event that a recipient defaults under the terms and conditions of a funding arrangement. It applies to all funding arrangements except legislated self-government agreements. It includes the triggers for intervention, the levels of intervention, the process, roles and responsibilities, performance review, accountability framework (for third party managers), and exit strategy.

There are four triggers for intervention for use with First Nations and tribal councils:

The Intervention Policy does not apply to election disputes and allegations and complaints except where an election dispute or concern raised gives rise to a default. If the recipient is a corporation, bankruptcy, insolvency or liquidation can also lead to intervention.

According to the policy, in the event of a default, INAC notifies the recipient, undertakes an assessment of the recipient's capacity and willingness to address the default, and determines the level of intervention required. There are three main levels of intervention:

  1. Recipient-managed level – a low level intervention when the recipient is willing and has the capacity to address the default based on a Remedial Management Plan (RMP).

  2. Co-managed level – a moderate level intervention when the recipient is willing but lacks the capacity to address the default. The Council of the First Nation or Tribal Council is required to enter into a co-management agreement with a co-manager in addition to developing or amending and implementing a RMP.

  3. Third party managed level – a high level intervention when INAC determines that there is a high risk to the funding provided under a funding arrangement or to the provision of programs and services, or that the council is unwilling to address the default. INAC then appoints a third party manager to administer its funding. Third party management is also used in a situation where no funding arrangement has been signed by the council.

INAC may also withhold funds, terminate the funding agreement, take any other reasonable action, or require any other reasonable action to be taken by the recipient.

According to the policy, RMPs are developed and implemented by the recipient. They are approved by the Council of the First Nation or tribal council (or the appropriate authority for other entities) and by the Regional Director of Funding Services, and are attached to the funding arrangement. RMPs should include the following elements:

At the co-management level, the council selects and contracts a co-manager. The role of a co-manager is to assist the council in remedying the default, developing sound management practices, restoring financial health, and building capacity. The co-manager is to be given financial signing authority for all accounts containing INAC funding. The council pays for the co-manager's services, usually from its Band Support Grant.

At the third party management level, INAC selects the third party managers from a pre-qualified list of individuals or firms and negotiates a third party agreement (see Section 7.1). All INAC funding, or funding for a particular program or service, is redirected to the third party manager to administer. INAC may also require that the third party manager assist the council in enhancing capacity to administer funding in order to remediate the default that gave rise to the intervention. The third party manager is required to maintain a system of accountability to the community in terms of the programs and services that are funded under the third party management agreement. The third party manager's services are normally paid from the INAC's Band Support Grant.

Councils are responsible for their credit transactions and any debt they may have incurred. The Crown is not obligated to pay or accept responsibility for any current or future debt. Employees remain the responsibility of the council but the third party manager determines which of the council's operational and administrative staff are necessary for the continuation of programs and services, and pays the salaries and benefits of those staff from INAC funding.

According to the policy, INAC monitors the progress of the intervention at least quarterly. Where possible, the council is included in the review of the performance of the third party manager. Based on the monitoring, INAC may decide to escalate or de-escalate the level of intervention or terminate it. In the event that co-management or third party management is being de-escalated or terminated, the council is required to develop an administrative transition plan for the transfer of full authority back to the council.

1.2.4 Policy Management

The Chief Financial Officer Sector manages the Intervention Policy and the regions implement the policy. According to the policy, INAC Headquarters is responsible for:

INAC regional offices are responsible for:

1.2.5 Policy Resources

The Intervention Policy is issued under the authority of the Chief Financial Officer (CFO) and no specific funds are currently targeted to this policy. Funding in support of the development of the policy is included in the CFO's A-base budget. Funding for regional implementation of the policy is included in regional A-base budgets. All fees related to services provided by a co-manager or a third party manager are the responsibility of the council and are normally drawn from the Band Support Grant provided to the council by INAC.

The Companion Initiative on Capacity Development was funded in fiscal years 2006-2007 (starting January 2007) and 2007-2008 using Budget 2006 funds authorized by the Treasury Board. Treasury Board approved $10.6 million. Seventy-five percent of the funding was allocated to the regions on the basis of the average regional distribution of required interventions over the past three years, and the remaining 25 percent of funding was allocated on the basis of regional population. Other funds (about $500,000) were available for tribal councils and other institutions, such as regional Aboriginal Financial Official Officers Association (AFOA) Chapters to provide support.

Return to Table of Contents 






2. Evaluation Methodology

2.1 Evaluation Objectives

In accordance with the Treasury Board Secretariat (TBS) Evaluation Policy, the evaluation examines issues related to relevance, success and performance. Design and delivery issues are also assessed to guide and improve the policy and performance measurement efforts. Best practices, improvements and alternatives are also explored. Terms of Reference were approved by INAC's Evaluation, Performance Measurement and Review Committee on September 24, 2009.

2.2 Evaluation Issues and Questions

The following broad evaluation issues guide the evaluation:

Evaluation of relevance looks at the extent to which the Intervention Policy addresses existing needs and priorities; is consistent with federal priorities, roles and responsibilities; and does not duplicate or overlap with the policies of other stakeholders.

Evaluation of success looks at whether the Intervention Policy objectives are being achieved and what factors have facilitated or hindered the achievement of the policy objectives. It also looks at the underlying causes and key success factors, as well as the unintended, positive or negative, impacts of the policy.

Measuring cost-effectiveness of the Intervention Policy involves identifying the cost of intervention in relation to the outcomes and comparing it to the cost of other intervention approaches (of other First Nation and tribal council funders in Canada, of provincial governments, and of other countries).

Evaluation of the design of the Intervention Policy looks at whether the policy is well designed to achieve its objectives and how it compares to the design of other intervention approaches.

Evaluation of the delivery of the Intervention Policy complements the work already done by the Audit of the Application of the Intervention Policy (2009) by looking at whether the policy is being implemented as planned, whether good guidance and monitoring is in place, what constraints impede implementation, and what are the best practices.

Finally, the evaluation looks at the alternatives to the Intervention Policy to achieve the same results or to improve the results. In particular, the evaluation examines other models of intervention used by other funders of First Nations and tribal councils in Canada, by provincial governments, or by other countries.

The evaluation questions related to each of these issues can be found in Appendix B, Revised Evaluation Matrix.

2.3 Evaluation Scope and Timing

The evaluation examines Intervention Policy activities undertaken since its adoption in December 2006 and up to March 31, 2010. Because the Intervention Policy has never been evaluated, however, historical information over the past ten years or more have been looked, drawing from INAC's financial information systems, previous audit data and key informants. This helps us to place recent activities into an historical context and identify any overall trends.

INAC is in the process of reviewing and revising the 2006 Intervention Policy. While the evaluation team received documentation related to the revised Intervention Policy, it has not been included in the analysis because it has not been finalized. Our focus is therefore on the current Intervention Policy.

The relevance, design, delivery, success and cost-effectiveness of the Intervention Policy were assessed through qualitative and quantitative data; and triangulate the information across a number of sources. In order to assess what may have affected the achievement of the outcomes of the policy and make recommendations for changes or alternatives, possible underlying causes, preventive measures, and exit strategies were explored.

The evaluation takes into consideration previous audits, other evaluations, reports on the Intervention Policy and the Companion Initiative, and other relevant documents. It also takes into account other related INAC policies on community development, transfer payments, risk assessment, capacity building, and funding arrangements.

The perspectives of various INAC officials as well as the recipients in the evaluation were solicited. Information from the Chief Financial Officer Sector and the Regional Operations Sector, as well as from all INAC regions, except Nunavut, were collected.[Note 6] Information from a range of First Nations, tribal councils and other Aboriginal organizations was also collected. Finally, the intervention approaches of other key federal departments, other jurisdictions, and other countries, as well as the perspectives of subject matter experts who have a knowledge and interest in the evaluation issues were examined.

The evaluation was conducted from January to June 2010. It was preceded by a pre-consultation phase conducted by the Evaluation, Performance Measurement and Review Branch (EPMRB), which led to the development of a draft Evaluation Methodology Report and draft Logic Model. From January to March 2010, the Institute On Governance (IOG) finalized the Evaluation Methodology Report and developed the data collection instruments; drafted the Document Review; and conducted the Literature Review. From April to June 2010, IOG made site visits with the assistance of EPMRB to five of the regions (Atlantic, Quebec, Ontario, Manitoba and Saskatchewan) and conducted the case studies and interviews. EPMRB also compiled and analyzed data during this period. Two subject expert panel meetings were held by IOG with EPMRB participation – one at the end of the planning phase and one at the end of the data collection phase. Preliminary findings and conclusions were presented by IOG to INAC senior management at the end of May 2010. EPMRB developed the recommendation based on the report submitted.

2.4 Evaluation Methodology

2.4.1 Development of a Framework Logic Model

EPMRB developed a draft Framework Logic Model for the Intervention Policy based on the purpose, aims and objectives of the policy; pre-consultations with key informants in INAC; and INAC's Strategic Outcomes (Appendix C, Draft Intervention Policy Framework Logic Model). The draft Logic Model was used by EPMRB to help clarify the evaluation issues and the short, medium and long-term outcomes.

According to the draft Logic Model, INAC carries out a number of activities in order to ensure that services are delivered and accountability is maintained, remedial measures are implemented, and recipient capacity is strengthened. In the long term, this should lead to improved performance and ultimately the de-escalation of intervention and earlier exit from intervention; a reduction in the number of First Nations that have seen under intervention and the duration of such intervention; and effective and responsibly managed and operated programs and services funded by INAC. The evaluation questions and the evaluation methodology were designed to measure the achievement of these outcomes.

2.4.2 Data Sources

The evaluation methodology incorporates six lines of evidence: a literature review, a document and file review, data analysis, key informant interviews, case studies, and a subject expert panel. The subsequent sections provide a summary of each of these lines of evidence. The Revised Evaluation Matrix (Appendix B) indicates which line of evidence was used to answer each of the evaluation questions.

1. Literature Review

The purpose of the literature review was to define and explore the key issues related to intervention by one level of government of another level of government, or by a funder to a recipient organization. The literature review has informed the findings and conclusions in this evaluation report, particularly related to design, delivery and effectiveness, as well as potential alternatives.

The literature review includes a brief survey of public policy and administration and social science, academic journal databases using key words like intervention, default, debt management, remedial management, third party management, supervision, trusteeship ("mettre en tutelle" in French) and remediation. As a result of the survey, three key lines of inquiry emerged as being most relevant to INAC's Intervention Policy – systemic intervention, intervention in distressed states and communities, and intervention in subnational entities.

Eight Canadian and international intervention examples were initially selected for further investigation, and another example of an alternative approach was identified during the data collection phase. The examples were used to illustrate the various approaches that have been taken to intervention and include:

  • First Nations and Inuit Health Branch of Health Canada;

  • First Nations Financial Management Board;

  • the Government of British Columbia and its municipalities;

  • the Government of Ontario and its school boards;

  • the Government of Ontario and its public hospitals;

  • the U.S. Federal Government and American Indian Tribal Governments;

  • the Commonwealth of Australia Government and Australian indigenous corporations;

  • the Queensland State Government and indigenous councils in Queensland, Australia; and

  • an alternative approach involving a number of federal and territorial government departments in an isolated community in Nunavut.

IOG gathered information on the other intervention approaches from government publications and other reports, and interviewed key informants for six of the examples.

The literature review concludes with a summary of the issues, lessons learned and potential alternatives.

2. Document and File Review

EPMRB identified relevant files and documents during the preliminary consultation phase of the evaluation and IOG supplemented them during the data collection phase. These documents included audits, evaluations, and policy documents on the Intervention Policy and related policies. IOG also conducted a Google search of media reporting on INAC's intervention over the past five years.

EPMRB asked INAC's regional offices to provide samples of RMPs for five recipients under remedial management and five recipients under co-management or third party management and were reviewed by IOG. All regions except the Northwest Territories (NWT) region provided a sample that covered a total of 63 First Nations, two Indian-administered organizations, two co-managers, and two third party managers. A summary of the sample of RMPs is provided in Appendix D.

The document and file review provided a background briefing for the evaluation team prior to the data collection phase and informed the findings for a number of the evaluation issues and questions.

3. Data Analysis

In collaboration with EPMRB, IOG conducted an analysis of INAC's performance and financial data related to the Intervention Policy from 1999/2000 to 2009/2010. EPMRB compiled a database for analysis, drawing from existing INAC management systems, financial systems and other relevant databases. EPMRB also cleaned the data and validated the data on intervention over the past 11 years with INAC's regional offices. PASW Statistics 18[Note 7] software was used by EPMRB for the analysis and EPMRB worked with INAC's Research and Analysis Directorate to integrate and analyze data on the Community Well-being (CWB) Index.

The databases and sources of information used in the analysis were:

  • First Nations and Inuit Transfer Payment System for financial information and audit opinion;

  • Regional offices for intervention level data validation and third party managers (TPM) costs;

  • Band Governance – Regional Operations for election system data;

  • Research and Analysis Directorate for registered population data and the CWB Index; and

  • Band Classification Manual for geographic zone data.

The purpose of the data analysis was to provide a description of the application of the intervention policy and to assess quantitative information on its underlying causes or impacts. This includes indicators and trends at a national and regional level related to: the incidence of intervention, the level of intervention, the duration, and the triggers. It also includes variables that may be related to underlying causes including: size, population, remoteness and the CWB Index. Where possible, EPMRB correlated the data on intervention with the data on underlying causes and interpreted the results.

Because the total number of Indian-administered organizations under intervention in any year is very small in comparison to the total number of First Nations under intervention, the data analysis for these organizations was limited to the incidence and level of intervention.

Appendix E – Summary of Data Analysis provides a summary of what was originally planned and what was actually achieved in terms of the data analysis. Appendix F provides the type of analysis that was conducted. We have incorporated relevant results into this Evaluation Report.

4. Key Informant Interviews

IOG conducted a number of interviews with INAC regional office and Headquarter managers, other key federal departments, and key informants from other organizations. We also interviewed a select number of First Nations and other Indian-administered organizations, third party managers, co-managers and funding service officers as part of the case studies. A total of 122 interviews with individuals or groups of individuals were planned, and a total of 82 interviews or 67 percent were actually conducted (Appendix G - Summary of Key Interviews). The interviews were conducted from April to June 2010.

The interviews were based on interview guides linked to the evaluation issues and questions. They were conducted in person, by phone or by teleconference. The results were written up and provided in the Key Informant Interviews Technical Report and incorporated into this evaluation report.

5. Case Studies

IOG conducted a total of 27 case studies covering all regions except the North. The cases include 24 First Nations and three other Indian-administered organizations. IOG and EPMRB selected the case studies to be broadly representative of the statistics on intervention over the past ten years, as well as other variables such as remoteness, size and community well-being. We consulted with INAC's regional offices about the cases selected and made a few changes due to limitations in accessing recipients or to INAC's funding.

The First Nation case studies cover:

  • Three Success Cases - First Nations that have never been under intervention and yet, appear to be disadvantaged in terms of their isolation, low own source revenue, or community well-being.

  • Seven Other Success Cases - First Nations that were under intervention for more than five years but managed to exit and have stayed out for two or more years.

  • Five Chronic Recipient-Managed Cases – First Nations that have been under recipient-managed intervention for five or more years.

  • Nine Chronic Co-Managed or Third Party Managed Cases – First Nations that have been under co-management and/or third party management for five or more years.

The other Indian administered organizations include a tribal council, a child and family services agency, and an education authority. A breakdown of the cases by region is provided in Appendix H - Summary of Case Study Sample.

The case studies addressed a number of the evaluation issues and questions, particularly underlying causes, delivery, and success. In particular, the evaluation team used the First Nation success and other success cases to analyze key success factors; the First Nation chronic recipient-managed and chronic co-managed and third party managed cases to analyze constraints to implementation; and the other Indian-administered organizations cases to analyze how the situation in a First Nation affects its service delivery organization or vice versa.

The case studies were conducted through a documentation review; a site visit where time and resources permitted; and interviews with the recipient, the relevant INAC funding service officer, and co-managers or third party managers, if applicable and where possible. EPMRB sent out an email or letter to the Chief and council, band manager, and co-manager or third party manager for each of the First Nations, and to the Executive Director for each of the other Indian-administered organizations, inviting them to participate in the evaluation. The IOG followed up through email and telephone with all of the contact persons in order to set up a site visit or a telephone interview. The contact person identified other relevant people to include in the interviews. As indicated in Section 2.4.3 Limitations, the evaluation team was only able to conduct interviews with about half of the recipients, three co-managers and one third party manager despite repeated follow-up. Appendix H indicates which cases included interviews with recipients.

6. Subject Expert Panel

IOG convened a subject expert panel of four members and facilitated two half-day workshops. Panel members were individuals with senior experience in INAC and Health Canada, in financial management, governance, and capacity building. IOG also invited members of the advisory committee to attend the workshops.

IOG held one workshop at the outset of the evaluation to review the key lines of inquiry and possible alternatives; and one workshop at the end of the data collection phase to review preliminary findings and conclusions. EPMRB participated to both workshops.

2.4.3 Limitations and Constraints

During the course of the evaluation, the evaluation team faced a number of constraints that limit the reliability of the data and the interpretation of the results. These limitations are presented below in terms of what the issue was, what efforts were taken to mitigate the impact, and how the issue ultimately affected the evaluation. Overall, the evaluation team attempted to address the limitations by using information from all of the data sources; complementing quantitative data with qualitative data; and incorporating different perspectives from the range of key informants and the subject expert panel. We have also qualified the findings where there are issues of data reliability, potential bias, and attribution of results to the Intervention Policy.

Limitation #1: No validated logic model for the Intervention Policy

As indicated previously, there was no logic model developed for the Intervention Policy at the time that it was adopted that outlined the link between the activities, outputs and outcomes.

Mitigation:

EPMRB developed a draft framework logic model based on components of the policy, pre-consultations with key INAC informants, and INAC's Program Activity Architecture. The draft logic model was used to detail the evaluation issues, to guide the data collected, and to assess the success and cost-effectiveness of the Intervention Policy. This logic model has, however, not been finalized.

Impact on Evaluation:

The emphasis in the evaluation and the interpretation of the results could change if there are changes in the logic model, particularly in terms of the long-term outcomes.

Limitation #2: Availability and reliability of data and data analysis

Mitigation:

Impact on Evaluation:

While the evaluation includes an accurate profile of intervention and a statistical analysis of some of the contextual factors that may affect the level or duration of intervention, other statistical analyses of underlying causes were not possible and have not been included. The analysis of cost-effectiveness is also limited by the lack of comprehensive quantitative information on costs.

Limitation #3: Potential Bias in Case Studies

In terms of the key informant interviews and case studies, about 60 percent of the recipients were interviewed despite repeated follow up by telephone and email with all of the contact persons. In terms of success cases – First Nations that have never been under any form of intervention – and in terms of co-managers and third party managers, it has been the least successful (33 percent of the sample). Interviews with First Nations that have been under co-management and/or third party management for more than five years were the most successful (77 percent of the sample). Three Indian-administered organizations were also interviewed.

In some cases, interviews were scheduled but the interviewees were no longer available at the designated time. There was not sufficient time to select alternate cases to replace those cases where recipient interviews could not be conducted. Funding Services Officers were, however, interviewed for all of the cases. There is, therefore, a potential for bias in the case studies towards INAC's perspective rather than recipients' perspective.

Mitigation:

The recipient interviews have been supplemented by the perspectives of national Aboriginal organizations, by the document review, and by the literature review.

Impact on Evaluation:

The evaluation may not reflect the full range of perspectives among recipients and among co-managers and third party managers and may be biased towards INAC's perspective. Where recipient views are referenced, they may emphasize views about co-management and third party management more than views about recipient-managed intervention.

Limitation #4: Availability of historical information

Another limitation in the case studies was the availability of historical information. Since many of the cases covered a decade or more of intervention, there was very little documentation from the earlier stages of implementation. In addition, remedial management plans for the cases were often not up to date (refer to Section 7, Evaluation Findings - Delivery). In a few cases, the recipient contact persons and the Funding Service Officer were relatively new and did not have a historical perspective.

 Mitigation:

The case studies were written up on the basis of the information available or received. For a few of the interviews, previous chiefs or councillors or funding service officers who would have a historical perspective were tentatively identified.

Impact on Evaluation:

The case studies vary in terms of the depth of the historical analysis.

Limitation #5: Availability of documentation

The only constraint in terms of the document review was that the NWT did not provide any remedial management plans for the RMP analysis.

 Mitigation:

An interview was conducted with regional officials in the NWT to obtain information on the implementation of the intervention policy in that region.

 Impact on Evaluation:

The NWT region is unique within INAC because only three of the 26 First Nation bands in the territory have a land base; essential services are funded by INAC through the territorial government and not directly to bands; and the regional office does not report to Regional Operations Sector at INAC HQ but rather the Northern Affairs Sector. The lack of RMPs for the NWT is therefore not seen as a major constraint in terms of the evaluation findings on RMPs.

Limitation #5: Analysis of impact and attribution to the Intervention Policy

As indicated in the findings on Success (Section 4), any positive changes in the incidence, level and duration of intervention and the cumulative deficit of First Nations cannot necessarily be attributed to the Intervention Policy. Evidence from interviews, case studies and the data analysis would indicate that other factors, such as access to own source revenue, economic risks in particular economic sectors, the degree of geographic isolation, very small size, lack of a land base, or the unwillingness of creditors to lend money can also have an impact on the incidence, level or duration of intervention. Variations across the regions in terms of these other factors also mean that it is not possible to attribute regional variations in results to differences in the ways that the regional offices are implementing the Intervention Policy.

Mitigation:

Limitations on attribution were noted in the findings and analysis, as well as the underlying causes and contextual factors that could affect the success of the policy.

 Impact on Evaluation:

The evaluation is not able to attribute success in the achievement of long-term outcomes or variations across regions to the implementation of the Intervention Policy. This is due to a number of other factors or intervening variables. The evaluation does, however, include suggestions for alternatives to the Intervention Policy that could address these other underlying factors.

2.5 Evaluation Governance, Management and Quality Assurance

EPMRB of INAC's Audit and Evaluation Sector directed and managed the evaluation in line with its Engagement Policy and Quality Assurance Strategy. Unless otherwise noted, the Institute On Governance carried out the evaluative work.

EPMRB set up an external Advisory Committee to provide it with ongoing advice on the evaluation. The Advisory Committee included representatives from the Department of Finance and the Treasury Board Secretariat and one First Nation organization, and members were invited to participate in the two workshops of the subject expert panel. EPMRB also set up an internal working group that included representatives of Transfer Payment and Financial Policy Directorate (part of the Chief Financial Office), Regional Operations Sector, and INAC regional offices.

The evaluation methodology report and the draft evaluation report were peer reviewed for quality assurance and compliance with relevant evaluation policies of Treasury Board, INAC and EPMRB.

2.6 Presentation of Findings and Analysis

The next section provides a summary of the findings and conclusions in relation to the evaluation questions, drawing from the different lines of evidence. For the qualitative data, certain terms were used to indicate the proportion of respondents, regions, or case studies to which the finding refers; or other terms to indicate the frequency. These terms are roughly equivalent to the following percentages:

Proportional TermFrequency TermPercentage Range
All Always 100%
Almost all Almost always 80-99%
Many Often, usually 50-79%
Some Sometimes 20-49%
Few Seldom 10-19%
Almost none Almost never 1-9%
None Never 0%

Findings may not apply to certain respondents, regions, or case studies because no response was provided or no response could be inferred from other comments; or because a different response was provided. Where possible, alternative views, as well as the views of the majority, are presented.

Return to Table of Contents 






3. Evaluation Findings - Relevance

3.1. Continuing needs and priorities

Is the Intervention Policy responding to actual needs?

According to almost all INAC officials, the Intervention Policy meets the needs of First Nation communities and other recipients by assisting them to manage within the resources available, to ensure the ongoing delivery of essential services, and to be accountable to INAC. Many of the recipients agreed that the Intervention Policy was necessary, especially at the recipient-managed and co-managed levels. Almost all of the recipients under third party management, however, did not agree that the Intervention Policy met their needs.

An analysis of the triggers for the First Nations under some form of intervention as of February 4, 2010, indicates that the cumulative operating deficit is overwhelmingly cited as a reason for intervention (i.e. in 82.12 percent of all First Nations). This is particularly true for co-managed intervention (88 percent of the First Nations) and recipient-managed intervention (85 percent of the First Nations). For third party management, the reasons are more varied, but the cumulative operating deficit as well as the risk of insolvency are still important considerations (72.22 percent of the First Nations).

The case studies indicate that operating deficits and debt can affect the delivery of essential services to First Nations by:

The next most prevalent reason for intervention is a denial or adverse audit opinion (10.6 percent of all First Nations). A denial, adverse or qualified audit opinion may be an indication that financial management and accounting procedures are deficient or that the First Nation has refused to provide information for a consolidated audit. An accurate picture of the financial health of the recipient is therefore not available. The recipient then runs the risk of increasing deficits beyond their carrying capacity, of not being able to recover certain expenditures because they have not been properly accounted for, or of losing the confidence of funders and creditors. Unqualified audits, on the other hand, provide accurate reporting to the council or other governing body, the members, funders, creditors and other stakeholders.

In terms of third party management, some of the other reasons recorded on FNITP for intervention are:

Our interviews with INAC regional officials and the case studies indicate that serious breaches or other reasons could include the refusal to sign a funding agreement or to provide a consolidated audit. A less common reason is a serious election dispute and no legitimate leadership in place. Even if no funding agreement can be concluded, essential services still need to be delivered to First Nation members.

Many of the recipients and other key informants noted that First Nations also needed access to other sources of funding, increased funding from INAC, or increased flexibility in order to address their needs and build capacity.

3.2 Consistency with federal government and INAC priorities and policies

To what extent does the Intervention Policy align with a) federal government's priorities; and b) with the departmental strategic outcomes?

Over the past 30 years, program delivery has been transformed from direct delivery by INAC to delivery by First Nations through transfer payments. The Intervention Policy is consistent with federal government priorities and departmental strategic outcomes by supporting adequate oversight and stewardship of these resources; ensuring that First Nations receive and provide essential services; and ensuring that health and safety are protected; while also helping to build the administrative capacity of First Nation governments and organizations.

The Intervention Policy is most closely aligned with the Governance and Institutions of Government Program Activity of INAC's Program Activity Architecture. This Program Activity seeks to foster "stronger governance and institutions of government through supporting legislative initiatives, programs and policies, and administrative mechanisms that foster stable, legitimate and effective First Nations and Inuit governments that are culturally relevant, provide efficient delivery of services and are accountable to their citizens."[Note 8]

3.3 Duplication or overlap

Does the program duplicate or overlap with other programs, policies, or initiatives delivered by other stakeholders? (including other federal departments)

Research demonstrated that the INAC's Intervention Policy is one of its kinds and there were no duplication with other initiatives delivered by other stakeholders. There is little guidance on intervention in the Treasury Board Secretariat's Policy on Transfer Payments as the purpose was to provide flexibility to departments. According the Policy, in the event of a default under a funding agreement, the repayment of the grant or contribution may be required,[Note 9] the funding agreement may be terminated, or "other remedies and procedures" (unspecified) may be provided for.[Note 10]

Health Canada has used INAC's Intervention Policy as the basis for the design of its own intervention policy; and participates with INAC in the development of the third party management framework and the pre-qualification of third party managers. INAC and Health Canada HQ officials indicated that they let each other know if TPM is required and INAC advises Health Canada if a Canada-First Nation Funding Arrangement is going to be terminated due to intervention. Health Canada looks to INAC to support the development of general management and administrative capacity within First Nations and tribal councils and focuses on building the capacity of health related entities.

The First Nations Financial Management Board (FNFMB) has also used INAC's Intervention Policy as the basis for its own intervention policy. FNFMB is working closely with INAC on revisions to the policy and exploring how information can be better shared between the two for mutual benefit.

3.4 Federal government role and responsibility

Does the Intervention Policy align with the federal government role and responsibility?

The Intervention Policy is an operational policy that supports the federal government's roles and responsibilities as it helps to maintain the continuity of essential services to First Nations. However, the federal government is not responsible for First Nations' debt,[Note 11] but debt can have an impact on the delivery of essential services.

3.5 Conclusions related to relevance

Return to Table of Contents 






4. Evaluation Findings - Performance (Effectiveness/Success)

4.1 Intervention trends

What are the intervention trends?

Indicators

A review of documentation demonstrated that no performance measurement has been developed for the Intervention Policy. But since the beginning of the implementation of this policy, internal services measured some indicators, such as the number of communities under intervention, their level of intervention, the deficit ratio and the reasons for being under intervention. However, in the recent Report on Plan and Priorities information on the percentage of First Nations communities under financial intervention has been added[Note 12]. This information depends on different factors, such as the consistency of the end-user to input the information in the information system some reliability issued rose during the collection of the data. The information below on the incidence and level of intervention, as well as the duration were collected through regional offices rather than the FNITP system to ensure the reliability of the information. In this case, there may be inconsistencies with the information used by the Transfer Payment Directorate (i.e. departmental performance report) as the process to collect the information differs. Indicators based on a review of the documentation on Intervention Policy since the `90 as well as based on the observations of the evaluators during this process are listed in Appendix K.

Incidence and Level of Intervention

One measure of the success of the Intervention Policy over the long-term is the incidence and level of intervention – i.e. the more successful the policy, the lower the number of recipients under any form of intervention (incidence) and the lower the level of intervention. As Chart 1 below and Table 1 (Appendix I – Data Tables) indicate, the total number of First Nations under some form of intervention actually increased for the last 11 years (1999/2000 to 2009/2010). However, the total number has however declined from its peak in 2003/04 and has been on a downward trend since 2007/08 when the Intervention Policy was introduced. As of March 31, 2010, there were 168 First Nations under some form of intervention, or 29 percent of the total number of First Nations to which the Intervention Policy applies.[Note 13]

Chart 1 - Number of First Nations Under Intervention

 

Source: FNITP and Regional Offices

Chart 1: This figure shows a time series chart and legend, which represent the evolution of the total number of First Nations under intervention and by level of intervention, since fiscal year 1999/2000 up to 2009/2010.

Fiscal years (1999/2000 up to 2009/2010) are segmented along the horizontal x axis and the numbers of First Nations under intervention are indicated along the vertical y axis.

Four time series are represented in the chart, each represented by a color: Remedial Management Plan (blue), Co- Management (red), Third Party Management (green) and total of the three levels (purple).

Remedial management plan – In this time series, the number of First Nations under Remedial Management Plan fluctuated in a range where the minimum is 81 First Nations in fiscal year 2009/2010 and the maximum is 103 First Nations in 2001/2002

Co-Management – In this time series, the number of First Nations under Co-Management fluctuated in a range where the minimum is 36 First Nations in fiscal year 1999/2000 and the maximum is 66 First Nations in 2007/2008

Third Party Management – In this time series, the number of First Nations under Third Party Management fluctuated in a range where the minimum is 15 First Nations in fiscal year 1999/2000 and the maximum is 37 First Nations in 2003/2004.

Table 1 in Appendix I presents the detailed numbers for each level of intervention and total for every fiscal year.

From 1999/2000 to 2009/2010, the number of First Nations under recipient-managed intervention (RMP) declined, the number of First Nations under co-management (CM) substantially increased, and the number of First Nations under third party management also increased. From 2007/08 to 2009/2010, however, the number of First Nations under co-managed or third party-managed intervention has remained fairly stable. As of March 31, 2010, 81 First Nations were under recipient-managed intervention; 64 First Nations were under co-managed intervention; and 23 First Nations were under third party managed intervention.

Chart 2 and Table 2 (Appendix I) provides information on the level of intervention in terms of the proportional share of each level of intervention. From 1999/2000 to 2009/2010, the proportion of recipient-managed interventions has declined from 64 percent to 48 percent while the proportion of co-managed interventions has increased from 25 percent to 38 percent. The proportion of third party-managed interventions has fluctuated over the same period. From 2007/08 to 2009/2010, however, the proportion of each level of intervention has been relatively stable. As of March 31, 2010, 48 percent of the First Nations under some form of intervention were under recipient-managed intervention; 38 percent were under co-managed intervention; and 14 percent were under third party-managed intervention.

Chart 2 - Level of Intervention As A Proportion of the Total Number of First Nations Under Intervention

 

Source: FNITP and Regional Offices

Chart 2: This figure shows a chart and legend, which represent the level of intervention as a proportion of the total number of First Nations under intervention since fiscal year 1999/2000 up to 2009/2010.

Fiscal years (1999/2000 up to 2009/2010) are segmented along the horizontal x axis and the percentage of First Nations under intervention at a particular level are indicated along the vertical y axis.

Three levels are represented in the chart, each represented by a color: Remedial Management Plan (blue), Co- Management (red), and Third Party Management (green).

Remedial management plan – In this time series, the proportion of First Nations under Remedial Management Plan fluctuated in a range where the minimum is 48% of First Nations in fiscal year 2006/2007, 2008/2009, and 2009/2010 and the maximum is 64% of First Nations in 1999/2000.

Co-Management – In this time series, the proportion of First Nations under Co-Management fluctuated in a range where the minimum is 25% of First Nations in fiscal year 1999/2000 and the maximum is 38% of First Nations in 2009/2010.

Third Party Management – In this time series, the proportion of First Nations under Third Party Management fluctuated in a range where the minimum is 10% of First Nations in fiscal year 1999/2000 and the maximum is 20% of First Nations in 2003/2004.

Table 2 in Appendix I presents the detailed proportions of First Nations for each level of intervention and total for every fiscal year.

Ontario, Manitoba and Saskatchewan regions have the highest number of First Nations under some form of intervention; followed by the Atlantic, British Columbia and Quebec regions (Table 3, Appendix I). When the number of First Nations in each region is taken into account, Manitoba, NWT, Saskatchewan and Atlantic regions have the highest proportion of First Nations under some form of intervention; followed by Quebec, Ontario and Alberta regions. British Columbia and Yukon regions have a low proportion of First Nations under intervention, but 11 Yukon First Nations are self-governing and the Intervention Policy does not apply to them.

INAC regional officials noted some key variations across the regions that could lead to a higher incidence and level of intervention:

Some regions also noted that they had an active early warning system that they thought kept the incidence and duration lower. Officials in one region noted that they may have moved First Nations into multi-year flexible funding agreements too soon, leading to a higher incidence of intervention later on.

As indicated in Section 2.4.3, Limitations and Constraints, any changes in the incidence and level of intervention cannot necessarily be attributed to the Intervention Policy – for example, evidence from interviews and case studies would indicate that increased access to own source revenue also has a positive influence.

The incidence of intervention in other Indian-administered organizations increased after the introduction of the 2006 Intervention Policy, but declined in the following two years (Chart 3 below and Table 4, Appendix I). All of these organizations were under recipient-managed intervention except three cases in the past where co-management or third party management was used. As of March 31, 2010, there were seven organizations under recipient-managed intervention.

Chart 3 – Number of Other Organizations Under Some Form of Intervention

 

Source: FNITP and Regional Offices

Chart 3: This figure shows a time series graph indicating the total of organizations under intervention policy over the fiscal years 1999/2000 up to 2009/2010. The total numbers are indicated along the y axis and the fiscal years are indicated along the x axis. In this time series, the number of organizations under a level of intervention fluctuated in a range where the minimum is 3 organizations in fiscal year 1999/2000 and the maximum is 10 First Nations in 2007/2008. Table 4 in Appendix I presents the total number of organizations under intervention for every fiscal year since 1999/2000.

In comparison to INAC, Health Canada has a lower incidence of intervention, although they do not track recipient-managed interventions and information on co-managed intervention is not compiled at the national level. For example, there are currently only nine First Nations under third party management by Health Canada compared to 23 First Nations under third party management by INAC. Health Canada regional officials also indicated that their co-management and third party management interventions are of shorter duration.

Health Canada and INAC officials' explanation for the differences between the two departments were: Health Canada funding may be provided to a separate health authority and not the band council; a separate health authority can be set up if the band council is under third party management by INAC; there has been less flexibility in Health Canada funding; and capital funding is managed separately from program funding. Health Canada also revised its approach to its funding arrangements in 2008 to invest more time and effort up front in risk assessment and capacity building, and less time during implementation on compliance monitoring and intervention.[Note 14] According to Health Canada officials, however, it is too early to tell whether this change has had the desired impact on the incidence and level of intervention.

Duration of Intervention

One objective of the policy is to encourage the recipient in default to enhance their capacity to effectively provide programs and services so that intervention measures are no longer required. As mentioned in the 2003 Report of the Office of the Auditor General of Canada[Note 15] "intervention is intended to be temporary" and the 2005 Office of the Auditor General Report[Note 16] added that there was a lack of strategy for building the capacity of First Nations management to end third party management. A measure of success is the duration of intervention. In the past eleven years (1999/2000 to 2009/2010), a total of 349 First Nations have been under intervention at some time or another. About half of those First Nations are no longer under intervention. There are, however, First Nations that have been under some form of intervention for a long time. For example, of the 168 First Nations under some form of intervention as of March 31, 2010, 42 percent had been under intervention for 10 or more of the past eleven years; 29 percent had been under intervention for six to nine years; and 29 percent had been under intervention for one to five years.

Table 5 - Duration of Intervention as of March 31, 2010

Number of Years
(1999/2000 to 2009/2010)
No. of First Nations
Under Some Form of
Intervention
% Total
10+ years 70 42%
6-9 years 49 29%
1-5 years 49 29%
Total 168 100%

Source: FNITP and Regional Offices

Saskatchewan, Manitoba and the Atlantic regions account for most of the First Nations that have been under some form of intervention for a long time.

Table 6 - Long-Term Duration by Region as of March 31, 2010

RegionNo. of First Nations
under some form of
intervention for 10+ years
(1999/2000-2009/2010)
% Total No. of
First Nations under
some form of intervention
for 10+ years
Atlantic 11 16%
Quebec 4 6%
Ontario 7 10%
Manitoba 19 27%
Saskatchewan 24 34%
Alberta 3 4%
BC 2 3%
Yukon 0 0%
NWT 0 0%
Total 70 100%

Source: FNITP and Regional Offices

Cumulative Deficit

Since a key trigger for intervention is the cumulative operating deficit, another measure of success is the trend in the cumulative deficit – i.e. the more successful is the implementation of RMP under Intervention Policy over the long-term, the fewer the number of First Nations with a cumulative operating deficit greater than eight percent of total operating revenues. In fact, the total number of such First Nations has declined by 32 percent from 2004/05 to 2009/10 (see note). British Columbia and Saskatchewan regions have seen the most substantial improvements – i.e. a reduction of over 50 percent in the number of First Nations with a cumulative operating deficit over eight percent. Quebec, Manitoba and Ontario have seen moderate improvements – i.e. a reduction of over 35 percent. The number in the Atlantic region has marginally increased from nine to ten First Nations.

Table 7 - Number of First Nations with a Cumulative Operating Deficit >8%

Regions# FNs
2004/05
# FNs
2005/06
# FNs
2006/07
# FNs
2007/08
# FNs
2008/09
%
Change
Atlantic 9 12 13 13 10 +11%
Québec 18 9 16 10 11 -39%
Ontario 31 35 37 27 20 -35%
Manitoba 34 33 27 28 21 -38%
Saskatchewan 23 23 16 12 11 -52%
Alberta 6 9 7 10 6 0%
BC 30 28 22 19 13 -57%
NWT 6 3 5 3 5 -17%
Total National 157 152 143 119 97 -32%

Source: FNITP financial data

Note: This report reflects information entered into the FNITP Statement of Revenue and Expenditure where the Audit Reporting Requirement "Approved" has been completed. First Nations with qualified audits or a denial of audit opinion are therefore excluded.

Based on this sample, as it would be expected with a decline in the number of First Nations, the total size of the cumulative deficit of those First Nations has also declined (Table 7, Appendix I). The average size of the deficit for each First Nation has however increased. Manitoba First Nations account for about 30 percent of the total cumulative deficit of First Nations with a cumulative operating deficit greater than eight percent.

The cumulative operating deficit is only one of a number of indicators that can be used to measure the indebtedness or financial health of a First Nation. It is not a totally accurate reflection of the extent of First Nation debt because it does not cover those First Nations with qualified audits or a denial of audit opinion. It will also no longer be applicable with the adoption of Generally Accepted Accounting Principles for First Nations in 2010/2011.

Changes in the cumulative deficit are also not necessarily attributable to the Intervention Policy – for example, an increase in own source revenue or a decrease in the willingness of creditors to lend money could also reduce the cumulative deficit.

In counterpart, a decline in certain key economic sectors (fisheries, forestry or oil and gas) and funding arrangements could be factors that increase the cumulative deficit. Funds are transferred to First Nations for programs and services on reserve through different types of arrangements. Since the implementation of the evaluation policy of 2006, when a third party manager is appointed to a First Nation community, the community should fall under the Comprehensive Funding Arrangement (CFA) which inhibit the third party manager ability to use surpluses to pay off debt as according a clause of this arrangement, any surplus need to be sent back to INAC. Furthermore, some respondents believe that the implementation of the revised Policy on Transfer Payments could bring the same impact certain First Nations that are assessed to be medium to high risk.

4.2 Underlying causes

What are the underlying causes of the intervention trends?

Based on the review of the sample RMPs, case studies, and interviews, the following list of the immediate causes that can lead to financial difficulties have been compiled (refer to Appendix J – Causal Factors Framework)[Note 18]:

As mentioned in Section 3.1, the immediate causes that lead to third party management are somewhat different and more varied. They may include the threat of creditors seizing funds, but also include other non financial reasons, such as the refusal to provide consolidated financial statements, the refusal to sign a funding agreement, or less commonly, a serious election dispute.

A list of potential underlying causes has been developed from the literature review, and supplemented this list through the review of sample RMPs, interviews and the case studies (Appendix I). The underlying causes are summarized into four key categories: poor governance, poor management, community factors and issues with funders and applies to First Nations and/or INAC. In the list below, highlighted underlying causes are addressed in whole or in part by the Intervention Policy, with its focus on financial management and some aspects of governance and funding arrangements. The other underlying causes that are not highlighted may, however, affect the success of the intervention – and would require a different approach to address.

Poor governance

Poor management

Community disengagement

Poor funding practices

Various contextual factors shape these underlying causes (Appendix I) and are related to community endowments, community member endowments and social cohesion and security. These contextual factors were also initially developed from the literature review and supplemented during our interviews and case studies.

Community endowments

Community member endowments

Social cohesion and security

Again, the Intervention Policy cannot address these factors but the factors can influence the success of the Intervention Policy.

The relationship between a number of these contextual factors and the level and duration of intervention in our statistical analysis have been tested. The key variables that were examined were: the CWB Index[Note 19] and its components (education, labour, income and housing); remoteness; registered population on reserve; and the type of electoral system (Appendix E - Summary of the Statistical Analysis). In brief, the analysis shows:

4.3 Achievement of objectives

Is the Intervention Policy successful in achieving its objectives and it intended results? Does this vary regionally? a) to what extent does the Intervention Policy reinforce the recipient's accountability? b) does the Intervention Policy contribute to building local capacity in order to deliver higher quality services? c) does the Intervention Policy contribute to the continuity of service delivery, appropriate use of funds?

Section 4.1 above looked at the achievement of the longer-term objectives of reducing the level, incidence and duration of intervention, and Section 4.2 looked at the underlying causes and contextual factors that affect the achievement of the long-term objectives, many of which the Intervention Policy has no influence over. This section will look at the achievement of the short to medium term objectives of ensuring the delivery of essential services, maintaining accountability, and building capacity.

Delivery of essential services

One of the objectives of the Intervention Policy is to ensure that essential services continue to be delivered. Essential services are education, social assistance and some community infrastructure. Non-essential services are governance support, economic development, post-secondary education and some minor capital. Regional officials and recipients indicated that INAC funding for both essential and non-essential services has continued to flow regardless of the level of intervention. First Nations under some form of intervention are also eligible to access targeted funding and many do. Under third party management, however, no new Ministerial Loan Guarantees (MLGs) can be approved by INAC for CMHC-funded social housing. For other levels of intervention, the RMP has to have been working for six months in order for a MLG to be approved.

The impact of intervention on the quality of the programs and services that are being delivered was not assessed because of the lack of program performance data. Twelve of our case studies[Note 20] indicate that implementing remedial measures and reducing debt had numerous benefits for First Nations and other Indian-administered organizations, including:

Ensuring accountability

INAC officials were of the view that the Intervention Policy maintains the external accountability of the recipient to INAC for funding. Although, INAC officials did not think that the Intervention Policy necessarily increased the internal accountability of the recipient to its members.

According to the Intervention Policy, third party managers are expected to account to the community in terms of the funding that they are administering. A few regional officials noted that some third party managers were accounting to community members and this had the benefit of increasing community expectations of their leadership. Regional officials and recipients under third party management, however, noted that some third party managers do not visit communities. The one third party manager interviewed indicated that a lack of guaranteed security while on reserve was a major reason for not visiting the community. A couple of the recipients under third party management also indicated that INAC needed to monitor the performance of third party managers more closely.

Capacity building

According to the Intervention Policy, the onus is on the recipient to enhance their capacity but INAC recognizes the need to support capacity development and commits to use the limited resources available for assisting those recipients most in need.[Note 21] At the time that the 2006 Intervention Policy was adopted, additional funds for capacity building were provided through the Companion Initiative. However, those funds were for short-term projects and were only available for two years.

A total of $10.6 million was allocated to the Companion Initiative in 2006/07 and 2007/08. 115 projects were funded from the 2006/07 funding allocation, or an average of $29,304 per project;[Note 22] and 147 projects were funded from the 2007/08 funding allocation, or an average of $36,000 per project.[Note 23] The projects covered three broad categories: financial development, management development, and intervention system, including the development of RMPs and RMP implementation projects. The funding was provided for tools, training and resources to increase capacity. In addition, the Aboriginal Financial Officers Association (AFOA) developed three new seminar-based and on-line courses for band office staff, including "Developing an Effective Remedial Management Plan."[Note 24]

Professional and Institutional Development (P&ID) funding[Note 25] is available on an ongoing basis for governance and management development, but the amount of funding is relatively small compared to INAC's other programs. For example, the P&ID program transferred $13.9 million to First Nations, Aboriginal organizations and tribal councils in 2007/08.[Note 26] P&ID funding is proposal driven and the demand outstrips the funds available.[Note 27] P&ID also has other priorities, although four regions mentioned that they have managed to set aside funds from P&ID for First Nations under intervention or at risk.

An internal audit of INAC's capacity development programs and authorities, including P&ID, found that INAC does not have a coordinated approach to the delivery of capacity development programs; there are no specific resources to support the design and implementation of effective capacity development programs; capacity development programs often address common issues, such as governance design and implementation, community engagement or skills development; and appropriate performance measurement data to demonstrate program effectiveness is not being gathered.[Note 28] The audit also found that underlying issues at the recipient level impede capacity development including: lack of basic education, difficulty in attracting and retaining qualified staff, periodic changes in leadership, and lack of a comprehensive plan.[Note 29]

The most significant and sustainable capacity building in our case studies occurred through the recruitment by First Nations or other Indian-administered organizations of qualified managers and financial directors or the contracting of expertise, including co-managers. For very small First Nations, as well as isolated First Nations, attracting and retaining qualified staff was an issue and raised questions about the sustainability of any progress.

According to regional officials and recipients, co-managers vary in terms of their contribution to capacity building. Some co-managers set up systems and procedures, control expenditures, train staff and the leadership, and improve financial results. Other co-managers lack the required skills and experience, are overstretched, or do not contribute adequately to the capacity building of band staff. Some regional officials attributed the poor performance of co-managers to inadequate controls by INAC over their selection and performance. Co-managers themselves said that their success was dependent to a large extent on the quality of their relationship with chief and council.

The Intervention Policy requires that INAC review the Co-Management Agreement between a co-manager and the council before it is executed in order to ensure that minimum requirements have been met and that the co-manager will be effective in remedying the default identified. [Note 30] The policy also provides examples of criteria for the selection of a co-manager and of the various roles and responsibilities that could be defined in the Co-Man[Note 31] Some regions have used this provision in the policy to ensure that co-managers have the required level of expertise and experience that they have been given co-signing authority at least for INAC funding, that they develop an exit strategy, and that their performance is reviewed regularly by the council and they are replaced if necessary. Other regions have exercised less control over the selection and management of co-managers by councils. Officials in those regions thought that they should exert more control and that more control might be more tenable to First Nations if INAC also provided funding for the co-managers. One region raised the issue of INAC's potential liability for the performance of co-managers if more control is assumed for their selection and performance.

The consensus among INAC HQ and regional officials and recipients was that most third party managers do not enhance capacity. They are too expensive and getting more expensive; they may not have the required expertise if the underlying cause is non-financial; they meet with resistance from the First Nation; and they usually only deal with INAC's funds and not all sources of revenue.

INAC's Funding Services Officers (FSOs) can provide limited support but according to INAC HQ and regional officials, they are not necessarily qualified and are overstretched. There is also a potential conflict between the FSOs' supportive role and their monitoring role. Many FSOs themselves indicated that they needed capacity building on remedial management.

Tribal councils are another potential source of capacity. INAC has devolved the provision of advisory services for economic development, financial management, community planning, technical services, and band governance to tribal councils or larger unaffiliated bands. Tribal councils are required to provide all five advisory services to their members, but an evaluation in 2009 found that not all were, particularly, financial management advisory services. Communities may resist sharing financial information with their tribal council, and remote and isolated communities have difficulties accessing their tribal council.[Note 32] These findings were echoed in our interviews and case studies - some tribal councils are very supportive and some are not, some communities resist getting support from their tribal council, and some tribal councils lack the skills and resources themselves.

AFOA was most often mentioned by key informants as a potential source for capacity building – training, certification, and advisory services. However, AFOA itself recognizes that it has variable capacity – some of its regional chapters are strong and some are weak. First Nations councils also have to pay for AFOA's services and it is often the bands that are most in need that use the services the least.

4.4 Unintended impacts

What are the unintended impacts of this policy? (positive or negative)

Unintended potential positive impacts that were identified in the interviews and the case studies are:

A number of negative impacts were mentioned in our interviews and case studies, but often these negative impacts were related more to the debt situation than to the intervention itself - such as, a reduction in the availability of flexible funding, a reduction in access to MLGs, a reduction in access to credit, staff lay-offs, shortages of staff, loss of benefits for staff, declining staff morale, and band members losing access to loans and advances.

Other potential negative impacts that were noted in the interviews are:

One major impact that can affect the success of the Intervention Policy itself was the reduction in the availability of band support funding because of payments to co-managers and third party managers. INAC provides band support funding as a grant so there is a high degree of flexibility in terms of how it is used. However, the evaluation of Indian Government Support Programs conducted in 2009 found that band support funding has not kept pace with the realities of modern governance and administration.[Note 33] The cost of third party management in particular can take most or, in a few cases, all of the band support funding, leaving the band to find other sources to employ a band manager and administrative staff.

There are also perverse incentives for co-managers and third party managers to remain in place rather than to work themselves out of a job. There is no incentive for First Nations to save under contributions or for third party managers to realize savings since they cannot be used to pay off debt.

4.5 Key Success Factors

What are the key success factors?

Success and other success cases were used to explore the reasons why some First Nations have never been under intervention despite being disadvantaged, and why some First Nations were able to get out of intervention after a long period of time under intervention. The reasons for success were also explored in the interviews and the document review. Based on these sources, the key success factors identified are:

As with the underlying causes and the contextual factors, the Intervention Policy has little influence over the majority of these key success factors. Our case studies and interviews also noted that improvements in the financial situation or capacity of a First Nation can be fragile – for example, a band-owned business can start to lose money or a key staff person can retire.

4.6 Conclusions related to Success

Return to Table of Contents 






5. Evaluation Findings - Performance (Efficiency and Economy)

5.1 Costs of intervention

What are the costs of intervention for the recipient and INAC?

The direct cost of the Intervention Policy for INAC cannot be quantified since it is part of the transfer payment process and, therefore, part of INAC's ongoing operations. HQ officials reported that the cost of policy development, monitoring and compliance for the Intervention Policy is not significant. INAC regional officials mentioned that the cost of implementing the policy can be significant, especially in the time and travel of FSOs. Some regional offices also have a specific person dedicated to supporting intervention.

The direct cost of the Intervention Policy for First Nations can be significant, depending on the amount of funding they receive and what level of intervention they are under. Recipients have to pay for a co-manager directly, and payment of the third party manager is taken from the recipient's band support grant. Travel costs for co-managers and third party managers can also be high for remote and isolated communities.

The costs of co-managers have not been quantified due to the lack of comprehensive information for all recipients. Regional officials or recipients mentioned total costs ranging from $100,000 to $300,000 per year for co-managers. These costs would need to be analyzed in relation to the total amount of INAC funding provided in order to gauge the significance for the First Nation.

Cost of Third Party Managers

Since fiscal year 2006/2007, the portion of Band Support Funding allocated to third party managers is recorded as a unique account in the First Nations Information Transfer Payment System (Band support funding for third party management services). Prior to that, there was no distinction in the funds allocated from the Band Support Funding. Even if only two regions were recording the information in this specific account in fiscal year 2006/2007 and the following years, there were a lack of inconsistency in the data inputted in the system (e.g. First Nations under TPM missing, First Nations under co-management listed into this account), in fiscal year 2009-2010, almost all regions recorded the proper information in a consistent manner.

The cost of third party managers varies depending on the fee rates of the service provider, the type of services provided, their level of effort required and the travel and living expenses. The TPM life cycle is also a factor as regional officials also told us that there is a higher level of effort required from TPMs at the beginning of any contract and whenever a TPM is changed.

As shown in Table 8, the amount paid to TPM varies depending on the selected firm or type of resources. Examination of a sample of TPMs agreement for fiscal year 2009-2010 showed that, according resources selected, fees per day, per resources could vary within a range of $557 to $2300 for an administrator, within a range of $575 to $1950 for an advisor and within a range of $275 to $1200 for an administrative support. The latest tender for TPMs (2009) required that third party administrators have an accounting designation (Chartered Accountant, Certified Management Accountant or Certified General Accountant).[Note 34] Some regions noted that this mandatory requirement had resulted in a doubling or tripling of daily fee rates and a smaller pool of potential service providers. There were concerns that the cost of TPM could start to exceed the band support grant of smaller communities and those in the North.

Table 8 - TPM remuneration for fiscal year 2009-2010

ResourcesPer diem rate per day
MinimumMaximum
Administrators $557 $2300
Advisors $575 $1950
Administration support $375 $1200

Source: Third Party Management Agreements

Data analysis of the fees for TPMs since 2006/07 demonstrated that the cost of TPMs has been stable. While in fiscal year 2006/07, the fees ranged from $29,842 to $768,200, in fiscal year 2008/10 the fees ranged from $43,825 to $339,197. Percentage of the Band Support Funding remained almost the same. In fiscal year 2006/07, the fees ranged from 8.5 percent to 125.8 percent and in fiscal year 2009/10, it ranged from 23.2 percent to 100 percent. It should be noted that in this sample, only one community used all its band support funding in fees for TPMs. Our analysis of TPM professional fees also indicates that smaller bands pay a higher proportion of their INAC's funding and Band Support Funding for TPMs - it could be due to the fact that there are certain fixed costs, the complexity and the amount of INAC program funding that are not linked to the amount of funding that the First Nation receives from INAC's Band Support Funding.

Table 9 - TPM fees and % of total Band Support Funding and total INAC's funding

Fiscal Years2006-20072007-20082008-20092009-2010
Sample of First Nations under TPMn= 28n=12n=19n=21
Cost of Third
Party Managers
Average $180,214 $181,055 $134,353 $142,969
Median $120,000 $160,144 $120,000 $117,000
Min $29,842 $25,000 $61,425 $43,825
Max $768,200 $314,665 $262,200 $339,197
% of the total Band
Support Funding
Average 41,1% 46,1% 42,9% 46,4%
Median 33,8% 45,1% 36,4% 38,9%
Min 8,5% 13,7% 13,6% 23,2%
Max 125,8% 72,1% 100,0% 100,0%
% of the total transfer
payments from INAC
Average 4,1% 5,3% 5,4% 5,8%
Median 3,0% 3,8% 2,3% 3,1%
Min 0,6% 2,1% 0,7% 1,1%
Max 13,1% 12,4% 36,2% 37,6%

Source: FNITP (FY 2007- 2008 to 2009-2010) and Q51 for fiscal year 2006/07

5.2 Comparison to cost of other intervention approaches

How does this cost compare to the cost of other intervention approaches?

The costs for Health Canada and its recipients are similar to those of INAC, except that the incidence and duration of intervention is lower. FNFMB has not yet implemented its intervention policy but will have few staff for all of its functions and will be contracting out for a lot of the expertise required. FNFMB has also not decided whether or not to assume the cost of co-managers and third party managers.

The intervention approaches of other countries and provincial governments are discussed in more detail under Section 8, Improvements and Alternatives. Like INAC, the cost for the funder cannot be separated from other costs related to managing transfer payments. In most cases, however, the funder assumes the cost of any special assistance, financial controllers, or third party administrators. The incidence and duration of intervention is also much lower in these other jurisdictions than in INAC's case, with one exception – Queensland Indigenous Councils.

5.3 Cost-effectiveness/value for money

Do recipients and INAC believe that they have obtained value for money?

The Intervention Policy is not cost-effective because it has not been successful in achieving most of its long-term and some of its short-term objectives. The high incidence and duration of intervention drives the costs up for both INAC and the recipient. Unwillingness, lack of capacity and low revenues cannot, however, be addressed by the Intervention Policy alone. They require other "interventions" from other players.

A lot of time and effort is being invested by INAC and recipients in remedial management plans, but they are of questionable value if they are not adhered to, and INAC has limited means to monitor and enforce compliance. It is difficult to think of an alternative, however, other than simplifying them – for example, an action plan to respond to the audit observations, and a brief description of how surpluses are going to be generated. Monitoring could still be carried out quarterly.

Co-managers can perform a useful treasury or controller function and build capacity and their cost appears to be reasonable in most circumstances. The alternative would be a full-time financial controller but that might be more than a recipient needed given their other staff. Some recipients have retained their co-managers on contract after co-management is no longer required by INAC, or contracted co-managers when they are under recipient management. One recipient suggested that INAC pay for the position of a financial director in every First Nation, to manage all revenues, not just INAC funding.

Third party managers are not cost-effective, but they are intended to be used as a last resort when no other option is available. As one region noted, a third party manager who draws thousands of dollars a year from a band budget, has no authority over own source revenue, and leaves nothing behind in terms of infrastructure (or often less than when he/she started), is not a good investment of public money. One recipient noted that the cost of its third party managers for 14 years would have been enough to pay off their debt.

The costs of not intervening are very high as well – loss of accountability, the risk of INAC funds being seized, the risk that essential services will not be delivered, and the risk to the health and safety of band members.

5.4 Conclusions related to Efficiency - Economy

Return to Table of Contents 






6. Evaluation Findings - Design

6.1 Clarity and understanding of objectives

To what extent are the objectives of the Intervention Policy clearly articulated and understood?

According to HQ and regional officials, the objectives of the Intervention Policy are clear – to ensure funding is used for intended purpose, essential programs and services are delivered, accountability is achieved, and to address the problems that led to intervention. INAC officials were of the view that recipients understood these objectives as well. Recipients themselves did not express any views on the clarity of the policy.

INAC and recipients both agreed that there needs to be a parallel piece on capacity building or other actions to address the situation. Capacity building is not really a focus of the Intervention Policy and there is little direction in the policy about how capacity building is to be delivered.

The financial institution interviewed thought that there should be greater transparency about the Intervention Policy so that lenders know the rules. For example, the Intervention Policy is not currently accessible on INAC's website.

6.2 Clarity and understanding of roles, responsibilities and accountabilities

 To what extent are the roles, responsibilities and accountability of key players clear and well understood? Is there a clear understanding of terminology used?

According to HQ officials and other interviewees, the roles, responsibilities and accountabilities of INAC and First Nations are generally clear and well understood. The onus is on the recipient to address the default and build capacity. Recipients are also responsible for their debt. The terminology is also clear.

Co-managers and third party managers were also of the view that their roles, responsibilities and accountabilities were well-defined but observed that often, they were not able to work in collaboration with chief and council and this impeded their effectiveness. Prior to be appointed with a Third Party Manager, some First Nations were not clear about their responsibilities or were not willing to accept them – e.g. signing the funding arrangement, arranging for a consolidated audit, and hiring staff.

A Google search of media coverage regarding Intervention Policy over the past year indicates that the media and the public do not clearly understand the Intervention Policy and there is a tendency to focus on third party management. INAC HQ officials and other interviewees suggested that there needs to be more effective communication and a clearer distinction among the levels of intervention.

6.3 Consistency of delivery and results nationally

To what extent does the design of the Intervention Policy lead to consistent national delivery/results?

The Intervention Policy provides triggers, assessment criteria, intervention processes, levels of intervention, records management, and roles and responsibilities within INAC. These components of the Intervention Policy are consistent across the regions (refer to Section 7, under Evaluation Findings- Delivery).

The implementation of the Intervention Policy requires judgement to be applied, however, the context and circumstances are different from one recipient to another, within regions, and across regions. First Nations vary by size, degree of isolation, access to capacity or access to own source revenue. There are also variations in the way that INAC's regional offices are structured, the availability of funding services and program staff, and their knowledge and abilities. All of these variations may have an impact on the decisions that are taken. According to the Intervention Policy, however, these decisions should be documented, reviewed, communicated to the recipient, and the recipient given an opportunity to respond (refer to Section 7, Evaluation Findings- Delivery). Some differences among INAC's regional offices have also been mentioned in the Audit of the Application of the Intervention Policy[Note 35].

6.4 Comparison to other intervention approaches

To what extent does the Intervention Policy design compare with other intervention approaches (e.g. other government departments, provincial governments, other jurisdictions, financial institutions, etc.)?

In the literature review, eight other intervention approaches have been looked at, including Health Canada and the FNFMB. As mentioned previously, Health Canada and FNFMB's approach are based on INAC's approach. The other examples were:

Following is a comparison of the key design features of those other approaches in comparison to INAC's approach.

Intervention objectives

The objectives of intervention are similar to INAC's and include:

It is usually not an option to dissolve a public sector entity in the event that it is insolvent. Funders or higher orders of government, therefore, have an interest in ensuring that the entity can continue to deliver services without jeopardizing public health or safety. Funders or higher orders of government also have an interest in protecting public funds and assets for present and future generations and ensuring accountability for the use of those funds.

Triggers

In the examples, the triggers that have been used for intervention are similar to those that are used by INAC – a breach in the funding agreements, contract or accountability agreements; operating deficits; adverse audit opinions; and risk to health and safety (or trust lands in the Aboriginal examples). In some cases, performance-based triggers are also being used. For example, the Ontario Ministry of Health and Long-Term Care can intervene in public hospitals if there are accessibility or quality issues. The Ontario Ministry of Education can intervene in school boards if education outcomes are not being attained.

Levels of intervention and remedies

The levels of intervention that have been used are similar to those that INAC uses, with variations in the terminology that is used. These levels include the provision of advice and support by the government or other parties; co-management or the appointment of a financial controller; and third party management or supervision. Co-management is less frequently used in favour of special assistance teams that go in to address problems. In some cases, a higher order of government also has the option of dissolving and reconstituting the governing body of a subnational government – an option that is not available to INAC. In the case of American Tribal Governments, the United States federal government has the option of reassuming responsibilities for the delivery of programs and services because delivery has not been fully devolved to tribal entities.

As with INAC, some form of assessment is conducted prior to assuming some measure of control over the affairs of a recipient. This assessment may be conducted by government officials, by independent bodies, or by peers, whereas only INAC officials conduct the assessment. As with INAC, the assessment not only determines whether intervention is required, but also informs the actions that need to be undertaken in order to rectify the problem.

There are also some examples where assessments or examinations are used more than audits. These assessment or examinations tend to be more frequent, more rigorous, more risk-based, and more preventive than audits. This is similar to the proposed general assessment that INAC will be conducting.

Capacity building for prevention and remediation

A lack of capacity is often identified in the examples as underlying the need for intervention. Many governments have provided a range of capacity building support to recipients, particularly related to financial management but also in governance. This support has been delivered through training, through guides and other tools, through peers, through technical assistance teams, and so on.

In some examples, capacity building is linked to the certification of individuals – for example, governance certification for board members on Ontario's school boards and public hospital boards or on Queensland's indigenous councils. In a couple of examples, capacity building is also linked to the accreditation of the organization itself – i.e. FNFMB or Ontario public hospitals.

Oversight and monitoring

There is a lot of emphasis on ex ante controls in the literature on subnational debt management, as well as in the examples involving provincial governments and municipal governments, school boards or hospitals. These controls include the requirement for balanced budgets, no operating deficits on current accounts, limits on borrowing, approval of long-term liabilities, etc.

In some cases, such as the FNFMB or the Registrar of Indigenous Corporations in Australia, the oversight and monitoring of intervention is carried out by an independent party, separate from the funders. This independent third party can address problems in the organization as a whole, and can act on behalf of all funders rather than just one funder or department.

6.5 Well designed to achieve objectives

Is the Intervention Policy well designed to achieve its objectives?

According to INAC officials, the Intervention Policy is not well designed to achieve its objectives. It does not include prevention measures; the categories of default or the triggers are too broad or ill-defined; capacity building is not adequately addressed and there is no framework to assist recipients to exit intervention; there is no performance measurement framework; the debt management rules are restrictive; recipients are not encouraged to take responsibility and build capacity at higher levels of intervention; and there is a lack of funding to assist recipients under intervention without having to resort to band support grants. These deficiencies will be addressed further under the suggested improvements to the policy (Section 8.1).

6.6 Conclusions related to Design

Return to Table of Contents 






7. Evaluation Findings - Delivery

This section presents the division of roles and responsibilities within INAC, the key activities that are being undertaken (prevention, detection, remediation, exit strategies and monitoring), and the key outputs related to the implementation of the Intervention Policy (refer to Section 1.2.3 Policy Framework and 1.2.4 Policy Management). A more thorough review of delivery is provided in the Audit of the Application of the Intervention Policy (2009).

7.1 Implementation as planned

Is the Intervention Policy implemented as planned?

Although the Intervention Policy does not address prevention, the Atlantic, Quebec and Alberta regions have instituted formal systems to detect potential early warning signs by using more comprehensive assessment grid that might lead to the need for intervention. Other regions have a more informal process of monitoring communities that might be experiencing difficulties more closely. Atlantic and Alberta regions also provide an orientation to new chiefs and councils on their roles and responsibilities under the funding agreements. All regional staff spend significant time explaining INAC programs, the funding available, and the terms and conditions of funding agreements in order to avoid the need for intervention.

According to regional officials, prevention and early detection requires effort by the FSO and a willingness by the recipient to address issues before they get to the point where intervention is required. FSOs don't necessarily have the time or the ability to monitor developments closely. There is, therefore, a tendency to rely on the audit report which comes in several months after the fiscal year end and the situation may have deteriorated further by then.

Interviews and the document review indicate that regional assessments are based on the Performance Review Model in the Intervention Policy. The model looks at the willingness and capacity of the recipient, financial indicators relating to expenditures and revenues, reporting, and audit results. Our case studies indicate that the imposition of remedial management appears to be linked more to the relatively small size of the deficit (e.g. 0 to 16 percent) than to the willingness and capacity of the recipient; co-management to larger deficits (e.g. 17 to 32 percent) rather than a lack of capacity; and third party management to very large deficits (33 to 100 percent) as well as unwillingness.

Other factors that may be considered when making a decision about intervention are complaints and allegations or calls from creditors. Co-managers may also conduct a situational review once appointed, or an interim audit may be commissioned to update the figures on the financial situation of the recipient.

Our review of the sample of remedial management plans indicates that RMPs are variable in quality and format across and within regions. In some of the regions, a standard format has been adopted and used by recipients. In other regions, the RMPs vary by the recipient. RMPs are not always available or if available, they are not always approved by the First Nation, particularly when co-managers or third party managers are appointed.

In terms of the minimum RMP components (refer to Section 1.2.3 Policy Framework), our review of the sample of RMPs indicates:

All regions have a regional review and approval process to recommend the level of intervention and approval of remedial management plans, and to monitor implementation. These processes are linked to the audit review process. Manitoba region invites the First Nation to make a presentation to the regional committee when a decision or recommendation on intervention is pending. Quebec region has an internal committee and an external committee that is involved in each intervention decision. In the Alberta region, chiefs and councillors or members of the band council administration may attend the review committee.

FSOs play a key role in the delivery of the Intervention Policy by maintaining a positive relationship with the First Nations that they are responsible for, providing assistance in the development of RMPs, and monitoring the implementation of RMPs. Some regions have a high turnover or high vacancy rate for FSOs that impedes delivery of the policy. In all regions, the workload and requirements of the position limits the amount of capacity building that an FSO can provide. Some FSOs also commented on their lack of training and orientation to the Intervention Policy.

Co-managers are considered the weakest link by regional officials. As noted in Section 4.3, Achievement of Objectives, some co-managers are not considered qualified or experienced, some are not co-signing for the expenditure of INAC funds, and some are not performing well. Some regions are proactive in approving the selection and monitoring of co-managers by First Nations. Others are reluctant to exercise authority in approving the selection.

Third party managers are appointed from the national list of pre-qualified firms or individuals and funded through a third party management framework agreement. The decision to impose third party management is made by the regional director generals. In term of qualification / pre-qualified firms or individuals, the MERX process requested in 2010 that the third party administrator have an accounting designation. However, some have concerns that the individuals without designation that were delivering third party services be hired by those firms to support third party administrator.

7.2 Constraints or pressures impeding implementation

Are there constraints or pressures that impeded the implementation of Intervention Policy? (Note: As the delivery has been addressed in the audit, the evaluationteam looked at this question regarding the impact on the success)

Case studies of recipients that have been under recipient-managed, co-managed or third party-managed intervention for a long time have been used in order to assess the constraints or pressures impeding implementation of the Intervention Policy. Analysis of those cases demonstrated that the situation and circumstances of each First Nation were different and there was no single answer for each level of intervention.

First Nations under recipient-managed intervention:

First Nations under co-management or third party management:

In the nine cases where First Nations have been under co-management or third party management, the evaluation team looked at whether capacity was being developed by the co-manager or third party manager while expenditures were being controlled.

The three cases involving other Indian-administered organizations are under recipient-managed intervention and in all three cases the intervention was triggered by an operating deficit. In two of the cases, there were also governance and financial management issues. All three organizations are working off their deficit, and two have made significant governance and management changes. In two of the cases, there was political interference in the organization by the First Nation(s) to which it was linked. In the third case, the band council did not interfere in the operations of the organization, nor did it provide any support.

In summary, therefore, the case studies indicate that progress under intervention is usually not linear. Progress is made for a while, then there are setbacks; First Nations move up and down the various levels of intervention. Many of the problems that have led to intervention cannot be addressed in the short-term, but require long-term solutions. It takes time to reduce deficits, especially if there is limited access to own source revenue; to build up financial policies, systems and procedures and to train staff; and to build up a set of accounts that can be audited. Meanwhile, the context is constantly changing – changes in governance, management and staff, the membership, funding conditions, access to own source revenue, natural disasters, etc.

Some of the explanations as to why it was so difficult for recipients to exit intervention are included in the analysis of underlying causes and contextual factors in Findings and Conclusions on Success. Some other constraints included:

FSOs were generally aware of the issues and problems and many offered support to the recipients, but the support was not always accepted.

7.3 Tools for collecting evidence

To what extent do the means and tools exist for: a) obtaining reliable evidence of consistent policy implementation; and b) success/performance (including indicators)?

The FNITP system includes an intervention module for capturing information related to the application of the Intervention Policy. According to the HQ officials responsible for the system and to the regional officials responsible for entering the data, the module is not easy to use. There are a lot of layers for input and for performance reviews.

It is also not easy to report on intervention using the system because the data does not match the policy. The reports produced are not useful for monitoring, diagnosis and problem-solving. The data had not been kept up to date and accurate until recently due to closer monitoring by senior management. HQ FNITP officers have to spend a lot of time validating the information and producing reports on request.

The FNITP intervention module is to be amended based on the new intervention policy and directive once approved.

7.4 Policy guidance and performance monitoring tools and mechanisms

To what extent is good guidance and monitoring in place?

Policy guidance tools and mechanisms

The Financial Policies and Procedures Manual contains various tools and templates to assist with implementation of the policy – a template for the contents of a remedial management plan, suggested criteria and a statement of work for a co-manager, sample letters to be sent to creditors, a third party manager agreement model, and a performance review model for assessing the willingness and capacity of recipients. Other tools are also available outside of the policy such as reporting requirements or complaints and allegations procedures.

At the time that the 2006 Intervention Policy was launched, an information tool kit was developed for FSOs to assist them in responding to questions from First Nations on the revised intervention framework and to provide guidance on the garnishment of First Nations funding, the selection of third party managers, and related technical and procedural matters.[Note 37]

Ongoing guidance is provided by INAC HQ on third party manager framework agreements and the use of the third party manager list.

Some of the regions have also developed their own policy guidance and tools whereas other regions consider the existing policy guidance to be sufficient. Quebec region had already developed its own regional intervention procedures just prior to the new Intervention Policy being adopted;[Note 38] British Columbia region subsequently issued a policy and procedures manual and intervention plan receipt procedure and FNITP guide;[Note 39] Atlantic region adopted its own Intervention Policy Directive and developed a set of criteria and a monitoring system to manage intervention; and Alberta region has developed standardized templates for RMPs and co-management agreements, Management Development Plans, and financial recovery and repayment plans.

Performance monitoring tools and mechanisms

Senior management has only recently been monitoring performance related to intervention. The Intervention Policy is now a standing item at the Financial Management Committee for monthly review. The Operations Committee is also interested in the issue. A Working Group that includes representatives from the regions and the Chief Financial Officer Sector has been reviewing the Intervention Policy.

INAC's Compliance Unit reviews implementation of the Intervention Policy as part of a review of transfer payments that is conducted on an annual basis across all regions. The Unit reviews files and the supporting documentation for decisions, prepares a report on its review, and follows up in two years time on the implementation of the changes required.

FSOs review the implementation of RMPs monthly or quarterly and regional committees review performance quarterly or annually.

According to the Intervention Policy, the performance of third party managers is supposed to be reviewed monthly, wherever possible with the recipient. Regional offices indicated that they review performance when monthly invoices for payment are submitted by third party managers. The monitoring of our third party management cases and a few of the co-managed cases was however difficult because approved RMPs were not in place, the band council was uncooperative or antagonistic, or the TPMs or CMs were changing frequently. A couple of First Nations also commented that INAC's monitoring of third party managers was poor – i.e. third party managers were not visiting the community, were contracting for services from unqualified contractors, or were allegedly misusing funds.

7.5 Implementationof audit recommendations

To what extent have recommendations from past audits been implemented as planned?

An audit of the management controls associated with the application of the Intervention Policy was conducted in 2008-09.[Note 40] The audit noted that there were significant variations between policy and practice and from region to region in terms of how the intervention policy was being applied. The audit concluded that management controls are not adequate to ensure effective monitoring of the application of the intervention policy and the application of judgement or decision making that is appropriate, deliberate and consistent across the Department.

The audit recommended an assessment of the effectiveness of the intervention policy, clarification of the policy objectives, revision of policy components to ensure consistency with the objectives, and the development of related tools and guidance to ensure the policy is implemented as intended. Policy revisions and related guidance included:[Note 41]

An Intervention Policy Audit Action Plan has been developed by INAC that includes actions to reassess the policy with a view to clarifying policy objectives, components and related tools and guidance; and actions to identify, recommend and develop tools and practices to support consistent application across regions based on risk and best practices. The actions have been undertaken by a joint Regional Operations Sector-CFO Intervention Policy Working Group. A new intervention policy and directive have been drafted and will be finalized in the fall 2010.[Note 42]

7.6 Coordination within INAC, with OGDs, with provincial governments, with others

What is the extent of coordination of delivery within INAC, with OGDs, with provincial governments, with others?

According to HQ and regional officials, coordination within INAC was generally good – i.e. with governance, education, capital and social programs and between the CFO and Regional Operations Sectors. However, some FSOs mentioned that they would like additional support from program staff when dealing with particular issues related to that program in recipients under intervention.

Communication with other federal government departments, particularly Health Canada, could be improved and formalized. While Health Canada HQ officials considered coordination at the HQ level with INAC to be good, Health Canada officials at the regional level said that they would like to be better informed about INAC's intervention in their recipients, including but not limited to third party management. One regional Health Canada official described a case in which significant capital funds from his department were seized by a creditor, a situation that might not have arisen had INAC officials informed them that the First Nation had been put under TPM.

Other federal governments departments that could be consulted more by INAC regional offices are the Department of Justice if there are election disputes; the Canada Mortgage and Housing Corporation if there are issues with social housing; the Department of Fisheries and Oceans, Agriculture and Agri-Food Canada or Natural Resources Canada if there are issues with economic development activities; the Department of Public Safety if there are policing issues; and Human Resources and Skills Development Canada if there are issues of employment, skills development or community development.

Quebec region has a formalized interdepartmental approach as part of their external review committee that also includes other federal departments, the provincial government, and First Nations. Manitoba and Yukon regions share information on intervention among the federal family in their rederal councils. In other regions, coordination is more informal and there are difficulties sharing information and no common approach. In one of our case studies under third party management, there was no coordination among the different federal government departments outside of INAC and Health Canada, even though those other departments were providing a lot of funding to the First Nation. On the other hand, one of our case studies included an interdepartmental initiative to address underlying causes and contextual factors.

There was almost no coordination with provincial governments outside of exceptional circumstances - for example, intervention in child and family service agencies that are regulated by the province; or joint audits of social welfare services under the INAC-Ontario agreement. Provincial governments are however affected if First Nations are not paying for tuition in provincial schools or for provincial water, electricity or telecommunications services.

Collaboration with financial institutions varied across the regions and by recipient. Financial institutions would like to be informed in a more timely way if third party management is imposed and to have a creditors meeting with the third party manager as soon as possible thereafter. In a few cases, INAC, the First Nation and the financial institution are working together on a common solution.

7.7 Best practices and lessons learned

What are the best practices and lessons learned?

The following best practices or lessons learned were noted by key informants:

7.8 Conclusions related to Delivery

Return to Table of Contents 






8. Evaluation Findings - Improvements and Alternatives

8.1 Improvements to the Intervention Policy

Are there improvements that could be made to better achieve Intervention Policy objectives?

The main improvements to the Intervention Policy that have been suggested by key informants relate to prevention, revised triggers, capacity development, and strengthening all three levels of intervention. It was also suggested that there needs to be other strategies to address issues, such as governance disputes and program management. These improvements are related to those underlying causes over which the Intervention Policy has some influence.

Prevention and early detection

In terms of prevention and early detection, interviewees suggested that it be linked to the general assessment that will be conducted as part of the new Policy on Transfer Payments (PTP). The PTP requires that transfer payment programs be designed, delivered and managed in a manner that takes account of risk and clearly demonstrates value for money. In addition to grants and contributions, Aboriginal recipients may also be eligible to receive fixed, flexible or block contribution funding. All recipients are to undergo a capacity and risk assessment and the results will influence the duration of funding agreements, the monitoring regimes, the frequency and type of reporting, the frequency and amount of payments, and the audit arrangements. The results will also identify possible capacity strengthening opportunities.[Note 43]

INAC has developed a new tool to conduct recipient risk assessments. Recipients will be categorized as high, medium or low risk based on eight potential risk factors: external factors or extraordinary events; major changes being undertaken by the recipient; the status of remedial plans; governance; planning; entity management (finance and administration); program management; and project management. The general assessments are intended to provide a more formalized process to identify emerging risks and implement risk mitigation strategies before intervention is required. They can also be used to inform the development of capacity building for recipients at risk. For those recipients in the high risk category, a more in-depth assessment would be required to determine capacity building requirements.[Note 44]

General assessments is planned to be conducted in the fall of 2010 for the 2011-2012 funding arrangements and annually, thereafter. They will be completed by the FSO, reviewed by a regional risk management committee, and discussed with the recipients.

Triggers

In terms of the triggers, the default on terms and conditions is considered by some officials to be too broad. They point out that for administrative defaults such as late reports, INAC can hold back the next tranche of funding. Although the evaluation team have focussed in the evaluation on the three levels of intervention (recipient-managed, co-managed and third party managed), the Intervention Policy does provide for other actions to be taken such as withholding funds, terminating the agreement, or requiring or taking any reasonable action. From that perspective, therefore, there needs to be a trigger that is sufficiently broad to cover a variety of circumstances that may arise and provision for a variety of responses that are not related to the operating deficit or adverse audits.

In terms of the eight percent trigger, First Nations are required to use the new Generally Accepted Accounting Principles for local governments in preparing their financial statements from April 1, 2010, moving from cash to accrual accounting. The eight percent cumulative operating deficit will, therefore, no longer exist as it is currently defined. A paper prepared for INAC has recommended the use of three ratios and a review of operating results (annual surplus or deficit) as a means to assess the financial health of a First Nations. The three ratios are: liquidity, net debt to total annual revenue, and debt servicing to total annual revenue. Taken together, the three ratios and the operating results should better indicate whether the First Nation can afford the goods and services provided; has a sufficient level of working capital, has the ability to settle the liabilities it has incurred, and can pay for the debt it has acquired.[Note 45]

Capacity building

INAC officials have suggested that capacity building be linked to a broader assessment of needs and a wider range of tools and mechanisms. A team of people (or special assistance teams) could assist with the assessment, the determination of what support is required, and the delivery of the support. This team of people could include consultants, peers or others such as AFOA.

This type of approach would be consistent with INAC's Community Development Framework, which proposes a differentiated approach to First Nations that recognizes their diversity. Targeted investments and supports are to be provided to build capacity and move communities out of intervention and stabilize fiscal management. The general assessment referred to above will assist in identifying where First Nations are on a continuum of development.[Note 46]

Capacity building could therefore be targeted at a range of potential underlying causes in our Causal Framework (Appendix J), including:

It was also suggested that training on the Intervention Policy be provided to INAC Funding Service Officers to increase consistency and improve monitoring.

Strengthening the Levels of Intervention

INAC regional officials suggested that co-management could be strengthened by INAC approving the selection of the co-manager based on the criteria in the Intervention Policy, as well as the co-management agreement to ensure that the co-manager has the authority to co-sign financial instruments. It was also suggested that INAC fund all or part of the costs of co-managers in order to increase the willingness of recipients under co-management and retain sufficient band staff to be trained.

INAC officials agreed that while third party management was undesirable, it was occasionally unavoidable, and should be reserved for situations where there is unwillingness or a total lack of capacity or where creditors are threatening to garnishee First Nation funds. Third party managers should, however, be supported by other capacity building or other measures that are separately contracted, with separate program funding. It was also suggested that INAC fund all or part of the costs of third party managers in order to retain sufficient band staff to be trained.

Communication and coordination within the federal family could also be improved and formalized as part of a more common approach to recipients.

Overall, regional officials made a strong case for continuing to provide regions with enough flexibility to deal with different circumstances in the most appropriate way, under a consistent policy framework.

8.2 lternatives to the Intervention Policy

Alternatives to the Intervention Policy

A number of other alternatives were suggested in the interviews or the literature review. These alternatives address other underlying causes or contextual factors than those mentioned in the preceding section. They relate to other policies or programs of INAC and are beyond the scope of the Intervention Policy, but affect the success of the policy.

In terms of improving governance and community engagement, the alternatives include:

In terms of improving INAC's funding practices or the funding regime, the alternatives suggested include:

Other longer-term alternatives suggested were:

An Alternative Approach

Our case studies and the literature review provide models or inspiration for a new intervention approach. This alternative approach could be used for the more intractable cases that have been under intervention for a long period of time with little hope of exiting any time soon. These intractable cases face a number of contextual factors that the Intervention Policy cannot address. The alternative approach would be holistic and long term and encompasses community rebuilding.

In one of our case studies in northern Manitoba, the INAC regional office, other federal departments and the province are working with the First Nation Council on a joint committee to address a set of priorities determined by the First Nation. The initial starting point was to deal with a growing number of youth suicides through the redevelopment of a partially completed community arena that will include a youth centre. (This partially completed arena was one of the principal causes of the community's financial difficulties that led to intervention.) Two federal departments in collaboration with the province and the First Nation were able to develop a solution complete the arena and develop the new youth centre. The joint committee is prepared to assist the First Nation with other community priorities, such as the development of a written election code.

An alternative approach that emerged from our literature review – a northern pilot project - provides additional insights. Like the Manitoba case study, it focuses on youth and has a similar underlying structure – a steering committee of officials from federal departments, the territorial government, Inuit organizations and the Town Council. It goes several steps further, however. First, the community has two important structures: i) a youth council on which a member of the Hamlet Council sits as an ex officio member; and ii) an interagency co-ordinating committee composed of front line workers from the hamlet, territorial and federal governments. Both of these structures help set priorities and are critical sources of information and advice for the steering committee. In addition, there is a hired youth worker who knows the community well but is not from the community. His job is to support youth in developing their increasingly important role in the community. Finally, the federal players are committed to developing a single window through which to funnel federal funding to reduce the reporting burden and alleviate cash flow issues.

For the past three years, annual expenditures on the northern pilot project have been in the range of $400k to $650k. Initiatives included the revitalization of a youth centre and its governance by youth, a small engine repair course, recreation activities, suicide prevention programming, cultural pursuits, education initiatives (e.g. a peer helper program for all grades), and youth carpentry programs. The next priority is crime prevention to counter, among other things, the influence of illegal drugs and alcohol.

The three year pilot phase produced mixed results. On the one hand formal and informal evaluations indicated positive impacts on youth (reduced crime, the growing importance of the Youth Council as a major force in the community, and evidence of a greater sense of pride and self-esteem among youth). On the other hand, the project failed to achieve a single window approach to federal programming support, resulting in some considerable hardship for the community (reporting issues, cash flow problems etc.). Furthermore, the project was under-resourced – i.e. a side project of various individuals working in different federal departments.

After some debate, the Hamlet Council agreed to extend the pilot for an additional three years. Changes to the initiative include the following:

In the literature review, examples in the fragile state literature with remarkable parallels to the two Canadian examples cited above, have been found, but on a larger, country-wide scale. Like the Canadian examples, a central feature is a co-ordinating mechanism for all of the aid agencies supplemented by an intervention team with high credibility among all of the players in the country. This co-ordinating mechanism works directly with the Government but draws in other elements in the country as well. Of note is the use of a comprehensive assessment of the conditions in the country shared by all of the members on the co-ordinating committee. As well, there is a significant emphasis on physical security and on training participants throughout the country in conflict resolution techniques. Finally, the literature suggests collecting baseline data on which to base an evaluation of the intervention.

8.3 Conclusions related to Improvements and Alternatives

Return to Table of Contents 






9. Conclusions and Recommendations

9.1 Conclusions

Program Relevance

The Intervention Policy remains relevant and there is a continued need to ensure that essential services are delivered, accountability to INAC is maintained, deficiencies in financial management and administration are addressed, and the capacity of recipient to provide services is enhanced. It is also consistent with federal government and INAC policies and priorities and the departmental strategic outcomes. As for federal roles and responsibilities, the federal government is not responsible for First Nations' debt. However, the Intervention Policy supports the federal government's other roles and responsibilities. INAC's Intervention Policy doesn't overlap or duplicate with other programs, policies or initiatives delivered by other stakeholders. In fact, it has served as a model for others stakeholders that deliver services to First Nations and is similar to the intervention approaches in other jurisdictions.

Program Success/Performance

There has been some recent progress in reducing the incidence of intervention but not the level and duration of intervention. A number of underlying causes related to poor governance, poor management, community disengagement and poor funding practices affect the long-term success of the Intervention Policy, in addition to certain contextual factors. The Intervention Policy is able to address only some of these factors, and progress across the regions varies partly because of differences in these factors.

The Intervention Policy has been successful in ensuring the ongoing delivery of essential services, maintaining the health and safety of band members, and ensuring the accountability of recipients to INAC. It has been less successful in ensuring the accountability of recipients to their members and building the capacity of recipients to exit and remain out of intervention. Third party management has the most negative consequences in terms of recipient accountability and community perception.

The Intervention Policy has positive unintended impacts in providing justification for political leaders to take tough political decisions, increase the awareness within the community or the council about financial problems and the required solutions, and an increase in support for those solutions. However, there are a number of negative unintended impacts, most notably on the availability of band support funding for governance and administration, and the negative consequences of third party management.

Key success factors to exit the intervention policy or remain out are mainly the willingness and a committed leadership, a stable and competent management, community engagement, and a positive external relationships – as well as access to significant own source revenue.

Efficiency and Economy (Cost-effectiveness)

The cost-effectiveness of the Intervention Policy could not be determined quantitatively because of the lack of comprehensive quantitative data. Similarly, the evaluation team could not compare the cost to other approaches because they also do not have comprehensive quantitative data. However, based on the limited success, it could be said that the policy could be more cost-effective. Costs include INAC's time to develop and deliver the policy, and the cost to recipients of co-managers and third party managers. The major cost drivers are the incidence, level and duration of intervention so that improving the results would decrease the costs. There is also a major cost associated with not intervening in a timely way.

Program Design

The Intervention Policy is not well designed to achieve all of its objectives as capacity development is lacking. The objectives, roles, responsibilities, and accountabilities of the policy are fairly well defined and the policy provides a consistent framework for use by the regions. INAC's Intervention Policy is similar to the intervention approaches of other jurisdictions in terms of objectives, triggers, levels of intervention and remedies, and oversight and monitoring. Other jurisdictions may, however, provide more capacity building and cover the costs of co-managers and third party managers. Provincial governments also have more ex-ante controls over the debt of their subnational entities (municipalities, school boards and hospitals).

Program Delivery

The Intervention Policy is being implemented as planned, with some gaps. The delivery processes for the Intervention Policy are fairly consistent across the regions, with some variations due to different contexts and different capacities of recipients and INAC staff. Some regions have a formal process for prevention and early detection. The performance of co-managers and third party managers varies, with co-managers better able to provide capacity building to recipients than third party managers.

The constraints that recipients face in implementing remedial measures vary; many of the problems require long-term solutions; and the context is constantly changing. INAC may also not have intervened early enough or escalated the level of intervention soon enough, and there are few tools available to enforce compliance by recipients.

The system for capturing the results of the Intervention Policy is not user friendly or useful for reporting and monitoring purposes at a national level. Policy guidance and tools exist at a national level and in some regions; and performance monitoring of implementation is taking place. Communication and coordination with other federal government departments and other key players are not optimized.

INAC is addressing previous audit recommendations through action plans and the development of a revised intervention policy and directive and there are a number of best practices in regions related to different aspects of the policy.

Improvements and Alternatives

The major improvements that have been suggested to the Intervention Policy relate to improved prevention; a revised trigger for financial health; a broader assessment of capacity needs and a more varied capacity building response that responds to some of the underlying governance, management and community causes; and a strengthening of the co-management level of intervention. Alternatives that have been suggested address other underlying causes, including INAC's funding practices and are outside of the scope of the Intervention Policy. Another alternative is a more holistic, long-term approach that addresses a number of underlying causes as well as some of the contextual factors.

9.2 Recommendations

It is recommended that INAC:

  1. Implement prevention and early detection strategies to prevent First Nations going into intervention status or escalating to a more serious level of intervention. Activities should include:

    1. Better identification of financial and governance capacity gaps and needs (linked to the general assessment);

    2. Better and broader identification of triggers for third party management and co-management;

    3. Identification of incentives for third party managers to build First Nations capacity, to be written into agreements with third party managers;

    4. Development of questions to assess properly key success factors to analyze trends in relation to escalation and de-escalation (consider undertaking community surveys to better assess community capacity factors for success); and

    5. Improve communication and coordination at the national and regional level with other federal government who deliver services and programs to First Nations.

  2. It is recommended that:

    1. Third party managers be prequalified and assessed against performance criteria;

    2. INAC audit co-management and third party management arrangement on a risk basis; and

    3. Revise the third party management agreement to request participation of third party managers in evaluations as well as in audits.

  3. The proposed new Default Prevention and Management Policy should address the design and delivery gaps of the current Intervention policy by:

    1. Implementing national tools and formalized processes in the assessment of First Nations;

    2. Clarifying FSO roles and responsibilities, developing job descriptions that identify competencies and knowledge needed, and identifying training to meet requirements; and

    3. Clearly communicate the new policy and assessment processes to stakeholders and First Nations.

  4. Develop and implement better monitoring and reporting systems that involve:

    1. A performance measurement strategy to allow for meaningful reporting through quarterly progress reports and the Departmental Performance Report. Indicators such as duration, incidence and level of intervention, and level of implementation of Remedial Management Plan could be considered;

       
    2. A cost tracking system at the Headquarters and regional level to capture cost data in order to measure cost effectiveness and inform future decisions; and

    3. Re-design the intervention policy module in FNITP that will involve a revision of the inputs, processing and output (reports) of the system at the regional and national level in order to make it more user-friendly, reliable and to inform performance measurement strategy to be developed in line with the new policy.
Return to Table of Contents 






Appendix A - Revised Evaluation Matrix

Revised Evaluation Matrix [Note 47]

Evaluation Question Methodology
  Document review Literature review Data Analysis KI Interviews HQ KI Interviews Regional Offices KI Interviews Recipients KI Interview OGDs KI Interviews Other Case Studies
(1) Program Relevance:
1.1 Is the Intervention Policy responding to actual needs?    
1.2 To what extent does the IP align with a) federal government priorities, b) with the Departmental Strategic outcomes?          
1.3 Does the program duplicate or overlap with other programs, policies, or initiatives delivered by other stakeholders? (including other Federal Departments)          
1.4 Does the IP align with the Federal Government role and responsibility?          
(2) Program Success/Performance: Is the IP achieving intended outcomes?
2.1 What are the intervention trends?          
2.2 What are the underlying causes?  

2.3 Is the IP successful in achieving its objectives and its intended results? Does this vary regionally?

a) To what extent does the IP reinforce the recipient’s accountability?

b) Does the IP contribute to building local capacity in order to deliver higher quality services?

c) Does IP contribute to the continuity of service delivery, appropriate use of funds?
 
2.4 What are the unintended impacts of this policy?    
2.5 What are the key success factors?  
(3) Cost-effectiveness: Is the IP cost-effective?
3.1 What are the costs of intervention for the recipient and INAC?      
3.2 How does this cost compare to the cost of other intervention approaches?          
3.3 Do recipients and INAC believe that they have obtained value for money?
  • Are the costs of remedying the default and building capacity minimized?

  • Could the same results be achieved with less money?

  • Are the most appropriate and efficient means being used to achieve the IP’s outcomes?
  • Is the process in place economic, efficient and effective?
 
(4) Program Design: Is the IP well designed?
4.1 To what extent are the objectives of the IP clearly articulated and understood?    
4.2 To what extent are the roles, responsibilities and accountability of key players clear and well understood? Is there a clear understanding of terminology used?        
4.3 To what extent does the design of the IP lead to consistent national delivery/results?        
4.4 To what extent does the IP design compare with other intervention approaches (e.g. OGDs, provincial governments, other jurisdictions, financial institutions, etc.)?
4.5 Is the IP well designed to achieve its objectives?
(5) Program Delivery: Is the IP being delivered effectively? 
5.1 Is the IP implemented as planned?    
5.2 Are there constraints or pressures that impeded the implementation of IP?    

5.3 To what extent do means and tools exist for:

a) obtaining reliable evidence of consistent policy implementation;

b) success/performance (including indicators).
   
5.4 To what extent is good guidance and monitoring in place?     *  
5.5 To what extent have recommendations from past audits been implemented as planned?            
5.6 What is the extent of coordination for delivery within INAC, with OGDs, with provincial governments, with others?     
5.7 What are the best practices and lessons learned in terms of intervention?    
6) Improvements and Alternatives
6.1 Are there improvements that could be made to better achieve IP objectives?  
6.2 Are there alternatives to better achieve IP objectives?  

* For other government departments, the response to these two questions relate to their own mechanisms and tools for performance monitoring and guidance.

Return to Table of Contents 






Appendix B - Draft Intervention Policy Framework Logic Model

Draft Intervention Policy Framework Logic Model

The charts depicts the draft intervention policy logic model. The model illustrates the links between the inputs, activities, outputs, short-, medium- and long-term outcomes, and ultimate outcomes. Its purpose is to show the expected effects of the Policy's activities and outputs.

Inputs

The inputs include the A-Based/B-Based budget, human resources, funding arrangement agreements (T&C), the Financial Administration Act and Regulations, national policies, compliance guidelines and procedures, and procurement guidelines. All of the activities in the logic model use the same inputs.

Activities

There are three types of activities:

  1. Service procurement – identifying and selecting co-manager or third-party manager;

  2. Compliance monitoring – determining whether the recipient should be placed under the Intervention Policy (triggers, T&C), and assessing the level of intervention required (performance review model);

  3. Capacity-building – delivering the Professional and Institutional Development program (capacity development initiative).

Outputs

The activities have four outputs. The outputs and inputs are linked as follows:

Short-term (or immediate) outcomes

Three short-term outcomes are expected:

  1. Accountability and program services delivery maintained during problem situation with co-manager or third-party manager reporting;

  2. Remedial management plan developed;

  3. Capacity-building strategy developed to have recipients administer own funding

More specifically:

Medium-term (intermediate) outcomes

The short-term outcomes lead to two medium-term outcomes.

The medium-term outcome Terms and conditions of funding agreement commitment met is the result of achieving short-term outcome 1 - Accountability and program services delivery maintained during problem situation with co-manager or third-party manager reporting.

The medium-term outcome Recipient capacity (skills, abilities and knowledge) to manage finances, to be accountable and to prevent future intervention enhanced (performance improvement) is the result of achieving short-term outcome 2 – Remedial management plan developed, and short-term outcome 3 – Capacity-building strategy developed to have recipients administer own funding.

Long-term outcomes

The medium-term outcomes lead to three long-term outcomes.

The long-term outcome Effective and responsibly managed and operated programs and services funded by INAC is the result of achieving medium-term outcome Terms and conditions of funding agreement commitment met.

The two long-term outcomes Intervention level de-escalated and Number of recipients under intervention reduced are the result of achieving the medium-term outcome Enhanced recipient capacity (skills, abilities and knowledge) and Capacity-building strategy developed to have recipients administer own funding (performance improvement).

Ultimate Outcomes

Achieving the three long-term results leads to the ultimate outcome: Good governance and effective institutions.

 
Return to Table of Contents 






Appendix C - Remedial Management Plans

RegionRecipient-managed RMPsCo-managed RMPsThird party managed RMPsNo. Of Case Studies Included in Sample
Atlantic 5 FNs 5 FNs 0 0
Quebec 5 FNs 5 FNs 0 1 FN
Ontario 5 FNs 3 FNs 2 FNs 1 FN
Manitoba 2 FNs 6 FNs 3 FNs 4 FNs
Saskatchewan 5 FNs 4 FNs 1 FN 2 FNs
Alberta 2 FNs + 2 Other Orgs. 1 FN 1 FN 1 FN + 1 Other Org.
BC 5 FNs 1 FN 0 1
Yukon 2 FNs 0 0 N/A
Total 31 FNs + 2 Other Orgs. 25 FNs 7 FNs 10 FNs + 1 Other Org.
Grand Total 63 FNs + 2 Other Organizations
Return to Table of Contents 






Appendix D - Summary of Data Analysis

PlannedAchieved
(Differences Highlighted)
INAC
Period covered:
April 1, 1999 to March 31 2010 (financial)

Census 1991, 1996, 2001 and 2006 (demographic)
Period covered: 
April 1, 1999 to March 31 2010 (intervention)
April 1, 2004 to March 31, 2009 (financial)
Census 1991, 1996, 2001 and 2006 (demographic)
Statistics on intervention: 
Total number of recipients
Number of Recipients under intervention – FNs, Other
Level of intervention: RMP, CM, TPM
Duration: No. of years under some form of intervention
By year, by recipient, by region, nationally
Statistics on intervention: 
Total number of recipients
Number of Recipients under intervention – FNs, Other
Level of intervention: RMP, CM, TPM
Duration: No. of years in the past 11 years under some form of intervention 
By year, by recipient, by region, nationally
Statistics on recipients: (ALL)
FNITP Band No.
Name
Population (size) on reserve (latest available)

Population Growth Rate on reserve (latest available)
Remoteness (geographic zone)

Type of funding arrangement (CFA, CFNFA)

CWB Index (last two Censuses, components and total)
Change in CWB Index (%)

Total INAC funding (latest year)
Funding per capita (on reserve population)
Dependence on INAC Funding (% total consolidated revenue, latest year)

Cumulative deficit (latest year)
Water assessment rating (if available)

Frequency of elections/change in leadership (if available)
Statistics on recipients: (ALL)
FNITP Band No.
Name
Registered population (size) on reserve (2004/05-2008/09)
Population Size in CWB (1996, 2001, 2006)
Population Growth Rate on reserve (1996-2001, 2001-2006)
Remoteness (geographic zone)
Community size – Registered population 2009/2010
Type of funding arrangement (CFA, CFNFA)
(TPM cases only)
CWB Index (last three Censuses, components and total)
Change in CWB Index (Value, 1996-2001, 2001-2006)
Total INAC funding (2004/05-2008/09)
Funding per capita (on reserve population)
Dependence on INAC Funding (% total consolidated revenue) (not reliable or complete)
Increase in INAC funding (2004/05-2008/09)
Cumulative deficit (2004/05-2008/09)
Water assessment rating (not available for total population)
Frequency of elections/change in leadership (not available for total population)
Election system (Indian Act or Custom)
Aggregated statistics on main trigger: 
Level of indebtedness: % cumulative deficit By year, by region, nationally
Aggregated statistics on main trigger: 
Level of indebtedness: % cumulative deficit By year, by region, nationally
Trend Analysis:
  • incidence
  • level
  • duration
Trend Analysis:
  • incidence
  • level
  • duration
  •  IP composite index (level x duration)
Correlations: 
Recipient statistics with Intervention level and duration (see separate table on Data Analysis)
Correlations:
Some recipient statistics with Intervention level and duration (see separate table on Data Analysis)
Cost of Intervention
Standing Offers
  • by amount contracted
  • fee rates for categories of consultants
Cost of Intervention
TPM costs and total INAC funding, 2002/03 to 2006/07
FNIHB
Period covered: 
April 1, 2000 to March 31 2010 (financial)

Statistics on intervention:
 
Band Number of Recipients under intervention
Name of Recipients under intervention
Total number of recipients
Number of recipients under some form of intervention
Level of intervention: RMP, CM, TPM
Duration: No. of years under some form of intervention
By year, by recipient, by region, nationally

Trend Analysis: incidence, level and duration

Correlation: by recipient with INAC's incidence and level
No national data available except on TPM. No regional data on recipient-managed intervention.
Return to Table of Contents 






Appendix E - Summary of Statistical Analysis

Relation
between
variables
Statistical
tools
Comment
CWB and the number of years under some form of intervention in the past 11 years. Anova Some relationship – i.e. the lower the CWB, the higher the number of years under intervention
CWB and the level of intervention Anova Significant relationship between the CWB and whether or not a First Nation will be under intervention. Not a clear relationship between the CWB and the level of intervention.
CWB components and the number of years under intervention Complex analysis – Multi Anova Some relationship except for education where the relationship is weak.
CWB components and the level of intervention Anova Significant relationship for the 4 components and whether or not a First Nation will be under intervention.
Not a clear relationship between the 4 components and the level of intervention.
Δ CWB 96-01
Δ CWB 01-06
and no. of years
under intervention
Correlation Due to the complexity of the analysis, not able to perform.
Δ CWB 96-01
Δ CWB 01-06
and composite of
no. of years under
intervention and level
of intervention
Complex analysis Composite not possible. Proof that CWB related to intervention but not level (see above).
Community size and the number of years under intervention Correlation No correlation
Community size cluster and the number of years under intervention Anova Not relevant
Option 1 Option 2 Not relevant. Not relevant
Very small: 0 – 250
Small: 251 - 750
Medium: 751 – 1500
Large: more than 1500
Small: less than 500
Medium: 501 – 1500
Large: More than 1500
 
Δ Pop 96-01
Δ Pop 01-06
and the number of years under intervention Correlation As no correlation with deficit, not relevant
Δ Pop 96-01
Δ Pop 01-06
and level of intervention Correlation As no correlation with deficit, not relevant
IP Composite Index and Geographic Zone Log e Composite not possible
GeoZone and the number of years under intervention Anova Significant relationship.
Remoteness has an impact.
GeoZone and the level of intervention Chi-Square Issue with analysis – Fischer analysis not possible
Δ Pop 96-01
Δ Pop 01-06
and the Cumulative Deficit Correlation Not correlated
Δ Pop 96-01
Δ Pop 01-06
and the Deficit ratio Correlation Not correlated
Election System and the level of intervention Chi Square Not correlated
Election System and the IP Composite Index Anova Composite not possible
Dependency % of unqualified BAND and the number of years under intervention Correlation Dependency not relevant – instead, used increase in INAC transfer payments. No correlation
Return to Table of Contents 






Appendix F - Summary of Interviews

Interview Category/GroupNumber
Planned
Actual
Number
INAC Headquarters 9 9
Chief Financial Officer Sector
  • Chief Financial Officer
  • Transfer Payments & Financial Policy Directorate
  • Compliance/Quality Assurance
  •  FNITP
4 4
Regional Operations Sector
  • Sr. Assistant Deputy Minister
  • Operations & Implementation Directorate
  • Governance Branch 
3 3
Policy and Strategic Direction 1 1
Legal Counsel 1 1
INAC Regional Offices(i.e.Regional/Associate Regional Director General, Funding Services/Field Services/First Nations Relations, Corporate Services)
  • Atlantic Regional Office
  • Quebec Regional Office
  • Ontario South Regional Office
  • Ontario North Regional Office
  • Manitoba Regional Office
  • Saskatchewan Regional Office
  • Alberta Regional Office
  • British Columbia Regional Office
  • Yukon Regional Office
  • Northwest Territories Regional Office
10 10
Other Federal Government Departments
  • Health Canada
  • Public Safety Canada
  • Office of the Federal Interlocutor
  • HRSDC
3 4
Other Organizations/Individuals
  • Assembly of First Nations
  • Aboriginal Financial Officers Association (3 – National, Alberta, BC)
  • National Centre for First Nations Governance
  • First Nations Financial Management Board
  • Canadian Executive Services Overseas
  • Financial Institutions (3)
  • Former Director of Finance, INAC
11 5
Case Studies (refer also to Appendix H) 89 53
First Nation Chiefs, band managers, managers 24 12
Other Indian Administered Organizations 3 3
Third party managers or co-managers 9 4
Funding Service Officers 28 28
Regional Health Canada officials 8 4
Provincial government officials 8 2
Regional police service 8 2
Grand Total 122 82
Response Rate 67%
Return to Table of Contents 






Appendix G - Summary of Case Study Sample

RegionCasesRecipient
Interviews
Atlantic 1 FN chronic RMP 
1 FN chronic CM/TPM 
Total: 2 FNs
Chief
Chief
Total: 2 FNs
Quebec 1 FN other success case
1 FN chronic CM/TPM 
Total: 2 FNs
None
None
Total: 0 FNs
Ontario 1 FN success case
2 FN other success cases
1 FN chronic RMP
2 FN chronic CM/TPM 

1 Other Organization

Total: 6 FNs; 1 Other
None
None (both)
None
  1. Chief, Councillors, Band Administrator (site visit)
  2. Co-Manager
Deputy Grand Chief, Executive Director, Program Manager
Total: 2 FNs, 1 Other
Manitoba 1 FN success case
1 FN other success case
1 FN chronic RMP 
2 FN chronic CM/TPM

Total: 5 FNs
Chief
Chief
Councillor
  1. Chief, Councillors, Band Manager (site visit)
  2. Chief
Total: 5 FNs
Saskatchewan 1 FN success case
1 FN other success case
1 FN chronic RMP 
2 FN chronic CM/TPM 

1 Other Organization
Total: 5 FNs; 1 Other
None
Band Administrator
None
  1. Co-Manager and program staff (site visit)
  2. Chief, former TPM
Executive Director
Total: 3 FNs, 1 Other
Alberta 1 FN other success case
1 FN chronic CM/TPM 
1 Other Organization
Total: 2 FNs; 1 Other
Band Manager
None
Director, Finance Director
Total: 1 FN, 1 Other
BC 1 FN other success case
1 FN chronic RMP 
Total: 2 FNs
None
None
Total: 0 FNs
Total 24 FNs
  • 3 Success Cases
  • 7 Other Success Cases
  • 5 Chronic RMP
  • 9 Chronic CM/TPM
3 Other Organizations
13 FNs
  • 1 Success Case (33%)
  • 3 Other Success Cases (43%)
  • 2 Chronic RMP (40%)
  • 7 Chronic CM/TPM (78%)
3 Other Organizations (100%)
Return to Table of Contents 






Appendix H - Data Tables

Table 1 - Number of First Nations Under Intervention

Level 1999/2000 2000/2001 2001/2002 2002/2003 2003/2004 2004/2005 2005/2006 2006/2007 2007/2008 2008/2009 2009/2010
RMP 92 90 103 101 100 91 94 83 91 84 81
CM 36 43 44 53 49 58 57 62 66 65 64
TPM 15 30 29 29 37 31 29 29 24 26 23
Total 143 163 176 183 186 180 180 174 181 175 168

Source: Intervention data extracted from FNITP as of March 31st in each year, and validated by EPMRB with INAC's Regional Offices

 

Table 2 - Level of Intervention As A Proportion of the Total Number of First Nations Under Intervention

Level 1999/2000 2000/2001 2001/2002 2002/2003 2003/2004 2004/2005 2005/2006 2006/2007 2007/2008 2008/2009 2009/2010
RMP 64% 55% 59% 55% 54% 51% 52% 48% 50% 48% 48%
CM 25% 26% 26% 29% 26% 32% 32% 36% 36% 37% 38%
TPM 10% 18% 16% 16% 20% 17% 16% 16% 13% 15% 14%
Total 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100%

Note: Because of rounding, percentages may not add up to 100%

Table 3 - Incidence and Level of Intervention by Region, as of March 31, 2010

Level of
Intervention
Atlantic Quebec Ontario Manitoba Saskatchewan Alberta BC Yukon NWT Total
RMP 9 6 15 14 13 4 12 2 6 81
CM 5 7 15 16 15 4 1 0 1 64
TPM 1 1 7 5 5 1 2 0 1 23
Total 15 14 37 35 33 9 15 2 8 168
Total No. Of FNs 33 39 127 62 70 44 198 26 16 615
% Total 45% 36% 29% 56% 47% 20% 8% 8% 50% 27%

Source: Intervention data extracted from FNITP as of March 31, 2010.

Table 4 - Number Of Other Organizations Under Intervention

  1999/2000 2000/2001 2001/2002 2002/2003 2003/2004 2004/2005 2005/2006 2006/2007 2007/2008 2008/2009 2009/2010
Total 3 6 6 5 5 4 4 4 10 9 7

Source: Intervention data extracted from FNITP as of March 31st in each year, and validated by EPMRB with INAC's Regional Offices

Table 8 - Trend in the Number of First Nations with a Cumulative Operating Deficit > 8% and the Size of the Deficit

  FY 2004-2005 FY 2008-2009 %
Change
Regions # FNs Amount $/FN # FNs Amount $/FN
Atlantic 9 -$41 969 486 -$4 663 276 10 -$49 034 852 -$4 903 485 5.15%
Québec 18 -$65 876 340 -$3 659 797 11 -$54 391 085 -$4 944 644 35.11%
Ontario 31 -$62 619 526 -$2 019 985 20 -$32 670 348 -$1 633 517 -19.13%
Manitoba 34 -$102 879 323 -$3 025 862 21 -$70 622 326 -$3 362 968 11.14%
Saskatchewan 23 -$35 146 865 -$1 528 125 11 -$18 265 810 -$1 660 528 8.66%
Alberta 6 -$10 564 698 -$1 760 783 6 -$17 696 227 -$2 949 371 67.50%
BC 30 -$20 863 380 -$695 446 13 -$12 033 091 -$925 622 33.10%
NWT 6 -$3 484 183 -$580 697 5 -$1 712 995 -$342 599 -41.00%
Total National 157 -$343 403 801 -$2 187 285 97 -$256 426 734 -$2 643 575 20.86%

Source: FNITP financial data.
Note: This report reflects information entered into the FNITP Statement of Revenue and Expenditure where the Audit Reporting Requirement "Approved" has been completed.

 
Return to Table of Contents 






Appendix I - Causal Factors Framework

Immediate Causes of Financial DifficultyUnderlying Causes
(causes that the IP can
address are highlighted)
Contextual Factors that
Shape Underlying Causes
  • Economic development (e.g. failed businesses)

  • Housing (overruns and new pressures from new home construction; no or poor rent collection; construction delays, mould)

  • Other capital projects (unfunded, cost overruns, delays)

  • Unfunded and non-recoverable expenditures on social assistance or other programs

  • Overruns in elementary and secondary education; decline in enrolment

  • Uncollected loans or advances to members & employees

  • Overruns on honoraria and travel costs of chief and council

  • Travel costs related to funerals

  • Financing court processes

  • Easy access to credit

  • Natural disasters – floods, fire, ice road issues

  • Unpaid HST/GST and tax remittances

  • Over-reliance on fluctuating OSR

  • Denial or adverse audit report

  • Fraud, corruption, nepotism

TPM Factors

  • Breach of terms & conditions (e.g. refusal to provide consolidated financial statements)

  • Refusal to sign a funding agreement

  • A sizable deficit and the risk of creditors seizing INAC's funds

  • A serious election dispute (i.e. no legitimate leadership in place)  
Poor governance
  • Disputed elections (legitimacy issues)

  • Leadership (lack of skills, knowledge, experience, health issues)

  • Leadership instability (no shared community vision)

  • Leadership – family v. whole of community orientation

  • Leadership – intimidation by gangs

  • Inappropriate political interference e.g. portfolio system

  • Poor accountability to members, funders

  • Few laws, policies

  • Ethical issues (conflict of interest etc.)

  • Unproductive relationships with external bodies (funders, business etc.)

  • Poor dispute resolution, redress

  • Low engagement with community

     
  • Other
Poor management
  • Lack of skills, experience

  • Instability (high turnover)

  • Lack of tools (IT systems)

  • Lack of policies, plans, budgets

  • Lack of basic understanding of money management

  • Poor information to Council
Community disengagement
  • Apathetic, low expectations of government

  • Lack of knowledge about government

  • Fear
Poor funding practices
  • Inappropriate demands, reporting requirements, late payments

  • Poor risk analysis

  • Poor monitoring

  • Underfunding e.g. isolated communities

  • ‘Dumping' money at year end leading to cost overruns

  • Poor co-ordination among federal family
Community endowments
  • Few natural resources

  • Geographic isolation

  • Lack of economic opportunities; low own source revenue

  • Dominance of public sector; small or non-existent private sector

  • Rapid population growth (especially if generated by teen pregnancies)

  • Small size – negative effects on capacity
Community member endowments
  • Low education levels

  • Significant health issues (TB, HIV/AIDS, FASD, mental health)

  • Lost cultural identity
Social cohesion and security
  • Fierce competition among families, groups (low social cohesion)

  • Psychology of victimization

  • Insularity (few external relationships)

  • High crime rates (gangs, gender violence, vandalism)
Return to Table of Contents 






Appendix J -Potential Indicators

As per evaluation and research

General

Leadership

Deficit/ Indebtedness/ Financial Ratio

Remedial Management Plans

Intervention Policy

Operations

Return to Table of Contents 






Footnotes:

  1. Please refer to the note in page v. In the following pages the expression First Nations under Intervention Policy will be used (return to source paragraph)

  2. Indian and Northern Affairs Canada, Management Framework 2006-2007, January 2007, p. 2. (return to source paragraph)

  3. Indian and Inuit Management Development was subsequently merged with Gathering Strength-Professional Development Program into the unified Professional and Institutional Development (P&ID) Program. One of the objectives of P&ID is to build sufficient and sustainable capacity that facilitates effective, efficient and culturally appropriate community government. (return to source paragraph)

  4. Indian and Northern Affairs Canada, "Backgrounder - Renewal of INAC's Intervention Policy" (return to source paragraph)

  5. Indian and Northern Affairs Canada, "Chapter 5.11 – Funding Arrangements: Intervention Policy", Financial Policies and Procedures Manual, updated 5/15/2008, Section 1.0. (return to source paragraph)

  6. Nunavut Region was excluded because there are no First Nations or tribal councils in Nunavut. (return to source paragraph)

  7. PAWS - Predictive Analytics SoftWare Statistics, former know as SPSS. (return to source paragraph)

  8. Indian and Northern Affairs Canada, Performance Report for the period ending March 31, 2009, p.18 (return to source paragraph)

  9. Treasury Board Secretariat of Canada, "Accounting Standard 3.2 - Treasury Board - Transfer Payments (Grants and Contributions)", (return to source paragraph)

  10. Treasury Board Secretariat of Canada, Directive on Transfer Payments, 1 October 2008. (return to source paragraph)

  11. Indian and Northern Affairs Canada, "Chapter 5.13 – Funding Arrangements: Credit and Solvency Policy", Financial Policies and Procedures Manual, Amendment 03, 2/10/07. (return to source paragraph)

  12. INAC, Report on Plans and Priorities 2009-2010, (return to source paragraph)

  13. There are a total of about 615 First Nations in Canada but some First Nations have been excluded from the analysis because they are under a self-governance agreement, have no funding arrangement with INAC, or receive too little funding to report to INAC. Total number of First Nations used for this purpose is 584 communities. (return to source paragraph)

  14. Institute On Governance, Special Study on Funding Arrangements Documentation and Literature Review, November 28, 2008, p. 29. (return to source paragraph)

  15. Office of the Auditor General of Canada, Chapter 10 Other Audit Observations, Report of the Auditor General of Canada to the House of Commons, November 2003 (return to source paragraph)

  16. Office of the Auditor General of Canada, Chapter 5 Management of Programs for First Nations, Report of the Auditor General of Canada to the House of Commons, May 2006 (return to source paragraph)

  17. Pour de plus amples renseignements sur les ententes de financement. (return to source paragraph)

  18. This model has been developed by the Institute On Governance and was used during the evaluation and some element has been verified. (return to source paragraph)

  19. The Community Well-Being Index was developed by INAC to help measure the quality of life of First Nations and Inuit communities in Canada relative to other communities and over time. It uses Statistics Canada’s Census of Population data to produce "well-being" scores of 0 to 100 for individual communities based on four indicators: education, labour force, income, and housing. (return to source paragraph)

  20. The 12 case studies cover all 7 of the Other Success Cases - First Nations that successfully exited intervention; 2 of the Chronic RMP cases – First Nations that have been under recipient-managed intervention for a long time, but in these two cases are about to exit intervention; and all 3 of the Other Indian-administered Organization Cases. Financial Policies and Procedures Manual, 6.0 Policy Statement, p. 190. (return to source paragraph)

  21. Indian and Northern Affairs Canada, "chapter 5.11 – Funding Arrangements: Intervention Policy", Financial Policies and Procedures Manual, 6.0 Policy Statement, p. 190. (return to source paragraph)

  22. Indian and Northern Affairs Canada, Final Report on the Companion Initiative on Capacity Development for Recipients under Intervention 2006-2007, Undated, p. 3 (return to source paragraph)

  23. Indian and Northern Affairs Canada, Professional and Institutional Development Directorate, 2008-2009 New Funding Pressures, Submission to the Financial Management Committee, February 2008, p. 12. (return to source paragraph)

  24. Ibid. (return to source paragraph)

  25. For more information on P&ID. (return to source paragraph)

  26. Indian and Northern Affairs Canada, Evaluation, Performance Measurement, and Review Branch, Evaluation of the Indian Government Support Programs, July 16, 2009, p. 11. (return to source paragraph)

  27. Ibid, p. 30. (return to source paragraph)

  28. Indian and Northern Affairs, Audit and Assurance Services Branch, Audit of Capacity Development, Internal Audit Report, June 3, 2009, p. 6-9. (return to source paragraph)

  29. Ibid, p. 7. (return to source paragraph)

  30. Indian and Northern Affairs Canada, Chapter 5.11 Funding Arrangements: Intervention Policy, Financial Policies and Procedures Manual, Updated 5/15/2008, Section 11, p. 199-200. (return to source paragraph)

  31. Ibid, Appendices B and C, p. 216-218. (return to source paragraph)

  32. Evaluation of the Indian Government Support Programs, p. 27-28. (return to source paragraph)

  33. Ibid, p. 27. (return to source paragraph)

  34. Indian and Northern Affairs Canada, "Invitation to Submit a Proposal for a Framework Agreement Consisting of Lists of Prequalified Individuals and Firms for the Provision of Third Party Management Services to Aboriginal Recipients", p. 6. (return to source paragraph)

  35. Indian and Northern Affairs Canada, Audit and Assurance Services Branch, Audit of the Quality Management Program and the Application of the Intervention Policy, February 26, 2009. (return to source paragraph)

  36. Indian and Northern Affairs Canada, Audit and Assurance Services Branch, Audit of the Quality Management Program and the Application of the Intervention Policy, February 26, 2009, p. 17. (return to source paragraph)

  37. Indian and Northern Affairs Canada, Information Tool Kit, March 7, 2007. (return to source paragraph)

  38. Indian and Northern Affairs Canada, Quebec Region, Regional Intervention Procedure With First Nations and Organizations in Financial and/or Administrative Difficulty, April 6, 2006. (return to source paragraph)

  39. Indian and Northern Affairs Canada, BC Region, 2008-2009 Intervention Procedures and Policy Manual, September 2008 and Intervention Plan Receipt Procedure & FNITP Guide, April 2008. (return to source paragraph)

  40. Indian and Northern Affairs Canada, Audit and Assurance Services Branch, Audit of the Quality Management Program and the Application of the Intervention Policy, February 26, 2009. (return to source paragraph)

  41. Ibid, p. 18. (return to source paragraph)

  42. Indian and Northern Affairs Canada, Regional Operations Sector/Chief Financial Officer Sector, "New Approach to Intervention", Presentation to Senior Management, February 17, 2010. (return to source paragraph)

  43. Treasury Board of Canada Secretariat, Policy and Directive on Transfer Payments, October 1, 2008. (return to source paragraph)

  44. Indian and Northern Affairs Canada, General Assessment Workbook (Test Document), Draft for Discussion Purposes, March 23, 2010. (return to source paragraph)

  45. Nola Buhr, Recommendations for New Financial Triggers for Remedial Action, November 24, 2008. (return to source paragraph)

  46. Indian and Northern Affairs, "A Program and Policy Framework to Support First Nations Community Development", Presentation to Executive Committee, September 9-10, 2009. (return to source paragraph)

  47. The original Evaluation Matrix was included in the Evaluation Methodology Report. It was revised and re-ordered for a clearer presentation of the evaluation questions and findings. Additions to the original Evaluation Matrix are marked in bold.(return to source paragraph)


 
 
Date modified: