Evaluation of Aboriginal Economic Development Strategic Partnerships Initiative

Notice

This website will change as a result of the dissolution of Indigenous and Northern Affairs Canada, and the creation of Indigenous Services Canada and the eventual creation of Crown-Indigenous Relations and Northern Affairs Canada. During this transformation, you may also wish to consult the updated Indigenous and Northern Affairs home page.

Final Report
September 2014
Project Number: 1570-7/13064

PDF Version (401 Kb, 90 Pages)

 

Table of contents

List of Acronyms

AANDC

Aboriginal Affairs and Northern Development Canada

EPMRB

Evaluation, Performance Measurement and Review Branch

SPI Strategic Partnerships Initiative
 

Glossary of Terms

Director General Investment Sub-Committee: The Director-General Sub-Committee is a sub-committee of the Federal Coordination Committee. It acts as an advisory body, making recommendations to the Federal Coordination Committee concerning the effective distribution of the annual Strategic Partnerships Initiative (SPI) budget. It also reviews and makes recommendations to the Federal Coordination Committee with respect to economic opportunities to be supported under SPI.

Federal Coordination Committee: The Federal Coordination Committee is an assistant deputy-minister level committee that validates and prioritizes investment opportunities from a whole-of-government perspective. It also monitors and ensures that relevant federal departments and agencies work together with Aboriginal groups to advance these opportunities.

Federal Coordination Committee-SPI Director General Committee: The Director General Committee for the Federal Coordination Committee brings together directors general from the signatory departments to assist in the overall implementation of the Federal Framework for Aboriginal Economic Development, to promote a consistent approach with respect to the management of SPI.

Governance Review: A review of existing program governance structures in order to improve inefficiencies and outcomes. The last SPI Governance Review took place in 2011.

Single-Window Approach: A single-window approach or a single-window system enables clients to engage with resources, submit documents and participate with the program through one, single transaction instead of relying on disparate lines of communication.

SPI Advisory Group: The SPI Advisory Group is made up of evaluation and program representatives, as well as some program representatives from the region, Aboriginal representatives/organizations, experts, and representatives from other governmental departments.

The Advisory Group will act in an advisory capacity throughout the evaluation, supporting in determining case study and site visit locations. The Advisory Group will review the evaluation’s methodology report, preliminary findings and final evaluation report.

Whole-of-Government Approach: The approach brings departments across levels of government together to create and implement programs supporting common needs and outcomes. Further, whole-of-government approaches are reflected in the 2005 Treasury Board Policy on Management, Resources and Results Structures, supporting the development of a common, government-wide approach to the collection, management and reporting of financial and non-financial information on program objectives, performance and results.

 

Executive Summary

The Strategic Partnerships Initiative (SPI) is a horizontal initiative, which aims to increase Aboriginal participation in complex economic opportunities by coordinating the efforts and investments of multiple federal departments. SPI is managed and administered by the Strategic Initiatives and Partnerships Directorate within the Policy and Coordination Branch of Aboriginal Affairs and Northern Development Canada (AANDC).

SPI is designed to increase economic opportunities for Aboriginal people by coordinating federal efforts around shared priorities, stimulating partnerships between federal and non-federal partners (including Aboriginal communities, provincial governments, the private sector) and addressing funding and program gaps that would otherwise limit or exclude Aboriginal involvement in key opportunities. These opportunities are primarily in, but not limited to, resource based sectors such as mining, fishing, forestry, agriculture, and energy.

This evaluation fulfills the requirements of the Treasury Board Policy on Evaluation to provide comprehensive evaluation of the relevance and performance of programs on a five year cycle. This evaluation covers the period from June 2010 to August 2014, but offers insights, observations and recommendations looking forward. This evaluation was conducted from September 2013 and completed in August 2014. Data collection, analysis and reporting was conducted by the Evaluation, Performance Measurement and Review Branch, with the consulting firm Stiles and Associates Inc. supporting the development of the methodology report and case studies.

Lines of evidence included 28 key informant interviews and a further 63 interviews that were completed as part of case studies; a literature and document review; and an analysis of the data collected to report on SPI. This evaluation sought to provide observations regarding the relevance (continued need, alignment with the priorities of the Department and the Government of Canada and alignment of roles and responsibilities) and performance (effectiveness, efficiency and economy) of the SPI while highlighting issues of design and delivery and governance.

To that end, this evaluation made the following findings:

With respect to relevance:

  1. SPI is addressing a need to support the community readiness of Aboriginal communities in engaging in complex regional economic development opportunities largely through the SPI process of horizontal collaboration to encourage strategic and targeted investment. However, community readiness needs of Aboriginal communities are broader than what SPI aims to address. Therefore, there is a continued need to consider broader community readiness needs in the context planning and a need to align objectives and expected results identified for SPI as a horizontal initiative.
  2. SPI is aligned with the priorities of Aboriginal Affairs and Northern Development Canada and the Government of Canada. Specifically, SPI provides support for Aboriginal communities to increase their involvement with economic development opportunities and the horizontal structure aligns with a government and departmental focus on streamlining services.
  3. SPI is consistent with key priorities, including Federal Framework on Aboriginal Economic Development.
  4. Complementary programming exists, however, the SPI's horizontal nature enables partnering on SPI initiatives in a way that doesn't duplicate efforts or investments. Rather, the SPI is seen to have the potential for being a model, one that could be replicated, for partnering and collaboration that can be extended to other areas beyond economic development.

    With respect to performance:
  5. SPI has resulted in a significant number of partnerships to establish and implement economic development projects, however, clear articulation of what it means to have partnerships within a sector, and why those partnerships are indicative of performance would be useful for performance measurement.
  6. Partnerships have increased buy-in amongst Aboriginal communities and are thus viewed as a success factor to Aboriginal economic development. From a governance perspective, partnerships have increased communication and an improved ability to both identify opportunities and eliminate potential overlap. There were concerns expressed, however, respecting the sense of "ownership" of projects between AANDC and other partners as well as issues with the communication with partners.
  7. A lack of awareness and understanding regarding the mechanisms for implementing a single window structure, including the use of the common Terms and Conditions, has hindered take-up and use of this funding method. However, in projects where a single window approach has been used it was successful and seen as an ideal for future projects.
  8. SPI has made progress towards supporting Aboriginal communities in engaging members, partners and stakeholders on complex regional development initiatives.
  9. There is some evidence that federal investments in Aboriginal economic development opportunities are aligned, however, it is difficult to assess the degree to which investments are aligned.
  10. There is some indication that communities have benefitted from federal strategic partnerships and investments. However, it is difficult to assess the relative impact of SPI due to both the newness of the initiative, and the fact that its contributions need to be assessed alongside the incremental contributions of other funders and initiatives. SPI programming performance should include performance measures that are laid out in specific projects to provide for a better assessment of economic impacts that are attributable to SPI-supported projects.
  11. Governance structures are still being defined and will need continued review and refinement given the relative newness of the initiative. Specific performance indicators for the effectiveness of SPI's governance structures should be included in future performance measurement strategies.
  12. Gender-based data is not being systematically considered or collected through SPI performance information.
  13. There are notable concerns with the SPI Performance Measurement Strategy, the most significant of which include assessing the relative contribution of SPI to longer-term economic outcomes in communities.

    With respect to performance (cost-effectiveness and efficiency):
  14. The value of investments leveraged and the in each project are not adequately captured so as to allow for a complete analysis of cost-effectiveness and cost-efficiency.
  15. Participants in this evaluation had mixed views regarding the efficiency of the SPI program. Generally, interviewees found that, while the horizontal approach may be more labour intensive than other economic development programs, the partnerships achieved are worth the effort.

It is recommended that AANDC:

  1. Strengthens the interface between SPI and existing Aboriginal Economic Development programming within the Department, and with federal and provincial partners to better align strategic objectives and expected results.
  2. Strengthens community engagement on potential opportunities to be considered under SPI with a view to ensuring needs of the community are considered adequately.
  3. Works with Central Agencies and federal partners to increase the use of the single window delivery approach under SPI.
  4. Continues efforts to monitor and review the SPI governance system, including the role of the Federal Coordination Committee, in order to ensure for consistency and maximize efficiencies.
  5. Continues to lead the development of the revised Performance Measurement Strategy with partners in order to enable the monitoring and measurement of results as they relate to both community readiness and longer-term outcomes related to partnerships and investments, including leveraging and to allow for gender-based analysis.
 

Management Response and Action Plan

Project Title: Evaluation of the Strategic Partnership Initiative

Project #: 1570-7/13064

Recommendations Actions Responsible Manager (Title / Sector) Planned Start and
Completion Dates
1. Strengthen the interface between SPI and existing Aboriginal Economic Development programming within the Department, and with federal and provincial partners to better align strategic objectives and expected results. Response: The objective of the program is to increase economic development opportunities for Aboriginal peoples by stimulating partnerships between federal and non-federal partners within key sectors of the Canadian economy. Since 2013, AANDC has worked with partners to focus program support on activities related to "community economic development readiness". The criteria for projects has been communicated with partners and can be found on the SPI GCPedia site established to support delivery of the program. Program officials interface regularly with all signatory departments – a meeting of the SPI partners takes places approximately four times annually to establish priorities and coordinate actions; SPI proposal are shared with economic development program areas before being put forward for approval and officials from other program areas, particularly regional offices, participate on interdepartmental teams that are established to advance individual SPI projects.

Action: AANDC will establish a definition for "community economic development readiness" and reconfirm the focus for the program, as well as expected results, with all partners, including existing Economic Development program areas within the Department. Additionally, staff from Economic Development program areas within the Department will be invited to participate at meetings of the SPI Interdepartmental Working Group.
Assistant Deputy Minister, Lands and Economic Development Sector January 2015
2. Strengthen community engagement on potential opportunities to be considered under SPI with a view to ensuring needs of the community are considered adequately. Response: Under program guidelines in place to support the delivery of SPI, proposals/applications for support are to be developed collaboratively between Aboriginal proponents and partnering federal departments and agencies.

Action: AANDC will explore the potential to work with the National Aboriginal Economic Development Board to identify and prioritize emerging opportunities that could benefit from a whole-of-government approach under SPI, as well as to determine the types of activities that should be prioritized for funding in order to effectively support community economic development readiness.
Assistant Deputy Minister, Lands and Economic Development Sector March 2015
3. Work with Central Agencies and federal partners to increase the use of the single window delivery approach under SPI. Response: to date, SPI has utilized a "single window" approach in the delivery of three large initiatives (fisheries; forestry; biomass) involving multiple projects. In 2013-14, AANDC established a Working Group to examine options for adopting a single window approach more broadly across the program.

The recent Horizontal Internal Audit of Compliance with the Policy on Management, Resources and Results Structures conducted by the Office of the Comptroller General in August 2012 made specific recommendation that "The Secretariat should assess the need to develop guidance relating to Horizontal Initiatives", which would provide additional guidance on the use of single window reporting requirements for horizontal initiatives.

Action Plan: AANDC will work with SPI partners and TBS to explore options for process that will facilitate a "single window" delivery approach under SPI.
Assistant Deputy Minister, Lands and Economic Development Sector June 2015
4. Continue efforts to monitor and review the SPI governance system, including the role of the Federal Coordination Committee, in order to ensure for consistency and maximize efficiencies. Response: AANDC completed a governance review of the program in December 2011. Based on its recommendations, AANDC established a Director General Investment Sub-Committee to make recommendations to Federal Coordination Committee regarding potential opportunities for investment.

Action: AANDC will continue to monitor the governance processes in place to support delivery of the SPI program. In particular, it will examine the role of the Federal Coordination Committee with respect to SPI and expand the Director General Investment Committee to include additional partnering federal departments.
Assistant Deputy Minister, Lands and Economic Development Sector January 2015
5. AANDC continue to lead the development of the revised Performance Measurement Strategy with partners in order to enable the monitoring and measurement of results as they relate to both community readiness and longer-term outcomes related to partnerships and investments, including leveraging to allow for gender-based analysis. Response: A revised logic model and draft performance indicators were developed for the program in July 2013. Information contained in the evaluation will allow the program to move forward with the development of a revised Performance Measurement Strategy focusing on community economic development readiness.

Action: AANDC will lead the development of a revised Performance Measurement Strategy with federal partners in order to enable the monitoring and measurement of results as they relate to community readiness and associated longer-term outcomes of the program.
Assistant Deputy Minister, Lands and Economic Development Sector March 2015
 

I recommend this Management Response and Action Plan for approval by the Evaluation, Performance Measurement and Review Committee 

Original signed by:

Michel Burrowes
Director, Evaluation, Performance Measurement and Review Branch

 

I approve the above Management Response and Action Plan

Original signed by:

Sheilagh Murphy
Assistant Deputy Minister, Lands and Economic Development Sector

The Management Response and Action Plan for the Evaluation of the Strategic Partnerships Initiative were approved by the Evaluation, Performance Measurement and Review Committee on September 25, 2014

 

1. Introduction

1.1 Overview

In accordance with the Treasury Board Policy on Evaluation requirements to provide a neutral and evidence-based assessment on programs, the Evaluation, Performance Measurement and Review Branch (EPMRB) of Aboriginal Affairs and Northern Development Canada (AANDC) undertook an evaluation of the Aboriginal Economic Development Strategic Partnerships Initiative (SPI).

SPI is a horizontal initiative, which aims to increase Aboriginal participation in complex economic opportunities by coordinating the efforts and investments of multiple federal departments. SPI is managed and administered by the Strategic Initiatives and Partnerships Directorate within the Policy and Coordination Branch of Aboriginal Affairs and Northern Development Canada.

The evaluation work was conducted by EPMRB between September 2013 and August 2014, with the assistance of one consulting firm, Stiles and Associates Inc for work associated with the development of a methodology report and case study work. The scope of the evaluation covers all activities undertaken between the program launch in June 2010 to March 2014, with financial data available for fiscal year 2010-2011 through to fiscal year 2012-2013. SPI is a new program and has not previously been evaluated.

The evaluation involved the participation of an Evaluation Working Group with representatives from 13 partnering federal departments and agencies in accordance with Treasury Board's Guidance on the Governance and Management of Evaluations of Horizontal Initiatives.

This evaluation report presents findings and recommendations on relevance and performance, including cost-effectiveness and efficiency, of the program. Due to the horizontal nature of SPI, issues related to implementation, governance and the facilitation of cross-departmental cooperation are an important focus for the evaluation. This evaluation also considers best practices and lessons learned.

1.2 Program Profile

1.2.1 Context and Background

Context

In June of 2009, the Government of Canada released the Federal Framework for Aboriginal Economic Development (the Framework). The Framework provides for a focused, government-wide (whole-of-government) approach to better align federal investments, respond to new and changing economic conditions and leverage partnerships in order to address persistent barriers that impede the full participation of Aboriginal people in the Canadian economy.

A key element of the Action Plan to implement the Framework included the establishment of a sector partnerships fund to develop strategies to focus the resources, efforts, and expertise of federal departments and agencies to support enhanced participation of Aboriginal peoples in complex developments, including major resource projects.

More than 20 federal departments and agencies of the Government of Canada have mandates that include Aboriginal economic development. Existing programs and processes do not always facilitate economic collaboration among federal partners nor promote the identification of shared priorities. This disaggregated approach to program delivery impedes the federal government from strategically targeting investments in economic opportunities that would maximize benefits for Aboriginal peoples while advancing government priorities and optimizing federal investments.

To access federal funding for Aboriginal economic development, existing programs and processes require clients to respond to a variety of departmental / agency application and reporting requirements for a single initiative, thereby, constraining supportive and timely federal investments, as well as comprehensive performance measurement.

The Strategic Partnerships Initiative helps advance two key strategic goals of the Framework: (i) Forging new and effective partnerships – primarily among federal departments and Aboriginal stakeholders, but also with provincial and territorial governments and the private sector'; and (ii) Focusing the role of the federal government- fostering a whole-of-government approach that aligns and targets investments toward market driven opportunities.  

Background

Launched in June 2010, SPI is a federal horizontal initiative designed to increase economic opportunities for Aboriginal people by coordinating federal efforts around shared priorities, stimulating partnerships between federal and non-federal partners (including Aboriginal communities, provincial governments, the private sector), and addressing funding and program gaps that would otherwise limit or exclude Aboriginal involvement in key opportunities. With significant investments in major projects anticipated in the next 10 years, SPI will focus increasingly on supporting economic development readiness activities so that communities are better prepared to engage with partners and participate fully in these developments. An annual budget of $14.45 million is available to support projects, particularly in key sectors of the Canadian economy such as mining, fisheries, forestry, agriculture, and energy. The funding is administered through umbrella terms and conditions that create the flexibility for signatories to engage in projects where they may have been otherwise limited.

SPI provides a mechanism for federal partners to collectively prioritize and sequence investments, assess and make project approvals, leverage non-federal sources of funding, monitor progress, and report on outcomes.

Investments are prioritized based on the extent to which they meet a number of criteria and objectives, including:

  • Alignment with Government of Canada priorities;
  • Alignment with objectives of the Federal Framework for Aboriginal Economic Development;
  • Demonstration of significant partnership potential requiring a coordinated federal approach and often spanning multiple fiscal years;
  • Proposed investments are based on the evidence of need;
  • The initiative does not overlap or duplicate existing federal and programs and SPI funding will fill a demonstrated gap; and
  • The federal role for the proposed initiative is clearly demonstrated.

Before SPI, funding decisions were typically made individually by federal departments and agencies in isolation of a broader strategy. Now, federal partners can make collective investment decisions and address any gaps in existing programs that would otherwise limit or exclude Aboriginal participation in complex economic development opportunities. It also enables federal partners to strategically engage Aboriginal communities, other levels of government and private sector partners so they may leverage additional funding or in-kind support.

The initiative also provides a mechanism for federal partners to combine their program application and approval processes, reducing the administrative and reporting burden on funding recipients.

SPI emphasizes a whole-of-government approach to identifying and investing in business opportunities by enabling the sharing of information and facilitating linkages between existing programs and services. Central to the success of the initiative is the ability to achieve a horizontal approach to identifying and prioritizing opportunities for investment, developing work plans that identify the role of the federal partners in advancing shared outcomes and establishing relevant performance indicators to measure results.

This unique, whole-of-government approach provides federal partners with a mechanism to collectively identify opportunities for investment, monitor progress and report on outcomes. SPI is designed such that the Government of Canada is able to maximize the results of federal investments and be better positioned to leverage funds from non-federal sources. Additionally, SPI creates the possibility of a single-window approach to federal investments in shared priorities.

There are 15 federal departments and agencies that are signatories to the program, including:

  • Aboriginal Affairs and Northern Development Canada
  • Industry Canada
  • Federal Economic Development Initiative for Northern Ontario
  • Natural Resource Canada
  • Agriculture and Agri-Food Canada
  • Employment and Social Development Canada
  • Canadian Economic Development Agency for the North
  • Parks Canada
  • Federal Economic Development Agency for Southern Ontario
  • Status of Women Canada
  • Western Economic Diversification
  • Department of Fisheries and Oceans
  • Canada Economic Development for Québec Region
  • Environment Canada
  • Canadian Environmental Assessment Agency
Previous Evaluations and Reviews

In response to the complexity of the program and the lack of existing horizontal initiatives that could be used as an example in the development of the SPI decision-making structure, a program-led Review of the SPI Governance Structure was called for after a full year of program implementation, in 2011.

In response to recommendations from the Governance Review, AANDC made some changes to the SPI governance structure for decision making, including the addition of a Director General Committee and a Director General Investment Sub-Committee. Decision was also taken to move beyond the sector-based approach originally used to a more opportunities-based approach for selecting and supporting initiatives.

As is often the case with horizontal initiatives, SPI faced challenges with respect to performance measurement due to the involvement of multiple partners and the varied nature and scope of its funded activities. SPI was included in a 2012 Office of the Comptroller General's Horizontal Internal Audit of Compliance with the Policy on Management, Resources and Results Structures. In this report, it was noted that the existence of numerous policies with performance measurement requirements led to misunderstandings within partnering departments and agencies in terms of what issues need to be measured and disclosed. This resulted in a lack of consistency with respect to the form and substance of performance reporting across the departments.

In response to their recommendation to finalize and implement frameworks for monitoring performance, AANDC initiated a review of the SPI Performance Measurement Strategy in 2012, including revisions to the logic model and performance indicators.

How SPI Works

The program is administered by AANDC and involves 15 federal departments and agencies, including AANDC, who partner with AANDC as signatories to the program's Terms and Conditions.

There are two categories of activities under SPI:

  1. Opportunity Assessments/Priority Setting, which includes processes and procedures to:
    • identify strategic economic opportunities through information gathering/research/identifying community needs/sector studies; and
    • assess and prioritize opportunities for potential investment (note that the Federal Coordination Committee validates and prioritizes sector specific economic opportunities best able to benefit from SPI).
  2. Building Relationships and Partnerships, which will enable partnerships with Aboriginal people, provinces and territories, and the private sector, as well as between federal departments and agencies through the:
    • engagement of partners (federal departments, agencies and others) with Aboriginal stakeholders; and
    • co-ordination of federal activities and investments in Aboriginal economic development.

These activities are undertaken by multiple federal departments and agencies. Some activities are undertaken with dedicated Operation and Maintenance and/or Grants and Contributions funds under the SPI, while others are funded through existing reference levels. It is expected that federal partners use existing resources to support SPI initiatives, and where it is deemed appropriate and necessary, resources from the SPI budget can be used to address gaps, which cannot be covered by other funding resources.

To facilitate decision making among partners, the SPI governance process involves four critical structures, including the Federal Coordination Committee; the Federal Coordination Committee-SPI Director General Committee; the SPI Director General Investment Sub-Committee; and the SPI Interdepartmental Working Group.

Federal Coordination Committee

Central to the governance structure of the SPI is the Federal Coordination Committee for Aboriginal Economic Development. The Federal Coordination Committee was established in 2010 as part of the Government's Federal Framework for Aboriginal Economic Development and is an assistant deputy minister-level committee that validates and prioritizes investment opportunities from a whole-of-government perspective. The Federal Coordination Committee monitors and ensures that all relevant federal departments and agencies work together with Aboriginal groups to advance these opportunities.

The Federal Coordination Committee is responsible for prioritizing opportunities for investment and determining budget allocations to advance prioritized opportunities; determining the departments and agencies responsible for collectively advancing the opportunities prioritized for investment under SPI; and for monitoring medium and longer term outcomes for funded initiatives.

Federal Coordination Committee-SPI Director General Committee

The establishment of the Director General Committee addresses the recommendation in the Governance Review to create a Federal Coordination Committee subcommittee in charge of SPI's operational matters. The Director General Committee for the Federal Coordination Committee brings together directors general from the signatory departments to assist in the overall implementation of the Framework and to promote a consistent approach with respect to the management of SPI. This includes the development and oversight of SPI governance processes, operations, allocations of budgets, communications, and reporting and evaluation.

Director General Investment Sub-Committee

The Director General Investment Sub-Committee is a sub-committee of the Federal Coordination Committee-SPI Director General Committee whose purpose is to act as an advisory body to the Federal Coordination Committee. This Sub-Committee makes recommendations to the Federal Coordination Committee concerning the most effective distribution of the annual SPI budget in order to support Aboriginal participation in economic opportunities.

SPI Interdepartmental Working Group

The SPI Interdepartmental Working Group has been established to promote a consistent approach and application of the SPI. The Interdepartmental Working Group ensures consistency regarding the delivery of SPI, including the overall direction and management of initiatives.

In addition to the SPI Governance Structure, each SPI opportunity is supported by its own working level committee, or working group, with members from all partnering departments involved that work closely with the Aboriginal stakeholders. The working groups, steered by the leading department on the project, identify community needs and promote increased Aboriginal participation in the sector of the SPI project. The groups support the SPI projects in the creation of working plans that are responsive to community needs, and oversee and monitor project progress through their working plans.

1.2.2 Program Objectives and Outcomes

SPI is situated as Program 3.2 under the Lands and Economy Pillar of AANDC's 2014-2015 Program Alignment Architecture.

SPI was launched in June 2010 as a component of the Government of Canada Action Plan to implement the Federal Framework for Aboriginal Economic Development. SPI was designed to help achieve the following strategic goals of the Framework:

  • Forging new and effective partnerships – primarily among federal departments and Aboriginal stakeholders, but also with provincial and territorial governments and the private sector; and
  • Focusing the role of the federal government – fostering a whole-of-government approach that aligns and targets investments toward market-driven opportunities in key sectors of the Canadian economy.

The stated objective of SPI is to "increase economic development opportunities for Aboriginal entrepreneurs and communities by promoting partnerships between federal and non-federal partners in priority sectors of the economy, including forestry, fisheries, mining, energy and agriculture"Footnote 1. While SPI has primarily sought to advance initiatives within five priority economic sectors – forestry, energy, mining, fisheries and agriculture - many government departments, including AANDC, are being asked to respond to other major development opportunities across the country (such as pipelines, potash, and major hydro development projects). In order to respond to emerging pressure for federal collaboration on these opportunities, SPI has adopted an opportunity-based approach as opposed to the sectoral approach that that was once used. This is intended to lead to greater participation by Aboriginal peoples in the Canadian economy.

SPI expected outcomes are aligned with the strategic priorities, as identified in the Federal Framework for Aboriginal Economic Development, by contributing to opportunity-ready communities, viable Aboriginal businesses and a skilled Aboriginal workforceFootnote 2.

The outcomes and performance indicators identified below are based on the revised Performance Measurement Strategy for the Strategic Partnerships Initiative (July 2013). The revised logic model is presented below.

Logic Model of the Strategic Partnerships Initiative (SPI)
Description of Logic Model

The logic model is oriented from top to bottom. On the left is the title for each row. The first title on top is: Inputs; the second title is: Activities; the third title is: Outputs; the fourth title is Immediate Outcomes; the fifth title is: Intermediate Outcomes and the last title at the bottom is Long-term Outcomes.

The first row on top is composed of two boxes. The box on the left is: Financial and in-kind contributions other resources from non-federal partners. The box on the right is: SPI resources (G&Cs), O&M, other. A dotted line is separating the first row and the second row. 

The second row is composed of three boxes. The box on the left is:  Building relationship and partnerships; the box in the middle is:  Opportunity assessment and priority setting and the box on the right is: Supporting targeted economic development opportunities. An arrow is connecting the box on the left and the box in the middle and an arrow is connecting the box in the middle and the box on the right. Arrows pointing down are connecting all three boxes to all four boxes in the third row.

The third row is composed of four boxes. The box on the left is: Partnership between Federal and non-Federal partners. The second box from the left is: Opportunity profiles, proposals or strategies and studies. The third box from the left is: Community profiles, readiness and capacity assessments. The last box on the right is: Awareness building initiatives in economic development opportunities. Arrows pointing down are connecting the box on the left and the box on the right to all three boxes in the fourth row.

The fourth row is composed of three boxes. The box on the left is:  Cooperation and collaboration among Federal and non-Federal partners; the box in the middle is: Simplified government application, monitoring and reporting (single window) and the box on the right is: Aboriginal communities engage members, partners and stakeholders on complex/regional development initiatives. Arrows pointing down is connecting all three boxes to the two boxes in the fifth row.

The fifth row is composed of two boxes. The box on the left is:  Aligned federal investments in Aboriginal economic development opportunities. The box on the right is: Aboriginal communities are ready to participate in economic development opportunities. An arrow pointing down is connecting the two boxes to the box in the fifth row. The box in the fifth row is:  Aboriginal communities benefit from federal strategic partnerships and investments in economic resource opportunities.


 

SPI aims to achieve the following three immediate outcomes:

  • Cooperation and collaboration among federal and non-federal partners;
  • Simplified government application, monitoring and reporting (single window); and
  • Aboriginal communities engage members, partners and stakeholders on complex/regional development initiatives.

These immediate outcomes are expected to result in the following two intermediate outcomes:

  • Aligned federal investments in Aboriginal economic development opportunities; and
  • Aboriginal communities are ready to participate in economic development opportunities.

Both intermediate outcomes will contribute to a single long-term outcome: Aboriginal communities benefit from federal strategic partnerships and investments in economic resource opportunities.

1.2.3 Program Management, Key Stakeholders and Beneficiaries

Aboriginal Affairs and Northern Development Canada

Aboriginal Affairs and Northern Development Canada, particularly the Assistant Deputy Minister of Land and Economy, supported by the Strategic Initiatives and Partnerships Directorate of the Policy and Coordination Branch, is responsible for the management, monitoring and reporting of SPI.

AANDC facilitates the organization of the Federal Coordination Committee for Aboriginal Economic Development, which oversees the implementation of the Federal Framework for Aboriginal Economic Development. AANDC's Headquarters is responsible for all of SPI's Parliamentary reporting obligations, with input from partnering departments. AANDC is also responsible for horizontally coordinating all of the signatories involved in SPI.

Regional Offices

AANDC regional offices are responsible for implementing the SPI initiatives and projects where AANDC is the lead department. On the ground, regional AANDC offices can identify and develop further opportunities for investment and measure projects' progress and performance.

Other Government Departments

SPI has signatories from 15 federal departments and agencies. The signatories have signed on to the Strategic Partnerships Initiative Terms and Conditions, recognizing the importance of collaborative work in developing and moving Aboriginal economic development forward. Those partners leading or involved in specific SPI projects will be involved through the full life-cycle of the project, from development to implementation to completion and they will be engaged in the project's performance measurement.

Beneficiaries

The Strategic Partnerships Initiative seeks to support members of Aboriginal communities (First Nations, Métis and Inuit); tribal councils, governments of self-governing First Nations; local governments of Inuit communities; Qulliit Nunavut Status of Women Council; Aboriginal for profit and not-for-profit corporations, partnerships, associations, co-operatives, and institutions, which are majority owned and controlled by Aboriginal people; and Aboriginal businesses, partnerships and joint ventures.

1.2.4 Program Resources

The total funding for this program is $85.6 million (fiscal year 2010-2011 to fiscal year 2014-2015) or $16.9 annually. SPI's annual budget is made up of approximately $14.5 million in Grants and Contributions funds and $2.4 million in Operations and Maintenance funds. The Grants and Contributions funding portion is predominately spent on multi-year initiatives.

Financial Summary Table by Estimates Vote Structure (Dollars)
Fiscal Year
  2010-2011 2011-2012 2012-2013 2013-2014 2014-2015 Total Ongoing
NEW FUNDING
Vote 1 (Operating Expenditures)
Personnel $144,149 $144,149 $144,149 $144,149 $144,149 $720,745 $144,149
Operating & Maintenance $2,532,715 $2,683,194 $2,683,619 $2,683,619 $2,683,619 $13,266,766 $2,683,619
EBP @ 20% $28,830 $28,830 $28,830 $28,830 $28,830 $144,150 $28,830
Total Vote 1 $2,705,694 $2,865,173 $2,865,598 $2,865,598 $2,865,598 $14,131,661 $2,856,598
Vote 10 (Grants and contributions)
Contributions $13,600,000 $14,450,000 $14,450,000 $14,450,000 $14,450,000 $71,400,000 $14,450,000
Total Votes $16,305,694 $17,306,173 $17,306,598 $17,306,598 $17,306,598 $85,531,661 $17,306,598
Accomodations @ 13% $18,739 $18,739 $18,739 $18,739 $18,739 $93,695 $18,739
Total new funding $16,324,433 $17,324,912 $17,325,337 $17,325,337 $17,325,337 $85,625,356 $17,325,337
 
 

2. Evaluation Methodology

2.1 Evaluation Scope and Timing

This evaluation examined SPI’s activities undertaken and outcomes achieved between June 2010 to August 2014, with financial data available for fiscal year 2010-2011 through to fiscal year 2012-2013.

As per the Treasury Board Policy on Evaluation, the evaluation focussed on core issues related to relevance (i.e. continuing need for the program, alignment with government priorities, consistency with federal roles and responsibilities) and performance (i.e. achievement of expected outcomes and demonstration of efficiency and economy) with a key focus on design and delivery, including governance issues.

The Evaluation Terms of Reference were approved by the Evaluation, Performance Measurement and Review Committee in September 2013. The evaluation was undertaken between September 2013 and August 2014.

2.2 Evaluation Issues and Questions

The evaluation issues focussed on AANDC commitments as per the program’s initial logic model (October 2010) and revised logic model (July 2013). The following core issues and questions were addressed:

Evaluation issue Evaluation Question
Relevance
Continued Need for the Program (assessment of the extent to which the program continues to address a demonstrable need and is responsive to the needs of Canadians)
  • Does SPI address demonstrable needs of Aboriginal communities, and to what extent are federal strategic partnerships and investments essential in meeting those needs?
Alignment with Government Priorities (assessment of the linkages between program objectives and (i) federal government priorities; and (ii) departmental strategic outcomes)
  • Are the objectives of the program consistent with government-wide priorities and AANDC’s strategic outcomes?
Alignment with Federal Roles and Responsibilities
(assessment of the role and responsibilities of the federal government in delivering the program)
  • To what extent are the objectives of the program aligned with the role and the responsibilities of the federal government? Is there duplication or overlap with other programs, policies or initiatives?
Performance - Effectiveness, Efficiency and Economy
Achievement of Expected Outcomes (assessment of progress toward expected outcomes (including immediate, intermediate and ultimate outcomes) with reference to performance targets, program reach, program design, including the linkage and contribution of outputs to outcomes)
  • To what extent is progress being made toward achievement of expected outcomes?
    • Cooperation and collaboration among federal and non-federal (Aboriginal communities, provincial governments, the private sector, etc.) partners.
    • Simplified government application, monitoring and reporting (single-window).
    • Aboriginal communities engage members/partners and stakeholders on complex/regional development initiatives.
    • Aboriginal communities are ready to participate in economic development opportunities.
    • Federal investments in Aboriginal economic development are aligned.
    • To what extent do Aboriginal communities benefit from federal strategic partnerships and investments and to what extent are they an essential part of the causal package enabling Aboriginal communities to benefit from economic resource opportunities?
  • How effective is the SPI governance structure?
  • What are the factors that are facilitating or hindering the achievement of SPI outcomes?
  • To what extent have the differences in the socio-economic realities of women and men been addressed in SPI initiatives?
  • To what extent do SPI initiatives support AANDC’s responsibilities under the Federal Sustainable Development Strategy?
  • How effective is SPI’s performance measurement strategy and how could it be improved?
Demonstration of Efficiency and Economy (assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes)
  • How could SPI be improved?
  • Are there alternative or more efficient ways of delivering SPI that would achieve similar results?
  • How cost effective is the design and delivery of SPI services?
  • Do more economic alternatives exist? Have simplified application and reporting processes resulted in cost savings for government or the private sector? Is it a good use of money to invest in a horizontal initiative?
Other Evaluation Issues
(best practices and lessons learned)
  • What are the lessons learned and best practices that emerged from implementation of SPI, or from similar horizontal programs from other jurisdictions, countries or the private sector that may enhance SPI effectiveness?
 

2.3 Evaluation Methods

The evaluation was undertaken by EPMRB with the support of the consulting firm, Stiles and Associates Inc., for assistance with the development of a methodology report and in undertaking case studies. The evaluation was undertaken according to the following three phases of work:

2.3.1 Planning Phase

This phase was undertaken approximately between July 2013 and December 2013.

Prior to undertaking the evaluation, the evaluation team met with SPI program representatives in order to develop a better understanding of SPI, identify key documents and discuss the development of a Terms of Reference, including the evaluation scope, approach, broad issues and questions and cost. Terms of Reference were approved by the Evaluation, Performance Measurement and Review Committee in September 2013.

SPI Evaluation Working Group

An Evaluation Working Group was formed with representatives with an in-depth working knowledge relevant to the evaluation of SPI. The SPI Evaluation Working Group comprised both evaluation and program representatives from AANDC, as well as at least one representative from the partnering federal departments and agencies. The Working Group comprised over 20 individuals. The Working Group supported the evaluation by advising primarily on the development of the methodology for the evaluation and by helping to facilitate access to data sources for evaluation purposes. An initial evaluation launch meeting was held in September 2013 and another meeting was held in February 2014 to review the methodology report. Preliminary findings were shared with the Working Group in June 2014.

SPI Evaluation Advisory Group

An Evaluation Advisory Group was also formed comprised of persons with knowledge of Aboriginal Economic Development from a broad government and non-government perspective, as well as knowledge specific to SPI. The role of the Advisory Group was to support the evaluation by providing advice and oversight on the overall evaluation process. An Evaluation Working Group meeting was held in February 2014 and Advisory Group members were invited to review the methodology report and preliminary findings were shared in June 2014.

Horizontal Evaluation

Evaluation directors of partnering departments and agencies were made aware of the evaluation and received a copy of the evaluation Terms of Reference, as well as the Evaluation Working Group contact list, which included representatives from each partnering department/agency. Given that the evaluation had program representation of other departments/agencies on the Evaluation Working Group, it was left up to working group participants to keep their respective evaluation units apprised of the development, progress and issues relating to the evaluation.

Methodology Report

A draft methodology report was developed by EPMRB and validated and enhanced by Stiles and Associates. The methodology report was informed primarily by a media scan, an initial review of program documentation, an initial review of literature and preliminary interviews with SPI program management and representatives of other government departments. In total, three interviews were conducted in support of the planning of the evaluation. These interviews contributed to clarifying the issues and questions, potential interviewees and considerations for undertaking case studies.

2.3.2 Data Collection/Analysis Phase

Data was collected using multiple lines of enquiry and analyzed by triangulating evidence against the issues and questions. This phase was undertaken approximately between January 2014 and June 2014.

Literature Review

The literature review examined theories of horizontal government, or the whole-of-government approach, as applied in different countries, how the program aligns with the Government’s role and priorities, alternative programs that support or contribute to similar outcomes, and best practices among partnership initiatives between Aboriginal communities, businesses, and national departments, or local governments and private sector partnerships.

Document Review

EPMRB undertook a comprehensive review and analysis of program related documents. Key documents reviewed included program documents, governance-related documents, sector strategies and reference papers, Aboriginal economic development papers, performance measurement strategies, progress and performance reports, project files, reviews, policy, planning and administrative documents, and other documents related to SPI.

Financial and Administrative Data Analysis

A review of available administrative data was undertaken. The financial data analysed included AANDC expenditures, as well as the expenditures of other departments and other non-federal and private sector partners. Where available, other qualitative and quantitative data related to all SPI Initiatives, as well as project-specific data were reviewed. These included reports from the Director General Investment Committee, the Departmental Performance Report and a sample of recipient reports. The following data were identified as part of this review:

  • Internal Progress Reports for Senior Management - Internal Progress Reports for Senior Management provided information on 19 initiatives of SPI, which included: funding received; funding spent; activities each initiative engaged in or planned; and progress towards outcomes and deliverables respective to each initiative.
  • Project Summary and Activity Summary and Expenditure Tables - Activity Summary and Expenditure Tables provided details on projects undertaken by the initiative, and the amount of SPI funds that were allotted for each project from 2010-2011 fiscal year until 2012-2013 fiscal year.
  • Initiative Financial Summaries –Financial summaries were provided for 19 initiatives from fiscal year 2010-2011 to early 2013-2014. The summaries supplied data on the number of partnerships (ongoing and unique), contribution agreements and the numbers and names of Aboriginal communities involved per fiscal year, as well as SPI contributions, federal, non-federal and private funding received per fiscal year. This analysis was intended to contextualize findings with respect to the amount spent per initiative, and to speak to questions of efficiency and economy.

The information was analyzed to assess the extent to which they could support indicators of outcomes as specified in the most recent (2010) Performance Measurement Strategy for SPI (i.e. numbers of trained sessions or individuals completing training, number of Aboriginal businesses, and/or number of workplans). The data review found that, much of the information being collected was anecdotal and the majority of the reports reflected the beginning phases of the initiatives, without concrete outcomes reported. To the extent possible, content analysis was conducted to determine progress towards the immediate and intermediate outcomes of SPI.

Key informant interviews

Key informant interviews were used to gain a better understanding of perceptions and opinions of individuals who have had a significant role or experience in management of SPI and those who may benefit from the program.

Interviews were completed through the key informant interview process at Headquarters as well as through the completion of case studies. In total, 91 individual interviews or group discussions were conducted (28 as key informant interviews, mostly completed at Headquarters and 63 as part of case studies).

Four separate interview guides were developed and interviews conducted based on the following:

  1. AANDC representatives Headquarters and regions - Interviewees included AANDC SPI managers and program officers at Headquarters and regions. A total of 16 interviews were completed (nine completed at Headquarters and seven completed as part of case studies – see Case Study Annexes for further details);
  2. Stakeholders - Interviewees included representatives of other federal departments provincial/territorial government representatives, external consultants, business representatives. A total of 41 interviews were completed (16 completed at Headquarters and 25 completed as part of case studies – see Case Study Annexes for further details);
  3. Aboriginal organizations and community representatives involved in SPI projects - n=30 (one completed at Headquarters and 29 completed through case studies – see Case Study Annexes for further details; and
  4. External experts - Interviewees included: academics and Aboriginal economic development practitioners. A total of four (two completed at Headquarters and two completed through case studies – see Case Study Annexes for further details).

Interview guides were designed to cover core evaluation issues and questions specified in the evaluation matrix and were tailored to the different respondent groups within each category, as appropriate. All interviewees were sent the finalized guide by email in advance of their scheduled interview to allow for preparation for the interview.

Case Studies

Case studies provided an in-depth assessment at the community level of the relevance of SPI, design and delivery of the program and the effectiveness of SPI to date. More specifically, the case studies allowed insight into the conditions for success and/or limitations in reaching the desired outcomes present in each project.  

The evaluation included case studies of five SPI Initiatives. The case studies were identified based on the following criteria.

  • Sector: a mixture of projects from different sectors
  • Location: projects chosen from Ontario, Quebec, the Atlantic, the West and the North
  • Lead department: a mix of projects led by AANDC and other federal departments
  • Funding level: a mix of smaller and larger projects
  • Length of support: a mix of projects approved in 2010-11 and 2011-12
  • Recommendations from program staff and working group member

A focus on five initiatives was considered sufficient to allow evaluators to explore SPI issues in depth across a range of communities in different regions of the country. (See detailed Annex B). Program staff were consulted to identify the sites and project partners that could provide the richest data and most efficient use of available resources. The following initiatives and their related projects were identified:

  • First Nations Power Authority
  • Forest Industry and Biomass Initiative
  • Ring of Fire
  • Labrador Trough
  • Atlantic Commercial Fisheries Diversification Initiative

Case study research included the development of a project profile (including available statistics to situate and contextualize the communities involved; a review of relevant documents (i.e. performance reporting, media reports) and administrative data (e.g financial reporting); 29 interviews with Aboriginal organizations and community representatives involved in SPI projects (note: as mentioned in section above) and on-site observation.

Econometric Data Analysis

An econometric analysis of financial data to make inferences about return of investment (economy) was attempted, however, there was insufficient financial and outcome data collected by AANDC to date to allow for an assessment of the value of projects relative to the incremental investment from SPI versus other department programs, or the total amount leveraged.

Also, due to the newness of the program, and because many initiatives are in early stages, a lack of concrete outcomes does not allow for an assessment of return on investment.

Given the limited range of data, and given that reliable economic data using variables from the Canadian Census have not been collected since 2011, it was impossible to make a reasoned assertion on economic impacts using these proxy variables.

2.3.3 Reporting

For each line of evidence, findings from all lines of evidence were analyzed and identified against evaluation issues and questions as key findings by using a findings template format (streamlined technical report). Findings were then cross-identified and triangulated in order to identify emerging themes and key findings.

Several documents and supporting technical-based reports were created to reflect the evidence collected from the evaluation based on evaluation issues as per the evaluation matrix. This phase was undertaken approximately between July 2014 and September 2014.

Preliminary findings were shared with Evaluation Working Group and Advisory Group members and presented to the Evaluation, Performance Measurement and Review Committee in June 2014.

2.4 Considerations, Strengths and Limitations

2.4.1 Considerations

Since SPI involves many departments with different roles and responsibilities, collaboration and access to data of those federal departments was essential to the success of this evaluation. To increase success, an evaluation working group and advisory group was established to increase the participation of SPI coordinators and partnered signatories as well as data sharing.

2.4.2 Strengths

As an integral part of the New Federal Framework for Aboriginal Economic Development, comprehensive research was conducted to identify key success factors for Aboriginal business and economic development. SPI is a cornerstone program under the Federal Framework for Aboriginal Economic Development and benefits from the resources and research conducted under the Framework.

Case studies allowed for the opportunity to collect information at a level of detail that was not possible to collect through broader and more representative methods (i.e. a survey) because of cost, schedule and other factors. The case studies highlighted best practices, lessons learned and key successes. The methodology for the case studies included a review of Initiative-specific documentation interviews, administrative data review and on-site observation. To conduct the case studies, evaluators traveled to the regional hub and, where possible, conducted site visits to the Aboriginal community / communities involved in the initiative. Some of the case studies involved multiple remote communities, and numerous partners.

2.4.3 Limitations

The Strategic Partnerships Initiative is a new program for the Department, launched in 2010. Very little longitudinal data exist with which to assess program outcomes, especially in relation to impacts on Aboriginal economic development, job creation, and community well-being. The evaluation therefore focuses on immediate outcomes and the design and delivery aspects associated with SPI.

The SPI Performance Measurement Strategy is currently being updated and may affect the availability of necessary information. Of particular importance, the evaluation mostly relied on the most up-to-date logic model (July 2013) to evaluate the program and not the model originally created for the program. As a result, some of the new outputs in this logic model have only had data collected for them in the past year, limiting longitudinal performance measurement.

In conducting the evaluation, it was apparent that neither strong nor comprehensive performance data was available from participating departments on their initiatives. Additionally, there was concern that different departments may have different ways of determining indicators and measuring outcomes, thus, making cross-departmental programming comparisons difficult. Furthermore, any inconsistencies in data measurement would make it difficult to determine the influence of the SPI initiative over time. Evaluators collected and assessed the performance measurement data available and developed tools that allow for a comprehensive assessment of the state of performance measurement, and the achievements of the program to date. Performance data was mostly qualitative and was assessed with content analysis, with additional financial information used to inform assessments where applicable.

SPI initiatives by their nature are complex and involve multiple players. Coordinating cross-departmental and community-based involvement was slow and cumbersome at times. Further, because projects are complex, the final impacts will become evident only in the long-term and not in the funding period under examination.

Due to the multi-stakeholder nature of the projects that SPI helps leverage, it was difficult to differentiate the impact between SPI dollars and other stakeholders’ contributions. It is important for the purpose of this evaluation to assess the effectiveness of SPI dollars in creating a cross-department leverage effect only and not to evaluate the impact of other stakeholder funds.

The evaluation had neither the resources nor the time to assess each major initiative funded to date.

2.5 Roles, Responsibilities and Quality Assurance

EPMRB was the project authority for this evaluation and managed the evaluation in line with EPMRB’s Engagement Policy and Quality Control Process. The majority of the evaluation work was completed in house, with assistance from Stiles and Associates Inc in contributing to the development of a methodology report and in undertaking case studies.

 

3. Evaluation Findings - Relevance

A program continues to be relevant if there is a clear demonstrable need for it, if it aligns with federal priorities and if it is consistent with federal roles and responsibilities. The continued relevance of SPI was explored primarily through literature and document review and the case studies and key informant interviews.

Based on the results of the lines of enquiry, SPI is relevant, based on the following findings and conclusions related to continued need, alignment with federal roles and responsibilities and consistency with government roles and priorities.

3.1 Continued Need

Finding: SPI is addressing a need to support the community readiness of Aboriginal communities in engaging in complex regional economic development opportunities largely through the SPI process of horizontal collaboration to encourage strategic and targeted investment. However, community readiness needs of Aboriginal communities are broader than what SPI aims to address. Therefore, there is a continued need to consider broader community readiness needs in the context planning and a need to align objectives and expected results identified for SPI as a horizontal initiative.

To assess relevance, the evaluation sought evidence of: (1) continued need to be responsive to the needs of Aboriginal communities with respect to community readiness and encouraging participation in the economy; and (2) whether there was a continued need to facilitate cross-sector/departmental approaches to support Aboriginal participation in complex regional development initiatives.Footnote 3

Community readiness of Aboriginal communities

The current evidence suggests that Aboriginal people are poorly positioned from a capacity and community-readiness perspective to fully benefit from the resource economy. Generally speaking, communities have a limited ability to negotiate with private sector businesses.

According to literature review, a lack of business expertise prevents the participation of some Aboriginal entrepreneurs in opportunities resulting from nearby major projects and can also impede communities from pursuing partnerships, such as equity participation or joint ventures with industry proponents.Footnote 4 Further, the literature review highlights capacity limitations with respect to overall business and management and further identified a need for support in accessing major financing, expertise and access to training opportunities.

Evidence from the case studies highlighted the relevance and usefulness of having an established business development team assists communities in developing business plans, obtaining funding and implementing business plans and was cited as a key success factor.

Given the challenges faced by many Aboriginal communities and some Aboriginal organizations with respect to community readiness, case study and interview respondents suggested that SPI provide technical assistance and expert opinions in a number of areas. These most notably included areas related to business planning, legal issues, environmental assessment and knowledge regarding specific sectors, such as forestry, mining and fisheries and the commercialization of energy technologies for example.

Case study and interview respondents also suggested, however, that the needs related to the community readiness of communities, which have an impact on the level of their economic participation are very broad. For example, while the majority of SPI projects have focused on increasing the capacity and preparedness of Aboriginal communities to participate in economic development initiatives, the success of some SPI community readiness projects have been hindered by other social issues, such as housing.

Recognizing that to SPI is not intended to address all social and economic needs of communities, evidence suggests that more clearly articulated outcomes related to the projects supported by the SPI initiatives might help to address this issue. For instance, the case studies found that for initiatives and projects that are more limited in scope, the need being addressed appears to be more clearly defined.

To address this issue, it was suggested by some that communities play a more direct role in articulating the needs related to the Initiatives through increased planning, through band council resolutions, letters of support and other means. Further, case study evidence suggested that community needs be considered more holistically in the context of the broader strategic objectives and expected results identified for SPI as a horizontal initiative.  

SPI horizontal approach

The evaluation found a demonstrable need for the horizontal approach SPI employs. This is primarily related to both the complexity of resource-based economic development in the context of Aboriginal participation in economic development initiatives, as well as the fragmented nature of Aboriginal programming.

According to documents reviewed, the landscape of Aboriginal economic development in Canada has changed due to a number of critical factors related to the complexity of resource development opportunities, including their scope, the greater number of resource development initiatives in close proximity to Aboriginal communities and legal requirements for industry and governments to consult with Aboriginal communities prior to economic developments. The involvement of multiple sectors is therefore necessary in order to manage the increased complexity of economic development opportunities.

The need for the involvement of multiple players, however, has resulted in the involvement of a wide variety of stakeholders. Documents reviewed indicate that there are currently more than twenty federal departments and agencies that presently have mandates that include Aboriginal economic development. Documents reviewed and interview respondents have further pointed out that programs offered by these departments have not necessarily encouraged collaboration or have helped to facilitate the identification of shared priorities.

Evidence suggests that without a horizontal and coordinated approach to Aboriginal economic development programming, investments may not be maximizing efforts to be strategic, targeted and timely, with projects and proposals often receiving funding that only address one aspect of the help they need. Further needs may remain unmet and communities are often not successful in preparing for participation in economic development opportunities. Furthermore, an approach that is fragmented and lacking in coordination typically requires that Aboriginal communities and organizations complete multiple applications and meet diverse reporting requirements to access the funds necessary to support a single economic development project. This places additional burden on communities that are already, in many cases, contending with limited capacity with respect to overall business management.

Therefore, a clear and demonstrable need exists for a strategic horizontal approach that will ensure federal investments in Aboriginal economic development are aligned in order to maximize benefits for Aboriginal people. This is supported by the 2007 Report of the Senate Standing Committee on Aboriginal Peoples, which recognized the need for greater coordination within the federal government and recommended it take a lead role in facilitating partnerships between Aboriginal people and the private sector.

Clearly, SPI is addressing a need to support the community readiness of Aboriginal communities in engaging in complex regional economic development opportunities. This is accomplished largely through the SPI process of horizontal collaboration to encourage strategic and targeted investment. However, community readiness with respect to economic participation extends beyond what SPI currently aims to address. Therefore, there is a need to view community readiness more holistically, and continue to consider those needs identified by communities, in the context of economic development planning. Further, SPI is addressing a need to reduce the fragmented nature of economic development programming through its collaborative and horizontal approach, however, there is a need for the SPI to continue to align its objectives and expected results identified for SPI as a horizontal initiative.

3.2 Alignment with Government Priorities

Finding: SPI is aligned with the priorities of Aboriginal Affairs and Northern Development Canada and the Government of Canada. Specifically, SPI provides support for Aboriginal communities to increase their involvement with economic development opportunities and the horizontal structure aligns with a government and departmental focus on streamlining services.

SPI is consistent with the priorities of the Government of Canada and the strategic outcomes established by AANDC. Specifically, in the 2014-2015 Program Alignment Architecture, SPI supports the Land and Economy Strategic Outcome of full participation of First Nations, Métis, Non-Status Indians and Inuit individuals and communities in the economy under the sub-activity of Aboriginal Economic Development and the Program 3.3 Strategic Partnerships.

Additionally, in AANDC's Reports on Plans and Priorities (2012-2013, 2013-2014, and 2014-2015), SPI contributes to the priority "Improving Partnerships and Relationships" by focussing on supporting community readiness activities so that communities are better prepared to engage with industry partners and participate more fully in economic development opportunities. Further, in the 2014-2015 Report on Plans and Priorities, it was noted that SPI contributes to the Land and Economy Strategic Outcome by aligning federal efforts, leveraging investments from other levels of government and the private sector and addressing gaps in programming to ensure that Aboriginal peoples can participate in and benefit from priority regional opportunities and major resource developments.

SPI is aligned with the Government of Canada's strategic, long-term economic plan Advantage Canada: Building a Strong Economy for Canadians, released in 2006, focusing government so that roles and responsibilities are aligned to maximize economic outcomes for Aboriginal peoples; and leveraging investment and promoting partnerships with the private sector to produce sustainable growth for Aboriginal peoples. Additionally, SPI addresses a fragmented federal approach to economic development, which was identified as one of the seven most salient barriers to economic development by the Senate Standing Committee on Aboriginal Peoples in its report Sharing Canada's Prosperity – A hand up not a hand out, released in 2007. SPI also responds to the recommendation of the Auditor General of Canada made in 2006 that AANDC take a more coordinated approach to its programming and streamline related administrative processes.

The alignment of SPI objectives with the priorities of the Government of Canada is further shown through the program's consistency with key messages contained in the Speeches from the Throne between 2010 and 2013, as well as Budget 2012 and 2014, emphasising resource development and the need to increase the participation of Aboriginal peoples in the economy.

The SPI program is designed to support AANDC's mandate to support Canada's Aboriginal and northern peoples in pursuit of healthy and sustainable communities and broader economic and social development objectives. Additionally, SPI is designed to contribute to the achievement of two strategic goals of the Federal Framework for Aboriginal Economic Development:

  • forging new and effective partnerships - primarily among federal departments and Aboriginal stakeholders, but also with provincial and territorial governments and the private sector; and
  • focusing the role of the federal government and fostering a whole-of-government approach that aligns and targets investments towards market driven opportunities in the five key sectors of the economy.

3.3 Alignment with Federal Roles and Priorities

Finding: SPI is consistent with key priorities, including Federal Framework on Aboriginal Economic Development.

Finding: Evidence of complementary programming exists, however, the SPI's horizontal nature enables partnering on SPI initiatives in a way that doesn't duplicate efforts or investments. Rather, the SPI is seen to have the potential for being a model, one that could be replicated, for partnering and collaboration that can be extended to other areas beyond economic development.

As stated in the section above, the SPI program is consistent with the priorities of both the federal government and the mandate of AANDC, as well as supporting the Federal Framework on Aboriginal Economic Development. Additionally, SPI key informant interviews and literature sources identified SPI's horizontal program structure and support for Aboriginal involvement in economic development opportunities as the right approaches to supporting federal departmental priorities.

Specifically, with higher unemployment rates on-reserve, targeted economic development programs, such as SPI, have the potential to generate employment opportunities. SPI's emphasis on increasing the preparedness of communities to play a more meaningful role in resource development projects is an effective approach to improving the economic participation of Aboriginal communities, and is consistent with the findings of the National Aboriginal Economic Development Board in its report Increasing Aboriginal Participation in Major Resource Development Projects. The report noted that increasing the participation of First Nations, Inuit and Métis people in major projects as equity partners, entrepreneurs and workers, as SPI intends, is the most effective way to see these economic opportunities move ahead and help close the socio-economic gaps between Aboriginal and non-Aboriginal peoples.

Additionally, there is a role for the federal government to play in furthering the progress of economic developments by facilitating consultation and collaboration between private enterprises and Aboriginal communities. The horizontal approach employed by SPI is intended to fill this role by aligning the services of the federal government to increase the ability of communities to participate effectively in negotiation or engagement sessions. Moreover, the horizontal approach should facilitate partnerships between relevant partners and eliminate bureaucratic or jurisdictional impediments to economic development projects. In this regard, key informants noted that SPI has increased collaboration between partners and contributed to increasing the involvement of Aboriginal communities and individuals in the economy.

However, an international example of a similar horizontal approach has resulted in disappointing outcomes and shows that placing an emphasis on horizontal collaboration may result in significant challenges in increasing the participation of Aboriginal communities and individuals in the economy. In Australia, the Closing the Gap framework includes an Indigenous Economic Development Strategy, similar to AANDC's Federal Framework on Aboriginal Economic Development, which is intended to provide a blueprint for the Australian Government to work with Indigenous Australians, all levels of government, the private and not-for-profit sectors to generate jobs and economic activities for Indigenous people.

In the most recent report on the status of the framework, it was noted that progress against targets has been disappointing and there has been no progress at all on the employment target. The report found that while a large number of partnerships had been formed, there was too much emphasis placed on bureaucratic processes rather than delivering services and outcomes. It should be noted that the context regarding the horizontal structure of the Closing the Gap framework and the Aboriginal population of Australia are different than those addressed by SPI. However, the framework can provide an example for SPI on the need to remain focused on addressing the needs of Aboriginal communities rather than the horizontal structure of the program.

Overlap and Duplication

From the perspective of economic programming, case studies and key informant interviewees described SPI as being complementary to other programs and noted that the risk of duplication of programming is lowered by having federal departments at the same table through SPI initiatives. The increased communication between federal departments not only allowed participants to identify instances where potential duplication might occur but also provided opportunities to ensure SPI projects are consistent with and complimentary to other government programming.

Evidence of other federal programming relating to Aboriginal economic development through the promotion of collaboration and partnerships was found, notably AANDC's Community Opportunity Readiness Program (previously named the Community Economic Opportunities Program) and the Community Support Services Program.

It was determined that both the Community Opportunity Readiness Program and the Community Support Services Program do not represent duplication with SPI as both provide a focused approach to project-based funding whereby SPI's funding is intended to fill gaps that cannot be addressed by existing programs. Neither the Community Opportunity Readiness Program nor the Community Support Services Program employs a horizontal approach intended to involve multiple partners through a single window approach to facilitate the involvement of Aboriginal communities in economic development opportunities.

Further, the evaluation found no evidence of duplication among federal or provincial jurisdictions in terms of SPI's horizontal collaborative approach to Aboriginal economic development. Documents reviewed and case study and interviewee respondents could not provide any other examples of programming or initiatives that support Aboriginal participation in the economy through a collaborative, partnership, single-window delivery approach.

AANDC Follow-up Audit of Capacity Development, which included a case study of SPI, identified it as an interesting model that could well be applied to other areas of capacity development, such as band governance and management, infrastructure management, and delivery of services.

Further, the willingness to explore the use of the single-window approach can be seen through the Pangnirtung Project: Making Connections for Youth. This AANDC-led pilot project was created to respond to the federal objective to reduce the administrative burden on grants and contribution recipients, with its goal to establish a single-window program/service delivery model that can be used as a tool to flow funds to northern and remote communities.

Indeed, SPI was cited by many case study and interview respondents as having the potential for being a model, one that could be replicated, for partnering and collaboration that can be extended to other areas beyond economic development.

 

4. Evaluation Findings – Performance (Effectiveness/Success)

4.1 Achievement of Expected Outcomes

4.1.1 Program Outcome: Cooperation and collaboration between federal, non-federal partners.

Finding: SPI has resulted in a significant number of partnerships to establish and implement economic development projects, however, clear articulation of what it means to have partnerships within a sector, and why those partnerships are indicative of performance would be useful for performance measurement.

Finding: Partnerships have increased buy-in amongst Aboriginal communities and are thus viewed as a success factor to Aboriginal economic development. From a governance perspective, partnerships have increased communication and an improved ability to both identify opportunities and eliminate potential overlap. There were concerns expressed, however, respecting the sense of "ownership" of projects between AANDC and other partners as well as issues with the communication with partners.

Cooperation and collaboration between federal and non-federal partners was examined using two key indicators as per the SPI Performance Measurement Strategy:

  • number of partnerships developed interdepartmental Letters of Agreement and other types of agreements; and
  • stakeholder perceptions of changes in understanding, co-operation and collaboration due to partnerships.

SPI has resulted in the cooperation and collaboration amongst 15 federal departments and agencies, including AANDC, involved in the key sectors of the economy targeted by the program:

  • Aboriginal Affairs and Northern Development Canada
  • Industry Canada
  • Federal Economic Development Initiative for Northern Ontario
  • Natural Resources Canada
  • Agriculture and Agri-Food Canada
  • Employment and Social Development Canada
  • Canadian Northern Economic Development Agency
  • Parks Canada
  • Federal Economic Development Agency for Southern Ontario
  • Status of Women Canada
  • Western Economic Diversification Canada
  • Fisheries and Oceans Canada
  • Environment CanadaFootnote 5
  • Canadian Environmental Assessment Agency
  • Canada Economic Development for Quebec Regions

SPI, through the Initiatives it supports, has also demonstrated success in the area of cooperation and collaborating through the establishment of partnerships between multiple federal and multiple non-federal partners with the aim of supporting First Nations participation in economic development. In some cases, this collaboration enabled partnerships between organizations that had never existed before. Case study respondents and interviewees indicated, for example, that the SPI initiative Ring of Fire, in 2010-2011, enabled partnerships between the Matawa Tribal Council, the Government of Canada (AANDC, Natural Resources Canada, Federal Economic Development Agency of Northern Ontario and Employment and Social Development Canada), the Government of Ontario (Ministry of Aboriginal Affairs, Ministry of Northern Development and Mines, Ministry of Natural Resources), and industry (Noront Resources, Cliffs Natural Resources, KWG).

The number of partnerships and unique partnerships for each of the sectors and SPI as a whole, based on data reviewed for the period between fiscal year 2010-2011 through to fiscal year 2012-2013 identify that SPI, overall, has resulted in over 100 unique partnerships being established, noting that multiple partnerships with the same entity are only counted once to avoid duplication. The largest share of these new partnerships is in the Forestry and Mining sectors.

Sector Partnerships Unique Partnerships
Agriculture 11 7
Energy 41 19
Fisheries 22 12
Forestry 40 28
Mining 65 27
Tourism 3 3
Shipbuilding 7 7
SPI Total 189 103
 

In addition to formal partnership arrangements, working partnerships have been established through SPI's governance processes via interdepartmental working groups, regional working groups, and implementation teams established for SPI sectors and projects.

However, while the number of partnerships provides a quantitative value with which to measure the achievement of the Immediate Outcome Cooperation and collaboration between federal and Aboriginal partners, it does not tell us a great deal about the level and value of cooperation and collaboration that occurred. For example, while partnerships may have been formed, it is unclear whether they resulted in full collaboration between the parties, if they resulted in benefits for Aboriginal communities, or if those benefits could have been achieved without the partnership. As a result, SPI could better articulate what it means to have partnerships within a sector, and why those partnerships are indicative of performance.

In addition, the evaluation examined the changes in stakeholder perceptions of understanding, co-operation and collaboration due to the partnerships as an indicator of the level of cooperation and collaboration reached between SPI partners. Stakeholders interviewed during the key informant interview process were nearly unanimous in their agreement that partnerships are necessary for Aboriginal economic development, that SPI has resulted in an increase in the number of partnerships and that more partnerships are necessary in the future. Key informant respondents noted that co-operation and collaboration through leveraging, community engagement, partnerships and agreements have all increased throughout the lifespan of SPI.

Specifically, stakeholders commented that partnerships increased the legitimacy of projects for Aboriginal communities, industry and other levels of government. Businesses and industry were more willing to take part in projects when federal partners were already involved. This increased legitimacy made it easier to leverage funds and increased the chances of success for specific SPI projects. However, it should be noted that despite the positive opinions of stakeholders, no additional evidence was found through the literature or document review to confirm that partnerships increase the success of economic development projects in Aboriginal communities.

Key informant respondents also noted that the increased cooperation and collaboration, particularly through SPI working groups, has resulted in increased communication and an improved ability to both identify opportunities and eliminate potential overlap. Respondents further noted that SPI working groups have shown departments who traditionally have not had a responsibility to promote Aboriginal economic development that there is in fact an important role for them to play. The involvement of a greater number of departments was seen as a positive aspect of SPI as it was noted that the Government cannot continue to work in "silos" and a more coordinated approach to supporting Aboriginal communities is needed. In particular, it was noted that with budget reductions taking place in governments across Canada, partnerships are more important than ever to continue to provide services with reduced resources.

However, while there are positive examples of cooperation and collaboration between federal partners and between federal and non-federal partners, progress has been impeded by weak communication of SPI to federal departments and Aboriginal communities. Specifically, staff, internal and external to AANDC, noted that they spent much of their time educating others about SPI due to a lack of awareness regarding the program and how it operated. In some instances, the lack of communication resulted in missed opportunities for valuable partnerships. In the case of one SPI initiative, the project proponents did not learn the role the new Director General Investment Sub-committee would play until shortly before its 2013-2014 application was due, and as a result, faced challenges.

The challenges relating to communication were also noted in the Governance Review conducted by Patterson Creek Consulting, 2011. The Review noted that at the time, the Federal Coordination Committee had not developed a communications strategy and as a result, the governance review recommended the creation of a communications and marketing director general sub-committee to address the problem, however, some key informant interviewees noted that it took a long time for communications materials to be developed.

Finally, case study interviews identified a challenge relating to ownership as it relates to cooperation and collaboration amongst partners. It was observed in case studies that other federal departments regard SPI as being "owned" by AANDC and not truly horizontal. Further, Aboriginal communities do not regard SPI as offering equal partnership with government and noted the minimal representation of Aboriginal people in the governance structure. Some respondents said there is a lack of true partnership and collaboration in developing annual workplans and it was felt that even when workplans were developed through extensive consultation with communities, they were eventually reduced in scope by the federal government.

4.1.2 Program Outcome: Simplified government application, monitoring and reporting (single-window approach)

Finding: A lack of awareness and understanding regarding the mechanisms for implementing a single window structure, including the use of the common Terms and Conditions, has hindered take-up and use of this funding method. However, in projects where a single window approach has been used, it was successful and seen as an ideal for future projects.

The achievement of the program outcome: simplified government application, monitoring and reporting (ie. The single-window approach) was examined using three key indicators as per the SPI Performance Measurement Strategy:

  • number of initiatives where "single window" approach was used;
  • stakeholder perceptions of the administrative burden of SPI projects compared to other initiatives involving multiple departments; and
  • stakeholder perceptions of the value added of the single window aspect of SPI.

According to the documents reviewed, SPI is intended to be different from the usual application based approval and funding process.

SPI allocates funding based on workplans developed collaboratively with Aboriginal communities, federal departments and other partners with a single window approach for recipient communities to access funds and for reporting requirements. In this way, it is hoped that SPI can employ a more coordinated opportunity identification process to eliminate the need for clients to navigate the many federal departments and programs involved with Aboriginal economic development while pursuing specific projects. This coordinated approach is also expected to provide a mechanism to harmonize monitoring and reporting processes as well as the management of financial agreements among relevant partners.

The evaluation found that take-up of the SPI's single-window approach is very limited. Based on the data reviewed and case studies examined, only two of the fourteen SPI Initiatives are currently using the single window approach - the only Northwest Territories Forest Industry and Biomass Initiative and the Atlantic Commercial Fisheries Diversification Initiative. In the case of the Northwest Territories Forest Industry and Biomass Initiative, funding from SPI allocated to AANDC and Natural Resources Canada, along with funding from the regional development agency Canadian Northern Economic Development Agency, all flowed through one contribution agreement between Canadian Northern Economic Development Agency and the Government of the Northwest Territories, due in large part to the persistence of staff in two federal departments.

Overall, respondents felt that in instances where a single window approach had been implemented, it was very beneficial to clients and overall represented an ideal model. In situations where project proponents were not able to fully implement a single window approach, they were able to incorporate some aspects, such as use of the common Terms and Conditions. Many respondents noted that, through SPI, they worked more closely with clients and other departments and this increased collaboration had benefits. Despite the challenges in implementing the single window approach, respondents commented that they believed it would result in a decreased administrative burden and that the decrease would primarily be seen on the client side of the projects.

There are also some examples where simplified reporting and administration have been achieved. For example, the Government of the Northwest Territories provides one annual report to the three federal departments (Canadian Northern Economic Development Agency, Natural Resources Canada, and AANDC), which support the Forest Industry and Biomass Initiative. Additionally, in SPI's tripartite agreement with Ontario as part of the Ring of Fire initiative, the province is using the SPI forms for applications and reporting. But the outcome of a reduced reporting and administrative burden has not been achieved on a large scale. In fact, some partner organizations say that the administrative burden is still significant and consumes the time of their program staff.

While case study and key informant interview respondents generally had positive comments regarding the goal of a single window approach, they identified some challenges.

One hindering factor is the short-term and unpredictable nature of funding for SPI projects. In some cases, projects have been given multi-year funding agreements yet, must have their annual workplans approved each year. In regards to one case study project, respondents from the organization, the federal government and First Nations said the lack of access to stable funding is an impediment. An example provided by one respondent is the difficulty of building a sustainable organization when staff members are all on short-term contracts.

Restrictions on how SPI funding can be used, and the lack of flexibility for re-profiling allocations, are also hindering factors because there is little latitude given for communities and organizations to respond to changing circumstances. Financial planning for complex regional developments requires flexibility since it is impossible to anticipate all contingencies in an annual workplan. For example, multiple interviewees commented that they underestimated the time needed for the community engagement portions of their projects. The time needed to properly engage with communities and gain their support for the project was longer than they expected and as a result, their projects fell behind schedule, which created challenges regarding the re-profiling of funds. Interviewees felt that additional flexibility regarding re-profiling would allow them to adapt to unexpected issues such as these and ensure they continue to effectively deliver their project.

Specifically, the single window approach requires a change in the culture of government and a change in how departments work together. It was noted by respondents that the current culture of governments is one of silos, wherein each department is focused on its own mandate and resources when undertaking a project. While SPI offers a broader and more collaborative approach, there is hesitancy within departments to merge human and financial resources with other organizations and unfamiliarity with the mechanisms to do so. The skepticism within departments regarding the use of collective resources and the confusion around the collaborative mechanisms of SPI are contributing factors to the slow progress in moving towards a single window approach.

A general lack of understanding and knowledge was also noted as a challenge, and may be the root of the skepticism mentioned above. Departments are reluctant to transfer their own funds through the SPI authorities for a range of reasons. The legal and contractual mechanisms for other departments to use the SPI authorities appear to have been poorly communicated. Specifically, interview respondents reported that there is frequently confusion between and within departments regarding how single window financing could work, how it would be reported, the results and measurements that would be required, the Terms and Conditions, and how the project would adhere to the different departmental mandates. Some departments have tried to ensure their work is consistent with the collaborative intent of SPI by delivering projects or programs in close consultation with others; however, they are not willing to commit to the single window approach. Departments interested in taking part in SPI generally have a positive intent to come together to fund projects, yet, acquiring the necessary information regarding the single window approach can be challenging and obtaining approval for such a funding mechanism can be met with resistance. Further, in an environment of deficit reduction departments are wary of signing over funds to another department.

Several misconceptions limit the ability of the single window approach to be used on a wider scale and illustrate the lack of understanding regarding the funding approach. As a result, the departments that do demonstrate some form of cooperation with SPI typically use their own authorities to support Aboriginal organizations and communities instead of the more flexible SPI common authorities. Many respondents spoke of specific departmental funding criteria that must be met, which prevent them from undertaking a single window approach. Specifically, with the exception of the Forest Industry and Biomass Initiative, the regional development agencies, interviewed through case studies, said it would be difficult, if not impossible, for them to use the SPI authorities to flow their funds through another department. A respondent from one of these agencies noted that they still must apply their agency's criteria to Aboriginal economic development, which restricts opportunities. Further, respondents noted that funds cannot be provided without knowledge that they would be used as defined by departmental grants and contributions requirements. However, as the departments and agencies have signed on to the SPI common Terms and Conditions, they do, in fact, have the ability to fund projects using the single window approach even if it does not comply with their regular departmental authorities. This information does not seem to be widely known among departments participating in SPI.

Finally, the evaluation found that it has been challenging to get some federal departments to engage, and participate in, SPI. Interviewees noted that in some instances, departments are fearful that someone else will get the "credit" for collaborative projects undertaken through SPI. The success of SPI partnerships is also hindered by structural impediments that exist in departments. Specifically, respondents noted that some departments were "stuck in their ways" and were unwilling to fund projects that fell outside of their usual funding scope. Further, constricting the ability for departments to partner are the different decision making structures that exist in federal departments. For example, some departments have decision making and funding authorities residing with the Deputy Minister or Minister while others have placed the authority with director generals. As a result, decisions are made by different departments at different times resulting in delays and other challenges to partnerships.

Related to this lack of information is a concern expressed by respondents regarding the high turnover of assistant deputy ministers, director generals and directors involved in SPI, as this often requires a renewed effort in "converting" new people to the value of the SPI single window approach. Respondents commented that there was still an education process to be completed, which would address these misconceptions and knowledge gaps. Additionally, in order to address the perception that SPI is "owned" by AANDC, respondents noted a need for increased support from the Treasury Board of Canada Secretariat to encourage the use of the single window approach and a more coordinated effort at the federal level.

4.1.3 Aboriginal communities engage members/partners and stakeholders on complex regional development initiatives.

Finding: SPI has made progress towards supporting Aboriginal communities in engaging members, partners and stakeholders on complex regional development initiatives.

The evaluation examined the extent to which the program achieved the following outcome: Aboriginal communities engage members/partners and stakeholders on complex regional development initiatives were examined using two key indicators as per the SPI Performance Measurement Strategy:

  • number of Memoranda of Understanding/Impact Benefit Agreements, contractual or other agreements established or in the process of negotiation, between Aboriginal communities and public or private sector partners; and
  • stakeholder perceptions of Aboriginal community and private sector engagement in economic development initiatives.

Based on data available for fiscal year 2010-2011 through to fiscal year 2012-2013, the following 14 initiatives have been reviewedFootnote 6:

 
Aboriginal Commercial Fisheries Diversification Initiative
British Columbia mining

Alberta Oil sands
First Nations Energy Mining Council

Northwest Territories Biomass Energy Strategy
North Vancouver Island Exploration Geosciences – First Nation engagement project

First Nation Power Authority (Saskatchewan)
Quebec mining Matimékush Lac-John

Lower Churchill Hydroelectric project (Atlantic)
Ontario Ring of Fire

Ontario off-grid initiative
Aboriginal Forestry Initiative

British Columbia Remote communities electrification initiative
Aboriginal Agriculture Initiative
 

Modest progress has been made towards supporting Aboriginal communities in engaging members, partners and stakeholders on complex regional development initiatives, specifically through contractual agreements such as Memoranda of Understanding and Impact Benefit Agreements.

For example, the case studies found that SPI had resulted in the following agreements:

  • The Forest Industry and Biomass Initiative, undertaken by the Government of the Northwest Territories, has led to a 2014 Forest Management Agreement between the Territorial government and a joint venture company owned by First Nations and Métis;
  • The First Nations Power Authority has reached agreements with SaskPower to supply a total of 30 MW of power generation (one agreement is still to be finalized);
  • In the Ring of Fire, respondents said that SPI support allowed community chiefs to move ahead on negotiations with the Ontario Government regarding infrastructure for remote communities and some communities have also made significant progress toward Impact Benefit Agreements; and
  • In Quebec, the True North Treasure Initiative supported two Aboriginal communities in negotiating successful Impact Benefit Agreements. Other communities are being supported in negotiations underway with mining companies on two other projects.

Case study respondents highlighted specific instances where SPI successfully increased engagement between Aboriginal communities and the private sector regarding economic development:

  • In one such instance, it was noted that SPI was successful in addressing conflicts between an Aboriginal community and a local mine regarding an existing Impact Benefit Agreement, which led to a renewed partnership going forward.
  • The First Nation Power Authority has also signed an agreement with SaskPower that defines a process for establishing power generation opportunities for First Nations in order to ensure communities, with varying levels of capacity, reach an effective agreement with the provincial utility.
  • In the Ring of Fire Initiative, SPI funding has enabled one tribal council to obtain legal advice and training in developing exploration agreements.

Additionally, SPI has focused on laying the groundwork for Aboriginal communities to effectively negotiate future agreements. For example, a current SPI initiative is focused on community planning with the aim of clearly defining community needs in order to form the basis of future negotiations. It is hoped that, by doing so, communities will be better positioned to negotiate Impact Benefit Agreements that truly represent the identified needs of the community rather than focusing on a few specific benefits.

Several interviewees felt, however, that a greater emphasis should be placed on developing partnerships with the private sector going forward. It was felt that a main focus of SPI so far has been partnerships between federal departments and, while that is important, effort should be made to increase the number of private sector partners involved in SPI projects.

4.1.4 Program Outcome: Aboriginal communities are ready to participate in economic development opportunities

Finding: By addressing community readiness through strategic and targeted investment, SPI is achieving progress towards Aboriginal communities being ready to participate in economic development opportunities. However, the concept of community readiness is vaguely defined, which may prove challenging in assessing progress toward this outcome and therefore, may necessitate further consideration of performance information being collected.

The achievement of the program outcome was examined, as per the SPI Performance Measurement Strategy, by:

  • evidence of increased community capacity to participate (e.g. Aboriginal people trained or mentored, new business partnerships created, business plans produced and implemented and procurement strategies developed).

Overall, the evidence indicates that by addressing capacity development and community readiness through strategic and targeted investment, SPI has made progress towards this outcome.  

Documents reviewed cited a key challenge to ensuring the readiness of Aboriginal communities is the general lack of capacity in the communities, including a need for skilled workers, management capacity, technical capacity of community leaders, and a more thorough knowledge and understanding of the process involved in the developmental projects and their long-term benefits. The Energy Sector Reference Paper from March 2011 noted that few communities had the required skill set in trades, leadership and technical capacity in relation to renewable energy, sustainable development, engagement and project planningFootnote 7.

In case studies as well as interviews, respondents continued to note that capacity building efforts related to engagement and skill and leadership development were required to allow Aboriginal communities to participate more fully in economic development. This increased capacity would allow communities to manage the economic processes themselves, as opposed to letting a developer manage projects for them, and potentially not take the communities' interests into consideration.

A review of data indicates that the majority of reported activities undertaken by initiatives were related to the development and implementation of partnerships, business plans, and procurement strategies, followed by engagement and training activities for both trades and management.

Efforts and progress towards engaging partners and developing relationships were continuously noted in internal progress reports, speaking to the ongoing nature of this particular activity – a pillar of all SPI initiatives. To date, 103 unique partnerships have been established across all initiatives. The collaboration of stakeholders, including Aboriginal organizations, has helped identify a variety of capacity needs at different levels and provided training and engagement activities to target areas, including trade skills development, management skills and training, cultural sensitivity training, negotiations training, and support and guidance in development and implementing business plans.

Across all 19 SPI Initiatives, there were close to 300 reports of activities related to determining needs and increasing the capacity of Aboriginal organizations to benefit from SPI.

The collaborative efforts of the partnerships have resulted in an increased capacity for management as well as skilled workers in the communities. Interviewees and case studies from initiatives that have been concluded indicate that increased capacity has been seen. For example, the Atlantic Commercial Fisheries Diversification Initiative established business development teams to provide technical business development capacity to interested Aboriginal communities. These in turn helped communities to develop proposals, establish business and procurement strategies, as well as establish several new businesses, expand others and create 250 new jobs for the communities involved. The First Nation Power Authority also reflects positive results, particularly noted by interviewees as a successful example of capacity building. Communities that could benefit from the First Nation Power Authority originally did not know how to get involved in the power generation opportunity. However, First Nation Power Authority created a vehicle for agreement negotiation and community education about the power generation industry. This gave the communities the information and capacity to take part in larger economic development opportunities.

Such positive outcomes from earlier initiatives seem to be an encouraging indication for newer initiatives that are just beginning their capacity building and engagement strategies. The British Columbia Liquid Natural Gas project is currently undergoing an engagement strategy that will increase awareness about the Liquid Natural Gas industry and inform communities about the employment opportunities. Similar to earlier initiatives, this will be followed up with feasibility and needs assessments to determine capacity gaps and inform training and business strategies. Another initiative, the Ring of Fire, has also focused on building capacity to increase the communities' ability to participate in discussions with mining companies and the provincial government, as well as increasing environmental capacity and engagement.

However, while planning and implementing business plans and strategies are essential to develop economic business, identifying these factors as indicators of community readiness may be an over extension. Identifying business plans as an indicator may also be problematic as each initiative would involve a business plan of some sort to identify skills and areas of interest. Most initiatives were able to discuss business plans and some others were able to discuss and implement procurement strategies, yet, these tended to revolve around skill development and engagement. As such, a more valid indicator may be required.

The majority of initiatives have demonstrated the considerable effort required before many communities are ready to even start planning for businesses and procurement. Such activities require a healthy community that is willing to engage in the development activities for the longer term, have certain skills within the workforce and management, and have considerable buy-in and understanding from the communities of how the process works and that it will work for them. In light of this, it may be more accurate to determine community readiness via the previously noted factors, and build on them as initiatives have done, as opposed to developing business plans and procurement strategies.

Finally, it was noted during case studies that for unhealthy communities, economic development can be challenging. Sudden increases in economic development and incomes can increase substance abuse, resulting in lost employment, as well as have detrimental impacts on child-rearingFootnote 8 Footnote 9. Further, increased availability of employment can result in decreases in school attendance, as well as a variety in career preparation outside of skills related to the sector-related economic development (i.e. fishing, mining, forestry, etc.)Footnote 10. Other research has indicated that Aboriginal community leaders, due to a lack of understanding in financial management, may struggle with adequately managing an influx of economic resources, resulting in more long-term harm to the community and a loss of investmentFootnote 11.

In light of this, a finding from the case studies suggests that an assessment of current social concerns be considered when assessing economic prosperity, and to potentially put in place programs and/or measures to ensure negative impacts are minimized. This in turn may assist the community to maintain ongoing economic prosperity after SPI is no longer investing in and guiding the economic development of the Aboriginal communities involved.

The evaluation recognizes however that as many of these SPI initiatives are in the developmental stages, there is a great deal of groundwork that needs to be completed before communities are fully prepared to participate in the variety of potential economic opportunities that are available. Overall, however, the evidence indicates that SPI has made progress towards Aboriginal communities being ready to participate in economic development opportunities but more consideration of the performance information being collected is required.

4.1.5 Program Outcome: Federal investments in Aboriginal economic development opportunities are aligned

Finding: There is some evidence that federal investments in Aboriginal economic development opportunities are aligned, however, it is difficult to assess the degree to which investments are aligned.

The achievement of this program outcome was examined, as per the SPI Performance Measurement Strategy, by:

  • Evidence of alignment (e.g. development /implementation of comprehensive workplans)

Proposals available for 12 of the initiatives include the articulation of overarching goals, current partners, funding received from SPI, estimated allotment of SPI funds for each planned activity, and the independent initiative governance structure, either planned or in existence.

A review of the process indicates that once proposals are approved, a comprehensive workplan is developed which identifies key players – including federal and non-federal organizations – as well as start up activities (i.e. engagement, training, procurement strategies, and negotiations), which were to be aligned with the overall goal (i.e. new and expanded Aboriginal businesses or other economic development opportunities realized).

Progress reports and summaries reflected the considerable investment and progress made on the beginning stages of workplans. Workplans for those initiatives, approved in 2013, were frequently broken down into their component parts, identified as activities, however, concrete numbers on how many workplans and projects existed are not available. Project summary and expenditure tables provided information for 15 initiatives from 2010-2011 until 2012-2013, two of which were renewed and renamed the following year, and two of which were funded for one year exploratory efforts only.

It is important to note that projects did not have a singular "type", and some incorporated a number of objectives that had some degree of interrelation (i.e. one project may identify any combination of engagement, feasibility analysis, capacity building, research, and procurement activities). Further, as the progress reports and project summaries are reported differently, the following values should be viewed as estimates.

The governance structures developed by the initiatives are designed to help alignment and include interdepartmental working groups and steering committees. Working groups bring together federal and provincial departments, private industry and Aboriginal organizations and help ensure that the actions of these partners are aligned with the objectives for SPI. For example, a document review of Ring of Fire indicates that that the governance structure coordinates the actions and investments of 12 partnering departments and agencies. The Aboriginal Business Development Teams by the Aboriginal Aquaculture Initiative are also a good example. These teams provided ongoing support for investments, and provide business development services to those Aboriginal business people who are interested in using the expertise of the Business Development Teams to improve or build their business. The Labrador Trough initiative also identifies a steering committee comprised of three sub-committees, each with a singular focus related to the end goal of Aboriginal Economic Development. Other initiatives identify that a governance structure has been developed or is being planned and will address and coordinate workplan related activities such as infrastructure-related negotiations, workshops and forums for input on skills and training programs related to the sector.

Generally, document review, interviews, and progress reports suggest that federal investments are aligned; however, there is limited quantitative data available to augment the qualitative and largely anecdotal evidence. While workplans provided some information (including the monies supplied by partners, the estimated SPI monies spent on activities, working group details regarding purpose and focus, and the areas of concentration for departments), this information was not available consistently or with the same level of detail for all initiatives. As such, a thorough analysis that would more clearly identify the degree of alignment with Aboriginal economic development was not available at this time.

Ideally, detailed information regarding the funds transferred from departments and where actual funds were spent (related to activity), should be articulated to have an ongoing measure of alignment.

At this stage, using the development or implementation of comprehensive workplan as an indicator of alignment with departmental objectives may not be appropriate. Specific workplans are reported infrequently, and instead, general details are provided regarding efforts made to increase capacity related to the initiative. Alternatively, reporting requirements may need to be altered in order to address this indicator. Specific details identifying a workplan, with its related activities listed, the roles of each partner, the funding contributed to it, and what needs the activities would address would provide more clear information on the project activities, as well as provide more relevant data on multiple lines of questioning for later assessment.

4.1.6 Program Outcome: Aboriginal communities benefit from federal strategic partnerships and investments

Finding: There is some indication that, communities have benefitted from federal strategic partnerships and investments. However, it is difficult to assess the relative impact of SPI due to both the newness of the initiative, and the fact that its contributions need to be assessed alongside the incremental contributions of other funders and initiatives. SPI programming performance should include performance measures that are laid out in specific projects to provide for a better assessment of economic impacts that are attributable to SPI-supported projects.

Due to the existing capacity of many Aboriginal communities, many initiatives had to "start from scratch" in terms of preparing Aboriginal communities to be ready to engage in economic development. As such, many communities have benefited from involvement with SPI, although perhaps not yet in terms of specific economic opportunities being realized or in businesses being started or expanded. Rather, for many communities, the benefit is in terms of the development that has occurred to make the community ready to engage in economic development opportunities.

Data review indicates that, in total, 489 individual communities have engaged in the initiatives from fiscal year 2010-2011 to fiscal year 2013-2014. In this time, 103 individual partnerships between communities and private, federal, and non-federal organizations have been established. As of yet, the exact number and value of economic development opportunities realized during this timeframe is unavailable, particularly due to SPI still being in its early stages.

Overall, there are 310 reports of progress towards new or expanded Aboriginal Business and other opportunities in Aboriginal communities. This includes feasibility analyses, negotiations and collaboration for business opportunities, inventory assessments, equipment procurement, supply and opportunity assessments, as well as the development of proposals and business plans, resource development specific to business opportunities, and guidance and support supplied to Aboriginal communities to be competitive in a variety of business opportunities. Each of these activities can be translated into benefits for the Aboriginal communities, as they allow greater knowledge sharing and add to the skills and abilities of the people in these communities, which can be applied to community development or with other opportunities.

Due to many of the communities still being in the early stages of initiative development, specific details on opportunities actualized are not yet available. However, one initiative was able to provide more specific details on new or expanded businesses. The Atlantic Commercial Fisheries Diversification Initiative identified 27 businesses that have been impacted by their initiative, either through expansion or creation. Further, there have also been 250 new jobs created, with 31 existing positions maintained. Other SPI Initiatives would do well to establish information collecting mechanisms that allow for this type of meaningful reporting.

As SPI has contributed a portion of the overall investment in these projects, the exact return on investment for SPI cannot be clearly calculated. However, collectively, the Atlantic Commercial Fisheries Diversification Initiative, the British Columbia Tourism Initiative, British Columbia Mining Initiative, Lower Churchill Hydro Initiative, and the First Nation Power Authority, anticipate combined revenues of an estimated $2.2 billion by 2016.

As discussed above, broader impacts, whether economic or social, are very difficult to measure for a number of reasons. Perhaps the most significant limitation is the ability to attribute any observed changes to the incremental contribution of SPI funding. Presumably, social impacts could be measured using time series measures from the National Household Survey to compare changes over time between "like" communities on economic and social indicators. Specifically, one could examine community-level data on income and labour force at multiple census years (i.e., 1996 to 2015) to see whether any increase is above the trends of other communities (controlling for isolation, region and population) and regressing any changes against the relative size of projects supported by SPI. However, with a myriad of other projects related or unrelated to government economic development investments, and other factors that may impact social and economic development, it is likely that impacts of SPI would be lost in measurement error and the measure of its relative impact obscured.

It is therefore essential for SPI investments to include, in Terms and Conditions or Contribution Agreements, some protocols for the measure of economic and social impacts. Specifically, when a project is funded it should have measures in place to assess job growth; proxy measures of economic growth and measures of long-term viability of business opportunities that are spun off of SPI-supported initiatives. It is also important to do this strategically and purposefully so as not to increase the reporting burden to recipients or participants. Thus, consideration should be given to emphasising reporting requirements for performance and reducing other compliance or administrative reporting requirements that may not be necessary.

4.2 Governance Structure

Finding: Governance structures are still being defined and will need continued review and refinement given the relative newness of the initiative. Specific performance indicators for the effectiveness of SPI's governance structures should be included in future performance measurement strategies.

The horizontal nature of SPI's activities represents a unique approach within government and as a result, a new and untested governance structure has been implemented. This governance structure has suffered from some "teething problems" as described by Patterson Creek Consulting in its SPI governance review and has been altered in order to improve its overall effectiveness.

Specifically, Patterson Creek Consulting noted that problems have arisen related to three areas: the Federal Coordination Committee and its structures and processes; barriers to horizontal collaboration; and sector working papers. As a result, the report included seven recommendations, the most substantial of which was the establishment of an investment board and a different approach to sector papers. Additionally, the governance review included a recommendation that Sector Reference Papers be updated, as the documents included only brief sections on opportunities and likely fell short of their intended Federal Coordination Committee purpose.

As a result, following the governance review the Strategic Initiatives and Partnerships Directorate refocused SPI with an opportunity-based approach and established a Federal Coordination sub-committee at the directors general level, in charge of operational matters. Interviewees felt that the Director General Committee has been successful by having a more varied representation than the Federal Coordination Committee; and that directors general are more engaged, provide thoughtful contributions to the approval process and have served as a valuable challenge function. In contrast, the evaluation has found that the Federal Coordination Committee has not played a sufficiently strong role in SPI implementation and in bringing other departments on board. Overall, these changes have been viewed as positive and the SPI governance has improved as a result of their implementation.

Views on the purpose and continued role of the Federal Coordination Committee were also mixed. Some case study and interview respondents felt there may be some duplication of effort between the  the Federal Coordination Committee and the Director General Investment Sub-Committee that is involved in project approval and that, as a result, some respondents were of the view that  the Federal Coordination Committee could be eliminated, leaving the Director General Investment Committee or senior management at AANDC to make final decisions on funding. Other respondents felt that there is a continued role for the Federal Coordination Committee to provide overall strategic direction for SPI in terms of sectors and opportunities to focus on for upcoming funding cycles and that this approach would be more valuable as it can ensure SPI remains consistent with the priorities of the Government and the Federal Framework for Aboriginal Economic Development.

Further, the governance structure extends beyond the Headquarters process as SPI funded all have their own governance structures with a steering committee and working groups on different themes. In some instances, most notably the Atlantic Commercial Fisheries Diversification Initiative, the project level governance structure was unique, innovative and represented a best practice.

4.3 Facilitating Factors

In addition to the information noted, the evaluation sought specific factors that helped facilitate SPI's achievement of objectives.  

One key factor identified was the ability of SPI to fill funding gaps. Case study and key informant interview respondents, including community representatives at the project level, indicated that projects could not have been undertaken without support from SPI. Interviewees praised the program for having funding parameters that reach projects that do not fit with other funding sources.

Secondly, the current interest in resource development by the federal government, industry and Aboriginal communities has been a facilitating factor for SPI as a whole and many of its individual projects.

Finally, and most frequently noted, was the value of having committed people in government and among the partners and stakeholders. Respondents in multiple case studies noted that SPI created valuable partnerships with Aboriginal partners, whose commitment allowed them to achieve a great deal with small amounts of financial support.

4.4 Extent to which differences in socio-economic realities of women and men have been addressed in SPI initiatives

Finding: Gender-based data is not being systematically considered or collected through SPI performance information.

Information reviewed as part of the data review, as well as information from case study and interview respondents indicate that no SPI initiative has conducted a formal analysis of gender, however, one project has been keeping track of the employment of women and men in the opportunities created, and will be including a gender analysis for training strategies that are being put in place. Other projects – the Northern Greenhouse Initiative and the British Columbia Mining Strategy – are incorporating projects focus exclusively on female participation. Those interviewed as part of case studies agreed that there would be utility in collecting gender-based information that would lend itself for conducting gender-based analysis.

4.5 Extent to which SPI initiatives support AANDC's responsibilities under the Federal Sustainability Development Strategy

Although SPI initiatives are still in the early stages of development and implementation, there has been varying degrees of consideration regarding the environmental impacts of SPI projects. One project is working with Environment Canada to increase the participation of Aboriginal individuals on environmental assessments. Some others are undertaking environmental studies to comply with legislation, so long as it is related to the developments being undertaken as part of the SPI project itself. Many of the projects are focused on community readiness, and in some of these cases, the environmental impact studies would be separate from the SPI projects, and would be considered by the organization that is undertaking the development. Another project, while not directly related to the initiative, has been moving communities from diesel generation to more renewable sources of energy, such as biomass. This has the potential to stimulate a market for biomass fuel as the initiative develops.

4.6 Effectiveness of Performance Measurement Strategy

Finding: There are notable concerns with the SPI Performance Measurement Strategy, the most significant of which include assessing the relative contribution of SPI to longer-term economic outcomes in communities.

Overall, a review of the data currently being collected found that much of the information being collected is anecdotal in nature and the majority of the reports reflected the beginning phases of the initiatives, without specific numbers related to the outcomes. A review of the Performance Measurement Strategy found that reporting practices were minimal and lacked structure for reporting on the breadth of results achieved by SPI projects.

There were notable limitations cited among interviewees with the current Performance Measurement Strategy. A noted concern was regarding what was being measured and being considered "success". It was noted, for example, that communities may define success quite differently than project proponents. Their focus, for example, may be simply to strengthen the community's ability to sit in and contribute to negotiations and bolster their collaboration with other departments.

Further, socio-economic concerns such as employment, education and substance abuse would likely detract from objectives related to SPI projects. Such issues need to be carefully considered when articulating objectives, and emphasis needs to be placed on establishing trust, buy-in, and strengthening capacity in communities. With current measures, social impact is not captured directly.

In further refining performance measurement, it is important to keep in mind the role of SPI in creating and strengthening partnerships, and to be cautious about ascribing social and economic impacts that are directly attributable to these partnerships. The emphasis of the relative contribution of SPI may be more appropriately placed on assessing the degree to which SPI has made projects a reality. Measuring broader economic and social impacts may be better left to performance measurement of the projects themselves; particularly given the high degree of variability of the relative contribution of SPI funds versus all other investments.

Finally, there was also concern that limiting the reporting would also not allow initiatives to report on their unique results that may not have fit specifically in the three identified performance indicators.

The evaluation notes that a new performance measurement strategy is due to be developed following the completion of this evaluation.

 

5. Evaluation Findings – Performance (Efficiency and Economy)

5.1 Cost-effectiveness/Cost-efficiency

Finding: The value of investments leveraged and the in each project are not adequately captured so as to allow for a complete analysis of cost-effectiveness and cost-efficiency.

Finding: Participants in this evaluation had mixed views regarding the efficiency of the SPI program. Generally, interviewees found that, while the horizontal approach may be more labour intensive than other economic development programs, the partnerships achieved are worth the effort.

Case study interviewees indicated that the majority of projects would not have gone ahead without SPI. However, at this time, few initiatives have been able to provide an accurate estimate of the value of the potential outcomes for communities, or the specifics on time and effort involved in the participation.

Measuring return on investment is difficult as there is a high degree of variability in the relative contribution of SPI to projects compared to all other investments, and therefore, incremental impact is unclear. However, four initiatives have provided estimated values of the anticipated revenues of SPI related projects. Collectively, the Atlantic Commercial Fisheries Diversification Initiative, the British Columbia Tourism Initiative, British Columbia Mining Initiative, Lower Churchill Hydro Initiative, and the First Nation Power Authority, anticipate combined revenues of an estimated $2.2 billion by 2016.

It is important to remember that the costs for program delivery will vary greatly from initiative to initiative, depending on the requirements of each community, such as basic needs in socio-economic requirements, to training in relevant trades and guidance in developing business plans. One initiative may require fewer funds due to the existing economic capacity of the communities, while another may require more investment. Still, other communities may have the knowledge and capacity but require procurement funds. Additionally, some initiatives may have more involved or more renowned partners who encourage greater buy-in, while others will be met with reticence in potential business partners or communities, which will result in greater time and effort to encourage these groups to get involved. Given that many of the initiatives are entrenched in start-up projects, there will be a great deal of up-front work involved before there is a considerable return on the investment of that time and money.

Currently, anecdotal evidence indicates that the existing design and delivery of SPI is not as cost-effective as it had been anticipated to be. Generally, interviewees found it to be more labour intensive than other economic development programs, yet, it was also acknowledged that it is a different way of working, and that the increased work involved was part of the "cost of doing business" when working horizontally. It was noted that as they become more familiar with a region, they become more efficient, and another commended the governance review, as it had increased their efficiency. It was also noted that as those involved in SPI are still adapting and learning the new method of doing business, true cost-effective results would be seen at a later time, when people were more experienced with the process.

There were few recommendations on alternatives. It was noted that a simple model that allowed communities to spend their funding on their own economic development priorities may be more effective. In this manner, it would reveal where communities would pool their funds, and if that would be on major projects, or smaller ones. However, it was acknowledged that this may not guarantee results on partnerships and collaboration, and that there would also be a risk that recipients could become dependent on these funds.

5.2 Alternatives, Lessons Learned, Best Practices

Improvements/alternatives identified

The evaluation found that SPI is operating efficiently given the unique nature of the program and its objective. As a result, no specific delivery alternatives were proposed to achieve greater efficiency. However, as with most programs, organisations experience significant pressure with delays in funding.

Another important suggestion was that the SPI governance structures engage Aboriginal stakeholders more directly. This suggestion is in line with a recommendation made by Douglas R. Eyford in his report to the Prime Minister Forging Partnerships Building Relationships: Aboriginal Canadians and Energy Development on the importance of early political engagement with First Nations in resource development. The need for better communication was also highlighted, especially around changes to the ways projects are selected and approved and by improving the information flow to communities to deepen their understanding of initiatives and potential impacts.

Lessons Learned

Through key informant interviews, multiple project proponents commented that they experienced unexpected delays due to the time needed for community engagement. Specifically, the time needed to properly engage communities and gain their support for projects was substantially longer than originally anticipated and led to multiple projects falling behind schedule. As a result, project proponents had difficulty spending the allotted funding within the time period. In response to this issue, interviewees suggested multiple potential solutions, including: increasing awareness among SPI applicants regarding the time needed for effective community engagement; increasing the flexibility around the re-profiling of funds to allow programs that have fallen behind schedule to move funds to later years of the project; providing funding for community engagement in addition to other aspects of SPI projects.

As noted earlier in this report, SPI has encountered challenges in establishing and maintaining the involvement of other federal government departments and regional development agencies. In specific cases, senior management from AANDC have reached out to other departments to provide information and education regarding the purpose of SPI and the opportunities the program presents. In such cases, this outreach was found to be effective and further outreach could encourage senior management throughout the federal government to increase their participation in SPI. Consideration should be given to ensuring the timelines for projects are reasonable to allow projects to achieve intended results. SPI is intended to fill gaps in funding and not provide ongoing funding for continuing projects, although some successful projects have resulted in becoming "mainstreamed" and incorporated into the budgets of participating departments, however, this is generally not the intention of the SPI Initiative. For example, in the case of Atlantic Commercial Fisheries Diversification Initiative, the initiative was put in place to respond to a short-term market issue resulting from a reduction in snow crab quota and called for a strategy/investments to diversify the industry. At the end of the three year initiative, the impacts on the community owned businesses operating in the sector were mitigated, and as result, the program was deemed to be a success, in spite of the program ending.

Best Practices

The project governance structure utilized by the Atlantic Commercial Fisheries Diversification Initiative was identified as a best practice. The initiative built upon an ongoing program operated by the Department of Fisheries and Oceans Canada in order to develop a governance structure, which provided assistance to First Nations, applied a rigorous review process to determine eligible projects and above all was trusted and valued by First Nations participants. In this case, business development teams, comprised of subject matter experts, were established in the Ulnooweg Development Group to assist communities in the development of business plans and proposals. A review committee was established to determine what projects would be funded and included representation from two federal departments and two Aboriginal organizations. The decisions made by the committee were informed by the project proposals as well as a proposal review conducted by an independent third party evaluator. Finally, project implementation teams, including community members and members of the business development teams, were established to assist with the establishment and operation of the newly formed businesses. The partnerships formed with regional Aboriginal organizations, the formation of business development teams and project implementation teams provided communities with support to ensure their projects were implemented successfully.

The success of the Atlantic Commercial Fisheries Diversification Initiative reinforces the opinions of interviewees who noted that engaging communities in consultative processes to determine priorities and plan strategically for capacity building and economic development was a best practice. Additionally, investing and partnering with regional Aboriginal organizations was also seen as a best practice, as opposed to attempting to build expertise in each community.

The horizontal nature of SPI and the promotion of partnerships were identified by multiple interviewees as a best practice itself. The Government often encounters challenges as departments operate in "silos" and SPI represents a unique approach to encourage collaboration to ensure Aboriginal participation in large and complex economic development opportunities.

 

6. Conclusions and Recommendations

Relevance

There is a clear need to support Aboriginal communities in engaging in complex regional economic development opportunities, through the SPI process of horizontal collaboration, which encourages strategic and targeted investment.  

SPI is addressing a need to support the community readiness of Aboriginal communities in engaging in complex regional economic development opportunities largely through the SPI process of horizontal collaboration to encourage strategic and targeted investment. However, community readiness needs of Aboriginal communities are broader than what SPI aims to address and therefore, there is a continued need to consider broader community readiness needs in the context planning and, as well a need to align objectives and expected results identified for SPI as a horizontal initiative.

Objectives are consistent and aligned with department and government wide roles and priorities. SPI is consistent with key priorities, including Federal Framework on Aboriginal Economic Development and is aligned with the priorities of Aboriginal Affairs and Northern Development Canada and the Government of Canada. Specifically, SPI provides support for Aboriginal communities to increase their involvement with economic development opportunities and the horizontal structure aligns with a government and departmental focus on streamlining services.

Complementary programming exists, however, the SPI’s horizontal nature enables partnering on SPI initiatives in a way that doesn’t duplicate efforts or investments. Rather, the SPI is seen to have the potential for being a model, one that could be replicated, for partnering and collaboration that can be extended to other areas beyond economic development.

Performance

Overall, SPI is making progress toward the achievement of its immediate objectives: SPI has resulted in a significant number of partnerships, progress has been made towards supporting Aboriginal communities in engaging members, partners and stakeholders on complex regional development initiatives and some, albeit, limited take-up has been made on using the single-window approach.

A lack of awareness and understanding on the part of partners regarding the mechanisms for implementing a single window structure contribute to its limited take-up and use, which could be addressed through better communication and Central Agency guidance.

The complexities and challenges, however, associated with tracking the performance of Horizontal Initiatives against expected program results present a challenge for the Initiative as a whole. Without a common set of strategic objectives and expected results amongst partners, SPI runs a risk of being delivered in a fractured fashion, without alignment.

There are concerns with the SPI Performance Measurement Strategy, the most significant of which include assessing the relative contribution of SPI to longer-term economic outcomes in communities. Notably, it is difficult to assess the degree to which investments of partnering departments are aligned.

The Initiative is still in its early days. New partnerships are being forged and governance structures are still being defined. While there is a need to improve upon the SPI programming performance for a better assessment of economic impacts that are attributable to the SPI Initiative, through its SPI-supported projects, the value of having committed people in government and among the partners and stakeholders resulting from the achievement of cooperation and collaboration amongst partners is a facilitating factor in SPI’s progress toward the achievement of success.

Recommendations

Based on the evaluation’s findings and conclusions, it is therefore recommended that with regard to the horizontal Aboriginal Economic Development Strategic Partnerships Initiative, that AANDC:

  1. Strengthens the interface between SPI and existing Aboriginal Economic Development programming within the Department, and with federal and provincial partners to better align strategic objectives and expected results.
  2. Strengthens community engagement on potential opportunities to be considered under SPI with a view to ensuring needs of the community are considered adequately.
  3. Works with Central Agencies and federal partners to increase the use of the single window delivery approach under SPI.
  4. Continues efforts to monitor and review the SPI governance system, including the role of the Federal Coordination Committee, in order to ensure for consistency and maximize efficiencies.
  5. Continues to lead the development of the revised Performance Measurement Strategy with partners in order to enable the monitoring and measurement of results as they relate to both community readiness and longer-term outcomes related to partnerships and investments, including leveraging and to allow for gender-based analysis.
 

Annex A - Draft Evaluation Matrix

Issues / Questions Indicators Literature Review Document Review Key Informant Interviews Financial and Admin Data Review Econometric Analysis Case Studies
AANDC Representatives at Headquarters and Regions Major Stakeholders (other government departments, Provincial / Territorial governments) Aboriginal
Organizations and
Representatives
Community Members
Non-Stakeholder Experts
Relevance (Continued Need)
1. Does SPI address demonstrable needs of Aboriginal communities, and to what extent are federal strategic partnerships and investments essential in meeting those needs? Evidence of continued need to facilitate cross sectoral/departmental approaches to support Aboriginal participation in complex regional development initiatives Selected Selected   Selected Selected       Selected
Evidence of responsiveness to the needs of Aboriginal communities   Selected Selected Selected Selected Selected     Selected
Relevance (Alignment with Government Priorities)
2. Are the objectives of the program consistent with government wide priorities and AANDC's strategic outcomes? Alignment of SPI objectives with government wide priorities and AANDC's strategic outcomes.   Selected Selected            
Relevance (Alignment with Federal Roles and Responsibilities)
3. To what extent are the objectives of the program aligned with the roles and responsibilities of the federal government? Linkage of SPI objectives to Federal roles and responsibilities.   Selected Selected            
Evidence SPI adds to or supports Federal Framework for Aboriginal Economic Development Priorities   Selected Selected Selected Selected Selected      
4. Is there duplication or overlap with other programs, policies or initiatives? Evidence of duplication or overlap between SPI and other initiatives. Selected Selected Selected Selected Selected Selected     Selected
Performance (Effectiveness/Success)
5. To what extent is progress being made toward achievement of expected outcomes? See below.                  
5.1 Cooperation and collaboration between federal, non-federal partners.
(Immediate Outcome)
Number of partnerships (ILAs and other types of agreements) developed   Selected Selected Selected Selected Selected     Selected
Stakeholder perceptions of changes in understanding, co-operation and collaboration due to partnerships     Selected Selected Selected Selected     Selected
5.2 Simplified government application, monitoring and reporting (single window).
(Immediate Outcome)
Number of initiatives where "single window" approach was used.   Selected Selected Selected Selected        
Stakeholder perceptions of the administrative burden of SPI projects compared to other initiatives involving multiple departments.     Selected Selected Selected Selected     Selected
Stakeholder perceptions of the value added of the single window aspect of SPI     Selected Selected Selected Selected     Selected
5.3 Aboriginal communities engage members/ partners and stakeholders on complex/regional development initiatives.
(Immediate Outcome)
Number of Memoranda of Understanding/Impact Benefit Agreements, contractual or other agreements established or in the process of negotiation between Aboriginal communities and public or private sector partners.   Selected             Selected
Stakeholder perceptions of Aboriginal community and private sector engagement in economic development initiatives.     Selected Selected Selected Selected     Selected
5.4 Aboriginal communities are ready to participate in economic development opportunities.
(Intermediate Outcome)
Evidence of increased community capacity to participate (e.g. Aboriginal people trained or mentored, new business partnerships created, business plans produced and implemented, procurement strategies developed)   Selected Selected Selected Selected Selected Selected   Selected
5.5 Federal investments in Aboriginal economic development are aligned
(Intermediate Outcome)
Evidence of alignment, e.g. development/implementation of comprehensive workplans   Selected Selected       Selected    
5.6. Aboriginal communities benefit from federal strategic partnerships and investments (Ultimate outcome) Number and value of economic development opportunities realized   Selected Selected Selected Selected   Selected   Selected
Number of jobs created   Selected Selected Selected Selected       Selected
Number of new or expanded Aboriginal businesses in SPI partner communities by sector (mining, forestry, agriculture, fisheries) and other opportunities.   Selected Selected Selected Selected   Selected   Selected
6. How effective is the SPI governance structure? Evidence of successful implementation of SPI's governance structure   Selected Selected Selected Selected        
Opinions of staff and key stakeholders on the effectiveness of SPI's governance structure.     Selected Selected Selected        
7. What are the factors that are facilitating or hindering the achievement of SPI outcomes? Evidence of appropriate mechanisms (e.g. adequate communication about SPI, opportunity identification processes, timeliness of investment decisions, partnership coordination, funding mechanisms, delivery processes)   Selected Selected Selected Selected Selected     Selected
Barriers identified   Selected Selected Selected Selected Selected     Selected
8. To what extent have the differences in the socio-economic realities of women and men been addressed in SPI initiatives? Percentage of initiatives/projects that have been subject to gender-based analysis   Selected Selected Selected Selected       Selected
Percentage of initiatives with plans to a) encourage the participation of women, and b) to fill gender gaps in relation to women's equal access to resources and the benefits of development   Selected Selected Selected Selected       Selected
9. To what extent do SPI initiatives support AANDC's responsibilities under the Federal Sustainable Development Strategy? Evidence that environmental impacts are considered in SPI projects.   Selected Selected Selected Selected       Selected
10. How effective is SPI's performance measurement strategy and how could it be improved? Evidence the Performance Measurement Strategy is being implemented and updated   Selected Selected Selected          
Disaggregated (male/female) performance measurement data is being tracked   Selected Selected           Selected
Opinions of staff and key stakeholders regarding the utility of the Performance Measurement system for accountability, management decision making and learning     Selected Selected Selected        
Performance (Efficiency and Economy)
11. How could SPI be improved? Are there alternative or more efficient ways of delivering SPI that would achieve similar results? Improvements/alternatives identified Selected Selected Selected Selected Selected Selected     Selected
12. How cost effective is the design and delivery of SPI services? Do more economic alternatives exist? Have simplified application and reporting processes resulted in cost savings for government or the private sector? Value of SPI investments compared to their costs (Time and effort required for participation) Selected Selected Selected Selected Selected   Selected Selected Selected
Program reach (# of communities/population) vs. program expenditures   Selected Selected       Selected Selected Selected
Alternatives suggested Selected   Selected Selected          
Other Evaluation Issues (Lessons/best practices)
13. What are the lessons learned and best practices that emerged from implementation of SPI, or from similar programs from other jurisdictions, countries or the private sector that may enhance SPI effectiveness? Evidence of lessons/best practices identified and applied Selected   Selected Selected Selected Selected     Selected
 
 

Annex B - Case Study Annexes

First Nation Power Authority Case Study Summary

Summary of Program information
Number of Communities Involved 1 - Meadow Lake Tribal Council
Number of Partnerships 3
SPI Funding: $484,500.00
Major Stakeholders First Nations communities
Tribal Council Development Corporations
Private sector
AANDC
Western Economic Diversification Canada
SaskPower
Economic/Business Category Energy
 

Interviews Held for Case Study

Interviews
Representative Number of Representatives Interviewed
Federal Government Representatives 2
First Nation Power Authority representative 1
First Nations Tribal Councils and communities 3
First Nations development corporations 3
Representatives of SaskPower 2
Independent expert 1
 

Key Findings from the Case Study

Summary of Key Findings
Type of Finding Finding
Relevance:
  • The findings indicate that the First Nation Power Authority addresses demonstrable needs of Aboriginal Communities. With an increasing population, and increasing unemployment rates, the power projects serve as an opportunity for long term employment and a reliable source of energy. Interviewees reported that First Nation Power Authority can be a reliable source to provide advice and knowledge on power projects, assisting both Aboriginal communities and SaskPower.
  • Views were mixed about the need for Federal strategic partnerships and SPI particularly. Generally, aside from the funding from SPI, the program was determined to be similar to current programming that was already available, such as the Community Economic Opportunities Program. Most value was seen in the First Nation Power Authority itself. Without federal funding, it still would have occurred, but more slowly.
Performance Effectiveness / Success
  • FNPA's strong relationship with the crown was determined to be the most significant achievement to date. No other external group has the same level of access and insight into Saskpowers internal operations as the First Nation Power Authority. First Nation Power Authority has also spent time to educate SaskPower on the needs and rights of First Nations and how they can be integrated into plans for power generation. First Nation Power Authority has also increased communication between First Nation communities, corporations and private sector companies.
  • The single window approach has not been used for funding the First Nation Power Authority, as other departments thought it would slow down the project by adding another layer of approvals. Further, departmental representative said that federal collaboration on funding the projects would have happened without SPI.
  • The First Nation Power Authorities most significant achievements to date are its agreements with SPI, which are 'set aside' agreements providing an opportunity for First Nation Power Authority to prove it can deliver on a large project.
  • The support of Meadow Lake Tribal Council also helped reinforce the relationship with SaskPower during negotiations of other projects, and First Nation Power Authority unveiled its first small power project in April 2014, a test project before it is scaled up to larger application. The advice of the First Nation Power Authority regarding power projects was also valued, as it prevented poor investment and business choices that had been suggested by other consultants. Overall First Nation Power Authority, with SaskPower, has made progress to engaging First Nations in complex regional development opportunities.
  • Direct benefits were determined to be seen more in the long term, and will be the result of First Nation Power Authority's assistance in helping First Nations learn how to structure opportunities and turn them into business adventures.
Design and Delivery
  • One interviewee reported that the governance structure was not effective.
  • FNPAs relationship with SaskPower was noted as one of the most important factors to the achievement of the outcomes. Also, First Nation Power Authority provided credibility to the organization and was able to provide access to funding.
  • Challenges included lack of planning and funding for the long term, as well as the timeliness of approvals.
  • There have been struggles in measuring First Nation Power Authorityies accomplishments and reporting those within the SPI matrix. These have been attributed to the limited staff available.
Economy and Efficiency
  • Regarding the First Nation Power Authority specifically, it was noted that improvements could be made by increasing funding and staff capacity.
  • Regarding SPI itself, the formation of a regional working group was suggested to improve efficiency.
Lessons Learned:
  • A volunteer board comprised of tribal leaders and experienced business people has been established, and works to maintain its independence from politics.
 

NWT Forest Industry and Biomass Initiative Case Study Summary

Summary of Program information
Number of Communities Involved 7
Number of Partnerships 3
SPI Funding: $350,000.00
Major Stakeholders AANDC
Canadian Northern Economic Development Agency
Natural Resources Canada
Government of Northwest Territories – Department of Environment and Natural Resources
First Nation Communities – Behchoko, Fort McPherson, Fort Resolution, Fort Providence, Jean Marie River, Kakisa, and Yellowknife
Economic/Business Category Forestry/Biomass
 

Interviews Held for Case Study

Interviews
Representative Number of Representatives Interviewed
Federal Government Representatives 2
Staff and consultants working for the territorial government 3
Representatives of Aboriginal Communities and Corporations 3
Northwest Territories Legislative Assembly Members 1
Business Representative 1
Independent consultant 1
Environmental Organization Representative 1
 

Key Findings from the Case Study

Summary of Key Findings
Type of Finding Finding
Relevance:
  • The Forestry Industry and Biomass Initiative are relevant and address a need to reduce high cost of energy and lack of employment in Northwest Territories communities. The jobs provided by the initiative fit well with the traditional activities, and will help address the high unemployment rates in the area. However, the small communities with the forestry resources have little capacity to develop them, and will need significant support to do so.
  • There is some concern about the economic viability of the mil, regarding the supply and demand of the pellets, the proposed exportation of the pellets unused by Aboriginal communities, and the competition from other pellet-producing operations
  • The partnerships between federal and territorial governments are crucial to initiative, and that with the funding through SPI, the initiative has been able to move forward at a much faster pace.
  • While respondents did not see any duplication, one noted that there were similarities between SPI and Community Economic Opportunities Program. However, the greater flexibility of SPI better addressed the need of the forestry initiative.
Performance Effectiveness / Success
  • Strong collaboration was confirmed between federal government departments. The Government of Northwest Territories and non-governmental partners. Relationships have been strengthened via annual meetings and the collaboration with Natural Resources Canada can be linked to SPI funding.
  • The Single Window approach has been successful in this initiative, and has significantly reduced the application, monitoring and reporting burden.
  • The most significant achievement to date has been the Financial Management Agreement signed between the Government of Northwest Territories and Timberworks Inc. Further, Timberworks Inc represents the first time the two opposing First Nation groups involved have worked together in over 20 years. The Financial Management Agreement also allows the community to have a stronger say in the harvesting and planning of the business venture.
  • Through the funding, an Aboriginal-owned biomass heating system has also been installed. Harvesting has been delayed as permission from the wildlife and management boards has not yet been requested.
  • Direct benefits have been limited, including short term employment related to harvesting and reduction in heating costs in the first season, which may provide benefits over the long term.
Design and Delivery
  • The SPI governance structure was not found to be effective by respondents, reporting that changes (i.e. the establishment of the Director General Investment Subcommittee) had not been communicated effectively and that there was duplication in the structure regarding the submissions to the Director General Investment Subcomittee and the Federal Coordination Committee. One suggestion for improvement was to have proposals and Committee questions shared across departments, to increase understanding.
  • Facilitating factors to SPI outcomes included having staff that were experienced in working with Aboriginal communities and committed to the goals, as well as the strong working relationships between partners. Annual in-person meetings were also seen as a facilitating factor.
  • Identified challenges included a lack of capacity in the Government of Northwest Territories and communities. With few human resources, progress on the initiative was limited. Also, other issues within the communities competed for the attention of the Chief and Council of the communities. Politics between the Dene and the Métis also impeded initiative progress.
  • Short-term and reduced funding was also noted as a challenge that impacted the initiative, as it resulted on having to scale back on planned projects
  • While generally found beneficial, Natural Resources Canada noted difficulties with the single window approach, citing that it took considerable time and effort, and consultations from lawyers and Treasury Board Secretariat to provide contribution to Canadian Northern Economic Development Agency
  • Finally, the selection process for SPI projects was also noted as a concern, and interviewees suggested that greater community input is needed to articulate the need for initiatives.
  • There has been no gender-based analysis  conducted, and disaggregated data is not collected.
  • Analysis has been conducted regarding Sustainable development and there are efforts to minimize impact on hunting areas and those of cultural significance. A full-scale environmental assessment has also been requested by environmentalists; however forestry officials are concerned that this may be too costly, and delay the project to the point of its downfall.
  • Regarding performance measurement, reporting was provided by activities, not by outcome, and some work has been done to developing indicators. However, there is no results chain in the PM strategy to identify how the activities will lead to project outcomes, and no data has been collected. There seems to be a limited understanding of results-based management.
Economy and Efficiency
  • Increased and multi-year funding was noted as potential improvements to SPI economy and efficiency. Also, better communication regarding changes and how projects are selected and approved, through all levels of partnerships, was suggested
Lessons Learned:
  • Results take time on such a complex initiative
  • Community champions and community buy-in are required for success
  • Initiatives benefit from experienced and committed officials.
 

Ring of Fire Initiative Case Study Summary

Summary of Program information
Number of Communities Involved 9
Number of Partnerships 7
SPI Funding: $2,911,669.00
Major Stakeholders AANDC
Employment and Skills Development Canada
The Canadian Environmental Assessment Agency
Environment Canada
Federal Economic Development Agency for Ontario
  Ministry of Northern Development and Mines
  Provincial Departments including: Aboriginal Affairs, Natural Resources, Children and Youth Services, Health and Environment
  Aboriginal Partner Organizations: Matawa First Nations Management (referred to as Matawa Tribal Council), Mushkegowuk Tribal Council, Wabatek, and Nishinabi Aski Development Fund
Economic/Business Category Mining
 

Interviews Held and Communities Visited for Case Study

Interviews
Representative Number of Representatives Interviewed
Federal Government Representatives 7
Provincial Government Representatives 1
Aboriginal Communities and Organizations Representatives 13
Independent Consultants 4
 

Key Findings from the Case Study

Summary of Key Findings
Type of Finding Finding
Relevance
  • The findings indicate that Ring of Fire addressed a limited range of needs which are defined by the Government. There is inadequate attention paid to those identified by the community, such as housing and community well-being. It was also found that the consultation processes of the communities and Aboriginal Partner Organizations were not respected. While acknowledging the efforts to address some social and health concerns, there is generalized frustration about the housing and nutrition concerns in the North. Further, it was noted that funding provided is not meeting the financial needs to be relevant players in the Initiative. Unmet needs also included: support to communities for a regional economic development strategy; support to economic develops officers in communities; community engagement; and tools to improve financial literacy.
  • While there are needs still to be addressed, the Aboriginal partners confirmed that the federal strategic partnerships are necessary for efficiency, and reduce the passing of requests between provincial and federal governments.
  • There was no report of duplication, but that with such broad conditions, there was some overlap. Duplication was avoided by including multiple federal players.
Performance Effectiveness/Success
  • There was mixed progress regarding the increased cooperation and collaboration between federal and non-federal partners. While this was increased for federal and provincial partners, Aboriginal communities noted that the type of "communication and collaboration" they are being offered is not in line with what they have requested. It was noted that the progress may be slowed by a difference in mandates between federal departments, and "territorial" attitudes. Specific concerns with communication and collaboration were also cited, including: poor communication and control of knowledge about SPI; a sense of partnership was not fostered, with partners feeling they were being spoken to, as opposed to spoken with; a common approach has not been established between the 9 Matawa communities, resulting in damaged trust between them.
  • There has been no progress in adopting a single window approach, but some success in aligning federal investments. Reasons for not using the single window approach included: perception that SPI is an AANDC program and cannot be used in other departments; in fiscally restrictive environment, there is a desire to show the value of the money the department spent, as benefits the department; SPI Terms and Conditions need some adjustment to work with all departments; unable to ensure accountability for funds signed to AANDC. Despite this, SPI has resulted in some money used from the Community Economic Opportunities Program, and there was increased departmental awareness to the region and improved inter-directorate co-operation.
  • There is limited progress on engaging partners and stakeholder son complex regional development initiative. In this, SPI has been seen as instrumental in helping chiefs proceed on complex initiatives. Communities have also made significant progress towards Impact Benefit Agreements
  • There has been modest progress on communities being ready to participate in economic development opportunities. While some communities have capacity concerns, the following progress has been noted: Increased capacity to engaging in complex discussion with other partners; increased capacity in environmental assessment and monitoring; knowledge exchange from counterparts; increased education and training; improving infrastructure; and addressing health and community wellbeing.
  • There is limited progress on communities benefiting from federal strategic partnerships and investments. Improvements noted above contribute to this. Infrastructure projects have contributed to building multipurpose centres, training efforts have led to leveraging money from a mining company, and SPI funding has helped develop proposals for an environment, health and financial initiatives.
Design and Delivery
  • Facilitating factors on the federal side were noted as the focus within departments and the profile placed on the initiatives. Aside from this, another assisting factor was the ability of Aboriginal organization to maximize the funding received from SPI.
  • Highlighted challenges were the uncertainty and delays in decisions on SPI funding allocation. This creates serious problems in project planning and completion. The restrictions on the funding were also noted as a challenge. This included the inability to carry over a surplus, as well as a number of procedural and accounting barriers faced when they wanted to make adjustments to spending and budget. Communities also commented that they found that provincial staff members were more cooperative than federal staff, and that SPI did not offer greater access to other federal departments.
  • It was also noted that SPI needs to have a greater presence in the federal government as it is currently being viewed as a temporary measure.
  • There has been no gender analysis completed on this initiative, and it is expected that any efforts will come from the grassroots. There has not been any discussion with AANDC around gender.
  • Sustainable development is led by the Canadian Environmental Assessment Agency, and this department is fully engaged. SPI has increased community engagement by helping communities understand the processes and by working with the Four Rivers unit. However, it was noted that more work could be done to help present information in ways that communities can understand.
  • Regarding Performance Measurement, the SPI Performance Measurement Strategy is not suited to the capacity of Ring of Fire communities, and should be addressing community readiness as opposed to tangible outcomes. Most communities are simply not ready to engage in that level of development.
Economy and Efficiency
  • One of the suggested improvements to SPI was allowing Aboriginal Communities to determine their own priorities. Another was for early political involvement, with more visits to communities to increase understanding and better inform funding decisions. It was also noted that have the Federal Coordination Committee and the Directors General Investment Sub-Committee was duplication in effort. Other suggested improvements included a longer funding term, with amendments to avoid yearly renegotiations, and increased funding for administrative support in partner organizations.
Lessons Learned
  • Generally, there is a need to allow a generous timeline for communities to develop. Those who are not on the ground of the initiative may have unrealistic expectations.
  • Best practices included: community consultation, and investing in a group that can provide specialized expertise at a regional level, as opposed to trying to create that expertise in each community.
 

True North Treasure Initiative Case Study Summary

Summary of Program information
Number of Communities Involved 26
Number of Partnerships 7
Working Groups 3
SPI Funding: $2,330,000.00 for 2013/14
Major Stakeholders Corporation de Développement Économique Montagnaise
First Nation Human Resource Development Commission of Quebec
First Nation of Quebec and Labrador Economic Development Commission
Private Sector: Arcelor Mital; Cliffs Natural Resources; Iron Ore Company of Canada; Labrador Iron Mines
26 communities comprised of Inuit, Innu, Naskapi, and Cree First Nations
Economic/Business Category Mining and Supply Services
 

Interviews Held and Communities Visited for Case Study

Interviews
Representative Number of Representatives Interviewed
Federal Government Representatives 4
Provincial Government Representatives 1 (Staff Member)
Aboriginal Communities and Organizations 6
Business Representatives 1
Education and Research Representatives 2
Communities and Dates Visited
Community Date of Site Visited
Uashat April 2014
 

Key Findings from the Case Study

Summary of Key Findings
Type of Finding Finding
Relevance:
  • The continued relevance of the project has been confirmed, in that it addresses demonstrable needs of Aboriginal Communities. However, which needs the program meets tends to vary. AANDC representatives state that the initiative addresses needs related to governance and economic development as well as social development. In comparison, First Nations communities see the program as fulfilling the needs of the governance and economic development, but are lacking in the area of social development. There has also been note of a lack of communication between Aboriginal and federal government stakeholders regarding community needs and that there is a Top-Down process that requires more interaction between Aboriginal, governmental and non-governmental stakeholders.
  • Regarding the need for a federal strategic partnership, this approach has also been viewed positively, as using a "one-stop shop" increased efficiency, as well as increasing communication between federal and Aboriginal partners. Overall, this method of planning and implementing a program did meet the needs of all players involved, and there was no concern regarding duplication or overlap with other programs.
Performance Effectiveness / Success
  • In terms of increased and improved collaboration between federal and non-federal stakeholders, there is not a strong sense of collaboration of using the SPI mechanisms among federal departments, particularly in the Quebec region. Respondents reported that participation between federal departments seemed cumbersome, and was not widely engaged in. Others noted that there were not enough human or financial resources to address the workload required by SPI, and that program criteria was not flexible enough to encourage SPI use. It is important to note that these concerns may be due to the fact that SPI is still in its early stages. Positive indications include: AANDC Quebec region in collaboration with Employment and Skills Development Canada and the provincial government; potential collaboration with the regions primary higher educational institution; and a new partnership between the AANDC regional office and an Aboriginal organization.
  • Establishing a single-window approach has also met with some difficulties, though, this may also be the result of SPI not yet being fully established. One Aboriginal organization found that the approach was getting closer to its intended design, however, this may be due to the abilities of the organization – they were able to carry out preliminary analysis of projects – making the process easier. Conversely, other organizations found no improvement to the overall process through SPI, finding that they had to contact other partners individually. As the implementation of SPI is still relatively recent, over time these concerns can be addressed and resolved appropriately.
  • Regarding the engagement of Aboriginal communities with members/partners and stakeholders, there has been a very positive response. The case study confirmed that an agreement is under negotiation for a new mining project, as well as a database project, with support from the Corporation de Développement Économique Montagnaise, to address accountability. A second project involved a feasibility study developed by the local Aboriginal community.
  • The case study found some progress regarding the readiness of Aboriginal communities to participate in economic development opportunities. Activities to this extent included workshops and training material on agreement negotiation, training sessions for elected officials and labourers, conducting studies of economic needs of companies and assessment of economic and labour development resources. Overall, the study indicates that there is still much to do bring communities to a level where they can participate independently in economic development opportunities.
  • There is evidence of progress towards Aboriginal communities benefiting from the federal partnerships and investments. As a result of True North Treasure Initiative, 40 jobs were created, legal expertise was acquired, 4 more agreements were signed, and new projects are underway.
  • Other: The Secrétariat aux affaires autochtones in the Quebec government is attempting to model the SPI approach.
Design and Delivery
  • There is little positive feedback regarding the SPI governance structure, with only one of the True North Treasure Initiative working groups noting progress, and one Aboriginal partner organization reporting satisfaction with its role and responsibility in the committee. Other respondents noted concerns about the structure and role of the Federal Coordination Committee, suggesting they need to provide strong guidance in what SPI and other guiding groups do beyond having departments sign the agreements. There is a general consensus that more guidance and assistance on the implementation of initiatives and problem-solving is required. Further there is concern regarding timelines for meetings, strict resource allocations and restrictions, as well as limited Aboriginal representation at committees.
  • There was good feedback on the facilitating and hindering factors towards the achievement of SPI outcomes. Facilitating factors included: SPI authority makes AANDC more effective and bring forward synergies; the knowledge of the AANDC True North Treasure Initiative team regarding partnerships and federal connections; and the partnerships made possible through SPI.
  • Limiting factors were varied, noting a lack of human and financial resources to address the increased workload of the initiative, as well as a lack of communication within and among communities regarding projects, and a need for clarity of communication regarding the fiscal environment for Aboriginal business. Also of concern was the timeframe for SPI initiatives – as opposed to careful planning, programming is reactive and may be rushed. Conversely, the slow pace of decision making by the federal government was also noted as a hindrance.
  • No gender-based analysis has been conducted, but a general analysis from SPI has been included in the Performance Measurement Framework. There was recognition that working women will need tools such as daycare. Also, some organizations and software being used already disaggregates by gender. There are still some Aboriginal Organizations that show a low level of awareness of the potential gender dimensions of SPI.
  • SPI also supports AANDC's responsibilities under the Federal Sustainable Development Strategy. While environmental assessment procedures are being followed, improvements could be made. In particular, the Initiative could provide assistance in Aboriginal companies gaining expertise in Environmental Restoration. Further, it was suggested to conduct an environment impact consultation, reaching out and receiving greater input from communities.
Economy and Efficiency
  • Areas where there could be improvements for efficiency include: stronger direction from senior federal levels to obtain the right engagement; improved implementation procedures; and better results targeting. There are also concerns to create a dedicated position or cultivate expertise within the regional office to manage the True North Treasure Initiative.
Lessons Learned:
  • Lessons learned and Best Practices Include: increasing and maintaining in-person meetings, due to the culture of Aboriginal communities; to continue competitive bidding for the external contracts; the experience and process used by the First Nation Human Resource Development Commission of Quebec in its work with the toolkit and the labour force profile – having Aboriginal colleagues evaluate the communities was appreciated by the Aboriginal people.
 

Atlantic Commercial Fisheries Diversification Initiative Case Study Summary

Summary of Program information
Number of Communities Involved 2
Number of Partnerships 3
SPI Funding: $8,000,000.00
Major Stakeholders Fisheries and Oceans Canada
Aboriginal Affairs and Northern Development Canada
The Atlantic Policy Congress
Ulnooweg Development Corporation
Economic/Business Category Fisheries
 

Interviews Held for Case Study

Interviews
Representative Number of Representatives Interviewed
Federal Government Representatives 2
First Nations communities 5
First Nations development corporations 2
First Nations organizations 1
Independent consultant 1
 

Key Findings from the Case Study

Summary of Key Findings
Type of Finding Finding
Relevance:
  • The findings indicate that the Atlantic Commercial Fisheries Diversification Initiative addressed a need to assist communities in diversifying their business interests related to fisheries. Atlantic Commercial Fisheries Diversification Initiative filled a funding gap by providing funds for fisheries related businesses (including trucking, aquaculture and restaurants) that fell outside of the scope of the Department of Fisheries and Oceans programming.
  • The partnership between AANDC and Department of Fisheries and Oceans was crucial to the initiative and provided funding for community projects that would not have qualified through other sources.
  • One respondent did note overlap between the projects funded by the Atlantic Commercial Fisheries Diversification Initiative and those funded by Aboriginal Business Development Canada. However, steps were taken to ensure that the funding provided by the two sources was coordinated. As a result, the funds from the two programs were used in complimentary ways and some unique funding packages were developed for communities.
Performance Effectiveness / Success
  • A single window approach has been used with Department of Fisheries and Oceans and AANDC using existing funding agreements to flow money to the Atlantic Policy Congress who in turn created a contract with the Ulnooweg Development Corporation to develop Business Development Teams to support communities. Applications and reporting was provided to one source, the Project Review Board, and assistance was provided for the reporting from the Business Development Teams.
  • The role played by the Ulnooweg Development Corporation Business Development Teams in supporting communities in the development of comprehensive business plans, applications for funding and the implementation of projects was seen as extremely valuable and led to an increase in capacity for participating communities.
  • The most significant achievement of the Atlantic Commercial Fisheries Diversification Initiative has been the creation of new jobs and businesses. Specifically, the Atlantic Commercial Fisheries Diversification Initiative resulted in the creation of 251 jobs, maintained 31 existing jobs, and created new businesses with an annual projected sales revenue of $27.8M.
  • Direct benefits to the communities include increased employment, cost savings/increased revenues for band owned businesses (in some cases), diversification of the local economy, increased business development capacity and increased community pride.
Design and Delivery
  • Few interviewees from the Atlantic region had experience with, or participated in the SPI governance structure. The proposals for SPI funding and reporting requirements were handled by Department of Fisheries and Oceans Headquarters staff, who were generally pleased with the governance structure.
  • A key challenge was identified regarding the "mainstreaming" of funding for SPI projects. SPI is intended to fill gaps in funding but not provide long term funding for programming. Instead, when programs are successful it is expected that they will be brought into the mainstream funding of a participating department. In the case of Atlantic Commercial Fisheries Diversification Initiative as the funding from SPI reached the end of its three year commitment participating departments were unable to mainstream the funding and as a result the program ended despite its success.
  • The project governance structure implemented for Atlantic Commercial Fisheries Diversification Initiative was highlighted as a major success and best practice. There are a number of critical levels to the governance structure that has led to its success, including:
  • Community Support: The Business Development Teams developed through the Ulnooweg Development Corporation were vital in providing support and capacity building services for communities through the development of business plans and project applications. The Business Development Teams continued to support the communities with the implementation of the projects and the completion of reporting requirements. This support was seen as valuable by communities.
  • Project Approval: A Project Review Board was established to review applications and approve projects. The Board had a varied and representative membership including: Department of Fisheries and Oceans, Atlantic Policy Congress, AANDC, First Nations chiefs and non-Aboriginal business people. Overall the review process was seen as fair and impartial.
  • Third Party Evaluation: A third party evaluator was employed to conduct a neutral assessment of all applications regarding their feasibility and the quality of the business plans. This process increased the quality of proposals and increased confidence in the impartiality of the process.
  • Reporting: The business plans developed by project applicants with the assistance of the Business Development Teams were used as the basis for the Terms and Conditions for. As a result, project applicants are more willing to report on them. Additionally, assistance from the Business Development Teams is provided for meeting the reporting requirements.
  • Reporting on gender was completed and projects met the environmental assessment requirements related to the specific projects they were implementing.
Economy and Efficiency
  • Respondents noted two potential improvements related to SPI: increasing the timeliness of funding decisions; and establishing a forum for Regional Economic Development Officers to discuss SPI projects.
Lessons Learned:
  • The structure of the Atlantic Commercial Fisheries Diversification Initiative led to increased buy-in from communities because the Business Development Teams were the face of the initiative, while the government departments only played background supporting role. Interviewees commented that due to the respect communities had for the Business Development Teams and the positive relationship they had developed the communities were willing to undertake application and reporting requirements they may have resisted had they been imposed by the federal government.
  • The projects were developed through the business plans developed by the communities, with assistance from the Business Development Teams. As a result, the projects represented the needs and interests of the communities and represented a ground-up approach to project development that was seen as contributing to its success.
 
 
 
 
Date modified: