Audit of Economic Development Programs

Date : February 2014
Project No. 13-44

PDF Version (184 Kb, 29 Pages)

 

Table of contents

Acronyms

AANDC

Aboriginal Affairs and Northern Development Canada

ABDP

Aboriginal Business Development Program

AEP

Aboriginal Entrepreneurship Program

AFI

Aboriginal Financial Institutions

ATC

Access to Capital

BC

British Columbia

CEDP

Community Economic Development Program

CEOP

Community Economic Opportunities Program

CORP

Community Opportunity Readiness Program

CIDM

Comprehensive Information Document Management system

CSSP

Community Support Services Program

DGIOC

Director General Implementation and Operations Committee

EDP

Economic Development Programs

FIP

Financial Institutions Program

GCIMS

Grants and Contributions Information Management System

LED

Lands and Economic Development

LEDSP

Lands and Economic Development Services Program

MRED

Major Resource and Energy Development

NACCA

National Aboriginal Capital Corporation Association

PDP Program Delivery Partners
 

 

Executive Summary

Background

An Audit of Economic Development Programs was included in the 2013-2014 to 2015-2016 Risk-based Audit Plan approved by the AANDC Deputy Minister on February 27, 2013. The audit was identified as a very high priority as the programming is significant to the strategic objective of full participation of First Nations, Inuit and Métis individuals and communities in the economy.

AANDC Economic Development Programs (EDP), is comprised of a number of programs which aim to build and promote viable Aboriginal businesses and opportunity-ready communities. This programming supports the vision of increasing the participation of First Nations, Inuit, and Métis individuals and communities in the Canadian economy and enables Aboriginal people to pursue the same opportunities for employment, income and wealth creation as other Canadians. Consistent with its mandate and vision, AANDC is the lead department responsible for carrying out the Government of Canada's Federal Framework for Aboriginal Economic Development, which was released in June 2009 to address commitments made in Budget 2008 to develop a new framework that was opportunity-driven, results-focused and partnership-based. The Framework guides federal actions across many departments and agencies by pursuing the following strategic priorities:

  • Strengthening Aboriginal entrepreneurship
  • Developing Aboriginal human capital
  • Enhancing the value of Aboriginal assets
  • Forging new and effective partnerships

Over the long term, the expected outcomes of these strategies are opportunity-ready Aboriginal communities, viable Aboriginal businesses and a skilled Aboriginal workforce.

AANDC delivers a number of programs aimed at economic development and is renovating its programming to better align with the Framework. Current programming can be categorized into three main types: Aboriginal Business Development, Community Economic Development and Lands and Environmental Management.

Through its Aboriginal Business Development programs and initiatives, AANDC aims to strengthen Aboriginal entrepreneurship by enabling access to capital, identifying and facilitating business opportunities and supporting Aboriginal business promotion activities.

AANDC’s Community Economic Development programs support First Nation and Inuit communities by providing core funding and proposal-based funding. Programming focuses on building the capacity of communities to provide the economic development services needed to activate and enhance their respective economic assets and opportunities.

Lands and Environmental Management are also considered key to unlocking the economic development potential of First Nation lands.

Economic Development Programming is managed at Headquarters by the Lands and Economic Development (LED) Sector. Program delivery, depending on the program, is by regional offices, LED Sector staff, or through a third party.

Based on the audit work conducted during the planning phase, the audit team determined that the audit scope should focus on Economic Development programs listed in Table 1 below. Considerations for selection included program funding amount, program activity during the scope period, and degree of program change during the scope period. For additional details on the in-scope programs listed below, please refer to Appendix B.

Table 1.

Program 2012-13 Funding
(approx)
Program delivery Funding type
Community Economic Development (funding to First Nations Communities)
Community Economic Development Program (CEDP) $52 million Regional offices Core (formula based) funding
Community Economic Opportunities Program (CEOP) $23 million Regional offices Targeted (proposal based) funding
Community Support Services Program (CSSP) $8.3 million Regional offices, and LED Sector Targeted (proposal based) funding
Aboriginal Business Development (Funding to aboriginal owned, and community owned businesses)
Aboriginal Business Development Program (ABDP) $35.6 million Third party delivery (Aboriginal Financial Institutions) Targeted (proposal based) funding
Major Resource and Energy Development (MRED) $6.8 million LED Sector, and third party delivery Targeted (proposal based) funding
 

Audit Objective, Scope and Approach

The objective of the audit was to assess the adequacy, effectiveness, and efficiency of the management control framework of the EDP. The audit assessed the extent to which the EDP processes ensure that program delivery is consistent with the policies, processes and procedures of the programs. More specifically, the audit assessed whether:

  • Effective management and oversight bodies are established and functioning to ensure that there is effective governance and monitoring necessary to achieve EDP objectives;
  • Management has in place an adequate, efficient and effective risk management approach to ensure that there is a documented, well communicated and understood process to identify, assess, and implement risk mitigating measures related to EDP; and
  • Adequate, effective, and efficient controls are in place to ensure that the EDPs are designed, delivered, reported, and monitored consistent with program requirements including policies, processes and procedures.

The scope of the audit included the responsibilities of the LED Sector and regional offices in support of the EDPs. The scope, however, did not include the responsibilities of the recipients of the EDP funding. Through audit planning, and based on program spending, level of activity during scope period, and the degree of change over the scope period, the programs identified in the background section were included in the audit scope.

The audit scope included the period April 1, 2011 to September 30, 2013 and included an examination of the controls in place in the LED Sector and the British Columbia (BC), Ontario, Alberta, and Atlantic Regional Offices.

The audit plan was developed using a risk-based approach through which key risks facing AANDC, particularly the LED Sector’s obligations in support of the EDP, were identified through interviews with management and documentation review. Key risks were then linked to the core governance, risk management and control policies, processes and practices in place within AANDC that are designed to mitigate the risks identified. This information was used to determine the objectives, scope, and criteria for the audit. Audit criteria developed for this audit are included in Appendix A.

Statement of Conformance

This audit conforms with the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program.

Strengths

Throughout the audit fieldwork, the audit team observed examples of strong practices in place in the delivery of programming. This resulted in the following positive findings:

  • BC Region created a strategic planning document that aligned available economic development programs to three tiers of community economic development; Tier 1 – Nation Building; Tier 2 – Asset Development; and Tier 3 – Access to Capital. This approach helped to clarify the regional strategy and priorities and related activities for the year.
  • BC and Ontario Regions have emphasized the use of program funding by communities to develop 5 year strategic plans (e.g. strategic economic development plans, land use plans, and community plans). The audit found the emphasis on the development of 5-year strategic plans by First Nation communities to be a strong practice for strategically managing Economic Development funding resources

Conclusion

Generally, the audit found that control practices related to the in-scope economic development programs are adequate and effective. Some opportunities for improvement were noted to strengthen management practices in the following areas: performance measurement; risk management; quality assurance and compliance; and information management.

Recommendations

The audit team identified areas where control practices and processes could be improved, resulting in four recommendations, as follows:

  1. The Assistant Deputy Minister of the Lands and Economic Development Sector should, as part of revising its Performance Measurement Strategy, design and implement key performance measures and indicators, which provide metrics to objectively evaluate and demonstrate the impact of community-based economic development programs as a key input into the ongoing assessment of the design and delivery of these programs. For the Aboriginal Business Development Program, LED should clarify how project performance data will be collected by third party Program Delivery Partners and communicated to AANDC.
  2. The Assistant Deputy Minister of the Lands and Economic Development Sector should include, as part of its annual planning for these programs, identification and assessment of Economic Development risks and determine appropriate mitigation measures required to further support the achievement of program objectives. Where possible and appropriate the program risk work already being done in regions should be leveraged to support the efficiency and effectiveness of this assessment.
  3. The Assistant Deputy Minister of the Lands and Economic Development Sector should develop a quality assurance review and compliance framework and practices to provide AANDC with assurance over the effectiveness of Program Delivery Partner program management processes and controls in the delivery of the Aboriginal Business Development Program. The compliance framework and practices should be adjusted to ensure AANDC continues to receive assurance over program delivery once the National Aboriginal Capital Corporation Association takes on the Department’s program coordination role.
  4. The Assistant Deputy Minister of the Lands and Economic Development Sector should, as program realignment is rolled out and roles and responsibilities are clarified, emphasize the use of the Grants and Contributions Information Management System as the system of record, and clarify the need for complete documentation, to support the efficient and effective management of AANDC economic development programs.

Management Response

Management is in agreement with the findings, has accepted the recommendations included in the report, and has developed a management action plan to address them. The management action plan has been integrated in this report.

 

 

1. Background

1.1 Introduction

An Audit of Economic Development Programs was included in the 2013-2014 to 2015-2016 Risk-based Audit Plan approved by the AANDC Deputy Minister on February 27, 2013. The audit was identified as a very high priority as the programming is significant to the strategic objective of full participation of First Nations, Inuit and Métis individuals and communities in the economy.

1.2 Economic Development Programs

AANDC Economic Development Programs (EDP), is comprised of a number of programs which aim to build and promote viable Aboriginal businesses and opportunity-ready communities. This programming supports the vision of increasing the participation of First Nations, Inuit, and Métis individuals and communities in the Canadian economy and enables Aboriginal people to pursue the same opportunities for employment, income and wealth creation as other Canadians.

Consistent with its mandate and vision, AANDC is the lead department responsible for carrying out the Government of Canada's Federal Framework for Aboriginal Economic Development, which was released in June 2009 to address commitments made in Budget 2008 to develop a new framework that was opportunity-driven, results-focused and partnership-based. The Framework guides federal actions across many departments and agencies by pursuing the following strategic priorities:

  • Strengthening Aboriginal entrepreneurship
  • Developing Aboriginal human capital
  • Enhancing the value of Aboriginal assets
  • Forging new and effective partnerships

Over the long term, the expected outcomes of these strategies are opportunity-ready Aboriginal communities, viable Aboriginal businesses and a skilled Aboriginal workforce.

AANDC delivers a number of programs aimed at economic development and is renovating its programming to better align with the Framework. Currently programming can be categorized into three main types: Aboriginal Business Development, Community Economic Development and Lands and Environmental Management.

Through its Aboriginal Business Development programs and initiatives, AANDC aims to strengthen Aboriginal entrepreneurship by enabling access to capital, identifying and facilitating business opportunities and supporting Aboriginal business promotion activities. Programs include:

  • Aboriginal Business Development Program (ABDP)
  • Access to Capital Program (ATC)
  • Procurement Strategy for Aboriginal Business (PSAB)
  • Aboriginal Workforce Participation Initiative (AWPI)
  • Financial Institutions Program (FIP)
  • Loan Loss Reserve Program (LLR)
  • Major Resource and Energy Development Investments Initiative (MRED)

AANDC’s Community Economic Development programs support First Nation and Inuit communities by providing core funding and proposal-based funding. Programming focuses on building the capacity of communities to provide the economic development services needed to activate and enhance their respective economic assets and opportunities. These programs and initiatives include:

  • Community Economic Development Program (CEDP)
  • Community Economic Opportunities Program (CEOP)
  • Community Support Services Program (CSSP)

Lands and Environmental Management are also considered key to unlocking the economic development potential of First Nation lands, and supporting EDP. Program areas include:

  • Commercial Leasing (CL)
  • Contaminated Sites Management Program (CSMP)
  • First Nations Land Management Regime (FNLM)
  • Reserve Land and Environmental Management Program (RLEMP)

Based on the audit work conducted during the planning phase, the audit team determined that the audit scope should focus on Economic Development programs listed in Table 1 below. Considerations for selection included program funding amount, program activity during the scope period, and degree of program change during the scope period. For additional details on the in-scope programs listed below, please refer to Appendix B.

Table 1.

Program 2012-13 Funding
(approx)
Program delivery Funding type
Community Economic Development (funding to First Nations Communities)
Community Economic Development Program (CEDP) $52 million Regional offices Core (formula based) funding
Community Economic Opportunities Program (CEOP) $23 million Regional offices Targeted (proposal based) funding
Community Support Services Program (CSSP) $8.3 million Regional offices, and LED Sector Targeted (proposal based) funding
Aboriginal Business Development (Funding to aboriginal owned, and community owned businesses)
Aboriginal Business Development Program (ABDP) $35.6 million Third party delivery (Aboriginal Financial Institutions) Targeted (proposal based) funding
Major Resource and Energy Development (MRED) $6.8 million LED Sector, and third party delivery Targeted (proposal based) funding
 

Delivery of the Aboriginal Business Development Program (ABDP)

As of fiscal 2013-14 AANDC has endeavoured to completely transition away from the direct delivery of the Aboriginal Business Development Program (ABDP) funding to Aboriginal entrepreneurs, businesses, and Aboriginal community enterprises. The new 2013-14 model is designed to have numerous Aboriginal Financial Institutions (AFIs), located regionally across the country, delivering program funding to recipients on behalf of the Department. AANDC is responsible to provide the program funding to the AFIs, referred to as Program Delivery Partners (PDPs), which are then responsible to provide the targeted funding to recipients according to program terms and conditions. AANDC provides the PDPs with program coordination and support through seven regionally located Institutional Development Officers. The Department has a further plan to delegate the ABDP support and program coordination role to the National Aboriginal Capital Corporation Association (NACCA) in 2014-15. This transition would have NACCA distributing the program funding to PDPs for the Aboriginal Developmental Lending Allocation initiative, and providing the coordination and management of program delivery. Based upon NACCA’s performance, the Aboriginal Business Flexible Financing initiative may also be transferred to NACCA in 2015-16.

Economic Development Program Suite Renewal

The LED Sector and regional LED Directorates are operating in an environment characterized by considerable change due to the Deficit Reduction Action Plan (DRAP) and related organization structure, and Economic Development program suite renewal. The audit team took into consideration these upcoming program change initiatives.

In 2009, the Government of Canada released the Federal Framework for Aboriginal Economic Development to better support the full participation of Aboriginal Canadians in the economy. As part of an Action Plan to implement the Framework, AANDC has assessed how well the current suite of Aboriginal lands, environment and economic development programs align with Framework priorities. Three Branches of the LED Sector - Community Opportunities, Aboriginal Entrepreneurship, and Lands and Environmental Management – worked together to review the existing authorities and consolidate them into three program areas under the one authority. The three programs will comprise the following:

  • Community Opportunity Readiness Program (CORP) intended to provide project-based funding for First Nation and Inuit Communities for a range of activities to support communities’ pursuit of economic opportunities. Where First Nation and Inuit communities, or their representative organizations, identify opportunities and lack the financial resources to pursue them, CORP will allow them to submit proposals to obtain additional funding;
  • Lands and Economic Development Services Program (LEDSP) intended to provide support to First Nation and Inuit Communities to assist with enhancing the economic development, land and environmental capacity of communities and to support the establishment of the conditions for economic development to occur, increasing their participation in the economy; and
  • Aboriginal Entrepreneurship Program (AEP) intend to increase the number of viable businesses in Canada owned and controlled by Aboriginal Canadians, provide a supportive business environment for Aboriginal Canadians, support partnerships with various public and private stakeholders and advocate and inform employers about the hiring of Aboriginal peoples.

While this process will realign the organization of programs, many will retain the current approach, design and delivery. For example, core program funding (currently CEDP) will continue to be based on a community funding formula, and the targeted funding will be continue to be proposal/project based.

The following chart depicts how the existing programs included in this audit will be consolidated into one of three programs under the new authority:

Current Program Future Program
  • Community Economic Development Program (CEDP)
  • Community Support Services Program (CSSP)
  • Lands and Economic Development Services Program (LEDSP)
  • Community Economic Opportunities Program (CEOP)
  • Community Opportunity Readiness Program (CORP)
  • Aboriginal Business Development Program (ABDP)
  • Major Resource Development and Energy Investments (MRED)
  • Aboriginal Entrepreneurship Program (AEP)
 

 

2. Audit Objective and Scope

2.1 Audit Objective

The objective of the audit was to assess the adequacy, effectiveness, and efficiency of the management control framework of the EDP. More specifically, the audit assessed whether:

  • Effective management and oversight bodies are established and functioning to provide assurance that there is effective governance and monitoring necessary to achieve EDP objectives;
  • Management has in place an adequate, efficient and effective risk management approach to provide assurance that there is a documented, well communicated and understood process to identify, assess, and implement risk mitigating measures related to EDP; and
  • Adequate, effective, and efficient controls are in place to provide assurance that the EDPs are designed, delivered, reported, and monitored consistent with program requirements including policies, processes and procedures.

2.2 Audit Scope

The scope of the audit included the responsibilities of the LED Sector and Regional Operations (Headquarters and Regions) in support of the EDPs. The scope did not include assessment of the management controls for recipients in receipt of EDP funding.

The scope included an assessment of the extent to which the EDP processes provided reasonable assurance that programs are consistent with the policies, processes and procedures of the programs.

Through the planning process, and based on program spending, level of activity during scope period, and the degree of change over the scope period, the following programs were included in the audit scope:

  • Community Economic Development Program (CEDP)
  • Community Economic Development Program (CEOP)
  • Community Support Services Program (CSSP)
  • Aboriginal Business Development Program (ABDP)*
  • Major Resource and Energy Development (MRED)

*Due to the significant change in the delivery of the ABDP effective 2013-14, from direct delivery by AANDC to delivery by a third party, the audit scope included an assessment of the adequacy of the design of the current program processes. The audit could not assess the effectiveness of most of the program processes, as they had not completed a full fiscal-year cycle.

The audit scope included the period April 1, 2011 to September 30, 2013 and included an examination of the controls in place in the LED Sector and the British Columbia (BC), Ontario, Alberta, and Atlantic Regional Offices.

 

 

3. Approach and Methodology

The audit was conducted in accordance with the requirements of the Treasury Board Secretariat Policy on Internal Audit, and follows the Institute of Internal Auditors’ Standards for the Professional Practice of Internal Auditing. The audit examined sufficient, relevant evidence and obtained sufficient information to provide a reasonable level of assurance in support of the audit findings.

The audit plan was developed using a risk-based approach through which key risks facing AANDC, particularly the LED Sector’s obligations in support of the EDP, were identified through interviews with management and documentation review. Key risks were then linked to the core governance, risk management and control policies, processes and practices in place within AANDC that are designed to mitigate the risks identified. This information was used to determine the objectives, scope, and criteria for the audit.

Audit criteria define the standards against which the EDP’s governance, risk management and control policies, processes and practices were assessed. They are expressed in terms of reasonable expectations for these processes and practices to achieve expected results and outcomes. The audit objective was linked to audit criteria (Appendix A) developed in alignment with Treasury Board of Canada Secretariat’s Audit Criteria related to the Management Accountability Framework: A Tool for Internal Auditors (March 2011).
 

 

4. Conclusion

Generally, the audit found that control practices related to the in-scope economic development programs are adequate and effective. Some opportunities for improvement were noted to strengthen management practices in the following areas: performance measurement; risk management; quality assurance and compliance; and information management.

 

 

5. Findings and Recommendations

Based on a combination of the evidence gathered through the examination of documentation and systems, analysis, and interviews, each audit criterion was assessed by the audit team and a conclusion for each audit criterion was determined. Where a significant difference between the audit criterion and the observed practice was found, the risk of the gap was evaluated and used to develop a conclusion and to document recommendations for improvement.

Audit findings and related recommendations for improvement are categorized below by the following: governance, risk management and controls.

5.1 Governance

Governance is critical to the effective management and oversight of EDP policies, processes, practices and delivery. The audit examined the extent to which governance and oversight was established and effective, whether management established an adequate management control framework for accountability and whether management has established and used performance measures to monitor and evaluate the EDPs.

The following observations confirmed that management is performing the following elements of management, administration and oversight expected for the EDPs.

Strategic and Operational Planning

The audit noted that EDP governance and oversight was supported through annual program (CEDP, CEOP and CSSP) planning and budgeting, regular management meetings and quarterly reporting of EDP program (Headquarters and Regional) activities and achievements.

Planning at the Program (CEDP, CEOP and CSSP) and regional level was noted consistently for allocation of resources for the in-scope period and across regions and Headquarters.

The audit also noted a good management practice in British Columbia (BC) Region, where LED directorate and economic development unit annual business plans are being completed. These plans identified priorities, key activities and performance targets that were monitored each quarter. These regional directorate and unit plans were aligned with the overall priorities and objectives of the Economic Development programs for LED Sector nationally to help ensure the effective and efficient achievement of objectives. Employee Performance Agreements goals can be prepared with improved alignment to the Directorate and Unit plans to efficiently and effectively align accountabilities.

A strong management practice noted in BC Region included a strategic planning document that aligned available economic development programs to three tiers of community economic development: Tier 1 – Nation Building (CEDP, CSSP) activities include: EDO training initiatives and other capacity building; Tier 2 – Asset Development (CEOP, ABDP, FNLM) activities include economic development plan, strategic plan, land use plan, feasibility studies; and Tier 3 – Access to Capital (Aboriginal Financial Institutions (AFI), ACC, FNFA) activities include equity gap financing, strategic partnerships, capital and business start ups. This approach helped to clarify the Regional strategy and priorities and related activities for the year.

The audit noted that each Region consults with First Nation communities on community development priorities by leveraging a Lands and Economic Development Advisory Committee (LEDAC) or similar advisory body. Management has indicated that these advisory bodies are effective at helping align the EDP priorities of the First Nations communities with those of the Department.

Organization Structure, Accountability and Communication

The audit noted that Regional offices are structured to include the Lands Unit and the Economic Development Unit under one Directorate, each with different management responsibilities, but still having underlying interdependencies. This structure facilitated and supported communication between the Lands and Economic Development Units, while ensuring clear accountability and responsibility for delivery of EDP. Effective communication between land management officers and economic development officers is important since development opportunities are often linked to reserve land as the key asset. The audit noted a variety of management practices across Regions to facilitate this connection and communication. Practices included: co-location of Lands and Economic Development Units; staffing Economic Development Units with land management (Commercial Leasing, Designations) staff; the formation of community teams (including LED and non-LED staff); standing internal management and project meetings between economic development and land management managers and staff; and both formal and ad hoc reporting of issues between economic development officers and lands management officers.

The audit noted LED Sector staff roles and responsibilities for the EDP programs were assigned and communicated through performance management agreements and were consistent with EDP objectives.

In addition, a management control framework was established with clear and communicated roles, responsibilities, accountabilities and delegation of authority. Each Region was consistently structured with a LED Director reporting to the Regional Director General. The Directorate in each Region was supported by Manager positions (both Lands and Economic Development) and Economic Development Officers supporting the management framework.

Management and Oversight Bodies

The audit found that management and oversight bodies exist, which meet regularly, where EDPs can be discussed and issues can be raised. AANDC's Director General Implementation and Operations Committee (DGIOC) provides the key national forum where Director Generals and Regional Director Generals can raise EDP related issues. Regions have regional operations committees to discuss regional program issues. Finally, a regular conference call is conducted that includes all regional LED Directors and representatives from the LED Sector where regions can discuss program delivery with each other, and Headquarters.

Performance Measurement

The audit expected that the LED Sector would have identified appropriate performance measures linked to planned results, and monitored the results of performance to evaluate the effectiveness of EDPs. Regional quarterly reporting included some measures, such as the number of projects funded and the amount of funding for CEDP, CEOP and CSSP.

The audit found that the performance measures currently in place for community based programs (CSSP, CEDP, CEOP) do not adequately demonstrate the impact of the programs on communities. The performance measures currently reported focus on the number of projects approved and the value of funding provided. As noted in the Community Economic Development Program Activity Report for 2012-13, the vision of these programs is to help Aboriginal people to pursue "opportunities for employment, income and wealth creation". Effective performance measures should be in place to monitor and report progress towards this vision. It is understood that LED is in process of revising its Performance Measurement Strategy.

Beginning 2013-14, the delivery of ABDP was transferred from AANDC to AFIs who were successful applicants to be Program Delivery Partners (PDPs). AFIs are required to report on a number of program management performance measures, such as, program activity (number of enquiries, applications received, applications approved), disbursements and cost per case. ABDP has also established strong Performance Measurement Indicators at the project/business level. There is a requirement for AFIs to track, and collect this performance data and provide it to AANDC. As this program delivery design is new in 2013-14, the audit was unable to conclude of the effectiveness of these performance measurement processes. Beginning 2014-15, LED Sector is planning to transition the coordination of ABDP (the current role of AANDC) to NACCA. At the time of the audit, there had not been a formal determination of how performance data of funded projects/businesses by AFIs will be collected and communicated to AANDC. The ABDP provides financing, business support and information to increase the competitiveness and success of Aboriginal businesses in Canadian and world markets. Effective performance measures should be in place to help track and monitor the level of success achieved over time regarding these program objectives.

Recommendation:

1. The Assistant Deputy Minister of the Lands and Economic Development Sector should, as part of revising its Performance Measurement Strategy, design and implement key performance measures and indicators, which provide metrics to objectively evaluate and demonstrate the impact of community-based economic development programs as a key input into the ongoing assessment of the design and delivery of these programs. For the Aboriginal Business Development Program, LED should clarify how project performance data will be collected by third party Program Delivery Partners and communicated to AANDC.

5.2 Risk Management

To support the efficient and effective delivery of the EDP, an effective risk management framework must be in place. The EDPs and individual projects have a number of objectives related to improving economic opportunities for Aboriginal people, communities and businesses. Effective risk management helps to identify risks that may preclude or limit the achievement of these objectives and to identify measures to put in place to mitigate or reduce the likelihood that the risks identified occur. The audit examined the extent to which there is an effective risk management process to identify, assess, and implement risk mitigating measures to help ensure the achievement of EDP objectives. Risk management was considered at the project/file level, at the regional level, and at the national program level.

Regional and Project Level

It was noted that risk management is being conducted at the project and regional level. The following examples were noted:

  • During the auditing of a sample of individual funded projects, general assessments completed by AANDC were reviewed as part of the project proposal due diligence completed by the Region. The general assessment is completed for each First Nation community and considers risk factors including governance, performance history, financial stability, project complexity and other considerations.
  • Each project submitted to Regions for consideration includes a Project Assessment and Summary Report that includes a risk assessment (section 13) that is completed by the Regional Economic Development Officer. The nine risk factors are both qualitative and quantitative and are assessed using a point rated scale with the sum used to derive low, medium or high risk.
  • Regional risk assessments were reviewed and noted to be included as part of Regional annual planning and quarterly reporting processes. Most of the regions visited also had included the Regional Risk Profiles as part of the Quarterly Report. The risks include the key regional corporate risks and related regional planning commitments for all programs. An example noted in one Region aligned the regional corporate risk - Government Partnership Risk ("ability to effectively and efficiently work with partners and stakeholders in the development and delivery of policies and programs") to the Regional planning commitment ("Community Development Framework").

National Program Level

Risk management at the project level by LED Sector involved assessment of the risk of each project and determination of type of funding arrangement and amount of monitoring that will be undertaken. As well, a national prioritization framework to ensure transparency of program funding decisions for larger economic opportunities was created to manage the client relationship risk. However, formal creation of a formal risk profile for the program through a facilitated process has not been undertaken for existing programs. The audit expected this to be in place for the EDPs as this is a good management practice and a gap that was noted by a previous auditFootnote 1. Without this consideration of risk, program objectives may not be met due to circumstances occurring (risks) that are not adequately controlled or mitigated and/or risk is not considered in the prioritization and reprioritization of program activities.

Recommendation:

2. The Assistant Deputy Minister of the Lands and Economic Development Sector should include, as part of its annual planning for these programs, identification and assessment of Economic Development risks and determine appropriate mitigation measures required to further support the achievement of program objectives. Where possible and appropriate the program risk work already being done in regions should be leveraged to support the efficiency and effectiveness of this assessment.

5.3 Controls

Key management control activities are critical in supporting accountabilities and efficient and effective delivery of EDP. The audit considered whether:

  • There are appropriate training, tools, resources, information and guidance available to support an effective EDP design and delivery;
  • There is an effective process to identify and leverage innovation change opportunities and requirements; and
  • EDPs are managed in accordance with program terms and conditions and funding agreements, and include regular and timely monitoring and reporting of compliance to support decision-making.

Training and Innovation

The audit team found there were appropriate training, tools, resources, information and guidance available to support an effective program design and delivery in-line with EDP.

The management and delivery of the community programs has not been changed over the audit scope period. The audit noted a number of tools and guidance including a training manual, management control regimes and example terms of reference for targeted programming. The approach to training is largely a combination of mentoring, coaching and on the job training. This approach was consistently noted in regions to be effective.

The audit also noted that changes related to the EDP suite renewal and consolidation of programming initiative will impact programming starting 2014-15. LED Sector has drafted tools and resources to assist in the roll-out of the initiative. Examples of these resources include program area descriptions, updated Management Control Frameworks, updated application and reporting templates, and updated program logic models.

For ABDP, the audit noted the development of supporting program documentation for the PDPs that outline program terms and conditions, along with the roles and responsibilities of the various stakeholders.

Strong management practice noted in two Regions (BC and Ontario): Regions have emphasized the use of funding (CEDP, CEOP) by communities to develop 5 year Strategic plans (e.g. strategic economic development plans, land use plans, and community plans). The audit found the emphasis on the development of 5-year strategic plans (economic development, land use) by First Nation communities to be a strong practice for strategically managing Economic Development funding resources. Benefits noted include:

  • identification of longer term objectives and planning for economic development projects;
  • understanding of projects that are "shovel-ready" which helps with resource re-allocation as circumstances change over time; and
  • ability to track individual projects that culminate in economic development over time (e.g. feasibility study to project design to project completion).

The other two Regions were either considering a similar shift to a longer term planning timeframe or were continuing with the annual focus. The audit recognizes benefits and encourages the LED Sector management to consider consistent focus on community strategic plans as a key priority as program re-alignment is rolled-out.

Management of ABDP: Compliance and Quality Assurance

The delivery of the ABDP was transferred from AANDC to PDPs during 2013-14. In all regions visited, the due diligence process to select participating PDPs followed the readiness evaluation and approval process set out by Headquarters. The expectation was that PDPs would take on the management and administration for all ABDP funding. It was noted, however, that in one region the only qualified PDP agreed to manage only a portion of ABDP recipients, and excluded community owned businesses. The management of this ABDP component in the region remained the responsibility of regional staff.

The audit expected clear and communicated compliance requirements for the ABDP, which were not present for the Program. Compliance is a requirement of the Funding Agreement Terms and Conditions between AANDC and the PDP and for any ABDP funding that remains with AANDC (as noted in one region). The regions visited consistently noted that they have not completed any compliance activities of the PDPs. In instances where third party service providers are used to perform program operational activities, the Department is expected to implement a regime that provides assurance over controls exercised by the third party so that the Department has comfort that program funds are being expended according to program terms and conditions. It is expected that Headquarters would provide the expectations for compliance to help ensure consistency across regions.

The audit noted that the ABDP program will be changing again with AANDC delegating to NACCA the management of funding to, and coordination of, the PDPs who deliver ABDP. Management has indicated that as of April 1 2014, NACCA will be distributing the program funding the Aboriginal Developmental Lending Allocation initiative. Based upon NACCA's performance, the Aboriginal Business Flexible Financing initiative, which represents the majority of ABDP's budget, may be transferred to NACCA in 2015-16.

As the program goes through these considerable changes, compliance requirements need to be clarified and implemented.

Recommendation:

3. The Assistant Deputy Minister of the Lands and Economic Development Sector should develop a quality assurance review and compliance framework and practices to provide AANDC with assurance over the effectiveness of Program Delivery Partner program management processes and controls in the delivery of the Aboriginal Business Development Program. The compliance framework and practices should be adjusted to ensure AANDC continues to receive assurance over program delivery once the National Aboriginal Capital Corporation Association takes on the Department's program coordination role.

Results of Testing Program Files (CEDP, CEOP, CSSP and MRED)

A judgemental sample was selected for each of CEDP, CEOP, CSSP and MRED. The audit tested key management control activities that were expected to help ensure the due diligence of CEOP, CEDP, CSSP and MRED project proposal and planning, funding approval, project/ program compliance and monitoring. Evidence was gathered through a combination of examination of documents retained by each Region and through discussions with key regional staff. Management controls over EDPs, including due diligence of funding plans, funding approval, project/ program compliance and monitoring appear to exist and are generally effective and efficient.

The audit found that generally processes and key controls were adequate and effective. The audit team tested 24 CEDP files, 24 CEOP files, 11 CSSP files across 4 regions, along with 2 MRED projects and 1 MRED Funding Arrangement. The audit team was looking for documentation to support the due diligence, approval, compliance and monitoring. The audit found that project proposal and project summary assessment, or similar documents, were received and filed, and displayed formal approval. Section 34 (Financial Administration Act) payment approvals were appropriate, and retained in GCIMS in most cases. No exceptions were noted for compliance and monitoring activities. Testing found that the majority of reporting requirements were received, reviewed and approved. In the few instances where reports were noted to be missing or late, such as Audited Financial Statements, subsequent funding is halted until outstanding reporting is received.

Regions did not note any concerns regarding capacity of delivering targeted, proposal-based funding compared to core formula-based funding. The funding budgeted for targeted and core programs was consistently issued during the year across all regions visited.

Information Management

The audit expected that information would be captured and categorized leveraging the Grants and Contributions Information Management System (GCIMS) as the "system of record", and that a minimum standard for program documentation would be met in aggregate. Test results varied in the degree to which GCIMS was used as the "system of record" for each program. While testing found that file documentation and information was generally complete, information tended to be stored in a number of systems and formats, including GCIMS, Comprehensive Information Document Management system (CIDM), spreadsheets, and hard copy files. The completeness of documentation and evidence captured in GCIMS, particularly for project based funding, was not consistent between programs and between Regions. Of particular note, the use of the GCIMS "Proposal Management module" as a repository of key project documents (proposal/applications, proposal review and approval, project deliverables) varied considerably across programs and regions, from considerable use, to very little.

The audit also noted during file testing that in one region visited, evidence of S.34 program payment approvals for the 2011-12 and 2012-13 sample files tested were not being uploaded into GCIMS in order to evidence that formal approval was executed. The audit team did, however, note that this region has since adopted the practice in 2013-14, and these approvals are now being captured in GCIMS.

  • Key documentation in GCIMS should include: Project applications, proposals, proposal review and approval, project reporting and key deliverables, and evidence of formal S.34 program payment approval (scan of batch S.34 payment approval)
  • Key evidence should always include: Documentation that evidences why project funding exceeds approved proposal amount.

The management and administration of the EDPs is improved through the use of GCIMS as the common system of record. Benefits resulting from consistent use of GCIMS can be expected to include: national accessibility for programs and projects, completeness of reporting requirements is demonstrated and tracked, and facilitated monitoring of compliance with terms and conditions.

Recommendation:

4. The Assistant Deputy Minister of the Lands and Economic Development Sector should, as program realignment is rolled out and roles and responsibilities are clarified, emphasize the use of Grants and Contributions Information Management System as the system of record, and clarify the need for complete documentation, to support the efficient and effective management of AANDC economic development programs.

 

 

6. Management Action Plan

Recommendations Management Response / Actions Responsible
Manager (Title)
Planned
Implementation Date
1. The Assistant Deputy Minister of the Lands and Economic Development Sector should, as part of revising its Performance Measurement Strategy, design and implement key performance measures and indicators, which provide metrics to objectively evaluate and demonstrate the impact of community-based economic development programs as a key input into the ongoing assessment of the design and delivery of these programs. For the Aboriginal Business Development Program, LED should clarify how project performance data will be collected by third party Program Delivery Partners and communicated to AANDC. To streamline program delivery and remove program overlaps, the Lands and Economic Development Sector recently completed a consolidation of program authorities. Performance measurement strategies for both the Community Opportunity Readiness Program and the Lands and Economic Development Services program are currently being developed to incorporate key performance indicators. Through the development of these strategies, AANDC will be better positioned to measure the impact of community based economic development programs.

Beginning in 2014-15, LED Sector is planning to transition the coordination of Aboriginal Entrepreneurship Programming, which is currently the role of the Sector, to the National Aboriginal Capital Corporation Association (NACCA).

LED continues to transition program management and program delivery responsibilities of renovated programming to Aboriginal Financial Institutions who agree to become Program Delivery Partners. NACCA has been asked to undertake a Quality Assurance Review and Compliance function in respect of the program delivery undertaken by Program Delivery Partners, including data collection about program recipients and beneficiaries.

This requirement has been discussed extensively with NACCA and representatives of the Aboriginal Financial Institutions. They understand the critical importance of collecting and remitting reliable data in a timely manner to be able to provide evidence of the beneficial effects of this program investment.

The Quality Assurance Review function will be an integral component of the business plan that NACA will provide to LED, prior to assuming renovated program management responsibilities.
Assistant Deputy Minister, Lands and Economic Development Sector April 1, 2014
As of April 1, 2014, program management responsibility for two new program elements will be transferred to NACCA. April 1, 2014
As of April 1, 2015, an additional program element will be transferred to NACCA, based on performance with program elements transferred in 2014-15. April 1, 2015
2. The Assistant Deputy Minister of the Lands and Economic Development Sector should include, as part of its annual planning for these programs, identification and assessment of Economic Development risks and determine appropriate mitigation measures required to further support the achievement of program objectives. Where possible and appropriate the program risk work already being done in regions should be leveraged to support the efficiency and effectiveness of this assessment. LED has implemented appropriate strategies to mitigate key program risks, such as a national prioritization framework which ensures that project proposals from across the country are considered each year on a consistent basis. The national prioritization framework provides a fair and transparent platform for communicating decisions regarding departmental investments in large economic development projects. LED will also complete a formal risk assessment process to ensure a consistent approach to risk identification and analysis. A program risk assessment is being developed in collaboration with the Audit and Evaluation Branch and will be included as part of the performance measurement strategy for new consolidated programs in 2014/2015. Assistant Deputy Minister, Lands and Economic Development Sector April 1, 2014.
3. The Assistant Deputy Minister of the Lands and Economic Development Sector should develop a quality assurance review and compliance framework and practices to provide AANDC with assurance over the effectiveness of Program Delivery Partner program management processes and controls in the delivery of the Aboriginal Business Development Program. The compliance framework and practices should be adjusted to ensure AANDC continues to receive assurance over program delivery once the National Aboriginal Capital Corporation Association takes on the Department’s program coordination role. A Quality Assurance Review and Compliance framework has been developed for Program Delivery Partners. Also, a Quality Assurance checklist has been provided to Regions. Activities under this framework will be initiated for PDP activities during FYs 2013-14 and 2014-15. Assistant Deputy Minister, Lands and Economic Development Sector April 1, 2014.
4. The Assistant Deputy Minister of the Lands and Economic Development Sector should, as program realignment is rolled out and roles and responsibilities are clarified, emphasize the use of the Grants and Contributions Information Management System as the system of record, and clarify the need for complete documentation, to support the efficient and effective management of AANDC economic development programs. The Lands and Economic Development Sector agrees that GCIMS should be used to its full potential as a tool for financial management, project management and information management, to provide national accessibility to key information and accurate data. This includes project applications and proposals, due diligence project reviews, evidence of project approval, monitoring, compliance, and project deliverables.

As the LED Sector implements the new consolidated program authorities, LED will require the proper use of GCIMS in the following:
  1. Management Control Frameworks, as a key program management role and responsibility; and
  2. Performance Measurement Strategies, as a data source and methodology for performance measurement and data collection.
In addition, as a requirement within LED program Management Control Frameworks, LEDS will conduct regular operational reviews. The objectives of the operational reviews are to: ensure compliance with established procedures; assure reliability of data and reports; and report the results of the operational inspections to senior management at the regional and headquarters level.

As of April 1st, 2014 full use of GCIMS will be implemented for Community Opportunity Readiness Program and Aboriginal Entrepreneurship Program.
Assistant Deputy Minister, Lands and Economic Development Sector April 1, 2014
 

 

Appendix A: Audit Criteria

The audit objective was linked to audit criteria developed in alignment with Treasury Board of Canada Secretariat’s Audit Criteria related to the Management Accountability Framework: A Tool for Internal Auditors (March 2011).

Governance and Oversight
Governance
1.1 Management and oversight bodies are effective at supporting the achievement of EDP objectives.
Accountability
1.2 Management has established an adequate management control framework with clear and communicated roles, responsibilities, accountabilities and delegation of authority.
Results & Performance
1.3 Management has appropriately identified & monitored appropriate performance measures and/or other tracking linked to planned EDP results/expectations.
Risk Management
2.1 There are adequate risk management mechanisms that are appropriately linked to the EDP objectives.
Program Delivery
Stewardship
3.1 EDPs are managed in accordance with program terms & conditions and funding agreement requirements, and include regular monitoring & reporting of compliance. Reporting is prepared on a timely basis to support management decision making.
People
3.2 There are appropriate training, tools, resources, information and guidance available to support an effective program design & delivery in line with EDP requirements including policies, processes and procedures.
3.3 There are appropriate information systems, and Information Management processes in place to ensure that key documentation is accurate and retained.
Learning, Innovation, and Change Management
3.4 There is an adequate process to identify change opportunities/requirements to ensure change initiatives are properly implemented & well communicated.
 
 

 

Appendix B: Economic Development Program Descriptions

In Scope Programs

  Community Economic Development Program (CEDP) Community Economic Opportunities Program (CEOP) Community Support Services Program (CSSP) Aboriginal Business Development Program (ABDP) Major Resource and Energy Developments (MRED)
Description The CEDP provides core financial support to First Nation and Inuit communities for the provision of economic development public service to their community. The CEOP provides project-based financial support to First Nation and Inuit communities for economic development opportunities. The CSSP funds the implementation ofnational and regional plans to deliver support services to First Nation and Inuit community economic development organizations. The support services are intended to increase the economic capacity with community organizations. The ABD provides financing, busines support and information to increase the competitiveness and success of Aboriginal businesses in Canadian and world markets. The MRED, provides contributions to private equity providers to establish or expand private equity instruments that foster Aboriginal participation in Major Resource Development and Energy projects
Primary Stakeholders First Nation Councils, governments of self-governing First Nation and Inuit communities, representative organizations of Inuit communities and other organizations mandated by the eligible recipients to carry out ongoing activities and projects on their behalf. First Nation Councils, governments of self-governing First Nation and Inuit communities, representative organizations of Inuit communities and other organizations mandated by the eligible recipients to carry out ongoing activities and projects on their behalf. First Nation Councils, governments of self-governing First Nation and Inuit communities, representative organizations of Inuit communities and other organizations mandated by the eligible recipients to carry out ongoing activities and projects on their behalf. 14 Aboriginal Financial Institutions that have been selected to deliver ABD to individual Aboriginal entrepreneurs, new and existing business and Aboriginal organizations 8 Investment funds have been established that are accessible to Individuals of Aboriginal heritage (Status or Non-Status Indians, Métis or Inuit) or a majority owned Aboriginal business.
Eligible Activities
  • Staffing and associated costs
  • Community economic development planning
  • Capacity development initiatives
  • Development of proposals and leveraging financial resources
  • Carrying out economic development activities on behalf of the community
  • Employment of Community Members
  • Business development
  • Development of land and resources under community control
  • Access to opportunities originating with land and resources beyond community control
  • Promotion of community as a place to invest
  • Research and advocacy
  • Capacity building initiatives for national and/or regional community economic development organizations and/or community economic development personnel funded by the CEDP
  • Provision of funding for economic development associations, capacity building events and/or training opportunities
  • Business Advisory Services and Training
  • Commercial Ventures
  • Market Development
  • Business Development and Advocacy Activities
Establishment or expansion of a private equity instrument to support business participation in major resource and energy projects
Other Comments The design and delivery of the program has not changed over the audit scope period.

Program consolidation will see CEDP combined into Land and Economic Development Services Program (LEDSP)
The design and delivery of the program has not changed over the audit scope period.

Program consolidation will see CEOP combined into Community Opportunity Readiness Program (CORP)
The design and delivery of the program has not changed over the audit scope period.

Program consolidation will see CSSP combined into Land and Economic Development Services Program (LEDSP)
The delivery of this program has largely been outsourced to Aboriginal Financial Institutions (AFIs) with AANDC providing management and oversight. This management and oversight role will be devolved to NACCA in a year or two.

Any residual program management retained by AANDC will see ABDP consolidated into Aboriginal Entrepreneurship Program (AEP)
A June 2011 independent review concluded that the initial capitalization amounts for investment funds were too small and the three-year investment period was too short to ensure the viability of investment funds.

Program consolidation will see MRED combined into Aboriginal Entrepreneurship Program (AEP)
 

Additional Programs (not in audit scope)

  Access to Capital Program (ATC) Financial Institutions Program (FIP) Procurement and Business Partnership (Including AWPI) Procurement Strategy for Aboriginal Business (PSAB) Loan Loss Reserve (LLR)
Description The ATC program provides assistance to strengthen the network of Aboriginal-controlled developmental lending institutions through funding agreements with the National Aboriginal Capital Corporation Association (NACCA) targeted at providing support services and training. The FIP provides contributions to the establishment, expansion or diversification of Aboriginal Financial Institutions. Access to Business Opportunities provides funding to cultivate a culture of entrepreneurship in the Aboriginal community, to improve access to business and employment opportunities for Aboriginal Businesses, develop partnership agreements with key Aboriginal and non Aboriginal stakeholders including the private sector/employers. The resulting participation frameworks, strategies, components focus on increasing Aboriginal procurement (including self employment) and employment opportunities through a wide variety of activities such as advocacy, research, promotion and major public and private economic opportunities. The PSAB initiative aims to increase federal contracting opportunities and to gain access to the overall federal procurement process for Aboriginal businesses. The LLR, part of the Framework Action Plan, encourages commercial financial institutions to provide debt financing to First Nation businesses with assets on reserve. A June 2011 independent review concluded that restrictive program guidelines and lack of promotion and marketing were key contributors to the low LLR take up and activity. No additional LLR arrangements have been negotiated.
Primary Stakeholders National Aboriginal Capital Corporation Association (NACCA) or any recognized Aboriginal Financial Institution (AFI). National Aboriginal Capital Corporation Association (NACCA) or any recognized Aboriginal Financial Institution (AFI). Individuals of Aboriginal heritage (Status or Non- Status Indians, Métis or Inuit), municipal governments, labour organizations, universities, colleges or other learning institutions and private and Aboriginal corporations. Individuals of Aboriginal heritage (Status or Non-Status Indians, Métis or Inuit) or a majority-owned Aboriginal business. 8 Investment funds have been established that are accessible to Individuals of Aboriginal heritage (Status or Non-Status Indians, Métis or Inuit) or a majority owned Aboriginal business.
Eligible Activities
  • Interest-rate buy downs
  • Enhanced access loan funds
  • Support, training and organizational development
  • Establishment, expansion or diversification of an AFI
  • Developmental loan funding, loan guarantees and other related financial services to Aboriginal businesses
  • Establishment and operation of one National Association
  • Institutional Development
Aboriginal Participation Components (APC) that include employment and labor initiatives and support business development and advocacy strategies to contract/hire Aboriginal businesses and employees.

Business development, planning and research activities, business and labour market capacity.

Business advisory and entrepreneurship development, advocacy activities, skills and capacity development and market and partnership development and promotion.

Aboriginal Procurement and Business Partnerships, Business and Employment and Capacity assessment and research
  • Business Advisory Services and Training
  • Commercial Ventures
  • Market Development
  • Business Development and Advocacy Activities
Establishment or expansion of a private equity instrument to support business participation in major resource and energy projects
 

 

Appendix C: Applicable Legislation, Regulations, and Policies

 
 
Date modified: