Summative Evaluation of the Elementary/Secondary Education Program on Reserve

Date: June 2012
(Project Number: 1570-7/09057)

PDF Version (844 Kb, 95 Pages)




Table of Contents




List of Acronyms

AFA
Alternative Funding Arrangement

BC
Block Contribution

B.C.
British Columbia

CEC
Cultural Education Centres

CFA
Comprehensive Funding Arrangement

CFNFA
Canada/First Nations Funding Arrangement

CWBI
Community Well-Being Index

ECE
Early Childhood Education

ECCD
Early Childhood Care and Development

EPMRB
Evaluation, Performance Measurement and Review Branch

EPMRC
Evaluation, Performance Measurement and Review Committee

ESE
Elementary/Secondary Education

FC
Fixed Contribution

FNEC
First Nations Education Council

FNESC
First Nations Education Steering Committee

FNRMO
First Nation Regional Education/Management Organizations

FTE
Full-Time Equivalent

FTP
Flexible Transfer Payment

HCSE
High-Cost Special Education

HQ
Headquarters

K-12
Kindergarten to grade 12

OAG
Office of the Auditor General

PSE
Post-Secondary Education

QAS
Quality Assurance Strategy

SC
Set Contribution

SEP
Special Education Program






Executive Summary

This summative evaluation of the Elementary/Secondary Education (ESE) Program was conducted in time for consideration of policy authority renewal in 2012-13. It follows a formative evaluation of the ESE Program in 2010, which provided a preliminary examination of the state of information on First Nations education at Aboriginal Affairs and Northern Development Canada (AANDC).

This evaluation was conducted concurrently with the summative Evaluation of Post-Secondary Education programming in order to obtain a holistic understanding of AANDC's suite of education programming and its impact on First Nation and Inuit communities.

The primary objective of elementary/secondary education programming is to provide eligible students living on reserve with education programs comparable to those that are required in provincial schools by the statutes, regulations or policies of the province in which the reserve is located.

AANDC's elementary/secondary education programming is primarily funded through seven authorities: Grants to participating First Nations and First Nations Education Authority pursuant to the First Nations Jurisdiction over Education in British Columbia Act; Grants to Indian and Inuit to provide elementary and secondary educational support services; Grants to Inuit to support their cultural advancement; Payments to support Indian, Inuit and Innu for the purpose of supplying public services in education (including Cultural Education Centres; Indians Living On Reserve and Inuit; Registered Indian and Inuit Students; Special Education Program; and Youth Employment Strategy); Grants for Mi'kmaq Education in Nova Scotia; Contributions under the First Nations SchoolNet services to Indians living on reserve and Inuit; and Contributions to First Nation and Inuit Governments and Organizations for Initiatives under the Youth Employment Strategy Skills Link program and Summer Work Experience Program.

The evaluation examined the following components of ESE programming: instructional services for Band Operated Schools, Federal Schools and Provincial Schools; Elementary and Secondary Student Support Services; New Paths for Education; Teacher Recruitment and Retention; Parental and Community Engagement; First Nation Student Success Program; Cultural Education Centres; Special Education; Education Partnerships Program; and First Nations SchoolNet.

In line with Treasury Board Secretariat requirements, the evaluation looked at issues of relevance (continued need, alignment with government priorities, alignment with federal roles and responsibilities), performance (effectiveness) as well as efficiency and economy.

The evaluation's findings and conclusions are based on the analysis and triangulation of seven lines of evidence: case studies, expenditures analysis, student data analysis, document and file review, key informant interviews, literature review and surveys (see also Appendix A, Evaluation Matrix).

Four contractors were contracted to handle specific lines of evidence for which they have expertise. Donna Cona Inc. undertook key informant interviews and case studies; Harris/Decima conducted surveys; KPMG provided the expenditures analysis; and the University of Ottawa carried out a meta-analysis of available literature. The evaluation team worked closely with them to determine the most appropriate and rigorous approaches to the methodologies, and conducted most of the final analysis in-house. Additionally, a secondary literature review was completed to assess broader issues of governance and the coverage of education programs.

Stemming from this review, this evaluation found the following:

Relevance

  1. There is a need for continued investment in the Authorities for Elementary/Secondary Education stemming from projected population growth and from the need for significant improvement in student outcomes.
  2. Education authority activities are generally aligned with Government of Canada priorities; however, recent major reforms are reflective of the need to better align activities and better ensure improvements in student success.
  3. The priorities as stated by First Nation Education Authorities and those of AANDC are aligned insofar as the need to address marked gaps in educational opportunities and success. However, First Nation participants emphasise key priorities in the areas of cultural and language retention as being critical to success, and emphasise the need to recognise key differences in learning needs and the current state of education gaps, rather than simple notions of comparability.
  4. The role of the Government of Canada in ESE programming is generally appropriate; however, policy changes with respect to service delivery and local operational control may have implications on this role in the future.

    Performance
  5. The intended outcome of education opportunities and results that are comparable to the Canadian population is not being achieved.
  6. Continued work is needed to better facilitate constructive engagement and collaborative networks between First Nation education authorities, and where appropriate, with provincial governments or other organisations, and there is evidence that AANDC programming is improving in this regard.
  7. Student success is associated with parental engagement, the level of education in the community and the strength of the local economy. There are deeper issues related to the historical trauma of residential schools that may be interrelated with these factors.
  8. Community governance, the quality of teacher instruction and the quality of school curriculum were suggested as key factors affecting student success.
  9. Expenditures to First Nations and tribal councils for the operation of schools do not appear to account for actual cost variability applicable to the needs and circumstances of each community or school, and particularly the cost realities associated with isolation and small population. There is a need for a more strategic understanding of resource needs and allocation methods.
  10. Jurisdictional issues, implications for the provision of second- and higher-level services, and early childhood and adult education are considerations of significant importance in future policy development.
  11. Many First Nation schools and communities are not adequately resourced to provide proper assessments and services to meet the needs of First Nation students with special needs.

    Efficiency and Economy
  12. The current approach to programming may not be the most efficient and economic means of achieving the intended objectives of ESE programming.

Based upon these findings, it is recommended that AANDC:

  1. Develop a strategic and transparent framework for the investment of new funds that are explicitly allocated to facilitate improvements in student success in the short-term;
  2. Undertake further research into funding allocation methodologies that are equitable to provincial approaches, while at the same time accounting for cost realities on reserve;
  3. Ensure that future policy and program exercises develop clearly defined roles, responsibilities, and accountabilities for Elementary and Secondary Education;
  4. Explore and pursue options for the comprehensive development of second and higher-level services where possible and appropriate to reduce administrative burden and overhead costs, while supporting First Nations in developing long-term capacity for service management;
  5. Work with First Nations to develop strategies to strengthen culture and language retention as it relates to better student outcomes;
  6. Examine the implications of integrating support for early childhood education in AANDC's education portfolio;
  7. Examine the implications of integrating support for adult education in AANDC's education portfolio; and
  8. Develop a strategy to work with First Nations in building the capacity to strengthen the provision of special needs assessments and services.





Management Response / Action Plan

Project Title: Summative Evaluation of Elementary and Secondary Education
Project #: 1570-7/09057

Recommendations Actions Responsible Manager
(Title / Sector)
Planned Start
and
Completion Dates
1. Develop a strategic and transparent framework for the investment of new funds that are explicitly allocated to facilitate improvements in student success in the short term. Agree in Principle, with respect to Recommendations 1, 2, 3, 4, 5 and 8. Stephen Gagnon,
Director General, Education Branch
ESDPP
Start Date:

Completion:
In Budget 2008, the Government of Canada announced the First Nation Student Success Program (FNSSP). The FNSSP supports projects aimed at improving student outcomes in reading, writing and mathematics as well as those aimed at school retention. The department's web site describes: the program objectives in detail; the assessment and selection process; as well as Program Guidelines.

Pursuant to commitments outlined in Budget 2012, the Government of Canada will:
  • work with First Nations to develop options, including new legislation, to improve the governance framework and clarify accountability for First Nations elementary and secondary education;
  • work with willing partners to introduce a First Nation Education Act and have it in place by September 2014;
  • explore mechanisms to ensure stable, predictable and sustainable funding for First Nation and elementary and secondary education
We will also invest $100 million over three years for First Nation education to provide early literacy programming and other supports and services to First Nation schools and students, and to strengthen their relationships with provincial school systems.

The findings of this evaluation will be taken into account when developing options for moving forward.
2. Undertake further research into funding allocation methodologies that are equitable to provincial approaches, while at the same time accounting for cost realities on-reserve.
3. Ensure that future policy and program exercises develop clearly defined roles, responsibilities, and accountabilities for Elementary and Secondary Education.
4. Explore and pursue options for the comprehensive development of second and higher-level services where possible and appropriate to reduce administrative burden and overhead costs, while supporting First Nations in developing long-term capacity for service management.
5. Work with First Nations to develop strategies to strengthen culture and language retention as it relates to better student outcomes.
8. Develop a strategy to work with First Nations in building the capacity to strengthen the provision of special needs assessments and services.
6. Examine the implications of integrating support for early childhood education in AANDC's education portfolio. Concur in part

Early childhood education and adult education are under the mandate of Health Canada and HRSDC, respectively. To the extent that integrating early childhood education and adult education fall within the mandates of other departments, the recommendation raises machinery of government issues that are beyond this Department's abilities/authority to change.

To the extent possible, the department will work to develop early literacy programming, high school retention and active measures to ensure that the coverage of the various department's responsibilities is coordinated.
Stephen Gagnon,
Director General, Education Branch
ESDPP
 
7. Examine the implications of integrating support for adult education in AANDC's education portfolio.

I approve the above Management Response and Action Plan

Original signed by:

Françoise Ducros
Assistant Deputy Minister, Education and Social Development Programs and Partnerships Sector (ESDPP)






1. Introduction

1.1 Overview

This summative evaluation of the Elementary/Secondary Education (ESE) Program was conducted in time for consideration of policy authority renewal in 2012-13. It follows a formative evaluation of the ESE Program in 2010, which provided a preliminary examination of the state of information on First Nations education at Aboriginal Affairs and Northern Development Canada (AANDC) and was required to approve the continuation of terms and conditions of the program as of March 31, 2010.

This evaluation was conducted concurrently with the summative Evaluation of Post-Secondary Education (PSE) programming in order to obtain a holistic understanding of AANDC's suite of education programming and its impact on First Nation and Inuit communities. This approach allowed the evaluation team to minimise costs and reduce the burden for individuals and communities by collecting all the information at one time.

1.2 Program Profile

1.2.1 Background and Description

Section 91(24) of the Constitution Act, 1867 gives the Parliament of Canada legislative authority in matters pertaining to "Indians, and Lands reserved for Indians." Canada exercised this authority by enacting the Department of Indian Affairs and Northern Development Act, the enabling legislation for the Department of Indian Affairs and Northern Development. The Indian Act (1985), sections 114 to 122, allows the Minister to enter into agreements for elementary and secondary school services to Indian children living on reserves, providing the Department with a legislative mandate to support elementary and secondary education for registered Indians living on reserve.

Since the early 1960s, AANDC has sought incremental policy authorities to undertake a range of activities to support the improvement in the socio‑economic conditions and overall quality of life of registered Indians living on reserve. These activities include support for cultural education for Indians and Inuit. Thus, AANDC considers its involvement in cultural education programs as a matter of policy.

Although there have been significant gains since the early 1970s, First Nation participation and success in elementary and secondary education still lags behind that of other Canadians. Drop‑out rates are higher for First Nations than other Canadian students. Increasing Indian and Inuit retention rates and successes at the elementary/secondary programming level will support the strategic goal of greater self-sufficiency, improved life chances, and increased labour force participation.

AANDC's elementary and secondary programming is primarily funded through the following authorities: [Note 1]

  • Grants to participating First Nations and First Nations Education Authority pursuant to the First Nations Jurisdiction over Education in British Columbia Act;
  • Grants to Indian and Inuit to provide elementary and secondary educational support services;
  • Grants to Inuit to support their cultural advancement;
  • Payments to support Indian, Inuit and Innu for the purpose of supplying public services in education (including Cultural Education Centres; Indians Living On Reserve and Inuit; Registered Indian and Inuit Students; Special Education Program; and Youth Employment Strategy);
  • Grants for Mi'kmaq Education in Nova Scotia;
  • Contributions under the First Nations SchoolNet [Note 2] services to Indians living on reserve and Inuit; and
  • Contributions to First Nation and Inuit Governments and Organizations for Initiatives under the Youth Employment Strategy (YES) Skills Link program and Summer Work Experience Program.  [Note 3]

In addition to core program funds for ESE, the evaluation looked at several complimentary programs, which aim to improve the quality of education in First Nation schools and student outcomes. They include:

  • the Special Education Program (SEP), which supports students with moderate to profound special education needs;
  • New Paths for Education, Teacher Recruitment and Retention, and Parental and Community Engagement – which provide proposal based funding for initiatives designed to strengthen First Nations education management, improve the effectiveness of classroom instruction, and support community and parental involvement in the education of children and youth;
  • the Education Partnerships Program, which supports proposals for tripartite education partnership arrangements between the Government of Canada, First Nations and provinces in order to help advance First Nations student achievement in First Nations and provincial schools;
  • the First Nations SchoolNet [Note 4] Program, which focuses on improving the connectivity and technical capacity of kindergarten to grade 12 (K-12) schools on reserve;
  • the Cultural Education Centres (CEC) Program, which currently funds 110 Cultural Education Centres; and
  • the First Nation Student Success Program, which provides support to proposals that are aimed at improving student and school outcomes through investments in school success plans, student learning assessments, and performance measurement.

1.2.2 Program Objectives and Expected Outcomes

The overall objective of ESE programming is to provide eligible students living on reserve with education programs comparable to those that are required in provincial schools by the statutes, regulations or policies of the province in which the reserve is located. The objective is that eligible students will receive a comparable education to other Canadians within the same province of residence and achieve similar educational outcomes to other Canadians, and with attendant socio-economic benefits to themselves, their communities and Canada.

In AANDC's Program Activity Architecture, education falls under the strategic outcome "The People," whose ultimate outcome is "individual and family well-being for First Nations and Inuit." Education is its own program activity, which includes the following sub‑activities covered in this evaluation: Elementary and Secondary Education, First Nations and Inuit Youth Employment Strategy, Education Agreements, Special Education and Cultural Education Centres.

1.2.3 Program Management, Key Stakeholders and Beneficiaries

The management of education programming at AANDC is undertaken by the Education Branch in the Education and Social Development Programs and Partnerships Sector (ESDPP).

In order to be eligible for elementary/secondary education, a student must:

  1. be attending and enrolled in a federal, provincial, band‑operated or private/independent school recognized by the province in which the school is located as an elementary/secondary institution;
  2. be aged 4 to 21 years (or the age range eligible for elementary and secondary education support in the province of residence) on December 31 of the school year in which funding support is required, or a student outside of this age range who is currently funded by AANDC for elementary and secondary education;
  3. be ordinarily resident on reserve; and
  4. have demonstrated need for education student support and that the Chief and Council have asked AANDC to deliver these services, thus, no other source of funding can meet such a need.

Grants are administered directly by AANDC on behalf of First Nations. Grants are provided when Chiefs and Councils choose to continue having AANDC deliver support services on reserve or some components of program funding. AANDC may make a grant payment directly to individual First Nation or Inuit recipients.

Funding for the ESE and SEP programs may be flowed directly to chiefs and councillors, or to organizations designated by chiefs and councillors (bands/settlements, tribal councils, education organizations, political/treaty organizations, public or private organizations engaged by or on behalf of Indian bands to provide education services, provincial ministries of education, provincial school boards/districts or private education institutions).

In addition, AANDC may also enter into agreements with provincial education authorities for the delivery of elementary/secondary education services; with private firms to administer program funds jointly with or on behalf of the First Nation (i.e., co-managers, or third-party managers); with education authorities or First Nation Regional Education/Management Organizations (FNRMO) or in some cases, AANDC may deliver services directly on behalf of First Nations (e.g., in the remaining seven federal schools).

Contributions for the SEP may also be flowed directly to FNRMO's who deliver direct and indirect services. The following six FNRMOs deliver direct and/or indirect services for SEP:

  • First Nations Education Steering Committee (FNESC) in British Columbia
  • First Nations Education Council (FNEC) and l'Institut Culturel et Éducatif des Montagnais in Quebec
  • Prince Albert Grant Council and Battlefords Tribal Council in Saskatchewan
  • Manitoba First Nations Resource Centre
  • Mi'Kmaq Education in Nova Scotia
  • New Brunswick Education Initiative

Funding for First Nations SchoolNet is flowed directly to the FNRMOs, which deliver direct and indirect services. They include the following organizations:

  • FNESC servicing British Columbia
  • Keewatin Career Development Corporation servicing Alberta and Saskatchewan
  • Keewatin Tribal Council servicing Manitoba
  • Keewaytinook Okimakanak/Northern Chiefs Council Network servicing Ontario
  • FNEC – Technology, servicing Quebec
  • Mi'kmaw Kina'matnewey servicing the Atlantic provinces

Additionally, AANDC funds the Inuit Tapiriit Kanatami. For Cultural Education Centres, AANDC directly funds First Nations, Inuit and Innu cultural education centres. AANDC also funds the First Nations Confederacy of Cultural Education Centres who manages the funds and administration for a majority of First Nations Cultural Education Centres.

Eligible recipients for block contributions are First Nations and tribal councils.

1.2.4 Program Resources

In 2010-11, AANDC spent nearly $1.8 billion on education, [Note 5] approximately $1.5 billion of which was spent on ESE programming. Table 1, below, provides a breakdown of total ESE funding from 2006-07 to 2010-11.

Table 1: Total K-12 Program Expenditures from 2006-07 to 2010-11 by Region
Region 06/07 07/08 08/09 09/10 10/11 Total ($)
Atlantic $35,371,340 $39,930,003 $41,618,262 $47,727,998 $47,596,941 $212,244,544
Quebec $87,482,268 $89,929,515 $92,310,524 $97,098,323 $101,184,025 $468,004,655
Ontario $234,021,762 $237,866,555 $243,442,766 $251,463,050 $261,988,043 $1,228,782,176
Manitoba $215,828,958 $226,466,397 $232,172,382 $239,441,473 $242,482,991 $1,156,392,201
Saskatchewan $174,891,081 $179,603,443 $185,579,852 $200,162,367 $211,673,006 $951,909,749
Alberta $184,400,660 $190,523,973 $196,374,326 $203,657,405 $208,047,581 $983,003,945
British Columbia $169,536,542 $172,976,689 $175,774,570 $181,449,148 $183,183,848 $882,920,797
Yukon $1,277,722 $1,199,009 $1,190,529 $1,626,894 $1,513,789 $6,807,943
NWT $0 $0 $0 $0 $0 $0
Nunavut $0 $0 $0 $0 $0 $0
HQ Admin $12,242,757 $8,714,850 $10,333,780 $19,932,787 $15,563,273 $66,787,447
SUB-TOTAL [Note 6] $1,115,053,090 $1,147,210,434 $1,178,796,991 $1,242,559,445 $1,273,233,497 $5,956,853,457
Band Support Funding and Band Employee Benefits
 [Note 7]
$86,598,945 $88,533,520 $92,288,302 $92,973,141 $96,592,818 $456,986,726
SUB-TOTAL: K-12 EDUCATION $1,201,652,035 $1,235,743,954 $1,271,085,293 $1,335,532,586 $1,369,826,315 $6,413,840,183
Education Agreements
 [Note 8]
$127,969,860 $137,402,303 $143,170,317 $146,705,800 $156,842,473 $712,090,753
TOTAL: K-12 EDUCATION EXPENDITURES $1,329,621,895 $1,373,146,257 $1,414,255,610 $1,482,238,386 $1,526,668,788 $7,125,930,936

ESE programming is funded through annual Comprehensive Funding Arrangements (CFA) and five-year Canada/First Nations Funding Agreements (CFNFA). These arrangements include various funding authorities, such as grants, set contributions (SC), fixed contributions (FC), and block contributions (BC). [Note 9] Appendix C outlines ESE program components by funding authority.

All components of the ESE Program are eligible for funding under set contributions, fixed contributions and block contributions with the exception of the following program components: funding to Federal Schools, funding for New Paths, funding for Enhanced Teacher Salaries, the accountability initiatives including performance measurement systems, student assessments, school success plans and initiatives to encourage collaboration between Aboriginal organizations, federal, and provincial governments. The excepted items are eligible only for contribution funding.

Effective April 1, 2011, AANDC began using additional fixed, flexible and block contribution funding approaches for transfer payments to Aboriginal recipients, as described in the Directive on Transfer Payments [Note 10] and according to departmental guidelines for the management of transfer payments. Prior funding for PSE through Alternative Funding Arrangement (AFA) or Flexible Transfer Payment (FTP) mechanisms will remain in effect, as required, until the expiration of existing funding arrangements that contain these mechanisms. Following this, AFA will become Block Contributions and FTP will become Fixed Contributions.

The allocation of program funding involves the distribution of funds from Headquarters (HQ) to the regions and from the regions to the recipients. The Department-First Nation Funding Agreements (DFNFAs) are formula-driven, and are consistent throughout the country. The CFA allocation model for the program, on the other hand, varies sometimes significantly by region.

In terms of the allocation of funds from HQ to the regions, program funding is a component of each region's annual core budget. The Education Branch does not determine the amount of program funding to be allocated to each region. This is the responsibility of the Resource Management Directorate in Finance at HQ. National budget increases (currently two percent annually) are allocated to each region in proportion to their existing budgets.

The regions have the authority to allocate funds across the various programs included in their core budget and therefore, ultimately decide the extent of program funding to provide to their recipient.

1.3 Current Evaluations

The current evaluations of ESE and PSE looked at the entire suite of education programming offered by AANDC. In the past, AANDC has evaluated components of its education programming, such as the Evaluation of the First Nations SchoolNet Program (2009), the Evaluation of the Special Education Program (2007), the Summative Evaluation of the Cultural Education Centres Program (2005), and the Summative Evaluation of Band-Operated and Federal Schools (2005). While evaluating programs from this lens has its benefits, the result is a lack of understanding of how all programs affect overall student outcomes. It is expected that evaluating Aboriginal education in this light will provide the Department with a clearer sense of the current state of Aboriginal education and insight on future direction.

Internally, the Branch's evaluation work is informed by its Engagement Policy that provides Branch staff with a framework for including Aboriginal people and organizations in the evaluation process. The underlying principle is that, for evaluations to be relevant to, and have meaning for, Aboriginal people, Aboriginal people need to be fully involved in the evaluation process. As part of this evaluation, Aboriginal involvement was attained through interviews, surveys and case studies (including focus groups).

Furthermore, an Advisory Committee was established, composed primarily of national First Nation and Inuit education organisations, and with representation from AANDC. The purpose of the Advisory Committee was to provide guidance and insights on the development of evaluation tools, and on broader interpretation of findings and forming recommendations. The Advisory Committee representatives met at various stages of the evaluation process to discuss and provide advice on:

  1. Terms of Reference
  2. Methodology Reports
  3. Acquisition of Qualified Consultants
  4. Potential Key Informants, Case Study Communities and Literature
  5. Technical Reports
  6. Evaluation Findings
  7. Draft Final Evaluation Report





2. Evaluation Methodology

2.1 Evaluation Scope and Timing

The evaluation examined the following components of ESE programming: instructional services for Band Operated Schools, Federal Schools and Provincial Schools; Elementary and Secondary Student Support Services; New Paths for Education; Teacher Recruitment and Retention, Parental and Community Engagement, First Nation Student Success Program, Cultural Education Centres, Special Education, Education Partnerships Program and First Nations SchoolNet.

The Terms of Reference were approved by AANDC's Evaluation, Performance Measurement and Review Committee (EPMRC) on May 14, 2010. This evaluation provides information on program relevance, performance, and efficiency and economy to support the management of program authorities in compliance with the Policy on Transfer Payments

Four contractors, Donna Cona Inc., Harris/Decima, KPMG and the University of Ottawa were contracted to undertake primary data collection (see Section 2.3). Data collection and analysis was undertaken primarily between September 2010 and April 2012.

2.2 Evaluation Issues and Questions

In line with the revised evaluation questions approved at EPMRC in September 2010, the evaluation focused on the following issues:

  • Relevance
    • Is there an ongoing need for the program?
    • Is the program consistent with government priorities and AANDC strategic objectives?
    • Is there a legitimate, appropriate and necessary role for the federal government in the program?
  • Performance
    • To what extent have intended outcomes been achieved as a result of the program?
    • To what extent has the program influenced the constructive engagement and collaborative networks to further education and skills development?
    • What are the factors (both internal and external) that have facilitated and hindered the achievement of outcomes? Have there been unintended (positive or negative) outcomes? Were actions taken as a result of these?
    • To what extent has the design and delivery of the program facilitated the achievement of outcomes and its overall effectiveness?
  • Efficiency and Economy
    • Is the current approach to programming the most economic and efficient means of achieving the intended objectives?
    • How could the program be improved?

2.3 Evaluation Methodology

The following section describes the data collection methods used to perform the evaluation work, as well as the major considerations, strengths and limitations of the report and quality assurance.

2.3.1 Data Sources

The evaluation's findings and conclusions are based on the analysis and triangulation of seven lines of evidence: case studies, data analysis, expenditures analysis, document and file review, key informant interviews, literature review, and surveys (see also Appendix A, Evaluation Matrix). All lines of evidence were collected for this evaluation and the Summative Evaluation of PSE simultaneously to minimise costs and reduce the burden for individuals and communities by collecting all the information at one time.

  • Case Studies:

A total of 14 case studies were conducted by Donna Cona Inc. for both Elementary/Secondary and Post-Secondary Education evaluations. Case studies provided qualitative insights into whether intended impacts of education programs, policies and initiatives are occurring, and to allow for a thorough analysis of First Nation perspectives on needs, challenges and best practices in providing a quality education to First Nation and Inuit students.

Seven case studies were conducted using a regionally-based approach, where one community was selected randomly for each AANDC region (British Columbia, Alberta, Saskatchewan, Manitoba, Ontario, Quebec and Atlantic) while attempting to acquire a general representation of varying community sizes, local economies, and isolation. In these case studies, Donna Cona Inc. was asked to take a holistic approach to education in the community to gain a comprehensive understanding of issues facing individuals and communities more broadly. Case study tools were designed by Aboriginal researchers and customised to include various methods of communication if the respondent so chose.

The regionally focused case studies included, but were not limited to, participation from:

  • Chiefs and/or member(s) of band councils responsible for education profiles;
  • Education office/department managers and staff and/or persons responsible for elementary / secondary / post-secondary school management;
  • School principals;
  • School counsellors, on-reserve elementary and secondary teachers, cultural centre's staff, elders, staff directly involved in the delivery of on-reserve special education programming;
  • Parents of students who attend the school(s);
  • Students, aged 16 and over who are involved at school;
  • Group sessions may have included Elders, parents and student representatives;
  • Off-reserve principals and/or school board administrators; and
  • Off-reserve secondary school coordinators, off-reserve teachers responsible for the education of on-reserve students under Local Education Agreements.

Seven thematic case studies were also conducted: three focussing exclusively on PSE (including one with an Inuit student focus), two for CEC and two focussing specifically on Special Education. The purpose of these case studies was to obtain a comprehensive review of the programs being funded and the effect that they are having on recipients and communities more generally.

Similar to the regional case studies, the thematic case studies included individuals with similar portfolios to those listed for the regional case studies.

Prospective communities or organisations were contacted by AANDC in writing and subsequently contacted by Donna Cona Inc. to confirm whether participants were interested. Communities wishing to participate worked directly with the consultant to organise the site visit and information sharing. Participants were asked if they were willing to have their interviews recorded. Willing participants' audio files were securely stored electronically, and hand-written notes were securely stored for those not wishing to be recorded. Data was stripped of personal identifiers and securely transmitted to the Evaluation, Performance Measurement and Review Branch (EPMRB) for analysis and interpretation using NVivo 9 Qualitative Analysis Software. Data are stored in accordance with departmental guidelines on data storage, to be destroyed after five years, and are handled as "Protected B" [Note 11] information. Responses were organised by evaluation questions and open-ended thematic elements.

Concise case study reports were also completed by Donna Cona for each regional and thematic case study, which included community and/or program specific context, as well as relevant data and statistics. A technical report of case study analysis was conducted by EPMRB and reviewed by Donna Cona.

  • Costing Analysis:

KPMG was contracted to review detailed expenditures data and methodology, comparatively between AANDC and all provincial school districts. This comparison was to develop an empirical appreciation of the level of resources required to implement education programs and services in First Nation schools that are comparable to those in adjacent provincial schools. The study was to examine the most up-to-date financial data available for all districts (2009) and analyse funding formulae used by provinces, and to identify the principles and assumptions that apply to the funding methodologies, and the potential implications for the Department in moving forward under various scenarios. KPMG collected detailed expenditures data from most school districts, including detailed funding formulae. AANDC provided its funding break-downs by category for comparison.

  • Data Analysis:

A review of the Nominal Roll database was undertaken by EPMRB for fiscal years 2006-07 to 2010-11. Two databases were developed using extracts from raw Nominal Roll data on a per‑student basis. One was designed to assess student progress over time, and the other to contextualize student outcomes with other community factors, such as population, isolation, income, and Community Well-Being Index (CWBI) factors such as community education level, housing, labour market strength and household income. Personally identifying information was stripped from extracted data. Data were analysed to provide an assessment of drop-out frequency and characteristics; progression rates of students through high school; graduation rates and characteristics; and educational outcomes in the context of community characteristics. All data used for this exercise were treated as "Protected B" information.

  • Document and File Review:

A document and file review was undertaken by EMPRB and included, but was not limited to, Treasury Board Submissions, the Departmental Performance Report, the Report on Plans and Priorities, Budget announcements, previous audits and evaluations, the Education Performance Measurement Strategy, Office of the Auditor General (OAG) reports, annual reports, project files, Regional Health Survey publications, Education Information System documentation, the Final Report of the National Panel on First Nations Education [Note 12] and other administrative data. Relevant documents were scanned for content relating to key evaluation questions and the analysis was triangulated with other lines of evidence.

  • Key Informant Interviews:

A total of 40 interviews were conducted by Donna Cona Inc. with the following key informants: six AANDC HQ, 11 AANDC region, and 23 First Nation organizations. The purpose of key informant interviews was to obtain a better understanding of the results of education programming supported by AANDC and its impact on First Nation and Inuit communities. Key informants further helped to identify other potential key informants and communities for case studies.

Prospective key informants were contacted by EPMRB in writing and then subsequently contacted by Donna Cona Inc. to establish their willingness to participate. Those wishing to participate negotiated a time and interview process with the consultant. Participants were asked of their willingness to be recorded. Willing participant's audio files were securely stored electronically, and hand-written notes were securely stored for those not wishing to be recorded. Data was stripped of personal identifiers and securely transmitted to EPMRB for analysis and interpretation using NVivo 9 Qualitative Analysis Software. Data are stored in accordance with Government of Canada guidelines on data storage, to be destroyed after five years, and are handled as "Protected B" [Note 13] information. Responses were organised by evaluation question and open-ended thematic elements.

  • Literature Review:

Members of the Faculty of Education at the University of Ottawa conducted a "Meta-Analysis of Empirical Research on the Outcomes of Education Programming in Indigenous Populations in Canada and Globally," which included a comprehensive assessment of literature (published and unpublished) over the past 10 years, and emphasized peer reviewed empirical research related to educational outcomes for students in First Nations and other Aboriginal populations in Canada. The purpose of the review was to provide insight on the impacts of policies and programming in education (curricula, investments, governance) on student success, as well as economic integration.

Following this initial analysis, it was determined that a supplementary literature review [Note 14] would be conducted by EPMRB to provide further insight on the following themes:

  • Jurisdiction and control in First Nation education
  • Education systems and second and third level services
  • Early childhood education
  • Adult learning and higher education

The issues examined for each thematic area included current policy and practice, relevance, results, best practices and lessons learned. The bibliography was informed by EPMRC and the Advisory Committee, and drew primarily upon policy, research and evaluation studies as well as state-of-the art and state-of-practice literature reviews.

Other sources, such as websites, conference proceedings and working group papers, were also included where they proved useful in providing context or in identifying emerging research and issues.

  • Surveys:

Surveys were undertaken by Harris/Decima to gain insights from education authorities/managers, principals and Regional Management Organisations beyond what could be obtained with key-informant interviews. A total of 113 surveys were completed.

To maximise confidence in the generalisability of Education Administrator (approximately 600 in total) survey results, a minimum required sample size of approximately 200-225 respondents was necessary. Based on an anticipated response rate of 30 percent, it was decided that EPMRB would attempt to engage contacts from all First Nation communities in Canada to participate. In attempting to produce accurate contact information for all First Nation education administrators, EPMRB was able to gather contact information for 520 of 616 First Nations across Canada. EPMRB contacted all prospective participants in writing and then followed up via telephone to gauge interest in participating. Those agreeing to participate were given the choice to complete their survey online or via telephone. Details were logged and sent to Harris/Decima, who then followed up with participants.

Survey results were compiled by Harris/Decima, stripped of any identifying information and analysed by EPMRB using PASW 18 Statistical software and NVivo 9 Qualitative software for open-ended responses. A total of 16 follow-up interviews were also conducted with respondents who agreed to elaborate on some of the qualitative answers provided in their surveys. Qualitative data from the survey and its follow-up interviews were coded and analysed in a consistent fashion as with case studies and interviews.

In terms of regional representation among the 113 participants, regions were generally represented proportionate to the number of reserves in each, but with Ontario and Alberta somewhat overrepresented and Quebec underrepresented.

2.3.2 Considerations, Strengths and Limitations

Since the evaluation builds upon the critical analysis of existing data, it also favours reductions in the reporting burden on First Nations communities. Reducing the reporting burden for recipients is a key priority for AANDC and for the Government of Canada at large.

This evaluation will further consider AANDC's Sustainable Development Strategy's [Note 15] objective of enhancing social and economic capacity in Aboriginal communities through educational programming, with particular emphasis on factors affecting Aboriginal high school graduation rates.

Moreover, the evaluation adhered to AANDC's Policy on Gender Based Analysis [Note 16] by analysing multiple variables based on gender.

This evaluation comes at a time when several reforms for First Nations education are underway or being considered, such as "Reforming First Nation Education" beginning in 2008 and actions that will be taken as a result of the National Panel on First Nation Elementary and Secondary Education for Students on Reserve. While this evaluation focuses primarily on outcomes to date, it also discusses priorities moving forward.

Survey Respondent Selection: While every attempt was made to have a broad representative sample of First Nation education managers/directors, it is expected that a degree of respondent bias exists, as selection was not truly random, and those agreeing to participate may or may not have views aligning with those not agreeing to participate, or those we were unable to reach. Additionally, given that respondents were asked to send a survey link to post-secondary students on behalf of the evaluation team, there is no way to know how many students actually received the survey, and no follow-up protocol was available for this group. As only 113 of the target 200 surveys were completed, the minimum sample size needed for statistical validity was not officially met.

Interviewee Respondent Selection: While every attempt was made to have a broad representation of First Nation and Government of Canada viewpoints, it is expected that there could be a degree of respondent bias. Specifically, it is possible that those agreeing to participate could have viewpoints inherently different from those not interested in participating.

AANDC Internal Database Limitations: Data recorded for the Nominal Roll have only recently become reliable insofar as a consistent application of data entry nationally. Thus, this analysis is limited to only 2006-07 to 2010-11 data. Consequently, only five school years of statistics were analysed for trend analysis, and only one full cohort could be legitimately assessed for progression over time (grade 9-12). Additionally, Nominal Roll data are not independently verified and thus, it is not possible to assess the accuracy of the information recorded in the system. It is also important to note that community-based information used to create one of the two data bases are based on Census 2006 figures. While these figures are outdated, it is expected that as this analysis is national, changes in the subsequent six years in all probability would not be significant enough to change observations in the current analysis.

Expenditures Analysis: The most significant issue regarding expenditure analysis was inconsistencies in the way financial data are recorded between districts, between regions and over time. For example, for each funding component of education programming, each region uses different definitions and includes different line items in each category, thus, making legitimate comparisons extremely difficult. Interpretation was impeded by the fact that the expenditures recorded for each provincial district did not correspond to the official regional funding formulae. Additionally, the study was unable to account for all income sources for education authorities, thus, potentially underestimating the available resources. Given these limitations, and the significant complexity of analysing each formula in the context of on-reserve education, more research will be needed to provide a clearer sense of expenditure comparisons.

2.4 Roles, Responsibilities and Quality Assurance

EPMRB of AANDC's Audit and Evaluation Sector was the project authority, and managed the evaluation in line with EPMRB's Engagement Policy, outlined in Section 1.3, and the Quality Assurance Strategy (QAS). The QAS is applied at all stages of the Department's evaluation and review projects, and includes a full set of standards, measures, guides and templates intended to enhance the quality of EPMRB's work.

Oversight of daily activities was the responsibility of the EPMRB evaluation team, headed by a Senior Evaluation Manager. The EPMRB evaluation team was responsible for overseeing all aspects of data collection and validation, as well as composition of the final report and recommendations.

All information from consultants was thoroughly reviewed by the Project Authority (EPMRB) for quality, clarity and accuracy. Subsequently, the consulting firms thoroughly reviewed the analysis of data conducted by EPMRB to ensure their information was well-captured. The Advisory Committee assisted in the development of tools, interpretation of findings and the development of final reports.

In addition, status updates and key findings were regularly provided to EPMRC for strategic advice and direction.






3. Evaluation Findings – Relevance

3.1 Continued Need

Is there an ongoing need for investment in First Nations Education programming?

Finding: This evaluation found that there is a need for continued investment in the Authorities for Elementary/Secondary Education stemming from projected population growth and from the need for significant improvements in student outcomes.

The need for continued investment in Education programming is amply demonstrated by the significant gap in educational opportunities and outcomes between First Nation and other Canadian learners [Note 17]. Findings in the current study are consistent with observations made in the Formative Evaluation of the Elementary/Secondary Program On Reserve [Note 18] - that the expected growth in the on-reserve population will undoubtedly generate greater needs for ongoing investment in education services and the sustainability of programs and facilities to accommodate this growth. Population projections obtained from Statistics Canada show a projected growth in the population of children on reserve of approximately 15 percent, or an additional 30,000 children (see Figure 1) over the next two decades. This does not include the potential implications of persons relocating to reserves as a result of amendments to Bill C-31 [Note 19] and Bill C-3 [Note 20].

Among the greatest concerns raised by First Nation interview and case study participants were significant ongoing and projected increases in operational costs relative to available resources. It was broadly agreed among most case study and interview participants, both First Nation and government, that there are serious gaps in the ability of First Nation schools to attract and retain teachers and support staff with competitive salaries and benefits, and in the ability to manage increasing costs for programming and infrastructure. There was broad consensus that these issues need to be addressed strategically and expeditiously to ensure improvements in student outcomes.

Figure 1: Projected Population Growth by Age Category of Interest from 2010 to 2026

Projected Population Growth by Age Category of Interest from 2010 to 2026

Text description of Figure 1 is available on a separate page.

As stated in the Report of the National Panel on First Nation Elementary and Secondary Education On Reserve [Note 21], the lack of a high school education is strongly tied to significant unemployment, high use of social assistance and a high incidence of involvement in the justice system. Given the broad and excessive social costs associated with these elements, coupled with the current rate of graduation (estimated between 25-30 percent on reserve and 35-40 percent for reserve students attending off reserve, according to Nominal Roll data), and the potential for successful First Nation graduates to address projected labour shortages over the next two decades, there is likely a far greater cost associated with status quo than with strategic investments into the future.

Case study and interview participants frequently raised the issue of transferability of skills and knowledge from schools on reserve to secondary schools off reserve. It was noted that a key priority of education programming investment needs to be in ensuring that students in any given grade on reserve are receiving equal quality instruction and transferable knowledge and skills as those in provincial schools to increase the probability of success.

3.2 Alignment with Government Priorities

3.2.1 Are Education Authority Activities strategically aligned to Government of Canada Priorities?

Finding: Education authority activities are generally aligned with Government of Canada priorities; however, recent major reforms are reflective of the need to better align activities and better ensure improvements in student success.

Insofar as Education Authority activities relate to better education outcomes for First Nation students, there is a clear strategic alignment with government priorities. The Department's approach to ESE, however, was in a state of significant change through the course of the current study and continuing into the coming years based on the need to further refine its activities to be better aligned with strategic outcomes.

Critically, the 2004 Report of the Auditor General [Note 22] stated that, "The Department has not clearly defined its roles and responsibilities. The way it allocates funds to First Nations does not ensure equitable access to as many students as possible, and the Department does not know whether the funds allocated have been used for the purpose intended." The report recommended that the Department implement a comprehensive strategy and action plan to close the education gap. The 2011 Status Report of the Auditor General, however, noted that while work had begun in response to those recommendations, there was not a consistent approach and it could not demonstrate improvements. The Reforming First Nations Education Initiative was launched in 2008 with a stated target to have 75 percent of First Nation students achieve education outcomes comparable to those of the rest of Canada by 2028.

The Government's continued investments and stated objectives for First Nations education – as outlined in budgets 2011 [Note 23] and 2012 [Note 24] – further demonstrate recognition of the need for strategic changes and innovative planning in order to better align AANDC activities with desired outcomes for First Nation students. Specific recent examples include the formation of the National Panel on First Nations Education; the Economic Action Plan's purported additional $275 million in new spending over the next three years; the introduction of a First Nations Education Act by 2014; the $100 million investment in early literacy and other supports; and the $175 million investment in school renovations.

Additionally, the Reforming First Nation Education Initiative, which received significant praise from Aboriginal and government interview and survey participants, has facilitated significant new funding initiatives through its First Nation Student Success Program and Education Partnerships Program. These initiatives are designed to provide extra support for educators on reserve and to promote greater collaboration with provincial schools, the federal government and other stakeholders to facilitate better student outcomes.

Major revisions in education-related activities reflect a common sentiment expressed by government interview participants in this study – that while AANDC education programming outcomes are aligned with the Government's objectives, its broad activities are undergoing significant reform because outcomes are not improving in a satisfactory manner.

3.2.2 Are the education-related priorities of AANDC aligned with those of First Nations?

Finding: The priorities as stated by First Nation Education Authorities and those of AANDC are aligned insofar as the need to address marked gaps in educational opportunities and success. However, First Nation participants emphasise key priorities in the areas of cultural and language retention as being critical to success, and emphasise the need to recognise key differences in learning needs and the current state of education gaps, rather than simple notions of comparability.

When asked to explain what they saw as the ultimate priority for First Nations education programming, nearly all respondents indicated that it was essential to close the gap in terms of both the opportunities provided and the successes achieved by students. This was seen both as an intervention to address personal, community and economic well-being, and as an opportunity to bring significant numbers of skilled professionals into the labour market.

Additionally, however, it was the broad consensus among First Nation participants that the retention of heritage languages, and learning with culturally-relevant curricula were essential to improving outcomes. Specifically, participants suggested that retaining a strong cultural identity, coupled with knowledge of their history and competence in their heritage language creates a sense of personal belonging within a community, and a stronger sense of self-worth necessary to feel motivated to be successful in school. While there is not an abundance of peer-reviewed literature directly linking culturally-based learning to longer-term educational success, there have been some promising results in short-term goals in language and heritage immersion programs [Note 25][Note 26]. Much of the research that exists generally suggests that such programming increases the congruence between school, language, and culture; includes topics of relevance to youth; provides accurate images of both the past and present; improves the self-esteem and pride of Aboriginal youth and increases engagement [Note 27][Note 28].

While internal data from the Nominal Roll is not designed to account for the availability of culturally-relevant curriculum or language immersion programs, what is clear is that students perform more poorly when their language of instruction is different from their language spoken at home. Specifically, when examining progression from one grade to the next, a difference between the instruction and home language was strongly associated with increased time to progress through high school [Note 29]. These findings, as well as other research reviewed, suggest the need not only for greater English and/or French proficiency, but also the clear benefits of heritage language retention.

For example, a study of heritage language immersion programs [Note 30] examined students enrolled in English or French language instruction and either course work or full immersion in Inuktitut. Students in the Inuktitut immersion program showed steady improvement in the heritage language and made gains in English. Students in the English program showed steady gains in English, even though it was not spoken at home. However, they did not have the same proficiency in Inuktitut as peers in the immersion program. Students in the French program showed good progress in that language despite the lack of instructional materials, but their skills in Inuktitut showed the slowest gains of the three groups. Only students in the Inuktitut immersion program had attained a level of proficiency that allowed them to solve complex mental problems in that language. The researchers also noted that personal self-esteem scores at the end of kindergarten were the highest among the Inuktitut immersion students.

First Nation interview and case study participants particularly suggested the need to incorporate traditional forms of learning and communication such as traditional and creative arts; music; dancing; and story-telling, into broader efforts to improve numeracy and literacy. They further suggested that where such learning is present, there is evidence of positive effects insofar as attendance and engagement in the subject matter. Some participants suggested First Nation students need to see themselves in the learning as an engaged actor in the process, and this will facilitate a better identification with the subject matter, a stronger sense of identity and pride, and thus, a motivation to attend and do well in school. An example of a holistic learning process of this kind involves the math teacher engaging students in the rhythm of algebraic equations, for example, with physical movements as they take in, understand, and internalize the mathematical learning so the math is integrated within the 'whole' person.

First Nation participants also suggested that their students learn much better with hands-on opportunities. They indicated the need to move beyond classroom instruction and to offer more applied learning relevant to their culture, as well as extra-curricular activities. In this way, while the broad desired outcome of success is consistent between First Nation education administrators and AANDC programming, the notion of comparable opportunities may be somewhat of an oversimplification. The opportunities that should be comparable, according to all participants, are the availability of quality teaching in quality schools, opportunities for broad educational growth and development, opportunities to pursue post-secondary, and to be competitive in the labour market. Where this becomes more complex is that many First Nation participants see the specific opportunities for learning as very different between First Nation and non-First Nation learners. Specifically, participants suggested a significant divide between a holistic approach to learning versus the dominant approach of focussing on the mental processes, with little to no attention given to the spiritual, emotional, and physical learning/intelligence, which participants suggested cannot be separated from the mental processes in a holistic model for education.

This is particularly apparent in the expressed need for more culturally-relevant learning in the off-reserve schools systems. The concern focussed on the limited success among First Nation students attending off-reserve schools. The First Nation students' inability to see themselves in the subject matter and a general lack of welcoming and culturally relevant learning environments lead to an array of negative outcomes for many students, according to participants. Therefore, First Nation participants indicated that opportunities, while needing to be equitable, needed to be tailored to the needs for First Nation students and their families.

Caution was also raised by respondents from all groups around the concept of comparability. It was particularly noted that currently First Nation students lag far behind their non-First Nation counterparts in overall performance – particularly evident in the rates of graduation and drop-out. Additionally, it was noted that there is a degree of resistance to the education system among some parents, largely linked to trauma associated with the residential schools system. Thus, participants were clear in the notion that comparability should not imply receiving the same services; rather that starting at a significantly deficient point, First Nation communities and students will need significantly more supports than their counterparts.

3.3 Role of the Federal Government

Is the current role of the Government of Canada legitimate, appropriate and necessary for the improvement of First Nations education success?

Finding: The role of the Government of Canada in ESE programming is generally appropriate; however, policy changes with respect to service delivery and local operational control may have implications on this role in the future.

All AANDC participants stated that the role of the Government of Canada, and specifically of AANDC, was appropriate with respect to ESE programming. The Reforming Education Initiative was frequently cited as the ongoing mechanism of the Government's prioritising its support of First Nations to better ensure positive student outcomes. There were also references made to the Government's stated notions that First Nations youth represent an enormous potential for the current and future labour market, and that currently the potential is not being met.

Similar to issues discussed in Section 3.2.1, however, the precise role of the Government has the opportunity to evolve given the major reforms underway, and the desire to increase First Nation control of First Nations education. While there was no clear sentiment expressed by interview participants of the broad role of the Government needing to change, as discussed in Section 4.4, issues with respect to jurisdiction and control, as well as evidence of a need for broader education initiatives from early childhood, and potentially innovative structural changes designed to better meet learning needs, have all been raised. Evidence from this study, and that of the National Panel on First Nations Education, have pointed to key aspects needing change, which may have implications on the Government's current specific roles in program design and delivery.






4. Evaluation Findings – Performance

4.1 Achievement of Intended Outcomes

To what extent have intended outcomes been achieved as a result of AANDC programming?

Finding: It is the finding of this evaluation that the intended outcome of comparable education opportunities and results to the Canadian population is not being achieved.

Evidence from case studies and key-informant interviews is consistent with notions in the National Panel on First Nations education that suggest First Nation students do not have comparable education opportunities to other Canadians. Issues related to this lack of comparable opportunities are discussed in detail in sections 4.3 and 4.4. Without comparable opportunities, there can be no reasonable expectation of comparable outcomes, and it is clear from the evidence below that neither the opportunities nor outcomes are comparable.

As shown in Figure 2 and as discussed in the Formative Evaluation of ESE [Note 31], data from Statistics Canada shows that while the rates of Aboriginal Canadians with a high school diploma or greater living off reserve have typically been about 20-30 percent lower than the Canadian average, the gap is slowly narrowing. The same cannot be said of those living on reserve. Generally speaking, the on-reserve population had only seen a five percent increase in the proportion of individuals with a high school diploma or higher on reserve – from about 25 percent to 30 percent of adults under 25 between 1996 and 2006.

Figure 2: Proportion (%) of individuals by Age Group, Status (Aboriginal Off Reserve, Aboriginal On Reserve, Non-Aboriginal), Gender, and Census Year with a High School Diploma or Higher

Proportion (%) of individuals by Age Group

Text description of Figure 2 is available on a separate page.

Examining Nominal Roll data for the years subsequent to 2006, there appears to have been little to no improvement. The drop-out rates [Note 32] have remained consistent, although significant improvement was noted in the pre-grade 11 cohorts. Being above the average age for any given grade was strongly associated with an increased likelihood of drop-out, and the likelihood becomes stronger the older the student. Generally speaking, the drop-out rate is higher among females before grade 9; and roughly even between males and females for higher grades. Being older than average within a grade, however, has a far greater impact on the likelihood of males to drop out than of females. In particular, males over 18 are far more likely to drop out and this trend becomes progressively sharper with age [Note 33].

The average age of a school-aged student dropping out is 17.5 years old and as shown in Figure 3, the drop-out rates increase sharply with age and overall the rate is high, from about five to eight percent on average in any given school year for students between grade 9 and 12, and much higher for older aged students. It is important to note that these figures reflect only official and logged drop-out (a student who was present in the previous year and is not enrolled the following year but did not graduate) thus, the actual figure may be underestimated.

Figure 3: Drop-out Rates by Age Collapsed across 2006-07 to 2010-11

Drop-out Rates by Age Collapsed across 2006-07 to 2010-11

Text description of Figure 3 is available on a separate page.

In examining those that dropped out, approximately 23 percent eventually returned, but only 13 percent of those who returned and reached their final year actually graduated.

The rate of graduation [Note 34] has shown virtually no improvement at approximately 34 percent overall. Looking only at school-aged individuals (21 or younger), the rate has remained steady at 37 percent. The average age at graduation is 20 years old and this has not changed over time.

Examining regional trends, there was a significant [Note 35] improvement in Ontario from about 23 percent to 30 percent and this was particularly pronounced for female students, from about 23 percent to 34 percent. Atlantic region had the highest rate of graduation overall and also showed significant [Note 36] improvement from about 55 percent to 65 percent.

The graduation rate was higher for females (33 percent) than males (30 percent), and while the percentage difference appears small, it represents a difference of about 200 students per year and is significant. [Note 37] This trend, however, reverses with age with males tending to have a higher likelihood of graduating among cohorts over 21 – a trend that increases with age. [Note 38] Thus, the data appear to suggest that older males are more likely to drop out than females and more likely to graduate. In other words, among older cohorts, male students in their final year will more often either graduate or drop out, whereas female students, while less likely to graduate, are also less likely to withdraw.

For the cohort [Note 39] examined in this study, only 44 percent progressed through grades (to the start of grade 12) within the expected time frame. Nearly a third of these students repeated one grade and 15 percent repeated two grades. More than 600 students in this cohort repeated three or more times within the five years examined. There is no obvious gender difference in these trends. Importantly, education administrators surveyed in this study suggested schools were generally well-equipped to prepare students to transition through earlier school years, but much less so in later years, and certainly there was very little optimism about the ability of schools to facilitate transitions to post-secondary or the labour market.

When speaking with educators, there was a general sentiment that while there is little to no improvement in raw outcome figures, they have noticed some improvement in attitudes toward education in their communities, among their leadership and among students. They further indicated that it will take time for these changes to translate into obvious results.

Survey participants suggested that where there was strength in the quality of learning, it came mostly from the dedication of staff and innovative programming at the school level. With respect to off-reserve schools, issues mostly concerned the lack of culturally-relevant learning, the tendency to push students through the system without adequate preparation, and issues with transferability of knowledge from on- to off-reserve school systems (mostly from elementary to secondary).

4.2 Program Influence on Collaborative Engagement and Networks

To what extent has the program influenced collaborative engagement and collaborative networks to facilitate education and skills development?

Finding: Continued work is needed to better facilitate constructive engagement and collaborative networks between First Nation education authorities, and where appropriate, with provincial governments or other organisations, and there is evidence that AANDC programming is improving in this regard.

Partnerships were said to be integral to future feasibility of a strong First Nations education system, according to participants from all groups. There were key examples from First Nation representatives of partnerships with larger organisations and neighbouring First Nations where collaboration was highly beneficial. It was noted that AANDC must support this type of cooperation in its programming and incorporate partnership development in its support to education programming – which was noted as beginning to improve in recent years with the Education Partnerships Program.

For example, communities that worked together on coordinated transportation, or shared social services, such as social workers and special needs specialists, were noted as seeing improvements in this regard in recent years. Partnerships with mainstream school systems also support the inevitable transition from band-operated schools to public schools off reserve.

Support from the tribal councils and bands was also seen as crucial. There are varying degrees of involvement in education via the leadership, but ultimately First Nation participants viewed their engagement as highly valuable when it happens. Some noted extensive efforts by their band and/or tribal council to facilitate cooperation and the availability of better services, such as school psychologists and recreational staff. There was certainly a sense that leadership needs to take an active role in encouraging students to attend school and succeed through a combination of: facilitating better labour market opportunities locally; promoting labour market opportunities broadly; acknowledging and encouraging students for their efforts and success; speaking directly to students at general assemblies periodically; and lobbying for more effective programming and better resources.

There was also a strong sentiment, especially among AANDC participants, that there needs to be a strengthening of partnerships between First Nation communities and provincial education ministries, as well as leadership and specific Aboriginal education policies embedded within the ministries of Education. Specifically, it was felt that First Nation authorities, school boards and other education service providers could mutually benefit from cooperation to facilitate an economy of scale and better shared services to ensure equity, and a better understanding of First Nation student needs when attending schools both on and off reserve. Additionally, it was noted by respondents from all groups that strong links to provincial ministries were essential because so many of their students eventually attend provincial schools, especially for secondary. It was noted that the perceived gap in readiness for secondary could be better addressed with better linkages and coordination between provincial ministries, bands and tribal councils.

4.3 Factors Facilitating or Hindering Achievement of Outcomes

What factors have facilitated or hindered the achievement of outcomes?

Finding: Student success is associated with parental engagement, the level of education in the community and the strength of the local economy. There are deeper issues related to the historical trauma of residential schools that may be interrelated with these factors.

Finding: Community governance, the quality of teacher instruction and the quality of school curriculum were suggested as key factors affecting student success.

Among First Nation participants in this study, where there were successful educational outcomes, they were mostly attributed to good leadership and governance, and a stronger economy. This latter finding is unsurprising in light of the fact that when examining data on communities, the strength of the labour market and household income were strongly associated with positive educational outcomes. [Note 40] For example, Figure 4 shows a significant [Note 41] tendency for higher median household incomes to be associated with lower drop-out rates as defined by Nominal Roll. Unsurprisingly then, there is also a significant relationship between higher income and higher graduation, as shown in Figure 5.

The education level in the community as measured by the CWBI was the strongest predictor of graduation among the four composite CWBI indicators [Note 42], although income and labour force were also strong predictors. By far, the best predictor of drop-out rates appears to have been labour force activity. [Note 43] Other factors such as community size and isolation were examined, and while community size did not appear related to educational success in Nominal Roll indicators, there was a significant [Note 44] tendency for greater isolation (as measured through geo-zone [Note 45]) to be associated with lower graduation, as shown in Figure 6.

Nominal Roll data also suggests that the likelihood of drop-out is dependent on school type, with high school students in band-operated schools significantly more likely to drop out than those in provincial and private schools. [Note 46] Similarly, the graduation rate is highly dependent on school type, with those attending grade 12 on reserve in any given year are far less likely to graduate (25.8 percent) than those in provincial (36.7 percent) or private (39.7 percent) schools. [Note 47]

Figure 4: Scatterplot of Median Individual Income and Community Drop-out Rate

Scatterplot of Median Individual Income and Community Drop-out Rate

Text description of Figure 4 is available on a separate page.

Figure 5: Scatterplot of Median Individual Income and Community Graduation Rate

Scatterplot of Median Individual Income and Community Graduation Rate

Text description of Figure 5 is available on a separate page.

Figure 6: Mean Community Drop-out Rate by Geo-zone

Mean Community Drop-out Rate by Geo-zone

Text description of Figure 6 is available on a separate page.

Perhaps unsurprisingly, a significant [Note 48] predictor of the likelihood of graduation is the rate of progression through grades. The likelihood of graduation for the cohorts examined dropped between five to seven percent for each extra year taken to complete any given grade between grade 8 and 11. Among graduates in any given year, only half of them had not repeated any grades past grade 8. Additionally, consistent with observations regarding school type, students in band-operated schools are significantly more likely to take longer to progress through grades than those in provincial or private schools. [Note 49] For any given cohort of the five examined in this study, among grade 12 students, 44 percent had progressed from the start of grade 9 to the start of grade 12 within three years.

Survey participants saw strength in the ability of schools to equip students to progress through elementary school, but less so through high school, and much less so to post-secondary or into the labour market. This was not attributed to the secondary schools being weaker than elementary; rather to the cumulative skills of students not being attained over time – with respondents suggesting more students are being "pushed ahead" than actually learning. The sentiment across interview, survey and case study participants was that students are not learning through elementary school and are, thus, at a serious disadvantage at the start of secondary, causing a cumulative impediment to progression and very low graduation success.

The most pervasive programming issue affecting success, according to First Nation participants, was the ability to pay teachers competitive salaries to facilitate recruitment and retention at the same level as provincial counterparts. They suggested that the result was a high proportion of inexperienced teachers and a high staff turn-over. They further suggested that where many students transfer into the provincial system for secondary school, they are at a significant disadvantage because of an often weak or inconsistent teaching base at the start, coupled with transitional issues and a lack of culturally-relevant teaching off-reserve.

The quality of teacher instruction was generally not viewed favourably by most survey participants. Two key themes emerged: For teachers off reserve, the main concern was a lack of awareness and understanding of First Nations culture. This concern was also expressed by some respondents with respect to teachers from outside the reserve coming to the community to teach. For teachers on reserve, the main concern was a lack of professional development opportunities for those who need to enhance their skills, coupled with an inability to attract and retain the most qualified teachers because of a lack of available funds to make teaching salaries and benefits competitive.

There were mixed views on the impacts of local school curricula, and those views were correlated [Note 50] with views on the quality of teacher instruction. In much of the commentary from both case study and survey participants, this largely was related to perceptions of the retention of native language and culture. The vast majority of respondents indicated that the retention of native languages and culture has at least somewhat of an impact on student success, but very few suggested that students are actually retaining their culture and language.

Most participants suggested that the most significant factors affecting student success were parental engagement and poverty. This was linked by participants to the historical trauma of the residential schools system, and parents' resultant resistance to strictly enforce school attendance and studying, and the tendency to create a distance between school and home life. Consistent with the analysis of student outcomes, participants also indicated that the strength of the local economy can mediate this factor, in that with a stronger local economy and job market, parents and the community in general are much more likely to encourage active participation and school success.

Additionally, most First Nation respondents spoke at length about addictions and poor lifestyle behaviours among both parents and children as being major impediments to school success. Links were also made between these issues and the legacy of residential schools.

First Nation participants highlighted the point that improvement in student outcomes is unlikely where basic communities needs, such as basic infrastructure, safety, quality and sustainable housing and family stability are unmet.

Most survey participants had a generally positive view of their community's governance structure to have a positive impact on student outcomes. There seemed to be a mix of opinion, however, with respect to whether the community's system of governance should have more or less involvement in education both among those with positive and negative views of their community's governance. In other words, among those who viewed it positively, some felt that way because the band did not interfere, and others because the band were heavily involved. Among those viewing it more negatively, the same reasons were applied. Overall, however, there was a general desire for more involvement from the band, rather than less. There was a large mix of ideas on what the band could improve in terms of governance, but by far the most common issues raised were the need to not allow education programming to fall victim to political issues, and the need for more advocacy for education funding.

Other issues raised by education administrators included the presence of positive role models; the prevalence of bullying, drugs and alcohol; the general health of the community (mostly related to parenting, drugs and alcohol, and the availability of services); the availability of extra-curricular activities; and the quality of education facilities. Ultimately, however, when asked to comment specifically on factors outside of the education system that factor in to student performance, the overwhelming majority of respondents referenced drug and alcohol issues; parenting and parental engagement; child nutrition; and the local economy and poverty.

4.4 Design and Delivery

To what extent has the design and delivery of the program facilitated the achievement of outcomes and its overall effectiveness?

4.4.1 Available Resources for School Operations and Student Support

Finding: Expenditures to First Nations and tribal councils for the operation of schools does not appear to account for actual cost variability applicable to the needs and circumstances of each community or school, and particularly the cost realities associated with isolation and small population. There is a need for a more strategic understanding of resource needs and allocation methods.

A key issue that has been raised in numerous reports and discourse is an apparent gap in the amount of resources available to schools on reserve compared to their provincial counterparts. As stated in the recent report of the National Panel on First Nations Education On Reserve, there are serious concerns with uncertainty and unpredictability of funding, particularly given proposal-based funding requirements. The figures regarding the actual gaps in resource availability vary; however, analysis completed for this study (data gathered by KPMG from provincial education ministries and AANDC), assessing 2009 revenue and expenditure figures from most provincial school boards across Canada and comparing them with AANDC expenditures reveals interesting findings.

First, it is important to note that expenditure and revenue are not captured in a consistent fashion within regions, between regions, or between AANDC and provincial districts. To the extent possible, adjustments were made to figures to allow for meaningful comparison. Second, it is apparent from the data that there is a great deal of regional variability [Note 51] in these comparisons, and thus, broad comparisons or commentary on a national level would not be productive or informative.

Funding for total education expenditures on reserve exceeded that of provincial districts on average between $5,000 and $7,000 per full-time equivalent (FTE) in British Columbia, Ontario and the Maritimes. There was no apparent difference in the remaining regions. Importantly, however, these costs are largely made up of broader operations and maintenance, as well as project costs, which would naturally be higher on reserve given the higher operating costs and the complex needs of students living in distant and sparsely populated communities [Note 52] and thus these figures do not undermine the point of underfunding. Based on the notion expressed by interview, case study and survey participants – that the resource gap pertains largely to instructional services (mostly teacher salaries) – the current analysis confirms that this gap exists.

While broadly examining a funding amount per-FTE for instructional services, it is only in Saskatchewan that provincial divisions receive significantly more per FTE for instructional services – in the remaining regions the average difference is either nil or First Nation schools receive more.

In comparing these differences, however, it is essential to consider the relative size of each district, considering that an economy of scale will no doubt apply to districts with larger populations and higher numbers of FTEs. Conducting the same comparison of funding for instructional services only for districts where the FTE count is less than 1000 – characteristic of most First Nation communities – there is a marked and somewhat dramatic change in these differences for all regions.

With the exception of Manitoba, for districts with these smaller populations, the provincial districts receive more than First Nation counterparts for instructional services. For British Columbia, the difference reversed from the previous analysis to funding per FTE for instructional services on average to $2,029 greater than their First Nation counterparts. It also changed dramatically in Quebec, with an average $5,953 per FTE higher in the one provincial district with fewer than 1000 FTEs. For all other regions [Note 53], the difference widened markedly, as shown in Figure 7 below. This analysis was replicated with all funding types combined and the same general findings were observed.

Given the larger expenditures on instructional services funding per-FTE in provincial districts with smaller populations, it is clear that provinces are accounting for a higher cost per FTE associated with a smaller pool of students and thus, an economy of scale. This relationship was very apparent in provincial districts (with the exception of Manitoba), and does not appear to be the case on-reserve. On reserve, the only regions observing any clear relationship between the number of FTEs and instructional services funding per FTE on reserve were Manitoba [Note 54] and Ontario. [Note 55] Additionally, while there was no ability in this study to assess the relationship between remoteness and funding per-FTE in provincial schools, it was clear that there is no obvious relationship between remoteness and funding on reserve.

Figure 7: 2009 Instructional Services Dollars per FTE between AANDC and Provincial Districts, by Region for Districts with Fewer than 1000 FTEs

2009 Instructional Services Dollars per FTE between AANDC and Provincial  Districts, by Region for Districts with Fewer than 1000 FTEs

Text description of Figure 7 is available on a separate page.

Taken together this analysis suggests that AANDC does not account for actual cost variability in the same way as provincial education ministries. Deeper analysis of provincial district formulas showed that the broad rates and formulas applied in provincial districts do not translate into the actual expenditures at the school board level. This is because funding allocations in provincial districts are not made based on raw formulae in isolation; rather based more on the specific and unique circumstances in each school.

Assessing the gap in funding between First Nation and provincial schools by assigning average dollar funding gaps per-FTE is not meaningful or useful as the real issue is cost in light of need in each school, which would be more aligned to provincial methodology. If comparability of educational opportunity is the goal, then provincial methodologies, not just raw funding amounts, need to be fully understood and applied to First Nation districts to fully appreciate the degree of gap.

There is also a need to account for the key differences between First Nation and provincial divisions in the realities on reserve that do not exist in other "like" communities off reserve. It also needs to be considered that most First Nation divisions have very few revenue streams and an extremely limited ability to leverage resources, which sets them at an immediate disadvantage when compared to the typical provincial school district. Additionally, the fact that much of the funding for instructional services is proposal-based, and not based on a consistent and predictable funding stream as observed in provincial districts, makes long-term planning for teaching and resources a significant challenge.

Even understanding the differences in revenue to districts between on- and off-reserve communities, it has become clear through examining provincial formulae that there are numerous and quite complex methods of leveraging resources from multiple streams and allocating funds. Thus, there is a need for considerable research into better understanding resource needs and applying resource allocations strategically based on what a First Nation school will need to meet standards in teaching, quality of facilities and access to programs.

4.4.2 Alternative Program Components and Design

Finding: Jurisdictional issues, implications for the provision of second- and higher-level services, and early childhood and adult education are considerations of significant importance in future policy development.

Jurisdictional Considerations

Over the course of the past four decades, the Government of Canada has gradually revised its position on First Nation control of education, and where the federal government had previously controlled schooling options, including residential and day schools, there are now over 500 band‑operated schools in First Nations communities. Approximately 64 percent (69,500) of First Nations students living on reserve now attend these schools, with the majority of the remaining 36 percent (39,000) of students attending provincial schools.

As discussed in further detail in the literature review, the majority of the federal government's actions respecting First Nations education have not been undertaken through the invocation of a constitutional authority; rather, through the Government's statutory obligations under the Indian Act. There are, however, two fundamental shortcomings of the Indian Act as identified in the literature. Firstly, it does not allow the Government to enter into agreements with First Nations organisations to run First Nations schools (although, it does allow for such with the provinces and religious organisations). Secondly, the Act'sprovisions respecting education (sections 114–122) are commonly considered to be woefully inadequate in detail, scope and quality to guide the delivery of education on reserve. These constraints are identified as having contributed to the following outcomes:

  • First Nations education has often been '... vulnerable to the "arbitrariness and inequities' of federal policies and directives' in which direction, roles, responsibilities, funding and accountabilities have never been clearly articulated (tending Senate Committee on Aboriginal Peoples (2011: 48). [Note 56]
  • In the absence of an effective transfer mechanism, federal efforts to delegate control to First Nations resulted instead in the devolution of responsibilities for administering AANDC's own education programs.
  • Until recently, funds and responsibilities have been devolved almost exclusively to individual communities (e.g., the 'Band Operated School Program'). This process was not mindful of the extensive range of supports and resources typically provided to schools by second- and third‑level services (see next section on Second and Third Level Services) (OAG, 2004).

These outcomes have fueled tensions respecting the federal government's commitment to First Nations objectives for control over education. Moreover, without supports and with limited clarity around responsibilities, the potential for developing a culturally relevant education system (a key First Nations policy objective) and/or achieving provincial comparability (a key objective of federal education programming) was limited at best.

The approach used to support education has also contributed to a division amongst some First Nations on the value of local control over education, if that vision remains rooted on an antiquated village school model, long abandoned in Canada and elsewhere as education systems matured.

As summed up in AANDC's 2010 formative evaluation of the Department's K-12 programming:

'… [Much] like other programming in the Department […] AANDC has assumed the position of a funder and has given the control of education programming over to First Nations communities and organizations. In theory, this is respecting the principle of First Nations control of education. The reality, however, is that AANDC still requires a large amount of reporting and has a statutory obligation for education under the Indian Act.'…"without appropriate capacity and resources, many communities are unable to maximize the impact that First Nations control of education could have over something as fundamental as education of children' (Note this passage was also cited in Standing Senate Committee on Aboriginal Peoples, 2011).

These issues contributed to the development of a wide variety of initiatives [Note 57], partnerships and strategies to further First Nation control over education, including:

  • Opting out of the Indian Act through sectoral self-government agreements, which provide for delegated jurisdiction. These include, for example, the Nova Scotia Mi'qmaq Education Act (1997) and the First Nation Jurisdiction over Education in B.C. Act (2006);
  • The development of a diverse range of First Nations-led second level services;
  • The assumption of control by First Nations over school boards or districts. These initiatives have been facilitated by the demographics of specific locations. First Nations which have done this include the James Bay Cree, under theauspices of the 1975 tripartite James Bay and Northern Québec Agreement and the Nisga in B.C. who have entered into an agreement with the B.C. government to manage B.C. School District #92; [Note 58]
  • The establishment of Aboriginal controlled institutes of higher learning, following upon the experience of American Tribal Colleges. To note, unlike their American counterparts, the near majority of Aboriginal institutions in Canada are not eligible to be accredited as degree granting institutions in their own right; [Note 59]; and
  • Efforts to advance culturally appropriate curriculum/language training and pedagogical approaches in communities as well as in provincial/territorial systems (including, for example, the Cree of James Bay who as a result of the 1975 tripartite agreement, have a special mandate and resources to create a separate elementary-secondary program).

Many of the initiatives have been recognized as noteworthy and seminal, encouraging local solutions to local challenges. [Note 60] Yet, concerns are also being raised about the Government's often ad hoc or 'piecemeal' [Note 61] approach, particularly in the case of sectoral self-agreements. Others highlight the length and current unpredictability of self-government negotiation processes and variations in the supports provided to date. [Note 62]

To provide one example, as late as 2004, the Auditor General expressed concern that the Mi'kmaq Education Agreement, signed in 1997, was being used as a model for other sectoral education agreements. In her reasoning, she noted that both parties (the federal government and First Nations) lacked a common understanding of what was meant by jurisdiction, who had it over infrastructure expenditures, and what was to be done with new programming funds since the agreement was signed. She also noted the fact that no determination had been made about what information was required to account for results.

All of these developments have contributed to current calls for reforms, which could serve to support First Nations policy objectives as well as to clarify roles, responsibilities, commitments, funding and accountabilities. In response to the recommendations of the National Panel on First Nations K-12 Education, the federal government committed in the 2012 Budget to:

'… work with willing partners to introduce a First Nation Education Act and have it in place for September 2014, with the goal of establishing the structures and standards to support strong and accountable education systems on reserve… The government will also work to explore mechanisms to ensure stable, predictable and sustainable funding for First Nations elementary and secondary education.'

Further,'[t] o help ensure readiness for the new First Nations education system to be outlined in legislation, this budget will invest $100 million over three years for First Nations education to provide early literacy programming and other supports and services to First Nations schools and students, and to strengthen their relationships with provincial school systems.

To complement these investments in First Nations education programming, this budget will invest $175 million over three years to build and renovate schools on reserve, providing First Nations youth with better learning environments [Note 63]

Second- and Third-Level Services [Note 64]

Given the significant costs associated with operating multiple school districts on reserve and the lack of economy of scale caused by second-level services most often being the responsibility of bands, many interview and case study participants suggested themes consistent with some of the literature reviewed; that AANDC should work with First Nation communities in facilitating the creation of, or links to, First Nation directed second- and third-level service providers that are able to manage funds, operations and curriculum for larger conglomerates of schools and communities. Participants, and the reviewed research, emphasised that in considering such approaches, it is essential to consider the specific needs and capacities of communities when determining the degree of centralisation, and the importance of legislative frameworks outlining clear responsibilities and accountabilities between layers of governance. Further, in many cases, participants suggested that increased partnerships with provincial ministries are essential because many students will attend provincial schools at some point, and the emphasis needs to be on transferability so First Nation students are not at a disadvantage even before entering high school. Additionally, participant groups from interviews, case studies and surveys overwhelmingly suggested that the Department abolish proposal-based funding and develop consistent and predictable funding allocation methods – a finding echoed by the National Panel.

In an analysis of twenty school systems internationally, McKinsey et al. [Note 65] found that school board-like entities were present in the most successful and promising systems. These entities were seen as critical to ensuring success and improvements over time. Other issues that need to be considered, however, are the appropriateness of various models of second- and third-level services given the size and dispersion of many First Nation communities; the composition of the various service levels; the levels of support required and for how long; as well as the roles, responsibilities and relations within existing initiatives, including those resulting from treaties, claims and self-government agreements.

Most of the works reviewed to date highlight the importance of having a framework with roles and responsibilities, and dispute mechanisms established to guide the development of these levels and their potential evolution over time. For example, McKinsey et al. [Note 66] noted that most of the twenty systems undergoing reforms had established or expanded second-level services only after having addressed school level (immediate basic needs) and ministerial (framework, roles/responsibilities and policy) issues.

It was also noted that not all boards are created the same, and they do not all serve the same purpose. While geographically-based boards are the most common in Canada, boards can also be embedded at the school level where they are subject- or issue-based, drawing their membership from schools (most often principals) with additional technical supports. They can also be level-based – with separate boards for primary and elementary, for example.

Perhaps most importantly, however, is the need for long-term investment in capacity. Lessons from First Nations, Canada's Minority Language School Boards, and other national and international experiences suggest that extensive investment and significant capacity building is required where boards are being newly established and where the boards are serving schools or communities with significant needs. [Note 67],  [Note 68]

Specific Programming Components

Many First Nation survey and interview participant stated that current programming does not adequately account for programming needs beyond the classroom – specifically for after school programs, tutoring, summer school, and upgrading. Specifically, case study participants stated there was a lack of adequate resources for libraries, sports and teaching assistants.

Early Childhood Education

Importantly, the issues of early childhood education (not covered under AANDC authorities) were common in interviews and case studies, and pervasive in the literature. Specifically, and consistent with the findings of the National Panel, the research examined in the literature review suggests the need to examine the place of Early Childhood Education (ECE) in programming.

First Nation, Métis and Inuit education policies recognise the importance of ECE while the National Panel has identified ECE as an element of its child-centred approach to education.

A substantial body of international research, including the findings of a number of well-cited longitudinal studies, suggests that appropriate participation in high-quality ECE leads to short- and longer-term improvements in educational achievement, particularly for disadvantaged children. Additionally, positive outcomes (albeit more nuanced and mixed) can also be expected on child health, well-being and parenting skills, and ECE has been shown to contribute to increased income over the short (for parents) and longer-term (for children) as well as a reduced use and reliance on public services (e.g., social services, welfare, justice-related, among others). Further, research sponsored by the World Bank, which has taken a lead role in Early Childhood Care and Development (ECCD) states that 'Policymakers must identify the educational investments that yield the highest public returns. Here, the literature is clear — Dollars invested in ECCD yield extraordinary public returns'. [Note 69]

It is important to note that there is no single standardised early education or care model. It could be integrated with other ECCD services; it could take place in a variety of settings, including schools or child care centres; and delivered by any number of types of agencies. It is also important to account for the fact that the benefits are commensurate with quality and time spent in high quality ECE. Factors commonly associated with high-quality ECE in the research have included pedagogical quality and sustainability, educational governance, cultural relevance and ownership/participation by families and communities.

The majority of the best practice ECCD systems identified by the Organisation for Economic Co-operations and Development are controlled or led by ministries of Education and are well integrated with other key sectors, including health and social services. While Canada has not been considered a leader or 'best case' in this area in part because of its lack of a cohesive policy, fractured delivery systems and uneven participation of the education sector, provincial/territorial ministers of Education are now assuming a greater role in ECE across Canada and in support of Aboriginal people. The Council of Ministers of Education, Canada and National Aboriginal Organizations have included ECE in a recent joint action plan and the subject is also included in the recently released National Strategy for Inuit Education.

Adult Education

Given the state of education results among Aboriginal people, particularly on reserve, there is a need to explore the potential role of adult learning in facilitating better outcomes in the long‑term; particularly as current generations of working-aged individuals without a high school education or higher would be more likely to pursue further education with the proper supports. Thus, there may be the opportunity to facilitate work and earning potential and address labour market gaps in the short-term. While empirical research on adult learning initiatives is scant with respect to measuring actual outcomes, particularly long-term success and earning potential, there are some practices in Canada and abroad that seem promising.

There is strong evidence to suggest that options with respect to supporting First Nation adult learners in pursuing their education should be examined. In doing so, it is important to explore the potential role of the Government of Canada, pursue the recognition and accreditation of Aboriginal institutions of higher learning, and identify the potential role of adult learning in emerging education reform proposals.

4.4.3 Special Education Programming

Finding: Many First Nation schools and communities are not adequately resourced to provide proper assessments and services to meet the needs of First Nation students with special needs.

Data from Nominal Roll suggest that students under SEP are not more likely to drop out in any given year than their counterparts (roughly five percent of enrolled students per year). In fact, in some cases, the drop-out rates are actually lower, and the age at dropout is slightly higher (18.5 years old). The graduation rate, however, is much lower at roughly 20 percent and this has not changed between 2006 and 2010.

This was an issue frequently raised among survey and case study respondents. Specifically, there were three main concerns: 1) that schools are inadequately resourced to provide students with special needs with required supports; 2) that the assessment criteria for eligibility is variable between provincial school boards and AANDC; and 3) there are inadequate resources in the community to provide adequate special needs assessments.

Concerns regarding the capacity of schools to meet the needs of special education students were pervasive, and largely centred on the availability of human resources to provide accurate assessments, reliability of mainstream assessments, and a lack of consistency of eligibility and services between districts. For example, participants on several occasions noted that students that are ineligible for High-Cost Special Education (HCSE) support through AANDC criteria on reserve would be qualified in the adjacent school district. First Nation participants largely expressed frustration with the lack of ability to attract and retain qualified specialists to their communities due to inadequate salary dollars. Where possible, communities have come to agreements to share resources for such specialists, but of course this is not always possible.

The sites visited for case studies specifically focussing on SEP were models of success for promoting the success of students and meeting their needs within extremely limited resources. Participants illustrated how success among special education students was strongly related to the school's determination to work with them, to never give up on them, to provide a supportive and safe environment that accounts for behavioural and cognitive needs and variability, and to provide customised learning plans with ambitious but realistic goal-setting with active involvement of the students. Early identification of students with special needs was facilitated by working with local ECE programs and through putting focussed resources into primary classrooms. The incorporation of cultural teachings was also seen as key to success. From the small groups of administrators, teachers and students involved in the case study, there was a sense that attitudes were very positive, and that most students had aspirations, including post‑secondary studies and labour market participation. Similar to other groups, however, the most significant barrier noted was the involvement and encouragement of the family and educators.

There was generally positive feedback regarding the rationale behind individual learning plans, but participants were critical of AANDC's general programming models, noting that funding is unpredictable and inconsistent. They suggested that while they feel their regional AANDC colleagues generally understand the realities of education for students requiring HCSE, they have limited means to influence the decisions and budgets allocated by AANDC HQ. In Quebec, there were several positive references made to the work of FNEC in assisting with special education programming.






5. Evaluation Findings – Efficiency and Economy

Is the current approach to programming the most economic and efficient means of achieving the intended objectives?

Finding: The current approach to programming may not be the most efficient and economic means of achieving the intended objectives of ESE programming.

As discussed in Section 4.4.2, there are key considerations (for example, second- and third-level services) in future policy development that could change the general structure of program and service delivery insofar as minimising the administrative overhead caused by a lack of economy of scale. Key informants did not make any general suggestions for improving economy or efficiency within the departmental operations itself, however, there was widespread acknowledgement that while there is a consensus that more strategic resourcing is necessary, there is also room for reducing overhead costs with some of the approaches discussed above. Additionally, applying a comprehensive methodology to the allocation of funds as discussed in Section 4.4.1, including accountability measures to ensure schools have operating funds that are equitable to provincial counterparts, is essential to better ensuring the ability of schools to provide quality instruction and learning environments.

While there is no research on the potential of these measures to result in cost savings, it is clear that there will be a significant return on investment in a scenario of improving outcomes into the future. For example, as illustrated in a study by Stewart Clatworthy [Note 70], the potential impacts on improving educational outcomes for registered Indians could be significant for both educational attainment levels and employment rates. For instance, should the educational attainment gap between non-Aboriginal and registered Indians be reduced by one-half, the proportion of the population with a high school diploma or higher levels of education would increase from the 2001 level of 48.6 percent to 61.7 percent in 2026. Likewise, the proportion of the population with a university degree would also increase from the 2001 level of 3.9 percent to 10.2 percent in 2026. It was further estimated that this could lead to employment of an additional 29,300 registered Indians in 2026, or a 16.6 percent incremental impact over the 2001 employment level. Obvious benefits under such scenarios include increases in employment income, reductions in federal government transfer payments, increased tax revenues, significant growth to gross domestic product, and reduced costs related to criminal activity and reduced incarceration rates.

Thus, cost-saving measures designed simply to reduce short-term expenditures may not necessarily net cost savings in the long-term. It is necessary to examine future expenditures in the context not only of improved individual and community well-being for First Nations people, but also of long-term reductions in other expenditures and significant potential benefit to the broad Canadian economy and local economies on reserve.






6. Conclusions and Recommendations

6.1 Conclusions

Results over the past two decades have not been encouraging with respect to student success, and all information included in this evaluation strongly suggests a need for significant revision to AANDC's approach to ESE programming. While there is no doubt that the Government has an integral role to play in ensuring ESE programming is functional and is providing First Nation students with learning opportunities and environments equitable to their counterparts off reserve, there is also a clear need to better ensure programming and delivery are suited to First Nation learner needs and optimal school operations on reserve.

The general role of the Government of Canada in ESE programming is generally appropriate; however, policy changes with respect to service delivery and local operational control may have implications on this role in the future. As demonstrated in the evidence presented above, there is a strong need for the Department to consider significant revisions to its approach to ESE program delivery to address key deficiencies in performance. To that end, this report has observed the following:

Relevance

  1. There is a need for continued investment in the Authorities for Elementary/Secondary Education stemming from projected population growth and from the need for significant improvement in student outcomes.
  2. Education authority activities are generally aligned with Government of Canada priorities; however, recent major reforms are reflective of the need to better align activities and better ensure improvements in student success.
  3. The priorities as stated by First Nation Education Authorities and those of AANDC are aligned insofar as the need to address marked gaps in educational opportunities and success. However, First Nation participants emphasise key priorities in the areas of cultural and language retention as being critical to success, and emphasise the need to recognise key differences in learning needs and the current state of education gaps, rather than simple notions of comparability.
  4. The role of the Government of Canada in ESE programming is generally appropriate; however, policy changes with respect to service delivery and local operational control may have implications on this role in the future.

    Performance
  5. The intended outcome of education opportunities and results that are comparable to the Canadian population is not being achieved.
  6. Continued work is needed to better facilitate constructive engagement and collaborative networks between First Nation education authorities, and where appropriate, with provincial governments or other organisations, and there is evidence that AANDC programming is improving in this regard.
  7. Student success is associated with parental engagement, the level of education in the community and the strength of the local economy. There are deeper issues related to the historical trauma of residential schools that may be interrelated with these factors.
  8. Community governance, the quality of teacher instruction and the quality of school curriculum were suggested as key factors affecting student success.
  9. Expenditures to First Nations and tribal councils for the operation of schools do not appear to account for actual cost variability applicable to the needs and circumstances of each community or school, and particularly the cost realities associated with isolation and small population. There is a need for a more strategic understanding of resource needs and allocation methods.
  10. Jurisdictional issues, implications for the provision of second- and higher-level services, and early childhood and adult education are considerations of significant importance in future policy development.
  11. Many First Nation schools and communities are not adequately resourced to provide proper assessments and services to meet the needs of First Nation students with special needs.

    Efficiency and Economy
  12. The current approach to programming may not be the most efficient and economic means of achieving the intended objectives of ESE programming.

6.2 Recommendations

It is recommended that AANDC:

  1. Develop a strategic and transparent framework for the investment of new funds that are explicitly allocated to facilitate improvements in student success in the short-term;
  2. Undertake further research into funding allocation methodologies that are equitable to provincial approaches, while at the same time accounting for cost realities on reserve;
  3. Ensure that future policy and program exercises develop clearly defined roles, responsibilities, and accountabilities for Elementary and Secondary Education;
  4. Explore and pursue options for the comprehensive development of second and higher-level services where possible and appropriate to reduce administrative burden and overhead costs, while supporting First Nations in developing long-term capacity for service management;
  5. Work with First Nations to develop strategies to strengthen culture and language retention as it relates to better student outcomes;
  6. Examine the implications of integrating support for early childhood education in AANDC's education portfolio;
  7. Examine the implications of integrating support for adult education in AANDC's education portfolio; and
  8. Develop a strategy to work with First Nations in building the capacity to strengthen the provision of special needs assessments and services.





Appendix A – Survey Guides

Question # Question Possible Responses 7-point Likert Scales
S1 In your capacity, would you say you are involved in elementary and secondary education (junior kindergarten to Grade 12), or post-secondary education, or both? Elementary/Secondary
[WHEN SPLIT QUESTIONS, ASK ONLY "ES" VERSION]
 
Post-secondary
[WHEN SPLIT QUESTIONS, ASK ONLY "PS" VERSION]
 
Both
[WHEN SPLIT QUESTIONS, ASK BOTH "ES" and "PS" VERSIONS]
 
Neither [THANK AND TERMINATE]  
S2 Which of the following positions do you hold within your band? Chief  
Councilor  
Education director/manager/co-ordinator  
Other (PLEASE SPECIFY)  
S3 How many years have you been in your current position? [NUMERIC DROP DOWN]  
S4 And for how many years have you held any type of education-related position in your community? [NUMERIC DROP DOWN]  
Q1ESa Please rate the quality of learning children receive in elementary and secondary schools in your community (in on-reserve schools, if applicable). Very poor 1
Excellent 7
Dk/ref 9
Q2ESa Do you wish to elaborate on your response? OPEN END  
No further comment  
Q1ESb Please rate the quality of learning children living in your community receive in elementary and secondary schools off-reserve (if applicable). Very poor 1
Excellent 7
Dk/ref 9
Q2ESb Do you wish to elaborate on your response? OPEN END  
No further comment  
Q3 Overall, how well or poorly do you feel that the schools in your community equip young people to:    
a Progress appropriately through Elementary school, i.e. to complete elementary (grade 6) by 11-12 years of age? Very poorly 1
Very well 7
Dk/ref 9
b Progress appropriately through Secondary school, i.e. to complete high school by 17-19 years of age? Very poorly 1
Very well 7
Dk/ref 9
c Progress from Secondary to Post-secondary school, i.e., to have students who graduate high school prepared for post-secondary studies? Very poorly 1
Very well 7
Dk/ref 9
d Proceed into the workforce after school, i.e., for students who want to proceed directly to the workforce after high school? Very poorly 1
Very well 7
Dk/ref 9
Q3a Do you wish to elaborate on your responses? OPEN END  
No further comment  
Q4ES What are the most important challenges First Nations people face in obtaining a good education? OPEN END  
Q4PS What are the most important challenges facing First Nations students in obtaining a post-secondary education? OPEN END  
Q5ES Generally speaking, have you seen an improvement or decline in student success in elementary/secondary in your community over the past 10 years? Strong decline 1
No change 4
Strong improvement 7
Q5aES Do you wish to elaborate on your response? [OPEN END] OPEN END  
No further comment  
Q5PS Generally speaking, have you seen an improvement or decline in the success in post-secondary students in your community over the past 10 years? Strong decline 1
No change 4
Strong improvement 7
Q5aPS Do you wish to elaborate on your response? [OPEN END] OPEN END  
No further comment  
Q6PS Which of the following do you see as significant barriers to students from your community in attaining a post-secondary education? (Please rank each on a scale of 1 to 7, with 1 meaning rarely a barrier and 7 being a significant barrier for most students) Access to funding for tuition 1-7
Affordability of living costs 1-7
Exposure to a different culture away from their home community 1-7
Finding suitable residence 1-7
Finding suitable child care for students with children 1-7
Other barriers : PLEASE SPECIFY  
Q6aPS Do you wish to elaborate on your response?
[PROVIDE THIS OPTION FOR EACH RESPONSE ABOVE]
OPEN END  
No further comment  
Q7.  Do you agree or disagree that the following represent the right priorities for the education of First Nations students? Please use a 7-point scale where 1 means you completely disagree with the priority and 7 means you completely agree with the priority.    
a Having access to education that is comparable to that of other Canadians Strongly disagree 1
Neither agree nor disagree 4
Strongly agree 7
Dk/ref 9
b Having similar graduation rates to other students in the province Strongly disagree 1
Neither agree nor disagree 4
Strongly agree 7
Dk/ref 9
c Having similar math skills (numeracy skills) to other students in the same grade in the province Strongly disagree 1
Neither agree nor disagree 4
Strongly agree 7
Dk/ref 9
d Having similar literacy levels (reading and writing skills) to other students in the same grade in the province Strongly disagree 1
Neither agree nor disagree 4
Strongly agree 7
Dk/ref 9
e Having similar enrolment rates in post-secondary institutions as other Canadian students Strongly disagree 1
Neither agree nor disagree 4
Strongly agree 7
Dk/ref 9
Q8. Are there other priorities that should be considered? [OPEN END] OPEN END  
No further comment  
Q9ES. Generally speaking, do you feel the quality of teacher instruction is satisfactory, or does it need improvement? Please respond on a scale of 1 to 7 with 1 meaning "needs significant improvement", and 7 meaning "very satisfactory" Needs significant improvement 1
Generally acceptable, but there's room for improvement 4
Very satisfactory 7
Dk/ref 9
Q9ESa Why do you say that? OPEN END  
No further comment  
Q10 Do you feel the governance in your community has an impact on the effectiveness of educating people in your community? Please respond on a scale of 1 to 7, with 1 meaning "a negative impact" and 7 meaning "a positive impact." A negative impact 1
Little to no impact 4
A positive impact 7
DK/ref 9
Q10a Why do you say that? OPEN END  
No further comment
Q10b What changes are needed in your community's governance to better meet students' educational needs? OPEN END  
No further comment
Q11 Do you feel AANDC's (INAC's) approach to First Nation education programming has an impact on education in your community? Please respond on a scale of 1 to 7, with 1 meaning "a negative impact" and 7 meaning "a positive impact." A negative impact 1
Little to no impact 4
A positive impact 7
DK/ref 9
Q11a Why do you say that? OPEN END  
No further comment
Q11b What changes do you feel AANDC (INAC) needs to make to better meet students' educational needs? OPEN END  
No further comment
Q12. To what extent do you see the following as factors necessary to ensure student success?    
a To what extent is a quality school curriculum a factor necessary to ensure student success? Not at all a factor 1
Somewhat of a factor 4
A significant factor 7
Dk/ref 9
a-a Would you say the curriculum is satisfactory or in need of improvement in your community? In need of significant improvement 1
Generally satisfactory, but there's room for improvement 4
Very satisfactory 7
DK/ref 9
b To what extent is the Band or Tribal Council's support for education a factor necessary to ensure student success? Not at all a factor 1
Somewhat of a factor 4
A significant factor 7
Dk/ref 9
b-b Would you say your Band or Tribal Council's support for education is satisfactory or in need of improvement? In need of significant improvement 1
Generally satisfactory, but there's room for improvement 4
Very satisfactory 7
DK/ref 9
c To what extent is parental engagement in their children's education a factor necessary to ensure student success? Not at all a factor 1
Somewhat of a factor 4
A significant factor 7
Dk/ref 9
c-c Generally speaking, would you say parents in your community are adequately engaged in their children's education, or do they need to be more engaged Parents need to be much more engaged than they are 1
Many are engaged, but generally more parental engagement is needed 4
Parents are sufficiently engaged 7
DK/ref 9
d To what extent is the strength of the local economy a factor necessary to ensure student success? Not at all a factor 1
Somewhat of a factor 4
A significant factor 7
Dk/ref 9
d-d Would you say the state of your local economy is having an effect on student success? It is having a negative effect 1
It is having no effect 4
It is having a positive effect 7
DK/ref 9
e To what extent is the retention of native language and culture a factor necessary to ensure student success? Not at all a factor 1
Somewhat of a factor 4
A significant factor 7
Dk/ref 9
e-e Would you say students are retaining their native language and culture in your community? There is little to no retention of native language and culture among students in our community 1
There is some retention of native language and culture among students in our community 4
Students in our community mostly are retaining native culture and language 7
DK/ref 9
Q13. Are there any other factors that you feel are significant drivers of student success on reserve? YES: OPEN END  
No other factors  
Q14. Are there other factors, OUTSIDE of the school environment, that you see as detrimental to student success? YES: OPEN END  
No other factors  
Q14ES. Can you give some specific examples of what's working well in your community with respect to elementary/secondary education? OPEN END  
DK/ref  
Q15ES. Can you give some specific examples of what's NOT working well in your community with respect to elementary/secondary education? OPEN END  
DK/ref  
Q16ES What aspects of AANDC's approach to funding elementary/secondary programming are working well? OPEN END  
DK/ref  
Q17ES What aspects of AANDC's approach to funding elementary/secondary education are NOT working well? OPEN END  
DK/ref  
Q16PS What aspects of AANDC's approach to funding post-secondary programming are working well? OPEN END  
DK/ref  
Q17PS What aspects of AANDC's approach to funding post-secondary education are NOT working well? OPEN END  
DK/ref  
Q18ES. Do students in your community's schools have access to (check the most applicable response):    
a A library There are no libraries  
They have access to limited library resources  
They have access to good library resources  
Dk/ref  
b A computer lab There are no computer labs  
They have access to limited computer lab resources  
They have access to good computer lab resources  
Dk/ref  
c The Internet There is unreliable or no internet access  
There is limited internet access  
There is reliable internet access  
Dk/ref  
d Extra-curricular activities (such as music lessons, sports teams, etc) There are few or no extra-curricular activities  
There are some extra-curricular activities  
There are ample extra-curricular activities  
Dk/ref  
e A gymnasium There is no gymnasium  
There is limited gymnasium access  
There is adequate gymnasium access  
Dk/ref  
f Special education teachers There are no available special education teachers  
There is limited access to special education teachers  
There is adequate access to special education teachers  
Dk/ref  
g Elders There are no elders available/accessible  
There is limited availability of elders  
There is adequate availability of elders  
Dk/ref  
h Teaching assistants There are no teaching assistants  
There is limited access to teaching assistants  
There is adequate access to teaching assistants  
Dk/ref  
Q18ESa Do you wish to elaborate on your responses? [PROGRAMMING NOTE: OPEN END FOR EACH INDIVIDUAL QUESTION ABOVE]    
Q19 Are there any specific recommendations you would make to your band or tribal council that you feel would improve student success? OPEN END  
No further comment  
Q20 Are there any specific recommendations you would make to AANDC that you feel would improve student success? OPEN END  
No further comment  
Q21 In which region do you live? British Columbia  
Alberta  
Saskatchewan  
Manitoba  
Ontario  
Quebec  
New Brunswick  
Nova Scotia  
Prince Edward Island  
Newfoundland and Labrador  
Yukon  
North-West Territories  
Nunavut  
 







Appendix B – Key Informant Interview Guides

Some previous First Nation education evaluations have focused individually on the following topics:
  • First Nations SchoolNet Program (2009),
  • Evaluation of the Special Education Program (2007),
  • Cultural Education Centres Program (2005), and
  • Band-Operated and Federal Schools (2005).

The current evaluation is designed to examine the combined effect of AANDC's education programming on the education outcomes of First Nation students, while still providing insights on specific programming. The evaluation is intended to provide evidence on program relevance, program impacts, student success, as well as efficiency and economy and the design and implementation of education programming.


Department: ______________________________________________________

Position of Respondent: ______________________________________________

Length of time in Position: ____________________________________________

I would like to lead a discussion with you about the manner in which First Nation schools are funded, run, and controlled. To guide us, I want to discuss several different aspects of the AANDC-funded education system. These areas include:

  • Relevance – meeting the needs of the students.
  • Performance – benefits of the programs and services.
  • Efficiency and Economy – cost-effectiveness and administrative abilities.
  • Recommendations for Improvement.




Donna Cona Johnston Research Inc.,


The following questions provide a framework for our discussion, and not all questions will apply to each interview. Please offer any information or perspective that you feel would be of value.

Before we begin, I want to mention that this evaluation follows the Personal Information Protection and Electronic Documents Act, as well as the Privacy and Access to Information Acts. As such, we would like to ensure that you fully understand the purpose of this evaluation. If you do not, please feel free to ask me more questions. Further, your name will not be associated with any of the data collected, the data will be securely stored and all information will be disposed of after five years.

I would like to tape our session. We are taping for several reasons: 1) it allows me to hold a conversation with you as opposed to being focused on note-taking; 2) it will allow for an improved record of the information you are willing to share with me today, and 3) it provides for a richer and more comprehensive analysis and reporting of what we are able to record. Your name and identifying information will be removed from any written records created from your interview. A summary of your interview can be emailed to you, for your own use. If you so wish, you may review the notes and make corrections as you see fit and send those along back to us. The data you provide will be stored in a security protected system both on the computer and in hard copy. Do you have any questions before we begin?

  1. What needs are AANDC's education programming trying to address and have these needs changed over time? To what extent does AANDC's education programming address/align to these needs? Think about the programs that you're familiar with, including A) Cultural Education Centres Program, B) Special Education Program, C) School Net, D) Teacher Recruitment and Retention, E) Parental and Community Engagement, F) New Paths, G) Elementary and Secondary Education, H) Post Secondary Education, I)Funding off-reserve schools, J) Other AANDC programs…
  2. Is AANDC's education programming consistent with government priorities and AANDC strategic objectives?
  3. What role do you believe the federal government/AANDC should carry-out in First Nations education and what is the role for First Nations? What roles do you believe other stakeholders should play (e.g. provinces, FN organizations, private sector, universities)?
  4. What linkages exist between the various types of AANDC programs and services? What linkages exist between AANDC's education programming and other programming for first Nations (e.g. with other government departments, provinces and the private sector)? How are these linkages working to improve access to education and skills development for First Nation students and education staff? Is there overlap or duplication?
  5. When you consider the intended outcomes for First Nations students, what progress has been made in the past 5 to 10 years towards reaching these outcomes? Please see Appendix A for a list of the intended outcomes.
  6. To what extent do AANDC's administrative systems and operational practices make it possible for First Nation education authorities to deliver elementary/secondary programming in a cost-effective manner? What are some of the barriers? What are some of the efficiencies?
  7. Is the AANDC Education suite of programs the most economic and efficient means for reaching First Nations student success? Do the costs outweigh the current outcomes or vice versa?
  8. What are the factors (both internal and external) that have facilitated or hindered the achievement of outcomes (e.g. capacity issues, the economy, policy, governance, etc.)?
  9. What best practices are you aware of within First Nations and in other jurisdictions that meet First Nation student needs?
  10. How could AANDC's education programs be improved?
  11. Can you direct us to any additional reports or documents that we should be aware of to help us better understand the relevance, performance, efficiency and economy of education programming?
  12. Can we contact you if there are any additional questions or if we need any clarifications?
  13. Do you have any other comments you would like to make about education programming?


APPENDIX – Education Programs Intended Outcomes

Immediate Outcomes

  • First Nations students actively participate in elementary and secondary education
  • Learning environments with appropriate education programs and support services are in place
  • Supported First Nations and Inuit students participate in post-secondary education

Intermediate Outcomes

  • First Nations learners progress in their elementary and secondary education, including students with special education needs reaching their full potential
  • First Nations and Inuit post-secondary students progress in their program of study

Ultimate Outcome

  • First Nations and Inuit learners achieve levels of education comparable to other Canadians

Strategic Outcome

  • Individual, Family and Community Well-Being for First Nations and Inuit





Appendix C – Case Study Tools

AANDC Education Case Studies

School Principals, Managers / Directors of Education

Some previous First Nation education evaluations have focused individually on the following topics:
  • First Nations SchoolNet Program (2009),
  • Evaluation of the Special Education Program (2007),
  • Cultural Education Centres Program (2005), and
  • Band-Operated and Federal Schools (2005).

The current evaluation is designed to examine the combined effect of AANDC's education programming on the education outcomes of First Nation students, while still providing insights on specific programming. The evaluation is intended to provide evidence on program relevance, program impacts, student success, as well as efficiency and economy and the design and implementation of education programming.


Name of First Nation: ___________________________________________

Position / Type of Respondent: ____________________________________

Length of time in Position: _______________________________________

I would like to lead a discussion with you about local education programs and services. To guide us, I want to discuss several different aspects of the local education programs and services. These areas include:

  • Parental involvement (parents volunteer, parents respond to letters sent home)
  • Student involvement (volunteers, attend events, sit on committees)
  • Community involvement (volunteers, attend events, sit on committees)
  • Cultural inclusion (language course, culture in the curriculum)
  • First Nation directed and controlled education




Johnston Research Inc., Donna Cona


The following questions provide a framework for our discussion, and not all questions will apply to each interview. Please offer any information or perspective that you feel would be of value.

Before we begin, I want to mention that this evaluation follows the Personal Information Protection and Electronic Documents Act, as well as the Privacy and Access to Information Acts. As such, we would like to ensure that you fully understand the purpose of this evaluation. If you do not, please feel free to ask me more questions. Further, your name will not be associated with any of the data collected, the data will be securely stored and all information will be disposed of after five years.

I would like to tape our session. We are taping for several reasons: 1) it allows me to hold a conversation with you as opposed to being focused on note-taking; 2) it will allow for an improved record of the information you are willing to share with me today, and 3) it provides for a richer and more comprehensive analysis and reporting of what we are able to record. Your name and identifying information will be removed from any written records created from your interview. A summary of your interview can be emailed to you, for your own use. If you so wish, you may review the notes and make corrections as you see fit and send those along back to us. The data you provide will be stored in a security protected system both on the computer and in hard copy. Do you have any questions before we begin?

  1. What is your definition of educating children in preparation for their lives as adults? [refer to laminated Card A to help facilitate discussion] (probes: A. What are their educational needs as infants, children, teenagers, and adults? B. What is the Ideal means for addressing the full educational needs of infants, children, teenagers, and adults? C. What are your limits in understanding what is best; do you feel you could adequately comment on how to best meet the needs?)
  2. Please describe the current education system offered through your Band. Please comment on the ways in which each of these types of services are provided:
    1. How does the education authority work? (probes: Does it have strong relationships with the community, parents and students; Does it coordinate an advisory committee, Parent Council, or Student Council; does the authority's office have its own targets for success defined and action taken to meet those goals; How are new programs implemented; do teachers, principals, families, community leaders, or other community members take part in designing the way education is delivered and designed?)
    2. How does/do the school(s) work to meet the basic needs of those students in need? (probes: How many counsellors are employed and how do they work to meet student needs; Do you have child and youth workers and how do they help students; Are there after-school and weekend programs; are there other support programs (e.g., breakfast and/or lunch programs); How are other community service providers linked directly to students in-need of extra supports (e.g., Mentors, Leaders, Guardians, Police, Social Services, a network of providers that case conference).
    3. What kind of outreach and linkages does/do the school(s) do between student groups at school and between the school and the community? (probes: Does the school put on community feasts/gatherings, how often; Are socials held at the school for students or parents; How are parents involved in the day-to-day school environment (e.g., home-visiting, parent supports/counselling).
    4. What types of student data do you collect and how is it stored? (probes: How are student data analyzed; Are data linked back to student success indicators; Is there a system in place that enables you to know if student targets are met).
    5. How are students supported to reach high-levels of self-confidence and academic achievement? (probe: In what ways are tutoring services provided; Are any other programs that target building a belief system around academic achievement offered (e.g., counselling, mentoring programs, guidance from specially trained teachers, specific credit-based courses, scholarships/bursaries, and physical activity); Are there any school courses that are designed around ensuring these elements are met and are there systematic checks to ensure this is happening and they are effective; How are high school students linked to the adult world: such as college courses, volunteer program, work for pay program, summer job opportunities, and exchange programs).
    6. How does the school exemplify an environment with cultural understandings of all staff and students? (probe: Would you characterize the school(s) as having a welcoming environment that promotes and embraces culture and traditions; do teaching staff have opportunities for professional development and access to cultural curriculum and resources specific to some courses, and grades; How are language, culture, history, and cultural-relevance a part of the core subjects; Are Elders and Cultural Teachers brought into the school(s); Is there a Cultural room designated with resources).
  3. Now I'd like to show you three models and ask you to place your First Nation's education programs and services within one of the models or to map out the way you see it on a blank model or a blank page. [Interviewer: explain the models, Cards D-F, and give respondent a blank model and a blank page, Cards G/H, and a pen]
  4. How does the current AANDC funding and the various programs your Band receives funding under, support and limit this First Nations children's ability to have their full educational needs met? [ask respondent to complete Card C for the programs he/she is aware that are offered within this First Nation] (probes: A. What are the specific strengths and weaknesses of the way you receive funds? B. What are the strengths and weaknesses of what you can do with the AANDC funding you receive? C. What are the strengths and weaknesses of the current resources you have in-place to offer educational services (e.g., other federal/provincial government programs, or private sector, such as colleges/universities)? D. What are the additional needs of students beyond what AANDC provides funding for? E. What needs do students have that you feel are beyond the scope of what an educational system can offer?)
  5. To what extent do AANDC's administrative systems and operational practices make it possible for this First Nation to deliver elementary/secondary programming in a cost-effective manner? What are some of the barriers? What are some of the efficiencies?
  6. Are the current AANDC Education programs the most economic and efficient means for reaching First Nations student success? Do the costs outweigh the current outcomes or vice versa?
  7. How do the various authorities impede or enhance the programs and services this First Nations educational system offers to its membership? Specifically, please discuss each of: A. Band Council, B. regional management organization, C. AANDC region, D. AANDC headquarters, and E. other education authorities (such as the local off-reserve school board/school)?
  8. What current mechanisms are in-place that let you know how well student success is being achieved? (probes: A. How do you define student success? B. How are you measuring success rates? C. What are your current rates of success? D. Can you share (give) any data with me? E. What are your success targets for five years from now? F. What key factors will contribute to you improving student success over the next five and ten years?)
  9. In what ways have these [show Card B] achievements been recently reached among students from this First Nation?
  10. How does this First Nation as a whole, both support and limit student's ability to have their full educational needs met? (probes: A. What barriers, if any, exist that prevent different/alternative educational approaches from flourishing? B. What kind of support does the education department get from the community to support student success? C. In what ways do parents support and get involved in the school, why and why not? D. In what ways do students support and get involved in the school, why and why not?
  11. How does/do this First Nations school(s) work to both support and limit student ability to have their full educational needs met? [show Card A] Again, go back to the life cycle and the educational programs and services available for infants, children, teenagers, and adults. (probes: A. What are they doing right? B. What are they doing wrong? C. What areas could they easily change to improve student success? D. What areas are difficult to change that impact student success? E. What could change, but would take more work/money/support?)
  12. [if off-reserve school that this reserve's students attend] How does/do the current off-reserve school(s) education programs and services both support and limit First Nation student's ability to have their full educational needs met? [show Card A] Again, discuss the educational programs and services off-reserve that your band members attend for infants, children, teenagers, and adults. (probes: A. What are the programs and services doing right? B. What are they doing wrong? C. What areas could they easily change to improve First Nation student success? D. What areas are difficult to change that impact First Nation student success? E. What could change, but would take more work/money/support?)
  13. With regard to student success, think back ten years (what first hit your mind when you thought back to 2000), and five years ago (what comes to mind about the 2005 mark). How have the achievements of the students changed with regard to: [show Card B]
  14. Now let's talk about where this First Nations education goes from here? [Interviewer: show the models again, Cards D-F, and give respondent a blank model and a blank page, Cards G/H, and a pen], let's draw this First Nations Education Programs and Services as you think they need to grow in order to better attain optimal student success? Again, you could use a blank paper of the blank model to map it out. (probes: A. What are the needs of students in the future? [show Card A] B. What are the best practices of this future educational system? C. What are the new approaches as well as programs and services you would see being offered?)
  15. Do you have any final comments or thoughts you can share?


The Seven Stages of Life



AANDC Education Programs Intended Outcomes:

Immediate Outcomes

  1. First Nations students actively participate in elementary and secondary education
  2. Learning environments with appropriate education programs and support services are in place
  3. Supported First Nations and Inuit students participate in post-secondary education

    Intermediate Outcomes
  4. First Nations learners progress in their elementary and secondary education, including students with special education needs reaching their full potential
  5. First Nations and Inuit post-secondary students progress in their program of study

    Ultimate Outcome
  6. First Nations and Inuit learners achieve levels of education comparable to other Canadians

    Strategic Outcome
  7. Individual, Family and Community Well-Being for First Nations and Inuit

List from AANDC Education Outcome Targets

AANDC Education Programs for 2011 Evaluation:

  • instructional services for Band-Operated Schools;
  • Federal Schools and Provincial Schools;
  • Elementary and Secondary Student Support Services;
  • New Paths for Education;
  • Teacher Recruitment and Retention;
  • Parental and Community Engagement;
  • Special Education Program (SEP);
  • First Nations SchoolNet,
  • First Nation Student Success Program;
  • Education Partnerships Program;
  • Cultural Education Centres (CEC).
  • Post-Secondary Student Support Program (PSSSP);
  • University and College Entrance Preparation (UCEP); and
  • Indian Studies Support Program (ISSP).

List from AANDC Evaluation Branch for the 2011 Education Evaluation




Education Authority
With Minimal Activities

Education Authority With Minimal Activities

Models adapted from Urban Aboriginal Education Pilot Project: Provincial Evaluators Findings, FINAL REPORT, 03 March 2011, by Johnston Research Inc.

Text description of the Education Authority With Minimal Activities graphic is available on a separate page.



Education Authority
With Activities

Education Authority With Activities

Models adapted from Urban Aboriginal Education Pilot Project: Provincial Evaluators Findings, FINAL REPORT, 03 March 2011, by Johnston Research Inc.

Text description of the Education Authority With Activities graphic is available on a separate page.



Education Authority
With Many Activities

Education Authority With Many Activities

Models adapted from Urban Aboriginal Education Pilot Project: Provincial Evaluators Findings, FINAL REPORT, 03 March 2011, by Johnston Research Inc.

Text description of the Education Authority With Many Activities is available on a separate page.





School Programming and Data Tool

In the past, AANDC has evaluated its education programming one stream at a time. You may have heard about one or more of the following evaluations:
  • First Nations SchoolNet Program (2009),
  • Evaluation of the Special Education Program (2007),
  • Cultural Education Centres Program (2005), and
  • Band-Operated and Federal Schools (2005).

To-date there has been little to no analysis on how all programs combined affect students, families and communities.This evaluation focuses on the following aspects:
  • Elementary, secondary and post-secondary education.
  • Wholistic understanding of the community's perspective of education.
  • Comprehensive understanding of the education portfolio from a community perspective of meeting demand and needs.
  • Broader perspective on First Nation mainstream educational advancement.
  • Balancing a historical perspective to viewpoints about the keys to success.


Name of First Nation: ________________________________________________



Position / Type of Respondents Length in Position / First Nation
   
   
   
   
   
   

I need your assistance in completing this environmental assessment tool that will assess the complex systems within your community and your community's education system.





Johnston Research Inc., Donna Cona


Principals of each school should first be approached to complete this Tool; they may choose to appoint someone once they have reviewed what we are asking.

 

ENSURE THESE NAMES MATCH: Situational Perspective Tool

 

School A:  ___________________________________________________________________

2010/11 Enrolment in A: _________ ; if off-reserve 2010/11 add non-FN enrolled: _________

 

School B:  ___________________________________________________________________

2010/11 Enrolment in B: _________ ; if off-reserve 2010/11 add non-FN enrolled: _________

 

School C:  ___________________________________________________________________

2010/11 Enrolment in C: _________ ; if off-reserve 2010/11 add non-FN enrolled: _________

 

School D:  ___________________________________________________________________

2010/11 Enrolment in D: _________ ; if off-reserve 2010/11 add non-FN enrolled: _________

 

School E:  ___________________________________________________________________

2010/11 Enrolment in E: _________ ; if off-reserve 2010/11 add non-FN enrolled: _________

 

C
Current activity
L
Last Year
Prv
Previous to one-year ago
Pnf
Planned in near future
Plt
Planned in long-term
N
Not offered/No plans


Current Activity or Not
Topic School
A
School
B
School
C
School
D
School
E
Systems Perspective
Increasing Education Authority Commitment
Education Advisory Committee          
Parent Council          
Student Council          
Agenda to address student success          
Success Targets Defined          
Programs designed to promote success          
Meeting urgent needs of First Nation students
In-school Social Counsellors          
After-school Social Counsellors          
Weekend Social Counsellors          
After-school program          
School-related weekend activities          
Breakfast program          
Lunch program          
Social Mentors          
Student Mentors          
Leaders helping students          
Guardians helping students          
Police visit-the-school          
Case Conferencing          
Outreach, Student-Data and linkages among First Nation students
Community Feast          
Parent socials          
Parent and Student Socials          
Special student groups facilitated/ supervised by staff          
Community meetings          
Parent meetings          
Students meetings          
Parent volunteers          
Guest Parents Speak to/Teach class          
Guest Parents Speak to/Teach class          
Home visits          
Home visits          
Family counselling          
Other family supports          
Success Data collected          
Electronic Data storage / data entered          
Data analyzed          
Data used for decision making          
Data used for change / new programs          
Meeting needs for improved self-confidence and academic achievement by First Nation students
Tutoring          
Student Tutors          
Academic skill building Counselling          
Academic skill building  Mentors          
Academic  Guidance from Teachers          
Course for credit aimed at self-confidence          
Course for credit academic skill building          
Volunteer in work-places program          
Work for pay program          
Summer job program          
Work placement for credit          
Exchange program          
Traditional Perspective
Meeting the needs for cultural understandings of all staff and students
Welcoming school environment          
Cultural training offered to teachers          
Culture is integrated into the curriculum          
Students have opportunities to learn FN culture          
First Nation language credit-course offered          
First Nation culture credit-course offered          
First Nation history credit-course offered          
Core Subject teachers infuse culture into the classroom          
Elders and Cultural Teachers are brought into the school          
Cultural rooms are designated with resources          


Basic Descriptors of Key Activities

Descriptions
Topic Description Answer option School
A
School
B
School
C
School
D
School
E
Systems Perspective
Increasing Education Authority Commitment
Education Advisory Committee Formalized Yes/No          
Parent Council Formalized Yes/No          
Student Council Formalized Yes/No          
Agenda to address student success Formalized Yes/No          
Success Targets Defined Formalized Yes/No          
Programs designed to promote success Formalized Yes/No          
Meeting urgent needs of First Nation students
In-school Social Counsellors Certified Yes/No          
After-school Social Counsellors Certified Yes/No          
Weekend Social Counsellors Certified Yes/No          
After-school program Transportation Yes/No          
School-related weekend activities Transportation Yes/No          
Breakfast program Grades Gr. x-y
Gr. a-b
         
Lunch program Grades Gr. x-y
Gr. a-b
         
Social Mentors Grades Gr. x-y
Gr. a-b
         
Student Mentors Grades Gr. x-y
Gr. a-b
         
Leaders helping students Grades Gr. x-y
Gr. a-b
         
Guardians helping students Grades Gr. x-y
Gr. a-b
         
Police visit-the-school Grades Gr. x-y
Gr. a-b
         
Case Conferencing School initiated            
Outreach, Student-Data and linkages among First Nation students
Community Feast School-led Weekly
Monthly
> often
Annually
         
Parent socials Frequency Weekly
Monthly
> often
Annually
         
Parent and Student Socials Frequency Weekly
Monthly
> often
Annually
         
Special student groups facilitated/ supervised by staff Frequency Daily
Weekly
Monthly
> often
         
Community meetings Frequency Weekly
Monthly
> often
Annually
         
Parent meetings Frequency Weekly
Monthly
> often
Annually
         
Students meetings Frequency Weekly
Monthly
> often
Annually
         
Parent volunteers Frequency Daily
Weekly
Monthly
> often
         
Guest Parents Speak to/Teach class Frequency Daily
Weekly
Monthly
> often
         
Guest Parents Speak to/Teach class Grades Gr. x-y
Gr. a-b
         
Home visits Grades Gr. x-y
Gr. a-b
         
Home visits Frequency Weekly
Monthly
> often
Annually
         
Family counselling Grades Gr. x-y
Gr. a-b
         
Family counselling Frequency Daily
Weekly
Monthly
> often
         
Other family supports Grades Gr. x-y
Gr. a-b
         
Other family supports Frequency Daily
Weekly
Monthly
> often
         
Success Data collected Frequency Daily
Weekly
Monthly
> often
         
Electronic Data storage / data entered Frequency Daily
Weekly
Monthly
> often
         
Data analyzed Frequency Weekly
Monthly
> often
Annually
         
Data used for decision making Frequency Weekly
Monthly
> often
Annually
         
Data used for change / new programs Frequency Weekly
Monthly
> often
Annually
         
Meeting needs for improved self-confidence and academic achievement by First Nation students
Tutoring Grades Gr. x-y
Gr. a-b
         
Student Tutors Grades Gr. x-y
Gr. a-b
         
Academic skill building Counselling Frequency Weekly
Monthly
> often
Annually
         
Academic skill building  Mentors Formal Yes/No          
Academic  Guidance from Teachers Trained Yes/No          
Course for credit aimed at self-confidence Grades Gr. x-y
Gr. a-b
         
Course for credit academic skill building Grades Gr. x-y
Gr. a-b
         
Volunteer in work-places program Grades Gr. x-y
Gr. a-b
         
Work for pay program Grades Gr. x-y
Gr. a-b
         
Summer job program Grades Gr. x-y
Gr. a-b
         
Work placement for credit Grades Gr. x-y
Gr. a-b
         
Exchange program Grades Gr. x-y
Gr. a-b
         
Traditional Perspective
Meeting the needs for cultural understandings of all staff and students
Welcoming school environment Embraces Culture Yes/No          
Welcoming school environment Some First Nation art/displays Yes/No          
Cultural training offered to teachers Formal Yes/No          
Culture is integrated into the curriculum Formal Yes/No          
Students have opportunities to learn FN culture Formal Yes/No          
First Nation language credit-course offered Grades Gr. x-y
Gr. a-b
         
First Nation language credit-course offered Specify language(s)          
First Nation culture credit-course offered Grades Gr. x-y
Gr. a-b
         
First Nation culture credit-course offered Specify culture(s)          
First Nation history credit-course offered Grades Gr. x-y
Gr. a-b
         
Core Subject teachers infuse culture into the classroom Grades Gr. x-y
Gr. a-b
         
Core Subject teachers infuse culture into the classroom Formal Yes/No          
Elders and Cultural Teachers are brought into the school Formal Yes/No          
Elders and Cultural Teachers are brought into the school Grades Gr. x-y
Gr. a-b
         
Cultural rooms are designated with resources Formal Yes/No          
Cultural rooms are designated with resources Grades Gr. x-y
Gr. a-b
         


1
Very Valuable
2
Valuable
3
Somewhat Valuable
4
Not Valuable
5
No opinion


Value of Activity for First Nation Student Success
Topic School
A
School
B
School
C
School
D
School
E
Systems Perspective
Increasing Education Authority Commitment
Education Advisory Committee          
Parent Council          
Student Council          
Agenda to address student success          
Success Targets Defined          
Programs designed to promote success          
Meeting urgent needs of First Nation students
In-school Social Counsellors          
After-school Social Counsellors          
Weekend Social Counsellors          
After-school program          
School-related weekend activities          
Breakfast program          
Lunch program          
Social Mentors          
Student Mentors          
Leaders helping students          
Guardians helping students          
Police visit-the-school          
Case Conferencing          
Outreach, Student-Data and linkages among First Nation students
Community Feast          
Parent socials          
Parent and Student Socials          
Special student groups facilitated/ supervised by staff          
Community meetings          
Parent meetings          
Students meetings          
Parent volunteers          
Guest Parents Speak to/Teach class          
Guest Parents Speak to/Teach class          
Home visits          
Home visits          
Family counselling          
Other family supports          
Success Data collected          
Electronic Data storage / data entered          
Data analyzed          
Data used for decision making          
Data used for change / new programs          
Meeting needs for improved self-confidence and academic achievement by First Nation students
Tutoring          
Student Tutors          
Academic skill building Counselling          
Academic skill building  Mentors          
Academic  Guidance from Teachers          
Course for credit aimed at self-confidence          
Course for credit academic skill building          
Volunteer in work-places program          
Work for pay program          
Summer job program          
Work placement for credit          
Exchange program          
Traditional Perspective
Meeting the needs for cultural understandings of all staff and students
Welcoming school environment          
Cultural training offered to teachers          
Culture is integrated into the curriculum          
Students have opportunities to learn FN culture          
First Nation language credit-course offered          
First Nation culture credit-course offered          
First Nation history credit-course offered          
Core Subject teachers infuse culture into the classroom          
Elders and Cultural Teachers are brought into the school          
Cultural rooms are designated with resources          


Students, aged 16 and Over who are Involved at School

Some previous First Nation education evaluations have focused individually on the following topics:
  • First Nations SchoolNet Program (2009),
  • Evaluation of the Special Education Program (2007),
  • Cultural Education Centres Program (2005), and
  • Band-Operated and Federal Schools (2005).

The current evaluation is designed to examine the combined effect of AANDC's education programming on the education outcomes of First Nation students, while still providing insights on specific programming. The evaluation is intended to provide evidence on program relevance, program impacts, student success, as well as efficiency and economy and the design and implementation of education programming.

Name of First Nation: ___________________________________________

Location of School and Grade: ____________________________________

Length of time in that school: ____________________________________



I would like to lead a discussion with you about local education programs and services. To guide us, I want to discuss several different aspects of the local education programs and services. These areas include:

  • Parental involvement (parents volunteer, parents respond to letters sent home)
  • Student involvement (volunteers, attend events, sit on committees)
  • Community involvement (volunteers, attend events, sit on committees)
  • Cultural inclusion (language course, culture in the curriculum)
  • First Nation directed and controlled education

The following questions provide a framework for our discussion, and not all questions will apply to each interview. Please offer any information or perspective that you feel would be of value.

Before we begin, I want to mention that this evaluation follows the Personal Information Protection and Electronic Documents Act, as well as the Privacy and Access to Information Acts. As such, we would like to ensure that you fully understand the purpose of this evaluation. If you do not, please feel free to ask me more questions. Further, your name will not be associated with any of the data collected, the data will be securely stored and all information will be disposed of after five years.

I would like to tape our session. We are taping for several reasons: 1) it allows me to hold a conversation with you as opposed to being focused on note-taking; 2) it will allow for an improved record of the information you are willing to share with me today, and 3) it provides for a richer and more comprehensive analysis and reporting of what we are able to record. Your name and identifying information will be removed from any written records created from your interview. A summary of your interview can be emailed to you, for your own use. If you so wish, you may review the notes and make corrections as you see fit and send those along back to us. The data you provide will be stored in a security protected system both on the computer and in hard copy. Do you have any questions before we begin?

  1. How should children and youth be educated in preparation for their lives as adults? [refer to laminated Card A to help facilitate discussion] (probes: A. What are their educational needs as infants, children, teenagers, and adults? B. What is the Ideal means for addressing the full educational needs of infants, children, teenagers, and adults? C. What are your limits in understanding what is best; do you feel you could adequately comment on how to best meet the needs?)
  2. Where do you go to school (on- or off-reserve)? How well is your school performing, how well are students doing in your school? (probes: A. How do you define a successful student? B. What are the challenges for students in reaching that success? C. What is working well for students?)
  3. How well would you say students within this First Nation are doing with reaching these [show Card B] achievements, most recently?
  4. How does this First Nation as a whole, both support and limit student's ability to have their full educational needs met? (probes: A. What barriers, if any, exist that prevent different/alternative educational approaches from flourishing? B. What kind of support does the education department get from the community to support student success? C. In what ways do parents support and get involved in the school, why and why not? D. In what ways do students support and get involved in the school, why and why not?
  5. How does the school you attend work to both support and limit First Nation students' ability to have their full educational needs met? [show Card A] Again, go back to the life cycle and the educational programs and services available for infants, children, teenagers, and adults. (probes: A. What are they doing right? B. What are they doing wrong? C. What areas could they easily change to improve student success? D. What areas are difficult to change that impact student success? E. What could change, but would take more work/money/support?)
  6. Now let's talk about where this First Nations education goes from here? [show Card A] Again, discuss the changes that will best help this First Nations' people do well in school as infants, children, teenagers, and adults? What else should stay that same – what is just right, now?
  7. Do you have any final comments or thoughts you can share?


The Seven Stages of Life



Immediate Outcomes

  1. First Nations students actively participate in elementary and secondary education
  2. Learning environments with appropriate education programs and support services are in place
  3. Supported First Nations and Inuit students participate in post-secondary education

    Intermediate Outcomes
  4. First Nations learners progress in their elementary and secondary education, including students with special education needs reaching their full potential
  5. First Nations and Inuit post-secondary students progress in their program of study

    Ultimate Outcome
  6. First Nations and Inuit learners achieve levels of education comparable to other Canadians

    Strategic Outcome
  7. Individual, Family and Community Well-Being for First Nations and Inuit





Appendix D – ESE Components by Authority

Funding Arrangement CFA-CFNFA
Program Components by Funding Authority Grants Contributions BC FC SC
Elementary/Secondary Education Program: Instructional Services - Band Operated Schools
- Instructional Services Formula   X X X  
- Administration – Instructional Services   X X X  
- Enhanced Teachers Salaries (teacher recruitment and retention)   X      
- Low Cost Special Education   X X X  
- High Cost Special Education   X     X
- Band Operated – Evaluation   X X X  
Elementary/Secondary Education Program: Instructional Services - Federal Schools
- Instructional Services Formula   X   X  
- Administration – Instructional Services   X      
- Enhanced Teachers Salaries   X      
- Devolution - Front end funding   X   X  
- Low Cost Special Education   X   X  
- High Cost Special Education   X     X
ESE Instructional Services - Provincial Schools
- Tuition Agreements   X X    
- Ancillary Services X X X X  
- Private Schools   X X    
- Low Cost Special Education   X X X  
- Accountability: School/Student Assessments   X      
- Accountability: Performance Measurements System   X      
- Accountability: School Success Plans   X      
- Collaborative Initiatives   X      
ESE - Student Support Services
- Student Accommodation Services (All school types)   X X X  
- Student Transportation Services (All school types) X X X X  
- Financial Assistance Allowances (All School types) X X X X  
- Guidance and counselling   X X X  
- Advice and Assistance, Provincial Schools   X X X  
- Comprehensive Education Support Services (Saskatchewan and BC)       X  
Teacher Recruitment and Retention
Teacher Recruitment and Retention   X      
Parental and Community Engagement Strategy
Parental and Community Engagement Strategy   X      
New Paths for Education
- School Board type services   X      
- Proposal driven projects   X      
Cultural Education Centres
Cultural Education Centres   X   X X
Inuit Cultural Education Centres X X   X X
First Nations SchoolNet
First Nations SchoolNet   X   X  







Footnotes:

  1. Authorities under Education also include contributions to the National Aboriginal Achievement Foundation ; however, this was evaluated separately in 2009 by Heritage Canada. (return to source paragraph)
  2. Effective 2011, SchoolNet funding was revised to New Paths with funding maintained, integrating the four proposal-based programming items into one. (return to source paragraph)
  3. The YES Skills Link Program was not covered under this evaluation as it is being evaluated under a Horizontal Evaluation led by Human Resources and Skills Development Canada. (return to source paragraph)
  4. Effective 2011, SchoolNet funding was revised to New Paths with funding maintained, integrating the four proposal-based programming items into one. (return to source paragraph)
  5. AANDC, 2011, Departmental Performance Report 2010-11, p.41. (return to source paragraph)
  6. Includes Elementary and Secondary Education Instructional Services, Supplementary Programs and Special Education. (return to source paragraph)
  7. Band Support Funding is estimated at 7.66 percent of the education-related services funding provided to First Nations. Band Employee Benefits for education related positions is estimated as 53.9 percent of total First Nation BEBs. (return to source paragraph)
  8. Such as the James Bay Northern Quebec Agreement, Mi'kmaq Education Authority and BC First Nation Education Authority. These agreements cover funding beyond K-12 Education. (return to source paragraph)
  9. For more information, Frequently Asked Questions - Funding Approaches. (return to source paragraph)
  10. TBS, 2008, Directive on Transfer Payments . (return to source paragraph)
  11. Guideline for Employees of the Government of Canada: Information Management (IM) Basics  (return to source paragraph)
  12. National Panel on First Nations Education, 2012, Nurturing the Learning Spirit of First Nation Students   (return to source paragraph)
  13. Treasury Board Secretariat, 2009, Guidelines for Employees of the Government of Canada: Information Management Basics . (return to source paragraph)
  14. AANDC, 2012, Supplementary Literature Review: Selected Governance Issues, Early Childhood Education, Adult Learning and Higher Education. (return to source paragraph)
  15. AANDC, 2007, Sustainable Development Strategy. (return to source paragraph)
  16. AANDC, 2006, Gender-Based Analysis Policy. (return to source paragraph)
  17. Formative Evaluation of the Elementary/Secondary Education Program On Reserve - Evaluation Findings - Relevance. (return to source paragraph)
  18. Formative Evaluation of the Elementary/Secondary Education Program On Reserve. (return to source paragraph)
  19. Legislative Summary of Bill C-3: Gender Equity in Indian Registration Act  (return to source paragraph)
  20. Legislative Summary of Bill C-3: Gender Equity in Indian Registration Act  (return to source paragraph)
  21. National Panel on First Nations Education, 2012, Nurturing the Learning Spirit of First Nation Students   (return to source paragraph)
  22. Office of the Auditor General. 2004. November Report of the Auditor General of Canada. Chapter 5: Indian and Northern Affairs Canada—Education Program and Post-Secondary Student Support. (return to source paragraph)
  23. A LOW-TAX PLAN FOR JOBS AND GROWTH   (return to source paragraph)
  24. Budget Plan 2012  (return to source paragraph)
  25. Taylor, D., & Wright, S. (2003). Do Aboriginal students benefit from education in their heritage language? Results from a ten-year program of research in Nunavik. Canadian Journal of Native Studies, 23(1), 1-24. (return to source paragraph)
  26. Goddard, J., & Shields, C. (1997). An ethnocultural comparison of empowerment in two districts: Learning from an American Indian and Canadian First Nation school district. Journal of American Indian Education, 36, 19-45. (return to source paragraph)
  27. Castagno, A., & Brayboy, B. (2011). Culturally responsive schooling for Indigenous youth: A review of the literature. Review of Educational Research, 78(4), 941 -993. (return to source paragraph)
  28. Demmert, W., & Towner, J. (2005). A Review of the Research Literature on the Influences of Culturally Based Education on the Academic Performance of Native American Students. Portland, OE: Northwest Regional Education Laboratory . Retrieved on March 27, 2011 (return to source paragraph)
  29. λ2 (5)= 721.37; p = 0.000 (return to source paragraph)
  30. Taylor, D., & Wright, S. (2003). Do Aboriginal students benefit from education in their heritage language? Results from a ten-year program of research in Nunavik. Canadian Journal of Native Studies, 23(1), 1-24. (return to source paragraph)
  31. Formative Evaluation of the Elementary/Secondary Education Program On Reserve - Evaluation Findings - Performance. (return to source paragraph)
  32. Defined as the proportion of students in a given year who are not enrolled in the subsequent year but do not graduate (return to source paragraph)
  33. Linear by linear association λ2 (1) = 6.886; p = 0.009. (return to source paragraph)
  34. Refers to a student in their final year (grade 11 for Quebec) having received a certificate at the end of that school year. (return to source paragraph)
  35. Ordinal λ2 (4) = 42.96; p = 0.000 (return to source paragraph)
  36. Ordinal λ2 (4) = 30.75; p = 0.000 (return to source paragraph)
  37. Pearson λ2 (1) = 19.13; p = 0.000 (return to source paragraph)
  38. Linear by linear association λ2 (1) = 37.2; p = 0.000 (return to source paragraph)
  39. In this case examining one group of students starting in grade 9 and examining progress over five years (return to source paragraph)
  40. Note that for this report, where statistical information is presented nationally, the general observation is consistent across regions unless otherwise specified. (return to source paragraph)
  41. Pearson R2 = - 0.189; F (1, 312) = 11.51; p = 0.001 (return to source paragraph)
  42. Model variables Income, Labour Force, Education, Housing: R2 Change = 0.389; F Change (4, 951) = 151.34; p = 0.000 (return to source paragraph)
  43. Model variables Income, Labour Force, Education, Housing: R2 Change = 0.083; F Change (1, 309) = 27.92; p = 0.000. (return to source paragraph)
  44. Linear F (1, 1768) = 221.22; p = 0.000 (return to source paragraph)
  45. As determined by Statistics Canada, Geo-zone is defined the distance a First Nations band would have to travel to reach a service centre, that is a place where community members can gain access to government services, banks and suppliers): Zone 1: within 50km; year-round access; Zone 2: within 50-350km; year round access; Zone 3: within over 350km; year-round access; and Zone 4: varying distances without year-round access. (return to source paragraph)
  46. Linear by linear association λ2 (1) = 113.06; p = 0.000 (return to source paragraph)
  47. Linear by linear association λ2 (1) = 527.09; p = 0.000. (return to source paragraph)
  48. Linear by linear association λ2 (1) = 120.40; p = 0.000. (return to source paragraph)
  49. Linear by linear association λ2 (1) = 320.35; p = 0.000. (return to source paragraph)
  50. Pearson r2 (1) = 0.369; p = 0.001 (return to source paragraph)
  51. Analysis was not completed for Newfoundland and Labrador as the number of FTEs was not obtained. (return to source paragraph)
  52. National Panel on First Nation Elementary and Secondary Education for Students On Reserve   (return to source paragraph)
  53. NB and NS are excluded for this portion of the analysis because of no provincial jurisdictions with fewer than 1000 FTEs. PE is excluded because of too few FTEs overall on reserve. (return to source paragraph)
  54. Pearson R2 (1, 62) = -0.200; p = 0.082 (return to source paragraph)
  55. Pearson R2 (1, 87) = -0.263; p = 0.007 (return to source paragraph)
  56. See also: McCue, various; MacPherson (1991); National Panel on First Nations K-12 Education (2012: 9); and Mendelson (2008). (return to source paragraph)
  57. Drawn from: INAC, Interim Report joint INAC/Treaty First Nations educational review, p.8 cited in Carr-Stewart and Steeves (2009: 8); OAG (2004); MacPherson (1991); McCue (2006); Mendelson (2008); Standing Senate Committee on Aboriginal Peoples (2011: 8, 20-30); Kirkness (1999); National Panel on First Nations K-12 Education (2012: 10-11), Morgan and Edzerza (1998), Nesbitt et al. (2004), Paquette and Fallon (2010), Restoule (2008), and the Standing Senate Committee on Aboriginal Affairs (2011: 9, Footnote 15, and Chapter 4). (return to source paragraph)
  58. Geographic concentrations have been noted to have supported First Nations and Inuit in northern Canada as well as the Métis in some areas of the Prairies. (return to source paragraph)
  59. Aboriginal Institutes' Consortium (2005); American Indian Higher Education Consortium (2009, 2001; 1999); Hill (n.d), Seymour and Woods (2008). (return to source paragraph)
  60. National Panel on First Nations K-12 Education (2012: 12-13); Standing Senate Committee on Aboriginal Peoples (2011: 11-13). (return to source paragraph)
  61. 'Piecemeal' was the term used by both the National Panel on First Nations K-12 Education and the Standing Senate Committee on Aboriginal Peoples to describe activities in this area. (return to source paragraph)
  62. This section draws primarily upon McCue (1998), Mendelson (2008), Morgan (2006), Restoule (2008), and the Standing Senate Committee on Aboriginal Peoples (2011). (return to source paragraph)
  63. Government of Canada (2012) and AANDC, (2012), Budget 2012 Highlights - Aboriginal and Northern Investments. (return to source paragraph)
  64. Refers to administrative functions associated with operating schools such as broad curriculum development, strategic vision, recruitment standards and practices, transportation, etc. (return to source paragraph)
  65. McKinsey et al. (2010). How the world's most improved school systems keep getting better, November 2010. (return to source paragraph)
  66. Ibid (return to source paragraph)
  67. Raham, H.(1998). Linking Evaluation and School Success: Building School Success through Accountability, Presented to the annual conference of La Société québecoise d'évaluation de programme (SQÉP) and l'Association pour le developpement de la mesure et de l'évaluation en education, Saint-Hyacinthe, Québec, October 30, 1998 (return to source paragraph)
  68. Maguire, P. (2003). District Practices and Student Achievement: Lessons from Alberta, SAEE Research Series16, Society for the Advancement of Excellence in Education, (Kelowna: Society for the Advancement of Excellence in Education). (return to source paragraph)
  69. Grunewald, R. and Arthur R. (2004). A Productive Investment: Early Child Development, in World Bank, Early Childhood Development From Measurement to Action: A Priority for Growth and Equity, (Washington: The World Bank). (Available in the AANDC library) (return to source paragraph)
  70. Clatworthy, S. "Some Estimates of Private and Social Benefits of Improving Educational Attainment among Registered Indian Youth and Young Adults," 322. (return to source paragraph)