This website will change as a result of the dissolution of Indigenous and Northern Affairs Canada. Consult the new Crown-Indigenous Relations and Northern Affairs Canada home page or the new Indigenous Services Canada home page.
This website will change as a result of the dissolution of Indigenous and Northern Affairs Canada. Consult the new Crown-Indigenous Relations and Northern Affairs Canada home page or the new Indigenous Services Canada home page.
Date: (March 2005)
PDF Version (357 Kb, 76 pages)
In 1998, the Government of Canada established the Urban Aboriginal Strategy (UAS) to work with partners to better address the serious socio-economic needs of Canada's urban Aboriginal population. It was part of Gathering Strength: Canada's Aboriginal Action Plan, a response to the Royal Commission on Aboriginal Peoples.
A strategy targeting urban Aboriginal communities was viewed as critical given demographic and socio-economic trends that were becoming apparent. In the 2001 Census of Canada, almost one million people identified themselves as Aboriginal (Inuit, Métis and First Nations - status and non-status - peoples). Approximately 50 percent of those live in urban centres, with one quarter of Aboriginal people living in just 10 Canadian cities: Winnipeg; Edmonton; Vancouver, Calgary, Toronto; Saskatoon; Regina; Ottawa-Gatineau; Montreal and Victoria.
Urban Aboriginal Youth: An Action Plan for Change,Footnote 1 a report of the Standing Senate Committee on Aboriginal Peoples tabled in October 2003, noted that "not only do Aboriginal people constitute a significant percentage of urban populations, especially in the western provinces, but on the whole they have higher rates of joblessness, less formal education, more contact with the justice system, and are in poorer health than their non-Aboriginal counterparts." It noted that these demographic indicators suggest that the well-being of Aboriginal people in cities has a direct impact on the wellbeing of the cities themselves, most especially in western Canada, where a substantial number of Aboriginal people reside.
The UAS seeks to enhance coordination, improve horizontal linkages and policy integration within the federal government and partner with other stakeholders to better address the needs of urban Aboriginal people. In its 2003 Budget, the Government of Canada (GOC) announced that it was allocating $25 million over three years through the Office of the Federal Interlocutor for Métis and Non-Status Indians (OFI) to support the UAS. The bulk of UAS funding is being used to support pilot projects in eight priority urban centres: Vancouver, Calgary, Edmonton, Saskatoon, Regina, Winnipeg, Toronto and Thunder Bay. The GOC is working with selected provincial and municipal governments and community members to plan, fund and help implement the projects, and has assigned UAS coordinators in each of those designated communities.
In late 2004 the federal government announced a further expansion of funding for the UAS to a total of $50 million, to enable the currently participating communities to carry their work forward and to allow several new communities to participate.
Implementation of the pilot project phase of the UAS is well underway, with communities planning their longer-term approaches to using the UAS as a vehicle for change, and working to make best use of the available project funding. In August, 2004 the Office of the Federal Interlocutor (OFI) (until recently at the Privy Council Office but now relocated within Indian and Northern Affairs Canada (INAC)) initiated a formative evaluation of the UAS pilot project program in order to assess early progress, ensure that performance measurement strategies and activities are on track, support a summative evaluation scheduled for 2007, and identify any improvements that should be made on the basis of the experience thus far.
The UAS Pilot Project Initiative was expanded through Budget 2004. This formative evaluation did not study additional initiatives undertaken through this expansion. See 2.6 for more details on the UAS renewal and expansion.
This is the final report of the UAS formative evaluation. It includes a description of the initiative and the identification of its objectives and associated activities; a section detailing the methods used for the conduct of the evaluation; a section providing detailed evaluation findings organized according to the major issue areas the study addressed; a set of conclusions about progress to date based on the findings; and, a set of recommendations to help guide future efforts under the UAS.
This section of the report provides a description of the objectives of the UAS, the delivery approach, activities being undertaken at the national and local levels, and resource allocations.
The ultimate objective of the UAS is to narrow the socio-economic gap (also often referred to as "closing the gap in life chances") between urban Aboriginal people and the mainstream population. The pilot project phase of the UAS continues earlier efforts in the eight designated communities, enhanced by project funding.
The objectives of the pilot project phase of the UAS are to:
It was recognized that these objectives would not be fully achievable in the 3-year funding period for the pilot projects, but that progress in each area was possible. It was believed that by 2006, the UAS Pilot Projects could work to achieve the following specific outcomes:
The logic model for the UASFootnote 2 identifies four major sets of activities.
Capacity building is one of the primary activities of the UAS. It is viewed as a process by which organizations increase their abilities to: set objectives and priorities, perform functions, solve problems and achieve solutions; and, understand and deal with urban Aboriginal needs in a broad, strategic context and a sustainable manner.
At the national level the UAS is working to help build community capacity to address urban Aboriginal issues by helping to build partnerships within the federal sphere and with provincial and municipal governments, by actively promoting awareness of urban Aboriginal issues within government and in other sectors, and by supporting, compiling and disseminating research. At the local level, UAS coordinators and staff are working to support Aboriginal community mechanisms to identify needs, set priorities and develop strategic approaches to address urban Aboriginal issues. In addition, they are working to establish better partnerships within government and between government and Aboriginal communities. Finally, they are providing support for the project funding component of the UAS, to help build service delivery capacity.
Horizontal co-ordination is the other primary activity of the UAS. The UAS Pilot Projects are seen as opportunities to test new and innovative ways for the federal government to work co-operatively in the regions and respond better to community needs. One way in which this is envisioned is via the horizontal terms and conditions of the Pilot Projects. A horizontal mechanism was set up that allows for co-ordination of federal efforts in urban areas, and that sets out when it would be appropriate for federal departments to implement these terms and conditions in the context of a UAS pilot project. The use of the UAS terms and conditions by other departments can be done where the UAS pilot project is: community-based; beyond the parameters of any one federal department (but could be carried out if two or more of the departments participate on the project); consistent with the mandate(s) of the departments and consistent with the federal UAS objectives
In addition to the use of the flexible UAS terms and conditions and the idea of a "single window" for funding of UAS projects, UAS coordinators have established local federal committees to share information and coordinate activities related to urban Aboriginal issues.
Under the 2003 Treasury Board Submission, the following departments have agreed to participate and use the UAS horizontal terms and conditions when they participate in a pilot project under the UAS: Western Economic Diversification, Human Resource Development Canada, Department of Justice, Health Canada, and Canadian Heritage. Partnerships continue to be developed with other federal departments and agencies with mandates relating to urban Aboriginal issues.
The Pilot Project phase includes funds available at the national level to conduct or to compile research into issues that affect urban Aboriginal people. The research is intended to assist federal officials and policy makers with future issues related to urban Aboriginal people.
The fourth important element of the UAS is to advance awareness and understanding of the urgency and nature of urban Aboriginal issues. This is being undertaken through conferences, publications, speeches and the use of a UAS website.
This phase of the UAS incorporated the availability of project funding in the eight participating communities. By working with selected provincial and municipal governments (and others), the Government of Canada intended to implement pilot projects to test innovative solutions to address local priorities that could not be undertaken through existing programs. In recognition that this initiative is dependent on the experimental nature of the pilot projects and that the UAS is meant to be largely community-driven, it was considered premature to identify the specific types of projects that would be undertaken in each of the eight UAS priority cities.
Early consultations did, however, provide some indications. For example, some pilot projects were seen as likely to focus on improving educational outcomes for urban Aboriginal children by increasing parental and community involvement in the delivery of the core curriculum in inner city schools; improving feelings of inclusiveness and cultural pride among Aboriginal students so as to increase their self-reported interest in the core curriculum; and ultimately aiming to remove barriers to high school completion by Aboriginal students. Another type of project that was being considered in some of the UAS sites was an innovative approach for funding community priorities by way of a collaborative granting process.
Other projects might focus on restoring vitality to disadvantaged neighbourhoods through improvements in community safety and crime prevention, housing, education, business community partnerships and supports and community service centres, developing transitional centres that provide Aboriginal people with comprehensive information and services for making an easy transition to major urban centres, including shelter for short periods of time. Mobile services may also be considered to communities from which Aboriginal people are moving from so that they have information before they arrive in the urban centre.Footnote 3
The Federal Interlocutor received a total resource allocation of $25 million from 2003-2006 to undertake the Pilot Project phase of the UAS. Of that total, $18M or $6M annually has been allocated to the participating cities for the funding of pilot projectsFootnote 4; this funding has been delivered through contribution agreements. In addition, regional offices of the lead federal departments in each province represented in the UAS have been allocated an amount to enhance federal horizontality and Aboriginal engagement and to conduct or compile research under the UAS. In British Columbia, Alberta and Manitoba the lead department is Western Economic Diversification (WD). In Saskatchewan and Ontario the lead department is Human Resources and Skills Development Canada (HRSDC).
|Purpose of Funding||2003-04||2004-05||2005-06|
|Pilot Projects Footnote 5||$6.0M||$6.0M||$6.0M|
|Federal engagement Footnote 7 Aboriginal capacity||$2.5M Footnote 6||$1.5M Footnote 6||$1.5M Footnote 6|
|Research Footnote 8||$300k||$250k||$250k|
As the lead for the UAS nationally, the OFI is responsible for the co-ordination and management of an inter-departmental committee structure that oversees the implementation of the UAS. The structure developed at the outset consisted of three groups: a National Committee of Federal Officials to provide strategic advice and guidance; an interdepartmental working group of Headquarters' officials to ensure policy and program co-ordination; and, regional committees of federal officials to coordinate on-the-ground implementation of the UAS from a federal perspective.
In each participating community, steering committees or other bodies have been formed, typically with membership from the Aboriginal community, the provincial and municipal governments and federal department representatives, to guide the implementation of the Strategy, set priorities and directions, make recommendations about the pilot projects to be funded, and otherwise oversee the implementation of the UAS within the terms and conditions of the Strategy.
Some consultations took place with provincial and municipal governments regarding the selection of the initial eight communities, and they were invited to recommend potential communities for participation under the 2004 expansion. The OFI made the final selection decisions based on those recommendations and on an assessment of statistics on the socio-economic circumstances of Aboriginal people in the communities and the communities' apparent readiness to take on a UAS initiative.
Horizontal initiatives require effective communications between all participants to ensure the expected outcomes are on track to be met. The inter-departmental committee structure was intended to serve this purpose and to ensure that all departmental stakeholders had access to the same information and opportunities to learn from one another. The committee structure supports the sharing and integration of common performance measurement data.
Budget 2004 delivered on the federal government's commitment to cities and their Aboriginal residents by doubling the government's investment in the UAS, from $25M to $50M, and by extending the UAS funding to four years (to 2006-07) so that current projects with promising results could be expanded and, in partnership with willing provincial and municipal governments, more communities could participate.
Expansion of the UAS provides the federal government with the opportunity to re-adjust its course slightly to clarify and strengthen the UAS Terms and Conditions, to have additional federal departments participate in UAS pilot projects (the Aboriginal Policing Directorate at Public Safety and Emergency Preparedness Canada and Industry Canada - Aboriginal Business Canada), and to consider potential federal roles in urban Aboriginal issues such as health, economic development, education, housing and labour market development.
A significant portion of the funding provided for the UAS in Budget 2004 will be allocated to increase funding to the current pilot project communities in Vancouver, Edmonton, Calgary, Saskatoon, Regina, Winnipeg, Thunder Bay, and Toronto. Other funding will go to develop the UAS in additional communities, which are being identified in consultation between the federal, provincial, and municipal governments. At present, four additional communities have been identified to receive pilot project funding. They are: Prince George, British Columbia; Lethbridge, Alberta; Prince Albert, Saskatchewan; and Thompson, Manitoba.
This section of the report describes the methods used to conduct the formative evaluation, and an assessment of the limitations to the quality of the findings.
The Terms of Reference for the project and the results of the first meeting with the project authority established seven areas of inquiry that the formative evaluation needed to address:
It was decided at the initial meeting that there had already been sufficient work done by OFI in reviewing the literature on urban Aboriginal issues, identifying the needs of urban Aboriginal communities, and establishing the rationale for, and relevance of, the UAS. It was decided that the project would not devote additional resources to this element. Instead, it was decided that the emphasis would be placed on issues associated with the implementation of the UAS: delivery models; activities; progress to date; what can be done better; whether different approaches work better in different circumstances. It was recognized that the study would be largely exploratory in nature, given the qualitative nature of the main evaluation issues and the lack of available data on outcomes to date.
A review of documents was undertaken at a number of points in the study, for a variety of purposes:
In order for the formative evaluation to have a sound analytic framework for assessing implementation progress, and to ensure that the information/data collection mechanisms are in place for a future summative evaluation, it was critical that there was agreement on a set of clearly defined outcomes and performance indicators for the Initiative. The outcomes need to reflect clearly the objectives of the UAS, and the performance indicators establish in a realistic way how progress (and later, success) will be measured.
Discussions at the first project meeting indicated that some time needed to be devoted to this aspect of the initiative. The evaluation included several steps in this regard:
The last item, the guide for the formative evaluation, reflects closely the original terms and conditions for the evaluation but also takes into account the decisions made at the early meetings about areas of emphasis, and the thinking that went into the development of clear and well-defined outcomes and indicators for the UAS as a whole. The table of issues, indicators and methods for this formative evaluation are presented in Appendix B, along with the similar table for the future summative evaluation of the UAS (which is a product of the formative evaluation).
Interviews were conducted with OFI managers and staff and officials from federal partner departments and agencies at a national level, to address issues related to the implementation of the initiative, the delivery model, horizontality and working relationships between the OFI and regional UAS coordinators and community leaders.
The interviews were open-ended in nature, based on an interview guide listing the issues and specific questions to be addressed. The interview guides (tailored to the individuals being interviewed but with a common core set of questions), were developed in collaboration with OFI and a team member from Consulting and Audit Canada.Footnote 9 Interviews were conducted with four OFI officials (including one who was at the Cities Secretariat at the time of the interview but has since returned to the OFI), and Headquarters officials at Human Resources and Skills Development Canada, the National Secretariat on Homelessness, and Western Economic Diversification.
The community case studies were the core work of the formative evaluation. They took place in all eight designated communities (Vancouver, Edmonton, Calgary, Regina, Saskatoon, Winnipeg, Thunder Bay and Toronto). They included the following elements:
|Community||# of Federal Government Interviews||# of Provincial and Municipal Government Interviews||# of Interviews with Aboriginal Community Members||Total # of Interviews|
The formative evaluation included an internet-based survey of Aboriginal organizations that are stakeholders in the UAS process in the participating communities, in order to expand the reach of the evaluation beyond what would be possible through interviews, and in order to address certain issues that were amenable to a more quantitative approach. The survey addressed issues including:
The survey was targeted to all Aboriginal organizations involved in any capacity in organizing around urban Aboriginal issues, service delivery, research, advocacy or political organization. The population of organizations was identified by the UAS coordinator in each participating city. Lists were provided to the evaluators based on criteria provided to the coordinators. In many cases the lists did not include e-mail addresses, so the evaluators contacted those organizations by telephone to obtain an e-mail address, or to arrange another mode of delivery. It was also found that many of the existing e-mail addresses were no longer in service, and these cases were followed up as well, to maximize the number of organizations receiving notice of the survey.
Organizations on the eight lists were sent an e-mail (or a fax or letter in some cases) that described the purpose of the survey and the evaluation, and directed them to the internet web-site where the survey was located. The contacts were directed to the executive director or president of the organizations, to be completed for the organization (as opposed to the individual respondent). It was left up to them to decide how to proceed internally with completion of the survey.
The survey itself provided some background information of a general nature about the UAS and some community-specific information to take into account local UAS implementation and local terminology being used. Footnote 10 The survey was designed to guide respondents easily from one question to the next, with drop-down boxes or choices that required only a click of the mouse on the appropriate response. It also allowed for respondents to add additional comments.
Ultimately, 402 Aboriginal organizations were informed directly about the survey and invited to go to the website to complete it Footnote 11. Reminders were sent twice in the course of the 3-week period of the survey. The table below provides a breakdown of the number of organizations identified in each community, and the number of organizations that responded to the survey. It shows that the response rate for the survey was about 29%. This was less than what was hoped for but did not present a major surprise to the evaluators for several reasons:
|Community||# of Organizations Informed of the Survey(N)||% of Total N||Responses (# and %)|
The formative evaluation was exploratory in most respects because the pilot project phase of the UAS was relatively new, and it was unknown how the eight communities had each decided to implement this phase, according to their own circumstances. The methods used focused to a large extent on key informant interviews, with supporting information from the review of documents and the survey of Aboriginal organizations.
In terms of the active participants in the UAS in each community, and in terms of coverage of the range of types of participants (federal lead departments, partner departments and agencies, provincial and municipal officials and representatives from the Aboriginal communities), the key informant interviews are considered to have provided a solid basis for the findings. With few exceptions interviews were conducted with all individuals identified by government and Aboriginal community leaders as being important to consult with. The interviews were lengthy (often up to two hours in length and rarely less than one hour), and covered all the key evaluation issues and other issues raised by respondents.
That being said, there are inherent limitations to a case study approach based primarily on key informant interviews. Interviews capture perceptions that can be influenced by many factors including pre-existing ideas or opinions, the kind of information that informants have been exposed to, and alliances that form during initiatives that help determine points of view. Interviews can also be used by respondents to promote an agenda. In short, interviews do not represent a reliable source of objective information about what has transpired. With careful planning, selection of respondents and crafting of interview guides, experienced interviewers can minimize the limiting effect of those factors. However, where circumstances permit and resources are available, it is preferable to have additional supportive sources of information.
In this case there were some documents available to support descriptive aspects of the interview findings. The fact that a large number of interviews were conducted with key informants in all eight communities allowed us to have confidence in findings where perspectives emerged consistently. The survey of Aboriginal organizations was conducted in an effort to expand the scope of the study to include community members who were not necessarily active in the UAS.
However, the survey itself presented limitations. Because of the methods used for establishing the population of organizations for the survey and the low response rate, it is clear that the results cannot be seen as representative of the views of the Aboriginal communities as a whole, in the eight cities. Even though response rates for internet-based surveys are often lower than they would be for mail or telephone surveys (reportedly often in the 20% range for large commercial surveys and in the 30-40% range for smaller, more targeted surveys), the fact remains that a response rate of at least 50% and preferably close to 60% is desirable in order to make inferences about the broader populations represented by respondents. However, the responses at a national level, and in cities where the response rate was relatively higher, offer some indication of community awareness of the UAS and community views on some other broad issues, given that the population the survey was provided to were identified as the most likely to be interested. Findings from the survey are included in the analysis only when they indicate a strong tendency across most communities.
Because of this limitation to the survey, the evaluation did not succeed in obtaining reliable information on the views of the Aboriginal communities as a whole, on a range of issues that proved important. These include the representativeness of the current steering committees, the view of the broader communities on how they could be represented effectively in a UAS process, views about priorities for the communities relative to priorities being established under the UAS, and views about the way the UAS has been implemented. There were good reasons for the methodological choices made. It was not considered feasible with the time frame and budget for the evaluation to undertake a more comprehensive approach to canvassing the broader Aboriginal communities. As well, it was considered premature, given that little was known at the planning stages about what issues might need to be raised within the broader communities.
The evaluation has identified findings related to these issues, but they are based primarily on the perceptions of key informants and to a limited extent the responses to certain survey questions.
This section of the report presents the findings from the research described above. The findings are organized around issue areas identified in the evaluation terms of reference and in the initial review process. In each issue area the source of the findings is identified and findings are presented. Although there are references to specific communities when they exemplify a certain trend or offer a relevant alternative from the norm, in general the findings are national in scope. Participating communities may be conducting their own evaluations to complement this national evaluation.
The findings are generalized when they represent a clear majority of the views of relevant informants and are supported, or at least not contradicted, by any available alternative lines of evidence. Where an alternative perspective was found in three or four communities or at least one-quarter of the relevant informants interviewed, this is reported. Also, where one or more communities presented a distinctly different finding from the norm, this is reported and the reasons are examined.
Finding: Members of the Aboriginal communities have been integral to the implementation of the UAS in the great majority of participating cities, primarily through steering committees that are setting priorities and making project funding decisions. However, the Aboriginal communities as a whole do not consider themselves necessarily represented by the existing steering committee members, and there are challenges to current member selection approaches from the broader communities and from Aboriginal political organizations.
Finding: Lead federal departments made initial efforts to engage the Aboriginal communities as broadly as possible, but since steering committee memberships were established there has been little communication with the broader communities, and this has effectively limited access to planning and decision-making. Calls for proposals for project funding have been advertised widely, but lack of communication has led to a wide perception of limited access to funding.
Findings in relation to these questions are based on interviews in all eight communities with the Aboriginal representatives sitting on UAS steering committees Footnote 12 or otherwise directly involved in the initiative at the local level, interviews with Aboriginal political leaders not involved directly in the UAS, interviews with federal, provincial and municipal government officials involved in the UAS, a review of relevant documents, and a survey of Aboriginal organizations in the eight communities.
The UAS was conceived as an Aboriginal community-driven initiative. The expectation was that Aboriginal communities would set priorities, determine how best to organize the long-term strategic planning inherent in the initiative, and make decisions about how to target expenditures, subject to UAS and other program terms and conditions.
Aboriginal representatives in seven of the eight communities have been actively involved in UAS planning and decision-making. With the exception of one community in which the planning process has not proceeded to date, Aboriginal members comprise at least 50% of steering committee membership and in most cases they have a strong voting power. In one of those communities the committee process was Aboriginal-based. However, the committee has recently been disbanded by HRSDC in Ontario because disagreements within the committee had proven extremely difficult to resolve, with the result that the committee was not able to put forward plans or project proposals that had broad support. The decision was made to disband the existing committee, establish clearer terms of reference and decision-making procedures, seek to broaden the base of Aboriginal membership to better represent the Toronto Aboriginal population, and establish a new committee under those new arrangements.
This issue of Aboriginal involvement is greatly complicated by the question of what constitutes representation or inclusion of the Aboriginal communities. The process for selecting representatives from the Aboriginal community has varied greatly. In general, community-wide meetings were held at the beginning of the pilot project phase of the UAS. Invitations were extended to all Aboriginal organizations that the local federal officials were aware of, and the meetings were advertised in the community in order to encourage as wide participation as possible. In some cases the meetings resulted in a vote for community representatives for a steering committee. In the majority of cases participants in the general meetings were asked if they would like to sit on a steering committee, and some lead members of the community stepped forward without being chosen through a formal process.
An additional element in the selection of representatives for the UAS was the role of the formal Aboriginal political organizations such as the provincial affiliates of the Métis National Council and the relevant Treaty organizations at the local level, and the Assembly of First Nations and their regional Chiefs, and the Métis National Council at the national and regional levels. Prior to the pilot project phase the major political organizations in each community had been engaged in a process of identifying Aboriginal community priorities and discussing longer-term strategies. However, in most communities that process did not progress to the extent that had been intended, in part because of differing views as to how to implement the UAS, and in part because the lack of dedicated resources for the initiative at the community level discouraged active and ongoing participation. It was noted in most interviews at the community level that there are a limited number of Aboriginal leaders in each community with the skills and experience to guide the UAS process, and that these leaders are already extremely busy with their full-time positions and other volunteer work. This means that participation in new initiatives can be difficult unless there is a clear benefit.
The political organizations, and in particular the Métis National Council through its provincial affiliates, have taken the position that they represent the Métis people in the participating communities, and should be given the authority and a proportion of the funds to implement the UAS on their behalf. This view is linked to the broader movement toward Aboriginal self-government. There is a concern among some members of the Aboriginal community that were interviewed, that the UAS could lead to the withdrawal of funds that are now provided to Aboriginal organizations in areas such as employment and training, in the interests of reviewing overall funding approaches. This is viewed by some as an intrusion by the federal government into areas that Aboriginal organizations are capable of planning and delivering themselves.
According to a large majority of the Aboriginal community members interviewed for the evaluation, this perspective is not widely shared among service providing agencies and among some local chapters of the political organizations. The common perspective reported was that urban Aboriginal populations are diverse and often are not affiliated with any Aboriginal political organizations. The service providers report being focused on providing better quality service to all Aboriginal people regardless of their origins, and do not see any political organization as representing Aboriginal interests in a broad way.
That being said, there is by no means a consensus that the existing UAS steering committees are representative of the communities, and there were a number of concerns raised by the people interviewed, and similar concerns expressed in the survey responses.
First, there is a perception in several communities that steering committee members were "handpicked" by federal officials, and that the government was driving the process more than it should be. Where Aboriginal representation is seen as resulting from selection by the lead federal department, Aboriginal community members who otherwise support the idea of the UAS are dissatisfied with this approach, and would prefer an Aboriginal community selection process, even if it requires more time to establish.
A second concern is that the terms of reference for the steering committees have typically been insufficiently developed, so community members are unclear about how the committees function, what the criteria are for membership, what provisions there are for turn-over of membership, and how decisions are made. Even where members of the committee were voted by the community at large, there is now dissatisfaction in the broader community (as reflected in interviews and the survey responses) because of this lack of clarity. In Toronto in particular, the lack of sufficient terms of reference has been a major factor in the disbanding of the committee. There does not appear to be a clear line of responsibility nationally for developing such terms and conditions and other related tools, and communities have undertaken to develop them to varying degrees.
Related to community perceptions of how they are being represented on the UAS is the issue of communications. It is acknowledged in the large majority of UAS communities that communication with the Aboriginal community at large has been lacking. Community awareness of the UAS was reported by Aboriginal respondents to be low, and this is corroborated by findings from the survey that while about 70% of respondents had heard of the UAS, about half had had no involvement at all, and those same organizations reported being dissatisfied with the amount of Aboriginal involvement in the initiative. Certainly a lack of communication is seen as impeding wider participation in the initiative. Community UAS leaders including federal and other government officials agreed that little emphasis had been placed on developing an ongoing feedback mechanism with the broader community or any systematic communication, and that this would have to be a high priority in the coming months.
Survey responses and interviews with community members outside the steering committee membership indicated a serious concern that UAS funds were being distributed primarily to the agencies represented on the steering committees, and that the member agencies constituted something of a closed club that excluded other agencies from a real opportunity to obtain project funding. This was expressed as a perception, rather than actual knowledge of how the funding process functions, and it may be that the lack of public communication and transparency has engendered this perception. Evaluators did not examine the awarding of project funds to determine the fairness or openness of the process. Where an open request for proposal (RFP) process has been used for funding allocation, the calls for proposals have been disseminated widely. As well, federal officials responsible for the UAS at the community level, and Aboriginal participants in the funding process, indicated that formal decision-making processes were established and that conflict of interest guidelines were used to avoid direct conflict, but they acknowledged that committee members might have an advantage by being more knowledgeable about the UAS and its priorities. As well, they noted that the individuals active on steering committees are typically from organizations that are active and well-established in the delivery of services to the community, so it would be expected that they would have an interest in applying for available funding.
Finding: Collaboration among all participating parties has been generally positive and constructive, and there is a reportedly high level of trust among Aboriginal participants in most communities that government participants understand the nature and extent of the problems in Aboriginal communities and are genuinely committed to a process for making improvements. However, relationships between the federal government and Aboriginal communities are still fragile due to:
Finding: Collaboration among federal departments and agencies is generally constructive and positive, but limited greatly by:
Finding: Federal government capacity is enhanced because of the development of UAS steering committees in most communities that provide a focus for setting priorities and developing approaches to address urban Aboriginal issues. Capacity to respond effectively to address those issues is still limited substantially by the above-noted factors.
Finding: Roles and responsibilities on a day-to-day basis under the UAS are clear. The lead department is responsible for implementation of the UAS. Other partner departments/agencies attend federal committee meetings or conference calls to share information and coordinate activities related to the UAS where possible, and may attend steering committee meetings. What is unclear to the partner departments/agencies is how they are intended to contribute in a more meaningful way in the longer term.
Findings in this area are based on a review of local UAS documents describing the structures and processes in place and any planning that has taken place, and on interviews with federal, provincial and municipal government officials, Aboriginal members of steering committees, and representatives of other Aboriginal organizations in the communities.
In most communities the participating government departments from all levels, and Aboriginal representatives, have developed collaborative and constructive working relationships. They meet regularly for steering committee meetings and less formally to deal with issues and projects that require attention. The great majority of Aboriginal members of steering committees report that the federal lead departments in particular, and government members in general, appear to understand the nature and extent of urban Aboriginal issues and be making a genuine effort to move the initiative forward. In addition, they report that the local government participants appear to be serious about allowing the Aboriginal community to lead the initiative (within the confines of the terms and conditions of the initiative, which requires an oversight role for the federal government in terms of expenditures and in the broad direction of the Strategy).
There is one UAS community, Toronto, in which the above findings do not apply. In that city there has been considerable acrimony between various Aboriginal groups and the lead federal officials, and among various Aboriginal participants. There appear to be as many versions of events in Toronto as there are people to describe the events, but it is clear that the UAS has not been successful to date in establishing positive working relationships between the federal government and the Aboriginal community or among the various Aboriginal interests that have come forward to work on the UAS. There is a considerable lack of trust that the federal government is genuinely interested in allowing the Aboriginal community to lead the Strategy in any meaningful way, and from the perspective of federal officials there has not been a real effort on the part of participating Aboriginal organizations to set aside agency interests and work collaboratively.
Aside from Toronto, the findings with regard to the development of partnerships and collaboration are encouraging. However, at this early stage of the pilot project phase these relationships are still very much works in progress, and are seen as somewhat fragile due to a number of factors.
As noted earlier there are organizations within the Aboriginal communities that do not share the vision of the UAS as a collaborative strategy between governments and Aboriginal communities and believe that federal funds to address Aboriginal issues should be devolved to representative Aboriginal political organizations. There continues to be pressure from these organizations to link the UAS to broader issues of Aboriginal self-government and to deliver it through the established political organizations. The federal government's position on this is that the federal government already has a number of programs and initiatives that flow directly through established Aboriginal political organizations. In the urban context, there are a large number of Aboriginal organizations serving the needs of Aboriginal residents, and representing to various degrees the diverse Aboriginal communities in the cities. Therefore, the UAS is designed to be community-based to allow the diversity of interests to work together collaboratively among themselves and with governments.
In some communities there are members of Aboriginal political organizations sitting on steering committees and these members are participating under the existing model, but the tensions between these two visions are reported to be detrimental to the process at times.
The working relationships are also strained at times because of long-standing strained relations between government and Aboriginal organizations. Aboriginal interview respondents almost invariably pointed to the historic lack of trust that Aboriginal people feel toward mainstream government, and also to frustration with traditional government approaches to delivering Aboriginal programs and services that are heavily bureaucratic, and that maintain control and decision-making authority in the hands of non-Aboriginals, often with little transparency. Most respondents expressed concern that the UAS may turn into "just another government funding program" with all the hoops to go through and restrictions that prevent them from making a real difference in the Aboriginal community.
This concern is supported somewhat by the short-term nature of the UAS and the lack of knowledge of whether the UAS will carry forward, and in what form. There is an apparent contradiction between the intended strategic focus of the UAS on the one hand, and the pressure exerted by federal officials to ensure that the funding allocations are spent in the fiscal years for which they are allocated. This contradiction places federal officials in a difficult position in that they are seen on the one hand to be encouraging a long-term strategic, collaborative approach while on the other hand they are focusing to a large extent on the project funding element, and imposing the usual budgetary restrictions and pressures. Most Aboriginal members recognize the reasons for the contradiction and do not view the local federal officials as being duplicitous, but it places a negative light on the Strategy as a whole, and heightens suspicion that there may not be a genuine federal government commitment to a long-term, strategic and innovative approach to urban Aboriginal issues. In practical terms it also means that the scarce available time of Aboriginal members is often used in relation to project funding instead of strategic development.
Collaboration within the federal "family" of departments and with other levels of government has shown some similarities to the government-Aboriginal relationships. In general there is a positive, collaborative approach to the UAS on the part of participating departments at all levels of government. Program officers who work with the Aboriginal communities in whatever capacity their department's mandate determines are uniformly clear about the depth of need in urban Aboriginal communities, and the need for governments to work more collaboratively and effectively. Representatives of the federal departments that have the most direct program relevance to the UAS (Western Economic Diversification, Health Canada, Human Resources and Skill Development Canada and Social Development Canada, Heritage Canada and in some cases Justice Canada and Indian and Northern Affairs Canada) attend steering committee meetings or internal federal meetings on the UAS regularly, and support the Strategy. Other departments have not been involved to date. Provincial and municipal officials attend meetings regularly as well in most communities. Where specific project ideas are raised that relate to their areas of interest they participate in planning and try to find ways to contribute either with funding or in-kind assistance.
It is worth noting that provincial representatives interviewed for this evaluation were extremely positive about the idea of the UAS as a new, strategic, collaborative approach that offers an opportunity to break through some of the long-standing barriers to improving conditions for urban Aboriginal people. They expressed frustration at times at the pace of progress and the level of bureaucracy that exists with federal programs, but they were unequivocal in their support for the Strategy and its basic concepts. This stands out as important because it differs from the experience with some other federal initiatives at the community level, where there is a perception that the federal government initiates a program, brings short-term funds to communities, heightens community expectations in areas of shared jurisdiction, and then changes priorities and leaves provincial governments under pressure to continue funding projects that have been initiated. Provincial officials have become wary of new federal initiatives in areas touching on their areas of jurisdiction for this reason.
There is little such apparent wariness about the UAS on the part of the provincial officials interviewed. In part this is because the project funding component of the UAS is relatively small, and does not present a large risk that projects will be initiated that will have major implications for provincial funding. As well, however, it is reportedly because the need for improvement in urban Aboriginal communities is so clear, the implications for provincial and municipal economies is now recognized as being significant, and the status quo in terms of government action is recognized as being insufficient. Urban Aboriginal affairs have fallen between the cracks of acknowledged federal and provincial responsibility for many years, and provincial officials see the UAS as a vehicle to break that deadlock and bring about some meaningful change.
In short, then, there is considerable support at the working level in all levels of government for the UAS, and some valuable collaboration taking place. The collaboration is still developing, however, and as with the Aboriginal community there are real barriers to truly effective collaboration to address urban Aboriginal issues.
One of those barriers is a lack of uncommitted funds at the local federal level and among provincial and municipal departments to allocate to the UAS. Since the UAS is intended to be primarily a strategic initiative and has a relatively small funding component, the leveraging of non-UAS funds is a critical element of any longer-term success. Even for the smaller UAS pilot projects being considered under the current phase, there are few resources available within existing federal, provincial or municipal program or operating budgets to contribute. Funds have been allocated already, often through multi-year agreements with third parties or programs with long-standing partnerships and funding procedures.
Beyond the question of contributions to UAS pilot projects, provincial and federal officials pointed to the existence of strong, well-entrenched program and policy instruments among participating government departments as presenting a major challenge to innovate ways of addressing urban Aboriginal issues. In a sense, UAS committees are forced to "think small" because there is no confidence that the status quo of fund allocations in major policy and program areas can be breached significantly. For example, in the area of education, funding is typically committed to an existing complexity of school boards, schools and training institutions, or targeted programs. Consideration under the UAS of what to do about urban Aboriginal education tends to be focused on the enhancement of existing Aboriginal student support programs, or consideration of a dedicated Aboriginal school, as opposed to a major rethinking of Aboriginal education in the urban setting that may involve complex changes to current delivery models, such as a revamping of curricula or specialized early childhood education for Aboriginal children. Footnote 13
A second barrier is the lack of impetus from senior management at the national level and at times at the regional level as well, with the federal government. As a rule the participating departments have not received directives to commit time and resources to the UAS. For the most part, federal regional action (aside from the UAS lead departments in each community) derives from on-the-ground knowledge of the extent of need in the urban Aboriginal community, and the commitment of individual officers, as opposed to direction from senior management. This reported lack of impetus from senior management limits the time available for the UAS, dictates that representation at UAS meetings tends to be by less senior officials, and means that action on the UAS tends to happen around the margins of department activities instead of being integrated in any substantial way.
One possible reason for the lack of senior management commitment among participating federal departments, and a factor in how local officials participate as well, is a lack of clarity about how they could contribute. It was noted earlier that there has tended to be a focus on the project-funding component of the Strategy because of pressures to ensure that the funds are spent in the fiscal year that they are allocated for, and the considerable amount of work involved in planning, decision-making and administration of project funds. One result of this is that the participating partner departments come to view the UAS as focusing largely on the project funding. When they examine how they might contribute, therefore, there is a tendency to focus on how the projects being considered fit into their mandate or specific programs, and whether there might be an opportunity for some additional funding or partnerships with existing projects. These opportunities are limited, as we have seen.
What appears to be missing at the officer level and, apparently, at a senior management level, is a clear understanding of the long-term, strategic nature of the UAS, and what that means for their departments. This message, if it is clear to UAS managers, does not appear to have been communicated effectively to partner departments.
The final factor influencing the effectiveness of collaborations and partnerships on the UAS relates directly to the communication aspect. The interviews in all participating communities pointed to insufficient strategic direction from the OFI on an ongoing basis. This is in part because of the pilot project focus that has tended to drive the initiative, in part because of the desire to maintain the community-driven approach to the UAS, and also because the UAS represents a departure from traditional government practice and is potentially very complex in its implications. Participants expressed the need to clarify exactly what it is they are trying to accomplish, what the possible mechanisms are, and what the full scope of the UAS might be. Better national-level communication is seen as needed to help maintain a clear vision for the UAS, and to continue to inform government and community participants as new members take part.
Finding: There is coordination with other initiatives such as the National Homelessness Initiative and various Heritage Canada and Health Canada programs, but coordination is ad hoc and dependent largely on cross-over in committee memberships. The few attempts to institute more formal linkages with other initiatives have met resistance from the Aboriginal community.
Finding: Overlap and duplication with other government programs does not appear to be a problem in the case of the UAS, in part because the UAS, by its nature, is a strategy to identify opportunities for more effective and coordinated use of public resources. Pilot project expenditures are by design targeted to innovative approaches that do not fit the funding streams of existing government programs. The UAS can be seen as a vehicle to identify overlap and duplication among government programs and determine the most effective ways to use those available resources in a coordinated way. The summative evaluation could identify cases of pilot project-specific overlap and duplication with other government programs.
Findings in this area are based on a review of local UAS documents describing the structures and processes in place and any planning that has taken place, and on interviews with federal, provincial and municipal government officials, and Aboriginal members of steering committees.
To date the major focus of the UAS in most communities has been on establishing steering committees and decision-making processes, and identifying projects on which to spend available UAS funds. Coordination with other initiatives has for the most part occurred through an overlap in committee memberships, and the informal shared communications that result from those overlaps. To the extent that other initiatives have representatives on UAS committees or are otherwise participating in the UAS, those other initiatives will have some knowledge and awareness of the UAS. This is occurring to varying degrees across the eight UAS communities, but it is fair to say from the interviews conducted for the evaluation that participants in major urban Aboriginal initiatives are likely to be aware of the UAS and have some knowledge of what it is about. These include in particular the Urban Aboriginal Homelessness (UAH) component of the National Homelessness Initiative (NHI), but also community renewal projects, education projects and training and skills development programs.
In addition, UAS project funding decisions will in many cases be made with an awareness of projects being considered under other initiatives, and some coordination is occurring in this area to ensure that the most appropriate initiative is used as the funding vehicle. The coordination is reactive rather than planned, but it nevertheless results in the avoidance of overlap and duplication and better overall planning and use of federal resources.
Efforts were made in several communities to establish a more formal link between the UAS and the Urban Aboriginal Homelessness initiative, but these were not successful. In Vancouver, for example, a well-established structure was already in place with representatives of Aboriginal service-providing agencies to establish community priorities related to housing and homelessness, and to decide on project funding. There was initially an interest on the part of federal officials in linking the UAS to this structure in order to avoid duplicating what already existed and duplicating the burden on Aboriginal members. However, Aboriginal community leaders were against the idea because they had experienced some difficulties with the bureaucratic requirements of the NHI, and preferred to deal with the UAS lead department in that community, Western Economic Diversification, which has less stringent requirements and more decentralized signing authorities than Human Resources and Development Canada (HRDC). They also wanted to be sure that they could control how the funds were allocated, and believed that this was more likely under the auspices of the UAS.
In Toronto, both the UAS and the UAH are administered by HRSDC, and the UAH officials were interested in establishing formal linkages or even administering the UAS together with the UAH. However, UAS officials in that department preferred to maintain a separate administration, and no significant collaboration took place. Because the UAS in Toronto never reached the point of funding projects before the steering committee process was disbanded, the issue of coordination of projects did not arise.
In Edmonton the Joint Planning Committee on Housing has an Aboriginal sub-committee that provides guidance on Aboriginal housing issues, and that sub-committee was originally considered as a possible committee for the UAS. However, that has not taken place, and there is now a newly formed (as of December, 2004) Aboriginal Review Committee functioning under the auspices of the Edmonton Housing Trust Fund (EHTF), which is charged with administering the UAS project funds. This committee is limited in function to the funding decisions, however, and work continues on the establishment of a steering committee with a broader mandate to address strategic considerations under the UAS.
The EHTF does provide a link between the UAS and the NHI because it is the community entity for the implementation of the NHI in the city. The link is thus established in the delivery mechanism and in community planning that goes beyond homelessness to look at a range of issues affecting Aboriginal people in the city.
The evaluation examined the issue of potential overlap and duplication between the UAS and other government programs directed to urban Aboriginal people. Through interviews with the full range of participants it was determined that the nature of the UAS minimizes greatly any risk of such overlap and duplication. The UAS is a strategy to identify opportunities for more effective and more coordinated application of public resources. It is not a program, and does not have a focused area of interest such as health, education or Aboriginal justice issues. Any pilot project expenditures are by design targeted to innovative approaches that do not fit the funding streams of existing government programs. If anything, the UAS can be seen as a vehicle to identify overlap and duplication among government programs and determine the most effective ways to use those available resources in a coordinated way.
There is the possibility that some UAS project funds could be allocated to projects that could have been funded under other existing government programs. The summative evaluation will want to review project expenditures to identify the extent to which the pilot projects resulted in innovate approaches or initiatives that could not be funded from other sources. Any potential project-specific overlap and duplication could be identified through such a review.
Finding: The UAS terms and conditions are viewed as flexible and sufficiently broad to allow most types of required expenditures, but the single-window approach itself does not appear to be bringing noticeable efficiencies to date. Delays in the final sign-off on project funding, in particular, are reported to be a problem.
Finding: There were initial concerns in participating departments that the use of the UAS terms and conditions by other departments, or even other programs within the lead department, would not meet accountability requirements. Some of these have been alleviated through consultation and clarification, but there remain some doubts, and few examples to date of the single window approach being used.
Findings in this area are based on interviews with federal government officials, and the survey of Aboriginal organizations.
At the outset of the initiative there was some optimism that the UAS terms and conditions, and the agreement with partner departments to allow for the application of UAS terms and conditions to funding from other programs, would reduce the level of red tape and reporting requirements that typically accompanies federal project funding. This was part of the overall effort to use the UAS as a test for streamlining the bureaucracy associated with horizontal, cross-departmental initiatives. It would mean that a project proponent could apply for UAS funding and funding from other federal programs under a single proposal, and have only one set of terms and conditions and reporting requirements to deal with.
Early indications are that while the terms and conditions of the UAS are viewed as quite flexible as far as the kinds of projects that can be funded, the benefits of the "single window" approach in relation to the terms and conditions are limited because the requirements of any participating departments, or programs within departments, must be built into any project involving joint funding. The funding recipient does not have to deal with multiple departments so there is a reported efficiency there, but proposal and reporting requirements may not allow for the kinds of efficiencies that were anticipated.
Thus far there have reportedly been few opportunities to test the use of the common terms and conditions, because there are few examples of other federal departments contributing to UAS projects. However, it is notable that even within HRSDC, efforts to use the common terms and conditions across programs were not seen as beneficial because the other program saw the need to maintain its own terms and reporting standards, and financial officers were reluctant to use the UAS terms and conditions exclusively because they had not been given instructions on how to do so.
In several communities it was noted that one positive result of the "single window" approach was that financial and program officers across departments had entered into a dialogue with a view to finding a way to make the idea work, so there is optimism in those communities that some benefits may be derived in the future.
The survey of Aboriginal organizations found that while specific elements of the project proposal process were seen as adequate (in the mid-range from positive to negative) in terms of fairness, timeliness, and the nature and extent of requirements for proposals, respondents found the overall UAS proposal process "about the same" or "worse" than other project proposal experiences. Additional comments provided by some survey respondents, and comments from Aboriginal respondents and local government respondents, indicate that the reason for this negative overall perception may be the reportedly lengthy delays in gaining approvals for project funding, once the projects were selected by the steering committees. It is acknowledged by OFI managers that the review and approval process in Ottawa has been a significant factor in those delays, and efforts are now being made to reduce the project approval turn-around time.
Finding: Steering committees in all but one community are highly satisfied with levels of support provided by lead departments. However, the provision of adequate supports to UAS processes has required supplemental contributions, at times substantial, from lead departments. In addition, it was found that while the available resources were usually sufficient for broad planning and project funding purposes, they were not sufficient to allow for the level of community outreach, project development, relationship building and community development that would further the intended longer-term, strategic focus of the UAS. At least initially, a lack of program delivery expertise and resources at OFI caused confusion and delays at the community level in implementing the initiative. At OFI, administrative responsibilities were found to have drawn resources away from broader strategic, partnership-building functions.
Finding: Supports provided by lead departments for project funding were found to be sufficient in terms of processing proposals in a fair and timely manner. However, resources have not typically been available for a more extensive project and proposal development function that may have strengthened projects and placed a greater emphasis on linking proposals to the longer-term strategic approach of the UAS.
Findings in this area are based on a review of local UAS documents, interviews with OFI and local federal officials, and resource allocation information provided by UAS regional coordinators.
The formative evaluation was asked to consider whether the resources allocated to the implementation of the UAS (as opposed to the project funding itself) was sufficient to enable communities to advance the initiative as intended. The focus of the inquiry was on the impact of resource levels on results, as opposed to an examination of resource allocations within communities and how the funds have been spent.
The inquiry has found that federal lead departments have been able to provide the supports necessary to establish the basic elements of the initiative such as initial community consultations, development and functioning of steering committees, federal and inter-government liaison, and project administration. However, to do so the lead departments have invariably had to supplement UAS budgets with contributions from their own departmental operating budgets, over and above what was allocated through the UAS.
The UAS allocations for the implementation of the initiative (not including pilot project funding), in the range of $300,000 per year for each province, cover the costs of salaries, travel and some expenses including training and meeting costs. Lead departments have contributed an estimated additional 20% to cover additional costs including staff time for related functions such as finance, administration and communications (or portions of the cost of UAS-funded staff), and other miscellaneous expenses. In Alberta and Saskatchewan the resource issue is a particular challenge because the provinces were allocated the same amount as the other provinces but they each have two participating communities. Saskatchewan in particular has experienced problems in this regard. That province reports salary expenditure close to double the UAS allocation for Regina and Saskatoon, without taking into account travel, meeting expenses and other costs associated with the UAS.
In the majority of communities the federal support that has been provided has been very well received and seen as an important contribution. Either directly or through a secretariat employed with UAS operating funds, lead departments prepare all the background documents for steering committee meetings and other community meetings, host the meetings (in most cases) and cover all expenses, attend the meetings and provide guidance and information as required, maintain records of proceedings, communicate with committee members and assist with any activities that result from decisions. In the case of the pilot projects, the lead departments (again either directly or through a third party) assist in developing the decision-making processes, assist in implementing the process, write up the project agreements and administer the contribution.
One area consistently seen as lacking is support for community development types of functions. There is a recognized need for more ongoing communication and consultation in the communities, and for the provision of greater guidance and expertise in the development of projects in keeping with community priorities (as opposed to simply responding to whatever project proposals come forward), in order to establish a more strategic direction for the UAS. These functions are not resourced at present.
In addition, the strategic elements of the UAS appear to be underdeveloped. This is in part due to time constraints and the focus on the project funding component. However, the resources available for the initiative at the community level are reported to have an impact here as well. Developing a long-term, strategic approach to addressing urban Aboriginal issues requires different sets of activities and different organizational structures and processes than are required for short-term community planning and project funding functions. It also may require different participants at the government and community levels to bring substantive expertise in priority areas. Certainly, it requires time and effort to develop these different functions, sell the strategy to potential contributors including senior officials in relevant government departments, and coordinate the implementation of any substantive initiatives that emerge.
The UAS is currently resourced, albeit insufficiently according to some coordinators, to implement the pilot project element of the UAS, but does not appear to be sufficiently resourced to tackle both the project funding and the broader strategic elements that are the core of the Strategy.
There are also resource-related findings at the national level. The evaluation did not examine OFI functions or resourcing, but interviews with federal officials at OFI and in the participating communities pointed to difficulties in Ottawa coping with the volume of work associated with the early implementation of the pilot project phase of the Strategy, and with the administration of pilot projects. Two officers were assigned to the UAS besides senior managers with responsibilities for the UAS and other files. Both were policy officers with little experience in program operations, and while the UAS was exclusively a policy initiative this was sufficient, but the OFI reportedly lacked the expertise and resources to handle the operational side of the Strategy as the pilot project phase was implemented. This caused delays and some confusion at the community level as officers there attempted to establish procedures and fit the UAS terms and conditions with their own departmental requirements.
At the national level, a repercussion of this resource deficit was that the strategic functions dependent on OFI did not receive sufficient attention. Little effort was devoted at the officer level to establish and develop working relationships with the partner federal departments at national headquarters, or with other levels of government and other interested national organizations. Periodic consultations took place at the senior management level, but this is acknowledged at OFI as having been insufficient to champion the UAS vision and develop the broader federal government engagement that was intended. OFI has begun staffing in key administrative and communications areas to enable the initiative to resume these functions.
As noted in section 4.4, the survey of Aboriginal organizations found that supports for the project proposal process were seen as adequate in terms of fairness, timeliness, and the nature and extent of requirements for proposals.
Finding: The main capacity building to date has been in developing trust and working relationships among those government and Aboriginal representatives involved in UAS committees. This "asset" is still a work in progress and fragile, but is viewed widely as a necessary step in what the UAS is trying to accomplish. In some communities capacity has also been developed through a planning process, enabling community leaders to identify priorities and begin to establish longer-term community assets. There is a recognized need to increase substantially the pool of Aboriginal community members with the experience and expertise to participate effectively in the kind of long-term, strategic process envisioned by the UAS, and to contribute to the development of new and innovative approaches to service delivery and government support on priority urban Aboriginal issues. This capacity issue is not being addressed significantly at present.
Finding: With the relatively small number of projects funded to date that are directed to individual service agencies or service agency partnerships, service delivery capacity cannot be said to have been enhanced appreciably thus far. For the most part the UAS has not offered resources for capacity building in individual organizations, such as project and proposal development. Services have been provided in some communities to help refine project proposals that have been formally submitted and have been approved in principle.
Finding: Data on project expenditures were only partially available at the time of this report. Those partial figures indicate that the UAS was succeeding in attracting some investments from other sources. However, it is not clear from this evaluation whether the UAS leveraged these other investments, or if they may have taken place in any case without the UAS.
Findings in this area are based on a review of local UAS documents describing the structures and processes in place and any planning that has taken place, and on interviews with federal, provincial and municipal government officials, Aboriginal members of steering committees, and representatives of other Aboriginal organizations in the communities.
Capacity building is a concept that is used to describe a variety of different aspects of community and individual development. For the UAS, capacity building is an objective that takes a broad view-to help provide Aboriginal communities with the tools they need to flourish and ultimately to close the gap between themselves and the non-Aboriginal population. Within that broad framework there are elements of capacity building that could include:
The evaluation has found that the main capacity building to date has been in the first area listed above. As noted earlier, there has been notable progress in all but two communities in establishing steering committees to oversee the UAS and in developing trust and working relationships among those government and Aboriginal representatives involved in the UAS committees. There is also widespread (but not unanimous) support among both government and Aboriginal participants for the idea of the UAS, and an apparent willingness to deal with the complexities and bureaucratic hurdles and continue to move forward.
This is no small achievement given the strong perceptions in the Aboriginal community that governments have not had Aboriginal interests at heart in previous initiatives, and the abiding lack of trust that pervades the relationship. The time and effort associated with building this aspect of capacity can be considerable, especially considering that most participants, including Aboriginal representatives, are volunteering their time over and above their other responsibilities.
In several of the UAS communities capacity has been expanded as well through a community planning process associated with the UAS. Most communities already had an Aboriginal community plan that had been developed through the UAH, but these plans are typically targeted to housing and other homelessness related issues, whereas the UAS is broader in scope. Where communities have succeeded in developing a broad urban Aboriginal community plan, this can be seen as enhancing the community's capacity to address urban Aboriginal issues.
In Calgary and to some extent in Winnipeg as well, UAS planning is being undertaken in a context beyond the UAS itself. The Calgary Urban Aboriginal Initiative (CUAI) has established a model that includes a large committee of Aboriginal service providers representing different substantive "domains", representatives of Aboriginal political organizations, an elder, an Aboriginal youth member, members from the three levels of government and other members at large from the community. A smaller executive committee oversees day-to-day operations. The detailed planning and recommendations for action (projects, initiatives, research) takes place in the individual domains (e.g. health, education, justice, housing, employment), where organizations and individuals who are active in the area and have expertise to contribute meet to identify priority issues and develop new approaches to addressing those issues. The UAS is one funding program that contributes to projects emerging from these domains. While the UAS cannot take full credit for these developments in Calgary, efforts in that city to implement the UAS since 1998 did contribute directly to the CUAI. The CUAI can certainly be seen as an important community asset and an example of enhanced community capacity arising in part from the UAS.
In Winnipeg the UAS is also subsumed under a broader urban development strategy for the city. An Aboriginal Partnership Committee has been formed under the umbrella of the Winnipeg Urban Development Agreement, to recommend strategies and projects to address Aboriginal issues in Winnipeg, and to ensure that the Aboriginal community has input into broader initiatives in the community. The Committee is still at an early stage. But there is optimism among members that it will provide a basis for a planned approach to addressing urban Aboriginal issues.
The other communities have also undertaken smaller scale planning processes in order to set priorities as a basis for selecting projects to fund under the UAS. In Vancouver, that process led to a decision to forego a second RFP call, and instead develop four initiatives designed to build sustainable capacity and further community development. Working groups have been struck to implement these initiatives. In Thunder Bay the UAS committee identified the alleviation of child poverty as their community priority. It is a single, multi-dimensional project to test new approaches to service and program delivery to address Aboriginal poverty issues in Thunder Bay.
In a number of communities the Tamarack Institute was engaged to lead community sessions on capacity building. These sessions were viewed positively where they took place, and were reported to have contributed to some of the early work in bringing Aboriginal members together to establish committees, identify community priorities and otherwise move the UAS process forward.
Other capacity building initiatives have also been undertaken, including a workshop in Vancouver to assist community agencies to develop project proposals, and one-day sessions in Saskatchewan to assist agencies in conducting project evaluations for their own purposes and to meet evaluation requirements in their contribution agreements.
These are examples of one type of capacity that the UAS has contributed to. In all participating communities, though, it is recognized that other aspects of capacity are low and in need of attention. In terms of continuing to develop the UAS as a strategic initiative, there is a concern that the individuals currently sitting on the UAS committees will not be able to continue to participate indefinitely, and that there are few other community members with the skills and experience to take their places, much less to contribute to an expansion of the UAS into substantive working groups or other such mechanisms to make advances in specific issue areas. This aspect of capacity has yet to be addressed in any direct way and is seen as a potential barrier to progress.
In terms of enhancements to services and facilities in the participating communities, it is too early in the pilot project phase to determine the extent to which sustainable assets are being put in place with the assistance of the UAS. The purpose of UAS funding was, after all, to enable communities to test innovative approaches to addressing urban Aboriginal needs, and not to fund new services or facilities. As the projects progress and are completed, there will be an opportunity to assess the extent to which they are resulting in approaches that can make a sustained contribution.
Beyond the pilot projects themselves, the UAS has the potential to make a much greater tangible contribution to community assets through its strategic focus. If this approach can lead to effective collaboration between Aboriginal communities and all levels of government in specific service areas, there may be considerable gains in community capacity. There is no evidence to date of any such gains.
Figures to date on project expenditures are only partially available. Figures for projects receiving multi-year projects are included but the figures do not include new projects that have been or will be identified for fiscal year 2005/06. These figures do not offer evidence of actual gains for communities because it is too early to assess results. However, they indicate that investments are being made through the UAS that offer opportunities for capacity building. They also indicate that UAS projects are attracting investments from other sources, but this evaluation did not examine the extent to which the UAS actually leveraged investments that would not otherwise have taken place, or simply contributed to some projects that were already being planned. The summative evaluation will need to examine this issue.
|Community||Number of UAS-Funded Projects||Total Cost of All Projects||Total UAS Contribution||Average UAS Contribution Per Project|
|* includes $297,832 in WD funding that was provided under UAS Terms and Conditions|
Finding: It is premature to expect measurable results in this area. These results are expected to take place to a limited extent through innovations developed in pilot projects, but primarily through the broad strategic UAS approach and ensuing actions in specific service areas. Such results will be evident only in the medium to long-term.
Finding: As above, there is no expectation that the UAS will have led to a narrowing of the gap at this early juncture. The evaluation did note, however, that there appears to be a misalignment between apparent expectations (as delineated in initiating documents and the UAS RMAF) and realistic opportunities to affect change in the short to medium-term. There appears to be insufficient recognition of the long-term nature of the UAS building process, as a necessary precursor to meaningful and sustainable change in service delivery and in life opportunities and circumstances for urban Aboriginal people.
Findings in this area are based on a review of project expenditure data and interviews with federal, provincial and municipal government officials, Aboriginal members of steering committees, and representatives of other Aboriginal organizations in the communities.
As noted in the previous section it is too early in the pilot phase to assess the results of the projects being funded or longer-term results such as improvements in the circumstances of urban Aboriginal people. While projects terms of reference require reporting on results in broad terms, project reports are not yet available to provide a basis for assessing whether they contain sufficient information for project evaluation purposes.
There are, however, two findings that are worth noting that relate to the longer-term objectives of the UAS. While the logic model for the UAS identifies narrowing the gap between urban Aboriginal people and the mainstream population as an expected outcome, few people involved in the initiative expect to see significant measurable results of that sort in the current funding period. That may be obvious given the complex and intransigent nature of many of the issues facing Aboriginal people in Canada, but it is worth raising here because it has a bearing on the question of what are realistic expectations for the UAS, and what time frames might be appropriate to use in mapping out the future of the Strategy.
The finding of the evaluation is that there do not appear to be any commonly shared ideas about what are reasonable expectations for the UAS to accomplish in the current funding period, or about a timeframe for different stages of development. This finding emerged from discussions during interviews in response to questions about the progress that had been made to date.
A related finding is that the design for the current funding period may have underestimated the time required, and the work involved, in establishing even the most basic components of this strategic initiative-the community-based steering committees that link Aboriginal communities with government partners to address urban Aboriginal issues. We have identified in this report some of the modest successes that have been achieved, and also the substantial barriers that still exist. In the view of many federal officials involved in the UAS at the community level, the expectations from Ottawa may be unrealistic. Those expectations are reflected in part in the RMAF and other descriptive documents, but perhaps more importantly in the pressures to spend annual funding allocations whether or not the structures and procedures are in place to do so in a planned and deliberate manner in keeping with the strategic focus of the UAS. In several communities there has been a deliberate decision to not fund projects even if it meant leaving some of their allocation unspent, in order to maintain the focus on establishing effective bodies and mechanisms to advance the Strategy.
Finding: The evaluation noted the following strengths and weaknesses of the UAS model:
Finding: Findings on the challenges faced in implementing the UAS have been discussed already in the context of findings in other issue areas. They indicate that while there have been some encouraging developments in all but one of the eight communities, progress has been hampered by the complexity of the task, the challenges of building new working relationships, and systemic and bureaucratic barriers.
Finding: The strengths of the UAS are in fact primarily in the delivery model itself. Implementing the model as it was intended has proven to be a challenge, but there is no evidence thus far that the model is unrealistic because much of it remains untested. The evaluation has found a number of areas requiring remedial action, but these are directed to specific elements or activities of the Strategy, rather than the overall delivery model.
Findings in this area are based on the full range of evaluation methods.
There is wide (although not unanimous) agreement among the full range of people interviewed and surveyed for the evaluation that the current UAS model is, potentially, a very effective model with which to address urban Aboriginal issues. The findings listed above have been discussed already in the context of findings in other issue areas. They indicate that while there have been some encouraging developments in all but one of the eight communities, progress has been hampered by the complexity of the task, the challenges of building new working relationships, and systemic and bureaucratic barriers. The strengths of the UAS are in fact primarily in the model itself, and in the wide support it is receiving in government and in Aboriginal communities (with the exception of some Aboriginal political organizations, as discussed earlier). Implementing the model as it was intended has proven to be a challenge, but there is no evidence thus far that the model is unrealistic because much of it remains untested. If more effective alternative approaches are found at this early stage of the UAS, they are most likely to reflect individual elements of the initiative that may be developed in specific communities that may be applicable in other participating communities, rather than a different overall approach.
Finding: Lead departments are maintaining documentation relating to the structures and processes that have been put in place, the implementation of those structures and processes (including minutes of meetings and all resulting reports and decisions), the projects being funded, and all financial transactions related to the UAS.
Finding: The great majority of administrative information/data for the summative evaluation that relates to the early outcomes of the UAS is currently available in a form that will be sufficient for the evaluation. There are two exceptions. First, information on non-UAS public expenditures on urban Aboriginal issues at all levels of government is not currently available in any systematic or comprehensive form. If it is deemed necessary for the summative evaluation, the collection of this information will need to be undertaken prior to the evaluation itself, since it will require some time to compile. Second, the summative evaluation will require a full accounting of the resources used for the UAS and how they have been used, including resources provided under the UAS budget and resources contributed by the lead departments in the participating communities. If this is deemed important for the summative evaluation a systematic approach and common categories of expenditures will need to be identified and put in place.
As is to be expected, data/information for the medium and longer-term outcomes will derive primarily from methods such as interviews, surveys, case studies and expert panel reviews. There are some performance indicators in the current evaluation strategy relating to these longer-term outcomes that are not currently being tracked. A decision will have to be made regarding realistic expectations for the UAS in the current funding period, the appropriate performance indicators to apply, and therefore what data collection mechanisms will need to be put in place in preparation for the summative evaluation.
Findings for this section are based on the draft UAS Evaluation Strategy document developed in the early stages of the formative evaluation, interviews with UAS coordinators and staff, and an e-mail request for specific information about data availability sent to the UAS coordinators.
The review of data for evaluation indicators began with an initial assessment of each performance indicator in the Table of Issues, Questions, Indicators and Methods presented in Appendix A of this report. Each indicator and method was examined from the point of view of whether the required information would be readily available through existing sources, would be available but require collection and compiling, or would not be available unless a recording mechanism was put in place.
In some cases it was clear that the information/data would be readily available, especially for the outputs and early outcomes of the initiative. This is true, for example, of Aboriginal policy documents produced by OFI and partner departments, data on UAS project expenditures, copies of any community plans and other planning documents and reports produced under the UAS at the community level, membership lists of UAS committees, international literature on urban Aboriginal issues, and individual project budgets and types of projects funded.
In other cases it was clear that the information/data would not be readily available because it required evaluation methods that would not be expected to take place in the normal course of the ongoing performance management of the initiative. These include information to be derived from interviews with the full range of participants, any surveys to be conducted, case studies and expert panel reviews. The sources of information for these methods will be readily available, and while there may be barriers to the effective implementation of some methods, there is no apparent need for early preparation of data collection mechanisms prior to the summative evaluation.
From the initial review a number of specific types of information/data were identified for which the availability was in question. These items were pursued during interviews and included in a request for information from the UAS coordinators. They are identified and discussed below.
Data/information for the medium and longer-term outcomes will derive primarily from methods such as interviews, surveys, case studies and expert panel reviews. There are some performance indicators in the current evaluation strategy relating to these longer-term outcomes that are not currently being tracked. These indicators include:
The first of these indicators would be relatively easy for UAS managers to monitor through an ongoing tracking mechanism, integrated into the effort to track non-UAS expenditures on urban Aboriginal issues. The remaining indicators, however, relate to more complex issues and would require some concerted effort to define terms and more specific outcome elements, establish populations and possible sampling frames, and design research strategies. They assume that there will have been some considerable progress in improvements to service delivery through the UAS, and the findings of this formative evaluation suggest that this assumption may be optimistic. A decision will have to be reached about what are realistic expectations for the UAS in the time period leading up to the summative evaluation, and what indicators will therefore be appropriate. On that basis, decisions can be made about any required data collection strategies prior to the evaluation itself. In addition, there will need to be a review of project reports to assess the extent to which they will contribute to an evaluation of the success of the projects in meeting UAS objectives.
Implementation of the pilot project phase of the UAS began in 2003, with communities planning their longer-term approaches to using the UAS as a vehicle for change, and working to make best use of the available project funding. This formative evaluation was conducted to assess early progress, ensure that performance measurement strategies and activities are on track, and identify any improvements that should be made on the basis of the experience thus far.
The previous section presented the findings of the evaluation in each of the issue areas in its terms of reference. Here, the report draws those findings together into a set of conclusions about UAS progress to date and areas that require remedial action.
The main focus of the evaluation was in assessing whether the pilot projects, and the pilot project phase of the UAS as a whole, is proceeding as intended, and identifying any problems with implementation that need to be addressed. There are a number of conclusions in this regard.
A second area of inquiry was with regard to the UAS approach to performance measurement and the collection of information for that purpose.
The formative evaluation also looked at whether the pilot projects, and the initiative as a whole, were making progress toward the achievement of desired outcomes.
The evaluation also examined the issue of overlap and duplication of the UAS with other government programs.
Finally, the formative evaluation examined the resourcing of the UAS, and whether the UAS had leveraged partner resources.
These conclusions summarize the gains that have been made to date under the UAS, and the strengths of the existing model. They also point to some significant barriers to success that will need to be addressed. The findings suggest that there is an opportunity for the UAS to offer an important, innovative approach to resolving problems in urban Aboriginal communities, problems that to date have appeared to be intractable. They also suggest that failure to address the barriers will greatly limit the value of the UAS, and may undermine the support the Strategy has received thus far. The next section sets out recommendations to address those barriers, for the consideration of UAS managers.
Based on the conclusions presented above, evaluators offer the following recommendations for consideration by UAS managers. The recommendations do not imply that the suggested actions should necessarily be undertaken with existing resources. Some of the recommendations may require resources beyond what has currently been allocated for specific functions.
The reaffirmation should reflect the fact that the UAS is primarily a strategy, and that the legacy that the UAS wants to leave is a capacity for Aboriginal communities and governments to work together in a strategic, collaborative way to address urban Aboriginal issues. The reaffirmation should also make reference to the role of the pilot project funding, and offer clearer guidelines about how the projects are intended to feed into the broader strategic vision.
Ideally, the statement would embody ongoing support for Aboriginal community strategic processes at the broad level such as the current steering committees, and just as important at the "sector working group" level, where the specific, targeted strategic planning work will likely take place.
The guidelines should not unduly restrict communities in the paths they choose to implement the Strategy, but they should establish clearly the kinds of directions that are in keeping with the UAS vision, and the kinds of mechanisms that are seen as most likely to succeed. The guidelines should include recognition that the collaborative, Aboriginal community-driven committee process that is being fostered in the communities is a critical stepping stone that should be allowed to develop at a moderate pace if that is necessary.
Part of the strategy should be internal to UAS participants, directed to UAS coordinators, government partners at the national and local levels, and steering committee members. A second part should be targeted externally, mainly to the broader Aboriginal communities but also to the general public. The established communication approach should include on-going delivery of those messages throughout the initiative to counteract the forces that will tend to steer the initiative in other directions, such as an undue focus on project funding or approaches that accept the barriers identified in this report as being insurmountable.
The main purpose of the local communication strategies would be to engage the broader Aboriginal communities through improved awareness and opportunities for participation, and to make the existing planning and decision-making structures and processes much more transparent than they are at present.
This work should rely heavily on the renewed vision and the reaffirmation referred to above. It should include clear guidance in practical terms about ways in which partners can contribute nationally and at the community level. As specific strategic initiatives are developed at the community level, senior managers of relevant departments should be informed of those developments and provided with guidance about how best to contribute.
The tools could include standard terms of reference, guides for procedures, conflict of interest guidelines, guides for strategic planning, and other tools identified by regional coordinators. The tools should be designed so as to encourage a standard approach to the extent possible, while allowing sufficient flexibility to be useful in all participating communities.
In particular, OFI should develop an approach to address:
The current plan is for the conduct of a summative evaluation of the UAS by early 2006, in order to provide managers with information to support an anticipated Cabinet Submission later that year. However, the findings of this formative evaluation suggest that most participating communities may not be at a stage where a summative evaluation will appropriate, because few projects will have been completed sufficiently early to allow for the assessment of results, and because the strategic elements of the UAS are still very much under development. The inclusion of four new communities in 2004-2005 has further complicated planning for the summative evaluation. Given these realities, evaluators make the following recommendations for planning a summative evaluation.
The delay would be in order to have sufficiently broad information on the results of projects, to give communities more time to develop their strategic approach, and to allow for development in the four new communities.
The interim evaluation should examine results to date (including project results where they are available), assess progress in the four new communities, and examine the extent to which the findings and recommendations of this formative evaluation have been acted upon, and to what effect.
It should use as a starting point the draft Evaluation Strategy developed at the outset of this formative evaluation, and integrate the relevant information from this formative evaluation.
The review should include an examination of pilot project reports to assess the extent to which they provide the required information for evaluation purposes.
The purpose of the coordination would be to ensure that common measures are used, and to avoid any unnecessary duplication of effort and expense. Consideration should be given to working with regional coordinators to develop common approaches to evaluation and to providing resources for training in evaluation where it is required. Consideration should also be given to providing one-day seminars on project evaluation to project proponents, as is the case in Saskatchewan.
Most participating communities do not have reliable information in this area as a basis for furthering the strategic elements of the initiative and as a basis for assessing the contribution of the UAS in a future summative evaluation.
The resources should include those provided under the UAS budget and resources contributed by the lead departments in the participating communities. The approach should include common categories of expenditures for UAS coordinators to use to allow for standardization of the process.
|1. Are the UAS pilot projects, and the pilot project initiative as a whole, proceeding as intended, and are there any problems with implementation that should be addressed?||1.1 To what extent have UA communities been involved in planning and implementation of the initiative?||
|1.2 Is UAS planning and decision-making and project funding accessible to all interested Aboriginal stakeholders?||
|1.3 How effectively have federal and provincial governments, Aboriginal organizations and other stakeholders worked together to implement the UAS? What are the factors influencing effective collaboration?||
|1.4 Has there been coordination with other initiatives for UA people?||
|1.5 Do federal departments participating in the UAS have a common understanding of roles and responsibilities?||
|1.6 Have federal departments succeeded in coordinating their efforts in relation to UA issues?||
|1.7 Has having single contribution agreements and common terms and conditions for pilot projects brought efficiencies for community participants, and if so, what are the main areas of efficiency?||
|1.8 Are participating federal departments satisfied that their information and accountability needs are being met through this "single window" approach?||
|1.9 Have federal government supports and resources assisted communities in UA planning and decision making?||
|1.10 Have federal government supports and resources assisted project proponents?||
|1.11 Have the UAS pilot project delivery models compared favourably to previous models for UA project funding in terms of the time and effort required by stakeholders?||
|1.12 How has the UAS pilot project delivery model compared to previous models for UA project funding in terms of the time and effort required by federal officials?||
|1.14 What challenges have been encountered in implementing the UAS, and how have these been addressed to date?||
|1.15 Are there alternative delivery approaches that might work better?||
|2. Is the existing approach to performance measurement and the collection of information for that purpose sufficient, and are there any gaps in the types of measurements being applied or the information being collected, that present a risk to the strength of future evaluation findings, and that require changes to the performance measurement strategy?||2.1 What information/data on UAS planning and projects is currently being collected in communities for UAS reporting purposes?||
|2.2 Referring to the summative evaluation strategy, which evaluation questions and indicators does the information/data that is currently being collected pertain to, and is it sufficient for evaluation purposes in terms of validity and reliability?||
|2.3 Are there evaluation questions/indicators that will not be addressed sufficiently through current information/data collection mechanisms, and if so, is it feasible for communities to collect that information/data?||
|3. Are the pilot projects, and the initiative as a whole, making progress toward the achievement of the desired outcomes?||3.1 To what extent has the UAS initiative contributed to the development of partnerships among orders of government, Aboriginal groups and communities, and other stakeholders?||
|3.2 To what extent have these partnerships resulted in improved coordination of UA related activities?||
|3.3 To what extent has the federal capacity to respond in a coordinated way to community UA needs increased as a result of the initiative?||
|3.4 To what extent has the capacity of the Aboriginal communities in participating communities to address UA issues been increased as a result of the UAS?||
|3.5 To what extent has the capacity of individual Aboriginal organizations in participating communities been increased as a result of the UAS?||
|3.6 To what extent has the UAS resulted in new or enhanced services for urban Aboriginal people in participating communities?||
|3.7 To what extent has the UAS initiative narrowed the gap between urban Aboriginal people and the mainstream population?||
|4. Are there areas where UAS activities overlap with other government programs and services?||
|5. Are there indications that UAS has succeeded in leveraging partner resources, and has the size and distribution of UAS project and administrative allocations enhanced or hindered initiative objectives?||5.1 Are there indications that UAS has succeeded in leveraging partner resources?||
|5.1 Has the size or distribution of UAS project allocations and administrative allocations enhanced or hindered initiative objectives?||
|Rationale and Relevance||1. To what extent do UAS related activities continue to address GoC priorities?||
|2. To what extent are UAS activities relevant to current and evolving urban Aboriginal needs and community needs?||
|3. To what extent are UAS pilot project activities consistent with current research and thinking about how best to address urban Aboriginal issues?||
|Cost effectiveness||4. Are there areas where UAS activities overlap with other government programs and services?||
|5. Are there indications that UAS has succeeded in leveraging partner resources?||
|6. Has the size or distribution of UAS allocations supported or hindered initiative objectives?||
|Partnerships and Coordination|
|Success||7 To what extent has the UAS initiative contributed to the development of partnerships among orders of government, Aboriginal groups and communities?||
|8. To what extent have UAS partnerships resulted in a more coordinated approach to addressing UA issues?||
|9. To what extent has the UAS initiative contributed to the establishment of better federal coordination mechanisms across departments||
|10. To what extent have UAS-based federal coordination mechanisms led to improved coordination of federal activities related to UA issues?||
|Research and Communications|
|Success||11. To what extent has UAS research (national, regional and local) resulted in new information about UA issues?||
|12. To what extent have UAS research and communications activities at the national and community levels resulted in an increased interest in UA issues among academics||
|13. To what extent have UAS research and communications activities at the national and community levels contributed to increased investments and other activity on UA issues at the community level?||
|Success||14. To what extent has the capacity of the Aboriginal communities to address UA issues been increased as a result of the UAS?||
|15. To what extent has the UAS resulted in new or enhanced services for urban Aboriginal people in participating communities?||
|16. To what extent have UAS-based improvements in coordination contributed to better government service delivery to Aboriginal communities?||
|17. To what extent has the UAS initiative narrowed the gap between urban Aboriginal people and the mainstream population?||