Value for Money Audit of the Giant Mine Remediation Project

Date: September 2012
Project No. 12-32

PDF Version (666 Kb, 47 Pages)

 

 

Table of contents

Acronyms

AACE

Association for the Advancement of Cost Engineering International

AANDC

Aboriginal Affairs and Northern Development Canada

ADM

Assistant Deputy Minister

AES

Audit and Evaluation Sector

DWP

Detailed Work Plan

EA

Environmental Assessment

FCSAP

Federal Contaminated Sites Action Plan

GMRP

Giant Mine Remediation Project

GNWT

Government of Northwest Territories

KPI

Key Performance Indicator

MCVR

Monthly Cost Variance Report

MPO

Major Project Office

NAO

Northern Affairs Organization

NCSP

Northern Contaminated Site Program

NT

Northwest Territories

PEP

Project Execution Plan

PMC

Project Management Committee

PWGSC

Public Works and Government Services Canada

PWP

Phase Work Plan

RACI

Responsible, Accountable, Consulted and Informed

RMP

Risk Management Plan

SLA

Service Level Agreement

SPAC

Senior Project Advisory Committee

SSA

Specific Service Agreement

WBS

Work Breakdown Structure

YTD

Year-to-date

 

 

Glossary of Terms

AACE

Association for the Advancement of Cost Engineering International (AACE) is a non-profit organization that certifies members in the area of cost engineering and provides them guidance and tools to facilitate project management, estimations, risk management, and claims.

DWP

A Detailed Work Plan (DWP) is a planning document that outlines the activities and deliverables to be performed during a given year. A DWP can include information on the planned activities for the year, their estimated cost, planned completion dates, and change orders for any changes made throughout the year to the project budget or project scope.

EA

An Environmental Assessment (EA) is a tool through which the environmental effects of a proposed project are predicted and evaluated, and a subsequent decision is made on the acceptability of the project.

KPI

Key Performance Indicators (KPIs) are set of quantifiable measures that an organization uses to assess their performance in terms of meeting their strategic and/or operational goals.

MCVR

A Monthly Cost Variance Report (MCVR) is a monthly financial document prepared by the project management team to track and report on the difference between planned and actual project expenditures.

PEP

A Project Execution Plan (PEP) is the core document for the management of a project. It is a statement of policies and procedures defined by the project manager for the project sponsor/project director's approval. It sets out in a structured format the project scope, objectives, milestones, communication plan, Project change control procedures and other key project information.

PWP

A Phase Work Plan (PWP) is a planning document that outlines the activities and deliverables to be performed during a specific phase of a broader project. The PWP provides information on the resource requirements needed to complete the specific phase of the project.

RACI

A RACI chart is a matrix of all the activities or decision making authorities undertaken in an organization set against all the people or roles. At each intersection of activity and role it is possible to assign somebody Responsible, Accountable, Consulted or Informed for that activity or decision.

RMP

A Risk Management Plan (RMP) describes how project risk management will be structured and performed on a given project.

SLA

A Service Level Agreement (SLA) is a formal negotiated agreement between two parties – the service provider and the service receiver – concerning a measurable level of service (e.g. levels of availability, serviceability, performance, operation or penalties in the case of violation of the SLA).

SSA

A Specific Service Agreement (SSA) is a written agreement between PWGSC and a client organization that defines the scope of work to be undertaken, terms and conditions for providing the services, and the basis for invoicing/payment. The SSA is negotiated between the client organization representative having authority and the PWGSC service provider authority.

WBS

A Work Breakdown Structure (WBS) is a tool used to define and group a project's discrete work elements in a way that helps organize and define the total work scope of the project. A WBS also provides the necessary framework for detailed cost estimating and control along with providing guidance for schedule development and control.

 

 

Executive Summary

Background

The Minister of Aboriginal Affairs and Northern Development Canada (AANDC) requested that the Audit and Evaluation Sector (AES) conduct a Value for Money Audit of the Giant Mine Remediation Project (GMRP). Value for money auditing is intended to examine the ability of a government organization to discharge its responsibilities and control their costs by ensuring that resources are managed economically and that activities are organized efficiently.

Giant Mine is a defunct gold mine located in the city of Yellowknife, Northwest Territories. The mine site was first put into production by Giant Yellowknife Gold Mines Limited in 1948 and ran continuously under various owners until 1999. In 1999, the mine owner (Royal Oak Mines) was assigned into receivership and the Government of Canada, under the Department of Indian Affairs and Northern Development Act and the Northwest Territories Water Act, assumed administration and control of the mine with the responsibility for the protection of the environment and human health safety. It was immediately sold to Miramar Giant Mine Limited, after providing Miramar with indemnification for the underground contamination. Under the agreement, the mine site continued to operate until 2004, with gold ore being shipped off-site for processing and AANDC assuming the role of caretaker for pre-existing environmental liabilities on the property, including the arsenic trioxide (As2O3) dust stored underground.

Mining operations at the site, which grew over the years to encompass more than 870 hectares, including a number of ponds and small lakes, were halted in July 2004. Since 2005, AANDC and the Government of Northwest Territories (GNWT) have co-managed the site, with the Deton'Cho Nuna Joint Venture providing on-site care and maintenance at Giant Mine.

Currently, there are 237,000 tonnes of arsenic trioxide stored underground at Giant Mine in five chambers and 10 mine stopes. The mine site area has other sources of arsenic located in the abandoned roaster, tailings, waste rock, underground mine workings, and contaminated soils.

In 1997, AANDC and the mine operator, along with Environment Canada, the GNWT and the City of Yellowknife, began holding technical workshops to discuss the management of arsenic trioxide at Giant Mine. Over the next six years, AANDC, together with various stakeholders, worked to finalize an action plan to address mine contamination.

The broad objectives for the GMRP are to clean up the surface; stabilize and secure the underground; and maintain and monitor the site for health and safety. Surface and underground remediation activities, which have been identified following several years of extensive technical and scientific research and public consultation, are designed to minimize the release of contaminants from the mine site to the surrounding environment and to protect the health and safety of local residents. AANDC has ultimate accountability and responsibility for the GMRP.

Audit Objective and Scope

The objective of this audit was to assess the efficiency and effectiveness of the management practices, financial controls and accountability structures in place to support governance, risk management, project management, budget management and cost estimation and procurement management of the GMRP. The ultimate outcome for the GMRP is the remediation of the site within certain quality, time, and budget constraints. As a result, the attributes of management practice for which value for money were assessed included those considered fundamental to help ensure the planned outcome for the GMRP could be achieved.

The scope of this audit covered the timeframe from 2006-07 to July 2012. The audit included an assessment of the efficiency and effectiveness of the process to action key decisions related to governance, risk management, project management, budget management and cost estimation and procurement management, for the GMRP. The audit did not seek to examine the approaches selected to remediate the site (e.g., frozen block method). There were 56 options considered and rejected in favour of the Frozen Block Method, outlined in an Options Analysis of Remediation Method document.

Statement of Assurance

In my professional judgment as Chief Audit and Evaluation Executive, sufficient and appropriate audit procedures have been performed and evidence gathered to support the accuracy of the conclusions reached and contained in this report. The conclusions were based on observations and analyses of the situations as they existed at the time against the audit criteria. The conclusions are only applicable to the Value for Money Audit of the Giant Mine Remediation Project. The evidence was gathered in accordance with the Internal Auditing Standards for the Government of Canada and the International Standards for the Professional Practice of Internal Auditing.

Conclusion

Overall, we found that AANDC has made progress in building and improving on its project management practices to help ensure the achievement of its planned outcomes on the GMRP. AANDC has introduced and/or drafted a number of sound practices to strengthen its activities in the areas of governance, risk management, project management, cost estimation and budget management, and procurement management. Processes and tools that have been implemented or drafted by AANDC to improve practices include: project governing bodies; operational risk assessment practices; a Major Project Office; improved cost estimation practices and budget reporting; and Service Level Agreements to Public Works and Government Services Canada (PWGSC) procurement requirements.

Although a number of sound practices have been implemented, or are in the process of being implemented, inconsistencies were identified in the extent to which new practices and processes have been fully operationalized. In this regard, five key recommendations were identified through the audit.

Recommendations

  1. The Assistant Deputy Minister, Northern Affairs Organization, should ensure that the proposed governance framework for the Giant Mine Remediation Project is finalized and implemented, with minutes and records of decision documented for the key governing bodies, such as the Oversight Committee and the Senior Project Advisory Committee.
  2. The Assistant Deputy Minister, Northern Affairs Organization, should ensure that a comprehensive risk management framework (including strategic and project level risks) and a risk management plan are established for the Giant Mine Remediation Project to monitor risks on a regular basis.
  3. The Assistant Deputy Minister, Northern Affairs Organization, should ensure, as part of the Major Project Office implementation, that:

    • Clear roles and responsibilities between PWGSC and AANDC for the Giant Mine Remediation Project are defined, documented, communicated, agreed upon, and finalized such that the project management can focus on optimizing, resourcing, and delivering the project in accordance with its objectives. The resulting final organizational structure should be as streamlined as possible to support a strong and clear focus on the project.
    • A proper baseline schedule (prepared according to industry standards), an integrated project schedule, a management plan and a reporting regime are established to link all parties together into a standardized Critical Path Methodology. The establishment of these documents will help to ensure that a clear window to the overall position and progress for the Giant Mine Remediation Project is provided to the project management team(s).
    • A rigorous and effective change management program be finalized to support the consideration and reporting of changes to the scope and total estimated cost of the Giant Mine Remediation Project.
  4. The Assistant Deputy Minister, Northern Affairs Organization, should ensure, through consultation with the Chief Financial Officer, that budget management and cost estimation practices for the Giant Mine Remediation Project are strengthened by:

    • Developing a Master Project Total Estimated Cost to Complete document that includes a cost variance simulator, such as a 'Monte Carlo Simulation', to highlight the potential and probable deviation ranges in total cost estimates. The establishment of such a document would help to communicate key assumptions in the cost estimate and variance vulnerabilities to all project stakeholders.
    • Incorporating multi-year requirements of future work and costs in a Master Project Total Estimated Cost to Complete document based on industry best practices and the Association for the Advancement of Cost Engineering International Cost Estimate Classification system. Roles and responsibilities for maintaining, reviewing, and monitoring this document should be defined and communicated.
  5. The Assistant Deputy Minister, Northern Affairs Organization, should ensure, through consultation with the Chief Financial Officer, that the current approach to procurement management for the Giant Mine Remediation Project is strengthened by:

    • Improving the tools and guidance to support contract monitoring and reporting of expenditure variances and performance measures and targets.
 

 

1. Background

Introduction

The Minister of Aboriginal Affairs and Northern Development Canada (AANDC) requested that the Audit and Evaluation Sector (AES) conduct a Value for Money Audit of the Giant Mine Remediation Project (GMRP). Value for money auditing is intended to examine the ability of a government organization to discharge its responsibilities and control their costs by ensuring that resources are managed economically and that activities are organized efficiently.

Mine History

Giant Mine is a defunct gold mine located in the city of Yellowknife, Northwest Territories. The mine site was first put into production by Giant Yellowknife Gold Mines Limited in 1948 and ran continuously under various owners until 1999. In 1999, the mine owner (Royal Oak Mines) was assigned into receivership and the Government of Canada, under the Department of Indian Affairs and Northern Development Act and the Northwest Territories Water Act, assumed administration and control of the mine with the responsibility for the protection of the environment and human health safety. It was immediately sold to Miramar Giant Mine Limited, after providing Miramar with indemnification for the underground contamination. Under the agreement, the mine site continued to operate until 2004, with gold ore being shipped off-site for processing and with AANDC assuming the role of caretaker for pre-existing environmental liabilities on the property, including arsenic trioxide (As2O3) dust stored underground.

Mining operations at the site, which grew over the years to encompass more than 870 hectares, including a number of ponds and small lakes, were halted in July 2004. Since 2005, AANDC and the GNWT have co-managed the site, with the Deton'Cho Nuna Joint Venture providing on-site care and maintenance at Giant Mine.

Site Contamination

Gold ore deposits at Giant Mine were found in arsenopyrite (FeAsS), the most abundant and widespread arsenic mineral. To extract gold from the arsenopyrite ore, a roasting process was employed, which created, as a by-product, arsenic trioxide dust. Most of the dust created through extraction was collected and pumped underground into purpose-built chambers and mined-out stopes. This approach to storage relied on the area's natural permafrost, which worked as a frozen barrier to prevent the dust from migrating to other areas. However, due to on-going mining in the area, the permafrost thawed, which allowed water to seep into and out of the underground storage areas.

Currently, there are 237,000 tonnes of arsenic trioxide stored underground at Giant Mine in five chambers and 10 mine stopes. The mine site area has other sources of arsenic located in the abandoned roaster, tailings, waste rock, underground mine workings, and contaminated soils.

The Remediation Project

Development of the Remediation Strategy

In 1997, AANDC and the mine operator, along with Environment Canada, the GNWT and the City of Yellowknife, began holding technical workshops to discuss the management of arsenic trioxide at Giant Mine. Over the next six years, AANDC, together with various stakeholders, worked to finalize an action plan to address mine contamination. These activities included: the appointment of a technical advisor to study the possible alternatives for site management; the creation of an Independent Peer Review Panel to assess the proposed alternatives for long-term site management; the completion of health and ecological risk assessments to examine the risks of current and potential future releases of arsenic under the various alternatives proposed; and, the commencement of a public communications campaign to provide information regarding management alternatives for Giant Mine.

About the Remediation Project

The broad objectives for the GMRP are to clean up the surface; stabilize and secure the underground; and maintain and monitor the site for health and safety. Surface and underground remediation activities, which have been identified following several years of extensive technical and scientific research and public consultation, are designed to minimize the release of contaminants from the mine site to the surrounding environment and to protect the health and safety of local residents. AANDC has ultimate accountability and responsibility for the GMRP.

Surface Remediation

The remediation plan for the surface calls for the disposal of all buildings and infrastructure on-site that no longer serve a purpose or function. Surface remediation also includes: the on-going collection and treatment of contaminated water on-site; the rehabilitation of Baker Creek (an estuary that runs adjacent to the main pit of the Giant Mine); covering of the tailings and sludge areas with quarried rock and fine-grained soil; the removal of unnecessary mine roads; and, the diversion of a portion of Highway 4 away from the surface facilities required for ground freezing.

Underground Remediation

In 2004, AANDC announced a decision to proceed with the "Frozen Block Method" as the preferred long-term alternative for the storage of arsenic trioxide dust. Under the Frozen Block Method, the area around and within the 15 chambers and stopes is to be frozen using a super-cooled liquid circulated through pipes located beneath and beside each chamber and stope, creating an artificial permafrost around each container. Once each storage area has been completely frozen, thermosyphons – closed-loop vaporization and condensation circuits that operate without the use of a pump – will be installed to regulate the underground temperature.

A study of the Frozen Block Method (hereon referred to as the Freeze Optimization Study) was initiated in June 2009 while the full site remediation plan underwent an environmental assessment. The results of this study will be used to inform the environmental assessment process and to provide information about the operation, including: power requirements; rates of freezing; and, more accurate cost estimates.

Project Status

AANDC and the GNWT applied for a water license to begin the GMRP in October 2007. In March 2008, the project was referred to an environmental assessment (EA) by the City of Yellowknife. The Mackenzie Valley Environmental Impact Review Board co-ordinates this process, which will includes various hearings and other activities associated with an EA. Currently, the GMRP is in its fifth year of an EA, which has limited remediation activities as no work at the site may begin – unless the remediation activity is deemed an immediate health and safety risk and is removed from the EA submission – until the regulatory process is completed.

A breakdown of the reported remediation expenditures for the project from 2006-07 to 2011-12 is provided in Appendix B. A breakdown of the current total estimated cost to complete the Giant Mine remediation project is contained in Appendix C.

 

 

2. Audit Objective and Scope

2.1. Audit Objective

The overall objective of the audit was to assess the efficiency and effectiveness of the management practices, financial controls and accountability structures in place to support governance, risk management, project management, budget management and cost estimation and procurement management of the GMRP. The ultimate outcome for the GMRP is the remediation of the site within certain quality, time, and budget constraints. As a result, the attributes of management practice for which value for money were assessed included those considered fundamental to help ensure the planned outcome for the GMRP could be achieved.

2.2. Audit Scope

The scope of this audit covered the timeframe from 2006-07 to July 2012. The audit included an assessment of the efficiency and effectiveness of the process to action key decisions related to governance, risk management, project management, budget management and cost estimation and procurement management for the GMRP. The audit did not seek to examine the approaches selected to remediate the site (e.g., frozen block method). There were 56 options considered and rejected in favour of the Frozen Block Method, outlined in an Options Analysis of Remediation Method document.

The audit criteria used during the audit were based on recognized models of control, Treasury Board of Canada Secretariat's draft Core Management Controls: A Guide for Internal Auditors, and performance audit and special examination criteria, used by the Office of the Auditor General and other leading practitioners.

 

 

3. Statement of Assurance

In my professional judgment as Chief Audit and Evaluation Executive, sufficient and appropriate audit procedures have been performed and evidence gathered to support the accuracy of the conclusions reached and contained in this report. The conclusions were based on observations and analyses of the situations as they existed at the time against the audit criteria. The conclusions are only applicable to the Value for Money Audit of the Giant Mine Remediation Project. The evidence was gathered in accordance with the Internal Auditing Standards for the Government of Canada and the International Standards for the Professional Practice of Internal Auditing.

 

 

4. Approach and Methodology

The Value for Money Audit of the Giant Mine Remediation Project was conducted in accordance with the audit program and follows the Office of the Auditor General's value for money guidelines contained in the Performance Audit Manual. and the Canadian Comprehensive Audit Foundation's Comprehensive Auditing Reference Guide.

The principal audit techniques used included:

The approach used to address the audit objective included the development of audit criteria against which observations, assessments, and conclusions were drawn. The audit criteria developed for this audit are included in Appendix A.

 

 

5. Findings and Recommendations

Audit findings and related recommendations for improvement are categorized below by the following key elements of major remediation projects: governance; risk management; project management; budget management and cost estimation; and, procurement management.

5.1. Governance

To deliver on the ultimate objective of the GMRP, effective project governance is critical. The audit examined whether an effective oversight body with clear governance protocols has been established, and whether information to support efficient and effective project oversight exists and is implemented.

An oversight body has been defined with clear protocols to support project governance. However, delays have been experienced in the implementation of components of the oversight structure.

The audit confirmed that the GMRP has had two distinct governance frameworks in place from fiscal year 2006-07 to July 2012. A consistent element in both frameworks has been the Oversight Committee, which is comprised of representatives from the Government of Canada and the GNWT. Under the newly revised governance framework, the Oversight Committee is supported by additional committees at both the operational and strategic levels.

5.1.1. Oversight Committee (2006-07 to July 2012)

Fiscal Year 2006-07 to August 2010

The Terms of Reference for the Oversight Committee indicate that its mandate is to provide a forum for the parties (Government of Canada and the GNWT) to cooperate on the administration of the Cooperation Agreement, specifically:

  • The Oversight Committee shall provide general direction and guidance to the GMRP Interim Office;
  • The Oversight Committee shall develop options and recommendations for a project implementation office to implement the approved Remediation Plan, if the parties agree to establish such an office; and,
  • If there is a joint decision to establish a project implementation office, the Oversight Committee shall provide direction and guidance to the project implementation office.

In addition, the Terms of Reference notes that the Oversight Committee will meet four times per year and that all decisions of the Committee will be recorded. Minutes or records of decision for the Oversight Committee do not currently exist.

September 2010 to July 2012

Overall responsibility for the Giant Mine Remediation Project lies ultimately with the Minister of AANDC, represented by the Assistant Deputy Minister (ADM), Northern Affairs Organization (NAO) In September 2010, a new project governance framework and management approach was developed to recognize the complexities of this project and the current weaknesses in the governance structure. In addition to the Oversight Committee, the Management Board and the Senior Project Advisory Committee (SPAC) were established. The Management Board was designed to provide oversight, issue resolution and decision-making for the GMRP. The SPAC was intended to provide an interdepartmental forum for the GMRP, strategic policy and regulatory advice, and guidance and senior level support. The SPAC is supported by an independent engineer and Peer Review Panel. Since its creation, the SPAC has not met and elements of its mandate are still in process of being finalized. A diagram of the current governance structure is provided in Section 5.1.2.

5.1.2. Project Governance (revised September 2010)

The following illustration presents the draft project governance structure, dated September 2010. While the project governance structure had not been approved as of the end of the audit fieldwork (July 2012), the Terms of Reference for the Management Board was recommended for approval in January 2011 and approved by AANDC's ADM, NAO on May 13, 2011.

Governance
Description of Governance

This chart depicts the project governance structure for the Giant Mine Remediation Project (GMRP). At the highest level, the project is overseen by the Project Leader (Assistant Deputy Minister, Northern Affairs Organization, AANDC), in collaboration with the Senior Project Advisory Committee. The Project Leader is supported by a Management Board that includes the Director General of the Natural Resources and Environment Branch (AANDC), the Regional Director General of AANDC's Northwest Territories Region, and a Regional Director General from Public Works and Government Services Canada (PWGSC). The Management Board works in collaboration with the Oversight Committee, comprised of the Regional Director General of AANDC's Northwest Territories Region and a representative from the Government of the Northwest Territories, and is supported by an Independent Review process that includes an independent engineer and a Peer Review Panel.

The Management Board governs the Project Leadership Committee, which includes the Project Director from AANDC (a representative from AANDC's Northwest Territories Region), the Project Sponsor (Director of Contaminated Sites Program, Northern Affairs Organization, AANDC) and the Project Director from PWGSC.

The Project Leadership Committee is responsible for overseeing Stakeholder Engagement, the Project Implementation Team and the AANDC Headquarters-based Major Projects Office. Stakeholder Engagement consists of the following four components: Community, Regulatory, Aboriginal and Environmental.

The Major Projects Office, which reports to the Project Leadership Committee through the AANDC Project Sponsor, is also linked to the Project Implementation Team.

The various committees and responsible parties listed above have been grouped into two broad categories: project governance and project management. The Project Leadership Committee contributes to both the project management and project governance of the GMRP. Those structures that support the Project Leadership Committee are considered as part of the project management of the GMRP, whereas the structures that lie above the Project Leadership Committee are considered as part of the project governance of the GMRP.

Through interviews and documentation review, it was noted that AANDC's ADM NAO has the ultimate accountability for the GMRP and that this has been the case over the entire audit scope period. However, as noted in Section 5.3.5 – Roles and Responsibilities, AANDC has not clarified roles and responsibilities which has led to uncertainty with respect to the leadership and execution of the GMRP.

Minutes from the Management Board meeting of October 19, 2010 noted the need for a SPAC. Interviews noted that AANDC and PWGSC had met once with the aim of forming the SPAC. Since the Committee has been formed, no formal meetings have been held and a Terms of Reference has not been approved.

5.1.3. Senior Project Advisory Committee

The membership of the SPAC comprises AANDC's ADM NAO (Chair), PWGSC's Associate ADM, Acquisitions and PWGSC's ADM Real Property. According to the draft Terms of Reference, the Committee is expected to meet bi-annually.

The GMRP is a significant and complex project that has been the Government of Canada's responsibility since December 1999. Governance and related documentation is critical for all projects, especially those like the GMRP whose scope will span many years. Given the likelihood of key personnel turnover throughout the life of the project, it is important to ensure records of decision and other key governance documents are prepared and retained for the continuity of knowledge and for future decision making.

5.1.4. Management Board

The Management Board is accountable to the Project Leader (ADM NAO) and is responsible for providing overall direction and guidance to the project. Activities of the Board include: monitoring the project's performance in accordance with the draft Project Charter; monitoring progress in accordance with the Project Execution Plan; and, providing information and advice to the Oversight Committee. Based on our review of a sample of meeting minutes from the Management Board, we confirmed that the Board considered elements of the project, including governance, operational matters, risk registry updates, project director roles and responsibilities, business continuity and senior management engagement.

Observation: The proposed governance structure appears to be reasonable as it includes senior management representation from the Government of Canada and the Government of the Northwest Territories through the Oversight Committee. In addition, the SPAC provides a mechanism for federal departments (AANDC and PWGSC) to provide strategic, policy and regulatory advice to the project. Since it was first proposed however, the governance structure has taken almost two years to develop and is still not finalized or fully implemented. A more effective governance structure at the inception of the project would have provided enhanced oversight and governance for the GMRP.

The GMRP is a significant and complex project that includes numerous stakeholders, including all levels of government, First Nation communities, and residents of Yellowknife. The project is currently engaged in numerous activities, including: site stabilization; care and maintenance; essential design work; and, regulatory applications to obtain a water license. Major projects like the GMRP require a strategy, approach, and ongoing and effective oversight to support the achievement of its objectives. The need for efficient and effective governance and oversight is also critical to determine and communicate the overall strategy for the project that will help inform decisions to be made in consideration of quality, time and cost.

Recommendation #1: The Assistant Deputy Minister, Northern Affairs Organization, should ensure that the proposed governance framework for the Giant Mine Remediation Project is finalized and implemented, with minutes and records of decision documented for the key governing bodies, such as the Oversight Committee and the Senior Project Advisory Committee.

5.2. Risk Management

To support the efficient and effective delivery of the GMRP, an effective risk management framework must be in place. The audit examined whether AANDC has a documented approach to risk management that includes risk identification, assessment, mitigation and monitoring.

Risk assessments are conducted through various mechanisms on a regular basis, but to date, have been largely operational in nature.

AANDC has identified a number of risks in a variety of documents over the scope of the audit from fiscal year 2006-07 to July 2012. The focus of risk management activities throughout this period has been on operational risks. The following is a summary of risk related information provided to the audit team:

  • A detailed risk register was initially developed in fiscal year 2005-06 and has since been updated on an annual basis. The risks included in this registry are operational in nature and include such considerations as dams, diversions, tailings and sediments, open pits, underground, water treatment, and infrastructure. This document is referred to as the GMRP Risk Registry. An example risk is "Dam 1" which is copied below to illustrate the approach taken:
Giant Mine (Giant Yellowknife Mines; Royal Oak Mines; A, B & C Shafts
Dams
Name: Dam 1
Number: 1.1
Abbreviation:  
Classification: Dam – water retaining
Location: Polishing Pond, 1500N, 8500E
Description: Polishing Pond Dam. The dam foundation is on tailings
Key Issue: Seepage or breach
Concerns: Premature release of treated water
Key Individual: Ron Connell, Ron Breadmore, site security
Key Information: SNP Reports, dam inspection reports – annual. Daily visual inspection
 
1.1.1 Dam breach due to dam failure leads to premature release of treated water
ConsequenceSeverityLikelyhoodRisk
Community/media/reputation Moderate Unlikely Moderate
Consequence costs Major Unlikely Moderately High
Environmental Impact Minor Unlikely Low
Human health & safety Critical Unlikely Moderately High
Legal obligation Major Very unlikely Moderate
Special Consideration Low Unlikely Low
Comment: Results in water release into B3 pit and into mine portal/ramp. ETP unable to operate, but able to divert water during period of dam reconstruction to avoid release. Dam has shown previous settlement and has been repaired. Pre-remediation implementation of dewatering system will result in increased worker activity in 138 Portal, B3 pit. {Pre-Migration Event Number=1.1.1}
Control: Regular dam inspections. Annual geotech inspection. Implementation of OMS, May 2007. Upgraded pumpback.
 

Through a comparison of the risk registers from 2009-10, 2010-11 and 2011-12, we noted the following observations:

  • Although risks are identified and a number of consequences for each risk are assessed for severity, likelihood and risk level, there is no overall assessment of the likelihood of the overall risk occurring.
  • There is no comparison of all of the risks identified to help identify interrelationships among risks and to help understand the relative priority of all risks in relation to one another.
  • Controls are identified in the form of the activities to be conducted in order to mitigate the likelihood of the risk occurring. However, current risk management tools and templates do not allow for the identification of who is responsible to complete the controls, the frequency of the control, and any required monitoring.

NAO program documentation from 2006 and 2009 discusses the Northern Contaminated Sites Program (NCSP) approach to risk management, and notes that strategic, operational and technical risks are identified and assessed for impact and likelihood. However, we did not observe any evidence supporting the identification of strategic project risks or project technical risks for the period 2009-10 to 2011-12. While we did note evidence of operational risks that had been identified prior to 2009, these risks were not detailed.

Other documentation reviewed included Detailed Work Plans (DWPs), Developers Assessment Reports, and Information Requests provided to the Mackenzie Environmental Impact Review Board. These documents consistently focused on operational risks.

A strategic project risk assessment workshop was completed in April 2012. The risk assessment focused on the objectives of the "current" phase of the project, i.e. Preliminary Design – Phase 2b (April 2012 – March 2015). The objectives of Phase 2b included: completing of the preliminary engineering and design to support Effective Project Approval; obtaining regulatory approval of the EA; completing the urgent site stabilization work; and completing the Freeze Optimization Study.

The workshop concluded that procurement is the most significant strategic project risk as it impacts three of the four phase 2b objectives. Planning and controls was determined to be the next highest strategic project risk, and human resources, governance, and community engagement were also considered significant strategic project risks. One of the next steps noted in the workshop included the monthly review of the risk register at Project Leadership Committee meetings.

NAO program documentation from 2006 stated that the NCSP has adopted AANDC's Integrated Management of Risks Framework. This suggests that the GMRP, which falls within the NCSP, has conducted a risk assessment that included strategic, operational and technical risks. The documentation reviewed by the audit team did not demonstrate that the project identified strategic project risks or project technical risks until recently. The consideration of risks beyond operational risks was addressed for phase 2b of the project in April 2012, which resulted in the development of the Strategic Project Risk Assessment to identify and assess key risks to the achievement of the overall objectives of the GMRP.

Observation: AANDC has not implemented a comprehensive risk management framework nor has it implemented a risk management plan that is monitored on a regular basis. In addition, the approaches to risk identification and assessment over the audit scope period have focused almost exclusively on operational risks.

The GMRP is a complex, multi year project with numerous stakeholders and requires a robust risk management framework and process. A risk management process should be designed, implemented, communicated and monitored to identify risks (strategic, operational and project level) to help inform project planning and demonstrate that mitigating actions are in place to address risks over the life of the project.

Standard and leading risk management practices include ongoing environmental scans, timely risk mitigation plans with clear accountability, and defined risk mapping (high, medium, low) for both likelihood and impact. The risk management process should be clearly documented, monitored for compliance, and risk management training should be provided for all project personnel.

The NCSP has drafted a Major Project Standards and Guidance Manual, comprised of 23 chapters. Chapter 6, Risk Management, includes project risk management tasks and requirements including: developing the risk management plan; conducting a strategic project risk assessment; establishing project risk management tools; and, developing the project risk tolerance matrix. Roles and responsibilities are identified for the project governing bodies and essential positions. This document was still in draft as of July 2012 and has yet to be implemented for the GMRP.

Recommendation #2: The Assistant Deputy Minister, Northern Affairs Organization, should ensure that a comprehensive risk management framework (including strategic and project level risks) and a risk management plan are established for the Giant Mine Remediation Project to monitor risks on a regular basis.

5.3. Project Management

AANDC and PWGSC are both currently accountable for certain project management responsibilities for the GMRP. The audit examined samples of project management documentation developed and/or in place to determine whether such documentation supports the effective and efficient achievement of the GMRP's objectives.

Project management processes, practices, frameworks and tools to support the effective and efficient achievement of the GMRP's objectives have been enhanced throughout the period of this project and are in varying stages of implementation. Despite the observed enhancements, areas for improvement were noted with respect to: roles and responsibilities; schedule development, integration, management and reporting; and change management.

Both AANDC and PWGSC have developed a number of documents to support and facilitate the project management process. Project timelines, frameworks, flow charts, and control documents existed for the majority of the project timeframe.

5.3.1. Major Project Office

To further strengthen project management, a Major Project Office (MPO) was initiated by AANDC. From the MPO, a Major Project Standards and Guidance Manual – 23 Chapters in total with 28 accompanying Process Maps – has recently been developed. The manual's documents, along with the Project Charter, developed by AANDC, and the Project Execution Plan (PEP), developed by PWGSC, contain within them relatively strong content in comparison to typical industry standards. However, at the time of this report, these documents were in "Draft" form and therefore have not been fully operationalized.

The Major Project Standards and Guidance Manual, Project Charter and PEP are voluminous. As a result, implementing and sustaining these documents and processes into the GMRP environment will be a very significant undertaking and it is unclear at this time, how or if the current organization is capable of sustaining these practices.

5.3.2. Project Reporting

Project reporting is currently prepared on a quarterly basis, capturing the cost of annually budgeted contracted services. No regular overall project level reporting (Key Performance Indicators [KPIs], cost, schedule, scope, and quality) was evidenced during the audit. Ongoing monitoring and measuring of KPIs, schedule, cost, scope, quality, progress, and stakeholder engagement are key metrics to support the success of a project.

During the audit, no clear reporting of the phases of the project appeared to be available to analyze the project. While annual activity-based reporting exists for key project activities, such as care and maintenance and Freeze Optimization Study and Quarterly reporting exists to monitor annual project progress and cost, limited overall project based reporting and KPIs exists.

5.3.3. Change Control

Change Control Process

Draft guidance documentation on change control has been prepared by AANDC, and is currently being implemented, as part of the MPO implementation and is consistent with leading practices. Change control management requires maintaining the integrity of the performance measurement baselines and coordinating changes across functional project areas. Projects lacking a formal change control system will be at risk of major cost overruns, contractor disputes, and the potential to overbuild/overspend.

Observation: AANDC has begun the implementation of a formalized change management process to control requested changes to project budget, scope, schedule and/or other deliverables. While some of the change management forms and procedures are currently in draft, these procedures and forms have been used to communicate, consider, and approve project changes over the last quarter.

Managing project changes within a billion dollar project such as this require change control systems that define the policies and procedures by which the project configuration (i.e. scope, timing, cost, and quality) can be changed. A change control system is an integral part of the overall project management system. A change control system should include, at a minimum:

  • A way to identify changes to the scope or configuration of the project;
  • Documentation of the change and its impacts;
  • A tracking mechanism such as a log or register;
  • Regular notification and reporting of changes to the project team; and,
  • Established levels of approval authority based on dollar thresholds and risks for each change.

Change control management for the GMRP should: involve managing changes for contract or vendor purchase orders (which may involve changes to scope requirements, schedule duration, cost, or means and methods); require notification by one party to another; and be reported through a notification of change or construction change directive. These changes may be driven by an event, alteration in regulation, risk or safety requisite, or change in stakeholder requirements.

The audit team noted that a change control process was implemented in May 2012 and that 12 change requests have been processed to date. Discussions to finalize components of the change control process were evident in minutes of Project Management Committee (PMC)Footnote 1 meetings held during Q2 2012. The audit team reviewed two approved change requests and based on our review, it appears that changes along with rationale, risk levels, potential impacts, and approval requirements are captured. A change approval matrix based on risks, dollar thresholds and types of changes, is also evident and attached to the change request form.

Change Control Reporting

Controlling and reporting are an essential part of scope, schedule and cost management. In order to maintain control, it is important to have an effective reporting framework in place by AANDC. This framework would include in-depth and on-going communication as part of the responsibility of project management to ensure that relevant information from other project processes (e.g., quality management processes or financial management processes) is taken into consideration, evaluated and consolidated.

The change request form has an area to allow the AANDC Project Controls Manager to indicate whether change details are added to the Change Request Register and/or communicated to Management Board, Project Execution Team, Project Leadership Committee, etc., as required. Currently, change requests are added to the Change Request Register, which is then used during the monthly Project Management Committee Meetings to communicate changes to work package managers. Although they are summarized in a central document, changes that impact overall project cost, scope, and schedule are currently not reflected at the overall project level such as an overall project cost schedule. Details to further formalize the tracking and reporting of changes are currently being refined and finalized. Given the complexity and length of time of the GMRP, a structured change control process that is consistently implemented to update changes in overall project scope and/or cost is critical.

The scope management plan for the GMRP should detail reporting requirements (internal and external) on scope management and project progress to reflect authorized and pending scope changes. This should take the form of a summary or an extract from the central scope change register. The Project Leadership Committee and other stakeholders will also need to receive specific reports on scope changes when required.

As part of an effective change control process, management reporting should be initiated to track changes to the scope of the GMRP, addressing the key objectives of the change control procedures:

  • Controlling change throughout all phases/stages of the project;
  • Transparency throughout the change process;
  • Implementing justified and authorized cost changes only;
  • Reporting on project changes, corrective actions taken and lessons learned;
  • Establishing roles and responsibilities for the change control process;
  • Understanding and acceptance by management of the impact of proposed changes;
  • Authorization of proposed changes at the management level; and,
  • Communicating to all relevant stakeholders as well as relevant project leaders and contractors.

5.3.4. Project Schedule Management

AANDC's project schedule management should include the processes required to help ensure timely completion of the project within the requirements for quality, cost, time, environment, and health and safety. Primary schedule management processes generally include: activity definition; activity sequencing; activity duration estimating; schedule development; and, schedule management.

Observation: Project scheduling practices and related project reporting are not clear or integrated to effectively support the delivery of project activities and the achievement of the GMRP's objectives.

Based on our review of a sample of project files, we noted the following discrepancies:

  • No clear "approved" baseline schedule was prepared according to the requirements of the project. Typically, the owner of the project retains the right to review and approve the baseline schedule for coordination and monitoring purposes. Due to the issues concerning the clarity of roles and responsibilities, which is explored in Section 5.3.5., it is unclear who the owner of this schedule would be at this time.
  • No clear Critical Path Methodology, which can help to determine the tasks required to complete a project, the time required for each task and the interdependencies of tasks, was noted. Reporting appears to be generated for review by the project's stakeholders for the GMRP, providing no overall view of the state of the progress against the baseline and critical path.
  • There is limited clarity at this time around the role of AANDC or PWGSC to manage revisions to the overall project remediation schedule. Although the AANDC Project Charter clarifies AANDC's project schedule tracking and reporting role for the GMRP overall, most project schedule reporting is currently being produced by PWGSC. While this inconsistency is likely due to the nature of the current project schedule reporting on the GMRP, the limited overall project schedule reporting by AANDC has supported this inconsistent understanding of roles. This lack of understanding is further enhanced by documentation that describes PWGSC's role, if this documentation is not considered as a subset of the overall project.

    PWGSC's role, per the PEP, identifies:

    "…Schedule Plan - 4.1 - Activity Definition and Sequencing - The following sections outline the activities and schedule relationships for GMRP. Preliminary Work Breakdown Structure (WBS) and schedule have been produced for Phase 2. The WBS and Schedule for Phase 3 will be developed in concert with the Main Construction Management and Engineer of Record near the end of Phase 2, prior to the initiation of Phase 3…"
  • Also, PWGSC documentation notes that it is project managing the delivery of construction management services (i.e. contractors), and thus, would in effect be responsible for controlling the project schedule at the current level of project schedule reporting that exists.

Within the delivery of a project, schedule change management and reporting is necessary for the planning, scheduling, and approval of schedule changes resulting from the change control process (which, as noted earlier, was not evidenced during this review). The primary objectives of managing schedule changes are to:

  • Identify direct and indirect changes to the schedule based on scope changes;
  • Perform a schedule impact and cost estimate for the identified change; and,
  • Track and communicate schedule and cost impacts to the project team through effective and timely reporting.

The audit did not evidence any documents or reporting (except those in a "Draft" state in the MPO's Major Project Management Standards and Guidance Manual) that provided any clarity as to the processes that are currently in place and being followed at either AANDC or PWGSC.

Schedule integration management requires the active involvement of all project stakeholders to remain informed about the status of the project and any delays that may impact schedule performance and how individual project schedules impact each other and the overall GMRP.

During the audit, there was no schedule integration management plan or reportingthat contained the following major elements:

  • Integrated or linked owner, contractor, or sub-contractor schedule information;
  • Project and level integration process and requirements; and,
  • Project integration meetings/updates.

5.3.5. Roles and Responsibilities

Interviews with AANDC senior management and a review of project team structure documentation identify AANDC as having ultimate responsibility for the overall project and the achievement of related project objectives. However, it was noted during interviews with AANDC team members in Yellowknife that no known input from AANDC was included in the development of PWGSC's PEP and that roles for the GMRP remain unclear. The current Project Charter and PEP remain in draft as of the date of this report.

Observation: Currently, AANDC has not formally approved the existing draft roles and responsibilities document that outlines the overall leadership and execution of the GMRP.

RACI (Responsible, Accountable, Consulted, and Informed) and project organization charts have been developed, but are in "Draft" status only and both of these draft documents appear to have been developed by AANDC or by PWGSC independently of one another. The "leadership" roles referenced for the project are based on a committee style approach. As a result, no clear understanding of roles and responsibilities can be determined. Without a clear understanding of the roles and responsibilities and an integrated view of the GMRP, the lead project manager may not be appropriately positioned to protect the project from unnecessary changes and loss of resources.

One exception to this for the leadership and execution of the GMRP can be found in the presentation material tabled at the May 15-17, 2012 meeting of the Project Management Committee. These materials (not referenced as "Draft") provide some clarity with respect to the identification of project governance and management. While this interim document is the clearest of all the documentation reviewed by the audit team with regard to roles and responsibilities, there is no conclusive evidence to support that this organizational structure is now in place and whether these teams and their leads are empowered to move forward.

Recommendation #3: The Assistant Deputy Minister, Northern Affairs Organization, should ensure, as part of the Major Project Office implementation, that:

  • Clear roles and responsibilities between PWGSC and AANDC for the Giant Mine Remediation Project are defined, documented, communicated, agreed upon, and finalized such that the project environment can focus on optimizing, resourcing, and delivering the project in accordance with its objectives. The resulting final organizational structure should be as streamlined as possible to support a strong and clear focus on the project.
  • A proper baseline schedule (prepared according to industry standards), an integrated project schedule, a management plan and a reporting regime are established to link all parties together into a standardized Critical Path Methodology. The establishment of these documents will help to ensure that a clear window to the overall position and progress for the Giant Mine Remediation Project is provided to the project management team(s).
  • A rigorous and effective change management program be finalized to support the consideration and reporting of changes to the scope and total estimated cost of the Giant Mine Remediation Project.

5.4. Budget Management and Cost Estimation

To support the efficient and effective delivery of the GMRP, an effective budget management and cost estimation process must be in place by AANDC. The audit examined the project's cost estimating practices and tools as well as the budget management and mechanisms to assess, monitor and report on the accuracy and completeness of project cost.

Budget management and cost estimation have been conducted to support project estimates, expenditures and monitoring on a periodic basis; however, improvements can be made to strengthen the communication of cost estimates and related vulnerabilities, and the maintenance of a Master Project Total Estimated Cost to Complete document.

A number of project complexities exist that have had a compounding effect on the estimated project cost and related variances since 2006 due to their impact on the project timeliness as well as inflationary and escalation impacts. The audit noted various project estimation practices and tools have been established to support consistent cost estimation. These tools and practices have been enhanced throughout the audit period and the clarity of communicated variances estimates has improved.

The GMRP's approach to cost estimation, cost reporting and monitoring is guided by the following documents:

  • AANDC NAO's draft Corporate Procedures Manual, which details the procedures to follow in the planning, implementation, monitoring, reporting, and review of project phases.
  • AANDC NAO's draft Cost Estimation Guide, which was developed with reference to the Treasury Board Guide to Costing, provides a framework intended to enable the generation of consistently reliable and dependable cost estimates. The Cost Estimation Guide uses the cost estimate classification of accuracy of the Association for Advancement of Cost Engineering International (AACE).

Contingency estimates for the GMRP are developed (i.e. 5%, 15%, 30%, etc.) based on set criteria and a 10% contingency for support services, and are reflected in the current (2012) revised total estimated project cost of $904M. The total cost estimate includes contingency amounts that have been prepared with consideration of the following cost estimate practices:

  • Government of Canada Estimate ClassesPWGSC uses estimate classes to identify the level of certainty regarding project definition, which is expressed as a percentage potential variance (A [+-5%], B [+-10%], C [+-20], and
    D [+-40%]).
  • Industry Standards for Cost Estimation – The draft March 2012 Cost Estimate Guide utilized the current AACE industry standard model for cost estimation and provides a transition grid to match the PWGSC Estimate Class Model.
  • Independent Peer Review – The Cost Estimate Report, prepared for AANDC by an external contractor, was reviewed for reasonableness of labour rates, material prices, and equipment costs by an independent, third party organization with deep experience in mining.

The initial (2006) cost estimate of $348M for the GMRP was broken down but provided limited detail and no classification of cost certainty. Costs included general ongoing care and maintenance, consultation, regulatory approvals, site assessment, and design work to complete the preliminary design and the development of a Substantive Costs Estimate.

Cost estimates are re-assessed and adjusted on an annual basis by the Project Manager, with assistance from PWGSC, as new technological advances occur and as new information becomes available on inflation, escalation, the extent of contamination found, and the progress towards remediation. According to AANDC management, a rigorous change control system is being introduced to control scope, schedule and budget through an appropriate delegated management authority. This change management system was implemented in May 2012 as noted in Section 5.3.3 – Change Management.

An AANDC Project Charter, PEP and project plans have been developed by the Northern Contaminated Sites Program to plan, execute, control, and manage processes, standards, as well as the project schedule, scope and budgets. These baseline documents are the critical planning documents for helping to ensure that the current detailed budget of $904M is monitored, controlled and tracked. The PEP describes the contract implementation plan in terms of scope, schedule, cost, risks, procurement, quality, safety, human resources, and communications.

Phase Work Plans (PWP) are compiled to group the activities and deliverables to be performed during a specific life cycle phase into a work plan. It extracts those sections from the PEP that apply to the upcoming phase and is not a source or control document on its own, like the Project Charter or PEP.

The purpose of the PWP is to define the funding and budgetary requirements for the current phase only. In the case of multi-year project phases, the PWP will be revised annually to coincide with the fiscal cycle. Based on our review of relevant documentation, we observed improvements in quarterly project reporting on annual project cost. The 2011-12 Quarterly Reports provide: significant detail on the original budget, adjusted budget, annual commitments to date, year-end forecast, variance numbers, variance explanations, Total Project Cost Estimate amounts for the fiscal year; and provide a breakdown by budget categories and financial accounts, Specific Service Agreement (SSA), and DWP.

Budgets are further tracked through the annual Cost Breakdown Structure, which tracks the current detailed budget. Monthly financial progress reporting to the Project Management Committee and Project Leadership Committee are used to track and control budgets articulated in the Cost Breakdown Structure. Quarterly financial progress reports are reported to the GMRP Management Board and to NCSP senior management.

Observation: There is no evidence that an overall Master Project Total Estimated Cost to Complete document, that incorporates key assumptions, changes in key assumptions, variances, and other critical information, is maintained and updated on an ongoing basis.

Examination of the cost estimates and related budgets indicate that from 2006 to 2012, overall project cost estimation occurred at three separate points in time: 2006, 2009 and 2012. As a result of the discrete process used to estimate the total project cost and the varying levels of detail in each period, the audit team encountered difficulties tracking changes in overall project cost estimates and the underlying support for these changes from 2006 to 2012. Documentation with respect to key assumptions and their potential vulnerabilities was extremely limited in 2006 and 2009.

An effective Master Project Total Estimated Cost to Complete document should enable: overall project cost monitoring; the detailing of changes and associated reasons for changes; actual overall cost variance reporting; and, forecasted cost variances. In the absence of such a document, the audit team found that the current reporting and cost tracking tools are not sufficient to readily enable an assessment of the overall project cost against the current cost estimate at any point in time, in a consistent manner.

Recommendation #4: The Assistant Deputy Minister, Northern Affairs Organization, should ensure, through consultation with the Chief Financial Officer, that budget management and cost estimation practices for the Giant Mine Remediation Project are strengthened by:

  • Developing a Master Project Total Estimated Cost to Complete document that includes a cost variance simulator, such as a 'Monte Carlo Simulation', to highlight the potential and probable deviation ranges in total cost estimates. The establishment of such a document would help to communicate key assumptions in the cost estimate and variance vulnerabilities to all project stakeholders.
  • Incorporating multi-year requirements of future work and costs in a Master Project Total Estimated Cost to Complete document based on industry best practices and the Association for the Advancement of Cost Engineering International Cost Estimate Classification system. Roles and responsibilities for maintaining, reviewing, and monitoring this document should be defined and communicated.

5.5. Procurement Management

To support the efficient and effective delivery of the GMRP, a strong procurement process must be in place by AANDC. The audit examined the procurement practices and processes in place at AANDC to support project approval and planning, the division of roles and responsibilities for the management of procurement contracts, and monitoring of supplier performance and targets. The audit also examined PWGSC procurement documents, such as contracts, strategy and risk plans, evaluation and results, and project monitoring tools and reports, relating to the Base Care and Maintenance and Freeze Optimization Study contracts.

Mechanisms are in place to identify, assess, evaluate, approve, and communicate projects for procurement; however, improvements can be made to reflect the multi-year nature of the GMRP and to strengthen the consistency of amendments and cost revision practices.

AANDC is responsible for managing the remediation efforts as part of the GMRP and is the overall Project Owner. PWGSC, the procurement agent for all large dollar value goods and services purchased by the Government of Canada, leads all AANDC procurement activities in excess of $2 million. In the fiscal year 2011-12, PWGSC-led procurement activities on the GMRP were approximately $19 million or 86% of the total project expenditures of $22 million. The remaining $3 million was managed and procured internally by AANDC.

From the listing of PWGSC's work plan for 2011-12, the audit selected and examined two ongoing contracts: i) Base Care and Maintenance (delivered through a contract with Deton'Cho Nuna), and ii) Freeze Optimization Study (delivered through a contract with Deton'Cho Nuna and sub-contracted to SENES/SRK). For each contract, the audit examined the efficiency and effectiveness of procurement planning and project approval, and performance monitoring and reporting.

5.5.1. Planning and Project Approval

Project needs for the GMRP are communicated in two main documents: PWPs and DWPs. Long-term objectives and multi-year contracts are communicated by the GMRP team to PWGSC through PWPs. The most recent PWP remains a draft working document dated March, 2012. DWPs form the basis of work to be completed in the current fiscal period and is the basis for the annual SSA. Change orders for any changes made throughout the year to the project budget or project scope of are also contained in DWPs. We noted that the PWP and change order process did not actually exist in 2006 and, to date, they remain in draft form. Thus, for the period under audit, project needs were primarily communicated to PWGSC through the DWP.

The other key planning documents are the Service Level Agreement (SLA) and SSA noted in the preceding paragraph. The SLA is the overarching agreement, authorized by the Director General, Natural Resources and Environment Branch at AANDC and the Regional Director General, Western Region at PWGSC. The SLA outlines the terms, conditions, and principles of the procurement services provided by PWGSC. The SSA is the agreement between AANDC and PWGSC for the specific services that PWGSC will undertake to procure. Taken together, the SLA and the SSA, along with the PWP, and DWP, form the basis of AANDC's communication to PWGSC of its procurement needs.

Observation: The assessment and evaluation of working projects is done annually and does not reflect the multi-year nature of projects. Project related components, such as risks, performance measures and monitoring plans, and reasonable budget level details, are not captured in one document and are not clearly communicated to the AANDC Project Manager.

Based on our review of a DWP developed at the initial stages of the project, we noted that the work plan included: a rationale to support the need for care and maintenance of the Giant Mine; related work completed and planned for the year; the actual cost to date; and, preliminary estimates for future fiscal years. This is also consistent with the 2010-11 DWP that outlined the rationale and objectives of the Freeze Optimization Study, related planned activities, and impact on budget variances. While both DWPs reviewed contained high-level estimated costs for future years, these estimates were provided for a specific year rather than considering project needs on a multi-year basis. Without a multi-year focus, the existing work plans are insufficient to demonstrate key milestones and deliverables along with the anticipated level of effort by resource category and project timeline over the life of the project.

The budget estimate captured in the annual DWP is shown in one line item by Federal Contaminated Sites Action Plan (FCSAP) category, (e.g. 'Care and Maintenance'), and does not tie back to expected deliverables or major milestones over the life of the project. In years subsequent to 2009, we noted that the DWP contained more detailed budgeting by tasks for a specific year under each project as well as actual versus budgeted costs and the overall estimated cost to complete the GMRP.

Observation: There are inconsistencies in the manner in which amendments and cost revisions are justified in the SSA. Currently, agreement amendments and extensions to completion dates are to be approved in writing in the SSA. A review of the SSA noted that the manner in which SSAs are completed varies considerably, and has included: a brief description of additional work required; an attachment of additional information of unfunded activities broken down by work orders and project levels; and, in some cases, no project description at all. To fulfill its accountability in the use of federal funding, AANDC has a responsibility to justify and provide appropriate reasoning behind significant cost revisions or additional funding requests to allow delegated authorities to understand the nature and cause of the amendment, whether it was a result of inappropriate planning, unanticipated occurrences, or inefficiencies in contracting, for instance, rather than variables outside the control of project management (e.g. new technology, inflation, escalation). Having a more rigorous process to document changes to cost and work schedules, providing transparent rationale for these changes, and seeking appropriate authority for change requests, will help ensure that funding is provided for eligible and reasonable expenditures.

Procurement roles and responsibilities

AANDC and PWGSC work closely together on all aspect of procurement to help ensure that applicable legislation and Treasury Board policies are respected. There are several mechanisms that outline the roles and responsibilities between the two parties over the management of procurement contracts, including:

  • The GMRP Contracting Strategy (summarized within the draft Project Charter dated March 2012). This strategy outlines the contracting framework required to deliver activities supporting the GMRP, such as the treatment of existing, and awarding of new, short- and long-term contracts. Roles and responsibilities for governance and project management are also defined in the draft Project Charter. The specific procurement roles and responsibilities are assigned to the Project Director, AANDC and the Project Director, PWGSC, and consist of:

    • (i) approving the procurement strategy, providing transparent information to the Management Board on the progress of the project, and accepting and approving deliverables from PWGSC (AANDC Project Director); and,
    • (ii) developing and implementing the procurement strategy, being accountable for monthly reporting, contracting, and overseeing Project Management/Construction Management teams. Both the Project Directors at AANDC and PWGSC are members of the Project Leadership Committee.

Observation: Roles and responsibilities over the management of procurement contracts were not clearly defined in years prior to 2011-12 and opportunities exist to improve the clarity of roles and responsibilities.

Based on the recent SLA dated March 2011, PWGSC's main role is to provide contracting and project management services to AANDC as a "Service Provider" through a separate SSA for each project. PWGSC's expected services, authority levels, and key project considerations are detailed in the SLA.

Any restrictions to delegated authority are agreed to by both AANDC and PWGSC, and written in the SSA. As well, PWGSC cannot take unilateral action or decisions on procurements related to the GMRP without consultation with and consent of the AANDC authority. AANDC remains the custodial Department and has ultimate authority the GMRP and has the responsibility and accountability for long-term management of all residual risks.

AANDC retains responsibility for internal, external, and multi-stakeholder communications on behalf of the Federal Government, but any external communications that relate to or impact PWGSC will be approved by PWGSC prior to their release.

Certain key roles and responsibilities were not clearly defined in 2005-06. While these roles are better defined within the current SLA and draft Project Charter, there continues to be some lack of clarity in the delineation of specific project management roles between PWGSC and AANDC, as each project procured by PWGSC has an AANDC Project Manager and a PWGSC Project Manager. As the number of remediation tasks evolve over the life of the project, it is even more critical that AANDC makes a clear distinction between the roles and responsibilities of both parties from a project management perspective, in writing and in practice.

Procurement Planning and Contracting Process

The majority of AANDC's procurement activities are in excess of $2 million and are therefore managed by PWGSC. The AANDC's GMRP risk profile indentified that the procurement process, in its current form, may not be adequate to support the project requirements. This was confirmed through discussions with AANDC program managers.

5.5.2. Performance Monitoring and Reporting

Monitoring and reporting on the progress of projects enables management to exercise their project management responsibilities to help ensure that each contract is successfully contributing to the expected results of the GMRP. The audit examined the efficiency and effectiveness of performance measurement, monitoring, and reporting practices and processes to oversee the status (quality, cost, and time) of each contract, and to demonstrate the impact of any results, issues, or changes on the overall project.

Reporting and monitoring

AANDC uses a number of procedures and tools to monitor the status of its projects, including:

  • Monthly Cost Variance Reports (MCVR) prepared by the Project Controls Group to track year-to-date (YTD) actual expenditures against the YTD budget approved by cost centre.
  • Project Monthly Reports provided to AANDC by the Senior Project Manager from PWGSC to consolidate and report on health and safety statistics, specific items for AANDC attention, status report by FCSAP category, program status summary, schedule status summary, program status summary, and YTD financial status summary.
  • Weekly conference call meetings on the Freeze Optimization Study to discuss roles and responsibilities, allocation of work, updates on the freeze system, study planning, and management of data and instruments used to conduct the freeze operation.

Observation: There are opportunities to strengthen the existing monitoring tools and procedures to enable more relevant monitoring and reporting of expenditure variances and performance measures and targets for GMRP contracts.

We noted that some monitoring tools, specifically the MCVR and the Quarterly Reporting of Activities Status Schedule and Costs, could be improved to provide more complete tracking of project progress and expenditures against plan. The Quarterly Reporting of Activities Status Schedule and Costs reports on the status of the project and on financial information. There is the potential to include information on the status of the project against key deliverables and milestones outlined in the SSA to determine how the project is tracking against planned timeframes and expected work deliverables, and whether procurement services are delivered in a timely manner.

In accordance with the SLA, each SSA is to have specific project level performance measures and targets, beyond the general performance measurement indicators that are applied to all contracts procured by PWGSC. We observed that the SLA does not require that project monitoring be risk-based. A risk-based approach would consider such factors as the nature and complexity of the project, the total amount procured variances between actual expenditures and budget, scope changes, and project progress delays. Projects with higher risks would require more stringent monitoring procedures, such as the weekly meetings for Freeze Optimization Study, than projects with lower risks. We understand there are draft plans in the March 2012 PEP for PWGSC to prepare contract risk management plans for each planned procurement project to identify major risks that could impact cost, schedule, or quality. This plan has not yet been developed and we encourage management to develop and implement defined monitoring procedures for each project based on identified contract risks. PWGSC indicated that it completes a Risk Management Plan (RMP) for every contract that it issues and any contract amendment refers back to the RMP.

Recommendation #5: The Assistant Deputy Minister, Northern Affairs Organization, should ensure, through consultation with the Chief Financial Officer, that the current approach to procurement management for the Giant Mine Remediation Project is strengthened by:

  • Improving the tools and guidance to support contract monitoring and reporting of expenditure variances and performance measures and targets.
 

 

6. Conclusion

Overall, we found that AANDC has made progress in building and improving upon its project management practices to help ensure the achievement of its planned outcomes on the GMRP. AANDC has introduced and/or drafted a number of sound practices to strengthen its activities in the areas of governance, risk management, project management, cost estimation and budget management, and procurement. Processes and tools that have been implemented or drafted by AANDC to improve practices include: project governing bodies; operational risk assessment practices; a Major Project Office; improved cost estimation practices and budget reporting; and Service Level Agreements to PWGSC procurement requirements.

Although a number of sound practices have been implemented, or are in the process of being implemented, inconsistencies were identified in the extent to which new practices and processes have been fully operationalized. In this regard, five key recommendations were identified through the audit.

 

 

7. Management Action Plan

RecommendationsManagement
Response / Actions
Responsible
Manager (Title)
Planned
Implementation Date
1. The Assistant Deputy Minister, Northern Affairs Organization, should ensure that the proposed governance framework for the Giant Mine Remediation Project is finalized and implemented, with minutes and records of decision documented for the key governing bodies, such as the Oversight Committee and the Senior Project Advisory Committee. The GMRP team accepts this recommendation. The proposed GMRP governance framework will be finalized, approved by the Project Leader, and fully implemented to operational status. This applies to all current governing bodies included in the proposed governance framework, specifically Senior Project Advisory Committee, Management Board, and Project Leadership Committee. Operational status for each body will include formalized and approved Terms of Reference, agenda, minutes and records of decisions (where not already on-going). Assistant Deputy Minister, NAO September 27, 2012
2. The Assistant Deputy Minister, Northern Affairs Organization, should ensure that a comprehensive risk management framework (including strategic and project level risks) and a risk management plan are established for the Giant Mine Remediation Project to monitor risks on a regular basis. The Giant Mine Remediation Project Team will expand the scope of the risk assessment process to include technical and strategic risks and incorporate a risk management plan. This more robust risk management approach will initially focus on the current life cycle phase (PPA Phase or Phase 2b/2c – Preliminary/Detail Design) and evolve to include the total project scope before the Effective Project Approval is sought from the Treasury Board. The Risk Management Plan will be included in the Project Execution Plan (PEP) for the project. Assistant Deputy Minister, NAO PPA Phase Project Execution Plan:
March 31, 2013

Overall GMRP Project Execution Plan:

December 31, 2014
3. The Assistant Deputy Minister, Northern Affairs Organization, should ensure, as part of the Major Project Office implementation, that:
  • Clear roles and responsibilities between PWGSC and AANDC for the Giant Mine Remediation Project are defined, documented, communicated, agreed upon, and finalized such that the project environment can focus on optimizing, resourcing, and delivering the project in accordance with its objectives. The resulting final organizational structure should be as streamlined as possible to support a strong and clear focus on the project.
  • A proper baseline schedule (prepared according to industry standards), an integrated project schedule, a management plan and a reporting regime are established to link all parties together into a standardized Critical Path Methodology. The establishment of these documents will help to ensure that a clear window to the overall position and progress for the Giant Mine Remediation Project is provided to the project management team(s).
  • A rigorous and effective change management program be finalized to support the consideration and reporting of changes to the scope and total estimated cost of the Giant Mine Remediation Project.
The GMRP team accepts this recommendation. As part of the Major Project Office implementation, the following specific actions will be taken:
  • The Giant Mine Remediation Project Team will finalize the draft Project Charter which lays out the roles and responsibilities.
Assistant Deputy Minister, NAO November 15, 2012
  • The individual organizational charts for both AANDC and PWGSC will be finalized and integrated into a single GMRP organizational chart. Staffing within AANDC is underway.
March 31, 2013
  • The current draft MPO Standards and Guidance Manual will be formalized, approved and implemented by the GMRP team, including PWGSC, AANDC, service providers and contractors.
March 31, 2013
  • A comprehensive Schedule Management Plan will be developed, which includes an integrated project master schedule for the total project duration, as well as a baseline schedule for the five-year period of the PPA Phase. The baseline schedule will form the basis for monitoring project performance/progress and controlling schedule changes. The Schedule Management Plan will form part of the PPA Phase PEP and finally overall GMRP PEP.
March 31, 2013
  • The current draft Change Control Process will be finalized in accordance with the MPO Standards and Guidance Manual to track changes to scope, schedule, cost, technical or other approved baselines. This plan will be part of the Project Control Plan, which will be included in the PEP.
October 31, 2013
4. The Assistant Deputy Minister, Northern Affairs Organization, should ensure, through consultation with the Chief Financial Officer, that budget management and cost estimation practices for the Giant Mine Remediation Project are strengthened by:
  • Developing a Master Project Total Estimated Cost to Complete document that includes a cost variance simulator, such as a 'Monte Carlo Simulation', to highlight the potential and probable deviation ranges in total cost estimates. The establishment of such a document would help to communicate key assumptions in the cost estimate and variance vulnerabilities to all project stakeholders.
  • Incorporating multi-year requirements of future work and costs in a Master Project Total Estimated Cost to Complete document based on industry best practices and the Association for the Advancement of Cost Engineering International Cost Estimate Classification system. Roles and responsibilities for maintaining, reviewing, and monitoring this document should be defined and communicated.
The GMRP team accepts this recommendation. The following actions will be implemented in consultation with the Chief Financial Officer:
  • A Cost-Management Plan will be developed for the overall GMRP, which includes a Master Project Budget (Total Estimated Cost to Complete), estimate basis, assumptions and cost-management processes. This Master Project Budget will utilize probabilistic simulation techniques such as a 'Monte Carlo Simulation' to establish potential and probable estimate deviation ranges. This Cost-Management Plan will form part of the overall GMRP PEP.
Assistant Deputy Minister, NAO



Chief Financial Officer
December 31, 2014

(Overall GMRP)
  • The Cost-Management Plan for the overall GMRP will incorporate multi-year requirements for future work, which will focus initially on the five-year period of the PPA Phase and finally for the total project cost. The Cost-Management Plan shall include clear roles and responsibilities for ownership of the document.
March 31, 2013 (PPA Phase)
5. The Assistant Deputy Minister, Northern Affairs Organization, should ensure, through consultation with the Chief Financial Officer, that the current approach to procurement management for the Giant Mine Remediation Project is strengthened by:
  • Improving the tools and guidance to support contract monitoring and reporting of expenditure variances and performance measures and targets.
The GMRP team accepts this recommendation. The following actions will be implemented in consultation with the Chief Financial Officer:
  • A Procurement Management Plan will be developed to ensure funds that are being expended by PWGSC are achieving objectives and that variances are integrated into the Giant Mine Remediation Project's Performance Measurement Strategy and reported regularly to the Project Leader and Senior Project Advisory Committee.
Assistant Deputy Minister, NAO March 31, 2013
Materiel and Assets Management Division will implement a process where the monitoring of the file will be done quarterly. As of September 4th, all call-ups will be done by MAMD, therefore the Procurement Officers will be able to monitor all expenditure on the files. Chief Financial Officer September 4, 2012
The Procurement Officer will also request adequate justification for all amendments and make sure that the checklist is completed and reviewed. Chief Financial Officer September 4, 2012
 

 

Appendix A: Audit Criteria

A value for money audit plan was developed using a risk-based approach to identify key risks facing the achievement of the GMRP objectives and expected results. Principle techniques used to identify the key risks included interviews with management and key stakeholders, and a documentation review. Key risks were then linked to the core processes and practices in place within AANDC and PWGSC that are designed to mitigate these risks. This information was then used to determine the audit criteria to support the objectives and scope of the audit.

Audit Criteria

Governance

1.1 An effective oversight body is established with clear governance protocols and information to support efficient and effective project oversight.

Project Management

2.1 Project management processes, practices, frameworks and tools, and human resource capacity exist and are consistently used to support the effective and efficient achievement of project objectives.

2.2 Project reporting mechanisms are relevant, timely, and useful to efficiently and effectively support project issue escalation and information for decision making.

2.3 Project activities are planned and coordinated in an efficient and effective way to meet project objectives and stakeholder needs.

2.4 A risk-based approach to the monitoring of and follow up on the progress of the project (Cost, Schedule, etc.) exists and is followed.  Results are communicated and reported on a regular and timely basis to support effective and timely management decision-making and governance.

2.5 Efficient and effective processes exist to support the ongoing identification and remediation of environmental, health and safety needs.

2.6 Authority, responsibility and accountability are clear, communicated and an effective project charter exists.

2.7 Mechanisms exist to share lessons learned and support change management throughout the project lifecycle.

Financial/Cost Management

3.1 Project estimating practices and tools are effective at supporting consistent cost estimation practices that provide reasonable, timely cost estimates with clearly communicated estimated variances.

3.2 Mechanisms to assess and monitor the accuracy and completeness of project costs are effective in helping to ensure that project expenses are in accordance with agreement terms and conditions.

3.3 A timely project plan and budget is developed at the appropriate level of detail, and forecasts/variances are reported and monitored to support efficient and effective project management and monitoring.

Risk Management

4.1 Management has a documented approach with respect to project risk management that includes risk identification, assessment, mitigation and monitoring.

Procurement

5.1 Value-added procurement services are provided to support the efficient and effective delivery or required goods and services.

 

 

Appendix B: Project Costs and Timeframe

A breakdown of the reported remediation expenditures for Giant Mine from 2006-07 to 2011-12 is provided below.

Category2006 Approval2009 ApprovalTOTALS
2006-20072007-20082008-2009*2009-20102010-20112011-2012
Care & Maintenance $12,001,260 $8,280,349 $8,944,306 $11,502,234 $7,690,163 $13,472,956 $61,891,268
Regulatory Approvals $71,022 $253,652 $74,249 $300,000 $863,335 $1,510,176 $3,072,434
Site Investigations
and Assessments
$282,698 $588,611 $382,459 $6,710 $1,619 $0 $1,262,097
Site Remediation $0 $0 $0 $15,857,370 $11,651,600 $5,251,891 $32,760,861
Community Consultations $28,139 $46,685 $13,095 $3,482 $46,190 $3,660 $141,250
Monitoring $209,090 $156,517 $169,881 $0 $65,334 $89,559 $690,381
Project Management $1,793,238 $1,426,584 $2,115,619 $3,443,490 $4,264,186 $5,115,613 $18,158,730
Total $14,385,447 $10,752,398 $11,699,608 $31,113,287 $24,582,427 $25,443,855 $117,977,022
The Government of the Northwest Territories Contribution - $3,428,504
SUB TOTAL (does not include Employee Benefit Plan or accommodation or the Goods and Services Tax) $114,548,517
Employee Benefit Plan (20% of salaries) $963,471
Accommodations (13% of salaries) $626,256
TOTAL $116,138,244

Note: This table was provided by management of the Northern Affairs Organization. An assessment of the accuracy of these figures was not conducted as part of the audit.

Figures may not sum to totals due to rounding.

 

 

Appendix C: Total Estimated Remediation Costs

A breakdown of the current total estimated cost to complete the Giant Mine remediation project is provided below.

 Fiscal Year
Previous Fiscal Years
(2006-07 to 2011-12)
2012-20132013-20142014-20152015-20162016-2017
Funding
Aboriginal Affairs and Northern Development Canada
Vote 1 (Operating expenditures)
Personnel $4,817,356 $3,091,000 $3,183,730 $3,279,242 $3,377,619 $3,478,948
Operating & Maintenance $109,731,161 $43,830,944 $78,775,642 $88,941,238 $39,686,064 $37,046,920
Employee benefit plan @ 20% $963,471 $618,200 $636,746 $655,848 $675,524 $695,790
Total Vote 1 $115,511,988 $47,540,144 $82,596,118 $92,876,328 $43,739,207 $41,221,658
Accommodation $626,256 $401,830 $413,885 $426,301 $439,090 $452,263
Subtotal $116,138,244 $47,941,974 $83,010,003 $93,302,629 $44,178,297 $41,673,921
Goods and Services Tax $0 $2,191,547 $3,938,782 $4,447,062 $1,984,303 $1,852,346
Grand Total $116,138,244 $50,133,521 $86,948,785 $97,749,691 $46,162,600 $43,526,267
 
Fiscal Year
 2017-20182018-20192019-20202020-20212021-20222022-20232023-20242024-2025Total Indicative Project Cost
(2017-2025)
Total Project
Costs
(2006-2025)
Annual Ongoing
Costs
(2025-2137)
Funding
Aboriginal Affairs and Northern Development Canada
Vote 1 (Operating expenditures)
Personnel $3,583,316 $3,690,816 $3,801,540 $3,915,586 $4,033,054 $4,154,046 $4,278,667 $4,407,027 $31,864,052 $53,091,947 $0
Operating & Maintenance $54,703,381 $59,635,046 $59,184,842 $50,977,213 $49,188,355 $43,554,168 $42,256,439 $40,973,684 $400,473,128 $798,485,097 $1,887,857
Employee benefit
plan @ 20%
$716,663 $738,163 $760,308 $783,117 $806,611 $830,809 $855,733 $881,405 $6,372,809 $10,618,388 $0
Total Vote 1 $59,003,360 $64,064,025 $63,746,690 $55,675,916 $54,028,020 $48,539,023 $47,390,839 $46,262,116 $438,709,989 $862,195,432 $1,887,857
Accommodation $465,831 $479,806 $494,200 $509,026 $524,297 $540,026 $556,227 $572,914 $4,142,327 $6,901,953 $0
Subtotal $59,469,191 $64,543,831 $64,240,890 $56,184,942 $54,552,317 $49,079,049 $47,947,066 $46,835,030 $442,852,316 $869,097,384 $1,887,857
Goods and Services Tax $2,735,169 $2,981,752 $2,959,242 $2,548,861 $2,459,418 $2,177,708 $2,112,822 $2,048,684 $20,023,656 $34,437,696 $94,393
Grand Total $62,204,360 $67,525,583 $67,200,132 $58,733,803 $57,011,735 $51,256,757 $50,059,888 $48,883,714 $462,875,972 $903,535,080 $1,982,249

*Note: Figures may not add to totals due to rounding.

 
 
Date modified: