Audit of Information Management/Information Technology Governance and Application Systems Integration

Date : November 2013
Project #: 13-47

PDF Version (192 Kb, 32 Pages)

 

Table of contents

Acronyms

AANDC

Aboriginal Affairs and Northern Development Canada

ASC

Architectural Standards Committee

BMT

Branch Management Team

CFO

Chief Financial Officer

CIO

Chief Information Officer

DGIOC

Director General Implementation and Operations Committee

DOC

Departmental Operations Committee

EA

Enterprise Architecture

FNCFS IMS

First Nations Child and Family Services Information Management System

IM

Information Management

IMB

Information Management Branch

INAC

Indian and Northern Affairs Canada

IT

Information Technology

ITSG

Information and Technology Stewardship Group

MAF

Management Accountability Framework

MOU

Memorandum of Understanding

NATS

National Additions to Reserve Tracking System

PMO

Project Management Office

PPMF

Project Portfolio Management Framework

SSC

Shared Services Canada

TBS

Treasury Board of Canada Secretariat

 

 

Executive Summary

Background

Aboriginal Affairs and Northern Development Canada's (hereon referred to as 'AANDC' or 'the Department') investment in Information Management and Information Technology (IM/IT) serves to support Aboriginal people and Northerners, enable strategic programs, and support operational activities within the Department itself.

In the 2011-2012 fiscal year, resources and personnel associated with the delivery of email, data centre and network services were transferred from over 40 government departments, including AANDC, to Shared Services Canada as part of the Government of Canada's enterprise-wide approach to IT. Other than these services, the Information Management Branch (IMB) of the Chief Financial Officer Sector provides IM/IT solutions and services across the Department. IMB endeavours to provide AANDC with an optimized, aligned, stable and secure IM/IT environment.

The IMB had a budget of $41.1 M and expenditures of $40.4 M in 2012-2013. The Branch's expenditure forecast for 2013-2014 was $43.5 M as of September 30, 2013. There are 220 Full Time Equivalents across five directorates.

The IMB is led by the Department's Chief Information Officer (CIO) who, according to the Treasury Board of Canada Secretariat's Directive on the Management of Information Technology, is responsible for IT governance, IT planning and IT strategies. In their most basic form, these responsibilities help to promote strong IT regimes by establishing effective governance structures, integrating planning into overall corporate planning, and aligning strategies with those of the federal government. Additionally, as set out in section 5.2 of the AANDC Policy on Information Management, the CIO, as the senior official for Information Management designated by the Deputy Minister, has the lead role to coordinate, promote and direct information management within the Department.

Previous internal audit findings spanning 2008 to 2011, such as those from the Preliminary Survey of IM/IT Governance conducted in 2011, suggested that IM/IT governance required improvement at AANDC. In recent years, the Department invested in governance processes under new executive leadership and included in its 2012-2015 IM/IT Strategy the objectives of improving IM/IT governance, enterprise Information Management and Project Portfolio Management which will be facilitated through the long-term development of a business-integrated Enterprise Architecture.

The Audit and Evaluation Sector of AANDC identified an audit of IM/IT Governance and Application Systems Integration in the 2013-2014 to 2015-2016 Risk-Based Audit Plan approved by the Deputy Minister on February 27, 2013.

Audit Objective and Scope

The objective of the audit was to assess the adequacy and effectiveness of IM/IT governance structures, processes and controls in place to support the effective management of IM/IT resources, including integration among AANDC's key IM/IT application systems.

The scope of the audit included:

  • an examination of the governance bodies and decision-making processes in place to establish priorities among, and recommend or approve funding for, IM and IT projects and initiatives;
  • an examination of roles and responsibilities and delegated authorities, taking into account the role of Shared Services Canada; and,
  • an assessment of the adequacy of planning for integration of application systems.

The period from April 1, 2011 to June 30, 2013 was included in scope. Testing was adjusted to place more weight on the most recent periods. Audit fieldwork was conducted at headquarters and included a selection of two regions. British Columbia and Ontario were engaged in the audit process via questionnaires and conference calls.

The audit included detailed testing of a sample of AANDC IM/IT-enabled projects to assess their compliance with the Project Portfolio Management Framework (PPMF). For each project selected, the audit team obtained and evaluated project documentation that supported adherence to the requirements of the PPMF.

Statement of Conformance

The Audit of IM/IT Governance and Application Systems Integration conforms with the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program.

Observed Strengths

The following strengths pertaining to IM/IT governance structures, processes and controls were observed:

  • An IM/IT Governance Committee framework is defined which clearly sets out the relationship among the various IM/IT governance committees. Key decision-making bodies defined in the framework were found to be in place, with terms of reference, membership from throughout the Department, and a regular schedule of meetings.
  • The IM/IT Strategy 2012-2015 has been developed and approved. An IM/IT Plan 2013 has been developed and sets out a number of initiatives to operationalize the strategy throughout 2012-2013 to 2016-2017. The Plan is refreshed on an annual basis. Additionally, a risk assessment for Chief Financial Officer (CFO) Sector, which included IM/IT risks, was conducted in 2012 and has been incorporated into planning.
  • A suite of IM and IT policies and directives exist and, except for those that are still in draft, are consistent with the requirements set out in the related IM and IT policies and directives issued by Treasury Board of Canada Secretariat. Policies and directives have been issued in the past two years to enhance control over projects and IM/IT expenditures throughout the Department.
  • A Project Portfolio Management Framework (PPMF) has been developed, documented and approved to provide governance over projects. Testing of a sample of active projects indicated adherence to the requirements of the PPMF, with a few exceptions noted. The PPMF promotes application systems integration by requiring an assessment of the extent of reuse of systems, data, and business rules as part of "Options Analysis" deliverable.
  • The organization structure is documented for IMB, with clear assignment of roles and responsibilities.
  • Progress of key IMB initiatives is tracked and monitored as part of the CFO Sector quarterly reporting.

Conclusion

Generally, IM/IT governance structures, processes and controls in place to support the effective management of IM/IT resources, including integration among AANDC's key IM/IT application systems, were found to be effective and adequate. Some areas for improvement identified in the following areas: information management governance; integration of departmental priorities into IM/IT strategy and plans; IM/IT policies and directives; documentation and enforcement of requirements of the Project Portfolio Management Framework; implementation of Enterprise Architecture to support application integration; and, monitoring of IM/IT spend in regions and sectors.

Recommendations

The audit identified areas where management control practices and processes could be improved, resulting in the following recommendations:

  1. The Chief Information Officer should strengthen governance and strategic direction related to information management by establishing or enhancing the following key elements:
    • Develop and approve an IM Plan consistent with the priorities identified in the Department's IM/IT Strategy, including prioritized initiatives for the five-year planning horizon. This may be part of an integrated IM/IT Plan or a separate IM Plan, but it is critical that IM direction be clarified. The draft Enterprise IM Strategy should be withdrawn to avoid confusion with having both an IM/IT Strategy and a separate IM Strategy.
    • Ensure required IM policies and directives are finalized and approved, and are communicated within the Department to all impacted employees. Of priority are those meeting TBS requirements, especially the Directive on Recordkeeping, where compliance is required by March 2015.
    • Ensure IM is a regular standing item at governance committees to monitor implementation of IM policies and directives as well as progress of key IM initiatives within the approved plan.
  2. The Chief Information Officer should continue the process recently initiated of engaging sectors and regions through the five-year call for investment plans in order to facilitate integration of the IM/IT Strategy and Plans with departmental investment planning. In addition to those approved by Operations Committee as part of the Department's Investment Plan, which includes projects greater than $1 million only, these results should be incorporated into the annual updates of the IM/IT Plan to help ensure the IM/IT Plan remains consistent with the overall priorities of the Department and to allow for appropriate Information Management Branch resource planning.
  3. The Chief Information Officer should establish a centralized function to manage all IM and IT policies and directives to help ensure that they are tracked from draft status through to finalization and approval, and to manage the posting of only approved policies and directives on the Information Management Branch intranet site. This centralized function should track when policies and directives require review based either on the established timeline or when TBS requirements are modified, and monitor the process for updating. A communication process should be established to communicate the requirements of all new or modified policies and directives on a consistent basis to all those responsible for implementing them.
  4. The Chief Information Officer should enhance the Project Portfolio Management Framework (PPMF) documentation/adherence by:
    • clarifying the requirements for information management considerations in the PPMF, including how these will be documented and monitored in the gating approval process;
    • developing, approving, and posting to the Information Management Branch intranet site a document that clearly outlines the current requirements for approval by governance bodies at each gate in the PPMF gating process;
    • clarifying PPMF requirements for projects requiring Treasury Board approval as these projects should follow the Department's defined and approved project governance process; and,
    • ensuring that formal project close-out as defined in the PPMF, including lessons learned and benefits realization, is enforced.
  5. The Chief Information Officer should ensure that Enterprise Architecture (EA) initiatives are addressed on a priority basis and leveraged to enhance application systems integration in future project investments. Specifically, timelines should be confirmed and resources assigned for the key EA initiatives that had been identified for 2012-2013, consisting of development of an EA strategy and integration of EA with the PPMF, as these were the basis for future initiatives. Further, the Architectural Standards Committee should be re-instituted and reactivated to perform its role as the governance body ensuring IM/IT investments comply with approved technology standards and enterprise architecture initiatives (including current terms of reference produced, membership confirmed, and regular meetings initiated).
  6. The Chief Information Officer should ensure that appropriately detailed information is reported by sectors and regions in the annual IM/IT spend analysis, especially information on specific IM/IT projects/initiatives undertaken, in order to effectively monitor compliance with departmental policies requiring pre-approval of all IM/IT expenditures. Once this additional information is gathered and evaluated, any indications of non-compliance with required pre-approvals should be evaluated and, based on significance, corrective actions should be initiated, such as direct follow-up with the region/sector or escalation to governance committees.

Management Response

Management is in agreement with the findings, has accepted the recommendations included in the report, and has developed a management action plan to address them. The management action plan has been integrated in this report.

 

 

1. Background

AANDC’s investment in Information Management and Information Technology (IM/IT) serves to support Aboriginal people and Northerners, enable strategic programs, and support operational activities within the Department itself.

In the 2011-2012 fiscal year, resources and personnel associated with the delivery of email, data centre and network services were transferred from over 40 government departments, including AANDC, to Shared Services Canada (SSC) as part of the Government of Canada’s enterprise-wide approach to IT. Other than these services, the Information Management Branch (IMB) of the Chief Financial Officer Sector provides IM/IT solutions and services across the Department. The Branch endeavours to provide AANDC with an optimized, aligned, stable and secure IM/IT environment.

The IMB had a budget of $41.1 M and expenditures of $40.4 M in 2012-2013. The Branch’s expenditure forecast for 2013-2014 was $43.5 M as of September 30, 2013. There are 220 Full Time Equivalents across six directorates. The Branch is organized according to the structure illustrated below.

Organizational chart

Figure 1

Text description of Figure 1

Figure 1 illustrates the top two levels of the organizational chart of the Information Management Branch as at September 30, 2013. The Branch is led by the Chief Information Officer who oversees a total of 6 direct reports. These direct reports are as follows:

  • Director, Corporate Information Management
  • Senior Director, Enterprise Strategic Services
  • Director, Business Decision Support
  • Director, IT Security
  • Director, Infrastructure and Operations
  • Director, Application Development, Databases & Data Administration
 

The IMB is led by the Department’s Chief Information Officer (CIO) who, according to the Treasury Board of Canada Secretariat (TBS) Directive on the Management of Information Technology, is responsible for IT governance, IT planning and IT strategies. In their most basic form, these responsibilities help to promote strong IT regimes by establishing effective governance structures, integrating planning into overall corporate planning, and aligning strategies with those of the federal government. Additionally, as set out in section 5.2 of the AANDC Policy on Information Management, the CIO, as the senior official for Information Management designated by the Deputy Minister as per section 6.1.7 of the Treasury Board Policy on Information Management, has the lead role to coordinate, promote and direct information management within the Department.

Previous internal audit findings spanning 2008 to 2011, such as those from the Preliminary Survey of IM/IT Governance conducted in 2011, suggested that IM/IT governance required improvement at AANDC. In recent years, the Department invested in governance processes under new executive leadership and included in its 2012-2015 IM/IT Strategy the objectives of improving IM/IT governance, enterprise Information Management and Project Portfolio Management which will be facilitated through the long-term development of a business-integrated Enterprise Architecture.

In contrast to benefits realized by Enterprise Architecture and integrated systems for efficient scaling, increased productivity and better information for decision-making, the historically more fragmented approach to IT investments resulting in disparate systems presents maintenance challenges and undermines effective and efficient delivery of programs and information for decision-making.

In the broader Government of Canada context of growing demand for efficient, modern services, the government is moving toward whole-of-government IT solutions. The government’s IT modernization strategy involves, in part, standardization and consolidation of back office applications and a Government of Canada-wide shared electronic records management capability.

At AANDC, many systems have evolved over different periods of time and/or are being developed to meet the various business needs of the Department and functional needs of AANDC program areas. In accordance with government direction, certain systems currently in use at AANDC are transitioning or will transition to common government-wide systems. Some of the Department’s systems include: Human Resource Management System (PeopleSoft, which will be upgraded to the government-wide PeopleSoft 9.1 release); OASIS (Oracle, which is transitioning to SAP in accordance with government direction); Grants and Contributions Information Management System (GCIMS, formerly FNITP, which Health Canada is adopting and AANDC will host), Comprehensive Integrated Document Management (CIDM, which will transition to GCDocs); a new Education Information System (EIS); a new First Nations Child and Family Services Information Management System (FNCFS IMS); Indian Registry System (IRS); National ATR Tracking System (NATS); Indian Land Registry System (ILRS); and, Netlands, to name only a few. The 2012-2015 IM/IT Strategy noted that the Department was maintaining 76 corporate and 88 regional applications.

 

 

2. Audit Objective and Scope

2.1 Audit Objective

The objective of the audit was to assess the adequacy and effectiveness of IM/IT governance structures, processes and controls in place to support the effective management of IM/IT resources, including integration among AANDC’s key IM/IT application systems.

2.2 Audit Scope

The scope of the audit included:

  • an examination of the governance bodies and decision-making processes in place to establish priorities among, and recommend or approve funding for, IM and IT projects and initiatives;
  • an examination of roles and responsibilities and delegated authorities, taking into account the role of Shared Services Canada; and,
  • an assessment of the adequacy of planning for integration of application systems.

The period from April 1, 2011 to June 30, 2013 was included in scope. Testing was adjusted to place more weight on the most recent periods.

The audit also included detailed testing of a sample of AANDC IM/IT projects to assess their compliance with the Project Portfolio Management Framework (PPMF). For each project selected, the audit team obtained and evaluated project documentation that supported adherence to the requirements of the PPMF.

Audit fieldwork was conducted at headquarters and included a selection of two regions. Ontario and British Columbia were engaged in the audit process via questionnaires and conference calls.

 

 

3. Approach and Methodology

The Audit of IM/IT Governance and Application Systems Integration was planned and conducted in accordance with the requirements of the Treasury Board Policy on Internal Audit and followed the Institute of Internal Auditors’ Standards for the Professional Practice of Internal Auditing. The audit team examined sufficient, relevant evidence and obtained sufficient information to provide a reasonable level of assurance in support of the audit conclusion.

The principal audit techniques used included:

3.1 Selection of Projects for Detailed Testing

A list of all projects by gate (gates 1 to 7 are defined in the PPMF) was obtained from the Project Management Office (PMO) in August 2013. Active projects are those in gates 4 (Project Charter) to 6 (Solution Complete) of the PPMF. Based on the total population of 15 active projects, three were selected for detailed testing. The projects were selected using the following risk-based selection criteria:

  • from gates 5 and 6, to ensure each project could be tested for a sufficient number of key deliverables;
  • across various magnitudes of cost – one over $4 million, one between $1 and $4 million, and one less than $1 million; and,
  • representing different sectors.

Using the above criteria, the following projects were selected:

  • First Nations Child and Family Services Information Management System (FNCFS IMS);
  • Nunavut Map Selection; and,
  • National Additions to Reserve Tracking System (NATS) First Nations Access.

3.2 Selection of Regions for Testing

Regional IM and IT staff do not report directly to the CIO but rather to the regional Director of Corporate Services. Two regional offices were therefore selected to be included in the audit to confirm IM/IT governance controls in the regions and to understand the extent of IM/IT projects undertaken in the regions. The table below sets out financial information for 2012-2013 and the number of applications by region. Based on this information, British Columbia was selected due to its high level of IM/IT expenditures and large number of applications, and Ontario was selected as representing a region with both moderate IM/IT expenditures and number of applications. Information from each region was gathered through questionnaires, phone interviews, and requested supporting documentation.

Region Number of
applications
reported in 20131
2012-2013 IM/IT expenditures ($ 000's)
(all amounts per AANDC financial system)
Information
Technology
Information
Management
Atlantic 0 $ 354 $169
Quebec 9  728 654
Ontario 10 843 572
Manitoba 2 242 590
Saskatchewan 2 619 427
Alberta 0 799 494
British Columbia 26 1,344 772
Nunavut 3 0 148
Northwest Territories 13 0 1,455
Yukon 1 0 0
Total 66 $4,929 $5,281

1 Figures obtained from the inventory of applications by region per application maintained by the office of the Senior Director, Enterprise Strategic Services.

 

 

4. Conclusion

Generally, IM/IT governance structures, processes and controls in place to support the effective management of IM/IT resources, including integration among AANDC’s key IM/IT application systems, were found to be effective and adequate. Some areas for improvement were identified in the following areas: information management governance; integration of departmental priorities into IM/IT strategy and plans; IM/IT policies and directives; documentation and enforcement of requirements of the Project Portfolio Management Framework; implementation of Enterprise Architecture to support application integration; and, monitoring of IM/IT spend in regions and sectors.

 

 

5. Findings and Recommendations

Based on a combination of the evidence gathered through the examination of documentation, analysis and interviews, each audit criterion was assessed by the audit team and a conclusion for each audit criterion was determined. Where a significant difference between the audit criterion and the observed practice was found, the risk of the gap was evaluated and used to develop a conclusion and to document recommendations for improvement initiatives.

Observations include both management practices considered strong as well as those requiring improvement. Accompanying the observations of management areas identified for improvement are recommendations for corrective actions.

5.1. Governance and Strategic Direction

5.1.1 Management and Oversight Bodies

To determine whether effective management and oversight bodies for IM/IT initiatives were in place and functioning, the audit team assessed whether:

  • IM/IT governance committees were in place;
  • Terms of reference and membership were defined; and,
  • Regular meetings were occurring and records of decision/ meeting minutes were retained.

We observed that an IM/IT Governance Committees framework was defined in 2011 which clearly sets out the relationship among the various IM/IT committees, as follows:

Figure 2 Diagram of the reporting relationship between IM/IT committees. Reproduced from the IM/IT Governance Committees Framework (2011).

IM/IT Governance Committees framework
Text description of Figure 2

Figure 2 illustrates the relationship between the various IM/IT Committees. The diagram depicts the interrelationships between each committee using solid (direct) and dotted lines (functional) and separates committees into the following three groups, depending on their focus: strategic, tactical and operational. Beginning at the operational level, the lowest level of the three, is the Change Advisory Board (CAB). The CAB reports directly to the Architectural Standards Committee (ASC), which is located at the tactical level. Within the tactical level, the ASC receives direct input from the Information Management Branch (IMB) / Branch Management Team (BMT). The ASC also has receives and provides input into the Project Steering Committee(s) and the Information and Technology Stewardship Group (ITSG). The Project Steering Committee(s) receives functional input from the Information Management Branch (IMB) / Branch Management Team (BMT) and the Information and Technology Stewardship Group (ITSG) and provides functional input to the Director General Implementation and Operations Committee (DGIOC), which is located at the strategic level, in addition to the direct input from the ASC. The Information and Technology Stewardship Group receives direct input from Regional & Sectoral IM/IT Committees, in addition to the direct input it receives from the ASC and the functional input it receives from the Project Steering Committee(s). ITSG reports directly to the DGIOC, which is located at the strategic level. The DGIOC reports directly to the Departmental Operations Committee, the most senior committee at the strategic level.

 

Key decision-making bodies defined in the framework were found to be in place, including Departmental Operations Committee (DOC), Director General Implementation and Operations Committee (DGIOC) and Information and Technology Stewardship Group (ITSG). For each, terms of reference were defined and approved, including membership from throughout the Department. It was noted that the membership of ITSG includes the regional Directors of Corporate Services, and DGIOC includes the Director General of the Regional Operations Sector, thus ensuring involvement of the regions in IM/IT governance. An examination of the six months from January to June 2013 indicated that regular meetings were occurring, attendance was tracked, and records of discussion prepared, including tracking of action items.

The Terms of Reference of ITSG sets out specific responsibilities for both IT and IM, including to provide consultation and recommendations on these areas to DGIOC and, if required, to DOC. A review of the records of discussion for both ITSG and DGIOC however indicate that the focus has been on IT while IM items are rarely discussed. DGIOC, at its July 2013 meeting, has recognized this as an area requiring improvement and is taking action to determine how to best advance and support IM priorities. This will be a key area as there are critical IM initiatives currently facing the Department, including recordkeeping compliance and the implementation of GCDocs.

Although the creation of Shared Services Canada does not change the overall IM/IT governance, the respective roles, responsibilities and processes are continuing to evolve between SSC and the Department, and the minutes examined reflected that SSC updates were regularly tabled.

An examination of a sample of IMB tactical committees, which included the Branch Management Team (BMT) and Architectural Standards Committee (ASC), confirmed that BMT meets on a regular basis and maintains records of decisions. For ASC, meetings have not been occurring regularly and the last record of discussion is from October 2012. The Director Enterprise Strategic Services, who is the Chair of ASC, has advised that ASC is being restructured to improve its effectiveness, including a revised Terms of Reference and membership. As noted in the IM/IT Governance Committee framework, ASC should play an important role in ensuring that "IM/IT investments comply with the approved departmental infrastructure design, hardware, software and information management standards, as well as Government of Canada policies and strategic investments". See further discussion regarding ASC in Section 5.2.2.

5.1.2 Strategic and Tactical Plans

The audit examined whether strategic and tactical plans for IM and IT are in place and are being implemented.

We observed that key planning activities have occurred. The IM/IT Strategy 2012-2015 has been developed and approved by DOC in February 2012. The Strategy outlines IMB's vision, strategic direction, and implementation strategy. An IM/IT Plan 2013 was then developed and sets out a number of initiatives to operationalize the strategy throughout 2012-13 to 2016-17, including initiatives relating to SSC transition planning. The Plan is refreshed on an annual basis. Additionally, a formal risk assessment for CFO Sector, which included IM/IT risks, was conducted in 2012, and has been incorporated into the CFO Sector 2013-2014 business planning exercise.

The IM/IT Strategy identifies as a core strategy the achievement of "IM/IT Plans approved through the corporate planning cycle and directly linked to government-wide and departmental objectives", which is consistent with TBS requirements as set out in the TBS Policy on the Management of Information Technology, which in turn has been adopted by the Department. To date however, there has been no structured process in place to ensure this alignment when developing the IM/IT Plan. This has been recognized as an area for improvement by IMB management and we have been advised by the CIO that TBS will also require this alignment in the IT Plan to be submitted to TBS by March 31, 2014. As a result, the CFO issued an integrated call in September 2013 to all sectors and regions for a five-year IM/IT, Procurement and Investment Plan, including templates for setting out specific initiatives with cost estimates developed with progressively less precision over each of the five years. Responses are due in late October 2013, and will be utilized as input to future IM/IT Plans and IM/IT resource planning, as this will provide information for planning purposes on all planned projects. These IM/IT Plans must also be developed within the context of the overall departmental Investment Plan, which is approved by DOC and which includes projects greater than $1 million.

Although the approved IM/IT Strategy was developed to integrate both IM and IT elements, a separate draft 'Enterprise Information Management Strategy 2010 to 2015' also exists, as does a draft IM Tactical Plan 2013-2014 to 2015-2016, which sets out 30 specific IM initiatives. We have been advised by the CIO that the IM/IT Strategy should supercede the draft Enterprise IM Strategy, and that a consolidated IM/IT Plan will be developed in the current fiscal year. Other than the IM toolset renewal, the current IM/IT Plan 2013 does not contain any IM initiatives, in contrast to the 30 set out in the draft IM Tactical Plan. At this time, it is unclear which IM priority initiatives have been approved, who has been assigned responsibility for each and whether progress is occurring.

Planning between IMB and the Sectors has been enhanced through the development of Memorandums of Understanding (MOU) in 2013-2014 between IMB and each sector to confirm funding for system development and maintenance activities. The MOUs address additional costs to be incurred throughout the year for non-core operational support and application development, including any related additional hardware/software and licensing. Costs charged under the MOUs relate only to incremental costs of hiring contractors and do not include any recovery of IMB employee time. This is the second year for the MOU process, and is enhanced from the prior year to establish MOUs by sector rather than on an individual application basis.

5.1.3 Policies and Directives

Policies and directives set out direction and guidance for the Department in the areas of IM and IT. In examining policies and directives, the audit team assessed whether:

  • Policies and directives for key areas of IM and IT were developed and approved;
  • Policies and directives were subject to regular review and updating; and,
  • Policies and directives were effectively communicated to impacted parties.

Based on a review of the IM and IT policies and directives provided by IMB management, we observed that the suite policies and directives provides coverage of the expected areas, and, except for those that are still in draft, are consistent with the requirements set out in the related IM and IT policies and directives issued by Treasury Board Secretariat (TBS). We noted three AANDC directives still in draft as follows: Directive on Recordkeeping, Directive on Data Sharing and Directive on Data Stewardship. A comparison of the requirements of the draft AANDC Directive on Recordkeeping to those of the related TBS directive noted three areas that are required by TBS but are not included in the Department's draft Directive on Recordkeeping:  conducting a risk assessment of resources with business value, establishing taxonomies or classification structures of resources with business value, and the communication of the risk of poor record keeping to employees. As departments must be compliant with the TBS Directive on Recordkeeping by March 2015, it is important that AANDC finalize and issue its Directive so that all related compliance activities can be defined and implementation appropriately planned.

The process for reviewing and updating IM and IT policies and directives is currently informal and inconsistent. We were advised by management that policies and directives are reviewed internally every three years and when the TBS issues new policies or directives or amends existing ones. Through our testing, we noted examples of policies that should have been reviewed according to these criteria but were not, including the AANDC Policy on Information Management (dated October 2008) and the INAC Security Management Framework (dated July 2008, and which contains the IT Security policy statement). Both examples have not been updated in five years, while the related TBS documents have both been updated since 2008. In addition, a recent IMB review of existing policies, directives and standards identified that a number of existing documents are improperly classified as policy versus directive versus standard.

IM/IT policies and directives are published on the IMB intranet page, to be referenced by impacted employees. We identified that some IM/IT directives that have not yet been finalized have also been posted, including the draft AANDC Directive on Recordkeeping and a draft AANDC Directive on Mail and Messenger Services. In addition, the AANDC Directive on IM/IT Project Portfolio Management which has been approved and was effective November 1, 2012 is posted on the Project Management page of the IMB intranet with 'track changes' active in the table of contents and a 'draft' watermark throughout.

The Department has recently issued two key documents related to IM/IT management – the AANDC Policy on Management of IT and the AANDC Directive on IM and IT Procurement Authorization – which set out requirements for CIO approval of all IM/IT-related architectures and procurement, and specifies financial coding for all such procurement. We were advised that the only communication of these items throughout the Department was through the records of discussion of the approving governance committees, with the committee members then responsible for further communication. This policy and directive have wide applicability across the Department, including to all program and project managers and to all responsibility center managers who provide financial coding for these items. A lack of appropriate and consistent communication of expectations to those impacted can have a negative impact on the rate of compliance.

Recommendations
  1. The Chief Information Officer should strengthen governance and strategic direction related to information management by establishing or enhancing the following key elements:
    • Develop and approve an IM Plan consistent with the priorities identified in the Department's IM/IT Strategy, including prioritized initiatives for the five-year planning horizon. This may be part of an integrated IM/IT Plan or a separate IM Plan, but it is critical that IM direction be clarified. The draft Enterprise IM Strategy should be withdrawn to avoid confusion with having both an IM/IT Strategy and a separate IM Strategy.
    • Ensure required IM policies and directives are finalized and approved, and are communicated within the Department to all impacted employees. Of priority are those meeting TBS requirements, especially the Directive on Recordkeeping, where compliance is required by March 2015.
    • Ensure IM is a regular standing item at governance committees to monitor implementation of IM policies and directives as well as progress of key IM initiatives within the approved plan.
  2. The Chief Information Officer should continue the process recently initiated of engaging sectors and regions through the five-year call for investment plans in order to facilitate integration of the IM/IT Strategy and Plans with departmental investment planning. In addition to those approved by Operations Committee as part of the Department's Investment Plan, which includes projects greater than $1 million only, these results should be incorporated into the annual updates of the IM/IT Plan to help ensure the IM/IT Plan remains consistent with the overall priorities of the Department and to allow for appropriate Information Management Branch resource planning.
  3. The Chief Information Officer should establish a centralized function to manage all IM and IT policies and directives to help ensure that they are tracked from draft status through to finalization and approval, and to manage the posting of only approved policies and directives on the Information Management Branch intranet site. This centralized function should track when policies and directives require review based either on the established timeline or when TBS requirements are modified, and monitor the process for updating. A communication process should be established to communicate the requirements of all new or modified policies and directives on a consistent basis to all those responsible for implementing them.

5.2. Application System Development, Acquisition and Integration

5.2.1 Project Portfolio Management

A Project Portfolio Management Framework (PPMF) has been developed and documented, based on TBS guidance, to provide governance over IM/IT-enabled projects. The PPMF includes seven gates, or decision points, in a project life-cycle as follows:

  • Gate 1 – Strategic Alignment
  • Gate 2 – Project Approach
  • Gate 3 – Business Case
  • Gate 4 – Project Charter
  • Gate 5 – Plan and Specifications
  • Gate 6 – Solution Complete
  • Gate 7 – Utilize and Retire

The PPMF includes requirements for approvals by governance committees at each gate and various project deliverables are specified at each gate to support the required approvals. Since its initial implementation in 2010, the PPMF was enhanced in 2011 to require an evaluation of opportunities based on their size, complexity and risk and to streamline the process for those of lower risk. The original PPMF and its enhancements were approved by the governance committees, including DOC. The PPMF documents and templates are published on the IMB intranet site. The AANDC Directive on IM/IT Project Portfolio Management was issued with an effective date of November 1, 2012 and sets out the governance to support adherence to the PPMF.

Our review of the project portfolio and detailed testing of a sample of three active projects indicated adherence to the requirements of the PPMF, except for the following areas:

  • The requirements for integration of IM into PPMF processes are unclear. There is an 'IM Engagement in Project Management' document but it is in draft, even though it is dated 2011. Evidence of assessment and integration of IM requirements for all three projects tested was not available. For one project – Nunavut Map Selection – a Recordkeeping Agreement was prepared but there was no evidence of approval. For the others, although IM involvement may have occurred, it appears to be informal and no evidence was available.
  • The Project Charter for FNCFS IMS was not signed off by the sponsor or Project Steering Committee. Instead of going through the standard departmental project approval gating process, the Branch submitted the Project for central agency approval. Although the Branch did not get the Project Charter approved by the sponsor or Project Steering Committee, the Project Charter was approved through the central agency approval process. 
  • Although gating approval by governance bodies is a key component of the PPMF, we were unable to locate current documentation that clearly specifies which governance bodies are responsible for approvals at each gate for each of the full and light PPMF processFootnote 1.  The current requirements for approvers by gate was only available by following changes approved by DOC in March 2011, with clarifications provided by the Project Management Office (PMO), who indicated that only the full and light processes have been implemented. A June 2013 presentation prepared by a customer relationship manager included the current requirements but the presentation was not generally available even within the PMO.
  • Project close-out is not occurring, although it is a requirement of gate 7. Based on the listing of projects obtained from the PMO, there are 15 projects in gate 7 requiring close-out, some since 2010. Project close-out is an important phase in documenting lessons learned and benefits realization.

We were advised that some existing applications had been developed outside the PPMF, although no current projects were cited. The Department recently issued a Policy on Management of Information Technology which is effective January 1, 2013 and which prescribes that all projects with an IM/IT component be pre-approved by the CIO. Also, as noted previously, CFO sector has recently issued an integrated call to all sectors and regions for information on five-year investment plans. The combination of these two initiatives should prevent and detect circumvention of the PPMF in the future, although the effectiveness of this has yet to be proven.

Recommendation

4. The Chief Information Officer should enhance the Project Portfolio Management Framework (PPMF) documentation/adherence by:

  • clarifying the requirements for information management considerations in the PPMF, including how these will be documented and monitored in the gating approval process;
  • developing, approving, and posting to the Information Management Branch intranet site a document that clearly outlines the current requirements for approval by governance bodies at each gate in the PPMF gating process;
  • clarifying PPMF requirements for projects requiring Treasury Board approval as these projects should follow the Department's defined and approved project governance process; and,
  • ensuring that formal project close-out as defined in the PPMF, including lessons learned and benefits realization, is enforced.

5.2.2 Application Systems Integration

Application systems integration has been identified as an issue in the Department, with some existing applications having been built in silos, resulting in a lack of data sharing and potential for inconsistency of data across applications. Since implementation of the PPMF in 2010, application systems integration has been promoted by the requirements of gates 2 and 3 of the PPMF – Project Approach and Business Case respectively – where an assessment of the extent of reuse of systems, data, and business rules is required as part of the "Options Analysis" deliverable. Also contributing to integration, technology standards have been defined and a 'common data elements' database exists, although it was noted that this database is limited to only eight data elements, including items such as band, reserve and tribal council. Additionally initiatives such as enhancements to the data warehouse in 2013 to include education and child and family services contribute to addressing the departmental priority of improving information for decision-making.

To facilitate increased integration and reusability of information going forward, the IM/IT Strategy has identified the move towards enterprise architecture (EA) as a key enabler and "the foundation for core processes that facilitate the appropriate level of business integration in developing IM/IT solutions." Consequently, the IM/IT Plan for 2013 identifies a number of initiatives around EA, starting in 2012-2013 and extending over the 5-year planning cycle. However, the key EA initiatives identified for 2012-2013, which included the development of an EA strategy and integration of EA into the PPMF, did not occur due to resource issues. EA initiatives for 2013-2014 and subsequent years were to build on these 2012-2013 initiatives, so will now be delayed.

The Department engaged Gartner Consulting in 2012 to benchmark the application portfolio and assess applications based on business value, technical condition and support costs. This is valuable input to a potential rationalization of the application portfolio, with a focus on retiring applications that are of lower value and may have technical issues or relatively high support costs. The application portfolio information is now available in a tool for assessment purposes, and this can be used in support of the EA initiative.

In addition, as discussed in section 5.1.1, ASC is currently inactive, yet it is the body responsible for enforcing compliance with technology standards and will be expected to play a role in implementing EA.

Recommendation

5. The Chief Information Officer should ensure that Enterprise Architecture (EA) initiatives are addressed on a priority basis and leveraged to enhance application systems integration in future project investments. Specifically, timelines should be confirmed and resources assigned for the key EA initiatives that had been identified for 2012-2013, consisting of development of an EA strategy and integration of EA with the PPMF, as these were the basis for future initiatives. Further, the Architectural Standards Committee should be re-instituted and reactivated to perform its role as the governance body ensuring IM/IT investments comply with approved technology standards and enterprise architecture initiatives (including current terms of reference produced, membership confirmed, and regular meetings initiated).

5.3. Monitoring and Accountability

5.3.1 Roles and Responsibilities

To assess accountability, we examined whether an organization structure is documented for IMB, with clear assignment of roles and responsibilities. We also examined whether roles and responsibilities as defined by TBS requirements had been clearly defined within IMB.

We found that the organization structure for IMB is set out in an organization chart dated April 2013, and this has been approved by the CIO and CFO. We were advised that the structure for the Enterprise Strategic Services Directorate as set out in the organization chart is in the process of being revised to more accurately reflect some of the Branch's priority areas, including implementing enterprise architecture, but that it is otherwise current. For IT, the roles and responsibilities per the organization chart were compared to the eight IT workstreams defined by TBS and were found to be clearly assigned to a directorate. For IM, the requirements of the TBS Directive on Information Management Roles and Responsibilities were examined and roles of IM specialists found to be consistent with the roles and responsibilities assigned within the IM directorate. The CIO has been designated as the senior official to represent the deputy head in discussions with TBS on IT and IM issues, as set out in the Department's Policy on Management of Information Technology and Policy on Information Management respectively.

5.3.2 Monitoring and Accountability

We examined whether key monitoring and accountability activities are performed in the following areas:

  • Initiatives as set out in the IM/IT Plan;
  • The portfolio of IM/IT-enabled projects; and,
  • IM/IT spending in the regions and sectors.

Monitoring of compliance with the IM/IT strategy and the initiatives set out in the IM/IT Plan occur both informally through the CIO monthly meetings with each of his Directors and formally through the CFO Sector Quarterly Report. An examination of the CFO Sector quarterly report for the 2012-2013 fiscal year indicated that key IMB initiatives as set out in the IM/IT Plan were being tracked on a quarterly basis and results reported. Where quarterly deliverables had not been achieved, the reasons were noted as well as a plan to remediate, and this was tracked in the subsequent quarter. The exception to this is the key initiatives for enterprise architecture which, per the CIO, were parked for 2012-2013 since they had no resources to address these items at the time.

Although annual Management Accountability Framework (MAF) reporting is also a key monitoring tool, TBS has taken a modified, risk-based approach to MAF reporting, assessing Departments on a selection of high-risk / high priority Areas of Management. As a result, Departments have not been required to report on the MAF areas related to either IM or IT in the past three fiscal years.

IMB has also recently introduced project portfolio reporting to DGIOC and DOC, with the first presentation to DGIOC in May 2013 and DOC scheduled for late October. This has been implemented in response to a concern by governance bodies that individual projects get approved through the governance committees, but they are then managed through their development and implementation by the project Steering Committees, without further oversight by the senior governance committees. As a result, a process has been initiated to monitor the health of the portfolio of projects more closely, with portfolio reports to be prepared for DGIOC semi-annually, although expected to become quarterly, and the frequency to DOC yet to be confirmed. An examination of the May presentation to DGIOC noted it included a summary of the portfolio of projects by state (active, on hold, etc), by gate, by strategic outcome and by program area. The related Record of Discussion of DGIOC indicated a detailed discussion of the portfolio areas with requested enhancements to address risk areas on a go-forward basis.

The AANDC Policy on Management of IT, which was effective January 1, 2013, and the AANDC Directive on IM and IT Procurement Authorization, which was effective March 31, 2012, were issued to enhance control over IM and IT expenditures across all sectors and regions by requiring CIO pre-approval for all IM/IT expenditures as well as common financial coding to enable consistent tracking of IM/IT expenditures. A key activity being implemented to monitor compliance with these instruments is the IM/IT spend review currently underway, which was originally undertaken since required by Treasury Board, but will be used for additional internal monitoring purposes. This IM/IT spend review requires reporting from each sector and region of 2012-2013 IM/IT expenditures, including both salary and non-salary items, and classified into five categories as follows: distributed computing, application/ database development and maintenance, production and operations computing, telecommunications, and IT security. This data is expected to be gathered annually.

If this IM/IT expenditure exercise is to be an effective monitoring tool in determining whether all projects undertaken by regions and sectors were appropriately pre-approved, further detail will be required as to specific projects undertaken in each of these categories. The current reporting templates provide information within each category by type of expenditure, including salary, hardware, software, etc., but no information as to specific projects or initiatives that make up these amounts.  As an example, our testing of BC region indicated approximately $159,000 was spent on a 'thin client' project, with the only evidence provided by the region to indicate approval by the Director of Enterprise Strategic Services is an e-mail in February 2013, which approved replacing desktops only and which was issued after the project was underway. Information on this project is not evident from the current IM/IT expenditure analysis. We note that this is the first year the related policy and directive have been in effect, and that the spend review is currently in process, with the immediate focus on submission of expenditure amounts to TBS for October 15, 2013. As a result, further analysis of the financial information and the follow-up by the CIO of any indications of unusual or potentially unauthorized spend activity has yet to be completed.

Recommendation

6. The Chief Information Officer should ensure that appropriately detailed information is reported by sectors and regions in the annual IM/IT spend analysis, especially information on specific IM/IT projects/initiatives undertaken, in order to effectively monitor compliance with departmental policies requiring pre-approval of all IM/IT expenditures. Once this additional information is gathered and evaluated, any indications of non-compliance with required pre-approvals should be evaluated and, based on significance, corrective actions should be initiated, such as direct follow-up with the region/sector or escalation to governance committees.

 

 

6. Management Action Plan

Recommendations Management Response / Actions Responsible
Manager (Title)
Planned
Implementation Date
1. The Chief Information Officer should strengthen governance and strategic direction related to information management by establishing or enhancing the following key elements:   Chief Information Officer  
  • Develop and approve an IM Plan consistent with the priorities identified in the Department’s IM/IT Strategy, including prioritized initiatives for the five-year planning horizon. This may be part of an integrated IM/IT Plan or a separate IM Plan, but it is critical that IM direction be clarified. The draft Enterprise IM Strategy should be withdrawn to avoid confusion with having both an IM/IT Strategy and a separate IM Strategy.
The Director of CIMD in collaboration with the CIO, other Directors in IMB, and members of ITSG and DGIOC, will develop and approve an IM Plan consistent with the priorities identified in the Department’s IM/IT Strategy, including prioritized initiatives for the five-year planning horizon. June 30, 2014
The draft Enterprise IM Strategy will be withdrawn. November 30, 2013SSE
  • Ensure required IM policies and directives are finalized and approved, and are communicated within the Department to all impacted employees. Of priority are those meeting TBS requirements, especially the Directive on Recordkeeping, where compliance is required by March 2015.
The AANDC Record Keeping Directive and E-Mail Management Directive will be reviewed, updated and approved. January 30, 2014
All approved policy instruments will be communicated to all employees via AANDC Express. March 31, 2014
All IM/IT Policy Instruments will be organized into a prioritized list for review, update and approval as part of a life-cycle review and update. March 31, 2014
An Intranet review and update process will be completed to remove draft policy instruments and ensure that only official/final versions are available. December 15, 2013
  • Ensure IM is a regular standing item at governance committees to monitor implementation of IM policies and directives as well as progress of key IM initiatives within the approved plan.
Forward Agendas for IMB’s Branch Management Team meetings as well as IM/IT governance committees (e.g. ITSG) will be reviewed and updated to ensure that there are IM agenda items discussed at least once a month, and escalated as required to DGIOC and/or DOC. December 15, 2013
2. The Chief Information Officer should continue the process recently initiated of engaging sectors and regions through the five-year call for investment plans in order to facilitate integration of the IM/IT Strategy and Plans with departmental investment planning. In addition to those approved by Operations Committee as part of the Department’s Investment Plan, which includes projects greater than $1 million only, these results should be incorporated into the annual updates of the IM/IT Plan to help ensure the IM/IT Plan remains consistent with the overall priorities of the Department and to allow for appropriate Information Management Branch resource planning. The annual call for IM/IT initiatives will continue to be included in all future calls for Investment plans. Chief Information Officer March 31, 2014
Results will be included in updates to the Tactical IM/IT Plan March 31, 2014
3. The Chief Information Officer should establish a centralized function to manage all IM and IT policies and directives to help ensure that they are tracked from draft status through to finalization and approval, and to manage the posting of only approved policies and directives on the Information Management Branch intranet site. This centralized function should track when policies and directives require review based either on the established timeline or when TBS requirements are modified, and monitor the process for updating. A communication process should be established to communicate the requirements of all new or modified policies and directives on a consistent basis to all those responsible for implementing them. The office of the CIO will:

  • Manage the development and updates to IM/IT policy instruments
Chief Information Officer November 15, 2013
  • Track the development/update of new and existing policy instruments
November 15, 2013
  • Manage the posting of existing/new policy instruments
November 15, 2013
  • Develop an update life-cycle for all IM/IT policy directives
March 31, 2014
  • Prepare communiqués for AANDC Express to highlight the requirements of all new policy instruments.
March 31, 2014
4. The Chief Information Officer should enhance the Project Portfolio Management Framework (PPMF) documentation/adherence by:   Chief Information Officer  
  • clarifying the requirements for information management considerations in the PPMF, including how these will be documented and monitored in the gating approval process;
The Chief Information Officer will create an Information Management overlay/underlay that will identify the information management considerations and requirements for each gate of the PPMF. February 1, 2014
  • developing, approving, and posting to the Information Management Branch intranet site a document that clearly outlines the current requirements for approval by governance bodies at each gate in the PPMF gating process;
The Director of Enterprise IM/IT Strategic Services will develop a document that clearly outlines the current requirements for approval by governance bodies at each gate in the PPMF gating process.  February 1, 2014
Once approved, the document will be posted on the Intranet. February 28, 2014
  • clarifying PPMF requirements for projects requiring Treasury Board approval as these projects should follow the Department’s defined and approved project governance process; and,
The Director of ESS will create a document clarifying PPMF requirements for projects requiring Treasury Board approval to ensure that PPMF gating requirements are known for projects requiring TBS approval. March 1, 2014
  • ensuring that formal project close-out as defined in the PPMF, including lessons learned and benefits realization, is enforced.
The CIO, in collaboration with ITSG and DGIOC will update the IM/IT Frameworks and documents to ensure that formal project close-outs are completed. February 1, 2014
5. The Chief Information Officer should ensure that Enterprise Architecture (EA) initiatives are addressed on a priority basis and leveraged to enhance application systems integration in future project investments. Specifically, timelines should be confirmed and resources assigned for the key EA initiatives that had been identified for 2012-2013, consisting of development of an EA strategy and integration of EA with the PPMF, as these were the basis for future initiatives. Further, the Architectural Standards Committee should be re-instituted and reactivated to perform its role as the governance body ensuring IM/IT investments comply with approved technology standards and enterprise architecture initiatives (including current terms of reference produced, membership confirmed, and regular meetings initiated). The Director of ESS, in collaboration with all Directors in IMB, will:
  • Develop an EA Strategy
  • Develop an EA overlay/underlay for the PPMF
  • Re-institute the ASC with Terms of Reference
  • Develop EA Principles/guidelines
Chief Information Officer March 31, 2014
6. The Chief Information Officer should ensure that appropriately detailed information is reported by sectors and regions in the annual IM/IT spend analysis, especially information on specific IM/IT projects/initiatives undertaken, in order to effectively monitor compliance with departmental policies requiring pre-approval of all IM/IT expenditures. Once this additional information is gathered and evaluated, any indications of non-compliance with required pre-approvals should be evaluated and, based on significance, corrective actions should be initiated, such as direct follow-up with the region/sector or escalation to governance committees. The CIO will develop and implement compliance process(es) and activities to ensure that appropriate spending is occurring outside of IMB. Chief Information Officer April 30, 2014
 
 

 

Appendix A: Audit Criteria

To ensure an appropriate level of assurance to meet the audit objective, the following criteria were developed to address the objective:

Audit Criteria
1 Governance and Strategic Direction
1.1 Effective management and oversight bodies for IM and IT are established and functioning.
1.2 Strategic and tactical plans for IM and IT are developed and are being implemented.
1.3 Policies and directives have been developed for key areas of IM and IT, are appropriately approved and have been effectively communicated.
1.4 IM and IT risks are identified and documented along with the risk response and mitigation strategies.
1.5 Agreements have been documented and approved between IMB and Sectors to confirm funding for system development, acquisition and maintenance activities.
2 Application System Development, Acquisition and Integration
2.1 A Project Portfolio Management Framework has been defined to manage IM/IT projects.
2.2 The Project Portfolio Management Framework is being adhered to by the Department.
2.3 The framework for system development and acquisition explicitly addresses system integration opportunities at key steps.
3 Monitoring and Accountability
3.1 Progress against IM and IT strategies and plans is monitored and reported, and remedial action taken where required.
3.2 Processes and procedures are in place to identify and monitor compliance with key Treasury Board and other Government of Canada requirements related to IM and IT.
3.3 Monitoring of HQ and Regional IM/IT spend in accordance with the Policy on Management of Information Technology and the Directive on IM/IT Procurement is performed on a periodic basis and unusual items investigated.
 
 

 

Appendix B: Relevant Policies/Directives

The following authoritative sources were examined and used as a basis for this audit:

  1. AANDC Policy on Information Management
  2. AANDC Directive on IM and IT Procurement Authorization
  3. AANDC Directive on IM/IT Project Portfolio Management
  4. AANDC Directive on Recordkeeping
  5. AANDC Directive on Data Sharing
  6. AANDC Directive on Data Stewardship
  7. AANDC Project Portfolio Management Framework
  8. INAC Security Management Framework
  9. TB Policy on Information Management
  10. TB Policy on the Management of Information Technology
  11. TB Directive on the Management of Information Technology
  12. TB Directive on Information Management Roles and Responsibilities
  13. TB Directive on Recordkeeping
 
 
 
Date modified: