5.2.1 Internal Client Service Standards
Effective client service is enabled by implementing, monitoring and managing client service standards. Service standards should accurately reflect service that clients are receiving, and client feedback should be solicited to accurately reflect value in services provided. The CFO Sector plays an important role in providing internal services to other AANDC sectors and business units within the Sector, including IM and IT , procurement and transaction processing.
While we found some service standards in place within the Sector, client-centered service was found to be the control area with the greatest opportunity for improvement. In particular, we found limited client-centered performance measurement across IM and IT , accounting, procurement and other Sector processes that serve clients. Specifically, we found few service standards and service level agreements across business units within the Sector. We also found limited evidence that feedback was solicited from clients to inform process design decisions.
220.127.116.11 IT Services
IMB has formalized its IT funding arrangements with internal clients, improving the Branch's financial control framework. We were informed that a memorandum of understanding (MOU) is now established with each sector, outlining what high-level IT services will be delivered by IMB for the Sector as well as the associated funding arrangement. Through a review of MOUs provided, we noted that, consistent with leading practices, a formalized funding specification with departmental sectors was in place.
We understand that IMB also maintains a service catalogue of the IT services available to the Department. We were informed by interviewees that the catalogue is outdated and currently under review by IMB. The Chief Information Officer confirmed to the audit team that the service catalogue is outdated and is not being used. It was also noted that the revised catalogue would place greater emphasis on client-centered service.
While the client relationships for IT have become more formalized, performance is not measured from a client standpoint. We found limited evidence of client-centered service standards for IT services. We also found that IMB was not aware of whether Shared Services Canada (SSC) was tracking service standards on AANDC's behalf.
We were informed of some informal channels for clients to express their concerns and satisfaction levels, including through discussions with IMB Customer Relationship Managers (CRM) assigned to each client group. While informal feedback loops may exist, no evidence was provided to assess if any client feedback is reported or documented to capture their input and how it is trending over time. Accordingly, we could not ascertain the level of service quality perceived by IMB clients through the existing service standards.
To continue the maturation process underway within IMB, client-centered service standards should be established with performance measures that consider both quantitative and qualitative measures of client service success.
18.104.22.168 Proposed Approach to Service Standards in Hubs
Timely processing of financial transactions is an important aspect of effective financial management and governance. Processing timeliness enables AANDC's sectors and regions to serve the Department's client base more effectively. The CFO Sector plays an even more critical role now in enabling such timeliness by providing direction to the newly formed accounting and procurement hubs. These hubs were implemented on an accelerated timeline, several months in advance of what we understand to be the originally intended go-live date. The Sector responded quickly and has been working to formalize business processes through documented workflows with revised service-level standards. Representatives interviewed had a consistent view of consolidation objectives, which included greater service efficiency and more consistent client service.
Similar to the lack of feedback solicited from clients on IT service performance, we found no evidence of feedback sought for accounting and procurement services (prior to implementation of the hubs as well as at the time of the audit). We understand procurement services managers are exploring the option of seeking client feedback through a survey once hubs have stabilized. However, we found no plans to seek feedback for accounting services. We also found no evidence to demonstrate that client-centered service standards for accounting processes have been, or will be, measured. While a post-verification service standard exists, it does not measure payment processing timeliness or performance from a client's perspective.
We found opportunities for improvement in how service standards are defined and measured in both the accounting and procurement hubs. We understand that service standards in the procurement hubs will be adjusted to measure a narrower scope of activities, which include only activities that are performed by the hubs themselves, rather than end-to-end processing times that reflect the full service that clients are receiving. Monitoring the intermediate steps that are only performed by the hubs, instead of monitoring the complete process, will not provide a full view of client services. While this approach provides a more accurate link between activities, performance and individuals accountable for performing the activities, it limits the view of how the end client is impacted. As a result, it becomes difficult to diagnose bottlenecks, identify synergies and opportunities for efficiency across activities, and incentive for organizational units to work together for the best interest of the client.
While a complete performance measurement framework may require a granular view of performance by activity, it is also essential to establish service standards that consider the end-to-end chain of activities that impact the client.
Client-centered service standards in hubs may be an important opportunity for improvement as the hubs are centralizing client services that were previously performed closer to the clients. These service standards should be designed to measure service delivery to the client. Intermediary steps (such as work steps or activities performed by other organizational units or departments) may require measurement to better understand performance and identify issues; however, there also needs to be an end-to-end measurement that is meaningful for clients served.
22.214.171.124 Reporting Against Existing Procurement Service Standards
Procurement service standards were implemented in 2012 and continue to be tracked within the procurement hub.
While having service standards defined is a leading practice, we found significant opportunities for improvement in how these performance metrics are measured and reported on. In particular, a key procurement service standard is the time between final contract signatures (i.e. Section 41 under the Financial Administration Act) and the transaction processing start date. The service standard is defined in business days for five types of contract requests. This standard is tracked using an Excel spreadsheet, which has given rise to data integrity challenges. In particular, we found that over 12% of the fields were inaccurate, displaying either a negative time differential between s.41 and fund allocation or a blank date field for a transaction processed. We were informed that these data integrity gaps result from operational challenges created by using Excel; specifically, only one person can access the spreadsheet at a time, increasing the likelihood that staff will not enter the necessary data fields as they process transactions.
Based on the data that remained (i.e. transactions with missing date fields or negative date differentials removed from the data set), we found high exceptionFootnote 2 rates by contract type based on the raw data. We found that less than two thirds of the requests met the service standards from a population of 2,000 procurement requests in 2012-13. For some contract types, we found that less than half of the requests met the standard. While a high exception rate may be expected in a time of significant change, the more significant observation noted by the audit team was how service standards were reported on.
Specifically, we found that the procurement performance reported by the NCR hub was unclear and inaccurate. The reports provided to us could not be reconciled against the analysis we performed on the raw data. In particular, the metric reported on is "average calendar days" per contract type; however, in calculating this metric from the raw data, we could not arrive at the same numbers reported. Further analysis suggested that the data integrity problems were impacting the rolled-up results. Through inquiry, we understood this variance was likely due to data changes made during the reporting process to address known data errors in the raw data spreadsheet. Interviewees described significant challenges in tracking service standards due to a lack of functionality in OASIS (the Department's financial system), specifically, a lack of data fields within the OASIS procurement module to capture the necessary information. However, interviewees noted that better measurements will be possible once the Department moves to SAP. The current solution relies on spreadsheets, lacking integrity and creating additional manual work.
We concluded that the data being tracked lacks integrity and is therefore being manually updated to arrive at reports on performance. The observations suggest that, even with service standards defined, the tools to track metrics are necessitating manual work, creating inefficiencies and resulting in unreliable reports.
126.96.36.199 Information Management Across Hubs
The accounting hub established a paperless process for payment processing. Unlike previous processes that relied on both paper and electronic records, with inconsistencies across regions, the centralized process is paperless and enforces central storage of electronic transaction records. In addition to supporting the federal Sustainable Development Strategy targets for greening government operations, a paperless approach also enforces a consistent system of record in support of process integrity and effectiveness.
The hubs are leveraging the Department's document management solution, the "Comprehensive Integrated Document Management (CIDM)" system, using defined naming conventions and standards. While leveraging the Department's supported tool for document management versus establishing other work-around systems is consistent with leading practice, the IM practices observed to work with the system are not effective. Specifically, we found ineffective document management procedures in place that are creating significant inefficiencies. As an example, both the accounting and procurement hubs had defined file naming conventions, but the conventions were not the same across the two hubs.
As described further in Section 4.5, document sharing across hubs is an essential activity for payment processing, but it is manually intensive for the Sector's clients due to a lack of a well designed document management process. The impact is a potential delay in payment processing due to manual work, in addition to increased workload on an already taxed workforce. Greater focus on integrating and embedding effective IM solutions within existing workflows should be applied to improve service delivery.