RCMP Findings Relating to the Office of the Comptroller General Horizontal Internal Audit of Compliance with the Policy on Management, Resources and Results Structures

Final Report: April 2013

This report has been reviewed in consideration of the Access to Information Act and the Privacy Act. An asterisk [*] appears where information has been removed; published information is UNCLASSIFIED.

Table of Contents

Acronyms and Abbreviations

ADM Associate Deputy Minister
APPS Annual Performance Plan System
BSC Balanced Scorecard
CIDA Canadian International Development Agency
CPA Canadian Police Arrangement
DFAIT Foreign Affairs and International Trade Canada
DPR Departmental Performance Report
IPPPOP International Police Peacekeeping and Peace Operations Program
MAF Management Accountability Framework
MRRS Management, Resources and Results Structures
OCG Office of the Comptroller General of Canada
PA Program Activity
PAA Program Activity Architecture
PMF Performance Measurement Framework
PS Public Safety
RCMP Royal Canadian Mounted Police
RPP Report on Plans and Priorities
TB Treasury Board
TBS Treasury Board Secretariat

Executive Summary

The Royal Canadian Mounted Police (RCMP) was one of fourteen large departments and seven small departments selected by the Office of the Comptroller General (OCG) to participate in a Horizontal Internal Audit of Compliance with the Policy on Management, Resources and Results Structures (MRRS). The planning phase of the audit, including development of the objective, scope and criteria, was completed by the OCG. RCMP Internal Audit (IA) was asked to complete the conduct phase of the audit using a methodology developed by the OCG and to report findings and conclusions to the OCG for inclusion in a consolidated report.

The OCG report concluded that "departments are generally compliant with the requirements of the Policy on MRRS". The report highlights the need for additional guidance from Treasury Board Secretariat (TBS) regarding performance measurement, the definition and allocation of internal services and program linkages across departments, and clarification with respect to the management of departmental resources and the use of the Program Activity Architecture (PAA) Footnote 1 as the basis for allocations. Six recommendations are included in the OCG report, two of which include the RCMP as an addressee.

The OCG consolidated findings are consistent with the situation within the RCMP. While the general principles of the Policy on MRRS are being adhered to, management practices are advancing towards full implementation. The department is working to ensure that their information systems, performance measurement strategies, and reporting and governance structures are consistent with their MRRS but further work is needed to ensure full integration.

This report provides additional detail with respect to RCMP's compliance with the Policy on MRRS and includes Management Action Plans which address the recommendations put forward by the OCG. RCMP IA will monitor the progress made in implementing the Management Action Plans and will undertake follow-up activity if warranted.

1. Objective, Scope and Methodology

1.1 Objective

The objective of the OCG horizontal internal audit was to assess compliance with the Policy on MRRS.

1.2 Scope

The RCMP was one of fourteen large departments & seven small departments included in the scope of the audit. The Secretariat, because of its role as a central agency was also included.

The audit examined the MRRS, including but not limited to, the design and clarity of Strategic Outcomes, the Program Activity Architecture (PAA) and governance structures that were in place to manage program performance within departments. Although the Policy on MRRS does not specifically refer to horizontal initiatives, the audit examined how the policy's principles apply to horizontal initiatives.

The primary period of the audit was 2011-2012 fiscal year. Plans and processes implemented for 2012-2013 planning were also taken into consideration to supplement evidence, as required.

1.3 Methodology

The planning phase was completed by the OCG. The OCG also determined the methodology to be used during the examination phase (RCMP Internal Audit conducted examination/information gathering). Participating departments submitted their results, and the OCG reported on the consolidated findings.

2. Audit Findings

2.1 Central Agency Guidance

Greater central agency guidance is required to support departments in the implementation of some aspects of the Policy on MRRS.

This finding is related to TBS's role in providing guidance to Departments to support compliance with the Policy on MRRS. Detailed findings can be found in "Finding 1" of the OCG's Horizontal Internal Audit of Compliance with the Policy on Management, Resources and Results Structures report.

This finding resulted in a recommendation directed to TBS.

2.2 Strategic Direction and Accountability Structures

Strategic directions are established in departments, and accountability structures have been developed in most departments to support them.

The RCMP has recently updated and defined three measurable Strategic Outcomes that reflect its mandate and vision and are linked to overarching government priorities and intended results.

The Strategic Outcomes are long-term and enduring in nature, measureable through the department's Performance Measurement Framework (PMF) and linked to the RCMP's mandate. The outcomes also provide the basis for establishing horizontal linkages between departments with similar strategic outcomes such as Canada Border Services Agency and Public Safety (PS).

In an effort to provide further strategic direction, the RCMP has identified 5 priorities based on environmental scanning and an analysis of crime trends and the national threat assessment. Identifying priorities allows the operations to be more strategically focussed in order to enhance public safety. For the period under review each strategic priority had outcomes and objectives which were developed and evaluated by a working group led by a Deputy Commissioner. The RCMP utilized the Balanced Scorecard (BSC) to inform decision making by senior management on strategic priorities. Each priority contributes to one of the department's Strategic Outcomes.

The Commissioner provides leadership over the organization's strategic direction and consults with key stakeholders through the Senior Management Team. As part of this process, Commanding Officers were involved with the development of the RCMP's Strategic Outcomes.

The Department has a governance structure in place that outlines decision-making mechanisms, responsibilities and accountabilities. The Commissioner approves the RCMP's MRRS components [Reports on Plans and Priorities (RPP), Departmental Performance Report (DPR), PAA, etc.)], and with the collaboration of the Senior Executive Committee, ensures timely updates as required. Senior management are held accountable for agreed outputs and outcomes set out in the MRRS. In 2011-12 each executive was required to include at least one initiative from their BSC in their performance agreement. The Department has been working to improve the way the process is administered.

The RCMP is working to ensure that their information systems, performance measurement strategies, and reporting and governance structures are consistent with their MRRS but further work is needed to ensure full integration and implementation. The Department is actively monitoring its compliance with the MRRS Policy and interacts with the TBS on any changes it intends to make as the MRRS is the basis for reporting to parliament.

No recommendations were made as a result of this finding.

2.3 Resource Allocation

The PAA is not the sole tool for resource reallocation within departments and across government.

The RCMP has a PAA structure which captures the activities of the department. As part of the PAA, all program activities, sub-activities and sub-sub activities are described and defined to specifically identify what the activity encompasses. Program descriptions have recently been updated to improve clarity and completeness as well as to ensure that the descriptions fulfil the requirements of the MRRS Policy.

Within the PAA, the sub-sub activities are aligned to the program sub-activities which are aligned to the broader program activities which are linked to specific RCMP Strategic Outcomes and further linked to the Government of Canada outcomes.

The PAA forms the basis for RCMP's external reporting, in the form of the RPP and DPR.

Each reporting cycle, TBS releases templates to complete these reports to ensure uniformity for reporting across departments. This template is then circulated to all programs directly responsible for activities enumerated in the RCMP's PAA. Performance indicators, which are consistent with the PMF, are then provided for inclusion in the RPP, along with a narrative that helps explain the priorities and planning highlights for each program, and linked to the resources provided in the Main Estimates.

The DPR follows this same process, as the performance measures and targets are reported upon, organized by PAA.

From a financial perspective, Main Estimates are prepared and presented at a Program Activity (PA) level as per Treasury Board (TB) policies and guidelines. The historical funding levels by PA reflect past submissions, transfers between departments and intradepartmental allocations, that have been approved throughout the years and that are now considered part of the departments base funding. These are the "opening balances" for each program activity that are then adjusted to reflect any approved changes resulting from new Treasury Board Submissions, other technical adjustments or internal reallocation decisions. The RCMP's financial system is currently configured to account for expenditures by PA within the department.

When requesting new funding, the department prepares a Treasury Board Submission, which includes information about the initiative and the financial implications to the department. The submission is supported by detailed business cases that outline where the new positions are going to be created and what the funding is going to be used for. A PA allocation of the funding that is being sought is also presented. The department will allocate within its reference levels the funding received according to the information presented in the submission.

Throughout the fiscal year, the department may request additional funding to meet new financial requirements that arise during the year. These funds are allocated to the proper PA through different mechanisms. Funding reductions such as those contained in the Strategic Reviews are allocated across program activities as well based on the approved reduction proposals.

Although the PAA serves as the basis for resource allocation by departmental management, the integration of MRRS information with all reporting requirements for the RCMP is still in development. The RCMP is not yet making full use of MRRS information to enhance decision making as observed in TBS' 2011-12 detailed Management Accountability Framework (MAF) assessment of the RCMP. The following observations with respect to resource allocation were made:

  • MRRS information is not being used to support documents submitted to central agencies to justify funding decisions.
    • In the three examples submitted to TBS no clear reference was made to MRRS information (PAA or PMF); RCMP has not demonstrated its use of MRRS information to support planning and decision making.
    • When submitting a document to central agencies, RCMP is invited to situate where in the PAA the program may be found and to which Strategic Outcome it contributes; or, for a new program, where it would be placed. With regard to the PMF, RCMP should identify what expected result and performance indicators are related to this program that will enable them to generate more relevant performance management information. Further guidance will be communicated to departments for MAF 2012-13.

This finding resulted in a recommendation directed at TBS.

2.4 Performance Monitoring

Departmental frameworks to measure performance do not always provide the results information needed to assess how programs are achieving their expected results.

The RCMP has created a PMF and is collecting performance data, however this information does not fully support planning and decision making.

An objective and measurable PMF provides assurance that intended results are achieved and, if necessary, facilitates proactive corrective measures. The timely collection and analysis of performance information enables management to make reallocation decisions and continuous improvements to business operations.

At the time of the audit, the RCMP's PMF articulated expected results and outputs for each program activity, sub-activity and sub-sub activity. Performance indicators have been identified for each along with targets and measurement reporting timeframes. Based on the examination of the 2010-11 reports, the performance indicators and targets are set out in the RPP and reported against in the DPR. It was noted that targets have not yet been defined for all performance indicators.

As part of its performance management regime, the RCMP relies heavily on the BSC. As explained in the information provided by the department to TBS for the MAF assessment, performance information from the BSC is intended to inform decision making by senior management on strategic priorities. In addition to the Senior Executive having the Balanced Scorecard to help guide decision making at the overall organizational level, each of the five strategic priorities of the RCMP had a working group who monitored progress on the priority through a dedicated BSC for the priority. Every objective had performance measures that were monitored and reported against on a quarterly basis, and adjustments were made based on information obtained.

A similar process is established at all levels of the organization through the use of the Annual Performance Plan System (APPS). The APPS is a web-based tool which assists front-line managers at the unit and detachment level to identify issues, develop objectives, set performance indicators and track performance on a quarterly basis.

Related to performance measurement, the 2011-12 detailed MAF assessment found that:

  • Most performance indicators are clear and valid measures for their respective strategic outcome(s), expected results and outputs.
  • Overall, performance indicators are valid and reliable measures of their corresponding expected results and most of them are clear and in the appropriate format. There are only a few that require further clarification.
  • RCMP's performance measurement framework contains a few indices, e.g., Client Satisfaction Index (performance, value for money, value to police learning, post course satisfaction level), but it is unclear how these indices will be measured.
  • Financial information in the DPR is sufficiently clear and explained with text, as necessary. Linkages between resources and results are adequately demonstrated.

In contrast, the RCMP had not demonstrated its use of MRRS information to support planning and decision making. The MAF assessment rated both the Quality of the Performance Measurement Framework and the Quality of Performance Reporting as "Opportunities for Improvement".

Quality of the Performance Measurement Framework

  • An incomplete performance measurement framework has been developed.
    • Targets have not been developed for all expected results' performance indicators in the PMF.
  • Many expected results are not clear outcome statements that are appropriate for their respective programs.
    • Some expected results do not represent clear outcome statements and are activity based.
    • RCMP needs to provide complete program descriptions at the Sub Activity and Sub-Sub Activity level of the PAA in order to validate the appropriateness of expected results developed at these levels.

Quality of Performance Reporting

  • The DPR is generally balanced - the report presents both positive and negative aspects of performance and substantiation or explanation is provided.
    • The DPR provides good balance in the discussion of Lessons Learned in Section II and the explanation of performance status.
  • The DPR makes limited use of the organization's MRRS (i.e., PAA and PMF) in linking plans to performance.
    • Expected results and targets vary from those identified in the RPP and PMF of record. The DPR would benefit from greater consistency in its use of the PMF.
  • Some information on the validity and credibility of data used is provided. A few relevant findings from audit and evaluation are included. It is difficult for the reader to identify the source of data and information reported in the DPR as well as the quality of the underlying data.
    • The DPR would benefit from inclusion of results and actions stemming from the audits and evaluation anticipated in the RPP and identified in the Internal Audits and Evaluations supplementary table in the DPR.
    • The DPR would benefit from greater discussion of the impact of resource changes on expected results at the PA level.

The existence and use of performance information to assist better decision making is evolving from early stages of development, especially at lower program levels.

Without targets for all performance measures, it is difficult to determine if program activities are meeting objectives. Performance data is necessary to provide management with information to support decisions about operational delivery.

This finding resulted in the following recommendation directed to the RCMP and other departments: "As part of their Performance Measurement Frameworks, departments should improve Management, Resources and Results Structures performance measures, in support of planning and decision-making processes."

2.5 Horizontal Initiatives

Central Agency guidance is required to support departments in the management of horizontal initiatives.

The horizontal initiative involving the RCMP which was examined in the audit was the International Police Peacekeeping and Peace Operations Program (IPPPOP). The IPPPOP is managed through the Canadian Police Arrangement (CPA) framework which consists of a partnership between the Department of Foreign Affairs and International Trade Canada (DFAIT), RCMP, Canadian International Development Agency (CIDA) and PS.

The IPPPOP supports the international deployment of Canadian police. The IPPPOP is a foreign policy tool that aims to stabilize fragile states and conflict affected situations through the re-establishment of effective public institutions. The RCMP plans and evaluates missions, selects and trains personnel and provides comprehensive support throughout the deployment and return to Canada; manages relationships with partner police services and ensures Memorandums of Understanding are in place to support the human resource requirements of the IPPPOP.

Governance

There was a clear governance structure for this initiative as laid out in the CPA Agreement which also outlines the RCMP's role and responsibilities.

Outcomes and expected results for this horizontal initiative are concrete and aligned with the IPPPOP objectives outlined in the TB Submission as well as with RCMP's mandate.

The expected results and indicators are outlined in the draft CPA logic model and associated performance measurement strategy clearly align with the RCMP's International Policing Operation Program Activity, and with the sub-program activity of International Peace Operations. Responsibility for each of the CPA performance indicators has been formally assigned to a CPA partner department/Agency in the draft CPA PMF. The PMF outlines how progress towards the achievement of expected results will be measured.

A CPA Steering Committee (ADM-level), a CPA DG Advisory Committee and a CPA Working Group are in place to support IPPPOP. Terms of Reference are in place for each CPA committee and working group, outlining the associated roles and responsibilities. RCMP is a member on each committee.

RCMP is ultimately accountable for delivering expected results for this initiative given it is highly operational. The CPA Steering Committee and working group play an active role in monitoring the initiative. RCMP executives are held accountable for tactical IPPPOP results and financial management performance through performance management agreements and associated performance reviews.

Formal reporting for the IPPPOP is in place. The CPA requires the production of an Annual Report. The 2010-2011 CPA Annual Report reflects financial information as well as operational performance information.

A formal Evaluation of the program is required every five years as part of the renewal of the TB Submission. An evaluation was completed in 2010 and found that the program was successful in meeting its objectives.

An opportunity for improvement was noted related to the publication of program results. Information about the IPPPOP is communicated on the RCMP's website but it is at a high level and very limited without making reference to or being provided a link to the Annual Plan and Annual Report.

Allocation of Resources

RCMP has designed appropriate processes and procedures to support allocating and monitoring resources in line with IPPPOP objectives (note that the processes are being modified to provide alignment with the recently developed draft PMF). The IPPPOP and related CPA objectives and expected results as well as those of the RCMP provide a framework for the allocation and monitoring of resources for this horizontal initiative.

With respect to specific IPPPOP missions, DFAIT determines the foreign policy priorities. International deployments are discussed and agreed upon at the CPA Working Group. The objectives for each mission, along with the number of police personnel to be deployed are reflected in a 'Concept of Operations'.

The Concept of Operations incorporates the objectives, expected results, duration and projected RCMP reference levels and incremental costs, as well as administrative and logistical details, consistent with the information contained in the memoranda to CPA Ministers, seeking approval of deployments. Resources for individual missions are to be determined by RCMP as part of the planning process and linked to the Concept of Operations for the specific mission. It was determined that of 11 missions, there was only one mission with a Concept of Operations in place.

The CPA outlines the costing model for the IPPPOP. RCMP is the lead for developing the budget for each mission consistent with the costing model outlined in the CPA. The costing model outlines the methods by which Program costs funded by the RCMP are to be allocated (out of the RCMP reference levels from the International Assistance Envelope).

While the monthly reporting is currently highly operational in nature, RCMP has developed a draft tool to capture performance information from the field in line with relevant elements of the draft CPA PMF.

This finding resulted in two recommendations for TBS and the following recommendation was directed to the RCMP and other departments: "Departments should finalize and implement frameworks for monitoring the performance of horizontal initiatives."

3. Conclusion

While the general principles of the Policy on MRRS are being adhered to, management practices are advancing towards full implementation. The department is working to ensure that their information systems, performance measurement strategies, and reporting and governance structures are consistent with their MRRS but further work is needed to ensure full integration. Continued effort by the RCMP along with additional guidance from the Secretariat is required to ensure continued progress towards full implementation with policy requirements.

4. Recommendations and Management Action Plans

Recommendation 1:

As part of their Performance Measurement Frameworks, departments should improve Management, Resources and Results Structures performance measures, in support of planning and decision-making processes.

Management Action Plan:

The RCMP's Strategic Policy and Planning Directorate (SPPD) agrees with the recommendation.

Work began in the spring of 2012 to improve the RCMP's PMF. SPPD attended workshops and training sessions with TBS to ensure the policy and its requirements were followed closely within the RCMP.

SPPD set its goal of clarifying expected results and selecting meaningful performance indicators that were appropriate for respective activities, sub-activities, or sub-sub-activities. Significant assistance was provided to RCMP business lines in drafting a PMF that would form the basis of planning and decision making by program managers.

The draft PMF was submitted on August 17, 2012 and a finalized version, incorporating feedback from TBS, was submitted in November 2012.

The integration of the PMF within the larger RCMP planning process will continue throughout the year. SPPD is developing a business plan to incorporate the Management, Resources and Results Structures (MRRS) within all aspects of planning and resource allocation. This performance management approach is estimated to be fully implemented by April 1, 2014.

Position responsible: Chief Strategic Policy and Planning Officer / OIC, Planning and Strategy Management

Recommendation 2:

Departments should finalize and implement frameworks for monitoring the performance of horizontal initiatives.

Management Action Plan:

The International Police Peacekeeping and Peace Operations Program (IPPPOP) is managed through the Canadian Police Arrangement (CPA) framework which consists of a partnership between the DFAIT, RCMP, CIDA and PS. CPA partners included in the Audit (all except CIDA) have reviewed the Audit Report and concur with its findings.

CPA partners will finalize the Logic Model and Performance Measurement Frameworks that were initiated for the IPPPOP and proceed with the full implementation of these tools in relation to the various missions where Canadian police are deployed. Through their application, CPA partners will assess and measure the outcomes achieved through police engagements abroad in order to monitor the overall performance of the IPPPOP and support planning and decision making related to mission participation.

A fulsome detailed plan will be developed collaboratively by CPA partners with the necessary guidance from TBS with regards to performance measurement of horizontal initiatives. Specifically, CPA partners will make use of the upcoming TBS guidance document planned for future release. CPA partners will work toward a full implementation of the frameworks for monitoring the IPPPOP performance by March, 2014.

Position Responsible: Deputy Commissioner, Federal Policing / Director, International Peace Operations Branch

Appendix A - Audit Objective and Related Criteria

The objective of the audit was to assess compliance with the Policy on Management, Resources, and Results Structures (Policy on MRRS).

1. Strategic Outcomes: With guidance and leadership from deputy heads, departments have established clear and measurable Strategic Outcomes that provide a basis for horizontal linkages.

  • Departmental MRRS have clearly defined and measurable Strategic Outcomes that reflect the department's mandate and vision as well as being linked to the government's priorities. (Policy on MRRS, Section 6.1.1.1)
  • Departmental Strategic Outcomes provide the basis for establishing horizontal linkages between departments. (Policy on MRRS, Section 6.1.1.1)
  • Deputy heads provide overall leadership in developing, in consultation with key stakeholders in the department, the changes required to the Strategic Outcomes and to the Program Activity Architecture (PAA). (Policy on MRRS, Section 6.1.4)
  • Deputy heads approve the departmental MRRS and ensure regular and timely reviews and updates take place to ensure its appropriateness. (Policy on MRRS, Section 6.1.2)

2. Program Activity Architecture: Departments have explained their PAA in sufficient detail to reflect how resources are allocated and managed to achieve the intended results.

  • Related program activities are identified, grouped and linked to the Strategic Outcomes they support. (Policy on MRRS, Section 6.1.1.2)
  • Planned resource allocations, expected results, and performance measures are linked to program activities against which actual results are reported, and are used in monitoring performance and for decision making. (Policy on MRRS, Section 6.1.1.2)
  • The PAA is the basis for resource allocation at all levels of government. (Policy on MRRS, Section 6.1.1.2)
  • The Treasury Board of Canada Secretariat (the Secretariat) approves the PAA level, which allocates and controls resources. (Policy on MRRS, Section 8.1.1).

3. Governance Structure: Departments outline responsibilities and accountabilities for decision-making mechanisms, monitoring and reporting.

  • The departmental MRRS outlines the decision-making mechanisms, responsibilities and accountabilities of the department. (Policy on MRRS, Section 6.1.1.3)
  • Departmental information systems, performance measurement strategies, and reporting and governance structures support the MRRS. (Policy on MRRS, Section 6.1.5).
  • Senior executives are accountable for outputs and outcomes set out in the MRRS. (Policy on MRRS, Section 6.1.6)
  • Departments monitor their compliance with the Policy on MRRS and notify the Secretariat of any changes they intend to make. (Policy on MRRS, Section 6.2.1)
  • Parliamentary reporting and estimates display are based on the departmental PAA. (Policy on MRRS, Section 6.1.1.2).
  • The terms and conditions of a horizontal initiative outline roles and responsibilities related to funding allocations, horizontal collaboration, and adjustments and improvements to the management of the horizontal initiative. (Horizontal Initiatives audit only)
  • Performance monitoring of horizontal initiatives is conducted on a regular and timely basis. (Horizontal Initiatives audit only)
  • Horizontal initiatives include provision for evaluation and adjustment. (Horizontal Initiatives audit only)
  • A framework for horizontal initiatives is used to report to Parliament through the Estimates documents and any other parliamentary reporting documents in the manner and form defined by Treasury Board or its Secretariat. (Horizontal Initiatives audit only)

4. Role of The Secretariat: The Secretariat provides leadership, advice and guidance on the implementation of the Policy on MRRS.

  • The Secretariat works closely with departments to ensure that the spirit and intent of the Policy on MRRS is understood and fully implemented. (Policy on MRRS, Section 7.1)
  • Changes contemplated to the Strategic Outcome and PAA levels are made only with further Treasury Board approval.(Policy on MRRS, Section 6.1.3)
  • The Secretariat provides leadership, advice and guidance in the use and advancement of integrated financial and non-financial program performance information across the government. (Policy on MRRS, Section 8.2).

Footnotes

Note 1

The Policy on MRRS was updated on April 1, 2012. The terminology used in this report refers to the previous policy that was in effect during the planning and examination phases of the audit (prior to April 1, 2012). The term "Program Activity Architecture" (PAA) is used in the report; however the 2012 policy refers to "Program Alignment Architecture." In addition, "program activities" are referred to as "programs" in the 2012 policy.

Return to footnote 1 referrer

Top of page
Date modified: