Evaluation of the RCMP’s Biology Casework Analysis
Evaluation of the RCMP's Biology Casework Analysis
November 20, 2017
Table of contents
- 1. Executive Summary
- 2. Introduction
- 3. Findings
- 4. Conclusions and Recommendations
- 5. Management Response and Action Plan
Acronyms and abbreviations
- Biology Casework Analysis
- Crime Severity Index
- Deoxyribonucleic acid
- Evidence Management
- Evidence Recovery
- Forensic Assessment Centre
- Forensic Science and Identification Services
- Full-Time Equivalent
- International Organization for Standardization
- Laboratory Information Management System
- National Forensic Laboratory Services
- Royal Canadian Mounted Police
- Standard Operating Procedures
1. Executive Summary
The Royal Canadian Mounted Police (RCMP) is the main provider of forensic analysis for many police agencies across Canada. The RCMP has three forensic laboratories located in Vancouver, Edmonton and Ottawa. The Biology Services Program provides biological evidence recovery (ER), DNA analysis, and reporting services in relation to a wide range of criminal code offences. Biology service requests are divided and analyzed under four categories: crimes against property; crimes against persons; sexual assault and homicides.Footnote 1
In 2014, the Government of Canada entered into new cost-sharing Biology Casework Analysis Agreements with each province (excluding Ontario and Quebec) and the three territories. Cost-sharing was phased-in, and is based on the average proportional usage of the service during the previous two-year period.
Included in this report are the results of the evaluation of the RCMP's Biology Casework Analysis (BCA). The evaluation was conducted by the RCMP's National Program Evaluation Services. The evaluation was national in scope, and covered the five-year period from April 1, 2011 to March 31, 2016.
What we Examined:
The evaluation assessed: the demand for Biology Services; the clarity of roles and responsibilities in the provision of services; the quality, timeliness and costs of services; factors impacting the provision of services; and whether any efficiency or economical gains could be made. Multiple lines of evidence were analyzed to determine the efficiency and effectiveness of the Program.
What we Found:
The provision of BCA falls within the mandate of the RCMP. The service is delivered through a clearly established process where roles and responsibilities are well documented. The BCA Agreements between the provinces/territories and the RCMP has contributed to the financial sustainability and predictability of the Program.
Over the five-year period examined, the demand for BCA remained relatively stable for the first four years of the Agreements with a 25% increase in service requests in 2015/16. Legislative changes and crime rates have contributed to the increased demand.
The RCMP targets a 40-day turnaround time for routine BCA requests, and this target was achieved almost half the time (47%) over the reference period. Priority requests, which have a negotiated timeline, were achieved 91% the time. Although the lab continued to utilize alternative mechanisms such as overtime and private sector contracting to address the 2015/16 increase in demand, the achievement of combined routine and priority requests fell from 50% in 2014/15, to 43% in 2015/16. While the majority of clients are satisfied with the timeliness of Biology Services, it was unclear as to how the 40-day target was determined given the varying nature and complexity of requests.
There were several internal and external factors that impacted the Program's performance. With regard to internal factors, RCMPs labs policies, processes, procedures and governance structures supported program delivery. The evaluation also found performance information is available to assist management decision-making. With regard to external factors, the evaluation noted increased demands, quality of service request submissions, and a need for increased client awareness and education as impacting program performance.
What was Recommended:
Based on the findings of the evaluation, two recommendations were made to improve the effectiveness and efficiency of biology casework analysis:
Recommendation 1: That National Forensic Laboratory Services (NFLS), in collaboration with clients, establish a service target that considers a balance between timeliness and cost of service to establish achievable diary dates.
Recommendation 2: That NFLS assess existing communication materials targeted to the needs of police services and prosecutors, to determine if updates are required.
2.1 Purpose and Scope of the Evaluation
This report presents the results of the evaluation of the RCMP Biology Casework Analysis. As per the Agreements Respecting Biology Casework Analysis between the Government of Canada and signatory provinces and territories (which does not include Ontario and Quebec), the RCMP was to conduct an evaluation the RCMP's Biology Casework Analysis within the National Forensic Laboratory Services (NFLS) by April 1, 2017.Footnote 2
For the purpose of this evaluation, only the RCMP's Biology Services as provided by the three laboratory sites were assessed. Other forensic services offered, as well as the National DNA Data Bank, were not included in the scope of the evaluation except to situate how Biology Services fits within the continuum of forensic services provided to a client.
The evaluation was conducted by the RCMP's National Program Evaluation Services. The evaluation was national in scope, and covered the five-year period from April 1, 2011 to March 31, 2016. The objective of the evaluation was to assess the demand for Biology Services; the clarity of roles and responsibilities in the provision of services; the quality, timeliness and costs of services; factors impacting the provision of services; and whether any efficiency or economical gains could be made.
Since 1990, the RCMP's forensic laboratory services, including biology casework, have been subject to various assessments to determine compliance and performance. For example, in 2007, the Office of the Auditor General conducted an Audit of the Management of Forensic Laboratory Services,Footnote 3 and in 2011, completed an Audit of National Police ServicesFootnote 4. In 2014, Public Safety Canada conducted an evaluation of its role in the Biology Casework Analysis ActivitiesFootnote 5. This evaluation did not assess the RCMP Laboratory or their related activities.
2.2 Program Description
Overview of Biology Casework Analysis
Forensic science refers to the examination of scenes of crime, recovery of evidence, laboratory examinations, interpretation of findings, and presentation of the conclusions reached for intelligence purposes or for use in court. Forensic science activities range from instrumental analysis with unequivocal results, such as blood alcohol determination, to the investigation of suspicious fires and vehicle accidents, to comparison work such as handwriting and tool mark examination, which is largely subjective in nature.Footnote 6
The RCMP is the main provider of forensic analysis for many police agencies across Canada. The RCMP's Biology Service is organized to operate as one laboratory with three sites located in Vancouver, Edmonton and Ottawa. The RCMP Forensic Assessment Centre (FAC) is the first point of contact for investigators and clients requiring RCMP Laboratory analysis, who are then directed to the appropriate unit for completion of the necessary analysis. Services can range from: detecting drugs and alcohol in bodily fluid and tissue; analysing firearms to determine the type of weapon; tool mark identification; bloodstain pattern analysis; the analysis of paint, fire debris, clothing and footwear; analysis of counterfeit and questionable documents; and, for the purposes of this evaluation, Deoxyribonucleic acid (DNA) analysis. DNA analysis includes the analysis of biological trace evidence recovered at crime scenes such as bodily fluids (i.e. blood, semen and saliva), hair, and cellular material from handled objects. The RCMP also uses the services of a private laboratory when demand for service exceeds the capacity of the RCMP's three laboratories.
The Biology Services Program reports organizationally to the National Forensic Laboratory Services (NFLS) under Forensic Science and Identification Services (FS&IS). Within FS&IS, the Quality Branch provides quality assurance for all forensic services. Within NFLS there are Policy and Program Support positions (including Research and Development) which provide support to all forensic services, including the Biology Services Program, which has Policy and Program Support personnel at the three lab sites.
Biology Services provides biological evidence recovery (ER), DNA analysis, and reporting services in relation to a wide range of criminal code offences. Figure 1 below demonstrates the process undertaken for Biological Casework Analysis.
Figure 1: Procedure for Biological Casework Analysis
Biology service requests received from investigators (also referred to as "clients" throughout this report) are divided and analyzed under four categories:Footnote 7
- Crimes against property – Involve unlawful acts to gain property, but do not involve the use or threat of violence against the person. They include offences such as break and enter, theft and fraud
- Crimes against persons – Involve the use or threatened use of violence against a person, however, for BCA it does not include homicides and sexual assault cases which are their own categories
- Sexual assault – defined as an assault of a sexual nature that violates the sexual integrity of the victimFootnote 8
- Homicides – A homicide occurs when a person directly or indirectly, by any means, causes the death of a human being.
Once the RCMP's forensic scientists determine the answers to the investigator's forensic question(s), the results are provided in a forensic report to the investigator. The current goal is to provide this report within a 40-day target 85% of the time. Forensic reports are used for securing a warrant, to further the investigation, or as evidence in court. If applicable, DNA results are also submitted to the National DNA Data Bank, which includes the Convicted Offenders Index (DNA profiles collected from convicted offenders) and the Crime Scene Index (DNA profiles obtained from crime scene investigations).
Cost of Biology Casework Analysis
In 2014, the Government of Canada entered into new cost-sharing Biology Casework Analysis Agreements with each province (excluding Ontario and Quebec) and the three territories. A key focus of the 2014 agreements was to implement a sustainable funding structure for the provision of the RCMP's Biology Services. The cost-sharing model used the average total cost from 2011/12 and 2012/13 ($27.7 million) as the baseline amount to be recovered from signatories. Cost-sharing was phased in, with the provinces and territories billed 28% of the $27.7M baseline costs in 2014/15, 40% in 2015/16 and 54% in 2016/17Footnote 9 based on their average proportional usage of the service in 2011/12 and 2012/13.
In accordance with the agreements, beginning April 1, 2017, the baseline for recovery is to be re-calculated every two years, using actual costs from the previous two years. In the subsequent two-years, the provinces and territories will be billed 54% of the revised baseline cost, based on their proportional usage during the previous two-year period.
The following table provides the actual total expenditures from 2012/13 to 2015/16.
|Activities Directly Supporting Biology||$7,313,396||$7,826,249||$6,560,698||$6,072,900|
|National Quality Management||$522,474||$570,378||$548,739||$492,070|
|FS&IS Corporate Support||$1,112,342||$915,542||$856,128||$710,687|
* Financial figures are provided annually to Public Safety Canada to support FPT discussions concerning Biology Casework Analysis Agreements. Figures listed in the table were approved annually by the Deputy Commissioner of Specialized Policing Services and the Chief Financial and Administrative Officer
**Actual expenditures does not include 2011/12 as 2012/13 was the first year of the new funding structure of the BCA Agreements with the Provinces and Territories.
Variations in cost can largely be attributed to staffing levels. In 2012/13, there were 128 full time equivalents (FTE) working in Biology Services. This dropped to approximately 118 FTEs in 2015/16. Additionally, three labs were closed during this time period. Overtime and private sector contracting, both of which are used to address any shortfalls in capacity, have also impacted the total costs.
2.3 Methodology and approach
A logic model was developed for the purpose of this evaluation, with input from the program. It provides an overview of the logical linkages between inputs, activities, outputs and various levels of outcome.
Evaluation Approach and Design
A theory-based approach was used for this evaluation. Program performance was measured against intended results articulated in the program logic model. Extensive use of triangulation was undertaken as an analytical method, where multiple lines of evidence helped corroborate findings.
The evaluation questions covered relevance and performance, including effectiveness, efficiency and economy. The data collection, research and report was conducted in accordance with the requirements of the Treasury Board's Policy on Results (2016). Qualitative and quantitative information was used to determine findings and recommendations for improvement, and to help inform senior management decision-making.
The following lines of evidence were used to assess the relevance, effectiveness, and efficiency of the BCA:
Document and Literature Review. Ninety-three written sources of information were consulted such as foundational documents, operational reports, meeting minutes, records of decisions, communication products, and previous assessments.
Review of Performance and Financial Data. Data relating to BCA housed in the RCMP financial management information system (TEAM); human resource information system (HRMIS); and Laboratory Information Management System (LIMS) was extracted and analyzed.
Interviews and On-site Visits. Evaluators visited the RCMP's three lab sites (Vancouver, Edmonton and Ottawa) to conduct interviews and focus groups, as well as to better understand the workflow.
A total of 52 interviews were conducted. There were seven categories of interviewees tailored to their specific area of expertise. These included: 17 biology lab staff; 16 clients of lab services (i.e. investigators and crown attorney); eight NFLS Support; eight internal RCMP stakeholders (i.e. have knowledge of the Program); and three Management positions. Lab staff and management were selected based on their level of experience and knowledge of biology casework analysis. Client interviewees included individuals who made the most use of RCMP Biology Services; within the client category there were five RCMP detachments and two police forces other than RCMP.
Focus Groups. Focus groups involving a total of 18 participants were held at the three lab sites. Participants included representatives from evidence recovery, analytical, and reporting.
Some challenges and limitations were encountered during the conduct of the evaluation, however these were mitigated as outlined below. The evaluation's assessment of demand for service, cost of service, and workflow timeliness was largely based on data recorded in LIMS. Some anomalies were noted in this data, including inconsistent or missing dates (e.g. processing date in advance of receipt date, receipt date in advance of date assigned to a lab, no date recorded for completion of intermediary steps). Records containing these types of errors were eliminated from the analysis. This approach was taken in order to demonstrate reliability and validity of the findings, and to ensure that conclusions and recommendations were based on objective and documented evidence
3.1 Demand for Service
Finding: Demand for biology casework analysis has increased since its introduction in 1989. In addition to its highly conclusive results, legislative changes, crime rates and court expectations have all contributed to the increased demand.
DNA evidence was first permitted in a Canadian courtroom in 1989.Footnote 10 Since then, the demand for DNA profiling has increased as a result of its highly conclusive results. DNA is the same in every cell of the body (blood, hair, teeth, tissue, semen, saliva, bone, epithelial cells), and does not change over time.Footnote 11 With extremely low odds of two individuals producing the same profile, DNA results have become fundamental for investigations and court proceedings. In 2006/2007, the RCMP received 3,401 biology service requests, and by 2015/16 this had increased to 8,102.Footnote 12
This evaluation specifically analyzed service demand trends from 2011/12 to 2015/16. From 2011/12 to 2015/16, the RCMP processed a total of 34,060 service requests (Table 2). As noted earlier, service requests are categorized by four offense types: crimes against property; sexual assaults; crimes against persons; and homicides. For the first four years covered by this evaluation, demand for services remained relatively stable, however, in 2015/16 the lab experienced a 25% increase in service requests over the previous fiscal year. While increases were observed in each of the four service streams, the greatest increase was in crimes against property where a 32% increase over 2014/15 levels was observed. In each of the assessed years, approximately 2% of the requests were documented as priority requests (priority requests are processed first and often ahead of the 40-day target). Over the examined five-year period, 792 requests were documented as priority.
|FY 2011/12||FY 2012/13||FY 2013/14||FY 2014/15||FY 2015/16||Total|
|Crimes Against Persons||1382||1362||1321||1246||1578||6889|
|Crimes Against Property||2730||2878||2750||2995||3950||15303|
* Note: Data extracted August 18, 2016, and based on the date the service request was received by the Forensic Assessment Centre.
The majority of requests were received from British Columbia (34%) and Alberta (31%) over the five-year period (Table 3). While 10 of the 11 signatory provinces/territories had increased demand in 2015/16 when compared to 2014/15, the greatest growth in demand (43%) occurred in Alberta.
|Province||FY 2011/12||FY 2012/13||FY 2013/14||FY 2014/15||FY 2015/16||Total|
|Newfoundland and Labrador||186||207||208||242||316||1,159|
|Prince Edward Island||22||31||16||20||31||120|
*Note: Numbers are based on the date the service request was received by the Forensic Assessment Centre.
Rationale for Increased Demand
While the extent of the increase in 2015/16 was unexpected, 19 intervieweesFootnote 13 stated the increase was a result of increase in crime ratesFootnote 14 , the belief that DNA evidence has become an "expectation" for the courtsFootnote 15 , and the popularization of forensic sciences in the mainstream media (referred to by some as the "CSI effect").Footnote 16
When clients were asked how they decide when to engage the services of the RCMP's forensic lab, they (12/12) described the process taken at a crime scene. If the investigation falls under a designated offence according to section 487.04 of the Canadian Criminal Code,Footnote 17 which allows for DNA analysis, it is likely that investigators will collect DNA evidence. Investigators may first consider the quality of the exhibitFootnote 18 and the likelihood that it will move the investigation forward prior to sending exhibits to the RCMP lab for analysis.
Clients were also asked how frequently and for what reasons they would use a private commercial lab for profiling services, rather than the RCMP. Interviewees (9/12) indicated that at times they use a private lab because it offers a service not offered by the RCMP lab (for example, mitochondrial DNA testing), or because they felt the private lab could process the exhibits faster in urgent situations.Footnote 19
Legislative Changes that have affected demand
Changes to legislation have had an effect on the demand for Biology Services. In 1995, Parliament enacted amendments to the Criminal Code under which a provincial court judge could issue a warrant authorizing a police officer to obtain a biological sampleFootnote 20 (hair, blood or saliva) from a suspect for the purposes of forensic DNA analysis for Primary offences under the criminal code.Footnote 21 The introduction of the 1998 DNA Identification Act established a national DNA data bank to help law enforcement agencies identify persons alleged to have committed designated offences.Footnote 22 In 2000, Parliament enacted a third piece of legislation relating to the collection and use of DNA forensic evidence.Footnote 23 In 2008, through the introduction of new legislation, many criminal code offences were elevated from Secondary to Primary offences and many non-designated offences were elevated to Secondary offences. This increase in the number of secondary offences, predominantly categorized as crimes against property, led to an increase of submissions to the RCMP's Biology Services. As demonstrated earlier (Table 2), crimes against property represented 49% of service requests submitted in 2015/16. Prior to this amendment, exhibits for secondary designated offences were accepted under a quota system attributed to each province/territory for forensic DNA analysis until September 2009 when the quota on secondary designated offences was eliminated.Footnote 24
Demand for Service is linked to Crime Rates
The overall frequency and severity of crime in Canada declined steadily between 1998 and 2014Footnote 25. However, a Statistics Canada report titled "Police-Reported Crime Statistics in Canada, 2015" noted that both the Crime Rate and the Crime Severity Index (CSI) increased in 2015 for the first time since 2003.Footnote 26 The report concluded that the change in the CSI in 2015 was driven primarily by increases in fraud, breaking and entering, robbery, and homicideFootnote 27 – all offences where the RCMP's biology casework analysis services can contribute to investigative work. The Statistics Canada report further noted that "the upward movement of the national CSI was fueled by a notable growth in crime reported by Alberta."Footnote 28 This result aligns with the provincial/territorial trend in biology service requests, where the demand from Alberta increased by 43% in 2015/16 over the previous year's level (Table 3).
3.2 Alignment with Mandate, Role and Responsibilities
Finding: The provision of biology casework analysis is within the mandate of the RCMP. The service is delivered through a clearly established process where roles and responsibilities are well documented.
The mandate of the RCMP's Biology Services is to "support police investigations through the provision of forensic identification and analysis of exhibit materials used as evidence."Footnote 29 The RCMP lab, therefore, provides an essential service to police service clients. The scope and terms of service are outlined within the Biology Casework Analysis Agreements between the Government of Canada and signatory provinces and territories.
In addition to providing Biology Services, interviewees (8/13) believed it was also the RCMP's role to be an innovator in the areas of forensic analysis techniques and technologies, and to identify and test new processes and technologies. This role is confirmed in the RCMP's National Police Services Charter, which notes that the RCMP will strive to provide a "dynamic" national police service that "responds to the constantly changing nature of law enforcement."Footnote 30 Furthermore, one of the Charter's five listed fundamental principles is to be "innovative" in operations and administration where National Police Services "quickly adopts new technologies, processes and approaches to ensure law enforcement and criminal justice officials have the leading-edge tools and techniques required to effectively investigate and prosecute criminal activity."Footnote 31
The evaluation noted that the roles and responsibilities of clients, NFLS Headquarters, and the three lab sites are well documented. The RCMP's biology service is organized to operate as one lab with three sites, with each site operating under the same Standard Operating Procedures (SOPs) and with rigorous quality assurance processes in place.
3.3 Achievement of Expected Outcomes
Finding: An increasing percentage of exhibits received by the lab contain sufficient DNA for processing.
An expected immediate outcome of Biology Casework Analysis is that, "Sample test results allow scientists to compare and interpret DNA information to determine forensic associations (i.e. suspect to victim, suspect to scene, scene to scene, etc.). (Appendix A)"
In order for this to occur, the exhibit received must contain a sufficient amount of DNA to interpret DNA results to form forensic linkages. Between 2011/12 and 2015/16, 67% of the overall service requests processed had DNA present that could be analyzed (Table 4). The percentage of requests with sufficient DNA for analysis increased over the five-year period examined. In 2011, and again in 2015, improvements in both processing and instrumentation enabled the Biology program to generate a DNA profile from lower quantities of recovered DNA.Footnote 32 As a result, following these improvements, some cases which would have previously been reported as having insufficient DNA for analysis would no longer fall in that category. While this increase contributes to the achievement of the immediate outcome, the collection of sufficient DNA is outside the control of NFLS.
|By Offense Stream||FY 2011/12||FY 2012/13||FY 2013/14||FY 2014/15||FY 2015/16||5-year average|
|Crimes Against Property||69%||65%||68%||76%||78%||72%|
|Crimes Against Persons||68%||68%||68%||71%||77%||70%|
|Average all samples||64%||65%||65%||68%||73%||67%|
*An exhibit submitted to the RCMP Lab may result in multiple samples for analysis.
Finding: While almost all (91%) priority requests were completed within the targeted timeframe, less than half of routine requests were completed within the 40-day target.
An intermediate outcome of Biology Casework Analysis is that "Forensic scientists provide answers to the forensic association question(s) in a forensic report that is submitted to the investigator (current goal is to provide this report within a 40-day target 85% of the time) (Appendix A)." The 40-day target applies to all routine requests while the priority requests have a negotiated timeline.
To assess the results of this outcome, LIMS data was analyzed to determine the percentage of both priority and routine requests that were completed within this targeted timeframe (Table 5).
|FY 2011/12||FY 2012/13||FY 2013/14||FY 2014/15||FY 2015/16||Over 5 year|
|Crimes Against Persons|
|Crimes Against Property|
*Routine requests have a target date of 40-days while priority requests have a negotiated timeline which is identified in the Laboratory Information Management System.
Timeliness of Service
During the reference period, the target to complete routine services requests was 40 days, 85% of the time. The LIMS system allows the targeted turnaround to be established for each individual service request. Over the five-year period covered by this evaluation, 95% of the service requests had a targeted turnaround time of 40-days.Footnote 33
When the four offence streams were combined, the achievement of this target for routine requests varied from a high of 55% in 2011/12, to a low of 43% in 2015/16 (Table 5). On average, the target was achieved 47% of the time for routine requests over the five-year period. Homicide service requests tagged as routine were the least likely to be completed within 40-days, with the target being achieved 28% of the time. However, homicide priority requests were highly likely to be completed within the negotiated timeline.
Mechanisms to increase Timeliness and Capacity
Overtime and private sector contracting are two mechanisms used by the lab to ensure requests are completed within the 40-day target. To adjust for the 25% increase in demand (2015/16), overtime was increased by 40% over 2014/15 levels. In addition to overtime, 714 service requests were sent to a private sector lab for analysis in 2014/15, and 1356 were outsourced in 2015/16 – an increase of 90%. Despite these strategies, achievement of the 40-day target fell from 50% of the time in 2014/15, to 43% of the time in 2015/16.
Challenges with a 40-day Service Target
The timeliness of the justice system as a whole is of importance to Canadians. The Standing Senate Committee on Legal and Constitutional Affairs is currently examining the key causes of delays and how these can be addressedFootnote 34. In 2016, the Supreme Court of Canada set a deadline of 30 months for criminal trials to be completed; provincial court trials are to be concluded within 18 months.Footnote 35 Ensuring a timely system that supports an accused's Charter right to be tried within a reasonable time may also impact the RCMP's lab as it works to support investigators.
The evaluation was unable to determine how the 40-day target was determined as a result of lack of evidence. This target is not a universally expected standard for Biology Services across Canada. For example, the Ontario laboratory which processes requests for police forces located in Ontario indicated that their "goal is to report most cases within 60 days. Some offences/incidents have completion targets of 30 or 90 days." Footnote 36
The nature and complexity of biology service requests varies. A service request may include the examination of multiple exhibits, and each of those exhibits may yield multiple samples. While the number of exhibits is known when the request is submitted, the number of viable samples is only determined during evidence recovery. Further complicating the determination of level of effort required, one sample may generate the necessary results and the other samples developed will no longer need to be analyzed. LIMS data was accessed to determine the typical number of samples analyzed for a service request in each of the offense streams (Table 6). For example, on average, each homicide service request resulted in the analysis of ten samples while crimes against property service requests typically only yielded one or two samples to be analyzed.
|Offense Stream||FY 2011/12||FY 2012/13||FY 2013/14||FY 2014/15||FY 2015/16||Average|
|Crimes Against Persons||3.8||3.6||3.3||4.6||3.6||3.8|
|Crimes Against Property||1.6||1.7||1.7||1.7||1.5||1.6|
The client Quality Assessment Questionnaires found that, while there has been a slight decline since 2012/13, the majority of clients are satisfied with the timeliness of Biology Services (Table 7). Results from the 2014/15 client questionnaires indicate that, while the majority of respondents indicated satisfaction with the timeliness, they would prefer that a more realistic turnaround time be communicated when exhibits are submitted." Footnote 37
Ten police officers representing clients were questioned regarding the importance of the 40-day service target in support of frontline policing. They outlined two general scenarios where DNA evidence is used in an investigation and how timing can impact these scenarios. In the first scenario, where the suspect is known, a DNA turnaround time of 40 days may not be significant. For example, the court date is the critical date for results to be available. A court date may take over a year to schedule and is highly unlikely to be within 40 days of laying a charge. In the second scenario, where the suspect is unknown, the timeliness of a DNA analysis would be more significant. Depending on other evidence left at the scene, the investigator may be completely reliant on DNA results to give the investigation direction including the identification of a suspect. Similarly, DNA results can also support obtaining authorization to initiate surveillance of suspects.
Given that service requests relating to certain offenses require more effort than others, and in general clients are satisfied with the current timeliness of the services, it is recommended the appropriateness of a 40-day performance target for all offence streams be reviewed.
Finding: New technologies and processes have been tested and adopted by NFLS, however the evaluation did not assess the improvements in efficiency and/or effectiveness.
NFLS identified three key outcomes (Appendix A) related to their core research activities:
- New technology options are validated and then implemented if testing results demonstrated effectiveness and/or efficiency gains
- New technology and/or techniques and procedures improve analysis and interpretation
- New technology and/or techniques and procedures improve processing times and/or quality/sensitivity
Since 2011/12, the RCMP's research team has implemented nine new technologies, techniques or procedures intended to improve either the efficiency or the effectiveness of biology casework analysis. Five changes to instruments or procedures were made as a result of the manufacturer no longer supporting previous ones, however these replacements did not result in an increase in efficiency or effectiveness. Six additional concepts were tested but not implemented.
Prior to implementing new technology and/or techniques and procedures, the RCMP's research team conduct rigorous academic journal reviews followed by validation exercises in the RCMP's lab. When results are positive, the team will then pilot the new technology or procedure prior to full adoption. For example, in 2014/15, a new kit for DNA analysis was introduced which can provide results that are more discerning, allowing for the analysis of more challenging samples.
In addition to validating and implementing new technology, the NFLS research and development team is also engaged in the broader forensic science research community. For example, RCMP scientists peer reviewed and published sixteen papers during the five-year reference period, as well as wrote twenty-six papers to support the improvement of internal systems, for court purposes, or to inform the development of standard operating procedures. In addition, each year research scientists participate in conferences and general meetings, including providing presentations.Footnote 38 NFLS continues to be aware of new technologies, and improvements of analytical processes.
3.4 Internal Factors Impacting Performance
Finding: While the governance structure and regulated assessments have attributed to quality, accredited lab results, HR issues – particularly vacancies and understaffing – have affected the performance.
The evaluation considered four key internal factors that support the delivery of biology casework analysis. These include: the existence and effectiveness of policies and processes; the clarity of the governance structure; the existence of a tracking system to monitor timeliness and quality of service; and the existence of a human resource strategy that ensures adequate capacity, training, timely hiring and staff retention.
Polices and Processes
The RCMP's lab operations are highly regulated with a substantial amount of oversight in order for the lab to maintain its International Organization for Standardization (ISO) accreditation.Footnote 39 The Standards Council of Canada is the accreditation body that annually assesses whether the RCMP is meeting the ISO standards. The accreditation assessment includes an on-site review of:Footnote 40
- Control of records
- Competency of Personnel
- Accommodation and environmental conditions (including access control to the lab and secure evidence storage)
- Documentation regarding test and calibration methods and method validation
- Maintenance and calibration of equipment
- Maintenance of chain of custody records
- Monitoring of the quality of tests and results
- Reporting of Results
To support the accreditation process, the RCMP operates under a Laboratory Policies and Procedures Manual. Biology Services, also has a Discipline Policy and Procedures Manual with SOPs that are developed for each technique and type of DNA analysis. To ensure compliance, the following internal oversight functions take place:
- Annual proficiency tests of lab staff
- Annual internal audits using checklists to determine the conformance of laboratory quality operations to the elements of the quality system as described by FS&IS and required by the ISO quality standards
- Monthly Incident Reports
- Daily tracking of investigations/evaluations of incidents
The FS&IS Quality Assurance branch provides NFLS and FS&IS management with an annual report that summarizes audit findings from each lab site for each discipline. The report includes input from the Operational and Policy Managers of NFLS and from the National DNA Data Bank.Footnote 41 The report includes recommendations for the upcoming year as well as data collected by the Quality Assurance Branch throughout the reporting period. Footnote 42
A review of these oversight mechanisms found that the RCMP's polices, process and procedures are effectively supporting service delivery. For example, since 2011, lab staff have undergone 777 proficiency tests with a 99.5% success rate.Footnote 43 Also, despite an increase in volume, from 2013 to 2015 there was a decline in corrective actions identified (Table 8).Footnote 44
With regard to provincial influence on the governance of the RCMP lab, partners play a role through their participation on technical and advisory bodies.Footnote 45 Overall, the Commissioner of the RCMP is "accountable to the Federal Minister of Public Safety Canada for the direction, management and operation of the National Police Services. The Commissioner receives advice and recommendations from the National Police Services National Advisory Committee."Footnote 46 Other consultative bodies, such as the DNA Working Group, provide an outlet for discussion concerning challenges and opportunities.
When focus groups were completed with representatives from each of the three lab sites, participants were asked if and how the three sites work together as one lab.Footnote 47 Participants indicated that although the RCMP model is one lab with three sites, collaboration between sites at the working level is minimal. The majority (5/9) answered "no" when asked if the labs are uniformly managed.Footnote 48 Some (4/9) qualified their response by stating that while the science is uniform, there are differences at the working level and with the application of policy (i.e. SOPs).
Overall, it was noted that the RCMP lab encourages a culture of identifying and implementing efficiency gains. During the five-year reference period of the evaluation, the lab was piloting a new initiative to promote further standardization between the three lab sites and to improve workflow and efficiency through the identification of best practices.Footnote 49 A team lead from each of the three sites exchanged roles for a week to observe any differences in local practices. Senior management was eager to identify and implement any resultant observations.
Tracking and Monitoring
Exhibit movements are closely tracked to ensure the Chain of Custody for all exhibits is properly maintained and documented. Service requests are tracked to provide detailed performance information, and allows NFLS to determine costs. Service request information and exhibit transfer details are primarily housed in LIMS.Footnote 50 Each service request is provided a LIMS identification number which is used to track the case and all associated exhibits.Footnote 51
Using LIMS information, a weekly dashboard is produced for management containing information on backlogs, turnaround times, service load, and the capacity of each section.Footnote 52 This allows management to monitor the service load and to re-direct service requests between lab sites when backlogs are detected. Half of HQ interviewees, including management, felt they had the information necessary for decision-making. One exception raised by three managers was the absence of data relating to staff availability (e.g. leave, training, etc.) preventing management from effectively forecasting lab capacity and resource requirements.
There are two types of reports that are shared with investigators and provinces/territories. Investigators are provided with results on individual service requests, and provinces/territories with quarterly or annual reports on performance and costs. Overall, investigators interviewed found the reports to be readable and understandable. The second type of reports (i.e. quarterly and annual reports) are provided to Provincial and Territorial clients in accordance with article 8.1 of the Biology Casework Analysis Agreements. These reports include:
- Number of Service Requests Received (Priority Service Requests and Routine Service Requests)
- Number of Service Requests Completed (Priority Service Requests and Routine Service Requests)
- Average turnaround time for (priority service requests and routine service requests)
- % meeting routine service standards
- % priority service requests completed within the negotiated "diary date"
- # of FTE supporting biology casework analysis
- Number of service requests by contributing agency
The evaluation found evidence that program tracking and reporting is occurring. Collecting additional HR data could further assist decision-making with regard to forecasting capacity and resource requirements.
Human Resource Strategy
As demonstrated in Table 9, FTEs declined in each of the reviewed years. At the end of fiscal year 2015/16 there were approximately 118 full time RCMP employees working on biology casework analysis.Footnote 53 The closures of the Halifax and Regina laboratories reduced the number of reporting scientists, most of which were experienced. This impacted the other lab sites by an increase in workload around the same time that demand began to increase (FY 2015/16).
|Laboratory Site||FY 2011/12||FY 2012/13||FY 2013/14||FY 2014/15||FY 2015/16|
* The Winnipeg lab site was also closed during the reference period, however, there were no FTEs for Biology Services, only administrative support staff. Numbers are likely lower than depicted as the table does not take into account parental and sick leave.
A 2016 Managerial Review Report prepared by FS&IS' Quality Assurance Branch highlights that an increase in casework combined with lab closures and declining HR capacity was a contributing factor for sites to stay on top of exhibit/DNA returns.Footnote 54
To address staffing challenges, management focused on hiring practicesFootnote 55 and developed a pro-active hiring model where a qualified candidate pool is continuously replenished.Footnote 56 When hired, new staff participated in an extensive, RCMP-lab developed training program ranging from 6-12 months for technologists within evidence recovery, to over a year for reporting scientists.Footnote 57 Thirty-five biology staff members were hired during the period under review. While this was necessary to meet future demands, it also required existing staff to dedicate time to providing training rather than casework analysis.
While FTEs declined during the reference period, the overall retention rates were positive. Analysis indicates that biology casework staff, on average, had a 90% retention rate in Edmonton, an 85% retention rate in Ottawa, and an 81% retention rate in Vancouver.Footnote 58 On average, biology casework staff stayed in their positions for seven years.
In 2015/16 the lab completed a Capacity Assessment to develop a proposal for hiring additional biology staff.Footnote 59 The assessment relied on data from FY 2014/15, and took into account several factors such as:
- How much work was each employee able to produce in a week/year?
- How often were staff available to produce work? For example, how often were they away at court? How much sick and vacation leave was used?
- Based on the advice of a demographer, how many parental leaves, extended leaves and retirements could be expected over the next three years?
The assessment found that ten additional staff (seven reporting scientists and three technicians) should be hired resulting in a total of 42 reporting scientists and 38 technologists in the evidence recovery unit.Footnote 60 There was however concern expressed with the training time that would be required for the ten new employees anticipated for 2016/17 and 2017/18. With already strained resources now dedicating time to teaching new employees, as well as overtime hours and private sector outsourcing at all-time highs, it will likely remain a challenge for the lab to meet the 40-day target over the next two years, especially if demand remains consistent with 2015/16 levels.
3.5 External Factors Impacting Performance
Finding: External factors have impacted the RCMP's ability to meet its objective of providing forensic evidence in support of investigations within 40-days.
A key area of risk for the RCMP is the inability to predict demand, and to proactively adjust funding and capacity accordingly. Beginning in 2017, and in accordance with the Agreements Respecting Biology Casework Analysis, payments from the provinces and territories will be adjusted every two years based on the previous two years' actual costs. While this methodology, which is based on prior years' volumes and costs, allows the provinces and territories to budget a fixed amount for these services, it results in the RCMP having to fund and adjust for any immediate fluctuations in demand. This challenge was highlighted in 2015/16, when the lab, which was designed to support 6,500 service requests on an annual basis, processed 8,100 requests. This was a 25% increase. This surge was handled through increased levels of overtime and outsourcing to private sector labs.
An additional demand on lab personnel is the requirement for disclosure and/or testifying in court. During the 2015 calendar year, biology staff spent 1,905.5 hours or 254 days providing court testimony (including court preparation and travel time).Footnote 61 To reduce the time involved in meeting this demand, testimonies by videoconference are being encouraged where possible.
Quality of Service Request Submissions
Although the percentage of requests with sufficient DNA analysis increased over the five-year period examined, Lab staff indicated that improper swabbing, packaging and/or labelling by investigators requires a higher level of effort from FAC and lab staff to resolve thus impacting timelines.
Need for increased Client Awareness and Education
While certain crime scenes are attended by Forensic Investigation Services to oversee the collection and handling of exhibits, this is not always the case depending on the nature of the crime and its location. Accordingly, the lab has developed guides, instructional pamphlets, and booklets to assist investigators. Some of these resources are available electronically while others are only available in hard copy at the lab sites. While resources are available, they may not have been widely communicated to those collecting forensic evidence. For example, interviewees provided a number of suggestions for improving education and communication between the lab and clients. These suggestions included: periodic or annual conferences involving both clients and scientific personnel; providing additional materials on the types of exhibits which are most likely to provide a viable sample; and providing an online portal for submitting and tracking service requests. It is recommended that NFLS assess existing communication materials to determine the need for updating their current materials.
Although clients expressed a desire for an improved communication strategy with an education focus, police service clients (9/13) responded that the quality of communication between the lab and clients was good overall.Footnote 62 Good communication with the lab was generally seen as an important part of the process.
Finding: An analysis of overall costs compared to level of demand demonstrates that the average cost per service request was lowest in 2015 / 16. This was also the year when the lab was least able to meet the 40-day target.
In 2014/15, Canada entered into new ten-year Biology Casework Analysis Agreements with provinces and territories for forensic DNA analysis services. The agreements fulfilled two key guiding principles of the RCMP's National Police Services' Charter,Footnote 63 specifically, they demonstrated a shared responsibility in the provision of services and financial sustainability.Footnote 64
Prior to the new contracts being established, Biology Services had "experienced significant increases in cost and demand, without a corresponding increase in provincial and territorial payments."Footnote 65 As articulated in the Agreements, beginning in 2016/17 "the total annual provincial and territorial payment will be 54% of the average of the actual cost incurred by Canada during the previous two fiscal years".Footnote 66 This amount is to be charged to the signatory provinces and territories, based on their proportional usage of the services over the previous two years.
The total cost (Table 10) includes expenditures for the following.Footnote 67
Biology Program Direct Costs: Direct costs incurred in the provision of Biology Services include Evidence Recovery, Biology Analysis, Biology Reporting, and Biology Policy and Program Support.
Activities Directly Supporting Biology: Laboratory activities directly supporting the biology program are Evidence Management, Forensic Assessment Centre (FAC), Regional Lab Administration Unit, Scientific Services, Laboratory Information Management System (LIMS), and the National Forensic Laboratories Services office. These activities support the seven main lab services, and costs are allocated among them based on full-time equivalent (FTE) ratios. Similarly, the biology program is allocated a portion of the costs for National Quality Management, IT, and FS&IS Corporate Support based on their proportion of FTEs.
Additional Indirect Costs: Internal services include functions such as Strategic Planning and Priorities Directorate, planning, professional integrity, departmental security, communications and legal. It also includes resource management services, such as HR, financial management, information management, information technology, travel and other administrative services, and Asset Management services such as Real Property, materiel and acquisitions. The Employee Benefit Plan is also included in this cost.
Private Sector Costs: when demand exceeds the lab's HR capacity, service requests are outsourced. These costs, included in the overall lab expenditures, have varied from approximately $1.1M in 2011/12, to $2.7M in 2015/16 as shown in Table 11.
|FY 2011/12||FY 2012/13||FY 2013/14||FY 2014/15||FY 2015/16||Total|
|Number of Service Requests||605||688||1,088||714||1,356||4,451|
Six Management and NFLS support interviewees were asked if the current costing-sharing model was appropriate and sustainable. All indicated that it supports financial sustainability and improves predictability. In addition, the decision to split the costs based on prior years' usage rates, as per the BCA Agreements, allows provinces and territories to plan their budget accordingly. Both the RCMP and provinces/territories worked at containing costs by limiting demand to those exhibits most likely to answer the forensic questions, and for the RCMP by containing the costs associated with the biology casework analysis. When annual biology casework cost was divided by the number of samples received each year, the cost per sample was lowest in 2015/16 at approximately $847.67 a sample (Table 12).
|FY 2011/12||FY 2012/13||FY 2013/14||FY 2014/15||FY 2015/16|
|Cost per sample||$1,031.59||$1,054.12||$1,130.16||$940.25||$847.67|
*Includes cost of samples processed by the RCMP Labs as well as the Private Lab
Homicides service requests are typically more expensive than other types of requests (Table 13) since homicides often require the analysis of more samples as demonstrated in Table 6.
|Offense Stream||FY 2011/12||FY 2012/13||FY 2013/14||FY 2014/15||FY 2015/16|
|Crimes Against Persons||$4,189.79||$4,218.02||$4,385.45||$4,554.83||$3,375.11|
|Crimes Against Property||$1,817.18||$1,932.79||$2,088.94||$1,698.41||$1,435.03|
|Annual Average Cost per Service Request||$4,251.19||$4,219.55||$4,664.69||$3,970.83||$3,087.07|
*The average cost per service request was determined, by multiplying the cost per sample by the average number of samples per request as documented in table 11. The annual average cost per service request does not take into account the level of effort required for each stream (i.e. homicides will result in more samples being processed than crimes against property)
While costs were lowest in 2015/16, this was also the year when the lab was least able to meet the 40-day turnaround target.
4. Conclusions and Recommendations
Over the five-year period examined, the demand for BCA remained relatively stable for the first four years of the Agreements with a 25% increase in service requests in 2015/16. The evaluation found that the increase in crime rates as well as legislative changes contributed to this increase in demand.
The BCA Agreements between the provinces/territories and the RCMP have contributed to the financial sustainability and predictability of the Program.
The provision of biology casework analysis is aligned with the mandate of the RCMP. The roles and responsibilities of clients, NFLS Headquarters and lab sites are well documented and operating within the same Standard Operating Procedures and quality assurance processes.
The NFLS research and development team validates and implements new technologies as well as actively engages in the broader forensic science research community. The intention is for research activities to support improvements in efficiency and/or effectiveness, however, the evaluation did not assess these areas in their analysis.
It was noted that on average, the 40-day target was achieved almost half the time (47%) for routine requests. Priority requests, with a negotiated timeline, were achieved 91% of the time. Although the lab continued to utilize alternative mechanisms such as overtime and private sector contracting to address the 2015/16 increase in demand, the achievement of combined routine and priority requests fell from 50% in 2014/15 to 43% in 2015/16. While the majority of clients are satisfied with the timeliness of Biology Services, it was unclear as to how the 40-day target was determined given the varying nature and complexity of requests.
There were several internal and external factors that impacted the Program's performance. With regard to internal factors, RCMPs lab policies, processes, procedures and governance structures supported program delivery. The evaluation also found performance information is available to assist management decision-making. With regard to external factors, the evaluation noted increased demands, quality of service request submissions, and a need for increased client awareness and education as impacting program performance.
Based on the findings of the evaluation, evaluators recommend that NFLS complete the following activities to improve the effectiveness and efficiency of biology casework analysis:
Recommendation #1: NFLS, in collaboration with clients, establish a service target that considers a balance between timeliness and cost of service to establish achievable diary dates.
Recommendation #2: NFLS assess existing communication materials targeted to the needs of police services and prosecutors, to determine if updates are required.
5. Management Response and Action Plan
NFLS agrees with the recommendation that it should establish a service standard(s). Guided by business intelligence related to performance and internal capacity, NFLS will engage with client/stakeholders in the upcoming year to establish standard(s) that best meets all needs.
NFLS agrees with the recommendation that it should assess existing communication materials targeted to the needs of police services and prosecutors, to determine if updates are required.
|Recommendation||Lead / Area of Responsibility||Planned Action||Diary Date|
|Recommendation #1: It is recommended that NFLS, in collaboration with clients, establish a service target that considers a balance between timeliness and cost of service to establish achievable diary dates.||NFLS||NFLS will engage with clients and stakeholders to establish service standards. (support steps are found below)||2018/03/31|
|NFLS||Briefed client base, via the Regional Client Consultation Committees (RCCCs) network, that NFLS is embarking on Service Standard model development.||Completed on 2017-06-16|
|NFLS + BI Team||In collaboration with FS&IS Business Intelligence Team, review and analyze lab performance information to generate service standard models for client consideration. Service standard model development will generally follow the principles outlined in The Guideline on Service Standards, developed by the Treasury Board of Canada Secretariat (https://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=25750)||2017/09/30|
|NFLS||Using NFLS existing client consultation network (Regional Client Consultation Committees - RCCCs), engage and receive feedback/outcomes from clients on the various service standard models.||Nov 2017 (several dates within November)|
|NFLS||Consult with clients via RCCCs to share consultation results and discuss refinements of any models if required.||February 2018|
|NFLS||Formally report consultation findings and final model selection via the RCCCs network.||2018/03/31|
Recommendation #2: It is recommended that NFLS assess existing communication materials targeted to the needs of police services and prosecutors, to determine if updates are required.
|NFLS||NFLS will review existing client targeted communication materials and update the documents if required. (support steps are found below)||2017/12/31|
|NFLS||Client Content Review NFLS will review and organize existing client communication products and update as required. The steps are:||2017/09/30|
Review and update client instructional products (Investigator's Guide, etc).
Status: The Investigator's Guide has been reviewed and updated.
|Completed on 2017-03-01|
|NFLS + Forms Management|| |
Review and update client submission form (C-414 Form).
Status: The first draft update of the C-414 – Request for Forensic Laboratory Analysis is going through final internal review (and then translation).
|NFLS + Program Managers|| |
Review and update client oriented submission policy information (exhibit acceptance criteria, lab service changes, etc).
Status: Reviews in progress
|NFLS + Program Managers|| |
Review and update all remaining client focussed communication products (bulletins, memos, etc.)
Status: Reviews in progress
- Date modified: