TY - JOUR AU - Emma, Piasecki, AB - Abstract Streamlined forensic reporting (SFR), introduced as part of the United Kingdom Ministry of Justice’s drive to deliver swift and sure justice, is credited with generating both time and cost efficiencies. Through the provision of radically abbreviated forensic reports at an early stage in criminal proceedings, SFR is said to avoid the cost of long form reports, facilitate agreement between the parties, secure more guilty pleas and reduce the number of defence challenges to forensic science evidence. This article questions these claims and the value of SFR as conceived. It suggests that the limited empirical evidence is mixed, and that SFR is incompatible with emerging trends and the best advice on the presentation of forensic science evidence. SFR directs little attention to the quality—that is, the validity and scientific reliability—of forensic science evidence. In overlooking quality, SFR introduces new risks of misrepresentation, misunderstanding and mistakes, and is unlikely to align with long-standing and fundamental criminal justice values (such as transparency, rationality, rectitude, equality of arms and fairness), and so is unlikely to fulfil the fundamental goal of dealing with cases justly. 1. Streamlining Justice? In 2013, the Ministry of Justice introduced a new system of radically foreshortened expert reports for criminal proceedings under the guise of streamlined forensic reporting (SFR). The stated purposes were to: avoid the costs associated with long form reports and the delays attributed to them; facilitate agreement between the parties; encourage defendants to plead guilty as early as possible; and restrict defence challenges to real (ie genuine) issues. The scheme was presented as ‘a revised case management procedure’ with ‘a more proportionate approach to forensic evidence’.1 SFR has reformed the way information about crime scenes and some types of forensic science evidence are reported. The most conspicuous change is the provision of results as a bare conclusion for a range of forensic science procedures, notably: fingerprints, DNA profiles, drugs, footwear, marks and traces, firearms, toxicology and digital evidence.2 For those who are not scientists or familiar with recent controversies in the forensic sciences, the introduction of SFR might appear as a reasonable, indeed proportionate, reform. This article offers an alternative perspective on SFR, its underlying assumptions, its rationale, the prevalence of ‘real issues’ confronting the contemporary forensic sciences, and the effectiveness of SFR in terms of efficiency and justice. Drawing upon criminal justice norms and commitments (particularly around reliability, rectitude and fairness) and mainstream scientific advice, we question the utility of SFR.3 In doing so, we are sensitive to: the allocation of the burden of proof in accusatorial trials; experience with wrongful convictions; emerging perspectives on the scientific foundations of forensic science procedures; recent revelations on the limited efficacy of traditional trial safeguards (such as cross-examination, adduction of contrary evidence and judicial directions), and the ever-increasing inequality of arms between the expertise and resources available to the state as opposed to criminal defendants in adversarial proceedings. For reasons developed below, we suggest that it is unlikely that SFR will provide significant time and costs savings over alternative forms of reporting. Our concerns are reinforced by formal reviews. Among the most striking findings of the Law Commission’s review of expert evidence in criminal proceedings in England and Wales in 2011 were that too much questionable forensic science evidence was admitted and that conventional trial safeguards had not been effective in exposing limitations.4 The Law Commission ‘shared the [House of Commons Science and Technology] Committee’s concern that expert opinion was being admitted in criminal proceedings too readily with insufficient scrutiny’.5 The Commission recommended the introduction of a statute-based admissibility standard requiring lawyers and judges to attend to the reliability of expert evidence.6 That recommendation was not enacted by the Conservative government, largely on the basis of imputed costs. Instead, the Criminal Procedure Rule Committee was invited to amend the Criminal Procedure Rules (CrimPR) on the basis that it ‘could increase the likelihood of the trial judge and the opposing party, where appropriate, challenging expert evidence’.7 Simultaneously, the government closed the Forensic Science Service, in an attempt to expand the marketplace and rationalise costs associated with the provision of forensic science evidence.8 Shortly thereafter, the Ministry of Justice adopted SFR as part of its Swift and Sure Justice initiative, declaiming its consistency with criminal justice values and economic prudence. SFR sits awkwardly alongside the best advice on the presentation of forensic science evidence—discussed in Section 4. The uncomfortable juxtaposition was magnified by the reform to the CrimPR part 19 and the creation of Criminal Practice Direction (CrimPD) part 19 (together Part 19), which seek to incorporate the Law Commission’s recommendations.9 Intended to enhance reliability, comprehension and transparency, these procedural rules and practice direction impose stringent requirements on expert evidence—discussed in Section 5. There are, in consequence, now two streams of forensic science reports: (i) the ‘traditional’ statement or report, where Part 19 applies; and (ii) SFR, where it does not. Precisely why some types of forensic science evidence should be pre-emptively exempted from conventional disclosure obligations, including the need to ‘include such information as the court may need to decide whether the expert’s opinion is sufficiently reliable to be admissible as evidence’, remains obscure.10 Though, as we explain, it is not necessarily because the findings reported in streamlined forensic reports are reliable or expressed in scientifically defensible terms.11 2. An Introduction to the SFR Scheme SFR is built around four pro-forma documents set out in the schematic diagram reproduced as Figure 1. Two of these documents, namely Streamlined Forensic Report—SFR Stage 1 Forensic Result (SFR1 or MG22(b)) and Streamlined Forensic Report—SFR Stage 2 Forensic Issues (SFR2 or MG22(c)), are the primary focus of this article. Figure 1. View largeDownload slide Streamlined forensic reporting process flowchart (taken from the National Streamlined Forensic Reporting Guidance, s 1, 7 accessed 1 June 2018). Figure 1. View largeDownload slide Streamlined forensic reporting process flowchart (taken from the National Streamlined Forensic Reporting Guidance, s 1, 7 accessed 1 June 2018). Introduction of the SFR1 was intended to facilitate the provision of ‘sub and source level’ results, relating to the identification of persons via intelligence databases—such as the National DNA Database (NDNAD) and the National Fingerprint Database—and to provide evidence about prohibited items, substances and traces. The SFR1 is typically a page or so in length, with the ‘Results/Findings’ presented very succinctly. Section 1 of the SFR1 summarises the results of a test or comparison, though it may not incorporate an evaluation of any findings—see Figure 2.12 Section 2 documents the status of other exhibits related to the case. It is intended to list exhibits regardless of whether they have been examined or tested, and record any tests that produced a negative or inconclusive result. The SFR1 is not a witness statement (and therefore contains no statement of truth) or an expert report to which Part 19 applies.13 It may be completed by a forensic practitioner or by a non-expert intermediary who ‘will not be able to testify as to the forensic process or the accuracy of the conclusions’.14 A non-expert may complete the SFR1 where it reports the results from a database search, as in the case of a matching DNA profile or in respect of fingerprints, where it provides a summary of the conclusions of the forensic practitioner who actually undertook the analysis following a database search. Exhibit(s) Examined Results/Findings MRN/1 Lift from picked out glass, kitchen A finger/palm mark in exhibit MRN/1 is identified to a set of fingerprints held on the National Fingerprint Database in the name of XXXXX reference XXXXX MRN/2 Lift from picked out glass, kitchen A finger/palm mark in exhibit MRN/2 is identified to a set of fingerprints held on the National Fingerprint Database in the name of XXXXX reference XXXXX Evidence Type Supporting/Technical Information [no information provided] Exhibit(s) Examined Results/Findings MRN/1 Lift from picked out glass, kitchen A finger/palm mark in exhibit MRN/1 is identified to a set of fingerprints held on the National Fingerprint Database in the name of XXXXX reference XXXXX MRN/2 Lift from picked out glass, kitchen A finger/palm mark in exhibit MRN/2 is identified to a set of fingerprints held on the National Fingerprint Database in the name of XXXXX reference XXXXX Evidence Type Supporting/Technical Information [no information provided] Exhibit(s) Examined Results/Findings MRN/1 Lift from picked out glass, kitchen A finger/palm mark in exhibit MRN/1 is identified to a set of fingerprints held on the National Fingerprint Database in the name of XXXXX reference XXXXX MRN/2 Lift from picked out glass, kitchen A finger/palm mark in exhibit MRN/2 is identified to a set of fingerprints held on the National Fingerprint Database in the name of XXXXX reference XXXXX Evidence Type Supporting/Technical Information [no information provided] Exhibit(s) Examined Results/Findings MRN/1 Lift from picked out glass, kitchen A finger/palm mark in exhibit MRN/1 is identified to a set of fingerprints held on the National Fingerprint Database in the name of XXXXX reference XXXXX MRN/2 Lift from picked out glass, kitchen A finger/palm mark in exhibit MRN/2 is identified to a set of fingerprints held on the National Fingerprint Database in the name of XXXXX reference XXXXX Evidence Type Supporting/Technical Information [no information provided] Figure 2. View largeDownload slide Section 1 (or conclusion) from an SFR1. The name(s) and reference number(s) have been redacted. Figure 2. View largeDownload slide Section 1 (or conclusion) from an SFR1. The name(s) and reference number(s) have been redacted. The SFR1 does not include information about the method used, the quality of the sample or information about collection, analysis and interpretation, personnel involved, limitations and so forth. A redacted example from an actual SFR1, set out in Figure 2, illustrates the extremely limited nature of the information provided. On receiving the SFR1, the defendant may accept the finding, by way of an admission, under section 10 of the Criminal Justice Act 1967 (and pursuant to CrimPR 19.3). The admission becomes proof of the fact. In Figure 2, this would be the identity of the person who deposited the fingerprint(s). Alternatively, the defendant bears the burden of identifying one or more ‘real issues’ that prevent the making of an admission.15 Where ‘real issues’ are identified, the prosecutor should request that an expert address them in an SFR2—through the provision of additional information or further analysis and/or evaluation. The prosecutor may also request an SFR2 in order to evaluate or contextualise the evidence. The resulting SFR2 is not a full evaluative report, considering all forensically relevant aspects of a case. Rather, it is intended only to address specific ‘real issues’ raised by the defendant in response to the SFR1. It is, however, a witness report, and the expert may be required to testify (in compliance with Part 19). Alternatively, the prosecutor may contest the ‘reality’ of issues raised by the defendant. The two remaining SFRs predominantly deal with examinations by crime scene investigators (CSIs). They are SFR Initial Forensic Investigation/Examination Report (MG22(a)) and SFR Crime Scene Investigation/Forensic Examination Statement (MG22(d)). The MG22(a) documents the details of a scene or exhibit examination. It provides a summary of results, and may provide interpretation or intelligence information as to the significance or otherwise of any initial examinations. It is not intended as evidence, but should be disclosed where appropriate. It is typically completed by the CSIs undertaking the examination and is intended to relay, in a timely and usable format, the results to date—such that further questioning or inquiries may be initiated or deemed unnecessary. Information from an MG22(a) may be incorporated into an SFR1. The fourth of the pro forma reports is the MG22(d). Typically completed by a CSI, it documents observations at a crime scene and provides a record of continuity of crime scene exhibits and/or contextual or evaluative information relating to the scene and materials observed or collected. It sometimes includes photographs. The MG22(a) and (d) are less contentious than the SFR1 and responsive SFR2, and many of the issues explored in this article are not applicable to them. 3. Overselling Streamlined Forensic Reporting At this juncture, it is useful to begin to place the emergence of SFR into a broader context. Here, we introduce some of the ‘purposes and benefits’ attributed to SFR. In light of the ensuing analysis, these claims are revealing. They embody controvertible claims and assumptions about forensic science evidence, the effectiveness of criminal justice processes and the ability of legal participants (and others) to consistently identify problems with, or effectively regulate and understand, forensic science evidence. According to the guidance documents produced to assist with its implementation, the main ‘purposes and benefits’ of SFR are: Enabling experts and forensic science providers to produce their findings as early as possible after a forensic result is obtained; Enabling experts to produce their findings in the most cost effective way; and ensuring that further analysis is confined to cases and issues where there is a real issue with the forensic evidence. Ensuring investigators have accurate information to conduct investigations, make arrests and conduct interviews; Ensuring police and prosecutors can make early and informed charging decisions; Ensuring that the defence are able to have early sight of the forensic reports in the Initial Details of the Prosecution Case; Ensuring that early and informed pleas can be entered by defendants; Providing a form of evidence that the prosecution can serve as part of its case for service and/or trial; Providing a form of evidence which summarises the conclusions of the forensic evidence in a form which enables Defence advocates to take clear instructions as to whether those conclusions are accepted; Providing a form which facilitates the agreement of the forensic evidence by way of section 10 admission; Providing a summary which enables the forensic evidence to be agreed by way of admission, in accordance with Crim.PR 19.3(1) and 19.3(2) Assisting the courts to fulfil their CrimPR duties actively to case-manage cases; Providing a platform and means for the real issue(s) which any further forensic work must address, to be identified in cases where such work is necessary.16 These goals are generally laudable. Claims about the impact and value of SFR are, however, largely declaratory. Consider the provision of ‘accurate information’, ‘informed pleas’, and forensic science evidence summarised in a manner that ‘enables’ lawyers to ‘take clear instructions’ or ‘facilitate[s] agreement’. These and other benefits rely explicitly on the evidence being robust, accurate, appropriately qualified, comprehensible and comprehended. Yet, not one of the aims relates to ensuring that the evidence contained within the SFR is demonstrably valid and/or scientifically reliable, or to identifying issues that might assist with comprehension and evaluation. The introduction of SFR is said, employing the language of civil justice, to be ‘proportionate’. It is intended to reduce costs, encourage early pleas and reduce expert disagreement. To the extent that it does, that may be reasonable where persons are guilty or the overall case is compelling. However, where SFR misrepresents the evidence (including by expressing a summary or impression as expert opinion) or omits limitations, or the result is mistaken (for any number of reasons), the new regime will make it more difficult to evaluate the findings. Mistakes will be harder to identify and effectively challenge. Furthermore, it is unclear how any proposed ‘proportionality’ aligns with such long-standing and fundamental criminal justice values as transparency, rationality, rectitude, equality of arms and fairness. The remainder of this article is organised around a series of problematics, all with a tendency to destabilise assumptions supporting SFR, along with its purposes and claimed benefits. The following questions, considered in the ensuing discussion, tend to undermine SFR—particularly the SFR1 and SFR2—as conceived and applied. They focus attention on issues central to understanding and evaluating SFR. For example, does SFR accurately embody the known value of forensic science evidence and are findings presented in a scientifically appropriate form?17 Is the expectation that the defendant will identify problems with the forensic science evidence (so-called ‘real issues’) consistent with values central to our system of accusatorial justice? Is the inability to understand, or make sense of, the conclusion in an SFR1 sufficient to ground a request for further information?18 Is it unreasonable for the defendant to put the state to proof when the state chooses to rely upon expert evidence?19 Has the introduction of SFR, and particularly use of the SFR1, saved time or money and delivered other claimed ‘benefits’?20 Are there too many (spurious) challenges to forensic science evidence?21 And how were the forensic science procedures included in SFR selected, and on what grounds could SFR be legitimately expanded to include other procedures?22 Few of these questions seem to have been explored in the design and promotion of SFR. Revealingly, the reforms associated with SFR do not engage with the recommendations of the Law Commission or scientifically informed advice. Indeed, the introduction of SFR overlooks historical problems with forensic science evidence and a lamentable series of wrongful convictions linked to inadequate disclosure, overclaiming, and insensitivity to scientific research and advice.23 Proponents and users have tended to adopt credulous approaches to the forensic sciences where the state’s evidence is treated as presumptively reliable. We now turn to consider just this issue. 4. Independent Scientific Advice, Validity and Scientific Reliability The introduction of SFR is revealing given that the primary concerns about forensic science evidence, beyond the Ministry of Justice and perhaps the courts, have been focused on technical and methodological issues—what lay people refer to as trustworthiness or reliability. In recent years, there has been unprecedented interest in validity, scientific reliability, standards, proficiency, error and uncertainty, the terms used to report (ie communicate) findings and how human factors might contaminate results.24 At a time of unprecedented concern about the value of evidence produced by forensic science providers, the Ministry of Justice’s response, driven by a desire to reduce costs, has been to restrict the provision of information, encourage early guilty pleas and discourage challenges by defendants. Revealingly, the number of guilty pleas and defence challenges have been the primary metrics used to assess SFR. The reforms associated with SFR might be at least comprehensible if the forensic sciences were in a healthy epistemic condition. Lamentably, many are not. Coinciding with the dissolution of the Forensic Science Service (the forensic science agency of the Home Office) and greater in-sourcing by police, comprehensive technical reviews of the forensic sciences were taking place both at home and internationally.25 For the first time in their history, many forensic sciences were subjected to the intense scrutiny of independent scientists, engineers and statisticians.26 To the surprise of many, the resulting reports have been remarkably critical of procedures and derivative evidentiary claims, along with the history of legal responses to forensic science evidence.27 Reports by the National Research Council of the US National Academy of Science (NRC), Lord Campbell, the US Department of Justice and National Institute of Standards and Technology, President Obama’s Council of Advisers on Science and Technology (PCAST), the American Association for the Advancement of Sciences and the Royal Societies of England and Edinburgh have all questioned the scientific pretensions or accuracy of forensic science procedures, including some in routine use in investigations and prosecutions (and subject to SFR).28 Along with advice from the UK’s own Forensic Science Regulator (hereafter the Regulator), all of the reports insist on the need for more research, recommend procedural reform and, importantly for our purposes, advocate changes to the traditional reporting of results.29 There are a few particularly important points to take away from the recent reviews. First, all place emphasis on the need for forensic science procedures to be formally evaluated—so-called validation. Validation provides evidence that the procedure does what is claimed, and lays out the conditions under which it is known to work and how well. Validation studies assist with the generation of standards (or protocols), and provide information about accuracy and how results can be legitimately expressed. Along with rigorous proficiency tests, where the correct answer (or ‘ground truth’) is known, they provide evidence about the abilities and performance of forensic practitioners, practices and procedures, as well as insight into limitations, uncertainties and the frequency of errors.30 Secondly, many procedures, including some in routine use, have never been formally evaluated. In consequence, some forensic science procedures have not been shown to be valid and some of the common expressions used by forensic practitioners are, in consequence, impressionistic.31 Overall, the reports present the forensic sciences as a disparate set of activities, ranging from scientifically robust and stabilised procedures—exemplified by simple DNA profiling, with results expressed in statistically informed terms (eg a likelihood ratio)—to those plagued by residual doubts about their evidentiary value—such as shoe print and toolmark comparisons.32 Others, such as ballistic and latent fingerprint comparisons, seem to lie somewhere in between. Thirdly, the various reports, along with materials produced by the Regulator, document threats to forensic science procedures posed by human factors, particularly threats to the cognition of forensic practitioners involved in any kind of subjective assessment or interpretation. The reports recommend studying these risks and, where appropriate, eliminating dangers, such as priming and confirmation bias, through blinding. That is, practitioners should not be exposed to domain-irrelevant information, and any exposure to such information, such as the suspicions of investigators, should be assiduously documented.33 In conjunction with the primacy attributed to validation, most of the reviews have insisted on the need for greater transparency in forensic science reporting—specifically, the provision of more detail in expert reports prepared by, or on behalf of, the state and its agencies. It is useful to consider recommendations from the NRC on the content of expert reports to be used for investigations and prosecutions. The NRC insisted that: As a general matter, laboratory reports generated as the result of a scientific analysis should be complete and thorough. They should describe, at a minimum, methods and materials, procedures, results, and conclusions, and they should identify, as appropriate, the sources of uncertainty in the procedures and conclusions along with estimates of their scale (to indicate the level of confidence in the results). Although it is not appropriate and practicable to provide as much detail as might be expected in a research paper, sufficient content should be provided to allow the nonscientist reader to understand what has been done and permit informed, unbiased scrutiny of the conclusion … Forensic science reports, and any courtroom testimony stemming from them, must include clear characterizations of the limitations of the analyses, including associated probabilities where possible.34 Information about limitations and uncertainty is required to enable ‘understanding’ and ‘unbiased scrutiny’ of results. No fields or procedures were exempted from these recommendations. The Regulator has issued similar advice on the content of expert reports (and more documents are forthcoming). Those who have formally reviewed expert reporting practices—both scientists and lawyers—have uniformly insisted on the need for more rather than less information.35 Those promoting SFR do not appear to have engaged with these reviews and recommendations. Independent reviews of the forensic sciences have also cast doubts on some of the expressions used by forensic practitioners in their reports and testimony.36 Terms and expressions assume considerable import in a system that relies on very succinct ‘summaries’ of forensic science results, sometimes relayed by non-expert intermediaries. Several of the reviews recommend that interpretive conclusions, particularly those produced using forensic comparison procedures (such as fingerprints), should not be expressed categorically (or as fact) and should explicitly incorporate limitations.37 Attentive scientists have been highly critical of categorical conclusions and positive identifications.38 They caution: Troublingly, expert witnesses sometimes go beyond the empirical evidence about the frequency of features—even to the extent of claiming or implying that a sample came from a specific source with near-certainty or even absolute certainty, despite having no scientific basis for such opinions. From the standpoint of scientific validity, experts should never be permitted to state or imply in court that they can draw conclusions with certainty or near-certainty (such as ‘zero,’ ‘vanishingly small,’ ‘essentially zero,’ ‘negligible,’ ‘minimal,’ or ‘microscopic’ error rates; ‘100 percent certainty’ or ‘to a reasonable degree of scientific certainty;’ or identification ‘to the exclusion of all other sources.’39 The conclusion in Figure 2, for example, is a categorical conclusion.40 It purports to identify a specific person (implicitly to the exclusion of all others).41 The reliability and probative value of a particular fingerprint ‘match’ is dependent on the quality of the latent print, particularly the amount of comparable detail.42 These kinds of insight are not incorporated into SFR. Significantly, technically sophisticated reviewers have expressed an emphatic preference for the use of statistical expressions derived from empirical studies to equating a match with categorical identification. SFR is revealing in its derogation.43 We can obtain a sense of the magnitude of these issues by considering recommendations produced by PCAST. In its review of seven forensic feature comparison methods, PCAST concluded that only two—simple DNA profiling and latent fingerprint comparison—were foundationally valid.44 That is, these procedures can accurately determine whether a crime sample could be associated (note: not individualised) with a reference sample taken from a known individual or source. Notwithstanding the seemingly positive assessment, PCAST expressed concern at the manner in which opinions about latent fingerprints were routinely reported.45 PCAST insisted that the known limits of a procedure must be reported alongside the opinion to avoid misleading those trying to understand or evaluate the result. So, rather than report a fingerprint ‘match’ as being ‘identified to a set of fingerprints in the name of [a specific individual]’, as in the SFR1, PCAST explained that: it would be appropriate to inform jurors that (1) only two properly designed studies of the accuracy of latent fingerprint analysis have been conducted and (2) these studies found false positive rates [ie misidentifications] that could be as high as 1 in 306 in one study and 1 in 18 in the other study. This would appropriately inform jurors that errors occur at detectable frequencies, allowing them to weigh the probative value of the evidence.46 Such qualifications and limitations are not disclosed in the SFR1.47 The ‘real issues’ plaguing the contemporary forensic sciences, clearly identified by independent scientists, engineers and statisticians, are the lack of appropriate testing and modesty in reporting. Interestingly, these observations imply that legal procedures and personnel have not effectively exposed serious and continuing problems. The NRC report was critical of the performance of US courts and doubted their ability to unilaterally improve the quality of the evidence and its presentation in expert reports and courtroom testimony.48 This seems like an important observation, to the extent that SFR has been designed and implemented primarily by lawyers and investigators. Of particular interest, one of the main reasons for the introduction of SFR, namely reducing unmeritorious challenges to forensic science evidence, seems to be fundamentally misconceived. The scientific reviews insist, in a manner that is hugely disruptive to SFR and its foundations, that legal challenges have typically been inadequate and legal personnel and institutions far too credulous. Along with the Law Commission, attentive scientists have repeatedly questioned the effectiveness of legal safeguards. On this point, we should not overlook the fact that most of the recent revelations about the condition of the forensic sciences did not emerge through quotidian criminal proceedings—whether prosecution disclosure, defence challenges or successful appeals. Recognition of problems emerged out of formal reviews by independent (non-forensic) scientists. 5. Part 19: The Criminal Procedure Rules and Criminal Practice Direction Earlier, we referred to the existence of two streams of forensic science reporting. Standing alongside SFR is a more traditional approach to expert evidence regulated by revisions to Part 19. In responding to the prevailing laissez faire attitude to the reception of expert evidence identified by the Law Commission, revisions to the CrimPR and CrimPD direct attention to validity, uncertainty, reliability and limitations in the context of the overriding objective that cases be dealt with justly. According to the CrimPR, dealing with a case justly includes: ‘acquitting the innocent and convicting the guilty’; ‘dealing with the prosecution and the defence fairly’; ‘recognising the rights of a defendant’; ‘dealing with the case efficiently and expeditiously’; and ‘dealing with the case in ways that take into account … the gravity of the offence … the complexity of what is in issue … [and] the severity of the consequences for the defendant and others affected’.49 Furthering the need to deal with cases justly requires the court to ‘actively manage the case’ by: ‘the early identification of the real issues’; ‘ensuring that evidence, whether disputed or not, is presented in the shortest and clearest way’; and encouraging the participants to co-operate in the progression of the case.50 ‘Each party must actively assist the court’ by determining ‘what information, or other material, is required by one party of another, and why’.51 These expectations are now embodied within Part 19 and are applicable to all expert evidence, with the exception of SFR. Experts are required to give opinions that are ‘objective and unbiased, and … within the expert’s area or areas of expertise’.52 More specifically, CrimPR 19.4 dictates that an expert report must: (b) give details of any literature or other information which the expert has relied on in making the report; (c) contain a statement setting out the substance of all facts given to the expert which are material to the opinions expressed in the report, or upon which those opinions are based; … (e) where the expert has based an opinion or inference on a representation of fact or opinion made by another person for the purposes of criminal proceedings (for example, as to the outcome of an examination, measurement, test or experiment)— (i) identify the person who made that representation to the expert … (f) where there is a range of opinion on the matters dealt with in the report— (i) summarise the range of opinion, and (ii) give reasons for the expert’s own opinion; (g) if the expert is not able to give an opinion without qualification, state the qualification; (h) include such information as the court may need to decide whether the expert’s opinion is sufficiently reliable to be admissible as evidence; (i) contain a summary of the conclusions reached; (j) contain a statement that the expert understands an expert’s duty to the court, and has complied and will continue to comply with that duty; and (k) contain the same declaration of truth as a witness statement CrimPD 19A.5 refers to ‘factors that the court may take into account in determining the reliability of expert opinion, and especially of expert scientific opinion’, such as: the extent and quality of the data on which the expert’s opinion is based, and the validity of the methods by which they were obtained; if the expert’s opinion relies on an inference from any findings, whether the opinion properly explains how safe or unsafe the inference is (whether by reference to statistical significance or in other appropriate terms); if the expert’s opinion relies on the results of the use of any method (for instance, a test, measurement or survey), whether the opinion takes proper account of matters, such as the degree of precision or margin of uncertainty, affecting the accuracy or reliability of those results; the extent to which any material upon which the expert’s opinion is based has been reviewed by others with relevant expertise (for instance, in peer-reviewed publications), and the views of those others on that material; the extent to which the expert’s opinion is based on material falling outside the expert’s own field of expertise; the completeness of the information which was available to the expert, and whether the expert took account of all relevant information in arriving at the opinion (including information as to the context of any facts to which the opinion relates); if there is a range of expert opinion on the matter in question, where in the range the expert’s own opinion lies and whether the expert’s preference has been properly explained; and whether the expert’s methods followed established practice in the field and, if they did not, whether the reason for the divergence has been properly explained. CrimPD 19A.6 directs the court ‘to be astute to identify potential flaws in such opinion which detract from its reliability’. Further, CrimPD 19B requires the expert to provide independent assistance by way of unbiased, objective opinion on matters within the expert’s area of expertise. There is an expectation that the expert will disclose any conflicts of interest and resist importunity from those who provide instruction or payment. The expert must exercise reasonable care and skill to ensure accuracy and completeness, and disclose any matter that might adversely affect the validity of their opinion along with necessary qualifications. In addition, the expert is required to confirm that CrimPR 19 and the relevant Code of Practice for the discipline have been complied with. For experts called by the prosecutor, it also requires compliance with the Crown Prosecution Service (CPS) guidance document Disclosure—Experts’ Evidence, Case Management and Unused Material.53 The revisions to the CrimPR and CrimPD are a positive development consistent with independent scientific advice and the Law Commission’s recommendations. The reader might reasonably wonder about the reason for maintaining two parallel streams of expert reporting. Why are these criteria deemed unnecessary, and presumptively ignored, in relation to the provision of forensic science results within SFR? As the NRC and PCAST reports explain, the reason cannot be because the procedures are without non-trivial limitations or that categorical conclusions (eg the examples in Sections 3 and 4) fairly summarise the forensic science evidence.54 While the provisions of Part 19 may not be rules of admissibility per se, the terms reflect emerging concerns among senior members of the judiciary about the quality of forensic science evidence.55 There have been no efforts to explain the exemption of SFR from these requirements, apart from claims about cost and efficiency and the presumption that procedures included within SFR are implicitly valid and scientifically reliable. 6. Problems with SFR The introduction of SFR has placed new expectations on defendants. In order to obtain information, they must precisely identify issues with the ‘findings’ or specify what additional work ought to be undertaken and why. Ignorant of what was collected, how it was selected, sampled, removed, examined, tested, and interpreted, how the form of expression was selected, the risk of error, the type of review (if any), the personnel involved and so forth, the defendant and any legal representatives are now required to identify issues or risk the evidence being admitted as proof of the conclusion stated. There are many problems with this approach and the new evidentiary and resource burdens it imposes, most conspicuously on defendants. These problems directly threaten the effectiveness of SFR. A. Do Streamlined Forensic Reports Save Time and Money and Secure More Guilty Pleas? Justifications for SFR tend to be couched in terms of the objectives of the CrimPR. Claims abound that SFR enables experts ‘to produce their findings as early as possible after a forensic result is obtained’ and ‘to produce their findings in the most cost effective way’.56 We question whether the use of an SFR1 saves time and resources, or contributes to making criminal proceedings just. Indeed, we wonder where the purported savings are coming from. We also wonder how the provision of less information (and, importantly, the provision of potentially misleading findings) will produce earlier and more reliable guilty pleas and convictions.57 For procedures that are standardised and in routine use, we doubt that an SFR1 is cheaper to produce than the kind of report required by Part 19 (discussed in Section 5). In order to produce the results underlying the SFR1 (or an SFR2), a properly qualified forensic practitioner must undertake some analysis in an accredited laboratory (or deliberate and prepare a written response). The forensic practitioner must, on the basis of that analysis, offer an interpretation. That result must be formally recorded in an SFR1 or documented and conveyed to the non-expert author of the SFR1. Where the result is simply ‘reported’ by a non-forensic practitioner, the only obvious saving is the difference in salary. Here, we draw attention to the possibility of using pre-prepared (ie pro forma) text, standardised descriptions of methods, protocols, relevant literatures and accepted forms of expression—including appropriate qualifications and caveats. This sort of template is available and in use in several jurisdictions.58 Entering case-specific details into such a pro forma would seem unlikely to take much more time than is required to enter the same results into an SFR1, and it could automatically generate a (properly qualified) summary as a front page. There is no reason why a report compliant with Part 19 could not enter the criminal justice process just as early as a putatively streamlined one. More importantly, such a report would be much less likely to mislead the user, much less likely to require an SFR2 and any associated legal or scientific costs, could count as a potentially admissible witness statement, and could avoid the need to rely upon non-experts. Even if we were to persist with the SFR1, we should use empirically based expressions and appropriate qualifications to enhance transparency and comprehension. If costs are actually being saved through the use of an SFR1, then it might be because analysis and interpretation is restricted in comparison with full evaluative reports. More limited forms of testing and reporting might save money, but they simultaneously reduce the value of the evidence and, if not properly disclosed, distort the conclusions, introducing serious risks into criminal justice processes. To the extent that limits are not disclosed, they increase the risk of mistakes—both errors and misinterpretations—among investigators and other users.59 Users must be informed when analysis and interpretation is limited.60 Any derogation in the kinds of procedures used and the reliability of conclusions needs to be disclosed in all expert reports to avoid misleading those relying on findings, and to avoid tunnel vision and other institutional dangers.61 We are also highly sceptical of the (perhaps implicit) contention that properly qualified results—offered in a timely manner—will significantly alter the way defendants plead.62 And the alternative is disconcerting. Are proponents of SFR suggesting that the provision of forensic science evidence in a scientifically based and comprehensible manner will reduce the number of pleas? If so, we wonder about the willingness to provide bare conclusions, without the caveats and qualifications required to evaluate them, apparently in order to obtain (more) guilty pleas. The state has a responsibility to disclose real issues with its evidence.63 Better information will focus efforts and reduce attempts to explore unmeritorious issues. It will simultaneously provide public assurance that guilty pleas are based on reliable evidence. Regardless of whether there are actually fewer challenges, as a general principle, the state should not obtain any benefit derived from the failure to validate procedures (in routine use), measure error rates, and disclose limitations and uncertainties. In an accusatorial system of criminal justice, it cannot be the responsibility of individual defendants—including innocent defendants—to retrospectively identify errors or methodological limitations with procedures or terminology in routine use. Justifications for SFR place an emphasis on costs savings, but there are no references to new or additional costs incurred in the identification of real issues (a cost shifted to the defendant). Legal preparation, communications between defence lawyers and prosecutors, determining whether a defence expert is required and available (even at this preliminary stage), formally applying for an SFR2, potentially contesting the ‘reality’ of issues, along with the production of a bespoke SFR2, and questions about its adequacy are all non-trivial costs that appear to be ignored. Repeatedly identifying issues and challenging evidence is likely to be more expensive—perhaps much more expensive—than the costs of embedding findings in a standardised reporting template that presents results accurately while addressing many of the ‘real issues’ left begging by the opaque findings currently reported in the SFR1.64 Moreover, the effort and additional costs involved in challenging forensic science evidence may discourage challenges and perpetuate legal complacency. Ironically, the failure to provide a more detailed report may elide the existence of real issues, especially where defendants are not represented, and where legal representatives are busy, poorly resourced, inexperienced, technically challenged, lazy or—as many appear to be—oblivious to contemporary issues in the forensic sciences. B. ‘Real Issues’ and Doubt This requirement upon the Defence does not require them to identify a technical or scientific issue with the conclusions in the SFR 1, as it is accepted that at this point, they will not have the benefit of their own expert opinion. The requirement to identify the issue is simply a request that the Defence identifies a reason why the defendant does not accept the conclusions in the SFR 1 …65 The nature of any issues raised by the Defence should be clarified with sufficient precision to allow the Stage 2 Report (SFR 2) to be completed.66 The apparent simplicity of ‘real issues’ is deceptive. In a rational system of justice, the defendant should be placed in a position to understand the charge and the (scientific) evidence assembled against him or her. To that end, forensic science evidence should be presented fairly and in a manner that facilitates comprehension. The defendant, legal representatives and, where the evidence is in issue, the judge and tribunal of fact should be in a position to understand and rationally evaluate the evidence.67 Despite the government’s proclamation that ‘none of these reforms will compromise historic legal rights or important principles of justice’, by requiring the defendant to identify problems, limitations and uncertainties with forensic science evidence in the absence of detail, SFR might not be considered entirely consistent with the principles enunciated in Woolmington v DPP.68 A defendant should not have to identify a ‘real issue’ in order to have expert opinion evidence presented in a form that is consistent with the known limits of a procedure or to have uncertainties and limitations disclosed. It is difficult to conceive of reasons for not expressing the conclusion in an SFR1, or any expert report, in epistemologically defensible terms. Expert reports should present conclusions ‘warts and all’.69 Appropriate—ie good-faith—disclosure is consistent with obligations on prosecutors and the expectations placed on experts by Part 19, the Regulator and professional bodies. The presentation of expert evidence in a way that misrepresents (including through omission) the value of the evidence will almost always constitute a ‘real issue’ for the suspect or defendant.70 In calling on the defendant to identify ‘real issues’, a new burden is placed on the worst resourced participant in the criminal justice system. The defendant is not a repeat player in the sense of the Crown and forensic science provider.71 The defendant is almost always lacking technical sophistication, and very often lacks access to scientific, technical and statistical advice. This new expectation may secure more acquiescence and more guilty pleas, but it will not always be because there are not ‘real issues’, misunderstandings or mistakes. In some cases, it will be because the defendant and his counsel are not adequately resourced or sufficiently competent to appreciate that the SFR1 misrepresents (through exaggeration or omission) the value of the evidence and, in some non-trivial proportion of cases, may be mistaken. All procedures have an error rate regardless of whether they are measured or disclosed.72 In expecting the defendant to identify legally cognisable issues, there is a real risk that even innocent defendants will be overwhelmed by the case assembled against them, the apparent strength of the scientific evidence, or the difficulty of retrospectively uncovering or demonstrating an error. Some, including a proportion of those who are innocent, will take advantage of the discount associated with an early plea of guilty.73 One of the problems with the SFR scheme, and the commitments reproduced at the beginning of this section, is that where a person accused of a crime indicates that they are unable to understand or make sense of the finding in an SFR1, or simply requires more information because they think there must be a problem or mistake, it is difficult to imagine that more could be required of them. In such circumstances, it would seem incumbent upon the prosecutor to provide information about collection and continuity, the validity and reliability of procedures, standards, laboratory accreditation, the training and proficiency of the forensic practitioners, how the specific terms of the conclusion were selected, alternatives excluded, cognitive bias managed and so forth (see Section 6C) in order for the defence to properly consider the evidence and its position. Another of the difficulties is that there appears to be no scope for a defendant who does not challenge the finding per se but, rather, requires the forensic science evidence to be presented in a form that accurately presents its known value—which includes limitations, uncertainty and an indicative error rate—such that these might factor into proof. Similar issues arise where the defendant contests the accusation but seeks to have the tribunal of fact consider the absence of validation (ie testing) or the significance of ongoing controversy and mainstream scientific criticism.74 C. So What Isn’t a ‘Real Issue’? SFR allows the prosecution to inform the Defence of forensic evidence in such a way as to enable them to comply with their obligations under the Criminal Procedure Rules … In this way, forensic issues are identified early and the need for forensic witnesses to attend court unnecessarily is reduced. The intention is to take to trial only the issues that are in dispute. What this achieves, in practice, is to minimize issues based on alleged technicalities being identified just before trial. … If agreed, it is admissible but if it is not agreed then the defence should state the contested issue(s).75 In many cases, the forensic science evidence will not be material or contested. However, where the defendant does not accept or cannot understand one or more findings, or seeks to rely upon limitations and uncertainties, why does the spirit of Part 19, applicable to all other expert opinions, not apply? In contested proceedings, we find it difficult to imagine grounds on which the prosecutor (as a ‘minister of justice’) or the court could avoid providing the defendant with further information in response to the following ‘real issues’: What procedure(s), equipment or algorithms were used in the analysis? Has the procedure been validated (and, if so, what is the evidence of validation)? Was the procedure used in a way that is consistent with validation testing and did it conform with relevant standards, protocols and accreditation?76 Was the analysis supporting the conclusion a full-scale analysis or some more limited, preliminary or presumptive assessment? What is the error rate associated with this procedure? (If it is unknown please state that.) What limitations and uncertainties are associated with the procedure and the interpretation of results? Is there published disagreement, controversy or criticism associated with the way the procedure(s) was applied or the result reported in this case? Can you identify relevant literature or debates? Was the analysis performed ‘blind’—ie was the forensic practitioner shielded from gratuitous contextual information? What did the forensic practitioner know about the case, the suspect, police suspicions, the scene, and so forth, when undertaking the analysis? Has the conclusion in the SFR1 been reviewed by other forensic practitioners? What did any reviewer know about the case or conclusion when undertaking their review?77 Are the procedure, its application, the analysis and expression consistent with the recommendations of the Forensic Science Regulator?78 Can you provide information about the forensic practitioner’s formal qualifications, training and experience? Can you provide the results of any proficiency testing undertaken by the forensic practitioner (and reviewer)?79 Have any of the forensic practitioners involved in the analysis or reporting been censured or criticised by an employer, court or other organisation in relation to their competence or credibility?80 Did the forensic practitioner consider alternative interpretations (or hypotheses) or undertake analysis of materials related to alternative explanations, particularly possibilities consistent with defence scenarios or non-guilt?81 What, if any, alternative hypotheses were considered? Why were they discounted or excluded?82 Can you provide the laboratory notes associated with the analysis and testing? These questions and related issues could hardly be characterised as other than real or information that a defendant ought to be entitled to. The questions and related issues are not, according to the NRC, PCAST and the Regulator, ‘alleged’ technical and methodological issues. In many cases, this information is necessary if a defendant intends to question a forensic practitioner, contest the evidence or evaluate it. Many of these issues align closely with CrimPD Part 19.5—‘factors that the court may take into account in determining the reliability of … expert scientific opinion’.83 What is accepted as a ‘real issue’ seems to be reflexively indexed to what observers know about the forensic sciences, validation, limitations and error rates, and so forth.84 The architects of SFR do not seem to appreciate just how ubiquitous real issues are. Their concern seems to be to eliminate challenges deemed spurious, yet they have not engaged with prominent scientific recommendations and do not appear to be in a position to properly determine what is ‘spurious’ and what is ‘real’. For those more conversant with the forensic sciences, real issues abound. To require those suspected or accused to provide reasons before the state provides sufficient information to understand the evidence seems unreasonable and inconsistent with fundamental criminal justice commitments and norms. D. Ascertaining Probative Value (in the Rational Tradition) The Crown Prosecution Service offers the following advice to prosecutors: Expert reports should spell out with precision its conclusion and the basis for them.85 Conclusions in reports with degrees of support for those conclusions should not be overstated.86 [Expert reports] should be robust, logical, transparent and balanced.87 These expectations, applicable to traditional expert reports (under Part 19), do not seem to be entirely consistent with the operation of SFR. But how are report users expected to rationally evaluate the probative value of findings that do not comply with these sorts of expectations (and those in Part 19)?88 Recall the recommendation of the NRC: ‘sufficient content should be provided to allow the nonscientist reader to understand what has been done and permit informed, unbiased scrutiny of the conclusion’. By ignoring these issues, the SFR regime accentuates prevailing problems with the legal reception of expert opinion evidence.89 As it stands, SFR encourages decision making in an information vacuum, and speculation and even irrationality. To the extent that defendants and their lawyers wish to challenge evidence, they must do so ‘blind’ or obtain funds for expert assistance—an additional hurdle that, once again, does not appear to be factored into the scheme and the claims about efficiency, costs savings and effectiveness. E. Greater Reliance on Trial Safeguards and Legal Personnel Where the results of the SFR1 are not accepted and the defendant intends to rely on uncertainties, limitations and the real chance of error, SFR places greater reliance on legal participants identifying and explaining issues in order to obtain additional information and, where appropriate, challenging the evidence at trial. While this might seem reasonable in the abstract, and might appear well suited to preventing challenges based on ‘alleged technicalities’, we should not overlook the historically weak performances of trial personnel in recognising and exposing the frailties in many forensic procedures—especially the feature comparison methods. In its review of expert evidence, the Law Commission concluded that: Cross-examination, the adduction of contrary expert evidence and judicial guidance at the end of the trial are currently assumed to provide sufficient safeguards in relation to expert evidence, by revealing to the jury factors adversely affecting reliability and weight. However, … it is doubtful whether these are valid assumptions. A more credible assumption, at least in relation to complex scientific or technical fields, is that juries will often defer to the expert providing the opinion.90 Legal personnel have typically struggled to incorporate the advice and insights of mainstream scientific and technical organisations into their consciousness and practice.91 The design and implementation of SFR seems emblematic in this regard. F. Expanding SFR and the Selection of Suitable Procedures? It is conceivable that other procedures, perhaps even more epistemologically frail than some already included within SFR, will simply be deemed appropriate for subsequent inclusion without any engagement with the underlying research or credible explanation. The inclusion of additional disciplines or evidence types (eg face and body mapping, voice and forensic gait comparison) might seem to offer efficiency dividends, but exemption from Part 19 requirements introduces risks if procedures have never been formally evaluated, regardless of whether they have previously been admitted and relied upon in criminal proceedings. Once procedures are included within SFR, limitations, uncertainties, errors and appropriate forms of expression, and even cognitive biases, become issues for impecunious defendants (and their lawyers) to identify. The defence must persuade decision makers of their evidentiary significance in the midst of adversarial proceedings. 7. Blind Efficiency: Evaluating SFR Our final substantial concern relates to claims about the efficiency and effectiveness of SFR. Assessing SFR is in reality quite complicated.92 Attempts to evaluate a new set of practices, intended to change the provision of forensic science evidence, should not assume that the forensic science evidence is robust—ie valid and scientifically reliable. To do so risks conflating responses—such as more early guilty pleas and fewer defence challenges—as successes when there may be alternative explanations. Proponents of SFR attribute success to the efficiency of the new scheme, when they have no idea about its impact on the quality of the evidence presented or the rectitude of outcomes. Proponents of SFR have made no attempt to engage with issues of quality, comprehension, defence resourcing, accuracy or their implications for fairness. This is not intended to suggest that SFR has been introduced without some attempt to identify benefits. The current national SFR scheme was developed from a previous initiative, DNA and Fingerprint Staged Reporting, by the Association of Chief Police Officers (ACPO) Criminal Justice Business Area, the Metropolitan Police Service and the CPS National Strategy and Policy Directorate.93 It was piloted in 2008–9 by the courts serving and including Woolwich Crown Court in respect of a limited number of evidence types (DNA, fingerprints and firearms).94 Following a ‘successful evaluation of this pilot’, SFR was extended across all London courts, with the exception of the Old Bailey, in November 2010.95 Assessment of the success or otherwise of the expansion was made on the basis of a review of cases from Wood Green Crown Court involving 158 pre-SFR cases (November 2009–July 2010) and 126 post-SFR cases (November 2010–July 2011). A review found: An increase in early guilty pleas from 61% to 84%. An increase in guilty pleas before trial from 71% to 87%. A reduction in discontinued cases from 18% to 5%. A reduction in additional requirements for forensic evidence from 42% to 2%. An increase in total guilty pleas from 79% to 91%. A reduction in cracked trials from 9% to 2.4%.96 The results of this pilot, used as a basis for the national expansion of SFR, might appear impressive on their face, but this was a small review of a single Crown Court. On closer examination, the results become slippery and perhaps unsettling. The reported results are focused exclusively on institutional efficiencies and related savings, with no apparent interest in the quality or accuracy of the forensic science evidence or whether the non-provision of ‘additional requirements’ was reasonable, and provide no means of determining the rectitude of pleas (or verdicts).97 There are no references to rectitude, reliability or fairness. Proponents substitute pleas, and other case management dividends, as proxies for effectiveness and efficiency without clear causal contributions. Does, for example, SFR reduce the number of discontinued cases? And, while guilty pleas might reasonably be equated with guilt, research has revealed that a non-trivial number of persons make false admissions. SFR may have heightened the risk of the innocent pleading guilty because of the increased difficulty of obtaining information and effectively challenging forensic science evidence.98 Almost all of the claimed benefits accrue to investigators, prosecutors, courts and judges. More guilty pleas and fewer sophisticated challenges might be used to imply that the forensic science evidence is robust and that challenges are generally spurious. As we have endeavoured to suggest, however, there are alternative explanations. More recently, West Yorkshire Police undertook a pilot extension of SFR across all biological casework.99 The scheme was revised with input from a number of scientifically trained forensic practitioners who appear to share some of our concerns. Revisions seem to be tacit recognition of shortcomings with the original scheme. While the four pro forma templates are retained, with the exception of the automated National DNA database intelligence match reports, the MG22(a)–(c) reports (including the SFR1 and SFR2) are only to be produced by forensic practitioners. Practitioners are directed to report information that might assist the defendant and, where appropriate, to include the relevant section(s) from the National Body Fluid Guidance Notes.100 Revisions place emphasis on the risk of, and importance of avoiding, exaggerated findings and findings that might be misinterpreted through ‘simplification’ or omission. There is explicit recognition of the importance of disclosing ‘any further information in the case which may undermine the Prosecution case or may assist the Defence’.101 Emphasis is placed on the professional responsibilities of the forensic practitioner and a proviso that, should the forensic practitioner believe that the SFR is open to misrepresentation, a full statement ‘should’ be authorised. Overall, there have been no attempts to evaluate SFR involving independent scientists reviewing the underlying procedures, participants’ proficiency, the accuracy of findings, or the ability of users to understand results and their limitations. Rather, institutional benefits and costs savings have been substituted for epistemic legitimacy and fairness. This is misguided. Early pleas, more guilty pleas and fewer challenges cannot confer epistemic legitimacy. Early guilty pleas and fewer challenges should flow from the state’s reliance on valid and scientifically reliable procedures, used by proficient forensic scientists and presented in empirically predicated forms most conducive to lay understanding. 8. Conclusion SFR requires defendants and their lawyers to respond to caricatures of forensic science evidence. In many cases SFR will make rational decisions about pleas, defences and challenges more difficult. It will fall most harshly upon the factually innocent, the poorly resourced and those poorly represented (including self-represented litigants). Any savings in these cases will be at the price of comprehension, transparency, accountability, fairness, rationality and potentially rectitude. Problems with forensic science evidence in legal settings revolve around misrepresentation, comprehension, and the dearth of technically and methodologically informed challenges. As we have endeavoured to explain, SFR does nothing to improve the quality of forensic science evidence, legal understanding of expertise and its limitations, the quality of defence challenges, the fairness of proceedings or the accuracy of verdicts. It has a tendency to obscure, blur and conceal under the auspices of efficiency. It operates as though forensic science procedures are valid and reliable, and that reported conclusions are correct. It provides no indication of what was done and provides no indication of limitations, uncertainties, problems, disagreement, exposure to biasing information and so forth. SFR reduces transparency and accountability (and potentially fairness and accuracy) by deeming conclusions presumptively reliable subject to a defendant identifying ‘issues’ that are considered to deserve a response. SFR may not prevent challenges, but they are likely to be more difficult (and less frequent) and more speculative. The introduction of SFR, particularly the use of the SFR1 and the potential for a responsive SFR2, is curious because in its dominant incarnation the scheme is incompatible with authoritative advice, scholarly research and independent reviews by scientists and lawyers. The reasons for the inclusion of some types of forensic science evidence within the scheme and the application of more demanding procedural rules and practice directions to other types of forensic science—the two ‘streams’—is neither explained nor indexed to scientific research and advice. Moreover, there are few reasons to believe that provision of a more detailed report would add significantly to the cost of reporting. SFR seems to be ideologically driven—based on complacent assumptions about the forensic sciences and the performance of criminal justice personnel and institutions. Inattentive to epistemology and fundamental fairness, the scheme appears to be primarily motivated by costs, institutional savings and the misguided contention that challenges to forensic science evidence are by and large without merit. This is curious, because recent reviews by scientific and technical organisations have explained that many forensic sciences, including some long-standing procedures incorporated within the first tranche of SFR, are much less robust than forensic practitioners have claimed and lawyers and courts have typically believed. SFR aims to reduce disagreement and controversy, and associated costs, through the provision of less information and shifting a burden onto defendants. Forensic friction is reduced through the provision of a summary—effectively, a ‘small target’. The state relies on non-disclosure, non-transparency, technical ignorance and the limited resources provided to defendants to encourage pleas of guilty or to reduce the number of challenges to forensic science evidence. The practical difficulty of challenging controvertible forensic science evidence is how pleas are obtained and how most of the costs savings seem to be achieved. This might reduce costs and the number of challenges, but is not the result of (more) reliable expert opinion evidence or pleas that are more trustworthy. Footnotes 1 National Streamlined Forensic Reporting Guidance, Section 2—SFR Guidelines for Providers of Forensic Science and a Practical Step Guide (2015) Version 3.0, 3: ‘SFR is a proportionate forensic evidence procedure’ accessed 1 June 2018. See also National Streamlined Forensic Reporting Guidance, Section 1—Supporting Information (2015) Version 3.0, 3 and ‘Streamlined Forensic Reporting (SFR) Communications Q & A’, 1. 2 See Oriola Sallavaci, ‘Streamlined Reporting of Forensic Evidence in England and Wales: Is It the Way Forward?’ (2016) 20 E&P 235. 3 Anthony Duff and others, The Trial on Trial (Hart Publishing 2004); John Jackson and Sarah Summers, The Internationalisation of Criminal Evidence: Beyond the Common Law and Civil Law Traditions (CUP 2012). 4 Law Commission, Expert Evidence in Criminal Proceedings in England and Wales (Law Com No 325, 2011) [1.8], [1.20]. 5 Law Commission (n 4) [1.2]; Forensic Science on Trial, Seventh Report (2004–2005) HC 96-1. 6 Law Commission (n 4) [9.11]. 7 Ministry of Justice, ‘The Government’s Response to the Law Commission Report: Expert Evidence in Criminal Proceedings in England and Wales (Law Com No 325)’ (2013) [3] accessed 1 June 2018. 8 Paul Roberts, ‘Making Forensic Science Fit for Justice’ (2017) 49 AJFS 502. 9 Originally CrimPR 33 and CrimPD 33A, respectively. 10 CrimPR 19.4(h). 11 Kristy Martire and Gary Edmond, ‘Rethinking Expert Opinion Evidence’ (2017) 40 MULR 967. 12 The SFR1 resembles Forensic Science Service shortened forensic reports, and these might be viewed as a precursor, though there are fundamental differences in the authorship and transparency. 13 National Streamlined Forensic Reporting Guidance, Section 1 (n 1) 5. 14 National Streamlined Forensic Reporting Guidance, Section 2 (n 1) 3. 15 CrimPR 19.3(2)(a)(ii) specifies ‘disputed issues’ as opposed to ‘real issues’. If issues are disputed on other grounds, then those grounds should be specified. 16 National Streamlined Forensic Reporting Guidance, Section 1 (n 1) 4 (italics added). 17 Section 4. 18 Section 6B. 19 Sections 6A–E. 20 Section 6A. 21 Sections 6B and 7. 22 Sections 5 and 6F. 23 Viscount Runciman, Royal Commission on Criminal Justice (HMSO 1993); Paul Roberts, Chris Wilmore and Gywnn Davis, The Role of Forensic Science Evidence in Criminal Proceedings, Royal Commission on Criminal Justice Research Study No 11 (HMSO 1993); Criminal Cases Review Commission accessed 1 June 2018. See also Brandon Garrett, Convicting the Innocent: Where Criminal Prosecutions Go Wrong (Harvard UP 2011); Stephen Goudge, Inquiry into Pediatric Forensic Pathology (Queen’s Printer 2008). 24 ‘Human factors’ are psychological and physiological factors that threaten forensic science practices and results. See Expert Working Group on Human Factors in Latent Print Analysis, Latent Print Examination and Human Factors: Improving the Practice through a Systems Approach (US Department of Commerce, National Institute of Standards and Technology 2012) (NIST Report). 25 Roberts (n 8). 26 Independence (from police) is an important issue and was first among the recommendations in the seminal National Research Council, Strengthening Forensic Science in the United States: A Path Forward (National Academies Press 2009) (NRC Report) 19, 78. 27 See eg the account by the co-chair of the NRC Committee: Judge Harry Edwards, ‘Solving the Problems That Plague the Forensic Science Community’ (2009) 50 Jurimetrics J 5, 7. 28 NRC Report (n 26); Lord Campbell, The Fingerprint Inquiry Report (APS Group Scotland 2011); NIST Report (n 24); President’s Council of Advisors on Science and Technology, Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods (2016) (PCAST Report); American Association for the Advancement of Science, Forensic Science Assessments: A Quality and Gap Analysis—Latent Fingerprint Examination (2017); Royal Society and Royal Society of Edinburgh, Forensic Gait Analysis: A Primer for Courts (2017); Royal Society and Royal Society of Edinburgh, Forensic DNA Analysis: A Primer for Courts (2017). For an overview of the reports and recommendations, see Gary Edmond, ‘What Lawyers Should Know about the Forensic “Sciences”’ (2015) 36 Adel L Rev 33. 29 On the Forensic Science Regulator, see accessed 1 June 2018. 30 Validated procedures are a requirement of UKAS accreditation to ISO 17025 and the Regulator’s ‘Codes’. 31 By way of example, the first ever validation studies for latent fingerprint comparison were only published in 2011. Five of seven feature comparison procedures reviewed by PCAST—namely, microscopic hair comparison, shoe prints, bite marks, complex DNA profiling and ballistics—were found to lack foundational validity. 32 DNA is something of an exception, as it has received considerable scrutiny since the early 1990s. In part this was because DNA profiling and the population genetics and statistics were not monopolised by relatively secretive state-employed forensic practitioners. See Jay Aronson, Genetic Witness: Science, Law, and Controversy in the Making of DNA Profiling (Rutgers UP 2007). 33 NIST Report (n 24); Itiel Dror, David Charlton and Ailsa Péron, ‘Contextual Information Renders Experts Vulnerable to Making Erroneous Identifications’ (2006) 156 Forensic Science International 74; Itiel Dror and Greg Hampikian, ‘Subjectivity and Bias in Forensic DNA Mixture Interpretation’ (2011) 51 Science & Justice 204; David Krane and others, ‘Sequential Unmasking: A Means of Minimizing Observer Effects in Forensic DNA Interpretation’ (2008) 53 Journal of Forensic Sciences 1006. 34 NRC Report (n 26) 186 (italics added); see also ibid 21–2: Recommendation 2. 35 See the discussions in Bryan Found and others, ‘Reporting on the Comparison and Interpretation of Pattern Evidence: Recommendations for Forensic Specialists’ (2012) 44 AJFS 193; Gary Edmond, Kristy Martire and Mehera San Roque, ‘Expert Reports and the Forensic Sciences’ (2017) 40 UNSWLJ 590. 36 See eg Susan Lang, Motherisk Hair Analysis: Independent Review (Ministry of the Attorney General 2015) 9. See also Judith Beaman, Harmful Impacts: The Reliance on Hair Testing in Child Protection, Report of the Motherisk Commission (Ministry of Attorney General 2018) 105ff. 37 NRC Report (n 26) 142, 184; NIST Report (n 24) 77. This, as the Scottish Fingerprint Inquiry makes clear, is an expression of opinion: SFI Report, 740 (Recommendation 1): ‘Fingerprint evidence should be recognised as opinion evidence, not fact, and those involved in the criminal justice system need to assess it as such on its merits.’ 38 Ian Evett and others, ‘Finding the Way Forward for Forensic Science in the US—A Commentary on the PCAST Report’ (2017) 278 Forensic Science International 16. 39 PCAST Report (n 28) 54. 40 Jonathan Koehler and Michael Saks, ‘Individualization Claims in Forensic Science: Still Unwarranted’ (2010) 75 Brook L Rev 1187. 41 See Simon Cole, ‘Individualization is Dead, Long Live Individualization: Reforms of Reporting Practices for Fingerprint Analysis in the United States’ (2014) 13 Law, Probability & Risk 117. 42 Sophie Carr and others, ‘Clarifying the Reliability Continuum and Testing its Limits: Biometric (Fingerprint and DNA) Expert Evidence’ in Paul Roberts and Michael Stockdale (eds), Forensic Science Evidence and Expert Witness Testimony: Reliability through Reform? (forthcoming). 43 The results of some procedures (eg simple DNA profiling) tend to be expressed in scientifically orthodox (ie statistical) terms whereas other results (eg latent fingerprints, ballistics and digital) are often expressed in ways that are inconsistent with mainstream scientific advice and logic. See CPS, Expert Evidence (2014, revised February 2015) 49 accessed 23 March 2018. 44 See the critical analysis of footwear, ballistics, microscopic hair and bite mark comparison in the PCAST Report (n 28). The President’s Council recommended against the use of, or further research into, microscopic hair comparison and bite mark comparison. 45 In consequence, the PCAST Report (n 28) 9–10, 87–104 questioned whether latent fingerprint comparison, notwithstanding its foundational validity, was valid in its routine application. 46 PCAST Report (n 28) 9. See also Forensic Science Regulator, Fingerprint Comparison FSR-C-128 (2017) 13, which instructs that fingerprint comparison validation processes should include ‘some form of measure of uncertainty/known error rate’. 47 They are not always raised in traditional reports and are unlikely to be addressed in the SFR2. 48 NRC Report (n 26) 85, 12, 53, 96, 109, 110. 49 CrimPR 1.1 ‘The overriding objective’. 50 CrimPR 3.2 ‘The duty of the court’. Prosecutors have special obligations, as ministers of justice, to make appropriate disclosure. See also Criminal Procedure and Investigations Act 1996, ss 3–7: Attorney General’s Guidelines on Disclosure (Attorney General’s Office 2013) accessed 23 March 2018; Disclosure—Experts’ Evidence, Case Management and Unused Material, May 2010: Guidance Booklet for Experts accessed 23 March 2018 [emphasis added]. 51 CrimPR 3.3 ‘The duty of the parties’, CrimPR 3.1(1)(a) and CrimPR 3.3(2)(c)(iii). Note that court-imposed costs orders lurk in the background as a disciplinary mechanism. 52 CrimPR 19.2(1)(a)(i)(ii) ‘Content of an expert’s report’. 53Disclosure—Experts’ Evidence (n 50). 54 The NRC Committee concluded that: ‘With the exception of nuclear DNA analysis, however, no forensic method has been rigorously shown to have the capacity to consistently, and with a high degree of certainty, demonstrate a connection between evidence and a specific individual or source’: NRC Report (n 26) 7–8, 87, 100, 128. 55 The Rt Hon. Lord Justice Leveson, ‘Expert Evidence in Criminal Courts—The Problem: Lecture to the Forensic Science Society’, 16 November 2010 accessed 1 June 2018; Lord Thomas CJ (Baron Thomas of Cwmgiedd), ‘The Future of Forensic Science in Criminal Trials’, 2014 Criminal Bar Association Kalisher Lecture [17] accessed 1 June 2018. On the question of whether procedural rules and practice directions govern admissibility, consider Tony Ward, ‘Expert Evidence and the Law Commission: Implementation without Legislation?’ [2013] Crim LR 561; Michael Stockdale and Adam Jackson, ‘Expert Evidence in Criminal Proceedings: Current Challenges and Opportunities’ (2016) 80 JCL 344. 56 National Streamlined Forensic Reporting Guidance, Section 1 (n 1) 4. 57 False confessions appear to be much more common than once assumed. See eg Richard Leo, ‘False Confessions: Causes, Consequences, and Implications’ (2009) 37 Journal of the American Academy of Psychiatry and the Law 332; Gisli Gudjonsson, The Psychology of Interrogations and Confessions: A Handbook (Wiley 2003). 58 In Australia, Victoria Police have generated a comprehensive set of detailed pro forma reports. See also the example of a pro forma fingerprint report developed by New South Wales police reproduced in Edmond, Martire and San Roque (n 35). 59 Consider the reliance on presumptive testing in the Birmingham Six prosecution, criticised in R v McIlkenney (1991) 93 Cr App R 298. Consider also the recommendations of an inquiry into a miscarriage of justice in Ministry of the Attorney General, Report of the Kaufman Commission on Proceedings Involving Guy Paul Morin (Queen’s Printer 1998) 328: ‘Evidence of a preliminary test … does not have sufficient probative value to justify its reception at a criminal trial as circumstantial evidence of guilt.’ 60 Judiciary of England and Wales, ‘Streamlined Forensic Reporting’ (March 2017) 12 Better Case Management (BCM) Newsletter 2. The forensic practitioner preparing the SFR1 ‘may not have been asked to conduct a full analysis of the exhibits and they may not have performed a detailed evaluation. Indeed, the SFR1 may be complete not by scientists or other experts, but instead by members of the police, or other staff, who enter the relevant findings onto the form.’ 61 Keith Findley and Michael Scott, ‘The Multiple Dimensions of Tunnel Vision in Criminal Cases’ [2006] Wis L Rev 291. 62 This is particularly pertinent to the current climate whereby toxicology results presented using an SFR1 (and various other forms of report/statement) have subsequently been found to be compromised. 63 Criminal Procedure and Investigations Act 1996. The obligation on prosecution experts to assist the prosecution in complying with this duty is set out in Disclosure—Experts’ Evidence (n 50), summarised by the key actions of ‘record, retain and reveal’. See also Gary Edmond and Andrew Roberts, ‘Procedural Fairness, the Criminal Trial and Forensic Science and Medicine’ (2011) 33 Syd LR 359. 64 The provision of more detailed reports has additional advantages. Ironically, the use of report templates may avoid the need for, and expense of, the defence consulting with or calling an expert. Reports referencing validation research, standards, error rates, controversies and limitations will help with evaluation, cross-examination and deciding whether to call a defence expert—to advise or as a witness. 65 National Streamlined Forensic Reporting Guidance, Section 1 (n 1) 5. 66 CPS (n 43) 17. 67Davie v Magistrates of Edinburgh [1953] SC 34, 40. See also Gary Edmond, ‘Forensic Science Evidence and the Conditions for Rational (Jury) Evaluation’ (2015) 39 MULR 77. 68Woolmington v DPP (1935) AC 462. In contrast to many areas where the Crown’s burdens or obligations are shifted, the forensic sciences present a domain where defendants will rarely be in a position to understand what was done or identify limitations intrinsic to the procedure. The defendant may have no advantage over the state. See Ministry of Justice, Swift and Sure Justice: The Government’s Plans for Reform of the Criminal Justice System (2012); National Streamlined Forensic Reporting Guidance, Section 2 (n 1) 12. 69 The authors of the SFR may not be bound by the rules ordinarily regulating experts and may be less accountable. 70 As ‘real issues’, these are different from concerns about ‘obtaining detailed forensic evidence where such evidence adds no value to the administration of justice’: National Streamlined Forensic Reporting Guidance, Section 1 (n 1) 4. 71 Marc Galanter, ‘Why the “haves” come out ahead: speculations on the limits of legal change’ (1974) 9 L & Soc’y Rev 95. The Crown has numerous advantages, and as a repeat player should be insisting on validating procedures in regular use and studying error rates and other limitations. Anecdotal reports suggest that the CPS can be reluctant to recognise ‘real issues’ and commission an SFR2. There are also accounts of enthusiasm among magistrates to allow the prosecutor to adduce the contents of an SFR1 which have not been accepted by the defendant. 72 PCAST Report (n 28) 50, 53; NRC Report (n 26) 122, 184. 73 Contrast the assurances of the National Policing Improvement Agency, ‘ODIS Optimising Detections in Science’ (2011) 1–2: ‘Innocent suspects should have nothing to fear from material not being revealed, provided they are aware of what is alleged against them. Indeed there will be occasions when it is to the advantage of the innocent person to be allowed to provide a full and uncontaminated account without knowledge of some of the material.’ However, ‘premature reference to forensic investigative material may provide the “guilty” suspect with an opportunity to fabricate an explanation …’ 74 See CPS (n 43) 19–20. 75 National Streamlined Forensic Reporting Guidance, Section 2 (n 1) 9 (italics added). 76 CPS (n 43) 35; see also 11–12. 77 CPS (n 43) 14: ‘In some cases, it may be necessary to limit the information given to the expert to avoid the risk of their conclusions being affected by confirmation bias …’ 78 Or ‘primers’ for the Court. See Royal Society and Royal Society of Edinburgh, Forensic Gait Analysis (n 28); Royal Society and Royal Society of Edinburgh, Forensic DNA Analysis (n 28). See also CPS (n 43) 15. 79 CPS (n 43) 27: this appears to include details of certification and accreditation, staff training and even materials that might appear to be protected by intellectual property law. 80 CPS (n 43) 27, 28; CPS, Disclosure Manual (revised February 2018) chs 36, 37. 81 CPS (n 43) 2, 23: ‘In common with other participants in criminal proceedings, experts have a duty to ensure that cases are dealt with justly. This requires that they: address the issues, including any alternative hypothesis …’ 82 Law Commission (n 4) [7.23]. 83 We note that in R v Adams [2007] EWCA Crim 3025, [14]–[15], a case regularly cited, concerns about costs did not prevent the court imposing an expectation that considerably more would be done to locate witnesses. Where a report is inadequate or in issue, there are few reasons to doubt that the court might expect sufficient information to understand the evidence and any limitations or non-trivial alternative explanations. 84 ‘Real issues’ are related to technical and scientific literacy. See Harry Collins and Richard Evans, Rethinking Expertise (University of Chicago Press 2007). 85 CPS (n 43) 23. 86 CPS (n 43) 35. 87 CPS (n 43) 20. Ordinary expert reports, that is. 88 Edmond, Martire and San Roque (n 35). 89 Consider the previous example, where a finger mark is positively ‘identified to’ a set of fingerprints held on the National Fingerprint database. A single latent fingerprint that is of poor quality and distorted (on the margins of sufficiency) is a different proposition to identification based on one or more high-quality prints. If the fingerprint examiner who performed the analysis had struggled with proficiency tests, had made mistakes or had been censured, these factors should form part of the information available to those trying to make sense of the evidence. In some circumstances, exposing the fingerprint examiner to suggestive, though gratuitous, information (such as police suspicions) may dramatically increase the risk of a misidentification. These are all real issues (and there are many more) associated with a foundationally valid procedure. Interestingly, issues associated with the reporting of fingerprint evidence and the equality of arms were critically reviewed by the Court of Appeal at about the time of the introduction of SFR in R v Smith [2011] EWCA Crim 1296. 90 Law Commission (n 4) [1.20]. See also R v J-LJ [2000] 2 SCR 600, [35]–[36]. 91 Contrast R v Otway [2011] EWCA Crim 3 with Gary Edmond and Emma Cunliffe, ‘Cinderella Story: The Social Production of a Forensic “Science”’ (2017) 106 Journal of Criminal Law & Criminology 219; Royal Society and Royal Society of Edinburgh, Forensic Gait Analysis (n 28). 92 Related problems with the lack of evaluation of concurrent evidence are discussed in Tony Ward and others, ‘Assessing Concurrent Expert Evidence’ (2018) 37 CJQ 344–66. 93 The ACPO is now the National Police Chiefs’ Council. 94 National Streamlined Forensic Reporting Guidance, Section 1 (n 1) Version 2, 7. 95 Sophie Carr and others, ‘Opening the Scientific Expert’s Black Box: “Critical Trust” as a Reformative principle in Criminal Evidence’ (2016) 80 JCL 364. 96 National Streamlined Forensic Reporting Guidance, Section 1 (n 1) Version 2, 9; House of Commons Science and Technology Committee, Forensic Science, Second Report of Session 2013–14, HC 610 accessed 1 June 2018. 97 Generally, see Theodore Porter, Trust in Numbers: The Pursuit of Objectivity in Science and Public Life (Princeton UP 1995). 98 Leo (n 57); Gudjonsson (n 57). 99 Biology Casework—Streamlined Forensic Reporting—Guidance Document (2017) . 100 A National Body Fluid Guidance document has been produced to provide an additional resource of generic ‘technical’ information associated with the interpretation of various body-fluid-based evidence types. It is anticipated that relevant sections from this will be reproduced within the SFR Stage 1 Report. See accessed 1 June 1018. 101 Biology Casework—Streamlined Forensic Reporting—Guidance Document (n 99); MG22(b). © The Author(s) 2018. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model) TI - Science Friction: Streamlined Forensic Reporting, Reliability and Justice JF - Oxford Journal of Legal Studies DO - 10.1093/ojls/gqy025 DA - 2018-12-01 UR - https://www.deepdyve.com/lp/oxford-university-press/science-friction-streamlined-forensic-reporting-reliability-and-zt3qWPPubT SP - 764 VL - 38 IS - 4 DP - DeepDyve ER -