WATCH Joseph S. Fulda, CSE, PhD 701 VEst 177th Street #21, New l~rk, N Y lO033 htV,://www.cds .o,g/eight.html Copyright © 1999,Joseph S. Fulda Field Research in Artificial Intelligence and Law: A Case Study A recent research report on the development of an expert system for jury selection [1], that is for the deselection of grand panelists for service on the jury by using peremptory strikes (bizarrely called "preemptive strikes" throughout the report), is more interesting for the asides it provides on field research in AI and law than for its research results. (This is not to say that the research itself is not important; it is. It is an example of use of AI and decision-theoretic (statistical) techniques to predict human behavior, a general focus that will no doubt increase in importance as more algorithmic tasks are solved.) Going into the research, it was decided to start with some older published guidelines for jury selection of doubtful validity as well as rules of thumb suggested in interviews by practicing attorneys, also of dubious validity. There had not been, it: is claimed, any scientific studies of jury selection methods and their relative success and jury consultants, it is further claimed, are not retained on the basis of their track records, but on the basis of their renown, which is unrelated to prior successes and failures. The purpose of the research was to see how the expert system could learn from field experience to use rules of greater validity and to discard those shown to lack validity. For this to be possible, the co/Speration of the judges, litigants, attorneys, and courthouse staff would all prove necessary. The finished product, once it becomes sufficiently refined to be marketed as a product--a goal to which the author is still committed, should be highly sought after by litigants and their counsel to gain advantage and by the Court to ensure that neither side gains advantage because of jury composition. Yet, there was no shortage of impediments to the development of such a product and its validation. What follows is a brief synopsis of the difficulties work in the field presented the researchers: (1) The Judges. With the strong support of the Chief Judge, the federal district judges voted to allow full support of this research, with both parties to each case having access to the system. What happened: A number of cases were appealed to the U.S. Court of Appeals for the Fifth Circuit, arguing inter alia, improper jury selection, and the judges, fearful of reversal, refused to allow anyone access to the jury pool until after cases were decided. The system was hence to be used as a work-product of one side only, a 50% reduction in data. (2) The Litigants. With the strong support of a well-qualified domain specialist, the system was prepared for use in each of her cases. What happened: 96% of civil cases in that jurisdiction were settled by the litigants and the domain specialist's docket was no exception. One case that did go to trial ended up being a bench trial. Over a year passed with no progress for these reasons. Selection of cases that had proceeded to the trial stage was required, even though other attorneys without the domain specialist's interest and expertise in the area were involved. (3) The Attorneys. What happened: Some attorneys didn't use the system well; others conducting exit interviews with jurors concentrated on questions concerning their effectiveness to the exclusion of questions related to the project; still other attorneys--all of them, in fact-who took a mini-course to meet continuing legal education requirements refused to volunteer to use the system, even though it was offered pro bono. (The offer was made to the plaintiffs' bar so that the system would be used by plaintiffs as well as by the expert--a defense lawyer.) (4) The Courthouse. What happened: Threats on the life of one of the judges caused the implementation of security procedures by courthouse staff that significantly delayed the process. The conclusion is inescapable: Field research is just as frustrating and requires just as much dedication in artificial intelligence work as in any of the natural or social sciences. And, one can learn almost as much from what goes wrong and how the research team overcame their difficulties as from what went according to plan. • [1] Roy Lachman, "AI, Decision Scicnce, and Psychological Theory in Decisions about People: A Casc Study in Jury Selection," A1 Magazine 19(Spring 1998): (1)111-129. Note: Due to a production error in last quarter's AI Watch, the reference to A. K. Joshi's IJCAI-97 Research ExcellenceAward Acceptance l.ecture was omitted: It is Aravind K. Joshi, "Relationsbip between Natural Language Processingand AI: Role of Constrained Formal-Computational Systems,"A I Magazine 19(Fall 1998): (3)95-107. AI Watch solicits authors of technical papers in AI which reflect a perspectiveon cognitive processes,social processes,or philosophy of AI to submit reprints (or actual copies; no electronic submissions, manuscripts, or typescripts) to Dr. Joseph S. Fulda; 701 West 177th Street, #21; New York, NY 10033; fulda@acm.org.The reviewswill be technical and at most eight papers will be reviewedp.a. Computers and Society, March 1999

/lp/association-for-computing-machinery/al-watch-field-research-in-artificial-intelligence-and-law-a-case-zYsvL3Qzr5