Statisticians and Lawyers - Working Together?

What profession involves gathering evidence, determining what it might collectively mean and then making decisions based on that evidence according to both logic and an established code? I would contend that both the legal and statistical professions answer to this description.

However as a statistician who frequently works as an expert witness with lawyers, I am also aware of the perception of a gap. This gap was first illustrated to me when I was facilitating a workshop with senior judicial officers trying to determine their future needs. I thought I had reduced the technical aspects to a minimum but got pulled up by one of the participants, a Supreme Court Justice, who said "Slow down John, slow down. We are lawyers, we understand words, not numbers and diagrams and things!"

Another difference that is apparent from my experience is in the way that statisticians and lawyers are retained for their work. Both professions have a responsibility to give their clients objective advice, but many lawyers also have the role of being an advocate for their clients, presenting an argument that is almost by definition not balanced. Statisticians rarely have this advocacy role.

What are the similarities? We can discuss this in terms of evidence, assembling an argument, decision making and codes for doing the work. In each of these areas the different vocabularies used by lawyers and statisticians are a barrier, so the discussion must in part translate between two languages.

  • What a lawyer calls evidence a statistician might call data. In both cases it is information that is relevant to the questions at hand. Both professions are concerned with its quality and will use terms such as reliability and validity. Both are concerned with the impartiality of the sources. Clearly there is much that the two groups can agree on. Even the numerical aspect of a statistician's data is not very different from a lawyer's evidence - it is more about how each abstracts information.
  • There are significant differences in how each assembles an argument. For a start, a lawyer is often engaged to present a particular argument whereas a statistician would often say that their aim is to find what argument - it might be called a model - best fits the data. Both should use all the information available to them. It is probably in this respect that lawyers are often more flexible that statisticians. They will use a variety of sources (including common sense) and combine them as a verbal argument. A statistician is often more limited since formal statistical methods frequently cannot incorporate complex combinations of information.

  • Decision making is an area where the languages of the two professions almost meet. Lawyers talk of "beyond reasonable doubt" and "on the balance of the probabilities" when describing the basis of decision making in criminal and civil matters respectively, phrases that could easily come from a statistician

    In fact, when teaching hypothesis testing statisticians often use the legal metaphor of a criminal trial. For historical reasons, many statisticians are less comfortable with "the balance of the probabilities" approach but when working in business it is often most appropriate since a decision must be made, even when good information is lacking.

  • Codes of work provide a framework in which the above steps take place. For the lawyer these are typically the body of statutes, common law and court rules. While some of this appears arcane from the outside, it is generally a practical system developed over time to solve problems. Statisticians have a less well known body of standard practice, usually called rules of inference. These are much younger, with less than one century of history and there are still significant differences of opinion in how they should be applied.

The most frequent context where statisticians and lawyers work together is where the statistician is providing advice, often as an expert witness. For the statistician, this can be a challenging experience. While expert witnesses are usually treated with respect by the legal process on the assumption that they are there to help the Court rather than be partisan, the reality is that they are usually engaged by one side and that side hopes that the expert will be able to support their case.

The role of expert witnesses in the courts is similar across a number of professions. However sometimes the statistician's role is special since a statistician so naturally deals with issues of evidence [1]. For example, a statistician is often called upon to assess the significance of other evidence introduced in a case. There is a strong argument for statisticians being called on more frequently in this capacity.

A particularly well publicised series of recent cases in England related multiple SIDS deaths (Sudden Infant Death Syndrome) that had resulted in murder convictions. In one, a significant part of the evidence was the statement of a paediatrician that such cases would occur by chance only "once in over 73 million births". This calculation was based on an assumption that conditions such as SIDS did not have any genetic or environmental component that might make it more likely that a SIDS death occurs where there has already been one such case in the family. Statisticians refer to this assumption as independence. There is a large body of medical research that suggests that the assumption is wrong here - while the causes of SIDS are still a mystery there is strong indications that some of the contributing causes are genetic.

In the initial trial the "one in 73 million births" claim was not properly refuted although the presiding judge did give a warning that "we do not convict people in these courts on statistics. It would be a terrible day if that were so." However the subsequent conviction led to outcry over the misuse of statistics - to a statistician the mistake was so patently obvious. This led to a campaign for an appeal, including an open letter from the Royal Statistical Society.

The second appeal was successful, but it must be said more due to the discovery that a medical witness had withheld evidence helpful to the defence. However the Court of Appeal did recognise that the inappropriate use of statistics may have contributed to the original verdict [2].

From my experience as an expert witness in over twenty cases, I can relate to the difficulty that the Courts have with such cases. The legal community is generally not well equipped to handle statistical evidence and it requires great care in its presentation. In one case I was asked to give what amounted to a tutorial in Court - complete with whiteboard - to bring the Judge up to speed with probability concepts [3].

Language itself can become a major barrier. Statisticians, like most professionals, often use common words but give them special meanings. This can lead to serious confusion if a word such as "significant" is used without qualification. The reality is that the statistician needs to translate findings into common language in an honest but not condescending manner. Clarity is of greatest importance [4].

A challenge to lawyers is the way that statistics often displays a mathematical truth that sets it apart from many other areas. Expert witnesses are often asked for their considered opinion. What they will get from a statistician is often a statement that "if you assume this then the data implies that...". It can be a challenge for statisticians to firmly state their own interpretation of data.

Hence I would argue that lawyers and statisticians are natural partners in many situations. The commonalities of approach outweigh the differences and the application of the statistician's skills can be of great benefit to some legal situations.

Back to Top

John Henstridge
March 2005


Footnotes

[1] A good general reference for this is The Evolving Role of Statistical Assessments as Evidence in the Courts, ed. S. Fienberg, Springer, 1989.

[2] The transcript of the Appeal is worth reading. It gives insights to expert evidence in general as well as the lack of comfort that the judges had with statistical evidence.

[3] By contrast I had one memorable experience when a Judge interrupted the barrister cross examining me, saying "excuse me Mr. XX, may I ask Dr Henstridge a question? Dr Henstridge, when you mentioned in answer to the previous question that there was a linear relationship, did you mean something of the form y=ax+b where a and b are constants?" I could only answer "yes Your Honour" and Mr. XX gave up cross examining there and then, perhaps realising that the Judge was well ahead of him! This experience has never been repeated.

[4] Early in my career as an expert witness I was once cross examined for a day over my non-use of "Fisher's exact test". It was difficult to explain that the test was not necessarily exact but it simply used an exact calculation of a probability, and that the calculation assumed certain constraints on the data that did not hold in the context in question. The word "exact" had taken a life of its own.