3.

A general approach for predicting the behavior of the Supreme Court of the United States

A general approach for predicting the

behavior of the Supreme Court of the
United States
Yu Chen, 03722670

Summary
A generalized, consistent and out-of-sample applicable model based on time-evolving
random forest classifier is developed to predict the decisions (case outcome, justice vote) of
the Supreme Court of the United States
The model achieves 70.2% accuracy at the case outcome level, 71.9% at the justice vote level
(1816-2015), and outperforms a baseline model by 5% over the past century
The model can be applied out-of-sample to the entire past and future of the Court
representing an important advance in the filed of quantitative legal prediction

Three positive aspects

The model being able to predict judicial decisions can potentially produce significant financial
rewards to publicly traded companies
The model can serve as a strong baseline against which future science in the field of judicial
prediction might be cast, and stand the test of time across many justices and many distinct
social, political and economic periods
The authors' approach (ensemble) of developing the judicial prediction model can inspire a
number of future advancements in filed of legal informatics achieving

Three criticisms
Although the model is able to achieve a promising prediction accuracy in both case outcome
and justice vote, it still relies on experts to explain how the decision is made
The performance of the author's model only outperforms one in-sample optimized null
model by nearly 5%, so we need more comparison with judicial prediction models that use
advanced machine learning technology such as deep neural networks
The concluded data in the article is not visualized as graphs making difficulties for readers to
draw their own conclusion.

Three questions to the author

Is it possible to build and train the legal behavior prediction model based upon an advanced
deep learning model such as transformers?
Can you regroup the experimental results as plots or graphs to make the conclusion more
direct?
Can the robustness of the model be discussed and tested by some experiments?