In criminal cases in the United States prosecutors often add a lot of charges for a defendant to have ammunition for coercing the defendant into a plea deal. This is toxic because the defendant doesn’t know which of those charges are likely to hold up in court if the case would be decided by a jury. Given that there’s little cost to the prosecutor for adding additional charges, defendants are often overcharged. I propose that whenever a prosecutor files a charge for a defendant, the prosecutor should state the likelihood that in the absence of a deal the court will find the defendant guilty of the charge. The ability of the prosecutor to accurately access the likelihood can be measured via the Briers score or a Log score. The current score should be publicly accessible on the website of the court. This allows the defendant to know whether they can trust the likelihood values the prosecutor gives. The score should also be printed on ballots when the prosecutor seeks reelection to create much higher incentives for the prosecutor to give the correct likelihood then convicting a lot of people. After the prosecutor provides the likelihood for the charges it’s much easier for a defendant to make a good decision about whether taken a given plea deal is in their interest. Prosecutors with a good Briers score will be able to make more plea deals to reduce their overall workload because it’s easier for the defendant to know that a deal is in their interest. While this reform wouldn’t fix all problems with plea deals, as some plea deals are due to the defendant being given charges that would actually hold up in court given the existing criminal code, the reform will provide defendants with fairer plea deals. Defendants getting fair plea deals is good for the system given that it keeps overall legal costs down. I would expect that many juries will automatically throw out a 10% or 20% charges because there will be people on the jury who would argue that the prosecutor thinking that there’s only a 20% chance that a charge holds means that there’s reasonable doubt against the charge. This reform is both in the interests of citizens who care about law and order and citizens who care about reducing sentencing overall as everybody should be interested in prosecutors providing fair plea deals.
Interesting idea. I think that rather than a Brier score or log-score (which just indicates how accurate the prosecutor was) it would be better to use something that also indicates the extent to which they specifically over-stated charges. Maybe just something like "On average, the total sentence for defendants charged by this prosecutor was 4.7% of the total of all charges made", though it seems like there would be all sorts of difficulties actually doing that in a way that makes sense.
Comment
I think that accuracy is better for a prosecutor to optimize then not overstating the likelihood. If I’m charged with a crime and the prosecutor tells me that the real chance of getting put into jail is 60% when it’s actually 90% then I’m harmed by having more hopes then reasonable about being able to stay out of jail. Besides the main number, all the history should however be available so that a defense lawyer could also calculate whether a prosecutor with a bad Briers score under or overcharges. In addition to accuracy being useful to optimize for, I expect it to be easier to find a political consensus around "prosecutor should be accurate when making charges" as that’s desirable for both the right and left.
Comment
I think you want to give prosecutors incentives both to be accurate and not to overstate, and also to give people accused of crimes some indication of what they’re actually likely to end up with if they go to trial. You might be right that doing something that looks "bidirectional" might be easier politically.
For something that’s impossible to implement, this seems like a surprisingly small step toward EITHER futarchy and the embedding of good conditional predictions in more policy and behavioral decisions, OR to resolving the broken incentives of prosecutors (convictions, not truth).
Comment
Chance of being convicted is very weakly correlated with actual guilt, or even procecutor knowledge of guilt. Consider the case of an illegal search that definitively proves guilt, but cannot be presented at trial. The prosecution would probably want to proceed with their 10% case, since they know the defendent is guilty, even though the jury won’t have key information.
Also, too, predicting outcomes is a skill separate from ability to generate outcomes. The Tails Come Apart, Goodheart, and all that..
Also, also, too almost no prosecutors are directly elected, judging people by any raw metric is wierd and not done—there would be some arbitrary cutoffs added to translate it to a quantified grading scale, and, as other commenters said, the root cause here is way deeper than "the DA charges 17 counts of assault, one for each blow, plus 3 Kidnapping, one for each grab, etc. Etc. Etc.
Comment
An alternative approach that feels less gameable:Every time a plea bargain is reached, there is a small chance (1 in 1000?) that we randomly select the case for review. The defendant is imprisoned as per the terms of the plea bargain, but then we also bring the case to trial (federally funding a defense).If the review trial finds the defendant guilty of the charges they pled to, we say ‘okay, seems like the plea bargain was fair, the defendant would have been found guilty anyway’. If the review trial finds the defendant innocent, we release them and publicize this fact. If multiple reviews under the same prosecutor exonerate the defendant, it starts to look like that prosecutor is threatening charges to coerce defendants into unfavorable plea bargains. A 1 in 1000 chance means that the cost of this should not grow too excessive compared to existing costs of the legal system, but also means that a prosecutor who is habitually coercing defendants into taking unfavorable plea deals is very likely to get caught.
Comment
It seems to me like your proposed system is more gameable by influencing the people who do the review and who select which case to randomly review then the system I proposed. Furthermore, it does have no direct impact on the decision making process of the prosecutor when pressing charges.
Comment
You only score the predictions that actually have the court decide the outcome or that have the prosecutor drop the charges without a deal and not those that get decided by plea deals.
Comment
Suppose that the prosecutor has some random noise in their charges, such that they sometimes overcharge a bunch and sometimes undercharge a bunch. In that case it seems reasonable to suppose that things are more likely to go to court when the prosecutor is overcharging and the accused therefore thinks they can get more of the accusations dropped. But this would mean that the prosecutors are evaluated on a subset of the charges that are systematically too high, and therefore to compensate they end up lowering their assessed probabilities below what is actually counterfactually accurate if people just went to court about it always. I don’t know how big a problem this would be, but it seems like something that would be good to evaluate in the proposal.
Comment
So then nobody would assign low likelihood charges? I guess that’s a feature of the proposed system, but it seems like a flaw to me. (Probably minor)
Comment
I do think that the legal standard of reasonable doubt means not assigning low likelihood charges. That’s how I understand the principle of "reasonable doubt". The feature of this system it that it lets the jurors decide what they think reasonable doubt means and the system doesn’t set the meaning for it.
Comment
Yeah, I agree with that. But 20% seems within that range for me (especially for very bad crimes). Is 10%? Not sure.
Comment
The system as I proposed by design doesn’t say "Prosecutors are only allowed to bring charges with at least X%" but works to let an equilibrium play out. I think this feature is very neat because it’s a mechanism that doesn’t work through bureaucratic rules and arbitrary cutoffs but through jurors making decisions about what’s right in individual cases and the prosecutors learning from the judgements the jurors make.
What’s to stop the prosecutor from lying about their Briers score?
Comment
It would be a legal obligation for a prosecutor when making a charge to publically enter the charge along with the likelihood into a database. Making errors with filing charges will be problematic for the prosecutor the same way it’s problematic now, it can make a court throw out the charges on procedural grounds. Then when an election comes around the election commission has the job to calculate the Briers score to put it on the ballot. The job of administrating the database could be done be done by the court where the charges get filled, so that it’s outside of the realm that the prosecutor can influence.
If upfront assessments are provided, I expect the defense bar would gleefully keep track of such things.
They already informally track the behavior of the DA’s offices they deal with. They’re extremely organized in some areas and in near-constant communication with one another.
How would prosecutors be trained and incentivized to make and report accurate predictions? Who would be funding the defense attorneys, especially overworked public defenders, to learn to properly evaluate prosecutor’s claims? The US legal system already isn’t exactly known for being honest today even in terms of things like rules about disclosing exculpatory evidence to the defense. Also, the famous saying about indicting a ham sandwich is anecdotal evidence about how much credit (grand) jurors give prosecutors’ likelihood assessments in a pre-trial, non-adversarial environment, and I don’t find it encouraging. Also, I understand that one of the big issues with plea deals is that it is often in a defendent’s individual interest to plead guilty to a lesser charge they’re innocent of, even if they’re pretty sure they wouldn’t be found guilty at trial, either because they’re risk averse (not necessarily unreasonable when years of your life are on the line), or because the agreed-on punishment will be less than the time in jail, and cost in fees, they’d incur just by going to trial even if exonerated. Data on likelihood of conviction wouldn’t help with that. At a more basic level, I think this solves the wrong problem. If, as you say, it is in society’s best interests to keep legal costs low, then it follows that there is little benefit to society from the larger penalties imposed by overcharging defendants. This suggests that the problem lies in the criminal code and sentencing guidelines, and that these need serious updating. And given how biased our legal system is and has historically been, I believe a proposal based on conviction odds has basically the same problems as current AI tools for criminal sentencing and risk assessment. Large racial and economic biases, among other problems. Edit to add: I do like the idea of giving defendents honest info about their likelihood of conviction if a case goes to trial. I’m extremely skeptical of the idea of having prosecutors provide that evidence, and wary of letting prosecutors, juries, or judges see it.
Comment
Comment
Comment
Comment
Comment
Comment
On those points I completely agree.
While I think there are some flaws with this proposal, my biggest question is "Why not do other things that are done abroad and we know work?"
The US system for deals is really strange. In the UK and many european countries there’s no such thing as a deal. The police can charge you with a crime. The earlier in the process you admit guilt, the bigger % a sentence reduction you get. This seems to solve the overcharging problem, a prosecutor can’t charge you with crime X and then give you a deal for crime Y, and aligns incentives better as prosecutors can’t fear people into taking plea deals as easily. It also preserves much of the benefit of plea bargaining by still allowing suspects to confess when the evidence is overwhelming and hence save lot’s of time and money.
Comment
In Germany where I live a prosecutor has a legal obligation for prosecuting cases of crimes if he believes the defendent is guilty. A prosecutor is not allowed to say "Well this law is stupid so I won’t prosecute people for breaking it" while in the US that’s within the rights of a prosecutor. This right allows for plea deals to happen in the US. If you change this basic part about what the mandate of the prosecutor happens to be, you will suddenly get a lot of very bad laws enforced that currently aren’t enforced. If laws against oral sex in some states suddenly get enforced that will create pressure to get rid of those laws but it’s won’t be pretty. But even if the US would get rid of their current legal system and switch to the German one (which is obviously much better thought out and superior ;) ) and have a culture change to make American bureacrats as law abiding as German one’s there’s still a process whereby plea bargining could cut down on legal costs and time of the trial. There’s a benefit in getting a verdict in a few days instead of years through plea bargaining. In many situation a plea deal is about more then just getting the person to admit that they are guilty but also to reveal certain other information that’s of interest to the prosecutor. Being able to make a deal to get the information can be useful for the government.
Another idea: as things stand today, prosecutors are only allowed to file charges when they have "probable cause". Courts won’t put a number on what that means, but you might reasonably approximate it as 25% certainty that the defendant did it. We could insist on a higher standard—preponderance of the evidence (which courts do define as > 50% certainty) or clear and convincing evidence (which courts won’t put a number on but you could reasonably approximate as 75% certainty).
Comment
As I wrote, I don’t think such a number should be set directly by law. A judge can however take the number into account when deciding whether there’s "probable cause". Individual jurors might also decide to categorically treat charges with are under some number as baseless charges. A voter might decide that he doesn’t want to reelect prosecutors that fill charges with <50% or <75% certainty. Prosecutors who fill charges that they believe to be low probability can be judged for that behavior democratically and different voters can disagree with each other what standards they have for prosecutors. Having new dimensions based on which voters can evaluate prosecutors and thus incentvise different behavior for prosecutors then just a maximum of convictions is part of the point.
I think you are forgetting a major player in the criminal justice system: the defense attorney. The defense attorney already has both the expertise and the incentive to accurately advise the defendant as to the likelihood of a conviction on each charge.
Comment
The defense attorney doesn’t have access to all the evidence that the prosecutor has. This proposal essentially forces the prosecutor to state at the beginning how strong his evidence is. Having common knowledge about what the prosecutor believes about the likelihood that his charges stick is very useful for the defense attorney when it comes to negotiating a plea deal. Being able to argue in court "Even the prosecutor only believes that there’s a 10% chance that this charge sticks" is a valuable move even if the defense attorney knows that there’s a 10% chance.
Comment
That’s not a bug, that’s a feature! The prosecutor knows what evidence the prosecutor has, but the defendant knows whether he did the crime. We want the defendant to make the plea decision blind to the strength of the prosecutor’s evidence, because guilty defendants will guess that the prosecutor has strong evidence and plead guilty (even if that guess is wrong), and innocent defendants will guess that the prosecutor has weak evidence and proceed to trial. This is how we want the system to work.
Comment
I don’t think the intention of the plea deal system should be to get people who wouldn’t be found guilty in a court of law to plead guilty for any reason. I think subverting the right to a trial by jury by creating a system that encourages people to plead guilty who the jury wouldn’t find guilty is bad. Overall I think the defendant has to little power in the US system and moving the system to give them more powers is good even if that occasionally means guilty people go free.