Experiments on Bias in Patent Litigation OR Does Everyone Hate NPEs?

Lisa has written about the importance of experiments in patents, and I agree. I read about a really good one today. Bernard Chao (Denver Law) and one of his students, Roderick O'Dorisio, conducted an experiment to simultaneously test whether there is a bias against patentees sued for declaratory relief of non-infringement and against NPEs. To do so, they made identical patent vignettes used to resolve a close, but simple, infringement case. The only differences in the videos shown to the subjects were whether the defendant sued first and whether the plaintiff was an NPE (and in one, both were true). The abstract his here, for the paper forthcoming in the Federal Circuit Bar Journal:

Although everyone believes that telling a good story is an important part of jury persuasion, attorneys inevitably rely on their intuition to choose their stories. Experimental methodologies now allow us to test how effective these stories are. In this article, we rigorously test how two different narratives common to patent law affect mock jurors. First, we look at whether accused infringers can improve their chances of prevailing by being the aggressor. Prior studies have observed that accused infringers that file declaratory judgment actions to vindicate their rights win more often than those that are sued by patent holders. However, these results may simply be an artifact of the selection effects. For example accused infringers may simply be suing on stronger cases. To date, no studies have tried to control for these selection effects and determine whether it is truly the story that sways juries. Second, we looked at whether an accused infringer can influence mock jurors by making a few disparaging remarks about one kind of patentee’s business model, the non-practicing entity (NPE). NPEs, often pejoratively called patent trolls, may have a more difficult time prevailing at trial than practicing entities do.

To test how these narratives affect potential juries, we used a 2x2 between-subjects online experiment. We randomly assigned virtual mock jurors to watch one of four different scenarios of an abbreviated patent trial and render verdicts. The results showed that accused infringers that filed declaratory judgment actions prevailed more often than those where the patentee initiated the lawsuit. In addition, our study found that NPEs won less often than practicing entities. We discuss implications for strategy and policy.

The results are pretty clear - there were marked differences in favor of those who sued first and in favor of those sued by NPEs. And for the group that is both NPEs sued for declaratory relief, the numbers are the lowest of all. I consider this to be a validating check on the findings for each of the individual treatments (though more on that later, as statistically it is not so clear).

As the title of this post implies, there are a couple of ways to read this data. The results here may show an implicit bias against NPEs. Or, NPEs may be the baseline, and it shows a preference for practicing entities. The highest win rate was 39%, so it is not like the plaintiffs were running away with victory here. Or, it may show that taking the bull by the horns is rewarded - patentees prefer defendants who assert their "rights" to defend against infringement.

Nonetheless, the results are a bit shocking - a product making plaintiff was more than twice as likely to win than an NPE sued for declaratory judgment of non-infringement on identical facts and presentations. This makes me think that we have to talk about more than patent quality when we talk about low NPE win rates.

About the statistics: the Declaratory Relief effect was significant at p<.1 (and at p<.05 if you included demographics). The NPE effect was significant at p<.01. Interesting, despite the marked drop for both combined, when the entire model was tested, including the interaction of declaratory relief and NPE, then none of the treatments was statistically significant. This result is difficult to interpret, but my sense from eyeballing the data is that the NPE effect is doing most of the work in the combined model, and so combining the DJ effect with it confounds the model.

A final note on methodology - the authors use Mechanical Turk, and cite to literature that such users are reliable for research like this. They also use some techniques to ensure attention. Finally, if there are attention issues, it is unclear why they would affect one category more than any other. Nonetheless, to the extent that one is skeptical of mTurk, one might be skeptical of the results here.