Can ChatGPT Do Our Jobs Better Than Us? Not Yet.
Artificial intelligence and its impact on the practice of law is in the news again. Readers likely have heard about the attorneys that used ChatGPT, an artificial chatbot that synthesizes high volumes of data, to draft a legal brief that they submitted in a civil action in the U.S. District Court for the Southern District of New York. Unfortunately for these practitioners, ChatGPT cited multiple cases that did not exist, and the attorneys recently endured a sanctions hearing before the presiding district judge.
Whether artificial intelligence is friend or foe to lawyers is, like many legal questions, nuanced. We have found that AI platforms such as ChatGPT, if used properly, can be a helpful tool, particularly when compiling large amounts of data. But, for now at least, the talk of AI entirely replacing attorneys is overblown. Reader, don’t quit your day job.
We asked ChatGPT a series of questions about M&A and corporate governance litigation topics and assessed its knowledge of Delaware law. As you will see, when prompted within narrow and straightforward limitations, ChatGPT does a pretty good job answering questions. Where ChatGPT struggled more was to write a persuasive argument.
When asked to brief a seminal Court of Chancery case, Corwin v. KKR Financial Holdings, LLC, ChatGPT provided a summary that was largely accurate, and helpfully distilled the case.
When we asked ChatGPT to write a persuasive legal argument concerning a relatively complex topic, however, it faltered.
The argument was not necessarily bad; it was mostly accurate, if a bit rote and elementary. In particular, ChatGPT used broad legal principles that may apply, but it only scratched the surface of the relevant issues. An experienced human attorney likely knows that fiduciary duties interact with almost all stockholder vote scenarios, but ChatGPT did not explain how disinterested stockholders’ duties are implicated when the transaction is both beneficial to all shareholders and the voting stockholders personally (i.e., the entrenchment allegation). The heightened entire fairness standard may apply in the scenario presented, but ChatGPT did not argue why it should apply over the more deferential business judgment rule, and why its application would favor the stockholders bringing the post-close claim. ChatGPT’s best cite was to Zapata, where it acknowledged shareholders may bring post-close claims, but it stopped there. It did not analogize to a case where entrenchment was alleged.
If we were answering the question, we would have led with Corwin cleansing, the principle from the seminal case briefed above that a fully informed, uncoerced majority of a company’s disinterested stockholders’ approval of a transaction “cleanses” any alleged breaches of fiduciary duty and restores a lower business judgment review. And, because (unlike ChatGPT) we have access to and are familiar with the Court of Chancery’s recent decision in Edgio, discussed recently in this space, we would have argued also that Corwin cleansing’s application is limited to post-close damages claims only, not injunctive relief, as the prompt asked.
Of course, AI’s capabilities are limited to the data it has before it. The ChatGPT platform is limited to case law through 2021. Utilizing ChatGPT most effectively also requires work; one has to use trial and error to learn how to ask questions to generate a desired output.
But even if armed with all of these resources, it is dubious ChatGPT could—at least for now—draft a persuasive memorandum of law on par with an experienced practitioner. One would hope that all those long hours of motion drafting would lead not only to a body of knowledge, but advocacy expertise and skillful use of precedent.
AI likely will make our jobs as attorneys easier as the technology becomes more sophisticated. But our advice (or at least the advice of the human authors of this post) is that attorneys should use AI as a supplement, not a replacement.
This post is as of the posting date stated above. Sidley Austin LLP assumes no duty to update this post or post about any subsequent developments having a bearing on this post.
