AI and GenAI in LegalTech

Potential Applications of GenAI
Let’s take a look at some examples of new ways in which LLMs can be used to help lawyers with daily activities:

• Legal Document Generation
  There have been recent attention-grabbing headlines with lawyers opting to use ChatGPT, or similar LLMs, to generate court briefs and filings, that unfortunately failed miserably. Their motivation is understandable as generating briefs and filings is often a rote task ripe for automation, which would afford lawyers with more time to focus on more complex tasks, if not simply amplify their current case load. In-house counsel may similarly look to LLMs to help automate corporate contract creation or verification.
  
  On the surface, these types of tasks should be well suited for the current iteration of Generative AI – most briefs, filings and contracts re-use existing content from previously generated work product – and if any additional creativity is required it can always be added after the AI has generated its base product.
  The notable failings that have been subject to much criticism from the legal community and judges come from the so-called “algorithmic serendipity”. This is the ability of Generative AI to “hallucinate” and produce results that do not actually exist. This kind of creativity, while laudable in some fields, creates issues in a field that relies on true verifiable precedent, such as law. Alarmingly, some studies have found that when it comes to issues of law, LLMs hallucinate in as much as 70% of the cases, most of the time creating citations to non-existing cases or citing non-existing regulations.
  This does not mean LLMs cannot be used for the purpose of generating legal documents. They can, and can be reasonably expected to significantly simplify and streamline the process, but LLMs should not be used without verifying every source and citation.

• Q&A Chat Bots
  One of the most impressive aspects of LLMs is, unsurprisingly, their ability to generate interactions that feel natural and conversational as opposed to scripted. LLMs are able to interpret natural language queries very effectively, often providing a direct response to the query along with supporting context, while in some cases correcting typos and flagging unspoken assumptions in the original query.
  
  This functionality can be used to great effect to familiarize employees with company regulations by looking up provisions from Code of Conduct or other applicable internal rulebooks, and can also assist with legal document drafting by referencing provisions from applicable state and federal regulations.
  
  As a preventative measure, using LLMs to support internal chat bots can help educate employees on relevant regulations to mitigate potential misbehavior, and if subject to government investigations, may demonstrate to government officials that the company was not negligent with taking available measures to inform employees on relevant legal requirements. Although this aspect of AI usage can also be subject to hallucinations, as described in the previous segment, hallucinations should occur less frequently if used to support chat bots since this training would be based on smaller documents sets, which would be more narrowly tailored to the task at hand. Furthermore, this training can be expanded through reinforcement learning, which is a process whereby answers are validated against existing material to ensure their accuracy.

• Generate Rapid Insights for Investigative Purposes
  The ability of LLMs to generate content allows them to go one step further than previous AI models, which were already frequently used in investigative contexts to provide on-the-fly document summaries, identify facts related to custodians, and construct fact-based timelines. Considering that LLMs are pre-trained and can be deployed on document sets right away, thereby foregoing traditional model training, AI can significantly expedite fact-finding tasks by enabling investigators to dig into a case immediately after deploying the LLM. Given that time can be of utmost importance with investigations, the time-savings alone make the use of LLMs worth considering over traditional approaches.

• TAR X.0 / LLMs – Review Prioritization and Classification
  So far, a lot of attention has been paid to leveraging the generative capabilities of LLMs, but what about a more traditional approach when the model is required to enhance the document review process by prioritizing data based on relevance? Or by identifying a document review cut-off point, at which most relevant content has already been reviewed? Should eDiscovery practitioners still rely on older models like LSI or SVM, or should they consider migrating completely to using LLMs? The answer here is not very straightforward. There are pros and cons to all models and LLMs are no different.
  
  We have already discussed several benefits of using LLMs, and another notable addition would be LLMs ability to provide multiple label classification. In other words, step away from the binary classification of TAR 1.0 and TAR 2.0 (relevant or not relevant), and classify data based on multiple tags simultaneously. This can be particularly helpful when it comes to issue tags and issues classification. In some cases, the pre-trained nature of LLMs can also allow one to completely forgo initial model calibration and identify likely relevant material before any review takes place, saving valuable time.
  
  An example workflow might involve asking your LLM via a prompt to identify documents relevant to a particular question or subject (which can correspond directly to items from the RFP – although some prompt engineering might be necessary) and having LLM suggest the documents that might be responsive to that prompt. This can go one step further and have the same model identify the segment within each document that directly relates to the query fed into the model, and it can be done for multiple prompts at the same time. The reviewers can then focus on those suggestions to prioritize their initial review process, and add additional prompts or use other methods to pull in additional documents as necessary.
  
  That said, non-LLM models still have their place in your toolkit especially when it comes to review of highly specialized data that may not overlap with LLMs trained on general language data, or for instances in which non-LLM models are trained and calibrated specifically for a particular type of data. In such cases, it is likely that non-LLM models will produce higher levels of precision than LLM equivalents.
  
  Additionally, when it comes to audit requirements, the complexity of an LLM’s architecture may present some difficulty with substantiating why a particular document was labeled a certain way. In this manner, older models can provide more transparency into its AI decision making than current LLM models.
  
  Last but not least, many of the available LLMs have a token limit in terms of how many characters per document they would consider in their analysis. If a document happens to exceed the token limit (and long documents are, unfortunately, an all-too-frequent sight in the legal field) – quality of AI analysis might deteriorate.