THE LAW & eDISCOVERY
AN INTERVIEW
Thanks for taking the time to discuss with F’inn how AI is impacting various fields, this time focusing on law. Joining us are two colleagues from Tanenholz & Marr, a law firm specializing in discovery and document review, Founder & Managing Partner David Tanenholz and newly hired Counsel & Director of Legal Technology and AI Brandon Mack. Great timing and thanks for talking with us!
Greg Robison (GR): First topic, tell us a little more about your firm and what the day-to-day tasks of your associates are like.
David Tanenholz (DT): We are an eDiscovery law firm. Our firm specializes in assisting clients with the collection and legal review of documents in connection with litigation document requests, subpoena responses, and other business and compliance needs. A critical part of our document review process involves leveraging software and technologies to sort, prioritize, cull, and otherwise organize our client’s data for efficient and accurate legal review at the lowest feasible cost. Some of the existing eDiscovery tools use AI and AI-like technologies to identify similar documents and to propagate attorney coding decisions to unreviewed documents. For our associates, these tools are leveraged to create custom workflows that allow them to analyze similar documents together for more efficient review.
GR: How comfortable do each of you feel with AI?
DT: The applications in the eDiscovery field are ever-evolving. While I feel comfortable with the theory behind them, it is something you need hands on experience with before you can responsibly offer as a service to clients. That’s why we were so excited to bring on Brandon, who has both theoretical knowledge and practical experience using these tools.
Brandon Mack (BM): When it comes to operating these new analytics tools, there will obviously be a learning curve. However, extensive experience with multiple generations of prior analytics deployments in the eDiscovery industry gives me confidence that we will become as effective with GenAI as we are with other forms of analytics.
GR: What are the key AI tools you use personally?
DT: Our firm has used predictive coding, computer assisted review, and other advanced analytical tools in our document reviews. Those are based on similar concepts as the newer AI tools that are being developed now. I also use Chat GTP to make funny rhyming poems about friends and family members.
BM: I use ChatGPT almost daily. Same with Gemini. Actively, those are the two tools I use most. I’m still learning Copilot, but I see that use growing as well. Passively, I can’t even be sure how many AI tools I use (or how many use me). But, from Netflix to Spotify to Google Ads, I know that my interaction with AI tools is almost constant these days.
GR: When you’re handling confidential client information, do you have particular privacy/security concerns with using online AI tools like ChatGPT or Gemini? How do you properly handle confidential information with AI?
DT: I wouldn’t enter confidential information into an AI tool like ChatGPT. Any prompts I would use would be general as to facts or law and not contain client information. For advanced analytics tools used within existing document review platforms, those would be self-contained within your case materials and not shared with outside parties or the public.
BM: The short answer is yes, we have concerns about security and confidentiality. At this point, the way in which data is protected, how the data from prompts is used, what may be maintained are all relatively opaque issues with public LLMs. For that reason, we still have to be cautious. In general, the rule is, nothing confidential gets sent to one of these engines. You may be able to work around the risks by using pseudonyms, but that can still be dangerous.
GR: How do you see AI transforming the legal profession as a whole?
In the eDiscovery field, AI will eventually be used more frequently to reduce the billable attorney hours spent reviewing documents. Attorneys will still be needed to perform quality control checks on the AI work, and possibly to help train the AI systems, but hopefully the overall cost to the client will be substantially reduced. (DT)
DT: I think AI will also have a similar effect in other practice areas, such as contract drafting and probably many others. So far, it’s been a big disaster in the field of legal research, as the AI tools have been mimicking the language of legal arguments, while citing fictional cases. Several lawyers have been embarrassed (and sanctioned!) by relying on their AI researchers.
GR: In your opinion, what areas of law are likely to be most affected by AI in the near future?
BM: I think the easy ones are research and drafting. It’s been a joke for years now that much of contract/ property/ transactional law is the art of cut and paste. AI will make that closer to reality.
The ability to draft a document (albeit a first draft) that complies with the law of a given jurisdiction and contains the best clauses on a number of different issues and to do it all in moments will be a truly game- changing advance. (BM)
GR: How do you think AI will impact the way lawyers practice and deliver legal services?
DT: I think there will be many creative uses of AI in the legal field. But, generally, I think it will be used to take the place of the labor-intensive repetitive tasks that lead to large legal bills. There will be many situations where the AI can do the bulk of the work, and the attorneys would be the ones directing and checking the work.
BM: For those who embrace it, I think it will make them far more productive and it will clear their desks of the minutia that can sometimes be tedious and time-consuming. Some of these tasks are absolutely crucial, but so rote that they are difficult to bill to a client. AI can be the answer.
I see the introduction of reliable AI as akin to an attorney getting their first legal assistant. The ways in which that transition changes the work of the attorney are innumerable, and often particular to the attorney and the assistant. (BM)
GR: What role do you think human lawyers will play in an increasingly AI-assisted legal world?
DT: As mentioned in my previous answer, I think the human lawyers will be directing and checking the work of the AI tools. Some of the checks will be at an overarching—does this result make sense?—level and some will need to drill down on some of the minute details to confirm the process is working throughout.
BM: I don’t think the role changes that much. The reason someone hires a particular attorney rarely hinges on how adept they are at doing the things that AI may be able to take over. In large part, those things are the ‘metawork’ that supports the real reason attorneys exist and the ways in which they set themselves apart. Attorneys will continue to leverage the skill of converting knowledge and experience into wisdom and execution in a way that AI cannot replicate. They’ll just do it with a more reliable right hand and backstop in the form of the collected knowledge and speed that AI brings to the table.
GR: Given the necessity for human lawyers to review any results from these systems before signing off, do new lawyers need to develop any new skills for the upcoming AI-age?
DT: We think new lawyers always need to develop eDiscovery and document review skills, particularly regarding the functionality of searching and organizing documents within a review platform. For those attorneys who want to enter this field, keeping up with the latest analytic and AI tools would be very useful for their firms and clients.
BM: The most useful new skills are likely to be around learning how to properly communicate with these technologies via prompts and guidelines, as well as understanding how to interpret output and refine questions and prompts to get the answers you need (though not necessarily the answers you want). The technology is moving quickly enough that this gets easier every week. The current deployment of ChatGPT on mobile devices allows you to put on a headset and just have a conversation with the LLM directly. That format makes the feedback process not only faster and easier, but more familiar to the average user.
GR: How do you think AI technology will impact e-discovery and document review processes in the next few years?
DT: There have already been advanced technologies such as “predictive coding” to help leverage software to reduce billable hours and the human error involved in legal document review. As the AI tools become more sophisticated, reliable, and efficient, those benefits will increase.
BM: In the short term, AI is likely to increase the efficiency of existing processes and bolster the reliability of others. For instance, summarizing a binder full of data for a partner can now be done by an AI, rather than having a human being read through and draft something manually. At the same time, the speed and effectiveness of AI when it comes to pattern recognition and anomaly detection can be used to reduce the work required in quality control efforts across the entire span of the Electronic Discovery Reference Model (EDRM).
GR: What are some of the key challenges law firms face when integrating AI tools into their document review workflows?
DT: So many challenges. We still have challenges getting clients and co-counsel firms to use basic document review databases and review platforms. Some attorneys are just resistant to change and not comfortable using new technologies. With something as new and mysterious as AI, those challenges will be on steroids. Some people may not know that these tools exist, and some may simply not have the confidence to use them, at least right away. Also, in the context of litigation, usually you need to have both sides—and often the judge-- agreeing to the use of discovery tools that replace the actual document-by-document review by the attorneys. Hopefully, the potential for cost savings will help overcome these challenges.
BM: For review specifically, understanding how to properly prompt/direct an AI tool to provide the output that you need will be a challenge and will come with a real learning curve. Additionally, as with all other applications of AI in the legal space, understanding how to reliably validate the output of any AI system will be a challenge. It’s a challenge we will eventually solve. But, in the short term, it’s likely to be the largest.
GR: How do you balance the use of AI technology with ensuring defensibility and compliance in e-discovery?
DT: Your discovery obligations are not relaxed based on the technology you choose to implement. At the end of the day, the lawyer needs to be confident that whatever tools they use comply with the applicable discovery rule of that jurisdiction. Usually this takes the form of negotiating with the other side to get agreement on the tools that will be used by both sides, or sometimes, litigating the matter so the Court decides what tools can be used in a particular matter.
BM: This comes back to validation. If we look back to the work we have had to do over many years in integrating new technologies, we’ve always had to confront this. Moving from manual review of everything to using keywords to cull, relying on inclusive email reviews, adopting technology assist review (TAR)- all of these are technological advances that introduced a level of opacity to the process that raised concerns at the time. And, each time, we found ways to understand and validate what the program or machine was doing, and we used those measures until the processes became so commonplace that the trust became inherent. I believe the same will happen here. It may take a bit more time, but we’ll get there.
GR: Beyond opacity, what other concerns are you hearing from clients about using AI tools?
BM: The opacity issue is really an umbrella that covers a number of specific concerns. For instance, the opacity of the systems means that users don’t know how the tool is ‘understanding’ them, how it is reasoning through the creation of an answer or what resources it chooses to rely upon in generating a response. But, in addition to these, as new tools are created, many people have little knowledge as to what (aside from the final answer) the system may be generating. Are there artifacts created as part of the system’s reasoning process? If I’m using a tool like CoPilot and pointing it at various files and confidential information on our network, does any of that survive the conversation I’m having with it? As eDiscovery specialists, the questions around what types of data may be generated and where they’re stored may be a continually evolving one with each new tool and iteration.
GR: Can you share an example of a case where AI significantly improved the outcome or efficiency of the document review process?
BM: If we’re talking anything under the umbrella of AI, that’s an easy question.
We’ve seen scores of cases in which the amount of time needed for review was decreased dramatically by the use of TAR. Taking a case from a 20MM+ collection to a review set of less than 1.5MM documents is one example. In that case, key terms, concept searches, TAR and threading all came together to streamline a process and reduce a data set to something completely manageable. (BM)
GR: Are you familiar with Retrieval Augmented Generation (RAG) for interpreting and citing documents? Do you think this or another approach could be a key factor in the future of document management?
BM: I’m familiar with the term and how the process is supposed to work. In theory, yes, it could be a great help in building processes around the use of GenAI. However, to my knowledge, the work to be done to create an outside reference source for these tools which will be more reliable than their existing training data hasn’t yet been done.
GR: What are some common misconceptions about AI in e-discovery that you encounter from clients or other legal professionals?
DT: At this point, I think AI is probably not being implemented enough in eDiscovery to generate widespread misconceptions. But the old arguments against predictive coding and computer assisted reviews are that the algorithms just cannot be trusted well enough to rely on them for high stakes litigation. We think they often can be with the proper attorney quality checking and testing.
BM: From opposite ends of the spectrum, there are two: The first is that AI is an easy button. We can just throw data at the tool and its product will be all set to go- perfect on iteration one. The reality is that AI can make difficult processes easiER, but not always easy. And rarely is the process without human/machine interaction. On the other end, there are those who are uncomfortable with the tools and consider them unknowable, and therefore impossible to trust. While AI and all of its subgenres can be a really complex field, it’s not necessarily unknowable. And understanding what a tool is doing doesn’t require you to know the high-level algebra in order to get to a point of trust.
GR: What developments in AI technology are you most excited about for the future of e-discovery and document review?
BM: The biggest, of course, is the introduction of generative AI. We’ve hit on a number of different tools and functions in this interview. All are AI, but not all are as advanced as this application of AI which has taken the world by storm over the last 18 months. What we’ll be able to do with large language models, their depth of knowledge, and their mastery of language could be truly revolutionary.
GR: What key upgrades do you think GenAI will bring to your workflows in the future?
BM: We think it will make a number of processes faster, and will lower the barriers to entry on others. For instance, what we do relies heavily on a deep understanding of the data we are approaching for review.
Effectively and efficiently gaining understanding of large data sets is an art that can take people years to master. That is in part due to the level of expertise they need in understand patterns in data and the use of technology to search, analyze and treat that data. The simplicity of natural language inputs to GenAI tools can help those without that level of technical capability still participate in and become highly effective at that exploratory process. (BM)
GR: What are some of the primary ethical considerations surrounding the use of AI in legal practice in general?
DT: Attorneys have a large set of ethical obligations that apply regardless of their use of technology. One of the main obligations is the obligation of competency. This means that an attorney must provide competent representation to a client by having the knowledge, skill, thoroughness and preparation reasonably necessary for the representation. So, an attorney that tries to implement AI solutions without properly understanding them may be in violation of their ethical duty of competency. I would also argue that a lawyer who refuses to consider available technologies that would assist their client may also potentially be in violation of their obligations. In some cases, a lawyer who wants to use a certain technology may be obligated to affiliate with another attorney who does have the requisite skill or knowledge to use that technology.
BM: The biggest really comes from both our duties of supervision and competence. While we don’t feel that attorneys need to know everything about large language models (LLMs), we all have a responsibility to know enough to understand not just what we THINK we are asking a tool to do, but what the tool thinks we are asking it to do. Without this understanding, we’re not truly competent to be using the technology, and we can’t stand by our work product which contains output from that tool.
Additionally, we need to understand what outputs we need to check and validate and how to do so. It’s odd to think of ‘supervising’ a machine. But the spirit of the rules that govern how we practice would speak to our need to do just that, if we are giving these technologies the opportunity to create work product in any form.
GR: What regulatory frameworks do you think are necessary to govern the use of AI in the legal sector?
DT: I think the current ethics rules for attorneys probably already sufficiently covers the use of AI. Most of those rules deal with open disclosures to the clients, reasonableness, and informed client consent. If the clients understand the full risks and benefits of the proposed technology, the client should be in a good position to make an informed decision.
GR: Any other last thoughts about how AI will impact law in the future?
DT: Our mission has always been to leverage technology to lower client costs as much as is reasonably possible. We think the ongoing development of AI will continue the trend of lowering attorney costs, particularly in the eDiscovery field. In addition, we see it dramatically increasing productivity and reducing the cognitive load of many tasks in dramatic ways.
GR: Thank you, David and Brandon, for your discussion of the future of AI in law. We’ll check back in soon to see how the legal field continues to evolve.
The interview was moderated by Greg Robison, F'inn Founder and Chief AI Architect.
