Featured Case Studies
Blog

AI rEvolution – Testing Generative AI for eDiscovery Review – Planning 101

ediscovery and doc review with genai

As we begin another trip around the sun, there are a few eDiscovery topics that will remain top of mind through all 12 months of 2025 (and beyond). The use of next-generation AI in the legal industry is certainly near the top of that list.

While AI in eDiscovery and specifically document review is a rather small portion of that, it is an area in which Generative AI has established a foothold and represents a unique opportunity to explore what the latest AI can do in our field.

As we engage with clients on this topic, the discussions start with some variation of: “We want to understand it and be ready to use it but don’t know where to start or how to plan for the process.”

The process itself is relatively straightforward, not overly time-consuming or expensive. It does, however, exemplify the saying “failing to plan is planning to fail.” Furthermore, that failure could come with a variety of very costly consequences.

The remainder of this brief post offers cheat codes to avoid those pitfalls.

Approvals – Get them

While this is common sense, it is more than that. The past two years of rapid AI evolution have led to a dramatic increase in regulation, policy, and contract language surrounding its use. Your organization, and/or your clients’ organizations, almost certainly has a team (or teams) responsible for assuring compliance. Track them down and get their blessing.

The time it takes to check this box will vary. You may want to identify a good project for your testing (see next topic) before initiating the process(es). You don’t want to work through it only to find that there is no good testing data at the end of that particular rainbow.

Identify a Testing Project

The key to success here is maximizing the utilization of work you are already doing (or have done).

Good testing data will need “gold standard” review decisions with which to compare the AI decisions. Beyond identifying data for which you have (or will have) review decisions, you will want to be sure the review process creates decisions comparable with AI decisions. These adjustments are quite easy to make at the onset of a review. On the other hand, they may be virtually impossible to make on a completed review. If you have tried to reconcile “family complete” review decisions with traditional TAR workflow testing, you will recognize this challenge.

Beyond the data and review decisions, consider access to the subject matter experts (SME) familiar with the review you want to test against. As part of the testing process, you will need to iteratively adjust the review protocol used by the AI (“AI Prompts”). Your ability to do this quickly and without a significant time investment likely relies on access to those SME.

Subject Matter Expert Time   

Speaking of SME, how much of their time will you need? Not much really. That said, you must have it. You will need the SME to:

  • AI Review Prompt Authoring – Assist in translating review protocols into AI-compatible prompts. Please note, this is essentially a style translation. Note: you will not be asking the SME to become “prompt engineers.” Your technology and service partners should provide all the assistance you need to adjust a standard review protocol for effective use by AI.
  • AI Review Analysis – Analyze instances where the AI disagrees with human review decisions. The objective is to identify areas where the AI is consistently generating decisions inconsistent with the submitted protocol’s intent.
  • AI Review Prompt Adjustments – Complete / Consult on adjustments, based on the results of AI Review Analysis

How Will You Measure Results?

Know how you will measure AI’s success going into the process. Do you need to run it against the entire population, or can a properly sized control set-based test meet your needs? This decision will drive both workflow design and costs. Resources (time and money) for a large full-population test review are generally hard to come by. Don’t let that discourage you! Intelligent workflow design and statistics can give you the necessary familiarity and confidence.

Conclusion   

While testing Generative AI-enabled review tools is logistically straightforward, appropriate planning is critical to achieving the confidence you need and perhaps more importantly, avoiding potentially significant and costly consequences from planning failures. An intelligently designed workflow applied to properly selected data, implemented after obtaining the necessary approval(s), can safely deliver the data you seek.

Read all of our AI blogs here: www.purposelegal.io/embracingAI

Tags
ai ai for generative ai for managed review ai for review featured generative ai
Back to Blog

© Copyright 2023 Purpose, All rights reserved

It’s a self-appointed nickname, but it rings true. Our elite team approaches each project with precision, offering bespoke solutions that align with your objectives.

This will close in 0 seconds

Expect over-communication and real-time feedback, keeping you informed every step of the way.

This will close in 0 seconds

Leveraging AI and analytics for efficient review and quality control.

This will close in 0 seconds

With services in multiple languages and a dedicated review team in India, we handle international and multilingual projects effortlessly.

This will close in 0 seconds

Your data’s security is our top priority, with protocols that exceed industry standards.

This will close in 0 seconds

Purpose Legal has the capability to process thousands of documents across nearly every data type each hour. Our technology accelerates the production of ready-to-review files, enabling faster response times in critical review scenarios.

This will close in 0 seconds

Save significant manual administrative time with our automated redaction capabilities. Effortlessly redact personally identifiable information (PII) across multiple documents and spreadsheets with our bulk-redaction tools.

This will close in 0 seconds

Build accurate searches with confidence. Purpose Legal uses intuitive technology that supports highly detailed queries, offering accessible and clickable terms for efficient review.

This will close in 0 seconds

There is no project too large or too small. Purpose Legal creates customized workflows to perfectly suit our clients’ needs, ensuring they achieve their objectives with our comprehensive support.

This will close in 0 seconds

We believe in overcommunication to ensure you never face the unknown. Our reports provide data and insights, mapping out clear options and their implications for your business. With Purpose Legal, you’ll always know the status of your project and the impact of your decisions.

This will close in 0 seconds

Our dedicated team in India specializes in insurance defense matters. Our expertise also extends to supporting plaintiff firms in class action lawsuits. Our team boasts long-term loyalty and a broad language capability—including Mandarin, Japanese, and more—ensuring we meet global needs.

This will close in 0 seconds

Purpose Legal reviewers undergo extensive training on the specific issues and technologies of each project. Our real-time feedback and performance evaluations ensure a continuously improving team, ready to handle any challenge.

This will close in 0 seconds

Our review methodology is meticulous from start to finish, ensuring quality control through advanced analytics and AI processes. From project initiation to final quality checks, we implement a rigorous process to guarantee the highest standards of review quality.

This will close in 0 seconds

  • Monitoring and Enforcement: Vigilant monitoring and enforcement of applicable data protection laws and regulations.
  • Updates and Awareness: Regular updates to our Privacy Compliance Program and the promotion of workforce awareness of data protection risks and rights.

This will close in 0 seconds

  • Unauthorized Access Prevention: Implementation of controls to protect against unauthorized access to data, including internal controls for cyber threat detection.
  • Access Controls and Authentication: Strong access controls and multi-level user authentication are needed to ensure authorized access to data only.
  • Encryption and Secure Storage: Data encryption and secure, access-controlled data storage environments.

This will close in 0 seconds

  • Minimization of Data Processing: Ensuring minimal personal data processing and maintaining comprehensive records of processing activities.
  • Incident and Breach Management: Procedures for incident and data breach management, with periodic updates to stay compliant.

This will close in 0 seconds

  • Vendor Compliance: Periodic review of vendor contracts for compliance with data protection laws and oversight of critical vendors.
  • Contractual Obligations Review: Regular review and updates of contractual obligations to ensure compliance.

This will close in 0 seconds

  • Cyber Simulations: Conducting periodic cyber simulations to evaluate readiness for cyber incidents.
  • Security Audits: Regular security audits, including penetration testing and vulnerability scanning, to identify and address vulnerabilities.

This will close in 0 seconds

  • Client Consent: Preventing the sharing, sale, or disclosure of client data without express consent.
  • Use Limitation: Limiting the use of client data to approved business purposes and ensuring secure, VPN-based virtual desktop reviews.

This will close in 0 seconds

  • Remote Connectivity: Utilizing Accops remote connectivity tools with stringent controls on screenshots, copy/paste, and USB access.
  • Firewall Implementation: Use of firewall systems to authenticate incoming network connections and separate networks.

This will close in 0 seconds

  • Comprehensive Training: Providing extensive training on data security best practices, phishing recognition, and secure protocol adherence.
  • Security Awareness Programs: Regular programs to reinforce data security practices among employees involved in document review.

This will close in 0 seconds

  • Privacy Risk Management: Use policy to manage and monitor legal risks and compliance requirements around privacy and data management.
  • Privacy Rights Requests: Maintaining records of Individual Privacy Rights requests and informing clients of such requests.

This will close in 0 seconds

  • Assessment of Risks: Conducting risk and business impact assessments to determine necessary levels of authentication and security measures.
  • Dial-Back Procedures: Implementing dial-back controls for protection against unauthorized connections.

This will close in 0 seconds

With forensic collections and investigation, “it depends” isn’t a non-answer—it’s the starting point of our precision-tailored strategy. Our experts know the questions to ask to get to the heart of your “it depends” and find the answers you need. What data is crucial? Which platforms are involved? What are the legal implications?

This will close in 0 seconds

Our expertise shines in our adaptability. Whether you’re dealing with enterprise-level cloud storage or specialized communication tools, we tailor our methods to suit the platform, the data, and the stakes of your case.

This will close in 0 seconds

Forensic investigations (physical and electronic) require everything below the surface, uncovering artifacts, time stamps, movement, behavior, and exfiltration. We specialize in the normalization or parsing of data so you can understand it and use it to your advantage.

This will close in 0 seconds

When we talk to clients, we often hear “I didn’t know I didn’t know that.” And frankly, you don’t need to—that’s what we are here for. You care about the output of the data you must review and that WE know how to do it. We will handle the heavy lifting so you feel confident that the process is being handled quickly, efficiently, and producing the evidence you need.

This will close in 0 seconds

While the process may be complex, our communication isn’t. We provide clear, comprehensible explanations, so you understand the rationale behind our approach and how it’s the best fit for your case. You are never confused or in the dark.

This will close in 0 seconds

Our methodology guarantees comprehensive coverage and defensibility. It’s that simple.

This will close in 0 seconds

With over twenty years of trial-tested experience, Purpose’s computer forensics examiners can offer expert testimony, either in court or through deposition, to explain complex, technical concepts in layman’s terms, as well as present electronic evidence in a clear and comprehensible manner.

This will close in 0 seconds

We decipher the most complex data, offering you fast, accurate insights. Stay ahead with our innovative tools and methodologies, prescribed specifically to your needs and goals.

This will close in 0 seconds

Our attorneys are proficient in reviewing and redlining contracts based on established playbooks, which we can develop in collaboration with our clients. This ensures that the contract review process is both efficient and aligned with client-specific standards and expectations.

This will close in 0 seconds

Purpose Legal has extensive experience with a variety of specialized contract management tools including, Concord, Contract Sage, Wrangler, CS, Workiva, and more. Our team is skilled in utilizing these tools to offer cloud-based platforms, central repositories for contracts, workflow management for contract review and approval, collaboration tools, dashboards for tracking and reporting, e-signatures, and contract risk management detection.

This will close in 0 seconds

We provide contract drafting services based on templates, ensuring consistency and compliance with legal standards and client requirements. Our approach is flexible, allowing for customization based on specific client needs.

This will close in 0 seconds

Our team has tested and trained on leading contract management tools like Brevia, Luminance, ContractWorks, etc., which feature Al-enabled capabilities for auto-extracting key contract data points. We understand that while these tools provide a solid foundation, manual review and additional tagging are essential for accuracy and comprehensiveness.

This will close in 0 seconds

Purpose Legal offers a highly customizable solution that can include but isn’t limited to the following:

  • Email or FTP for contract review and data extraction, with data organized into client-approved Excel templates.
  • Support for contract review and data extraction via Relativity, with custom data fields and OCR search capabilities.
  • In-house tool utilization for data review or template preparation for data import.

This will close in 0 seconds

From pre-M&A due diligence contract review to lease abstraction, playbook creation, and data extraction, Purpose Legal has the expertise to support a wide range of contract management services. Our team is capable of drafting and reviewing a variety of contracts, from NDAs to loan agreements and beyond, with an emphasis on identifying and managing nonstandard clauses.

This will close in 0 seconds

Purpose Legal excels in creating best-in-class playbooks that streamline the contracting process, drawing on our deep experience and best practices to produce effective, client-specific playbooks.

This will close in 0 seconds

Our team includes seasoned Senior and Junior Indian Attorneys, providing reliable, efficient service with a proven track record of low error rates and high throughput, thanks to our focus on full-time, dedicated employees.

This will close in 0 seconds

NexGen Managed Services are designed to cater to your unique needs, whether you’re looking for Self-Service, Full-Service, or a tailored combination of both. Our Hybrid Approach seamlessly integrates skilled people and cutting-edge technology to support your operations, whether they’re cloud-based, housed in a data center, or on-premises. We offer robust infrastructure, ensuring that you have the dedicated environment you need to thrive without the complexities of a multi-tenant setup. For those who prefer a more human touch, our people-only option puts our expert team at your disposal.

This will close in 0 seconds

Our Configurable Tech Stack is designed with diversity and adaptability at its core. To streamline integration and ensure seamless data flow, we provide sophisticated middleware and data conversion solutions. We don’t compromise on storage, ensuring your data is always at hand when you need it. Security is paramount, with robust measures in place to protect your information. Our commitment to business continuity means that your operations keep running smoothly, no matter what.

This will close in 0 seconds

All of our solutions are bolstered by a team of experts who bring a wealth of knowledge to every project. We excel in Processing and Culling, streamlining data for efficiency. Our experts are dedicated to Forensics Collections and Investigations, ensuring thoroughness and accuracy. Our Advanced Technology harnesses cutting-edge tools to provide smarter, faster outcomes. Our Project Management acumen ensures that every project is executed flawlessly. With secure Data Hosting, we keep your data accessible and safe. Our NexGen Managed Review services blend expertise with precision, and our Consulting services provide strategic insights to drive your projects forward. The final touch to our services includes productions that meet all legal standards.

This will close in 0 seconds

Client Enablement is at the core of our approach, beginning with the creation of a personalized playbook to guide clients through complex scenarios. We ensure a smooth transition with comprehensive onboarding and implementation, setting the stage for success. Our workflow standardization makes operations efficient and predictable. We assign dedicated Project Management teams to maintain a consistent, high-touch service experience. Management of infrastructure, licenses, updates, upgrades, and patching is handled meticulously to keep client systems optimal without interruptions.

This will close in 0 seconds

Metrics That Matter form a critical feedback loop with an emphasis on both reporting/metrics and key performance indicators (KPIs). We provide tools to track backward & forward-looking metrics, enabling clients to assess past performance while anticipating future trends and outcomes. Our approach ensures that decision-makers are well-equipped with actionable data to drive continuous improvement and strategic planning.

This will close in 0 seconds

We aim to distinguish our services with a unique blend of expertise and innovation. Our team of AI & TAR experts push the envelope in technology research and testing. Our rigorous expert training and consulting services complement our team’s expertise, as do our product and application developers, who tailor solutions to meet the specific needs of each client.

This will close in 0 seconds

By redefining the traditional view of services as a Profit Center rather than a Cost Center, we focus on value creation. Our models ensure clients understand the financial impact. We offer clear Contract Termination Clauses and the flexibility to tier up and down in service levels without the worry of burst fees, providing cost control and predictability. Our pricing structures also include no burst fees for unexpected usage spikes and a commitment to cost predictability, so there are no surprises.

This will close in 0 seconds

We understand that the right technology can vary from case to case, which is why we provide access to a broad spectrum of leading eDiscovery and legal technology platforms. This breadth of choice ensures that you can always select the tool (or tools) that best fits your needs without the constraints of single-platform limitations.

This will close in 0 seconds

Purpose Legal sees the need for our industry to evolve beyond outdated practices and rigid contracts.
We don’t believe in forcing clients into non-flexible, 3- to 5-year contracts. We want to demonstrate our value continuously, proving ourselves indispensable month after month.
Flexibility and a cutting-edge tech stack are the backbone of our NexGen Managed Services.

This will close in 0 seconds

In just a few clicks, our technology can help you easily send notices to unlimited custodians, track compliance, issue reminders, and even preserve in place for Microsoft Office365.

This will close in 0 seconds

Automate your legal hold process with our user-friendly technology. Standardize every step, from issuing holds to reporting, with powerful dashboards and seamless integration with HR and matter management systems—no IT infrastructure needed.

This will close in 0 seconds

Easily create and access records of your compliance efforts anytime, allowing for customized reporting and enhanced accountability.

This will close in 0 seconds

Minimize manual administrative tasks and mitigate risk through automated notifications and reminders, ensuring custodians respond efficiently.

This will close in 0 seconds

Navigating data privacy regulations like GDPR, CCPA, and emerging global standards requires a proactive approach. Purpose Legal empowers your organization to meet—and exceed—compliance standards with ease. Our solutions are designed to handle any volume of privacy-related inquiries, such as Digital Subject Access Requests (DSAR) ensuring you can meet quick turnaround times without compromising on budget or quality.

This will close in 0 seconds

We equip you with the tools to sift through vast amounts of information quickly, identifying and securing the specific data you need to protect your privacy and comply with regulations.

This will close in 0 seconds

Purpose Legal’s automated redaction technology reduces manual administrative tasks by enabling bulk redactions across documents and spreadsheets, including the sensitive handling of personally identifiable information (PII).

This will close in 0 seconds

We take the security of your data as seriously as you do. Purpose Legal is proud to hold ISO-27001 and SOC2-Type 2 certifications, reflecting our unwavering commitment to maintaining the highest standards of data security and operational excellence.

This will close in 0 seconds

Efficiently map, locate, and identify the right information to respond to public records requests confidently, accurately, and thoroughly.

This will close in 0 seconds

Utilize reliable uploads and powerful search functions to accurately report, track, and share information, ensuring the protection of your data with vetted security measures and automated redaction tools.

This will close in 0 seconds

With the capability to process up to 900K documents per hour from any popular cloud storage solution, kickstart your review process quickly and efficiently.

This will close in 0 seconds

Automatically identify and redact sensitive information such as PII across documents, including native spreadsheets, with analytics, making the redaction process for public records requests straightforward and efficient.

This will close in 0 seconds

Our project management team and review attorneys have the deep experience needed to make sure you find what you need and produce what’s required to all parties completely and on time.

This will close in 0 seconds

Our project management team and review attorneys have the deep experience needed to make sure you find what you need and produce what’s required to all parties completely and on time.

This will close in 0 seconds

With Purpose’s technology, you can instantly identify critical concepts across millions of documents and quickly narrow a high-level analysis to a granular document review with comprehensive data visualizations and simple controls.

This will close in 0 seconds

Our team will collaborate with you to create interview question templates, document organization, report drafting, and more.

This will close in 0 seconds

We have managed over 30 significant second requests, covering high-profile cases, including pioneering the first Canadian second request. Our diverse group of experienced technologists, attorneys, paralegals, industry veterans, and project management specialists spans the country to ensure 24/7 coverage.

This will close in 0 seconds

Our team has decades of experience providing expert-level, customized, repeatable, and cost-effective solutions. We have a complete end-to-end chain of custody process, in which we document every step of every process, ensuring defensibility.

This will close in 0 seconds

Given the limited timeframe allotted for compliance in second requests, efficient and accurate data processing is the key to success. Purpose Legal has the team, infrastructure, and processes to handle the largest and most complex data sets.

This will close in 0 seconds

Purpose Legal’s consulting team designs and implements unique TAR workflows for predictive coding, email threading, near-duplicate identification, clustering, and categorization to drastically reduce document review time and spend.

This will close in 0 seconds

We begin every project with a planning meeting and ensure cost containment by using scoping and data minimization techniques to defensibly reduce the population for review and production. The account management team stays involved throughout the course of the project, providing weekly cost-to-date reports to main points of contact.

This will close in 0 seconds

We believe in identifying your desired outcome and matching you with the technology that best achieves your goals. Our suite of platforms is as diverse as the challenges you face. Our technology-prescriptive method allows us to consult and guide you toward the solutions that fit your unique needs, not the other way around.

This will close in 0 seconds

Ever wanted to experiment with a new platform without the overhead? Purpose Legal makes it possible. Trial new technologies on sample cases with our support, ensuring a risk-free exploration of what could be the next game-changer for your team. Use trial-based experiences to make data-driven decisions on which platforms and technologies work best for you—without the risk.

This will close in 0 seconds

From the latest in collection technology to hybrid managed services, we tailor our offerings to your in-house needs. Experience the freedom of choice, supported by professionals who know how to extract maximum value from each solution.
Step into a new reality with Purpose Legal, where your eDiscovery solutions are as adaptive and dynamic as your cases. Let’s redefine what it means to be supported in legal technology together.

This will close in 0 seconds

We’re not just providers; we’re partners in your tech journey. Purpose Legal offers a Swiss Army knife approach to eDiscovery, ensuring you have the right tools at the right time. Our technology-prescriptive method allows us to consult and guide you toward the solutions that fit your unique needs, not the other way around.

This will close in 0 seconds

We have no agenda. We have joined forces with our vetted partners to provide the best solutions for YOU, and we love them. But we don’t have any agenda to make you use their products. We look at each project holistically, and more importantly, we get to learn your goals and needs. That helps us build you a PERSONALIZED, prescriptive workflow.

This will close in 0 seconds

Everyone says that, we know. And every company has talented people, no doubt. We only hire dedicated people. Ambitious people. Helpful people. Honest people. GOOD people. Ask someone in the industry if they know someone from our leadership team. Not only will they likely know and have worked with these people, but they will tell you how kind, honest, and hardworking they are. Their reputations precede them. Our amazing crew attracts more of the same, and it overflows into every project we work on. We offer each other new perspectives, helping hands, laughs, and encouragement, and a happy team makes for better service, more innovative workflows, and stronger outcomes. We are here for each other, better poising us to be there for you.

This will close in 0 seconds

Being nimble and responsive allows us to adapt swiftly to the evolving needs of our clients, ensuring that we can provide tailored solutions with speed and efficiency. Our dedication to responsiveness means that our clients benefit from immediate support and customized advice, reinforcing trust and facilitating smoother projects.

This will close in 0 seconds

Corporate culture is not just about employee perks or office aesthetics. It’s an embodiment of values, beliefs, and practices that determine how a company interacts with everyone—employees, clients, and the community. In today’s world, it’s manifested prominently through ESG (Environmental, Social, and Governance) principles.

This will close in 0 seconds

We are steadfast believers that a diverse workforce is a competitive advantage. It leads to innovative solutions and ideas while boosting morale and productivity. When employees from different backgrounds and perspectives come together, they create a robust and versatile team. It’s not just about ticking a box; it’s about ensuring the best in terms of talent acquisition, retention, and performance.

This will close in 0 seconds

At Purpose Legal, the road to creating a purposeful corporate culture is paved with strategic planning. From garnering internal buy-in to ensuring candidate and client engagement, our focus is unwavering. We believe in the power of metrics to guide our strategies, always ensuring our diversity goals align with our broader company objectives.

This will close in 0 seconds

24-hour A/C power, UPS with emergency diesel generator backup, HVAC, fire detection and suppression linked to the local fire authority, and comprehensive security measures. Purpose Legal’s Datacenter is fully equipped with CCTVs in operation at all times.

This will close in 0 seconds

Proximity card readers, hand scanners, video surveillance, chain link fencing, and no external windows.

This will close in 0 seconds

Uninterruptible Power Supplies (UPS) are provisioned in an N+1 configuration and designed to supply full power to all essential services in the event of a main power failure or fault. Generator systems are configured for auto-start and will start within 45 seconds of a main grid power failure.

This will close in 0 seconds

Access is requested by submitting an Access Request Ticket via email to the Datacenter’s Service Desk prior to the visit.

This will close in 0 seconds

Visitors are escorted to the server cabinet through four security access points—each is secured by a biometric reader, number pad combination, and proximity card reader. After providing government-issued ID and biometrics, the relevant Purpose Legal personnel is given a security card and allowed to enter.

This will close in 0 seconds

All Purpose Legal employees are background checked and security vetted before being able to work on client projects.

This will close in 0 seconds