Disclaimer: This post is a personal reflection and does not constitute legal advice. Every situation is different — if you need legal advice, email me at rupen.shah@gpslegal.com.

“Why would I need a lawyer? AI can do it now.” I’m hearing this and reading about such a sentiment more often. Having recently attended Hong Kong FinTech Week, where AI seemed to feature in almost every conversation and panel discussion I encountered, I can understand the enthusiasm. There is a great deal of interest in what these tools can do, and how much time and money they might save.

What I am seeing in practice is more nuanced. I have recently worked with several small business owners who were very upfront about using AI, including to prepare their legal documents. But at the same time they recognised that they still needed legal advice. For me, the interesting question is where that boundary lies: how far can AI take you, and when do you need someone with the experience?

I previously wrote about the cost of fixing an AI-generated contract in “AI drafted my contract. It cost thousands to fix”. I am now seeing founders use AI to prepare whole suites of transaction documents. That raises questions beyond whether an individual clause is well written. Do the documents work together? Do they reflect the intended deal? And what has nobody thought to ask?

Why founders use AI

When I worked in a large law firm on multibillion-dollar transactions, my clients were often well-resourced financial institutions. They had internal legal teams, established processes, precedents and libraries of documents. They also had institutional knowledge built up over years of doing similar transactions.

Working directly with a founder feels much more personal. The business owners I meet know their businesses and industries inside out. But they may also be the CEO, CFO, COO, HR department, marketing team and IT support. They also take on the task of dealing with all the legals. They have to make the big decisions and deal with the small ones.

That makes time and resources precious. A tool that can explain an unfamiliar concept, organise a pile of information or help prepare a document has an obvious appeal. I use AI myself, although I still consider myself relatively new to it. I am learning and experimenting as I go, including working out where it helps and where I need to be more careful. The image accompanying this article is one example. I used AI to generate it, then asked it to remove the people and keep the computer and missing puzzle pieces. I could judge for myself whether the result worked. With a legal document, knowing what needs changing—or what is missing—can be much harder.

Being able to ask follow-up questions also makes subjects that previously you didn’t know about much easier to approach.

The risk is that founders may feel confident in an answer without knowing whether it is right for their business. Understanding the explanation of a clause is a useful start. Knowing whether that clause belongs in your agreement, and how it will affect the rest of the deal, requires a bit more.

Where things can go wrong

“You don’t know what you don’t know” is a familiar expression, but it is particularly relevant here. If you know that an issue needs addressing, you can ask about it. If you have never encountered the issue, you may not realise there is a question to ask.

In her Law Society article, “How AI tools hallucinate—and why it matters in law”, Mia Leslie explains how AI-generated answers can sound authoritative and follow familiar legal formats while still being wrong. It can look convincing even when it is wrong.

Invented information is only part of the problem. A contract can contain perfectly plausible provisions and still be unsuitable for that particular scenario. It may omit a protection, assume facts that are incorrect or give one party rights that the other party has not fully understood. In a transaction involving several documents, a provision that looks sensible in isolation may also conflict with another part of the transaction.

These problems can remain unnoticed while everyone gets along (and they probably will). That all changes when a payment is missed, someone wants to leave or the parties disagree. By then, you are probably looking at much higher costs trying to fix the issue or getting into expensive litigation. Everything is fine, until it isn’t.

Two recent matters I worked on illustrate these points.

An investment agreement that needed to work in practice

In one matter, a founder had used AI to prepare documents for bringing investors into the business. The documents captured commercial points that mattered to the parties.

The difficulties became clearer when we considered how the arrangements would operate. A requirement for unanimous investor approval above a low spending threshold offered investors oversight, but would make routine expenditure difficult. A commitment to distribute a substantial proportion of profits needed to be considered against the business’s need to retain cash.

There were also questions about what should happen to shares held by someone who stopped working in the business, and how a proposed investor exit or repayment arrangement would operate if circumstances changed. Recording an intention that an investor should be able to get their money back was only the beginning. The mechanism, who would pay and the ability to carry it out all needed attention.

The work involved reconsidering the arrangements and putting them into one set of documents (rather than a separate one for each investor), including amending the company’s articles. This was more than just tidying up the language.

These are commercial questions as much as drafting questions. How much control should investors have? What freedom does the founder need to run the business? What happens if they disagree? AI had helped set out the commercial terms, which was a valuable starting point. However, its legal drafting didn’t work in this instance.

A service agreement and the cost of cancellation

In another matter, a business owner had used AI to prepare a service agreement. Again, the draft captured the broad commercial arrangement. The gaps concerned what would happen when delivery did not go to plan as well as standard boilerplate terms.

For example, what would the provider recover if the customer cancelled shortly before the service started? What could the provider do if payment was late? Could it suspend performance, or would it be expected to continue incurring costs?

The draft also needed attention to liability and the protection of materials developed for delivering the services. The questions were how much the provider could be liable for, and whether the customer could reuse its materials.

The agreement prepared by AI did deal with bookings, payments, cancellation and delivery of the service, but didn’t capture some fundamental legal points that were needed to protect my client. A founder reading the initial draft might feel that this was a good enough document, as it captured the commercial agreement and dealt with payment issues. What a founder might not be able to do is identify issues that haven’t been properly addressed and need to be covered.

In both instances, the founders sought advice. They realised that they needed a professional to look through the documents and provide them with advice (as opposed to just signing off on the documents they had provided). My task was to identify and alert them to issues that AI had missed. Both founders understood the value of recognising when further judgement was needed.

In both instances, the AI documents helped explain the commercial terms, but the legal documents still had to be drafted from scratch.

Where AI can help

There is plenty of useful work AI can do before and during a conversation with an adviser. It can help a founder get an initial understanding of things they are not familiar with, organise correspondence into a transaction history and turn scattered notes into a clearer description of the proposed deal.

It can also help generate questions. What assumptions have been made? Which points remain undecided? What might happen if payment is late, the project is delayed or one party wants to leave? Used this way, it can make a founder better prepared to give instructions and discuss the advice they receive.

As with everything AI related, this all still needs checking and verifying. A summary can leave out a qualification, and a list of questions may miss an important issue.

A draft created by AI may be useful information. Before sending or signing it, you need to know whether it covers the deal and protects your business. A second AI agreeing with the first does not, by itself, establish any of those things.

There was recently a great Reddit post, which while tongue-in-cheek, captured this last point:

“The modern legal client has evolved.

Step 1: Ask AI to draft the legal notice.

Step 2: Ask another AI model to review the draft.

Step 3: Ask a third AI whether the first two AI models are legally correct.

Step 4: Develop a firm legal opinion based on all three responses.

Step 5: Finally approach a lawyer and say:

“Sir, I have already researched the matter. AI says I am absolutely right. I just need you to confirm it.”

The lawyer is now expected to function as a human verification plugin.

And when you point out that the AI has misunderstood the law, missed a precedent, invented a judgment or confused jurisdiction, the client looks disappointed.

Apparently, the problem is not the legal position. The problem is that the lawyer hasn’t updated his legal database with the latest ChatGPT hallucination.

I have absolutely nothing against AI in law. It is a fantastic research assistant, drafting companion and productivity tool.

But there is a difference between using AI to understand the law and assuming AI has replaced the application of law.

AI can generate a legal opinion in seconds. Unfortunately, it cannot always generate legal accountability when that opinion goes wrong.

Perhaps the next generation of law firms should advertise:

Human Lawyer Available. Specialising in Correcting AI’s Legal Confidence.”

Knowing when to seek judgement

In my guest article for Eight PR, I wrote about the growing value of clarity and judgement when information is abundant. Founders rarely need a longer list of everything that might possibly go wrong. They need help deciding what matters to their business and what to do about it.

That is where an experienced professional adds value: identifying what AI has missed and advising the founder on what needs to be done. Sometimes the advice will be that a risk is manageable and a simple arrangement is sufficient. Sometimes a seemingly minor provision needs much more attention.

Professional advice cannot guarantee the outcome. It should help you understand what you are agreeing to and the risks involved, so you can make an informed decision.

When it is time to seek advice, I would suggest:

Involve an adviser while important terms are still open, particularly where ownership, control, liability or exit rights are involved.
Explain the commercial objective and the decisions you need help with, rather than asking only for confirmation that the draft is correct.
Agree a proportionate scope and fee for the advice, so that cost is part of the discussion from the outset.

AI can help founders make better use of limited resources. The important thing is to recognise when you need a professional to help you understand what you are agreeing to and whether the documents do what you need.