By Wenting, from Ou Fei Temple
QbitAI | WeChat Official Account QbitAI
Wow, Zhipu has rolled its vertical model into the legal field!
Just now, Zhipu launched a new open-source model specialized in law —GLM-Legal。
It can not only chat with you about legal provisions, but also take on a series of concrete tasks such as reviewing contracts, stringing together evidence, and conducting cross-border compliance research, delivering results you can use as-is.
Its scores are quite impressive too.
In the latest legal evaluation published by Vals AI, GLM-Legal, with a19.2%full task pass rate and86.23%accuracy, took the top spot on the open-weight model leaderboards of both Harvey LAB and LegalBench.
Well then, after out-competing in chat, office work, and code, AI is now setting its sights on the fine details inside legal contracts and case files.
As soon as the news broke, folks couldn't sit still:
Great, great, great — from now on, GLM-Legal will be my legal team.
But wait — which country's law did it actually learn? Can it really do work for us Chinese users?
This is worth a closer look.
Reviewing contracts, stringing evidence, doing compliance — GLM-Legal can handle them all
The good news is,it is indeed adapted to our Chinese lawand won't suffer from being out of its element.
Whether it's Chinese legal practice, English legal tasks, or Southeast Asian cross-border compliance exam questions, it can handle them all.
But mock exams are only one aspect after all.
Real legal work is full of pitfalls — step on one and you'll fall silent instantly.
To see whether it can actually do the job, we still have to take it for a spin in a real legal work environment and see if it can help teams review contracts and determine privacy compliance matters before overseas release.
Let's first look at three official demos (On GLM-5.3-Flash battling GLM-Legal: fighting itself）：
I. Reviewing a contract and sorting out the handover arrangements after the cooperation ends
When we need to switch software service providers, we often need to first review the termination and handover clauses in the contract.
Can the code, data, and documents be smoothly retrieved? Can the new provider pick up where things left off?
And these landmines are often buried inclausesin them.
So, GLM-Legal suggested that we shorten the cooperation termination notice period from 60 days to 30 days, and add transition support of up to 90 days, making it convenient for us to arrange the handover of code, documentation, and knowledge.
In addition, it combined the contract's existing provisions on ownership of deliverables, requiring the return of all work products (including parts that were not yet completed or paid for).
Compared with GLM-5.3-Flash, GLM-Legal's advantage is obvious:
The former only offers fairly general suggestions, whereas GLM-Legal goes further by clarifying details such as timing and the scope of the handover, and aligning the modification plan with the contract's existing provisions.
Well then, this "breakup process" after ending a cooperation has been laid out clearly by it, all very gracefully.
That got me wondering: when I throw a pile of scattered evidence at it, how would GLM-Legal respond?
II. Analyze the evidence and piece together the scattered clues
This is actually a very practical question.
When a project collaboration dispute arises, how exactly can we determine, from a massive pile of reports, documents, and chat logs, whether the other party's claims have any problems?
Take, for example, this case given by officials: external reports claimed the project was 60% complete, while internal records showed only about 41%.
In actual testing, both GLM-5.3-Flash and GLM-Legal were able to identify this contradiction;
But the difference is that GLM-5.3-Flash only gave a suggestion, while GLM-Legal went further on top of the suggestion, linking the progress report to earlier risk warnings and internal communications, and turning it into concrete actions.
Once this thread has been sorted out clearly, we can then return to the original documents to check each item one by one and figure out which figure is actually the correct one that can continue to be used.
III. Compliance for Enterprises Going Global: Turning Regulatory Requirements into Action Checklists
And when we prepare to launch our website on overseas platforms such as Singapore, Malaysia, and Indonesia, we may face challenges where every aspect—user tracking, email marketing, overseas cloud storage, and more—could involvePrivacy Compliancedifficult problem.
At times like this, we may need an assistant to help us figure out exactly what preparations need to be made before going live.
In response, GLM-Legal in real-world testing broke down the compliance matters requiring attention into detailed steps that teams can put into practice and continue to verify, including but not limited to publishing a specific privacy statement, adding unsubscribe requests to a suppression list, and so on, all clear at a glance.
This is much more convenient now.
So looking back along this line of thought, we can more intuitively understand where the difficulty lies—and where the value lies—in Harvey LAB's 19.2% full pass rate.
All 19.2% passed—how hard is this exam paper?
Harvey LAB's reviews have never been paper tigers.
It adopts a rigorousThe "full conditional approval" mechanism, a model only passes a task if it meets every scoring criterion — missing even one means failure.
Its task design also directly mirrors how law firms assign work:give the Agent a request and a set of client materials, and have it produce a review-ready document directly.
The first version of Harvey LAB alone set upmore than 1200simulated law firm work tasks, covering 24 legal practice areas such as M&A, intellectual property, and banking and finance24legal practice areas.
This means that, from reading materials and calling tools to delivering results according to the rules, the model cannot slip up at any step if it wants to score well here.
Beyond the scores, there is also a contrast in test billing figures worth our attention.
With similar scores, GLM-Legal was nearly slower than Gemini 4 Argon by30 minutesslower than Gemini 4 Argon, but Gemini's cost per test was about70 times：
This result may also remind us that when choosing a legal work partner, we need to consider scores, cost, and time together.
So how exactly was GLM-Legal trained?
The answer may lie in Zhipu's accompanying development pathway.
Not only making AI read law, but also taking it to intern at a "law firm"
Zhipu's growth roadmap for GLM-Legal can be summed up in one sentence:
First send it to "law school", then bring it to a "simulated law firm" to practice.
Step oneGLM-Lega will undergo mid-training on legal corpora on the GLM-5.3-Flash base, learning legal concepts, statutes, and analytical methods to build a solid professional foundation.
Step twoGLM-Lega then needs to go through reinforcement learning in tens of thousands of legal task environments, continuously practicing reading materials, performing analysis, advancing tasks, and delivering results.
At the same time, the team keeps iterating on the Agent runtime framework, improving material organization, tool connections, and feedback processes, letting the model and the working environment break each other in together.
Does this look familiar? If it looks familiar, that's the point!
Isn't this exactly the level-up journey we law students went through?!
First, one fights their way into a university law school and devours books frantically to build up internal strength, then in the third and fourth years of undergrad dives headfirst into a law firm as an intern workhorse, finally after enduring several years, or even continuing to climb the exam ladder, only then can one evolve from a nobody into a reliable lawyer.
What AI is going through now is exactly the path we once walked!
It's just that GLM-Lega is still in the relatively green intern workhorse stage, so we can't yet expect it to reach partner-level right out of the gate.
After all, Zhipu's "legal specialist" still has many areas that need further testing; the 19.2% all-pass rate also means that under strict scoring conditions, most tasks have not yet achieved full compliance, and there is still plenty of room for improvement.
This corresponds to what Zhipu is doing now:
inviting law firms, legal tech companies, corporate legal teams, and independent practitioners to apply for the GLM-Legal Beta, letting professionals participate in trials, provide feedback, and jointly polish the model;
at the same time, Zhipu also plans to integrate these capabilities into AutoClaw to advance real-world applications.
Wow, this sends a clear signal behind the scenes:
Zhipu is bringing together a group of the most knowledgeable legal professionals to polish GLM-Lega, with a clear goal: to make it a truly useful auxiliary tool in lawyers' hands in the future.
As for whether Zhipu can actually realize this vision in the future, it's also worth looking forward to.
one more thing
In fact, Zhipu's entry into the legal model race is not a spur-of-the-moment decision.
As early as 2023, Zhipu partnered with PowerLaw AI, a legal tech company focused on smart contracts and other services, and jointly developed and launched a hundred-billion-parameter-scale legal vertical model based on the ChatGLM-130B frameworkPowerLawGLM, covering scenarios such as legal consultation, contract drafting, and review.
Since then, as an existing shareholder, Zhipu has continuously participated in PowerLaw AI's strategic financing and its Pre-B round of nearly 80 million yuan.
And Zhipu is not the only one sending a general-purpose model off to "study law".
As early as month 8 of this year, Harvey teamed up with Fireworks and, using Kimi K3 as the base, post-trained a legal model of its own —Tenet。
According to Harvey, after specialized training, the number of LAB held-out tasks it completed was nearly double that of the base model, while costs remained stable.
These explorations may also offer the industry some new insights, namely:
after choosing a good general-purpose base, continuing to train with specialized tasks, industry data, and work environments still offers the opportunity to push the model's capabilities one step further.
In the future, whoever can teach the model's "major courses" more solidly, build the "mock law firm" more practically, and train it to develop more skills usable in real work may truly win the recognition of this circle of users.
Reference links:
[1]
https://x.com/Zai_org/status/2109678174468534511?s=20
[2]
https://z.ai/blog/glm-legal
[3]
https://x.com/ValsAI/status/2109678921017168061
[4]
https://www.harvey.ai/blog/introducing-harveys-legal-agent-benchmark?utm_source=chatgpt.com
[5]
https://media.qimingpian.cn/2023/06/29/bd6b454d1bf2400827f682a6c5e7eacf.html?utm_source=chatgpt.com