If you announce that something is elegant or classy, it probably isn’t.
Another day, and more announcements by legal tech vendors of some fancy new Gen AI tools that will save the profession, if not the world. It’s a dizzying array of claims and promises. I received four email Gen AI announcements from legal tech vendors just today.
And for most of us, there is no way to really evaluate whether the tools will do what the vendors claim. Or whether the claims are basically being offered to show the vendor is keeping up with other vendors. It’s a full-blown hype cycle.
Of course, for those of us in the media, it’s hard to validate the claims being made. With consumer products–like say smartwatchs–we can play with the products and put them through the paces to see if they will do what the seller claims.
But with legal tech tools, lawyers and legal professionals have to use them in a legal setting to know if they will do what the vendor says. They have to be applied to actual legal problems to know whether they really work. And most lawyers and legal professionals aren’t likely to take the time to provide the kind of review that might enlighten others.
So, it’s an area that’s ripe for manipulation and exaggeration. But even with consumer-oriented products, the Gen AI arms race yields claims and demonstrations by well-known vendors that aren’t quite what they seem.
Last week, for example, Google announced the release of Gemini. Gemini is touted as an advanced Gen AI platform that can produce written, pictorial, and audio content. It’s claimed to have an easy to use interface. Google even released an entertaining demonstrative video to the public and media designed to trumpet the product.
The video is roughly six minutes long and demonstrates what appeared to be a fantastic product. It demonstrated spoken conversations between a user and the platform. It also purports to show Gemini’s ability to recognize pictures and physical objects. For example, it showed Gemini’s ability to recognize a drawing of a duck while the user was drawing it. It then recognized a rubber duck that the user showed it. All the while, the user was talking to Gemini, and Gemini appeared to be responding in real-time.
The video also showed Gemini recognizing that a user wanted to play the game rock, paper scissors with Gemini as the two of them talked out loud to each other. Gemini says, “I know what you are doing, you want to play rock paper scissors.”
The only problem was that after release and several media inquiries, Google fessed up. It confirmed to Bloomberg that the demo wasn’t conducted in real-time as it may have appeared. In fact, it was all done with still images and text prompts. In reality, Gemini had to be provided with several pictures and prompts before it figured things out. Even the voice in the video was being read from a script.
The reality is quite different from what the video seemed to suggest. Ultimately, Google told CNBC, “The video is an illustrative depiction of the possibilities of interacting with Gemini, based on real multimodal prompts and outputs from testing. We look forward to seeing what people create when access to Gemini Pro opens on December 13.”
In other words, it was mostly hype about some things Gemini can’t do. At least not yet.
Like vendors in the legal tech space, Google is in fierce competition with other Gen AI providers in the nonlegal space. All of this goes to show that the claims of vendors in the consumer space need to be taken with a grain of salt. This skepticism is even more important in legal. Legal has traditionally not seen the kind of competition among vendors and the hyped claims we are seeing now.
We have already seen legal tech vendors offering grand press releases and announcements about their Gen AI products. But all too often, we find that the products don’t yet exist or can’t yet do what they claim.
Think about it: if vendors in the consumer product space are playing a little fast and loose with the state of their products, imagine the mischief that legal tech vendors might be up to. Unlike the consumer space, where claims are easy to verify, legal tech products must be tested in a professional setting by professionals with real problems before you know whether they can do what is claimed.
So, for lawyers and legal professionals looking at GenAI products, remember the words of Sgt Phil Esterhaus in the old TV show Hill Street Blues: “Let’s be careful out there.”
