It seems as if every time we turn around, there is a new search system on the market claiming to use Artificial Intelligence and Machine Learning. Furthermore, these are going to replace our need for a traditional Boolean search system. Though there is no doubt these AI patent search systems continually improve, we have found it difficult to quantify their performance against the more traditional search approach. Additionally, it is challenging to figure out where the systems can add value in the traditional patent search process since each operates in a black-box fashion with varying features and transparency.
In using various search systems for our daily patent searching activities, we were able to compare and evaluate these AI systems. We discussed the results in our important whitepaper which addressed this revolution and provided five key takeaways:
- Professional patent searching has only marginally improved over time.
- Companies have introduced periodic innovations to address these shortcomings with limited success.
- Early semantic search engines, crowdsourcing, and offshoring have failed to improve quality.
- A few machine learning tools have demonstrated real progress, but not as substitutes for humans.
- Lasting innovations will blend the work of human patent search experts with the best artificial intelligence tools.
The question still remains – Friend or Foe? The answer comes down to your knowledge of the system being used and knowing when and how to use it most effectively.
Is it truly AI?
The use of an algorithm, such as creating a citation network, does not mean that there is AI in the system. Prior to subscribing to an AI tool, ask these important questions:
- What are the AI models used in the system?
- Does the system use supervised and/or unsupervised learning?
- How was the system trained? Was it trained in different technology areas?
- How is the system tested? What are the precision and recall metrics for the system?
Test the system before buying!
These systems are quite different than traditional Boolean search systems which are mostly differentiated by the data sources and user interfaces. You need to test the system with your typical use cases and compare the results with those generated from your traditional search methods. You want to know:
- How does the system determine relevance and how many “relevant” results does it return? A system that claims to return 80% of relevant documents in a returned set of 1000 is quite different than a system claiming to return the same 80% in a returned set of 250.
- Does the system’s definition of relevancy match with your definition of relevancy? This can be subjective, so you need to have a clear understanding of how you are determining relevancy.
- Is the system easy and intuitive to use? Is it easy to tweak your search?
Friend or Foe? How about Frenemy?
The right AI search system can be a good complementary tool and is most effective when used in conjunction with a standard search platform. We’ve found them to be most useful in the following situations:
- Researching the general state of the art
- When hitting a brick wall in your search process, they can help you uncover different and useful paths to explore
- To double-check your search results and avoid a surprise out of left field
As these AI patent search systems and the underlying technology change and improve over time, we are confident that they will become an increasingly important tool for the patent analyst. The less time the analyst spends searching through haystacks for those needles, the more time they can spend reviewing the references and applying them to the search at hand.
