How Large Language Models Can Help Your Search Project

The primary scope of this book is to communicate the current state of the art of large language model applications in the domain of information retrieval and search with a pragmatic perspective on industrial adoption via open source software.

To this end, the book is organised in three parts: “Large Language Models” gives an introduction to artificial intelligence and large language models, including an overview of open source and commercial options. Next, “Large Language Models and Search” describes techniques and strategies to integrate large language models in search projects, including how to choose the right model for a specific use case and how to avoid the classic mistakes that can happen in the process. Eventually, “How to Use Open Source Software to Interact with Large Language Models” gives an overview of open source technologies to interact with large language models and gives a detailed survey of how the most popular open source search engines support them.

The book lays the foundations, deeply analyses the building blocks and shows examples how to implement the ideas described. It highlights both the positives, negatives and possible mitigations of the limitations. This way, it caters primarily software engineers, data scientists and practitioners in artificial intelligence or information retrieval who are curious to learn about the latest trends, research and industrial applications related to search and large language models.

Alessandro Benedetti is R&D Software Engineer and Director at Sease Ltd. in London, UK. He also acts as an Apache Lucene committer, Apache Solr committer and chair of the Project Management Committee (PMC). His focus is on R&D in Information Retrieval, Information Extraction, Natural Language Processing, and Machine Learning. He firmly believes in Open Source as a way to build a bridge between Academia and Industry and to facilitate the progress of applied research.