News / All News / Article

Penguin Geeks | Unveiling the Intelligent Document Assistant: An Intelligent Experience Powered by Large Models!

Article images may contain Chinese text.

Image


As the information age advances, enterprises have accumulated large volumes of digital documents containing a wealth of information. To better manage and use these documents, they typically categorize them with tags and use search engines to retrieve their content. However, traditional document searches are often limited to keyword or phrase queries, making it difficult to achieve more advanced semantic retrieval and meet users' needs for an in-depth understanding of document content.


To address this issue, large language models (LLMs) have now made significant progress in natural language processing, gaining capabilities such as semantic understanding, document summarization, information extraction, and logical reasoning. By interacting with users through conversational questions and answers, LLMs can understand user needs more accurately and improve the efficiency and accuracy of searches for relevant documents.


Through a detailed breakdown of the solution and experimental demonstrations, this article will guide enterprises in building an LLM-based document semantic search system to deliver a more intelligent and efficient search experience. To do so, we first need to understand the basic principles and architecture of LLMs and learn the techniques and methods for applying them to document processing. We can then develop a search solution suited to the enterprise's actual circumstances and validate and optimize it through experiments. Ultimately, we will be able to build an intelligent search system tailored to the enterprise, improve the accuracy and efficiency of document searches, and give users an entirely new search experience.


Broadly speaking, intelligent document search can be divided into four main components: document chunking and vectorization, vector storage, vector search, and semantic synthesis by a large model. The rationale for this four-part division starts with the characteristics of LLMs.


First, because of the particular requirements and time sensitivity of retrieval tasks, large models with tens or even hundreds of billions of parameters are rarely used directly for information retrieval. Instead, they are often used for auxiliary data processing to produce better coarse-ranking and fine-ranking models.


Second, it is difficult for LLMs to encompass knowledge from numerous specialized domains during pretraining. However, they are highly capable in the comprehensive use of natural language, making them particularly well suited to summarizing and synthesizing answers to questions based on relevant background information and then expressing the meaning in a way people can readily understand.


In addition, when using large language models for quantized inference, the amount of text entered in a single input is limited to control computational costs. Longer source texts therefore need to be split to fit within the maximum token limit, a process also known as text chunking. Once a document has been split, an effective way is needed to find connections at the semantic level that are relevant to the question. This process relies on the preliminary processing of textual information during a language model's pretraining stage, namely vectorization. After vectorization, a passage of natural-language text is mapped into a high-dimensional space that represents meaning. Similarity calculations within this space can identify content with similar meanings.


Therefore, to accelerate the entire retrieval process, the text to be searched is first chunked, vectorized, and stored. At retrieval time, vector similarity is used to select chunks containing relevant content, and that content is then fed into the large model together with the question for inference, as shown below:


Image


The steps above can provide basic semantic-level document retrieval, but performance optimization issues still arise in practice. For example, when content from multiple documents is stored in the same vector database, a vector similarity search for a question will return multiple document chunks. Determining the scope of relevant documents more accurately becomes particularly important when an enterprise's internal documents contain substantial amounts of related business information, common entity concepts, process standards, and similar material.


We first tried optimizing the system with a hierarchical search strategy. A large model can be used to preprocess documents by summarizing their main content and, optionally, extracting a list of keywords, creating a front section similar to that of a journal article. Before searching, we calculate the similarity between the question query and the summary information for all documents, retaining the top 1–n results. These results then strategically guide subsequent searches of document chunks, reducing the error rate in cross-document searches.


Traditional machine learning methods such as LSA and LDA can also be used to extract the main topics of documents. Subsequent processing is similar to the approach above: document similarity, calculated using the question query and the topic matrix, serves as a prior that influences the subsequent retrieval of document chunks.


Based on the design above and using the large-model inference APIs provided by OpenAI and Baidu's ERNIE, we implemented an intelligent document assistant, as shown below.


Sample Q&A on DDI collection operations:


Image

Image


Sample Q&A on company rules and policies:


Image

Image

The implementation above still depends on platform-level large-model APIs and requires an internet connection to access the models' data-processing capabilities, which presents compliance risks for enterprises with stringent information security requirements. Next, we will select strong open-source models, such as Alpaca, Vicuna, and GLM, to build an offline document assistant and conduct preliminary technical research on model evaluation, fine-tuning, and deployment. Stay tuned.