Knowledge Assistant: grounding copilots in an organization's knowledge

Cat_Mackay

This is an innovation prototype that lives solely in Q Labs. Q Labs products are experimental and pre-roadmap, as such they are for awareness and co-innovation interest only.

Q Labs Status: Experimentation

Background 

A key problem for many companies is consolidating internal knowledge. This knowledge exists in multiple areas, from formal document sites and community blogs to code repositories and bug fix conversations. Searching for the right information can be a time-consuming task and, for less experienced employees, sometimes inaccessible.  

We aim to unify a company's knowledge, making it quickly and efficiently accessible. Copilots already provide a convenient chat interface people can use to find information, and have become an essential tool for enhancing productivity, supporting decision-making, and reducing the workload of their users. Whether you use them daily or believe they will take over your job, it is hard to argue with the change that this technology has brought about. But like most technology, copilots are only as good as the data they are working from, grounding this tech in context has the ability to unlock their true potential. Without a deep understanding of the specific environment, data, or nuances of the task at hand, even the most advanced copilot can falter.  

Combining the knowledge of a company with said copilot is an efficient step to solve this knowledge gap. Knowledge Assistant enables contextual knowledge to transform a copilot from a generic assistant into a trusted and effective collaborator. 

The Uses 

Following extensive research and feedback discussions, we have highlighted below some validated uses of this Knowledge Assistant alongside some of the expected benefits.  

Persona	Use Cases	Benefits
Support team: working in the support and triage space	A customer support ticket comes through regarding a new product for which no one on the team is an expert. They use Knowledge Assistant to find where the detailed documentation is and who made it in case they need further details.	Reduces response times to customers, supporting and improving KPI’s Reduces support teams' workload by supporting information collation and identification. Improve the diversity of information available on hand to support tickets Enhances support information response
Customer: a technical delivery person working and learning in Quantexa product.	A delivery engineer new to a project and working in a different time zone from the rest of the team. They find a bug that isn’t related to the libraries they have used before, and the support team has logged off for the day. They use Knowledge Assistant to search for information and instantly receive a high-level overview of what the bug could be and a link to a support conversation solving this bug for someone else. It also tells them what the acronyms stand for in this product.	Time save on accessing this information Give the ability to send a question and get an instant response in the IDE Reduce repeated queries and support tickets by providing instant access to self-help resources. Improvement in the user experience when looking for Quantexa specific information. Accelerate junior engineers learning
Internal Sales: support and content creation	A pre-sales person who has just received an RFP related to an area of the organization they are unfamiliar with. They use Knowledge Assistant to quickly access non-technical information on this area, including the relevant products they should include. They also ask Knowledge Assistant to include any RFP responses for similar questions from Salesforce.	Increase time to access data, efficiently accelerating initial responses Accelerate time to create new content by having consolidated materials Improve ease of creating new content Ensure tone of all sales content is the same
Content creator: product specific educational material	A content creator in the organization is looking to see which area of the product lacks educational materials. They use the statistics from Knowledge Assistant to discover that the topmost common questions all relate to a certain product. They then use Knowledge Assistant to create draft content based on these gaps using the writing style of the other educational material and the documentation.	Support identifying gaps in knowledge as well as gaps in documentation Accelerate upkeep of information in educational material Time to alter all content when change is needed Ensure tone of all content is the same

The Prototype So Far   

Knowledge Assistant is still in the prototyping phase of its innovation lifecycle. To start, we have built a copilot add-on that connects Quantexa’s documentation site and Community within GitHub’s copilot. These are two very distinct sources of Quantexa knowledge, so provide a good test to demonstrate how the LLM responds to this context. We utilize Microsoft’s Chat extensions feature, which allows VS Code extension developers to augment copilot in various ways, including introducing new participants (mentioned with @) This allows us to bring in additional data from other sources and pass them to the underlying LLM used within copilot.  

BEFORE Knowledge Assistant: 

AFTER Knowledge Assistant:

We use chat extensions within a VS Code extension to introduce new chat participants with 3 different modes:  

1. @knowledgeassistant : retrieves and combines knowledge from Quantexa docs and Community.  
2. @knowledgeassistant / qdocs: retrieves knowledge from Quantexa Docs using a custom RAG pipeline powered by ElasticSearch and LlamaIndex. 
3. @knowledgeassistant / qcommunity: retrieves knowledge from Quantexa Community using its underlying search API. 

This works exceptionally well, as showcased in the example shown below. Before Knowledge Assistant, the copilot had no knowledge of Quantexa-specific terminology, whereas after it not only clearly defines what this library does but also shares links to additional articles on this subject. 

https://player.vimeo.com/video/1062733355?badge=0&autopause=0&player_id=0&app_id=58479

Its Future 

As you can probably tell this prototype presents a world of opportunities and advancements. We are looking into expanding this to more data sources, utilizing analytics to detect repeated questions, spot areas requiring more documentation, and eventually adjusting how responses look for particular types of questions.  

If you would like to hear more about Knowledge Assistant, please get in touch with us. We are actively exploring different options for how to take it forward and are open to collaboration.

Please click here to get in touch and leave us any feedback