The Apprentice Project (TAP) is a non-profit organization that works towards empowering underprivileged children (ages 11-17) through WhatsApp chatbot (TAP Buddy)  offering self-learning electives in Arts, STEM, Digital and Financial Literacy, fostering 21st-century skills. They have been our partner NGO for more than 2 years and were among the first to use Glific.

Presently, TAP uses Glific to deliver weekly activities to a vast student community of over 9,600 via WhatsApp. While Glific has proven to be effective in sending messages to beneficiaries at a large scale, one of the challenges encountered while scaling is the limited interactivity of the chatbot, which relies heavily on the program’s design. Consequently, addressing individual user inquiries becomes increasingly challenging. Although a Help flow can be set up to handle FAQs, the responses provided by the flow are confined to its predefined design, making manual resolution of user queries impractical.

Recognizing this challenge, TAP wanted to use Natural Language Processing (NLP) and Machine Learning (ML) to provide accurate responses to user queries. They expressed their interest during one of our sprints to integrate this solution into Glific and test it with their extensive user base. This led us to start our research in early May, while also exploring the potential integration possibilities with ChatGPT. Since its release ChatGPT has taken the world by storm by revolutionizing the way people interact with technology. Its advanced natural language processing capabilities have enabled it to understand and respond to human queries and requests with remarkable accuracy and speed.

Through our initial finding, we realised that configuring ChatGPT right from the box does not adequately address the problem at hand as GPT primarily relies on its preexisting knowledge base, and the relevancy and contextual accuracy of its answers may vary. Thus we needed to enhance the existing knowledge base by fine-tuning the model with TAP’s data so that user queries are resolved based on their specific data. So we continue exploring openai documentation and other blogs to get an idea of how we can do it. Few good resources we came across
How to customize LLMs like ChatGPT with your own data and documents
How To Build Your Own Custom ChatGPT With Custom Knowledge Base

Based on our research, we have come up with two possible solutions for integrating ChatGPT into Glific:

1. The first solution involves setting up and hosting a dedicated application that generates embeddings and stores them. These embeddings are then used along with user queries to feed into ChatGPT and create response. Then through APIs this app can be integrated into Glific.
2. Alternatively, we can explore existing solutions and integrate them into the platform.

While the first option grants us greater control over the system, but it also dramatically extends the timeline because it will take time to develop and host the app. On the other hand, opting for the second solution allows for faster integration through APIs as we leverage established solutions, but it relinquishes some control as we rely on external APIs and responses. Ultimately, the decision rests upon striking a balance between control and timeline considerations.

As we were weighing our options on which solution to go with, we stumbled upon Jugalbandi.ai. They offer features such as intent recognition, entity extraction, sentiment analysis, and chatbot integration. But the best part? they also provide APIs that developers can use to incorporate natural language understanding capabilities into their applications.

Their APIs are fairly easy to understand and use and all our work reduced to two API calls

Exploring the capabilities of Jugalbandi APIs, We initially tested the APIs using Postman and were blown away by the responses we received! The results were so impressive that we couldn’t resist integrating it directly into Glific. We used webhook to use Jugalbandi APIs from within the Glific flow. We could now effortlessly call Jugalbandi APIs to process user replies and send back responses, creating a smooth and efficient user experience.

With the development phase complete, we shared this exciting integration with the TAP team and requested them to provide us with data related to their weekly activities. This data then will be used to generate a customized knowledge base for TAP, enabling them to test the integration and evaluate the accuracy of the responses. While waiting for the TAP team to gather the necessary data, we decided to conduct a small experiment of our own. We fed Glific’s help docs into the Jugalbandi APIs, to create a customized knowledgebase for Glific’s support. We then set up the flow and tested it internally by asking questions around Glific and saving user queries and response received in a Google sheet, while also rating the received responses.

Jugalbandi presents a range of options for querying like GPT3.5, GPT4, Langchain, and GPTindex. While GPT-4 offers more factual results, making it particularly useful when dealing with facts and actual textual information, GPT3.5 is preferred for enhanced outcomes in terms of natural language understanding. So we opted for GPT3.5 and after testing with over 50 questions with an accuracy of over 80% we were absolutely amazed by the results of our test.

Kudos to Saurabh Karn and his team at Jugalbandi for their amazing work.

While we are making progress each week, stay posted to know more about how everything turn out to be. We are already excited how this will turn out to be and actively seeking for more NGO partner to volunteer for testing out this feature as we strongly believe in its immense potential for the future, and we are excited to explore the possibilities it holds.