This blog is more of a journal of experiments we have done so far with AI (using OpenAI APIs mainly) in Glific, how things are right now, and things we are looking forward to in the future.
We have spent a significant amount of time in April in bringing a tighter integration of LLMs into Glific so it can aid NGOs in their program that are running in Glific
Overnight Infusions: Things We Did in the Past
Last year, we partnered with The Apprentice Project (TAP), which has been using Glific to deliver weekly activities to thousands of school students over WhatsApp. Using a chatbot for their program, they wondered about the chatbot’s limited interactivity and wanted to mitigate this to increase interaction in their program. As we realized that configuring ChatGPT right from the box does not adequately address the problem, we opted for Jugalbandi.ai, which allows users to upload documents and ask questions via API, so we hooked these APIs directly to Glific. This upgrade made it easier and quicker for the bot to handle and reply to user questions, really stepping up the chatbot’s game.
Read more about this here
- https://glific.org/chatgpt-comes-to-glific/
- https://glific.org/chatgpt-unleashed-journey-to-production-and-preparing-for-launch-on-glific/
Next, we hosted a sprint in October in Kochi for a deep dive with our Glific NGO partners on generative AI, along with the Agency Fund. In our early experiment with TAP, we kicked things off with Jugalbandi, a decent starting point for ChatGPT and generative AI exploration from NGO point of view. It gave them a chance to run a small pilot, see how it felt, and figure out if it’s their cup of tea. But as we start making more progress in this and talk with a few other NGOs, we come across a few other amazing capabilities of GPT, like function calling, and retaining conversation memory for better context-aware answers, exactly what many NGOs are after. So we chose to work on our own LLM project because these aspects match what many NGOs require. During sprint, we were able to setup a POC and started looking for NGO partners who could adopt this so we could validate the POC and decide on the next steps.
It works in two parts:
Uploading knowledgebase: Uploading few foundational knowledge documents, which can then be used the model to generate answers
- Upload a document using APIs
- The document is read, and its content is split into smaller chunks.
- These chunks are converted into embeddings using OpenAI’s APIs.
- Finally, the embeddings are saved in the database.
Generating response: Once the knowledge base is in place, the system uses this vast pool of embeddings to generate accurate and relevant responses to the queries it receives, ensuring each interaction is as informative and helpful as possible
Bubbling Brew: Current State
In March 2024, we talked with Udhyam, who wanted to experiment with LLM in their program, SBIC, which focused on cultivating an entrepreneurial mindset among school students. Currently, both students and teachers face challenges due to an insufficient support system. Students frequently ask business-related questions, but teachers and mentors, limited by availability and capacity, often struggle to provide satisfactory answers. This leads to unresolved queries and frustration, further exacerbated when teachers seek help from peers or the overwhelmed core team. Thus, WhatsApp chatbot designed to assist with these unstructured queries efficiently can be a potential solution.
Based on our conversation with them, we came up with MVP (the minimum viable product) for LLM that we can work on in April so starting May, the Udhyam team can test it and see how it fits their program
Read more about it here Glific_OpenLLM_Version 1
We recently wrapped up these deliverables and finished the MVP. Now, we’ve integrated LLM into Glific, allowing an organization to manage its knowledge base and use it to answer questions from beneficiaries. They can do this by creating flows to support their programs running on WhatsApp through Glific. Here’s how it works:
Get LLM4Dev Credentials: Before jumping into LLM4Dev, ensure your organization has OpenAI credentials ready to roll. These are essential to kick off your LLM journey. Once LLM4Dev is set up, you can add them from Glific Settings.
Set up Knowledge Documents: Before using LLM4Dev to create flows, you need few foundational knowledge documents, which can then be used the model to generate answers. Upload your PDF documents from Glific’s manage knowledge base section. Do add category, which helps it segregate different knowledge bases, which ultimately affects the response generated
Check Your Uploads: After uploading, you’ll be able to see the uploaded document in the knowledge base section along with category and category ID
Integrate LLM4Dev into Flows: Now, to integrate LLM4Dev into a flow, create a Call Webhook node and select “FUNCTION” from the dropdown menu. Enter “llm4dev” in the input field and add a result name. Go back to the manage knowledge base section and copy the category ID you need, Paste it into the function body tab of the webhook node along with prompt you want to use in the flow
The Next Cup of Brew
As we know, conversational AI is evolving rapidly, and LLMs (Large Language Models) are gaining traction for changing how we interact with technology. Many organizations have already started using AI in some capacity or are planning to, but it’s crucial to understand that AI isn’t a magical fix. It takes time and effort to fine-tune and improve efficiency along the way.
As mentioned in this blog, LLMs can be a bit unpredictable. Sometimes they give spot-on answers, while other times they spit out random facts from their training data. If they seem a bit lost at times, it’s because they are. LLMs understand words statistically, but they don’t grasp their meanings.
This is happening to us as well through our initial testing, where some of the time the model hallucinates and gives made up answers. That’s why we’re considering using the Retrieval-augmented generation (RAG) framework to enhance the quality of LLM-generated responses, so it will pull the most relevant embeddings to generate responses, Plus, we’ve got Jerome on board as the new AI head at Tech4Dev, so we’re starting discussions to make this happen in the next few months.
As for MVP, we had a call with the Udhyam team, and they will soon be putting it to the test, and will share their findings and observations. This qualitative data will help us fine-tune the model further. Stay tuned!
Cookie on the side
We’ve got another function that uses GPTVision APIs. It’s handy because it lets you use AI to analyze the quality of the images shared by beneficiaries, all within Glific Flow.
We’re still testing it out with our NGO partners to see if it helps them automate image analysis to some extent. Currently, ChatGPT by OpenAI only supports images, but we’re hoping they’ll add support for other types of media files soon.