Hey there! Welcome back to the second part of our ChatGPT comes to Glific! In case you missed it, in the previous post, we talked about the exciting integration of ChatGPT into the Glific platform, bringing in a whole new level of enhancing communication and engagement with beneficiaries. With the first version of ChatGPT out, it was time to focus on getting production ready and making preparations for launch as we move into the next phase. In this blog, We’ll be sharing how everything comes together and the enthusiastic preparations underway to introduce ChatGPT in WhatsApp chat through Glific.

With the integration of ChatGPT into Glific using the Jugalbandi APIs, we quickly jumped into the testing phase. Jugalbandi offers a variety of querying options, including GPT3.5, GPT4, Langchain, and GPTindex. As each model has its unique strengths, we had to make a decision and select the most suitable one. To aid us in this process, we collaborated with The Apprentice Project(TAP)  team to create three Glific flows for them to test out. Each flow used a different model, namely GPTindex, GPT3.5, and GPT4. To efficiently test these flows, we used Glific’s “Write to Google sheet” feature. This allowed us to involve a larger group in the testing process, while simultaneously capturing and saving every query asked and response received in a linked Google sheet. This data will later be analyzed to evaluate the accuracy and effectiveness of each model and help us in making a decision

The TAP team eventually decided to go with GPT3.5 after testing based on the pricing and the accuracy of the responses received.

While we were putting the integration to the test, we encountered a couple of bumps along the way:

1. Bot Freezing with Simultaneous Usage: One issue we encountered was the bot freezing when accessed by multiple users simultaneously. Since each API call takes approximately 20 seconds to respond, the bot would become unresponsive and fail to provide any output when multiple API calls were made concurrently.
2. Hitting Hourly Rate Limit: Another hiccup we faced was hitting the hourly rate limit. As the same Jugalbandi account is used across all APIs, heavy usage would occasionally trigger the hourly rate limit, restricting response from API call
3. Setting up Default Bot Behavior: In context of TAP program, where teachers and students interact with the bot, it became apparent that different sets of queries and answers were required for each group. To accommodate this, we needed a mechanism that allowed us to set a default behavior for the bot when conversing with students and a different one for the teachers.

As we could not afford these issues to carry forward into the production launch, we came up with the decision to host a separate instance specifically for The Apprentice Project (TAP), using their own OpenAI account credentials. This way we can mitigate the first two issues, ensuring a smooth and uninterrupted user experience. Additionally, we asked the Jugalbandi team to update an existing API, introducing an optional parameter known as a “prompt” that allows for customized bot behavior based on specific requirements.

We knew that sticking to a well-defined timeline was crucial to ensure everything was ready on time so we came up with one. Initially, the timelines seemed quite tight, considering the multitude of tasks that needed to be done within a limited time and required multiple teams to work interdependently, leaving little room for error. However, a big shout-out to both the Jugalbandi and TAP teams for their dedication and commitment to meeting the timeline with enough time to test everything before the actual launch

On June 26, 2023, with everything on track, the TAP team launched their program, incorporating the power of GPT integration into their program’s Registration flow. To configure this flow, the TAP team shared a document containing transcripts of previous interactions with the chatbot during registration which serves as a knowledgebase and is integrated into their registration flow. As a result, teachers can easily ask questions concerning registration, and using ChatGPT, response is generated by leveraging the knowledgebase. The teacher then receives this answer in a timely manner which aid them during registration. It’s an incredible fusion of technology and human interaction that empowers teachers and streamlines the registration process.

As the registration process continues, it’s only natural to encounter a few bumps along the way. We believe in being transparent about the reality of the situation, acknowledging that there is still room for improvement. As we continue to refine things on our end.

At the same time, we have been working closely with the TAP team to prepare for their second phase of the launch in the following weeks. In second phase, Students will be sent weekly assignments that are personalised for their chosen courses, skill levels current week of the program, offering classes in everything from financial literacy to STEM, digital, and the arts. TAP team envision that by leveraging the capabilities of ChatGPT integration, they can provide students with personalized support and guidance throughout their learning journey.

To prepare for phase 2, we needed a test flow to explore the possibilities and assess the potential outcomes. The TAP team shared five transcripts of the activity, which served as a knowledge base for setting up the test flow. We merged these five transcripts adding appropriate labels and headings into a single document to ensure it is easy to understand for GPT when setting up a knowledge base.

Furthermore, we recognized the importance of providing GPT with additional contextual information to generate tailored responses. To achieve this, we stored contact level information in contact fields for our test users. Subsequently, we incorporated this information into the user query(query_string), effectively supplying GPT with relevant details that would enhance its ability to formulate customized responses, and Voila!

Thus, when contact with preferred activity as “Art Activity” and level 1 asks the question

Similarly, contact with preferred activity as Coding Activity and level 1 asks questions and receives a different response

Through testing, we were able to effectively show that using a combination of Flow and ChatGPT integration could actually meet the TAP’s needs. This milestone validated our approach and confirmed that we were on the right path. With each iteration, we move closer to our goal of providing an innovative and effective communication tool

In conclusion, our journey to production and the imminent launch of ChatGPT on Glific has been an exciting ride filled with countless hours of hard work, collaboration, and innovation. We are grateful for invaluable support from our community that have propelled us forward and thrilled about the future and the endless possibilities that lie ahead.

Get ready to witness some serious AI-powered magic!