In our ongoing commitment to enhancing user experience, we’ve introduced a groundbreaking solution to streamline the support process for our product, Glific. As a platform that thrives on community interaction, we leverage Discord to address user queries efficiently.
Initially, our users were encouraged to utilize the forums channel on Discord to post their queries, allowing our support team to respond promptly. However, as we delved deeper into user interactions, a pattern emerged — many queries were simply seeking information already available in our documentation. This realization sparked a two fold approach: improving our documentation and finding a way to seamlessly redirect users to the relevant information.
Simultaneously, our team was engaged in Large Language Model (LLM) experiments with Glific, generously supported by Jugalbandi through free credits. One of the things that we were trying to do was to assess the accuracy of the LLM in providing responses based on the context of given documents. While manual testing and simulated questions proved insightful, the process was labor-intensive.
Connecting the dots, we envisioned leveraging the power of GPT to provide context-specific responses directly in the Discord forum. Drawing on my experience in developing Discord bots, I dove into the documentation to explore event triggers for forum channels.
The design of the bot unfolded as follows:
Event Trigger: New Forum Thread
- Upon the creation of a new thread in the forum, the bot captures the first message.
- This message is then fed into GPT to generate responses, which are subsequently displayed on the forum.
The implementation yielded promising results in some cases but revealed limitations in others.
Acknowledging the need for evaluation, Tejas took charge of assessing the bot’s overall performance. Discord chats were shared, and Tejas, along with Sangeeta, meticulously analyzed the interactions to get insights.
In a collaborative effort, Tejas experimented with different GPT models and stumbled upon success with GPT-4 using a custom prompt. The prompt served as a comprehensive guide, providing the bot with a detailed understanding of its role as an NGO Support bot. This identity was crafted as an intelligent AI assistant designed to aid NGOs utilizing Glific’s chatbot platform on the Discord channel.
The custom prompt included specific guidelines:
- Solicit the flow name once for improved responses.
- Identify keywords indicating specific issues with Glific’s features or services.
- Fetch search results relevant to identified keywords in NGO queries.
- Construct responses solely based on search results.
- In case of insufficient information, redirect users to Glific’s documentation webpage.
- Respond in a language comprehensible to a 21-year-old.
- Maintain responses within the 750–1000 word range.
- Include only the most relevant link to Glific’s documentation in the response.
With these refinements in place, we’re eager to evaluate the bot’s performance over time. Initial impressions suggest a marked improvement over the previous version.
As a next step in our evolution, we integrated the bot with a Google Sheet, consolidating queries and responses for a comprehensive view of accuracy trends over time.
In conclusion, our journey with the Glific GPT Bot represents a significant stride toward efficient and context-aware support on Discord. By harnessing the power of AI and continuously refining our approach, we aim to provide users with an unparalleled support experience. Stay tuned as we unveil more innovations on this exciting path of technological integration.