Report and blog is put together by the Civis team, contact atharva@civis.vote for more information on the details shared.

Introduction

In the rapidly evolving digital age, financial influencers, commonly known as ‘Finfluencers’, have emerged as significant players in shaping public opinions and decisions related to investments, insurance, and other financial ventures. Platforms such as Instagram, YouTube, LinkedIn, and Twitter have become hotbeds for such influencers, who often claim to provide informed advice on financial matters. However, with influence comes responsibility. The Securities and Exchange Board of India (SEBI), recognizing the profound impact of these Finfluencers, sought to introduce regulations ensuring that their influence is wielded transparently and responsibly.

To facilitate public understanding of and participation in the discussions surrounding these proposed regulations, Civis, an organization dedicated to enhancing transparency and fostering public participation, embarked on an ambitious project. Inspired by the successful AI-driven endeavors of NGOs such as TAP, Bandhu, and Myna Mahila, and with the support of grants like the one from CISCO, Civis envisioned an AI-backed solution to enhance public understanding of SEBI’s proposal.

Partnering with Glific, the goal was to craft a chatbot that could unravel the complexities of the consultation paper for the layman. This report documents this ambitious endeavor, capturing every facet of the journey, from the intricacies of AI training to the feedback and challenges encountered.

Spanning a concise window from 7th August to 15th August 2023, the chatbot served as a beacon of information and guidance. As the 15th September deadline approached for responses to SEBI’s consultation paper, the integration of the chatbot aimed to empower the public, ensuring informed voices were heard.

This comprehensive report endeavors to provide a panoramic view of this experiment, shedding light on the power and potential of integrating AI in public consultation processes.

Background

The term ‘Finfluencer’ has rapidly gained traction in today’s digital lexicon. These individuals, primarily active on popular social media platforms, provide insights, tips, and advice on an array of financial topics, ranging from investments in securities to real estate. The Advertising Standards Council of India (ASCI) aptly defines an influencer as someone who possesses both access to a substantial audience and the power to sway their purchasing decisions, largely due to their perceived authority, expertise, or rapport with their followers.

However, SEBI has raised pressing concerns regarding the unchecked proliferation of Finfluencers. The primary worry stems from the realization that not all of these influencers possess the requisite expertise in the subjects they discuss.

Additionally, in the absence of a regulatory framework, there’s no obligation on their part to disclose potential conflicts of interest, raising questions about the transparency and authenticity of their endorsements. This landscape painted a compelling case for the introduction of regulations and guidelines to ensure both accountability and transparency.

For a visual deep dive into the chatbot and the consultation, we invite readers to watch our video on YouTube, providing a comprehensive understanding of our endeavor.

Chatbot Development

In an era dominated by instant communication and digital interactions, Civis recognized the power of a chatbot in bridging the information gap. Collaborating with Glific, a renowned chatbot platform, the team embarked on the journey to develop a chatbot tailored to elucidate the intricacies of SEBI’s consultation paper on Finfluencers. The technical backbone of this chatbot was the Jugalbandi GPT 3.5 custom prompt model, a cutting-edge AI-driven tool known for its precision in generating contextually relevant responses.

The development process was meticulously orchestrated. Leveraging Jugalbandi’s swagger UI, the chatbot was trained using a curated dataset, which included the consultation paper’s summary, Civis’s content on the consultation, and the original draft. Throughout the development phase, the guidance and expertise provided by the Glific team proved invaluable, ensuring that the chatbot was not only robust and accurate but also user-friendly and engaging.

Chatbot Performance Metrics

The effectiveness of any digital tool is best gauged through tangible metrics, and the Civis chatbot was no exception. Operational for a concise window from 7th August to 15th August 2023, the chatbot witnessed a commendable user engagement. A total of 32 users interacted with the bot, posing 108 queries in total. Impressively, the AI-powered chatbot managed to satisfactorily address 89 of these questions, underscoring its robust training and capabilities.

Delving deeper into the numbers, the average engagement time stood at 1 minute and 41 seconds, indicating a keen user interest in the consultation topic. With an average of 3.08 questions posed per user, the chatbot effectively fielded a diverse array of inquiries. A key performance indicator was the Civis platform’s direct visits from the chatbot, which totaled 26 during the engagement window. It’s noteworthy that the 12th of August saw the highest user engagement, while the Civis home page registered 22 visits in the same time frame.

Chatbot Interaction Flow

The user experience is paramount in any digital interaction, and the chatbot’s flow was meticulously crafted to ensure clarity, engagement, and ease of use. Upon initiating a conversation, users were greeted with a welcoming and informative message that succinctly introduced the topic of the ‘Finfluencer Public Consultation’ and emphasized its significance.

This introductory message set the stage, elucidating the crux of the consultation and prompting the user to explore further.

Subsequently, users were presented with a choice, allowing them to either inquire further about the ‘Finfluencers’ consultation or to directly submit feedback. Tailoring the experience, those opting to pose questions were encouraged to delve deeper, with the chatbot poised to provide answers sourced from its training on the Jugalbandi GPT 3.5 model. If a user’s query veered outside the chatbot’s knowledge base, they were gracefully redirected to the Civis platform for a more comprehensive discussion.

Conversely, users who chose to submit feedback were provided a platform to voice their opinions, reinforcing the essence of public participation. The interaction culminated with an appreciative message, emphasizing the value of their input and steering them towards the Civis platform for continued engagement and to further amplify their voice in public consultations. The flow was not just a testament to technological prowess but also showcased a deep understanding of user behavior and preferences.

Chatbot’s Response Mechanism

The efficacy of a chatbot lies in its ability to interpret user queries and respond accurately and promptly. The Civis chatbot, built atop the Jugalbandi GPT 3.5 custom prompt model, was primed to handle a wide array of questions, especially those that pertained to the nuances of the ‘Finfluencers’ consultation.

Upon receiving a user’s query, the chatbot sifted through its trained dataset to identify the most relevant and contextually apt response. Given its extensive training on the consultation paper, Civis content, and the original draft, the chatbot was well-equipped to address the majority of questions. A staggering 89 out of 108 posed queries were aptly addressed by the AI.

However, it’s inevitable that some questions might fall outside the chatbot’s trained knowledge base. In such instances, the chatbot had a built-in mechanism to acknowledge its limitations. Instead of leaving the user without direction, they were advised to visit the Civis platform for a more in-depth discussion, ensuring that the user’s quest for information was never stymied. This dual approach – providing direct answers when possible and redirecting when necessary – ensured that users always felt guided and valued.

Challenges & Feedback

While the Civis chatbot showcased an impressive performance, the journey was not devoid of challenges. One of the most notable issues encountered was the occasional glitch in the chatbot’s response mechanism. Specifically, for three distinct questions, the chatbot outputted an erroneous response, “@results.jugalbandi.answer”, which understandably might have confused or deterred users. Such glitches underscore the intricacies of AI-driven systems and emphasize the need for continuous refinement and feedback-driven enhancements.

cja

The integration of the Civis chatbot for the ‘Finfluencers’ consultation represented a fusion of technology and public participation. In a brief engagement window, the chatbot facilitated informed discussions, clarified doubts, and garnered valuable feedback, highlighting the transformative potential of AI in public consultations.

While challenges were encountered, they also offered valuable insights into areas of potential enhancement. With a focus on continuous improvement, grounded in user feedback and technological advancements, chatbots like these can pave the way for a more engaged, informed, and participative citizenry in the future. The Civis experiment with collaboration with Glific stands as a testament to the power of innovation in fostering transparent and inclusive public dialogues. ____________________________________________________________________________