// DISCOVERY //

What are some of these core user journeys?

For a finance app (60% of Plotline's user base)

Lending money

Products such as personal, housing, auto loans and credit against investments, P2P lending

Investments

Securities and fund investments

Add/Withdraw money from wallets

Adding money and retrieving money for use across the platform

Why are users dropping off?

To probe further

How is the hidden cost and conditions data currently communicated? Where does the source of truth sit?

The fall-out: What are users doing after dropping off?

How this impacts

Adds to the operational overhead of not only costs but also time it takes to resolve queries and nudge user to resume the journey

To summarise, impactful problem areas are

1. App usage has a learning curve - FAQs and delayed support is inadequate

2. Current In-app interventions are static - one-dimensional. They just communicate what product owners feel the doubt is, not covering every organic query

3. Missed cross-sell opportunities due to delay in capturing user intent

“I am using Plotline's nudges, they are impactful and have led to an uplift of 10-20% in key journey completions, but there is still a lot of information not getting passed through to end users, causing delay in completion”

Rishabh

Growth team, Dream11

Let's take an actual user journey - applying for a loan in a fintech app.

1. User starts the loan application process

2. Has queries about the loan and doesn't build enough confidence

3. Raises support ticket or sales team reaches out - delay of ~1-2 hours on avg for resolution

4. Users drop-off and only those who feel confident enough apply for loan

This same journey with a real-time intent recognition and communication system

1. User starts the loan application process

2. Has queries about the loan and doesn't build enough confidence

3. Discovers missing information through context-aware live conversations

4. User feels confident enough and applies for loan without delay

// PROBLEM DISCOVERY //

What would these conversational agents do inside the app? What should we design for?

We asked product and growth teams at these apps to know where drop-offs were happening in their apps and which of these avenues could be best solved by conversational agents.

1. Providing generally available information at drop-off points

2. Highlight products for cross-sell and up-sell

3. Hand-hold users to complete workflows by guiding them

// PRIMARY RESEARCH //

Understanding existing workflows, uncovering any latent aspirations and concerns for a conversational AI agent inside their app

User interviews

To understand their challenges better, we started 1:1 semi-structured interviews which gave us insights into user needs and pain points, expected experiences from end users and what is the current experience provided to them.

What are the main customer interactions within your app that could benefit from AI-powered conversations (e.g., customer support, product recommendations, order tracking)?
How do you currently gather customer feedback, troubleshoot issues, and upsell products? Would a conversational agent be suitable for any of these?
How do you estimate the agent's impact in your app? (e.g., multilingual support, personalization, deep product knowledge)?
How important is AI-human handover in complex cases? What is your expectation of bot vs. human interactions?
What are your top concerns about integrating conversational AI agents? (Options: technical complexity, security and privacy, customer trust, handling edge cases, impact on brand, regulatory compliance)

“Over the past 2 years, we have seen that the time taken to resolve support tickets is inversely proportional to lifetime value of our customers"

ALETHIA TAN

SVP, Growth, Kredivo Indonesia

“An agentic experience inside my app should aid the overall discoverabilty and usage. It should intelligently understand when it is needed and what it should help with”

Rishabh

Growth team, Dream11

Key findings from interviews

From the outset, we were aware that this is quite a big paradigm shift in consumer apps as this transforms the app from one-way communication to a two-way context aware communication with end users. There were a lot of concerns from marketing/product leaders we talked to and thus I categorised all the concerns emerging from the primary and secondary research under 3 major themes

How will I train my system?

Learn in real-time to not answer any query with stale info

Should learn from feedback loops and every customer query to refine itself.

How will I test my system?

Will I be able to simulate real conversations?

Will I be able to test the latency of the system?

How will I deploy my system with confidence?

Escalation rules: when to switch from bot to human, how to route correctly.

AI must log interactions so humans can pick up without repeated context.

How might we

Design a system that enables marketers and product owners to understand user intent more closely and give generally available information to them in real-time

Why LLM based intervention could work?

1. LLMs are great at digesting information and forming unique responses

2. AI native solution can be hyper-personalised using vector DBs and provide real-time context for every user, thus understanding user intent more precisely

3. Can reduce the overhead cost on businesses and free up support for deeper queries

Configuring an agent - snapshot of the core UX journey

Setting up your agents

Since the whole concept of having an AI agent take care of your users' needs, aspirations and frustrations was new, I built a few pre-configured templates to help the marketers get started and explore in a low friction way.

Measuring success for this flow

It is very important to track the usability of a completely new product added to our core dashboard. Thus, we are closely tracking the performance.

Task success rate - Creation

Currently ~ 57%

Usability support ticket ratio

Currently ~ 45%

Time to first value

Currently ~ 8 minutes

How will I train my system?

Knowledge base

Collection of data points that the agent can retrieve as required such as FAQs, policy documents

Benchmark conversations

Marketers add benchmarks for the agent to learn and give addition feedback in blind testing

A dynamic, highly relevant knowledge base is essential for any agentic system to train and learn.

"Can I tag the sources of information precisely to reduce bloat"

"But, I can instantly recognise an AI system is talking to me"

How will I train my system?
Let marketers simulate real conversations and build trust on the agent

Testing each agent

Here’s a snapshot of how we can simulate the entire conversation experience, rate previous conversations and help the agent learn exactly how it is supposed to communicate with your users.

"How will I simulate my user's conversation"

"Can I test agent's responses at scale"

"What is causing latency in replies"

How will I the system with confidence?
Giving enough context and situation handling directions to the agent

Context broken down into communication styles, conversation guidance & escalation and hand-overs

Context broken down into communication styles, conversation guidance & escalation and hand-overs

Making sense of it all
Setting up iteration and improvement for agents

Agent performance broken down into actionable intelligence

Started by focusing on core metrics such as conversation volume, goal completion rate, and human handoff rate (when users are escalated to live agents). Real-time conversation logs, knowledge and tools performance also help in targeting the agent better.

Key learnings & next steps

Key UX decisions and learnings

Conceptualising how to build modular agentic experiences for platforms like Plotline, for marketers from leading consumer apps and visualising experience for their end users was a great opportunity to understand and design for

Building conversational agents in a modular way

Breaking the whole process into functions such as context, knowledge, tools & actions ensured a very gradual learning curve

Segregating what building blocks are global and what are at agent-specific

Centralising appearance, communication and brand guidelines reduces the potential for inconsistent experiences

Building trust and traceability into every AI decision

Breaking down every decision in every interaction solves for trust at a scale of millions
Building unbiased testing and learning flows for agent's training

Improvements & next steps

Working on agentic experiences opens a whole world of possibilities. For the next versions, ideations and concepts have already started!

Building 1:1 testing simulator

Will help simulate conversation with real time performance tracking for different users and use cases

Improving handover flow to manual agents

Giving more visibility into agent handovers and escalations.

Introducing new access points and interactions

Access points like floating buttons, pinned banners, gestures like long hold, bottom swipe can be introduced

Can users talk to my app in real-time and get their doubts and queries resolved? How should we guide users to complete key actions without asking for support?

Conversational AI builder for consumer apps

Conversational AI for consumer apps // Can AI actually help users in-context?

5M+

5M+

~20%

~20%

~30%

~30%

~40%

~40%

What does Plotline do?

What does Plotline do?

What does Plotline do?

Initial problem space

Users are dropping off from my core journeys. How can I plug this gap?

Who is getting affected

Who is getting affected

Why does it matter

Why does it matter

Next steps

Next steps

// DISCOVERY //

What are some of these core user journeys?

For a finance app (60% of Plotline's user base)

For a finance app (60% of Plotline's user base)

Lending money

Lending money

Investments

Investments

Add/Withdraw money from wallets

Add/Withdraw money from wallets

Why are users dropping off?

To probe further

To probe further

The fall-out: What are users doing after dropping off?

How this impacts

How this impacts

To summarise, impactful problem areas are

1.

1.

App usage has a learning curve - FAQs and delayed support is inadequate

App usage has a learning curve - FAQs and delayed support is inadequate

2.

2.

Current In-app interventions are static - one-dimensional. They just communicate what product owners feel the doubt is, not covering every organic query

Current In-app interventions are static - one-dimensional. They just communicate what product owners feel the doubt is, not covering every organic query

3.

3.

Missed cross-sell opportunities due to delay in capturing user intent

Missed cross-sell opportunities due to delay in capturing user intent

1.

1.

User starts the loan application process

User starts the loan application process

2.

2.

Has queries about the loan and doesn't build enough confidence

Has queries about the loan and doesn't build enough confidence

3.

3.

Raises support ticket or sales team reaches out - delay of ~1-2 hours on avg for resolution

Raises support ticket or sales team reaches out - delay of ~1-2 hours on avg for resolution

4.

4.

Users drop-off and only those who feel confident enough apply for loan

Users drop-off and only those who feel confident enough apply for loan

1.

1.

User starts the loan application process

User starts the loan application process

2.

2.

Has queries about the loan and doesn't build enough confidence

Has queries about the loan and doesn't build enough confidence

3.

3.

Discovers missing information through context-aware live conversations

Discovers missing information through context-aware live conversations

4.

Conversational AI for consumer apps //
Can AI actually help users in-context?