Using design thinking insights to reduce user experience debt

Introduction

This study is an activity for the Design Thinking (DT) module of Hyper Island's Digital Management Masters program. Its objective is to identify opportunities for improving the current design process at eBay Classifieds Group. Over the past years, the design practice within the company has grown considerably, driven by adopting frameworks such as DT. These new ways of work introduced a more user-centric way of designing and connecting multiple stakeholders and end-users to solve a problem.

This study is composed of two main sections. The first one is a critical review of DT's potential in corporate environments and ethical considerations regarding customer-centricity. In the second, the learnings and insights of part one are used as a base for iterating in the current design process to become even more customer-centric.

Part One: The Impact of Design Thinking

In the past five years, design teams across eBay Classifieds Group have gone from only one or two designers to a team of fifteen or more. While still considered a small group compared to other areas such as development, they face the challenge of solving numerous complex problems. According to Hehn et al. (2020), DT is primarily intended to be applied in issues known as wicked problems. That is characterised by volatile and partially hidden constraints and requirements, making it the perfect framework to support digital transformation. Hehn et al. (2020) also affirm that the best benefits come from applying DT from the early stages of a project, helping clarify user needs and providing a deep understanding of the context and what can be achieved. This integration was not achieved at the company from day one. Teams had to work hard on bringing a DT-based approach from just a fraction of the process, to what today can be considered an integrated practice across the complete product lifecycle. Nowadays, every team that builds features for users has a triad of product managers, engineers and designers, balancing business, technology and user needs in everything built, which proves that design has conquered space within the company strategy.

Ethical considerations regarding customer-centricity

According to Berkun (2020), it's a challenge to evaluate how much a company is customer-centric, since it's just a label that an organisation can apply to itself. The lack of an official measure may lead companies to believe that their process is centred around the user, while in reality, they may be far from doing it and consequently may be missing a chance of becoming more successful. It's mostly in the hands of designers to help these companies become genuinely user-centric, by building and applying the correct methods and frameworks.

There are also situations where a company fails to achieve success via a user-centric approach. Designers can overcome these issues by tackling common misconceptions, mostly on what can be learned from proper user research. As an example, Berkun (2020) highlights a statement of Laura Ballay, former director of Carnegie Mellon University's Master of Human-Computer Interaction program, saying that: "Business goals and user goals are often two very different things." At first, this may drive companies to move away from a user-centric approach, since their findings may be perceived as unusable and not aligned with what users want. But with proper methods and processes, these insights can be translated into actionable users that need to be explored.

Another discouraging issue may come from the fact that users are unaware of the constraints a particular company may have, therefore their feedback may be invalid by the lack of background on how the company operates (Johannessen and Ellingsen, 2012). As a solution, Berkun (2020) recommends designers assess what a company is trying to achieve, by asking a series of questions to stakeholders (Fig. 1), which intends to change the focus from a solution mindset to a user-centric one, by putting the end-user in the centre of the discussion.

The author claims that this approach's main point is to avoid guesses and reduce bias towards stakeholders' ideas and preconceptions about the end-user. The author also recognises that such questions may come across the wrong way and can even be considered annoying by business stakeholders. But good designers should know that this is part of their job, and once a person answers these questions, it may help them understand that its purpose is to avoid jumping into conclusions and solutions too fast. For companies, such as eBay Classifieds Group, where these questions are already part of the way teams work, Berkun (2020) offers a different set of reflective questions (Fig. 2). These complementary questions aim to help designers evaluate a solution through ethical lenses, by promoting a reflection and discussion of its consequences between stakeholders.

User Experience Debt

The questions presented by Berkun (2020) may be a good starting point for everything there is yet to be built. But a different approach is needed for what is already done. The act of revisiting past product and design decisions, with the intent to fix previously created issues, is usually referenced as User Experience Debt (UX Debt). Kaley (2018) explains that this type of debt is caused, among other reasons, by the fact that designers and researchers are working under tight timelines or impractical project constraints. As a reflection point, the author suggests these professionals reconsider if this need for faster deliveries is worth the risk of negatively affecting the user perceptions of a digital product. Kaley (2018) also affirms that the best way to uncover these issues is by getting real user feedback, using various methods, such as diary studies, user testing and user interviews. Moffett (2016) recommends that UX Debt should not only be avoided but also reduced when possible, since these constraints can generate debt that may seem small at first, but over time, it builds up to the point that it may become a blocker for new solutions.

Part Two: Problem Situation in the Workspace

This proposal is directed to one of the markets that eBay Classifieds Group operates, the Dutch market and its local brand, Marktplaats. This decision was made because of its design team's size and high design maturity. Over the years, the Marktplaats design team has been continuously iterating in their design process, an established way of working based on the DT framework principles. The maturity comes from the fact that the process contemplates all foundational DT elements, from discovery to validation. The practice of user research, for example, is a routine within all product teams. There are weekly sessions available for any designer that wants to get a project tested with real users. A research specialist conducts this session and also compiles the learnings and findings. The person or team who requests is invited to watch remotely, and can only interact with the interviewee by the end of the session. Besides this option, ad hoc studies can be requested at any given moment, with the research team's support.

Recently, the team adapted a process visualisation created by the Zendesk design team (Fig. 3), called the "Triple Diamond" (Chen, 2020). The expectation is that this visualisation can help stakeholders from different backgrounds understand how the design team operates.

Another benefit was that this visualisation also helped determine that essential parts of the process have not been iterated in recent months (Fig. 4). This discovery led to the motivation behind this study, which aims to identify opportunities to improve our user-centricity efforts, by iterating in the two stages of the design process where designers receive direct user input: Problem Discovery and Concept Validation.

Methodology

The Marktplaats design team relies on collaboration and co-creation for its process iterations. Guided by a very similar process used to create product features, the team's needs are treated as user needs, and success is achieved by creating a solution that caters for these needs. The methodology for this study is composed of the following stages:

• Discovery: A debate and analysis of current issues in the process and its possible causes, as well as opportunities to make it better.
• Problem and scope definition: Clear and shareable definition of the problem to be solved.
• Ideation: Exploration of possible solutions for the problem.
• Prototyping & Validation: Simulation and analysis of solutions using a minimal viable approach.
• Refinement & Implementation: Application of the validation stage insights and learnings, followed by implementing the changes in the team process and workflow.

After the stage, an assessment of the possible next steps will be presented, followed by a roadmap of suggested actions to scale the process in the company.

Discovery

For the discovery phase, the author organised an online workshop to discuss the challenges designers were facing and map all the identified issues (Fig. 5). The session was conducted in a moderated panel format, where participants were asked to share their recent experiences with the process. The other members were then invited to share their opinion about the issues.

The main topics were listed and then prioritised based on urgency and impact. By far, the most voted one was the lack of a repository for secondary issues identified during a research activity. When performing a user test or a user interview, the team collects insights and learnings about the main topic, but it's common also to identify other issues with the product. Examples can range from a simple copy that makes a message confusing, to other more complex problems, such as navigation structure issues. And currently, these insights get lost since there is no proper place to document them. As learned from part one of this study, these findings are UX Debt issues, and creating a process to handle them is an opportunity to be more user-centric.

Problem and scope definition

The findings of the Discovery phase provided a clear path and scope. The team also opted to frame the problem using the recently discovered questions from Berkun (2020), also referenced earlier in this study. The main driver for this decision is to make sure this process will not generate further problems, and also to make the team's intentions clear to the rest of the organisation:

What are you trying to improve?
The collection and processing of secondary insights generated by user research methods, since currently, there is not a straightforward process or centralised location that they can be stored.

Who are you trying to improve it for?
We want to benefit users who face problems that are not yet known to our product teams, by fixing issues generated intentionally or unintentionally in the past. It also benefits designers, who can have a better view of the product issues.

How do you ensure you are successful?
By fixing the UX Debt issues. With a repository, identified issues become tangible, allowing success metrics such as the number of resolved issues.

Who might be hurt by your work, now or in the future?
If we don't act on these issues, we will make users take extra time to complete their tasks, or face problems that could block their flows. A direction that doesn't align with our objective of providing an effortless experience for buyers and sellers. It also directly affects the product and design teams by creating future blockers due to UX Debt's accumulation.

Ideation

The author organised a second workshop for this stage (Fig. 6), where the team explored ways of solving the problem. In this virtual session, the team had to collect and map recent user research findings that fit the UX Debt definition. To help the identification of these issues, we adopted the UX Debt categories described by Kaley (2018):

• User interface (buttons, links, and visual styling).
• Interaction (movement from page to page, progressive disclosure, etc).
• Copy, content, and messaging (labels, headlines, and written text).
• Information architecture (navigation structures and content classification).
• Accessibility (contrast, visual focus indicators, text alternatives, etc.).
• Customer journey consistency.
• Omnichannel seamlessness.

Besides identifying the category, all issues were separated by team ownership and received an estimation of size and complexity, based on other similar projects.

These issues were then moved to a UX Debt inventory prototype, using a shared spreadsheet as a minimum viable approach (Fig. 7). The team also identified the need for extra information, such as the task status and which designer would be responsible for the fix.

Prototyping & Validation

To validate the approach, the team decided to choose two of the priority tasks, so that the UX Debt process could be evaluated from the beginning to the end. The selected tasks differ in their complexity and platform to ensure that the process is being validated in different situations.

Task A: Web filters look like primary buttons

Size: Small
Type: User interface
Complexity: Low
Platform affected: Web

Problem: The design of the filter pills looks similar to the primary buttons on the page. This may be confusing to users, since filters are optional, but are currently designed with the same blue colour of primary buttons, an element reserved for the most important actions in a page (Fig. 8).

Solution: The task was then assigned to a designer that worked on a possible fix (Fig. 9), followed by a team review and development. The issue is now fixed and released to all platform users, fixing the team's first UX Debt.

Learnings

• The solution was an easy fix because the team has guidelines on how colours and UI elements should be used.
• A small fix like this one takes just a few hours to be made, indicating that dozens of these fixes can be done in a relatively short amount of time.
• Stakeholders got confused by the spreadsheet structure since it does not offer standard status options.

Task B: My favourites listings

Size: Medium
Type: User interface and Interaction
Complexity: Medium
Platform affected: iOS and Android

Problem: The favourites listing page on iOS and Android differ in features (Fig. 10), which causes a feature gap between the platforms. This is a problem because it makes fewer features available to iOS users compared to Android ones.

Solution: This task's approach was different from the previous one, starting with two designers assigned to work in a pair, due to the more complex scenario. To better understand how it could be fixed, the pair mapped and explored the current differences in interaction and visuals of the feature in both platforms. This task did not generate a solution. The duo decided to break it into smaller and less complex UX Debt tasks.

Learnings

• A time limit should be agreed for designers to work on UX Debt. The effort needed for this can quickly become a conflict of priority for those who work in different teams.
• To accelerate the process adoption and increase the chances of more fixed tasks, the team should prioritise only tasks considered small for now and move to bigger and more complex tasks in the future.

Refinement & Implementation

With the validation done, the team worked on iterations based on the previous phase's learnings and insights. No blocking issues were identified in the validation, but two main points would need to be solved to increase the project's chances of success:

UX Debt task template

The team expects a considerable amount of issues to be identified once the process is formally implemented. So it's essential to use a template to speed things up and standardise how UX Debt tasks are documented. Using a real-time collaboration tool, the team co-created different template versions (Fig. 11), that were discussed, refined and iterated into one final optimal version.

An integrated repository and task management tool

With the task template done, it was time to move to the workflow itself. With the product and development team's support, the UX Debt repository was formally implemented using Jira (Fig. 12), a flexible and customisable agile management tool (Atlassian, 2021).

Adopting this tool helps the UX Debt process be integrated into the company's product development workflow. This approach allows anyone in the company to access the board and evaluate each task's status and what is in line to be worked on next. Four columns were created, representing the four steps to resolve a UX Debt issue:

• Backlog: Once an insight or opportunity is identified in a user testing, design feedback session or heuristic evaluation, it should be added to this column using the task template.
• UX Design: Current tasks being worked on.
• UX Review: Tasks that are ready to be reviewed by the design team in the weekly design feedback session.
• UX Ready: Once a task is done and revised, it is moved to this last stage, and from here, it can be picked up by the respective product and development team responsible for the UX Debt.

This study's result turned out to be a completely new process instead of an iteration of the current one (Fig. 13). The separation allows for a more effortless design operation, where processes and workflows can be managed and iterated separately.

Next steps

The results were better than predicted, and the design team was able to validate and implement an entirely new process within a few weeks. This was only possible due to the team's maturity and autonomy, allowing iterations to be made with minimum bureaucracy. eBay Classifieds Group defines new strategies quarterly in a meeting with multiple stakeholders. This case will be presented as one of the possible strategies for the next quarter. If the process is selected, it becomes easier to scale. The diagram below demonstrates the steps needed for scaling the process (Fig. 14):

Conclusion

As a design team's maturity evolves, it's crucial to dedicate time to its process and workflows. This operation side of the design practice allows the team to move forward by continuously fixing its problems. The User Experience Debt example shows that dedicating time to discuss and evaluate how the team works can reveal incredible opportunities. This study helped the team realise that users may face hundreds of unknown issues and demonstrated that there is still much work to be done on user-centricity. While also contributing to the speed and quality of deliveries, helping uncover needs, and promoting an ethical reflection about the consequences of a design. As designers, we should make sure that processes such as Design Thinking are adopted and improved continuously. Making users part of the way you solve problems is only the initial stage, the biggest challenge lies in what the company and teams do with these insights. Making sure the identified problems are solved is the first step to claim that a company is user-centred.

Bibliography

• Atlassian (2021) Jira | Issue & Project Tracking Software, Atlassian. Available at: atlassian.com.
• Berkun, S. (2020) How Design Makes the World.
• Chen, M. (2020) The Zendesk Triple Diamond process, Medium. Available at: medium.com.
• Collopy, F. (2019) 'Why the Failure of Systems Thinking Should Inform the Future of Design Thinking (06.07.09)', Design Issues, 35(2), pp. 97–100.
• Hehn, J. et al. (2020) 'On Integrating Design Thinking for Human-Centered Requirements Engineering', IEEE Software, 37(2), pp. 25–31. doi: 10.1109/MS.2019.2957715.
• Johannessen, L. K. and Ellingsen, G. (2012) 'Lightweight Design Methods in Integrated Practices', Design Issues, 28(3), pp. 22–33. doi: 10.1162/DESI_a_00159.
• Kaley, A. (2018) UX Debt: How to Identify, Prioritize, and Resolve, Nielsen Norman Group. Available at: nngroup.com.
• Kimbell, L. (2011) 'Rethinking Design Thinking: Part I', Design and Culture, 3(3), pp. 285–306. doi: 10.2752/175470811X13071166525216.
• Moffett, J. (2016) 'Eliminate UX Debt: Improving Consistency & Usability', Studio by UXPin. Available at: uxpin.com.