Efficiently documenting and sharing research insights at ecobee by building an insight database.

Project Roadmap

Discovery & Frame

Stakeholder Interview

I spent the first month conducting user interviews to understand the needs, pain points, and challenges of the current research workflow. I spoke with 15 stakeholders, including various user researchers, designers, product managers, lab managers, content strategists, and VP product in the team. Moreover, I also sat in some of the research meetings and followed through research initiatives to closely observe their research workflow and experience from different stakeholders and product lines.

Jobs-to-be-done Statements

After gathering data, I organized the notes as individual jobs-to-be-done statements for each research phase. I had also identified which stakeholder was involved in each phase along with corresponding pain points, opportunities, and context of the action. Based on the list of jobs-to-be-done statements, I then created a journey map that reflected the emotional ups and downs during the research workflow for various stakeholders.

This framework of organizing research data was very helpful because the jobs-to-be-done method allowed me to define different tasks each stakeholder needed to perform, while combining these jobs-to-be-done statements in a journey map allowed me to see an overview of the task flow for various roles within the product team. It became easier to identify which task or stakeholder required specific research support in the overall workflow, and it also revealed cross-functional relationships within the research process.

Current-State Workflow Journey Map

What is the purpose and value of creating an Insight Portal?

What would be the evidence that prove its value?

Based on Google’s HEART framework of user-centered metrics and Sharon’s concept of Key Experience Indicators (KEIs), which provides a quantitative score of a specific, important, and actionable phenomenon related to using a product or service, it will be valuable to focus on measuring the happiness, adoption, and task success dimensions of the Insight Portal’s user experience in this case.

Why and how to focus on happiness, adoption, and task success dimensions of KEIs?

Concept & Design

What are the possible solutions of addressing these problems?

Inspired by Tomer Sharon’s Atomic Research and the concept of managing research knowledge that redefines the atomic unit of a research insight as an insight nugget, I begin to see the value of designing a research repository that documents insights as “insight nuggets” with proper tags to enable better discoverability, consistency, and connectivity between individual research findings. An insight nugget is a tagged observation supported by evidence.

First iteration: What if the Insight Portal looks like [x]?

In the first round of design, I experimented with a database structure that would gather all information needed for the product team. For research insights documentation, I had foundational insights and usability issues placed under two sections as the type of findings are very different between generative and evaluative research. I have also designed the information architecture ranging from the most abstract (behavioural pattern and insights) to the most specific (user profile): with a descending order of generative insights, usability issues, studies, sessions, surveys, and users.

Evaluation & Iteration

First Round of Usability Testing

After the design of the initial database structure is completed, I input current research data from existing documentation platform to experiment with its usability. I tested its user flow with researchers in the team and there were several major problems with the current database structure:

1. It was difficult to navigate through the “Insight” section to find relevant research because the listing of insight nuggets was not organized by any theme. For a first time user who was not familiar with any key words to begin with, they did not know where to start.

2. Many researchers indicated that it took too many steps to input data from the administrator’s (mostly researchers) side. Although Airtable could link content between sheets, when the database structure followed the “insight-study-session-userprofile” logic, it was easy for “visitors” (the product team) to browse through, but complicated for “administrator” (mostly researchers) to organize and link information between many spreadsheets.

3. There was also privacy issues with directly linking certain insights with user profiles and relevant demographic information.

Second Iteration

Based on the insights and feedback from the first version, I reprioritized the database structure by conducting a participatory workshop with several researchers and designers that involved both mind mapping and brainstorming activities, which helped to understand how potential users may value different types of information in the portal. The mind mapping exercise and the participatory workshop helped me to refocus on the most critical component of a research repository. I also further probed into the field of information architecture and data science to understand how best to manage “many-to-many” data relationship.

What has changed between the first and the second iteration? Why?

New features/concepts added to the Insight Portal:

1. Introduced “Top Keywords” as a tag system that organized insight nuggets with different themes and product lines for better discoverability and search experience, especially for first time users

2. Positioned “Session” as a Junction Table that connects the “Insight”, “User” and “Finance/Legal” data into one centralized place

3. Introduced “Session ID” to protect user privacy and it was used as a thread in the junction table

4. Separated the compensation and consent information from “Session” and moved them to a new spread sheet called “Finance/Legal” because this portion can be automated in the future as the logistics part of the process

Features removed from the first version Insight Portal:

1. Removed “Study” because it contains repetitive information with “Session” and creates extra steps for data input

2. Removed “Usability Issue” and “Generative Insights” and combined them in the “Insight” section as different types of insight nuggets

3. Removed “Surveys” and added survey information to “Session” as one type of research methods

Ideate Future-State Research Workflow

Together with the user research team, we had a round table discussion and defined an ideal research workflow that the ecobee product team aspire to achieve and adhere to. With this proposed research workflow, each role in the product team understand the overall research process and their respective role in the workflow.