Building a research program for non-researchers
When I joined the Windows/HoloLens Experiences team, I recognized that we had a need for early-stage strategic design research that could not be adequately met by the user research team. I saw this as an opportunity to build a stronger, more integrated research culture, and pitched a solution to our creative director – to provide designers, program managers, and developers with the tools to conduct much of this research themselves.
I built a cross-organizational team and led them in developing a common set of research standards and resources for the org, along with KPIs to assess adoption success. Our deliverables ranged from websites to training programs, for which I managed our roadmaps, planning, prototyping, and launches. By the end of the first year, we had driven 87% adoption across the org, regular usage of resources by 50% of designers and PMs, and had observed multiple cases of reduced research bias as a direct result of participation in the program.
HoloLens customer segments
Selecting the right target customer is especially challenging for emerging technologies with no established user base. At the time of this work, Augmented Reality was still in its infancy. Microsoft’s HoloLens product teams had collected data on a wide range of potential customers, but had not come to consensus on how to consolidate this data and target design work.
I spearheaded an effort to produce an actionable, prioritized set of target customer segments to be used by HoloLens first-party app design teams. Assembling subject matter experts from a broad set of disciplines (strategy, sales and field, design, market analytics), I led exercises to synthesize our dataset into three usable, high-priority customer segments; to define key workflows and pain points; and to validate our output.
The segments were initially intended for short-term use, but were subsequently validated by follow-up research and continued to be used for over a year. They drove design work on four first-party apps and were later adopted by the hardware and marketing teams to coordinate work.
Building and executing on a new education product strategy
In 2014 Microsoft made the decision to invest more strongly in education audiences with its Bing in the Classroom program. I created a research program to rapidly get the product team up to speed on user needs in this new space, and to provide ongoing feedback for concept development and design iteration. I conducted remote interviews, site visits, focus groups, and surveys with hundreds of teachers and students, all while navigating a strict set of policy and legal requirements for working with government employees and minors. I also developed panels and processes for obtaining ongoing user feedback with turnaround in under a week (a 75% time savings over our standard recruiting process).
Early in the project, I partnered closely with program management to generate and prioritize our feature backlog and to scope engineering work with the development team. After our program manager suddenly transitioned off the project, I further engaged with developers to communicate user-centered design requirements for 9 shipped features. I also tackled the issue of cross-product experience, sponsoring a Microsoft-wide executive summit that highlighted opportunities for the newly formed Microsoft in Education strategy team.
Within 9 months of launch, Bing in the Classroom was being used by over 4.5 million students in more than 5,000 schools across the US.
Introducing a new biometrics research capability
One of Bing’s design priorities is user delight, but success is difficult to gauge through click metrics alone. I led Bing’s first attempt to quantify the emotional impact of our designs through biometrics (facial EMG, EDA). I assembled a 9-person cross-functional team, designed the study, directed production of 7 videos as stimuli, and analyzed and presented the results.
The project strongly (and quantitatively) demonstrated the value of our emotional content, has been presented numerous times both internally and externally, and was recognized by Bing’s executive general manager as one of our major accomplishments of the year.
Chatbot design principles
I believe the greatest impact often comes from effective perspective-taking.
Microsoft had assembled an incubation team to identify opportunities in the newly emerging space of chatbots. Over the course of several months, our 3-person research team conducted multiple studies to understand what makes chatbots successful in messaging applications such as WeChat, SMS, and Facebook Messenger. This work provided important foundational understanding, but was not getting uptake by the design team.
I felt that the answer was not more research, but better presentation with an emphasis on straightforward design application. To this end, I condensed the entirety of our research into 10 chatbot design principles and presented it not only to our design team, but at an org-wide chatbot hackathon as well.
My goals were usage and sharing, and by those metrics the work was successful. The principles were presented across Microsoft at multiple high-profile events. They continued to be added to and evolved by other researchers even two years after I left the group. And they were shared with the public as a part of Microsoft’s Bot Development Framework design guidelines.