Listening to Users at Scale is Product Management

James Colgan, Senior Director of Product Management  for Outlook Mobile Enterprise at Microsoft, shares his approach to scaling a product and generating user feedback for an app with over 100 million monthly active users.

Microsoft Outlook has 100 million active monthly users. James Colgan is in charge of making sure that they’re happy. As the Senior Director of Product Management for Microsoft Outlook Mobile Enterprise, he works across multiple teams to drive adoption within the largest companies in the world.

Outlook Mobile is the largest-scale project that James has worked on, but he’s stuck to the tried and true principles that he’s developed throughout his career.

First, Outlook Mobile needs to be trusted by IT and loved by end users, James says. Second, he's committed to moving fast. His team ships new features to the iOS and Android app stores on a weekly basis. This gives his team the agility and ability to be responsive to challenges and opportunities in the market. When operating at scale, James warns that it can be tempting to slow things down to be more contemplative and analytical, but says that this is a false tradeoff. His team is both data-driven and fast.

There wasn’t much data for James to rely on at first, so he and his team turned to app reviews aggregated across the iOS and Android app stores and focused on the 1-star reviews. They then prioritized feature roll outs that addressed common complaints. Though the app started at around 2.2 stars, today it is up to 4.7 out of 5.

The feedback his team was processing was, however, very North American- and European-centric. They had blindspots in Asia and South America. The team had some data but didn’t understand the “why” behind it. James sent teams to various countries in those regions to get a better sense of the context of how consumers used Mobile Outlook in their work and home lives by going to their homes and asking how they communicate and manage their time. They found that the work-personal life balance differed greatly among countries, which proved crucial in understanding why and how people use Outlook.

Mobile Outlook is unique in its highly developed in-app support, wherein users can have an instant message chat session with an Outlook Mobile expert. This helps the user get more value out of the app and acts as key signal for user pain points, which in turn feeds into how the team prioritizes new features.

The product manager is responsible for digesting all of the user insights from these various forms of research. James says that it’s crucial to contextualize long-term goals so that they can develop a tactical execution roadmap, which is actionable for engineering teams.

Product managers also have to be willing to recalibrate when presented with new insights. James had originally assumed that the more engaged users are, the more likely they would be to become monthly active users. But when looking at the data, he saw that the average session length was only 22 seconds long. Therefore, rather than focusing on email composition, he and his team shifted to focus on the consumption of information and ease of use for app navigation.

James says that there’s a common misconception that in order to increase the value of your product, you need to add more features. Often a product is born through the creation of a couple important features, but once the product devolves into a collection of features, then the user has a difficult time using it and loses sight of where your product fits into their life. He says that sometimes the need to pare back unnecessary bells and whistles.

You’ll learn a lot from this episode about customer-centricity, user experience, and user growth.

Here are the highlights:

• How James generates user feedback from the huge number of Outlook Mobile users (6:46)
• How user insights inform James’s product roadmap (15:35)
• James's strategy for growing Outlook Mobile's user base (20:34)
• How James decides when to de-prioritize feature ideas (23:52)