Segment case study  ⸻

A code-less data tracking tool



Design a no-code tool that helps Segment customers set up website tracking quickly by simply pointing and clicking on their own websites.







Role
Product Designer
Company
Segment
Duration
8 month (September 2019 - April 2020)




My Role

I was fortunate to work with a team of driven and passionate people. As the sole product designer, I worked as a strategic counterpart to my PM and EM on shaping the vision for this product and creating a product roadmap. I collaborated with my PM on discovering customer pain points, and worked with 5 engineers on building the designs. We shipped this product to GA after 8 months of hard work.

During this process, we worked very closely with cross functional partners on collecting user feedback and engaging with our customers. I loved building a user centric culture with my PM, by sharing user feedback often with our teammates, and engaging engineers in design workshops and design process. In the end, I was blown away by the depth of customer empathy that my fellow engineers have built up.




Business Objectives

A collective dream at Segment is that every company in the world uses good data to power their growth in a customer-centric way.

The first step in making this dream a reality is to enable all of our users to get their customer data into Segment. Unfortunately, due to the technical effort involved in instrumenting applications, companies of all sizes find friction in this phase of their Segment journey.

This no-code/ less-code tool, Visual Tagger, was developed to help Segment customers set up website tracking quickly by pointing and clicking the elements they want to track on their own websites.

💡 A bit more about Segment: Companies can send their user data from their mobile app, websites, and various sources to Segment, use Segment UIs to apply standardized controls and filters to the data, and then send it to destination tools, such as: Amplitude, Intercom, SendGrid, etc. Our power users consist of engineers, PMs, marketers, and data analytists.


Goals/ Metrics

Primary: Increase % of new customers set up track events with a broader initiative to increase customer activation rate
Secondary: Shorten time for new customers to start sending track calls

🤓 Analytics 1 min classroom: Events are user interactions with content that can be measured independently from a web-page or screen load. Downloads, link clicks, form submissions, and video plays are all examples of actions you might want to analyze as Events. 




Digging into Customer Pain Points

This tools was initially built as a 1 week long hackweek project. Our first goal of was to test its product market fit before we bring it to GA.

The first thing I, as a product designer, need to understand was the user personas and their pain points. Both of those things were certainly not clear in the beginning at all. In order to build the customer empathy, we ended up interviewing 20+ customers in the discovery phase.

Here is one of the insights we found after couple months of research while actively building the product:

Our users are less technical and are looking for analytics guidance.

Our users are marketers, product managers and business owners who don’t necessarily possess technical knowledge about coding or/and analytics. They want to learn analytics by doing it, and they want to enable analytics for their companies without relying or waiting on developers, which is a scarce resource for them.

“The vision for building a no-code tool did not just suddenly come to us one day. It was actually based on what we learned about our users.”

After gaining this insight, we worked on desiging less technical UIs and using simpler analytics terms.





Product Gaps

Orginally, the product was built as a Chrome Extension. Customers were struggling to use the product resulted in numerous Zendesk tickets flushing in. User satisfaction was low and the product was unable to meet the needs of our customers. I worked with my team to discover usability problems via numerous rounds of user testing and design iterations.

One of the key insights from our testing, which is also the biggest product gap that we had to tackle:

The mental models were mismatched between product experience and how our users expect it to work.

Our users had hard time using the product, because fundamentally, it was not how they expected it to work. The mental models were mismatched. In order to nail the flow that suits users’ mental model, we went through multiple major re-designs.





Before


The orginal product experience as Chrome Extension that was built as a one week hackweek project. We were passionate about building this concept into a robust product and bringing it to GA. I joined the team to lead the redesign as it was being rebuilt in the Segment web app.



After




The final product experience is drastically different than the initial one. Some key changes I made with the team along the way:
  • Remove the concept of “switching between modes (tag and navigate)” as it does not match with users’ mental model.
  • Make editing a separate step. After users tag an event, they need to type in event name, and they can add couple things to it. They need to click “add” to confirm saving. 
  • A “home page” was introduced, where a list of events are shown in a table view. We brought the interactive tagging-experience to full screen.

New product features:
  • Users can add properties to an event. (🤓 Analytics 1 min classroom: When to use properties? For instance, a user can attach “product name” as a property to an event “Add to Cart”. So that they can understand what product has been added to cart and how many clicks. They can also conduct more complex analytics in destination tools, such as Amplitude.)
  • We introduced “event template” as a starter kit for our users. One template consists of 5 to 10 events for users to add based on their industry, for instance ecommerce template will have: Products Searched, Product List Viewed, Pomotion Clicked, Product Clicked, Product Added, Cart Viewed, Checkout Started, Order Completed, etc.)



“It’s been awhile since I took a close look at this product but OMG. I was blown away so bad... It has changed a lot and it feels so click and so intuitive and y’all added a truckload of features!!! ... The biggest markers of success imho was that (1) It worked really well, and 2) because the experience was so smooth, as user there were times I forgot I was even using Visual Tagger (the product) because it doesn’t call attention to itself by being difficult to use - I was able to focus on my task and feel really reliable about it. Amazing work. I am such a fan of you.”
- Segment customer




Process

1. Onboard to the team and understand the technology




2. Identify customer pain points through user interviews
As mentioned above, we discovered that our users are less technical and are looking for analytics guidance. And they want to learn analytics by doing it.


3. Identify product gaps in the original product experience



After the initial research led by our PM and myself, we found a couple major product gaps:
  • The information architecture has failed to guide users through the product experience. For instance, the placement of the “modes”  (navigating mode and tagging mode in this case) is at the left bottom corner which is very far removed from the rest of UI, where the other actions are taking place.
  • The content and languages used in this UI are not well understood by our users.

With above reasons, our users are not able to achieve their goals using this UI, and they feel frustrated.


4. Iterate on user flow based on initial feedback


Questions we think about:
  • How might we reduce frictions as much as we can in order to help users achieve their goals as efficient as possible?
  • How might we give users confidence about their tracking implementation?
  • How might we help our customers to obtain clean data even if themselves are not aware of the best ways to do so?

We removed couple steps in the user flow and enabled users to “add tracking” quickly and easily with auto-save. Instead of the action of “saving an event,” we deliberately introduced another concept of “draft” and “publish” to indicate the formality and assist users to review their events before pushing them to production.


5. Iterate on UIs

v1


v2


Couple highlights and decisions we made in this version:
  • Intentionally only surface one primary button to guide users to take the action when they land on the product. This was a solution for an MVP untill we have more bandwidth to build a holistic on-boarding experience.
  • Continue testing the concept of tagging and navigating modes. But make “switching between modes” clear and easy with just a toggle away.


6. Iterate on user flow & UIs several times based on feedback from user testing sessions


We ended up learning that users do not typically understand the concept of “switching between modes” in this product context. 

We needed to design a cohesive end to end product experience that enables users to do ongoing operations (managing their data).

We also wanted to help our new customers set up Segment faster and easier by introducing in-between steps (deciding where they want to send data to).


7. After nailing down the user flow and layout structure, it’s time to radically simplifying the UIs

Simplicity is complicated. It’s been a challegne since beginning. Because first of all, traditionally, people with technical background implement analytics in code; secondly, we are designing for non-technical or less technical users. It felt like a task of inventing GUI. I was over the moon that at this stage we are almost at the finish line. We just need to make the complicated tasks feel even more simpler, by removing the visual & cognitive complexity. 


Remove or cleverly hide controls that are rarely used.



Create visual structure through typography, aligning objects to a grid, and utilize Gestalt principles, such as proximity and similarity.


8. Validate with customers


9. Ship the product to GA!
Well... yes, before this, it went through Alpha & Beta phases.







Impact

Besides the excitement and positive feedback shared across our users, 5 out of 8 customers have rated the product above 8 out of 10 easy to use. An average score is 7.7 out of 10 easy to use in our qualitative research.

“Soooooo good! it's superb - I was about to give up with the traditional setup as im super busy but i just set up the first event in 30 seconds with the tagger.” - Segment customer

“Love it! Great product, made my life so much easier. Waited for a month on my engineering team to get this setup, visual tagger came out and it only took me 20 minutes.” - Segment customer

“The tool does work!  I was able to begin events & properties on the page, just as promised.  Bravo.” - Segment customer

Quantitatively, we see 50% increase in the number of workspaces start sending track events within 14 days. It is important, because without track events, our customers won’t be able to achieve majority of use cases, which results in not getting value out of Segment. Eventually, it’s very likely that those customers would churn.

Our broader initiative is to increase self service customer activation rate. A relatively high percentage of customers who use this product to track events get activated within 14 days.





Reflection

I enjoyed the close collaboration and building a culture full of customer empathy. I also appreciated how driven our team is. The product is certainly not perfect - there are so many opportunities of improvements! Yet, I feel very proud of the result that we’ve achieved and how excited our customers are. 






︎   ︎   ︎   ︎  ︎

© QIAO HUANG. ALL RIGHTS RESERVED