What is Product Analytics?

Product Analytics is the process of analyzing how users interact with a product or service. Companies use Product Analytics to optimize the user experience by identifying where the user is challenged and merchandise features around the product.

The Importance of Product Analytics

Before launching a new product or feature, companies often rely on collecting data from potential customers using surveys and interviews to learn about the challenges and validate their assumptions. Once launched, the company’s strategy must change to instead use hard evidence by analyzing behavioral data of how their users are interacting with the product.

Companies must validate their previous hypothesis on which they designed the product using real behavioral data. While prospects may have shared their interest in the form of a wishlist, they may never actually engage with the features designed to fulfill that wishlist.

For that reason, businesses need to analyze product usage and service experience using real product data.

Product Analytics Metrics & Examples

Product Analytics can answer different types of questions ranging from trends to analyzing feature adoption or engagement over time, to visualizing intricate experiences and user flows inside or outside the product.

Here are the five most important types of analysis you must use to get the most out of your Product Analytics.

Analyzing trends over time is one of the most commonly used types of reporting in Product Analytics. It helps companies visualize whether feature adoption is increasing or decreasing over time. A Trends Analysis is centered around one or more specific touchpoints in the user journey by slicing and dicing them and then zooming into their performance over time.

UX Designers can zoom into specific features and visualize user configuration changes over time.

Product teams can identify the most and least used product features and how each feature’s usage compares to the past.

Trends Analysis

2. Journey Analysis

For most product use cases, a user must go through a series of steps to reach the solution. Those use cases can occur over a long period of time, involving multiple users and multi-channel interactions (Macro Journey). Other use cases can be as short as a few product clicks, such as submitting a form or configuring an account (Micro Journey).

It’s critical to visualize the user’s path to reach every goal and where they could be dropping off along the way.

The Journey Analysis provides better visibility to bottlenecks in the user journey to further fine-tune the user experience.

For example, a Customer Success Manager can visualize the path taken by a users towards the resolving an issue.

Journey Analysis

3. Attribution Analysis

Customers who succeed can give you a lot of insight. The Attribution Analysis can help you identify the touchpoints that are highly attributing to success. Similarly to the Journey Analysis, the Attribution Analysis uses the user flow data, but it instead precisely hones in on the users who have completed their journey and analyzes their touches in reverse.

A Product Manager can, for example, break down revenue generated by the most recent premium feature usage attempts before the conversion. That will allow companies to set a dollar value to every premium feature and highlight the top converting opportunities.

Attribution Analysis

4. Cohort Analysis

As a company matures, its product messaging, website design, documentation, onboarding experience, brand awareness, product complexity, and many other factors will evolve with it. These changes will profoundly impact how every cohort perceives the product or services. For that reason, metrics should be broken down by cohorts to reveal how that perception is changing over time.

An older cohort who started using the product when the product was simple may outperform a newer cohort who began using the product more recently. The new cohort will not experience a slow and gradual increase in product complexity as the older one did.

A newer cohort who experienced a well-designed onboarding experience may develop a stronger foundation than an older cohort who may not have had any proper upfront onboarding.

Cohorts can also be defined based on many different factors. A cohort can be determined based on the period they made their first website visit, signed up to the service, spoke to a salesperson for the first time, or upgraded to a paid subscription. The cohort definition should be dynamic and unique to every cohort analysis.

For example, a Product Manager can visualize user submissions broken down by when they registered to the service.

Cohort Analysis

5. Retention Analysis

Most successful businesses experience some level of churn. Companies must analyze the rate of that churn. The Retention Analysis helps your business analyze the rate at which users continue to engage with a product or feature over multiple periods. Unlike Cohort Analysis, a Retention Analysis normalizes cohorts under the Initial Period and displays the aggregate retention rate for the following periods.

The cohort units can vary based on every use case. They range from days, weeks, months, or even quarters.

Like Cohort Analysis, every Retention Analysis requires a cohort definition (an action that defines the cohort, such as signing up) and a repeat action (action to be analyzed over time such as logging in).

For example, a Growth Marketer can analyze the daily login rate from the moment the user signs up. They will find 87% of the users logging in the next week, 78% the third week, and so on.

Retention Analysis

How To Implement Product Analytics

Implementing Product Analytics can be straightforward. Companies must start by defining a general discipline upfront to collect actionable data. A well-defined discipline streamlines the data collection process as part of the development and deployment process.

This discipline covers:

  • What are user actions
  • How to collect user actions
  • When to collect user actions

What Are User Actions

User actions consist of three parts:

  • Action name: The name of the action that the user is committing. It’s commonly defined by a verb followed by a noun. Some examples include: view tab, invite user, submit application, create project, update profile.
  • Action properties: Every action will have its own set of properties (or taxonomies). For example, the view tab action can have properties like name (name of the tab) and url (URL of the tab).
  • Action timestamp: The time when this action occurred, usually stored in Unix timestamp format.

Once this action definition discipline is documented, developers will no longer have to waste time and guess the proper action name and properties for every new interaction or feature they release.

How To Collect User Actions

Most app interactions will be tracked on the front-end using Javascript (for web), Android SDK, or iOS SDK (for native mobile apps). Developers often incorporate those SDKs in their development frameworks and are encouraged to abstract the implementation to keep the tracking consistent.

For example, a React developer can plug a view screen action in the React Router, which will automatically track it every time the user switches the screen. Developers never have to worry about instrumenting that action every time they introduce a new screen to the app.

This engineering discipline will prevent double-tracking actions or tracking the same action type under different names, especially when multiple developers are involved. It will also nearly automate the tracking process when launching new features.

When to Collect User Actions

The frequency of action tracking depends on the resolution of data that the company is aiming for. Some companies opt to track every possible interaction such as click button, scroll to view, or even hover mouse actions. Others may limit their tracking to more prominent actions.

While tracking more actions offers a better resolution to the user experience, the cost of data collection can exponentially grow. But tracking too little might not reveal important challenges in the user journey, such as the user failing to click a button.

See below the different layers of tracking resolutions companies should consider, ordered by importance:

  1. Sessions (must have): This resolution should cover the account creation action, loading the app, and logging in. It will be enough to provide high-level growth and user retention analysis.
  2. CRUD Operations (highly recommended): This resolution covers when a user creates, reads, updates, or deletes any entity in the product.
  3. User Selections (recommended): This resolution covers how users engage with a specific feature locally, such as making a dropdown selection or switching accessibility modes.
  4. User Interactions (optional): This resolution covers every possible interaction such as mouse clicks, hovers, or expanding a dropdown. These actions may be device-specific and may not have a significant effect on the macro user journey.

Defining a discipline around when to track and when not to track will streamline the Product Analytics implementation process. It will also guarantee a consistent resolution across the whole user journey.

What Product Analytics Tools Should You Use?

Having defined the data disciplines for their Product Analytics project, companies will need to pick the right tools.

The good news is that there are many tools to choose from. Before choosing the tools, though, companies must decide on one of two approaches to implementing Product Analytics.

One approach is to build an internal solution that consists of a Data Warehouse and Business Intelligence tool, and the other is to use a dedicated Product Analytics service.

Approach 1 — Data Warehouse with Business Intelligence Tool

There are a few benefits to following this approach:

  • Companies will have full control over their data architecture. That, of course, requires Data Engineering expertise at the company, which is typically a full-time job.
  • Companies will have full control over data governance and compliance. They decide what data to expose.
  • Companies will have full control over what and how every report will look.

On the other hand, the downsides are many:

  • It requires multiple people with different expertise to be involved in this project.
  • Data Warehouses are queried using SQL. SQL is not the ideal query engine for user flows. A simple attribution question could take over 200 lines of SQL code to get answered.
  • This approach can be a lot more expensive. Besides the human resources required to implement this approach, companies will still need a data collection solution, a data warehouse, and a business intelligence solution.
  • This approach can take months of work to implement.

This approach is suitable for larger organizations that want to leverage data for multiple purposes, including providing analytics to their customers. They will have all the control they need to develop an embedded analytics system for their customers while building their internal dashboards from the same source of truth.

Approach 2 — Product Analytics Solution

The benefits of using a cloud-based Product Analytics solution include:

  1. Zero maintenance solution. Companies never have to worry about managing the data architecture as it scales.
  2. A visual interface answers Product Analytics questions that require user flows. Unlike SQL, Product Analytics query engines are tailored to answer these types of problems.
  3. Democratizes data across all teams. The interface is simple and does not require SQL knowledge or technical expertise to answer the most sophisticated questions.
  4. More affordable, requiring fewer people and no infrastructure provisioning to implement.

The main disadvantage of using an off-the-shelf Product Analytics solution is that every solution implements an opinionated approach to generating reports. So, if the organization has a specific way to visualize data, it may get constrained by this approach.

This approach can be beneficial for most organizations. They can rely on the expertise of the people who have spent years designing those products and focus all their energy and resources on providing service to their customers.

The bottom line, it’s critical to start with a data strategy collectively with your team. That strategy will influence the approach your company uses to implement Product Analytics.