Any Order funnels vs. Closed funnels: key differences explained by mobile game analytics platform ByteBrew

User funnels are a powerful marketing benchmark. Cameron Hozouri, CEO and co-founder of mobile game analytics platform ByteBrew, has broken down the significant differences between Any Order and Closed funnels.

Cameron Hozouri

As the CEO of an all-in-one game analytics platform that gives developers access to various growth tools, I want to discuss funnels as they are one of the basic tools for quick and effective analysis of player behavior.

Before breaking down the differences between their types, let’s start by defining a user funnel.

Defining Funnels

Funnels are methods for understanding how users transition through stages, events, steps, etc. in your project. Most common use cases for utilizing funnels when analyzing your apps are to quantify how many users reach or drop off key conversion points like purchases or level completes.

Even though funnels are often highlighted within most analytic solutions, how a platform computes funnels varies based on the technological capability of the platform.

Any Order Funnels

Like their name suggests, Any Order funnels are a form of funnel that only calculates based on the number of users or events that complete an individual step without boundary.

Breaking that statement down further, steps in an Any Order funnel are queried to only search for users who have completed the series of specified steps in any order. Any user that has completed the steps listed in the query will be included when calculating.

This funnel logic does not take into consideration:

• Matching the consecutive order of steps completed;
• The chronological order of steps completed when computing results.

To illustrate this, look at the sample funnel query for a 4-step funnel where: “Step 1 = Event A”, “Step 2 = Event B”, “Step 3 = Event C”, and “Step 4 = Event D” using any order funnel logic:

In the query diagram above, we examine 7 users that have interacted with the specified steps. Using Any Order funnel logic, all of the events performed by the users are incorporated into the funnel calculation regardless of the specified order in the query of “Step 1 = Event A”. The visual funnel output for this data would be the following chart diagram:

Utilizing the 7 user paths, the Any Order Funnel will simply count the number of events that match the listed steps in the query as long as the user has interacted with all previous events in the query. It also sorts the data by the slotted step number, resulting in the Any Order Funnel Chart sample shown above.

This indicates that any event, regardless of it’s predecessor or required step number, will be included in results. For example, User #2 does not match Step 1 (Event A) of the query in their user path, but is still incorporated in the funnel data. Looking further, User #5 starts with the last step of the query (Event D) and ends their path with the first step of the query (Event A), but is still added to the calculation.

Any Order funnels are fabricated funnels that completely mislead the developer analyzing their apps performance or conversion paths. Data calculated in this fashion does not represent a funnel of any kind, but a simple representation of the count of events performed by your users.

Utilizing this funnel logic also restricts what data can be derived from the analysis. Vital KPIs for understanding your app’s funnel performance such as “time to complete”, “median time to step”, or real user “dropoff” can’t be computed because the events in the query aren’t chronologically organized. If your analytics provider describes their funnel analysis using Any Order, be aware of what their data is representing when making product altering decisions.

Closed Funnels

Closed funnels are a type of funnel calculated using data from users that participate in events based on a set step order. This funnel logic sets strict boundaries to only compute events that follow both:

• The consecutive step order of events listed in the query;
• The timestamped chronological order of the events by users.

Computing funnels using this methodology renders true user conversion data with user dropoffs down to each individual user and unlocks visibility into how long converting takes from one step to the next.

Using the exact same user data set shown in Any Order funnels, let’s examine how Closed funnels would calculate the 4-step sample funnel where: “Step 1 = Event A”, “Step 2 = Event B”, “Step 3 = Event C”, and “Step 4 = Event D”:

Examining the diagram above, only user paths that begin with Event A and follow a subsequent step order will be incorporated into the funnel output.

For example, User #4’s events will only partly be qualified as the system will rule out events that do not have all the matching predecessors in the set order. Instead the system will recognize Event A as the start of a user’s conversion path and begin looking for subsequent steps matching the query order to include — making Event A and Event B in User #4’s path accepted by Closed funnel logic.

Additionally, User #3 will only have one event qualify to be included in the funnel calculation since the start of the user’s conversion path matches “Step 1 = Event A”, but the next subsequent step is not “Step 2 = Event B”. The visual funnel output for this Closed funnel data set would be the following:

The Closed funnel chart paints a completely different picture for how a user group transitions through the funnel set in the query. Accurately excluding user events that do not abide by the queried order set by the developer uncovers exactly how cohorts of users either reach or drop off a path to Event D.

Comparing the Any Order and Closed funnel data

In mobile development, funnels are a benchmark used in key decision making that affects both the product in development, but also the user experience — and to that extent the success of your business.

Platforms that calculate Any Order funnels display a skewed misrepresentation of what developers are looking to achieve when creating a user funnel. The delta between the results of these two methods using the exact same data set is substantial and highlights how vital the decision is for choosing a proper analytic partner.

ByteBrew, for example, uses Close funnel logic to give a precise analytical breakdown of the app’s performance. This is in line with the entire purpose of any analytics platform, which is to provide developers with both the tools and technology to produce the insights they depend on to make impactful business decisions in their products.

You can sign up for ByteBrew here.