What is noise in the Privacy Sandbox?

Imagine a colorful mosaic, where each tile represents an individual user’s data. To protect each tile’s unique pattern, we sprinkle sand over the mosaic. This sand acts as “noise” to safeguard individual information, or tile patterns. In a small mosaic, the sand easily obscures specific details. However, in a larger mosaic, the same amount of sand only slightly covers the tiles, maintaining the overall pattern’s clarity while protecting individual privacy.

This analogy illustrates how noise functions in the Privacy Sandbox. When data is less aggregated, like the small mosaic, noise has a more significant impact, reducing clarity. Conversely, highly aggregated data, like in the larger mosaic, is less affected by the noise, preserving user privacy while still allowing for more meaningful insights. For a deeper dive into how noise operates, check out our conversation with Google privacy experts.  

Effectively navigating noise in attribution reports

The key takeaway is that higher data aggregation minimizes the impact of noise. However, the challenge lies in balancing data aggregation with the need for timely, granular reports. As Branch developed designs for this next-generation analytics experience, we identified the critical need for a noise management system to help marketers navigate this complexity.

We use ”variation” as a metric for noise, representing the percentage difference between actual and reported numbers. For example, if the actual install count is 100, but the “noised” report shows 50, the variation would be 50%. This metric helps us understand the impact noise has in our reports and determine whether campaign performance is comparable.

Apps with lower advertising traffic are more susceptible to noise, meaning the reported metrics may vary more significantly from the actual figures. To address this, we’ve developed a tiered system that categorizes apps based on their daily install traffic driven by ads. This allows us to tailor noise management strategies to the unique needs of each app, ensuring more accurate reporting. The tiers are defined as follows:

• Tier 1: > 100,000 daily installs
• Tier 2: 50,000-100,000 daily installs
• Tier 3: 10,000-50,000 daily installs
• Tier 4: 1,000-10,000 daily installs
• Tier 5: 100-1,000 daily installs
• Tier 6: < 100 daily installs

Each tier reflects different levels of noise susceptibility, with lower tiers (fewer installs) being more prone to noise. By knowing where your app falls within this tiered system, you can understand how we will apply noise management techniques to maintain accurate and reliable reporting.

Branch learnings on noise management

Navigating noise in the new Privacy Sandbox environment requires a strategic approach to ensure accurate and actionable insights. Through our research and simulations, we’ve identified several critical factors that influence how noise impacts reporting. Below are the key learnings that we use to surface noise diagnostics, which in turn will inform your campaign planning and optimization.

1. 1. The more granular the tracking dimensions, the more noise affects the data. For example, tracking installs at the country level might show 100 installs from the U.S. However, when tracking at the city level (e.g., San Francisco), you might only see 10 installs. Based on our simulations, we recommend tracking no more than seven dimensions to maintain manageable noise levels. Branch allows customization of tracking dimensions with noise level references to balance granularity and noise impact.
   2. Timeliness of reports also affects noise. A weekly report might aggregate 5,000 installs, while a daily report might show 1,000, and an hourly report only 10. For apps with more than 10,000 daily installs driven by ads, daily reports remain reliable, and accurate hourly reports are possible with fewer than five dimensions. For apps with fewer than 10,000 daily installs, limiting dimensions to fewer than five is crucial for high accuracy in daily reports.
   3. Customizing measurement goals enhances accuracy. Different marketers prioritize varying measurement goals, whether it’s total install count, in-app conversions, or revenue. To accommodate these diverse objectives, Branch allows you to customize these goals and assign different weights of accuracy to each. This ensures that the most precise measurement is applied where it matters most for your unique business demands, enabling you to focus on the metrics that drive your success.

Practical application: Noise simulations

Let’s focus on one metric to dig into some real test results: total install count. Below are the results of a real noise simulation in the Privacy Sandbox where we measured the accuracy of total install count across different time frames such as hourly, daily, or weekly. 

For Tier 1 apps (100,000+ daily installs), tracking across just two dimensions like country and platform reveals a noise level of 0.05%. However, if you increase the reporting dimensions to 10 — imagine you’re bolstering your report with columns like city, device type, and user acquisition source — the noise only rose to 0.54%, still less than 1%.. This highlights just how resilient to noise large campaign volumes can be. 

On the other end of the spectrum, in Tier 6 apps (fewer than 100 daily installs), noise becomes much more significant. With a simple, two-dimension report, we already see noise reaching 9.82% and doubling to 18.16% with 10 dimensions.

When the time ultimately comes that attribution for users on Android will rely on Privacy Sandbox systems exclusively, you’ll need a performance measurement partner like Branch that has invested significantly in research and development on noise impacts. Branch Performance analytics will surface the key insights you will need to make informed decisions on structuring reports and optimizing strategies. 

If you’re interested in further results of simulations we’ve run beyond the one featured below (such as the hourly, weekly, or monthly), reach out to us. 

Embracing the Privacy Sandbox

The introduction of noise under the Privacy Sandbox is a part of a larger shift toward a user privacy-focused digital marketing ecosystem. Branch is committed to delivering comprehensive and accurate performance measurement, with a strong focus on privacy-compliant attribution, empowering marketers and digital businesses to thrive in an ever-evolving privacy landscape. As we continue to collaborate with the Google Privacy Sandbox team, we’ll keep sharing practical insights and updates. Stay tuned to our latest posts for more in-depth learnings and strategies.

The post Navigating Noise: Managing Ad Measurement in the Privacy Sandbox appeared first on Branch.

If you are like many of the app growth marketers in this space, you may know a little about MMM but still have many unanswered questions. Hopefully, you’ve had a chance to tune in to our latest webinar on MMM — it provides an in-depth look at the fundamentals. (If you haven’t listened yet, this is a great place to start!)

This article picks up where we left off in our webinar. We cover some of the most common questions surrounding MMM and lay the groundwork for including MMM in your marketing strategy moving forward.

By the way, perhaps the biggest roadblock cited by advertisers is the technical lift and change management investment required to get their model to a productive point for their business. Here at Branch, we are excited to alleviate that burden by partnering directly with you via our Branch Media Mix Modeling closed beta program. To learn more about participating, reach out to your Branch customer success manager.

MMM isn’t a new concept. What’s changed?

First of all, let’s clarify the name itself. Is it “Media Mix Modeling?” “Marketing Mix Modeling?” Or “Mixed-Media Modeling?” Multiple legitimate names are currently in use, but “Media Mix Modeling” is the most common and what we are calling it at Branch. We also feel it is the most accurate, because using “Mixed-Media Modeling” implies a distinction between mixed-media and non-mixed-media which doesn’t exist in practice.

When it comes to the growing awareness of MMM, three forces are occurring simultaneously: 

• Attribution is getting harder.

The primary keys with which we connect data from different parties in the ads ecosystem are drying up faster than we can keep up with. This means that traditionally relied-upon touch attribution approaches are degrading in efficacy.    

• The rules keep changing.

The walls haven’t been breached. This means no open-ecosystem exists to rally behind. Rather, we must make sense of platform changes like SKAN for iOS and Attribution API for Android with no dominant reconciliation paradigm to make sense of them (yet).  

• The learning curve is steep.

Machine learning has been making meaningful steps forward, but its many applications for accelerating marketing haven’t yet been fully explored. Similar to AI, the efficacy curve is shows mostly gradual and sometimes sudden progress.

And — you guessed it — MMM uniquely benefits from or tackles each of these three forces. 

MMM is like MTA, right? 

This is a common misconception about MMM that we encounter at Branch. The short answer: From a technical perspective, MMM and multi-touch attribution (MTA) are completely unrelated. But they can address similar business needs when employed correctly. 

Others have mentioned previously that advertising IDs (such as IDFA and GAID) are becoming increasingly scarce. In order to adapt to this new reality, marketers need a solution that does not need to directly join individual events together. MMM considers aggregate sets of spend (paid channels), clicks and impressions (organic channels), as well as other signals. MMM then uses machine learning-powered statistical analysis to generate budget allocation recommendations and forecasts. 

MTA, on the other hand, compounds the traditional last-touch paradigm. MTA considers all touches leading up to conversion and distributes partial credit between them. This is based on some valuation logic like “linear decay” (giving progressively more credit to later touches) or “U-shaped” (giving more credit to the first and last touches, and distributing the remainder equally). 

Our observation is that MTA often garners attention but rarely represents a strong enough value proposition for customers to migrate off last click. And the reality of increasingly scarce advertising IDs means all touch-based methodologies (including MTA) are losing accuracy. We must collectively adapt. 

Isn’t MMM a solution for brand advertising that takes months to generate? 

There’s a lot to unpack in this one. 

The history of MMM did previously resemble this picture. Typically, media agencies contained the requisite reach, data, and resources to provide MMM as a consulting service for advertisers. It was also expensive ($70K-$100K per project), took a long time to get results, and was mostly adopted by brand-focused advertisers heavy in traditional channels like linear TV. 

The big downside: When they finally arrived, the results went quickly out-of-date. 

The winds have since changed. MMM is now more accessible and flexible via powerful automation — quickly delivering rich, diverse marketing data. With the robust, next-generation MMM tools available today, it is much easier to make timely decisions and optimize your budget without having to wait weeks or months for results. 

For Branch customers, data from all marketing channels is gathered in one place — from paid media, emails, social, mobile web, and organic search. Our MMM solution uses this data to run weekly refreshes and help you understand how potential budget allocation changes can drive incremental app growth. This means you’re no longer waiting three months for your next MMM report.

The Robyn MMM framework was built by Meta. Is there a conflict of interest?

This is a great question that indicates a healthy understanding of the importance of unbiased measurement practices. 

Branch is building our MMM solution on top of Robyn. Robyn was originally developed by Meta, but it is an open-source MMM code library. This means the code is open for reviews and methodology audits by any code user. By leveraging an industry-standard, open-source model, Branch can ensure we are aligning with MMM best practices to deliver a balanced, impartial, and insightful cross-channel analysis.  

Side note: Branch is also excited to be part of Meta’s MMM Incubation Program, a select group of partners working closely with Meta to improve Robyn and develop the future of privacy-first, paid media performance measurement.

This all sounds rather theoretical. How does MMM actually work at Branch?

The main deliverable of the Branch MMM solution is a set of budget allocation recommendations for your channels and a forecast of the impact of those changes. These recommendations can be used alongside your existing, touch-based attribution reports to make more informed campaign investment decisions.

Initially, we’ll ask for 12 months of data that we can use to run the MMM model. We will also verify the data by discussing the model’s inputs and determining whether any additional data should be included. After we run the model for the first time, we will share the results and discuss whether any tweaks or improvements need to be made. The next step is an interactive refinement process until the model delivers results.

MMM is most effective when it sees the full picture of your business. So, if you have other marketing activities like push notification campaigns that aren’t tracked in Branch, you will want to include that data as well. You can expect recurring meetings with our team to help you interpret the model as it “refreshes” (i.e., runs MMM with new data). Those meetings will also act as a standing office hour to brainstorm experiments based on Branch’s recommendations. 

How can we trust what the model recommends? 

This is the million-dollar question. MMM can seem like a bit of a black box at first, and a recommendation like “shift 10% of spend from Google Ads Search to Apple Search Ads” is not necessarily the type of quantifiable impact analysis most teams are used to. Making budget allocation decisions, however minor, is no small thing.

Our team will work with you over time to ensure the MMM model sufficiently comprehends the nuances of your business model, market, and competition. After you are satisfied with the completeness of the model’s inputs, we recommend first conducting at least one test based on its recommendations, then taking it from there. 

Let Branch show you how MMM works

In our increasingly privacy-focused world, MMM can be a powerful tool — for marketers of all sizes — to allocate marketing budgets more effectively. The insights MMM can provide into how various marketing initiatives work together can help you make the best decisions possible when planning your next campaign.

Interested in finding out more about Branch’s MMM closed beta program and how to get started? Just reach out to your Branch customer success manager!

The post What Does Media Mix Modeling (MMM) Offer for Mobile Apps? appeared first on Branch.