How We Built Our Machine Learning Model for Predictive Send Time Optimization

Get a peek at the inner workings of our Best Time To Send Solution from the data scientist who made it, Lisa Orr. Want to see it in action? Request a demo anytime.

---

Figuring out when to send notifications is a tricky problem. Personalizing the experience to each user is challenging — let alone sending individually-timed messages across an entire audience base.

We solved both problems with our Predictive Send Time Optimization Solution. The model decides the best time based on individual engagement patterns — and provides an easy way to send to each user’s best time in a single step.

The (Mostly Manual) Ways Brands Have Been Trying To Predict Best Time To Send

While building this model we polled customers about their general engagement strategies to determine how they decide the best time to message their users.

Some marketers send notifications the moment they are finished composing them, regardless of user timezone.

Others use their intuition to identify the best time to send a campaign: a marketer of a dating app scheduled messages at 6pm local time as it represents the window when people get home from work and start thinking about their potential dates for the night.

On the more data-driven side, we learned that a telecom company selected best time for each individual based on a few rules: send at the hour in which they last saw an open from a user, unless that time was outside day hours; then send to an early morning hour the following day.

Our conversations revealed that marketers want to be more data-driven with their send time decisions, but are still relying mostly on intuition and convenience.

Machine Learning To The Rescue

We created a model that:

• Selects optimal engagement time personalized to each individual user

• Is localized to each user’s timezone

• Is extensible to dormant users, and

• Is both actionable when building campaigns and explorable in our data products

Our models are based on a user’s recent engagement history.

To start, app opens are localized to the user’s timezone and aggregated to the hour over the last 60 days of app activity. Best hour is determined by striking a balance between the user’s engagement patterns and a generalized model of engagement patterns across an app’s total audience.

The model includes a preference for daytime hours to avoid assigning late night best times to users who may infrequently engage with apps late at night. (However, if a user truly spends most of their time engaging at night then their personalized best time will fall into night time hours.)

Our model also creates a generalized “best hour to send” recommendation, which brands can use as a rule for sending messages to dormant or low-activity users. The general best hour aggregates opens across app users and selects the best hour based on the most frequent opening time for each app and app operating system.

Testing & Results

To test the model we compared predicted best times to true open times for open events that occured after model generation. We compared the best time model to both a baseline model (e.g. everyone’s best hour is set to 12pm) and to the general best hour (e.g. everyone’s best hour is set to the general model best hour).

We found that personalized best time more accurately predicts the hour a user opens than either the general model or the baseline model. It performed better than both the generalized best time (52% higher match rate on average) and the baseline model (100% higher match rate on average).

“We found that personalized best time…performed better than both the generalized best time (52% higher match rate on average) and the baseline model.”

Making It Easy To Put the Model To Work for You

We want to make deriving insights and taking action with the Predictive Send Time Optimization model simple for the marketer and their analytics teams:  

• As with our Predictive Churn model, we ship weekly predictions as user tags to facilitate integration between predictive model output and our data solutions.

• We make it easy to schedule messages with Predictive Send Time Optimization

• We offer simple ways to visualize the user-level predictions

• Analytics teams can ingest predictive tags into their own systems with real-time data streaming.

• Through our customer analytics solution Insight we provide a deeper dive into the model, surfacing the cross-audience best hour as it relates to each day of the week and for each app operating system.

Since launch we’ve seen major retailers, a major research firm and an international media company adopt optimized notifications into their daily workflow. We’ve seen higher direct open rates with optimized sends and have heard positive feedback on how the tool is saving teams time and effort. One of our customers is even using Predictive Send Time Optimization to meter traffic into their app.

We are excited to hear about the success of the model in the wild and look forward to building more predictive tools that fit our customers’ needs.

Ready to get started with Predictive Send Time Optimization? Let us show you how it can work for you in a personalized demo! Contact us today!