This article features content from Revolution 2017. Join us for in London March 2020 for our next event. Interested in learning more? Click here.

Retail sales and growth is reliant on the depth of your customer knowledge. Do you know them well enough, for example, to predict what they’ll want to purchase from you next? While some patterns are obvious, like Black Friday sales, the further detail you go into with each customer, the more complicated your predictions become.

At the recent Emarsys Revolution event in Berlin, Physics Professor and Founder of BlueYonder, Dr. Michael Feindt, spoke about the state of prediction today and how artificial intelligence (AI) is taking data analysis far beyond what humans can do on their own.

Knowing Your Customer in an Uncertain World

Data not only helps us understand what happened in the past, but also provides clues about what we can expect to happen in the future. Some things like recency and spend amount can follow defined patterns that you can translate into a personalized email or offer. Without a technology solution, you might be able to sift through the data for a single customer by herself in a day.

Now think about doing that for thousands of customers, each with thousands of data points. That’s where marketing teams begin to look for software or third-party services to scale all that analysis. And because predicting customer behavior as a central strategic activity for marketing, we must go beyond finding faster ways to make sense of the growing amount of data. Our databases are rich with clues that can help us deliver more personalized experiences to our clients. We must become a lot more accurate in the decisions we make based on all of that data.

Accuracy in prediction depends on a system that accounts for every possible input and influence. Tracking only recency and average order value gives you a rudimentary picture of your customer. Now add in channel, preferred device, average incentives, and related items purchased, and the decision becomes more complicated, takes longer to make, and may still only be a guess.

“What we’re trying to do is to give companies — now mainly retailers — real value through data and scientific software. So it’s about bringing science into economy but hiding the complexity and trying to build applications that are end-to-end applications for real problems that people have in business.”

Professor Michael Feindt • Founder, Blue Yonder • @M_Feindt

The Anatomy of a Prediction

To predict with any accuracy, you first need data to pore through, the more data the better, because this will be what creates the most accurate and in-depth picture of each customer. Then you’ll analyze the data and look for patterns, causes, and the relationships between them. These predictive insights form the basis of the prescriptive action you’re going to take.

But there’s a lot to account for between poring through the data and deciding on the best action to take.

Causation and correlation

When you find patterns in customer behavior, you then have to determine why those patterns are occurring. This often begins with trying to correlate two data points around the pattern and see what insights you can glean, because ultimately, you’re trying to learn what created the pattern and whether it presents a marketing opportunity. The problem is that using only a few correlated data points doesn’t necessarily reveal the true cause — or even a true correlation.

Causation and correlation are mainstays of statistical analysis, but some people are confused by the relationship between the two. As Prof. Feindt’s colleague Lars Trieloff at Blue Yonder puts it:

“For instance, there is a distinct correlation between the number of people drowning in swimming pools in the US and the number of movies Nicolas Cage appears in. Yet not even his harshest critics would accuse Mr. Cage of causing people to drown and the hypothesis that swimming pool deaths inspire Hollywood studios to cast Mr. Cage does not align with Hollywood production timelines.”

To understand the real reason customers are behaving in a certain way, the marketer needs many data points to draw correlations between. The data’s out there, but it’s not humanly possible to analyze it all in such a way and still be able to act in time.

Predictive + prescriptive analytics

Once the data’s been analyzed, the next step is to identify what’s predictable and then come up with an actionable plan to address that.

You can use predictive analytics to predict customer behavior in the near future based on recent behavior. This stage essentially finds the pattern in customer behavior and calculates the probability that the pattern will occur again.

Prescriptive analytics offer an action recipe, a plan for execution based on a decision fed by many data inputs. The factors that most influence these kinds of decisions include:

  • Objective data from as many sources as possible
  • Predictions from the predictive analysis
  • Cost/Utility. How much budget do you want to spend based on the outcome you want to achieve?
  • Reinforcing your approach by measuring the approach’s performance and using that data to make your efforts better targeted and more efficient
  • Using technology to take all the analytical insights you’ve gathered to scale the workflow

Predicting retail probabilities

So when can you use predictive analytics? The world is a very uncertain place, so it’s hard to predict anything impacted by random chaos. Weather is chaos, and we’ve come a long way with our ability to forecast, but even so, we can’t predict what the weather will be more than 14 days into the future. There are too many variables that can occur over too long a period of time.

The key is balancing predictability and uncertainty. For marketing, this means they have to aim for the sweet spot halfway between two impossible extremes. Due to uncertainty, every decision is a compromise between complete unpredictability where everything is random, and absolute predictability where everything is foreseeable.

In retail, you’re trying to predict a behavior or an event where the main factors for calculating such probabilities are:

  • Items you want to sell
  • Stores where stock will be
  • Days the events will have the best chance of being successful

Once you predict the event, then you have to prescribe the action to take, and you have to make that decision based on the given KPIs. For a grocery store that stocks perishables, you can only improve one KPI at the cost of another. Too much stock, and you have to write it off. Too little, and customers will be displeased that you’re out of stock. This is impossible for a human to correctly predict, but this is where AI technology offers a more efficient way to align to business goals and make it easier to achieve both a decline in write-offs and out-of-stocks.

The human limits of prediction

Very important personal and professional decisions are done by us, by gut feeling, and they will not be automated, even also in the future. So that’s a good method. For example, these questions… like these once-in-a-lifetime decisions like: “Should I take this job?” or “Should I marry this woman?” — These are things that will never be automated. So that’s a good story.

But for example, if you’re a retailer — and I think it’s true for almost every enterprise — many decisions are to be done there. So what we already do is that about 99% of all decisions can be automated. Not only can we automate it, but at the same time, they are improved quite a bit.”

“One great example where machines are better than the humans is making decisions under uncertainty. Important personal and professional decisions are done by us, by gut feeling, and they will not be automated. But a retailer has many decisions to make that can be automated — in fact, 99% of all decisions can be automated. Not only can we automate them, but at the same time, the accuracy of those decisions is improved quite a bit.”

Professor Michael Feindt • Founder, Blue Yonder • @M_Feindt

As Prof. Feindt believes, the human “gut feeling” is not a reliable decision-making tool. Marketing is a complex system, and results and decisions can’t be measured with the gut. You want to influence customers by observing customer purchase behavior, but with so many external choices, the gut feeling is tantamount to numerology. Sure, there must be people out there who can do either accurately, but with no way to measure such a subjective and potentially mythic decision-making process, your success might just be coincidental.

Like automated science, you have to observe what happens, but in marketing, there’s too much to observe. You need to match up correlation and causation to be sure that you’ve correctly identified the relationship, and you can only do that by basing every predictive decision on data.

Prof. Feindt points out that among the decisions that machines would be more effective making than humans are those that are constantly repeated. How many of each product should you stock? What is the best price, depending on time of day, year, location, and a million other possible variables? Which customers should you send the catalog to?

The problem, though, as Prof. Feindt outlines it is that when humans are the ones making those repeat decisions, one of three things will happen:

  • Most companies will do nothing (BAU)
  • A few companies might consider their own business rules, mostly focusing on “How did we do this before?” or “What did we do last year?”
  • A sliver of companies will actually think about the process and aim for a way to do it better.

When these decisions become habits, the only thing you can predict is getting the same exact result as you have every time before, and worse, your competitors may kill you in the market for it.

“Just think about your company and your own process,” Fiendt said. “How often is the default ‘what did we do last year?’ Yes, last year. Good, huh? But that’s doing nothing. It’s even worse than doing nothing! It’s wrong because you’re predictable. It’s really dangerous to do that because it will be easy for your competitors to find out. ‘These idiots! They do the same thing as last year.’”

Even under the best of circumstances, a human marketer can’t keep up with the data or feed all available data in to making the decision in anything close to real time. That means the marketer makes most decisions based on an average result across many customers. This is not the data marketers need to effectively personalize their interactions. To do better than a guess, you must account for uncertainty, and the only way to do that with accuracy is to use AI.

“I automated the stuff that I did with my students and research in elementary particles physics,” Feindt said. “So we found out that, more or less, we do similar things again and again over the years in making statistical analysis of such elementary particle collider events. So we automated that, and in the meantime, we had 4 times superhuman performance. We have a computer program now that is 4 times better than 400 physicists over 10 years, including myself.”

Machine + Data = Most Accurate Prediction

The closest thing we will ever have to an accurate fortune teller is a machine running AI software. AI is already prevalently used in machine learning, image recognition, and deep neural networks, and one of the most amazing things that AI is really good at is reinforcement learning, because you build intelligent algorithms that can optimize very complex actions.

AI takes uncertainty into account

One of the reasons AI is so revolutionary for the marketer is that it can accommodate the unpredictable chaos of the real world. Where the standard approach predicts a number, with no accounting for uncertainty, AI offers complete conditional probability density (a range of possible numbers in a bell curve). Risk management is also accounted for.

“That the future is uncertain is bad. If we want to decide now how many to order, then we have to find out what is the cost if I order too many and what is the cost if I order too few. We have a probability distribution and a cost function, and now we can use mathematical optimization algorithms to find the best decision, telling us: Okay, today I have to order 280 apples so that we will have the best compromise of being out of stock and having to throw it away.”

Professor Michael Feindt • Founder, Blue Yonder • @M_Feindt

Where AI shines in retail

Specifically, AI can transform the retail supply chain through optimizing automation. With 99% automation, a retailer can see significant improvements:

  • Write-offs and waste decrease
  • Operations costs go down
  • Fewer out-of-stock events
  • In the case of food retail, the produce will be fresher
  • Inventory turnover goes up
  • Efficiency improves

Blue Yonder has successfully applied AI-driven supply chain automation to a variety of industries. For example, one German grocery chain used automation to bring their out-of-stock events way down from 7.5% to around 0.5% to 1.0%. The grocery chain had access to all the necessary data and information before, but they didn’t know how to put it to work.

The meat industry uses AI to waste less meat. The auto industry uses AI for predictive shipping. Not only does AI more accurately predict order times and volumes, but the technology can also be used to calculate the right price at the right times.

Making Uncertainty a Little Less Uncertain

AI solves the human limitations in predictive and prescriptive analysis. Causation and correlation are often intermixed, making it very difficult to really know what made a customer decide to purchase from you. There are simply too many variables at play for humans to get a good predictive picture quickly. Due to the incredible level of computing power we have these days, the machines are going to win in this respect every time.

However, with AI, you can use intelligent algorithms to determine the cause and effect by digging through loads of historical data, and by scientifically proving the relationship between cause and the desired effect, you’ll make far better predictions about customer behavior.

As Prof. Feindt has said, “No AI is no alternative.” BAU is already changing. As companies think about their strategies for tomorrow, AI is the best way available to step up to the next level and improve the customer experience.

 ► Catch Professor Feindt’s full-length, 35-minute Revolution presentation, here.