To Understand Big Data, Try Thinking Like a Psychologist
To Understand Big Data, Try Thinking Like a Psychologist
Machines make great weather predictions but it’s never wet inside a computer
Like a modern King Arthur and all of his knights, digital marketers
are forever on a quest for the holy grail: the website that personalizes
itself to the consumer.
If we can customize an individual’s digital experience, the thinking goes, we can encourage them, with all the right signals and calls to action, to complete their purchase. The conversion cup will runneth over.
The problem is that a perfectly customized experience is as elusive as the holy grail itself.
Businesses have so many different kinds of users requiring various types of experiences, and “personalization” — that million-dollar idea that we can predict customers’ behavior in advance and then offer them a tailored online experience — is touted as the ultimate solution. Enterprises are pouring resources into personalization in a bid to gain an advantage on their competition and increase their profits.
But so much of this is a waste of time and cash.
It’s easy to be lured by the idea that if we can identify similar behaviors among groups of customers it will allow us to predict how similar clients will behave. But by looking for quick fixes from big data and crunching a handful of numbers, businesses are actually falling into a trap.
You Need to Get Personal - But Not in the Way You’re Thinking
Businesses are swimming in data about their consumers. There is data that relates to user behavior (clicks, page views), social events (likes, shares), item details (category, price) and contextual information (time of day, weather, device) and much more.
But despite all this information at our fingertips, we really don’t know why your customers are (or aren’t) buying your product. That’s because this incredible, rich data tells us plenty about what our customers do, but next to nothing about why they4 are doing it.
Consider this example: A health food store does a data-driven analysis of its consumer behavior, and determines that average customer spending is much higher during the summer months than in the winter. The data-based conclusion here, then, would be that people are more willing to spend money on health food items in the summer.
But drawing such a conclusion ignores several potential confounding variables. Several other potential factors are at play — summer is also swimsuit season, which means people tend to engage in healthier activities and lifestyle choices in the hot months, in which case purchasing more health food items is then a secondary result.
To truly understand customer behavior, data must be organized in a way that shows a deep understanding of the way people think and behave. It's not just about blanket personalization. It's about thinking like a psychologist — or better yet, hiring a psychologist to help you crunch those numbers.
Like Human Life, It’s Complicated
Data scientists are prone to fall into what we call the “black box” assumption: thinking that human behavior can be understood simply by observing external data. Humans are not machines. We are complicated, intelligent and emotion-driven, and companies who rely too heavily on cold numbers in a bid to understand their consumers are cheating themselves by forgetting this.
Machine learning offers unlimited potential for reaching customers in new and exciting ways. Using machine learning methods, we can zoom in to identify patterns that are invisible to the naked human eye. But one thing machine learning can’t offer us, at least not yet, is a portal into the inner human experience.
Consider this popular analogy: machines make pretty good weather predictions these days. They can even predict the inside of a storm. But it’s never wet inside the computer.
Why does this matter? It matters because even though we’d like to believe that all human behavior is as clean and tidy as a row of numbers, the reality is most of our decisions come from gut feelings and hunches. If we don't accept this, we’ll never move forward in our quest to truly understand human behavior.
Data analysis should be driven by expert knowledge and psychological theory4instead of applying the "let’s just try and see" approach. For marketers to use data effectively, they have to zoom out and remember that a one-direction, one-size-fits-all approach to reading data will almost always sell themselves short.
Conversion is a process comprised of many stops and starts. Conventional data scientist wisdom has looked at conversion as a single action or event, isolated in time.
This is a woefully short-sighted approach.
The path from initial brand exposure to checkout is long and usually takes a few touch points for the user to convert. We need to stop talking about conversion rates, and start talking about conversion cycles.
A “conversion cycle” can involve many site visits, transitions between mobile and desktop, and even multiple stops between a company’s online and offline stores. It’s the result of many different intertwined decisions, factoring in a number of concerns: cost, style, personal finances, emotional pulls, family decisions and more.
For companies to truly zoom in on their customers and gain valuable insight that can pad their bottom line, they need to remember that when a customer visits their site, the site visit is only one tiny data point in a maze of factors that will ideally lead at the end to conversion.
But if the company can identify where the customer is in her conversion process when she is at that specific data point in the maze, then they will have a valuable tool to effectively influence her behavior.
To Make Use of Your Data, Get Digging
It’s tempting to think of conversion and data models as existing across a single dimension. But if we want to tap into the goldmine that machine learning has offered us, and use the precious data at our fingertips to truly make a difference for our customers, we need to take out the shovel and dig to its second, third and fourth layers.
Here’s an example. A data scientist at one of our biggest retail clients once told me he was on the verge of telling his manager to remove the filters from his website. Why? Because he had run an analysis on customers who converted, and found that while the filters were popular among visitors who came to the site and then left without making a purchase, visitors who did convert were actually not using the filters.
I decided to go deeper. It didn’t take long for this data scientist and I to realize that those visitors who were converting were returning visitors, who were already familiar with their site and no longer needed the filters to find what they were looking for. Those same visitors, on their first visit to the site, had used the filters to search for the very product they were now returning to buy.
Think of psychological models like a shovel: Beneath every layer of data there are the human problems of emotion, irrationality, cognitive bias and emotional cues.
So many factors play into our purchasing decisions. If enterprises want to help guide their customers to conversion, they need to start by peeling back the layers on their data and seeing these customers as the multi-dimension, complicated people that they are.
Psychological models of customers’ behavior can work hand-in-hand with data to identify customer intent and help companies understand where visitors are in their conversion cycle during each unique site visit. Algorithms could integrate visitor actions, attributes and contexts such as the type of page or the type of website in order to determine intent.
It is this kind of personalization — identifying customers as people and not just spots of data on a chart — that is the real holy grail. Any company that wants to get ahead should be aiming for this today.
If we can customize an individual’s digital experience, the thinking goes, we can encourage them, with all the right signals and calls to action, to complete their purchase. The conversion cup will runneth over.
The problem is that a perfectly customized experience is as elusive as the holy grail itself.
Businesses have so many different kinds of users requiring various types of experiences, and “personalization” — that million-dollar idea that we can predict customers’ behavior in advance and then offer them a tailored online experience — is touted as the ultimate solution. Enterprises are pouring resources into personalization in a bid to gain an advantage on their competition and increase their profits.
But so much of this is a waste of time and cash.
It’s easy to be lured by the idea that if we can identify similar behaviors among groups of customers it will allow us to predict how similar clients will behave. But by looking for quick fixes from big data and crunching a handful of numbers, businesses are actually falling into a trap.
You Need to Get Personal - But Not in the Way You’re Thinking
Businesses are swimming in data about their consumers. There is data that relates to user behavior (clicks, page views), social events (likes, shares), item details (category, price) and contextual information (time of day, weather, device) and much more.
But despite all this information at our fingertips, we really don’t know why your customers are (or aren’t) buying your product. That’s because this incredible, rich data tells us plenty about what our customers do, but next to nothing about why they4 are doing it.
Consider this example: A health food store does a data-driven analysis of its consumer behavior, and determines that average customer spending is much higher during the summer months than in the winter. The data-based conclusion here, then, would be that people are more willing to spend money on health food items in the summer.
But drawing such a conclusion ignores several potential confounding variables. Several other potential factors are at play — summer is also swimsuit season, which means people tend to engage in healthier activities and lifestyle choices in the hot months, in which case purchasing more health food items is then a secondary result.
To truly understand customer behavior, data must be organized in a way that shows a deep understanding of the way people think and behave. It's not just about blanket personalization. It's about thinking like a psychologist — or better yet, hiring a psychologist to help you crunch those numbers.
Like Human Life, It’s Complicated
Data scientists are prone to fall into what we call the “black box” assumption: thinking that human behavior can be understood simply by observing external data. Humans are not machines. We are complicated, intelligent and emotion-driven, and companies who rely too heavily on cold numbers in a bid to understand their consumers are cheating themselves by forgetting this.
Machine learning offers unlimited potential for reaching customers in new and exciting ways. Using machine learning methods, we can zoom in to identify patterns that are invisible to the naked human eye. But one thing machine learning can’t offer us, at least not yet, is a portal into the inner human experience.
Consider this popular analogy: machines make pretty good weather predictions these days. They can even predict the inside of a storm. But it’s never wet inside the computer.
Why does this matter? It matters because even though we’d like to believe that all human behavior is as clean and tidy as a row of numbers, the reality is most of our decisions come from gut feelings and hunches. If we don't accept this, we’ll never move forward in our quest to truly understand human behavior.
Data analysis should be driven by expert knowledge and psychological theory4instead of applying the "let’s just try and see" approach. For marketers to use data effectively, they have to zoom out and remember that a one-direction, one-size-fits-all approach to reading data will almost always sell themselves short.
Conversion is a process comprised of many stops and starts. Conventional data scientist wisdom has looked at conversion as a single action or event, isolated in time.
This is a woefully short-sighted approach.
The path from initial brand exposure to checkout is long and usually takes a few touch points for the user to convert. We need to stop talking about conversion rates, and start talking about conversion cycles.
A “conversion cycle” can involve many site visits, transitions between mobile and desktop, and even multiple stops between a company’s online and offline stores. It’s the result of many different intertwined decisions, factoring in a number of concerns: cost, style, personal finances, emotional pulls, family decisions and more.
For companies to truly zoom in on their customers and gain valuable insight that can pad their bottom line, they need to remember that when a customer visits their site, the site visit is only one tiny data point in a maze of factors that will ideally lead at the end to conversion.
But if the company can identify where the customer is in her conversion process when she is at that specific data point in the maze, then they will have a valuable tool to effectively influence her behavior.
To Make Use of Your Data, Get Digging
It’s tempting to think of conversion and data models as existing across a single dimension. But if we want to tap into the goldmine that machine learning has offered us, and use the precious data at our fingertips to truly make a difference for our customers, we need to take out the shovel and dig to its second, third and fourth layers.
Here’s an example. A data scientist at one of our biggest retail clients once told me he was on the verge of telling his manager to remove the filters from his website. Why? Because he had run an analysis on customers who converted, and found that while the filters were popular among visitors who came to the site and then left without making a purchase, visitors who did convert were actually not using the filters.
I decided to go deeper. It didn’t take long for this data scientist and I to realize that those visitors who were converting were returning visitors, who were already familiar with their site and no longer needed the filters to find what they were looking for. Those same visitors, on their first visit to the site, had used the filters to search for the very product they were now returning to buy.
Think of psychological models like a shovel: Beneath every layer of data there are the human problems of emotion, irrationality, cognitive bias and emotional cues.
So many factors play into our purchasing decisions. If enterprises want to help guide their customers to conversion, they need to start by peeling back the layers on their data and seeing these customers as the multi-dimension, complicated people that they are.
Psychological models of customers’ behavior can work hand-in-hand with data to identify customer intent and help companies understand where visitors are in their conversion cycle during each unique site visit. Algorithms could integrate visitor actions, attributes and contexts such as the type of page or the type of website in order to determine intent.
It is this kind of personalization — identifying customers as people and not just spots of data on a chart — that is the real holy grail. Any company that wants to get ahead should be aiming for this today.
