Information is the oil of the 21st century, and analytics is the combustion engine.” – Peter Sondergaard, Gartner
When most people think of advertising and marketing, an image of the “Mad Men” era agency comes to mind. But with surprising speed, the rise of digital–and the accompanying explosion of customer data–has revolutionized marketing.
Using technology and data, marketers today can better understand their customers, deliver personalized one-to-one experiences, and drive significant bottom-line results. To achieve these goals, they now spend over $20 billion annually on marketing technology, a market that has grown by over 67 percent in just two years. In addition, spending on big data hardware, software and infrastructure is forecast to grow to a total market size of $114 billion by 2018.
As the strategic importance of data has increased, new approaches to customer analytics have emerged as well. As customer interactions with companies grow and diversify, the need to integrate data faster and deliver real-time insights is critical. This post will explore the underlying trends driving companies to become more data-driven and invest in customer analytics. And, it will outline three types of approaches to capturing, managing, analyzing, and activating customer knowledge and insights.
Why Customer Data Is Critically Important
The Age of the Customer
Today, customer data, knowledge, and insights are more valuable and of more strategic importance than ever before. That’s largely because power has shifted over time from companies towards their customers. Customers have more options, greater access to pricing information, and, through social media, greater means to share their experiences with others, both good and bad. They have more power, choice and influence than ever before.
In this age of the customer, the only sustainable competitive advantage is knowledge of and engagement with customers.” – Forrester Research
Companies that succeed in this environment are those that become obsessed with understanding their customers, such as Amazon and Salesforce. These companies go to great lengths to exceed customer expectations by leveraging customer information and insights. They are masters at gathering data, turning data into insights, and making decisions and changes in faster and faster feedback cycles. In other words, they find out what’s working and what isn’t and adjust appropriately at lighting speed.
Changes in Customer Behavior
Customers have taken control of their purchase process. With websites, blogs, Facebook updates, online reviews and more, they use almost twice as many sources of information to make decisions as they did in the past and often engage with a brand dozens of times between inspiration and purchase. According to a survey conducted by Endeca Technologies, 50 percent of customers interact with an average of two touchpoints to research or purchase products, and 36 percent engage with an average of three.
Digital technology change is driving much of this. Customers are rapidly adopting new devices and new digital and social media touchpoints. In a sense, technology has turned our customers into moving targets. Consider how much your own shopping behavior has changed in recent years.
Farewell Funnel, Hello Decision Journey
Because of this dramatic shift, the traditional marketing funnel paradigm–the linear flow from leads, to prospects, to purchase that’s been with us since the “Mad Men” era–is being replaced. The new paradigm for marketing is the customer decision journey that places customer needs, not business needs, at the forefront. The customer decision considers the overall experience that a customer has with a brand and how that contributes to loyalty (or not). It also better reflects the ability of each individual to influence the purchase decisions of others to an unparalleled extent. The decision journey of most customers is non-linear and multichannel.
Customer decision journeys can be thought of as activities that fall into several different stages.
In this stage, customers discover an unmet need. They begin to think about a relatively narrow set of products and services that might meet this need. Most customers start their decision process with brands they are already familiar with.
In the next stage, customers begin to actively explore and evaluate their options. The customer is intent on purchasing. They begin to explore on the Internet, research options, read reviews, and pay closer attention to advertisements and promotions. In this stage, it is critical for brands to ensure that review and comparison information is widely available.
The third stage of the decision journey involves the customer’s decision and actual purchase. Customers tend to make their purchases in the most convenient channel.
In the Engage stage, customers engage with the product or service they’ve purchased. The experience that customers have during this stage determines loyalty and whether positive or negative feedback is shared via reviews and social media. They may also engage with customer service or a user community to receive support. Customer activity during this stage can also help companies identify other potential needs.
During the Advocate stage, customers share their experiences of the product or service with others. They may do this through reviews, social media or direct word of mouth. Companies can encourage this by providing invectives to customers to provide reviews.
Customer Insight Drives Business Results
Companies are finding that a better understanding of their customers and customer journeys can lead to significant business results. One of the reasons for this is that customer behavior has changed so much in such a brief period of time, and customer behavior continues to change. Technology has turned customers into moving targets. Companies can no longer assume that what worked last year–in terms of customer acquisition, engagement, retention and experience–will continue to work this year. Another reason for growing interest in customer journeys is that customer behavior now varies considerably across customer segments and even within broad, demographically-defined segments. Consider the case of two people with nearly identical demographic characteristics: both male; same generation, both born in 1946; grew up in England; 2 children; very wealthy; and both have large real estate holdings. However, most people would agree that these two people–Prince Charles and Ozzy Osbourne–have very different behavior! Mapping customer journeys and analyzing customer data allows business leaders to understand these differences and respond accordingly.
Here are a few examples of business outcomes that often result from improved customer knowledge and insight.
Marketing & Advertising
Companies are leveraging customer data to move ever closer to the elusive goal of truly personalized marketing: the right offer, at the right time, in the right location and context, to the right person. Even incremental improvements over traditional mass marketing approaches yield major gains in conversion and new customer acquisition rates. Gartner estimates that there is likely a 10X improvement in response rates for offers that are timely, relevant and convenient over those that aren’t.
Customer journeys and customer data are also being used by progressive companies to shape their marketing strategies and guide spending on marketing and advertising. For example, consider the case of a hypothetical company with an outdated website and the need to boost sales in the coming year. Conventional wisdom may lead the CMO to include a significant round of website investments in the marketing plan and budget. However, by mapping the customer journey of the company’s most important segments and analyzing the associated data, the company discovers that online reviews play a much more substantial role in the decision process for these customers. In fact, the majority don’t bother to visit the website at all. Armed with these insights, the CMO redirects investment to improve product photos on online review sites and encourages customers to post reviews. Total marketing spending doesn’t increase, but sales and return on marketing investment show marked improvement.
During the Engage stage of the customer journey, customers may interact with customer service or an online user community to receive support. By capturing and analyzing the data from these touch points, such as customer service notes and online forum postings, companies can identify customer pain points and issues proactively and update their customer service FAQs or other communications with existing customers. This not only improves customer experience, making it easier to resolve problems, but it also decreases customer calls into call centers and overall service costs.
Retention & Loyalty
Many companies have discovered patterns that customers exhibit before they cancel service, close an account or switch to a competitor’s product. Using customer data and analytics, these companies deploy and refine predictive models that help them retain customers with proactive approaches. Investments, in terms of offers and upgrades, can be made at the right time to increase the likelihood of retaining desirable customers.
The experience that customers have with companies matters a great deal. According to research conducted by Peter Dahlstrom and David Edelman of McKinsey & Co, fully two-thirds of the decisions customers make are informed by the quality of their experiences all along their journey. Other recent research has highlighted the critical connection between experience and company financial performance. Customer experience leaders have significantly outperformed laggards in cumulative stock returns.
Approaches to Customer Knowledge and Insights
So we’ve established why customer journeys and customer data are so important and how companies can use the resulting knowledge and insights to drive business value. But how are they accomplishing this? What data are they capturing? And, how are they putting it to use?
It turns out that technologies and approaches have emerged in recent years to help make sense of all the data that is being generated in the digital and social era. We live in a world of rapidly increasing data volumes where data plays a key role in determining winners and losers in the competitive arena. It’s of little surprise then that large enterprise technology vendors and startups have rushed to introduce new technologies to help companies unlock, analyze and activate their customer data. Many of these solutions fall into three types:
- Evolved Data Warehousing Solutions
- Data Blending Solutions
- Data Management Platforms
Evolved Data Warehousing Solutions
For many years, traditional business intelligence and data warehousing technologies and approaches have been used to capture and analyze customer data. Beginning in the 1990s, companies pulled data from their transactional systems into separate, centralized data warehouses to support reporting and analysis. The typical extract-transform-load (ETL)-based approach to data warehousing captures data housed in disparate source data systems, transforms the data, and then moves it into the data warehouse, where the data is arranged in a way to help facilitate access. By centralizing data in the warehouse, companies could create a “single version of the truth” and avoid the errors and discrepancies that often plagued them when reports were created from various transactional and source data systems.
Traditional data warehousing solutions are expensive to build, but they play an important role in companies. They are used to generate reports and visualizations on the holistic company performance that go to executives and regulators. The data warehousing approach, architecture, and vendor ecosystem is very mature and has been honed over the past few decades. Traditional data warehousing isn’t going away anytime soon.
However, the explosion of data, particularly unstructured data, generated in recent years has strained the traditional data warehousing approach and underlying technologies. The foundational infrastructure of data warehousing has been the relational database, which stores data into tables (or “relations”) of rows and columns and is used for processing structured data. As the volume (amount of data), velocity (speed of data in and out), and variety (range of data types and sources) of data has increased, relational databases often aren’t able to provide the performance and latency needed to process large, complex data. This volume, variety, and volume (3V) description of data was created by industry analyst Doug Laney (currently with Gartner) in 2001 and is one of the most common ways of characterizing “big data.”
In response, a new generation of big data technologies have emerged that make capturing and analyzing customer data, even large volumes of it, faster, easier and more cost effective. While not yet as mature as traditional data warehousing, big data technologies represent an evolution of data management and analytics technologies, architectures, and approaches. Many consider the foundational infrastructure of big data to be the Hadoop ecosystem. At the core of this ecosystem is Apache Hadoop, open-source software for distributed storage and processing of big data. In addition, the ecosystem includes other software such as Hive, Pig and Spark. The Hadoop technologies break up and distribute data into separate parts and then analyze those parts concurrently.
NoSQL and Massively Parallel Processing (MPP)
In addition to Hadoop, big data technologies include technologies such as NoSQL and MPP technologies. NoSQL, which stands for Not Only SQL, refers to a range of database technologies that are good at processing dynamic, semi-structured data. For large volumes of strucutred, semi-structured, and unstrcutured data, NoSQL databases are more scalable and provide better performance than traditional, relational databases.
MPP technologies process very large volumes of data in parallel across hundreds (or even thousands) of processors. MPP has a lot in common with Hadoop. However, while Hadoop typically runs on inexpensive commodity hardware, MPP generally runs on expensive, specialized data warehouse appliances that are optimized for CPU, storage and network performance.
Many organizations find that they require both traditional data warehouse and big data environments. Big data cannot yet provide the level of trust, security and metadata management offered by the traditional data warehouse. However, the business value associated with customer analytics and processing greater volumes, velocity and variety of data drive the need for big data approaches.
Operating on top of data warehousing and big data environments are a host of reporting, analytics, visualization and data mining technologies. These technologies continue to improve and evolve as well, making it easier to both discover customer insights from the data and to communicate those insights in easy-to-understand ways that help inform decisions and actions.
Another recent technology development, cloud computing, benefits both traditional data warehousing and big data approaches by driving down the cost of computing and data storage. The good news is that, while data volumes are exploding, the cost of storing data continues to decrease every year. Cloud computing makes it easier than ever to spin up new computing resources—accomplishing this with literately a push of a button. And, cloud computing infrastructure is often half the cost or less. Companies such as Amazon have revolutionized cloud computing with Amazon Web Services (AWS) and have driven down cloud computing costs over the past several years. Many organizations have discovered that cloud computing infrastructure is more reliable and secure while a fraction of the cost of on-premise infrastructure.
Data Blending Solutions
While a single, centralized data warehouse or big data environment offers many advantages, analysts sometimes need access to different customer data sources for critical decisions. The process of bringing a new data source into a data warehouse or big data environment, through a traditional extract-transform-load approach, can be time-consuming. Traditional data warehousing and business intelligence approaches and technologies almost always have a serious challenge: a constant backlog of IT requests. In a recent research survey, Forrester found that about 63% of business decision-makers now use an equal amount or more of homegrown or shadow rather than traditional, enterprise BI applications.
Historically, analysts used tools like Microsoft Excel or Access in situations where they needed to analyze data not available in the data warehouse. But, in recent years a new type of solution, data blending (also sometimes referred to as data discovery), has emerged. Using data blending tools, analysts themselves can access, cleanse, and blend data from multiple sources without having to write a line of code. These tools allow customer data to be blended together from multiple internal sources as well as external sources immediately to support a more agile approach to customer analytics. This is increasingly important because if companies know what their customers are doing better than their competitors, or can get to those insights faster, then they have a very distinct advantage.
Data blending solutions often include direct connectors to the most common sources of internal customer data as well as connectors to packaged third party data from leading providers. Because they allow analysts themselves to blend data from multiple sources, they help meet the imperative for speed and agility. However, data blending solutions have important limitations and caveats. For this reason, they remain a piece of the overall customer intelligence and data warehousing solution for most organizations rather than a replacement for traditional BI tools and approaches.
One serious drawback with data blending is the risk of end user error when pulling together different data sources. Typically end users don’t understand the underlying data semantics which can lead to confusion and increase the likelihood of inaccurate results. For example, “revenue” in one data source may be monthly revenue while “revenue” in another data source may be quarterly revenue. If an end user doesn’t realize the difference when blending data from the two sources, inaccurate and misleading results are likely.
Today, data blending solutions offer speed and agility but lack the robustness, support for mission critical applications, and single version of the truth provided by traditional BI tools and approaches. Solution providers are rushing to address this gap by providing the data quality and consistency of traditional approaches along with the ease and speed of data blending.
Data Management Platforms (DMPs)
Another recent solution for customer data management and analytics is the Data Management Platform or DMP. A DMP allows companies to centralize data, both their own online and offline data as well as third party data, and use it to create target audiences and optimize their online advertising. Using a DMP, companies can measure how campaigns perform for different customer segments and optimize their media buys and creative elements over time to improve effectiveness. They also help provide visibility into Return on Marketing Investment (ROMI) across campaigns and customer segments.
DMPs emerged just a few years ago in response to the needs of online publishers and marketers who began to capture and integrate larger and larger volumes of data from a wider variety of sources. DMPs differ from data warehouses since they more provide more rapid data integration and are tied to execution systems, such as digital ad execution, content management and marketing automation systems. DMPs are optimized to allow marketers to define target audiences and then activate campaigns to reach those prospects and customers.
DMPs help address the need to move closer to real-time execution in the digital and social era. They provide rapid capability to develop customer insights and push those insights to digital ad execution and marketing automation systems. Today, the primary use cases for DMPs include:
- Providing an automated, sophisticated approach for using customer data in digital ad targeting
- Enabling media efficiencies and programmatic approaches to marketing and advertising
- Developing insights that drive improvements to customer experience
In addition to these use cases, there are two high-level benefits of DMPs that are driving interest and adoption.
One of the most powerful ways that data and analytics can be used in modern marketing is in helping marketers improve segmentation, moving beyond the traditional large, demographically-defined segments towards more granular, demographically and behaviorally-defined microsgements. DMPs enable the definition of microsegments by allowing companies to overlay their own first party customer data with additional third party data that provides greater insights into behavior and interest in products and services.
Truly personalized marketing and customer experiences–the right offer, at the right time, in the right location and context, to the right person–continues to be the holy grail of marketing. DMPs enable personalization techniques that tailor the ads, content, offers, promotions, and experiences for prospects and customers, levels of personalization that are now considered essential. Without a doubt, marketing has moved from a “one size fits all” to a “one size fits one” approach and static, mass market approaches are no longer enough.
Interest in DMPs has been surging in the past few years as the technology becomes a critical component of the marketing technology ecosystem. In fact, some predict that DMPs will eventually emerge as the one-stop shop for all marketing data needs.
Comparing Approaches and Technologies
When it comes to customer analytics, most companies will find that these three approaches and technologies need to be used in tandem. It is generally less of a case of “either/or” and more of a case of “and/also.”
Data warehousing, optimized over several decades, represents a mature, highly-evolved approach and ecosystem of technologies. Data warehouses support mission critical reporting and analytics needs, providing a “single version of the truth,” data quality, security and robustness, and they aren’t going to disappear anytime soon.
Data blending solutions, while newer and less mature, meet a real business need to integrate and analyze customer data faster than ever. They help generate insights quickly and enable greater agility when it comes to customer analytics and actionable insights. However, these solutions have important limitations when it comes to the potential for end user error and inaccurate results. Software providers are rushing to address these challenges. In the meantime, these solutions should be used with care.
Data management platforms play a critical role by helping to integrate greater volumes and variety of sources of customer data and activate this data. Through connections to execution systems such as ad execution, content management, e-commerce, and marketing automation solutions, DMPs help companies activate their customer data and operate closer to real-time–moving ever closer to truly personalized marketing.
Your 90 Day Plan
The path forward for customer knowledge and insights is an exciting one. Given its rising importance, the rapid growth in new approaches and technologies to support customer analytics comes as no surprise. However, this complexity makes charting a course forward more challenging than ever. As companies consider evolving their approaches to customer analytics, here are a few recommendations to keep in mind.
- Customer analytics is not about the data or technology, but about the business decisions that the insights enable.
- Customer insights have maximum value when the focus is on real-time insights connected with front-line execution.
- Many customer insights can be found by mashing up different data pools. But, it is important to begin with whatever data is available today.
- The best approach is business question or hypothesis-driven. Often the biggest challenge is to follow the 80-20 rule and identify the 20% of the data that provides the right insights.
- Where possible, begin with simple and then evolve to more sophisticated approaches. For example, is it possible to approach early attempts at multi-channel, multi-touch marketing attribution with heuristic approaches? Can you begin predictive modeling using simple, linear regression models that are easy to understand and implement?
- Keep people, your prospects and customers, constantly in mind in terms of improving their experience and meeting their needs and expectations.
- Don’t just focus on customer acquisition and retention data. There is additional value in insights derived from the full life cycle of prospect and customer touchpoints.
- Gain an outside perspective. Consultancies can help provide an assessment of where you are today and recommend roadmaps and best practices based on their experience with other clients.
- Rather than approach customer analytics in terms of a single business use case, consider a full range of uses when determining appropriate levels of investment and communicating the full strategic value.
- Make learning and talent development a key part of the agenda.
- Take an agile, iterative approach to managing, analyzing and activating data.
- Approach customer analytics as a journey rather than a one-time project. Most companies require cultural, organizational and process change to become more data-driven–not just a new data store or technology–and this evolution takes time.
- Success with transforming to data-driven marketing also requires executive support and involvement. Persuade senior executives to champion and support these efforts.