After decades of application in IT departments, the ETL process (Extract, Transform, Load) is still an unsolved problem for marketers. As accessible data continues to mount around us, extracting it from its native source, transforming it into information we can use, and loading it into a data warehouse or reporting tool gets more complicated every day. For marketing departments especially, for whom the task of complex data management is relatively new, ETL remains a looming challenge. The problem is that to perform ETL well we need:
1. clear requirements for the desired output (by the way, those will continue to evolve, so we better be prepared for that as well).
2. deep marketing data strategy, software development, operations, and analytics skills (among others).
3. substantial experience managing large, and often only moderately structured data.
4. a high ability to execute complex technical projects and maintain them reliably.
5. sophisticated consumers of the output.
Marketing departments are typically lacking in all of the above. Marketers often aren’t at all sure what the requirements should be for transforming or enhancing data—it’s not been a required skill set in the past. With little data management capability or experience, execution is often patchy and inefficient, and those who receive the output (execs, agencies, peers and colleagues) often aren’t data management-savvy—or even reporting-savvy—either.
Unfortunately, the tools available for helping with ETL are marketed largely to business users—though in reality, they don’t do much to get marketers more independent from IT. They demo well, but when tasked with more complex scenarios, substantial technical expertise is always required.
In today’s world, the scenarios marketers deal with are enormously complex—multiple data sources that spit out messy data in formats that don’t align at all. At the same time, marketing data must be transformed in a programmatic fashion in order to use any of these business-focused reporting tools on an ongoing basis.
All that doesn’t even account for the constant change that marketing data in any company must go through, one of the reasons why so many dashboard projects are so short-lived. New sources of data are constantly added (and removed) as marketers experiment with new marketing strategies, marketing channels, and measurement capabilities. Data from all of those experiments needs to fit seamlessly into the structure we’ve set up in the data warehouse or reporting system so that we can perform integrated reporting and benchmarking across current and historic efforts, and decide what to adopt and what to abandon.
And that puts us right back into the IT support queue over and over again. And every time that happens we have at least two groups trying to solve the problem—two groups that need to talk to each other and understand both sides of the equation. IT needs a solid understanding of marketing needs. Marketing also needs to understand IT needs.
Marketing-savvy engineers? Engineering-savvy marketers? Both are very hard to find, particularly in the same organization. If this sounds like your business, ETL outsourcing is right for you. Here are the capabilities you should look for in an ETL partner:
1. Supports all data types.
In today’s integrated, omni-channel world, your ETL partner must support all data types from all sources—on- and offline. Ensure that they can take anything from PowerPoint slides (typical of agency reporting of offline data), to spreadsheets (typical of online data), to APIs (typical of modern execution tools). Unless you’re working together to build a repository of ALL your marketing data, you’ll be right back where you started—manually trying to align data from different sources with endless cutting and pasting.
2. Both marketing- and data management-savvy.
Making marketing data useful takes a top-down marketing point of view—it’s not just a technical capability. We must start with our intentions (e.g. increase lead flow) and then structure our data so it reveals which efforts drive our aims, and which don’t. ETL providers by definition are data management-savvy. But if they aren’t marketing-savvy, they’ll typically approach an ETL process from the bottom up. They’ll focus on not missing any data (“you never know if you might need it”), instead of solving for strategic marketing questions. For marketers especially, the way data is structured when it comes together has everything to do with how useful it will be for reporting and analysis. Chose a partner with deep marketing expertise.
3. Provides robust and flexible reporting.
Here’s where the advantage of a marketing point of view becomes clear. Your ETL partner should be able to deliver out-of-the-box industry best-practice marketing analytics. You should have access to real-time, omni-channel dashboards that show marketing effectiveness within and across all channels. You should also be able to perform ad-hoc analyses easily—query your marketing warehouse and get quick answers to anything from, “What was our most cost effective channel for customer engagement?” to “Which campaigns generated the most earned media impressions?”
4. Able to spearhead and manage an on-going process.
Your ETL partner should have the understanding and expertise to lead your organization down the right path. They should not only understand your requirements, they should be able to help you to better define them up front and adapt them over time. ETL is not a one-off activity. It’s an on-going process that must be sustained indefinitely. Your ETL partner needs to be just that—your partner. They must be well aligned with your business in order to maintain your marketing data over time and adapt your data structure to accommodate your ever-changing needs.