This year I had the opportunity to present at eMetrics London<\/a> and Berlin<\/a> on a topic that is very close to my heart. I\u2019m psychotically neurotic about data quality<\/strong>. I\u2019ve written about it many<\/a> times<\/a> before, and it\u2019s pretty much why I want to keep on blogging and writing about analytics and tag management customizations. At eMetrics, I stepped out of my comfort zone of development and implementation, and chose to talk about organization practices.<\/p>\n <\/p>\n <\/a><\/p>\n<\/div>\n For the past number of years, I\u2019ve had the fortune of working with some amazing clients and colleagues, helping entire organizations understand the concept of meaningful data<\/strong> and how data is always an investment<\/strong>. In this article, I want to pull some of these threads together, and explain how I approach<\/em> data quality and the optimization thereof within an organization.<\/p>\n \n \n Subscribe to the Simmer newsletter<\/a> to get the latest news and content from Simo Ahava into your email inbox!\n <\/p>\n <\/span><\/p>\n As always, I do not want to preach, nor do I have any<\/strong> authority to tell others how to conduct their business. Quoting my good friend Craig Sullivan<\/a>, experts and best practices are \u201crather rare and mystical beasts<\/a>\u201d.<\/p>\n All we can rely on are the experiences of people we trust<\/strong>.<\/p>\n If we find these experiences useful, and we can trust that the person who shared them knows what they\u2019re talking about, we can see if these experiences could be adopted by whatever environment or context we are working in. This, I think, is lost to many who\u2019ve succumbed to the ease of finding answers<\/em> instead of methods<\/em>. Too often, people ask for Please do this for me<\/em> instead of Please show me how it works<\/em>.<\/p>\n The way I approach learning new things is to find experiences with which I can relate, after which I dissect these experiences, trying to recreate them in a way that is most beneficial to me<\/strong> and my goals<\/strong>.<\/p>\n In this article, I will be sharing my experiences of working in and with various types of organizations. If some of these thoughts resonate with you, then I\u2019m glad, and I look forward to hearing what you think about what I\u2019ve written. If this sounds alien to you, then I\u2019m equally glad, and I look forward to explaining myself further in the comments.<\/p>\n Surprise, surprise, I\u2019m all about data collection. I don\u2019t hesitate to say that it\u2019s the most crucial part of the web analytics process. But what about analysis, insights, visualization, segmenting?<\/em> I hear you say. Yeah, those are really important as well, but if you screw up data collection, you screw up all the subsequent stages as well.<\/p>\n <\/p>\n <\/a><\/p>\n<\/div>\n That image above, lovingly adopted from online-behavior.com<\/a> (my favorite analytics blog), illustrates this nicely. I used my Photoshop skillz to copy-paste the cute little diamond bug on top of the \u201cCollect Data\u201d node. This represents a virus that has infected data collection. It means you\u2019ve either failed or forgotten to measure something on the site. As you might guess, it means that this data is either not available or is corrupt.<\/p>\n A typical example is in Ecommerce tracking. Ecommerce comprises such a huge number of variables that successful comparison across date ranges would require that these variables be encoded consistently across different points in time. How often does that happen? I\u2019m willing to say rarely<\/em>, and the likelihood decreases with time. However, still we see people comparing Revenue from last quarter (which included shipping and tax) with Revenue from a year ago (which did not include shipping and tax). The very fact that they have different aggregation methods means that they can\u2019t be compared unless you know how these methods differ<\/strong>. Unfortunately, this information is often misplaced when teams and organizations change.<\/p>\n In other words, data collection is an investment<\/strong>. It requires labor, love, and a crystal ball. It\u2019s one of the most difficult things to get right, and requires a lot of experience. But there\u2019s nothing, nothing<\/em> as frustrating as data that\u2019s missing or incorrectly collected.<\/p>\n Here we go: diving into the controversial<\/em> end of the pool. Let me juice it up with another claim.<\/p>\n Data quality is destroyed by ineffectiveness<\/em> and non-involvement<\/em>.<\/strong><\/p>\n Let\u2019s see if we agree on this. Are you, or have you been, affected by the following phenomena in your organization?<\/p>\n Monthly<\/strong> reports, which lack relevance<\/strong>, are rife with generic suggestions<\/strong> that lack research<\/strong> in the context of your business, reiteration<\/strong> of previous month\u2019s points, even if there are solid reasons<\/strong> why they weren\u2019t addressed<\/p>\n<\/li>\n Ridiculously ugly and ineffective JavaScript hacks<\/strong> for measurement points, which should be tackled in the Data Layer<\/strong><\/p>\n<\/li>\n Clueless managers who want you to fix metrics<\/strong> rather than the underlying reasons that output the data into those metrics (\u201cFix our Bounce Rate, please!”)<\/p>\n<\/li>\n Analytics features are deprioritized<\/strong>, and deployed extremely infrequently<\/strong><\/p>\n<\/li>\n<\/ol>\n These are all symptoms<\/strong> of data being treated as a project outcome<\/strong>! They smack of non-involvement. They are deliverables and off-shoots of mail-order projects, where an entire analytics process is treated as something that can be time-boxed<\/strong>, outsourced<\/strong>, and ignored<\/strong>.<\/p>\n In the agency world, these assignments were quite typical. It\u2019s not because the agencies are lazy (well, sometimes it\u2019s that too), but it\u2019s because organizations have not matured enough to turn analytics into a business-driver<\/em>, and treat it as a diagnostics tool<\/em> instead.<\/p>\n What I\u2019ve learned is that the only way to make analytics truly<\/strong> work in an organization is to implement it as a process<\/em>. As consultants, we can be \u201cpurchased\u201d to help get things moving. This might mean an implementation project<\/em>, or a consultancy project<\/em>, but the key deliverable of these projects<\/em> is a shift in the momentum of the organization that we are trying to help. If the organization chooses to externalize analytics to an \u201cextra pair of hands\u201d, which happens ever so often, they will lack the incentive to actually integrate an understanding and respect of data into their processes.<\/p>\n One of the reasons analytics is so difficult to integrate into organizations is because it\u2019s cross-disciplinary. I wrote about this in a recent article in my company\u2019s blog (10 Truths About Data<\/a>):<\/p>\n Data is the lifeblood of the organization. It flows through all departments, across job titles, permeating the very fabric of the organization, reinforcing its foundations for growth. It cannot and should not be contained in one vector (a dedicated analyst) alone.<\/p><\/blockquote>\n Now, try to think of anything in a modern organization that works like that. It\u2019s difficult, right? A typical organization is riddled with departments and silos, and there\u2019s just so much friction<\/em> in the seams.<\/p>\n In 1968, Melvin Conway said something pretty insightful, which was then coined into a law bearing his name. The law is approximated as (quoting Wikipedia<\/a>):<\/p>\n organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations<\/p><\/blockquote>\n In other words, if the departments within your organization (e.g. IT and marketing) are unable to communicate with each other, any systems they manage or want to create (e.g. the CMS and an analytics platform) will be unable to function together as well.<\/p>\n This is one of the reasons we resort to projects<\/em> so eagerly. By requesting a project, we are allocating our department\u2019s budget to said project, and we\u2019ll receive something that\u2019s optimized for our department, and our department alone. For example, the marketing department might want to implement Google Analytics<\/a> on some new landing pages, and they request the help of an agency to write the implementation plan. The result is an implementation plan that only tackles the problems the marketing department outlined, instead of treating data collection as something owned by the entire organization<\/strong>.<\/p>\n The agency might also utter the most ridiculous words in the history of language:<\/p>\n \u201cWe recommend you install Google Tag Manager, because then we won\u2019t need IT anymore to implement analytics measurement.\u201d<\/p>\n I\u2019m not a violent man, but when I hear or see that argument it makes my blood boil. I feel very strongly<\/a> about this, and for a good reason. Nothing is as detrimental to an organization as an outside influence (e.g. agency) that communicates only with a single point of contact (e.g. marketing), but inadvertently gets mixed up in something that spans the entire organization (the IT infrastructure).<\/p>\n Friends, compatriots, fellow analytics aficionados: data and analytics belong to the entire organization<\/strong>, not just a subset thereof. It\u2019s unnerving, and quite honestly impossible, to try to implement something as significant as a bloody JavaScript injector<\/strong> (which a tag management solution often is) without involving the developers or IT!<\/p>\n In fact, I want to make the following bold claim:<\/p>\n A tag management solution empowers DEVELOPERS more than others.<\/strong><\/p>\n We implement a tag management solution because we want to harness a very developer-centric tool to facilitate communication within our organization. This necessarily<\/em> involves multiple disciplines in the decision-making process. If you entrust a TMS solely in the hands of a department or individual that already has prejudices against working with developers, it will only inflame this relationship further.<\/p>\n The way I see it, an optimal implementation of a tag management solution is perfect<\/em> for mitigating the effects of Conway\u2019s law. Similarly, a cooperative IT or developer process is paramount<\/em> in an optimal implementation of a tag management solution.<\/p>\n You see, a tag management solution is built on top of a Data Layer<\/strong>. Typically, we have a number of definitions for a Data Layer, but here are the three I most often fall back to:<\/p>\n A set of business requirements, encoded as key-value pairs against each hit, visit, and visitor that your digital property might collect<\/p>\n<\/li>\n A uniformly encoded, global data structure that collects and distributes data on the digital property<\/p>\n<\/li>\n An internal data model in the tag management solution with which you communicate through the global data structure<\/p>\n<\/li>\n<\/ol>\n OK, let\u2019s try again. That was confusing.<\/p>\n The purpose of a Data Layer is to provide a **bilateral** layer on the digital asset, which **decouples**, **normalizes**, and **uniformly encodes** semantic information passed through and stored within.<\/p><\/blockquote>\n <\/p>\n <\/a><\/p>\n<\/div>\n Hmm, I\u2019m not sure that made any sense either.<\/p>\n Look, the Data Layer is a translator<\/strong>. It forces you and your buddies in IT to communicate using a common syntax. It\u2019s a joint venture<\/strong>, where people and systems communicate across silos<\/strong>. You can keep on hating the gremlins in the IT lair if you want, but you\u2019d better respect and love the Data Layer. If you don\u2019t get this right, then you can just chuck the TMS out of the window, as it won\u2019t be worth the migration.<\/p>\n Perhaps unsurprisingly, I\u2019ve written about the Data Layer before<\/a>, and the more I work with organizations, the more I place my faith in its transformational power.<\/p>\n When crafting an implementation or measurement plan, involve all the relevant departments and individuals<\/em> in the process, and speak only Datalayerian. Instead of writing vague descriptions of what you might want to measure, write down key-value-pairs<\/strong>. Start and end with the Data Layer, in all parts of the implementation and data collection process. Let she or he who knows the constraints of this structure best lead the discussion. Translate all requirements you hear into variables and values, associated with their respective measurement points.<\/p>\n My past year at Reaktor<\/a> has only strengthened something I already knew: agility<\/em> is extremely helpful<\/strong> in the proper implementation of an analytics process.<\/p>\n It\u2019s not easy in a world still dominated by the pitfalls of the waterfall<\/strong>, where projects comprised distinctive stages, each with entry and exit conditions.<\/p>\n The handovers<\/strong> between the discrete stages of the waterfall are where data is most often corrupted! When a deliverable is passed from one stage to another, it\u2019s always a release<\/strong> rather than a continuum. Whoever passed the information is absolved of responsibility outside that of the deliverable, and whoever received the setup is forced to survive the possible conflicts, quality issues, and misgivings of the previous stages.<\/p>\n The void between the project milestones is where good data turns bad.<\/p>\n Instead, what if we treat the process not as a collection of milestones but rather something that\u2019s on-going and part of everything<\/strong> we do in our front-end or app development?<\/p>\n The first thing I want to implement in an analytics process is a proper consideration of measurement when developing features.<\/p>\n <\/p>\n <\/a><\/p>\n<\/div>\n Since the agile process is, by nature, quite chaotic, we need something to facilitate quality assurance. Often, this comes in the shape of a Definition of Done<\/strong>. It\u2019s a set of conditions that determine what is considered a success<\/em> in the development process.<\/p>\n If we add analytics and measurement requirements into the Definition of Done, we are actually mitigating data quality issues even before they crop up. By making sure that developers consider measurement when developing or updating features, we are cultivating a perfect breeding ground for successful data collection.<\/p>\n This requires, of course, that the developers are aware of what measurement entails. I could refer you to read the previous chapters again, but I can just reiterate myself here:<\/p>\n A tag management solution empowers DEVELOPERS more than others.<\/strong><\/p>\n Whenever I start working with a client, I educate the developers first. This might include:<\/p>\n Helping them design modules and interfaces for communicating with the Data Layer<\/p>\n<\/li>\n Getting them up-to-speed in front-end development and the requirements that tag management solutions place on the document object model<\/p>\n<\/li>\n Inspire them to create a sandbox for tag management solution testing (this WASP playground<\/a> is pretty much what I have in mind)<\/p>\n<\/li>\n Only hire to educate, not to delegate<\/p>\n<\/li>\n<\/ol>\n Oh, that last point is so<\/strong> important. I respect the na\u00efvet\u00e9 and hypocrisy (as I\u2019m also a consultant) of the statement, but I truly believe in it.<\/p>\n A good consultant should work in a manner that inevitably makes them redundant.<\/strong><\/p>\n Take that statement with a grain of salt. What I mean is that if you hire consultants for outsourcing tasks that should be handled within your organization, you not only create an unfavorable dependency, but you also lose the opportunity to improve your own processes.<\/p>\n Data collection and analytics are prime examples of this. I have most success when working in the client\u2019s premises, being extremely transparent about everything I do, and working together with their key players so that they can eventually do everything I\u2019ve done themselves. This builds trust and helps the consultant find new areas to improve and focus on, rather than things that are trivial to the consultant but super important to the client.<\/p>\n This text is a summary of what I\u2019ve been talking about this past year in conferences. I\u2019m crazy passionate about data quality, data collection, and facilitating growth in organizations.<\/p>\n I believe in this stuff with every fibre of my being.<\/p>\n I believe that successful projects morph into processes. They rejuvenate organizations, and build trust across silos that are engaged in ridiculous trench warfare, propagated by inflexibility and complicated hierarchies.<\/p>\n I believe that an empowered developer is at the heart of a successful analytics project. However, the entire team works together to build bridges from business requirements, through the Data Layer, all the way to reports.<\/p>\n I also believe that data is difficult. I\u2019ll end this article with another one of my \u201c10 Truths About Data<\/a>“:<\/p>\n Data is difficult. It has to be. There are no short-cuts, no omniscient tools, no out-of-the-box solutions that give you perpetual satisfaction. The entire life cycle of a single data point, from collection to reports, requires knowledge and expertise to manage. Turning arbitrary numbers into actionable insights is not, and never will be, a walk in the park.<\/p><\/blockquote>\n Do you agree?<\/p>\n<\/p><\/div>\n![\"From](\"https:\/\/www.simoahava.com\/images\/2015\/11\/project-to-process.jpg#ZgotmplZ\")
<\/p>\n
\n X<\/span><\/p>\n
\n The Simmer Newsletter<\/strong>\n <\/p>\nDisclaimer<\/h2>\n
1. Don\u2019t screw up data collection<\/h2>\n
![\"Data](\"https:\/\/www.simoahava.com\/images\/2015\/11\/analytics-process-infected.jpg#ZgotmplZ\")
<\/p>\n2. Analytics as a PROJECT is the root of all evil<\/h2>\n
\n
3. Tag management solutions help us fight Conway\u2019s law<\/h2>\n
4. Data Layer is the universal translator<\/h2>\n
\n
![\"The](\"https:\/\/www.simoahava.com\/images\/2015\/11\/data-layer.jpg#ZgotmplZ\")
<\/p>\n5. Empower and facilitate the developers<\/h2>\n
![\"Definition](\"https:\/\/www.simoahava.com\/images\/2015\/11\/definition-of-done.jpg#ZgotmplZ\")
<\/p>\n\n
Summary<\/h2>\n