Since writing my rant about the schema conspiracy<\/em><\/a> of web analytics platforms, I\u2019ve been giving the whole idea of hit-level data collection a lot of thought. Sessionization is very heavily implemented in Google Analytics<\/a>, which is understandable, but the regular Google Analytics API just doesn\u2019t give you the kind of information you\u2019d need, if you wanted to stitch hits together differently in your own backend. In fact, there are four<\/strong> distinct levels of aggregation that are not exposed via the API, even though I think they should:<\/p>\n Hit timestamp<\/strong> – You can\u2019t query for the exact timestamp<\/em> of a Google Analytics hit via the API. You can\u2019t get it in your custom reports, either. However, this might be very meaningful information, especially if you want to verify data integrity across systems.<\/p>\n<\/li>\n Session ID<\/strong> – Hits are grouped together into sessions<\/em>. However, using the API alone, you will not be able to easily identify if two distinct hits belong to the same session.<\/p>\n<\/li>\n Client ID<\/strong> – Sessions are bunched under a Client ID<\/em>. Every instance of the User ID<\/strong> – The User ID<\/em> feature in Universal Analytics groups together hits, sessions, and Client IDs. It is the highest possible level of abstraction and aggregation available in Google Analytics. However, the data rows do not expose if a hit\/session belongs to a specific User ID.<\/p>\n<\/li>\n<\/ol>\n So, the purpose of this post is to give you the tools to annotate your incoming hit-stream data with all the information listed above. For data integration, this is almost a necessity, especially if you have a complex mesh of systems across which you want to join arbitrary data.<\/p>\n In the following chapters, we\u2019ll build four Custom Dimensions<\/strong> and four data collection methods that will let you include this information in your data set. We will, of course, be using Google Tag Manager<\/a> to make things more manageable.<\/p>\n \n \n Subscribe to the Simmer newsletter<\/a> to get the latest news and content from Simo Ahava into your email inbox!\n <\/p>\n <\/span><\/p>\n I\u2019ve created a Custom Dimension for each of the four use cases. Two of the dimensions are session-scoped, and two are hit-scoped. The dimensions are:<\/p>\n Client ID<\/strong> – session-scoped<\/em> Custom Dimension that collects the Client ID set by Google Analytics<\/p>\n<\/li>\n Session ID<\/strong> – session-scoped<\/em> Custom Dimension that collects a randomized Session ID<\/p>\n<\/li>\n Hit Timestamp<\/strong> – hit-scoped<\/em> Custom Dimension that collects the actual timestamp of each hit in local time, with the timezone offset included<\/p>\n<\/li>\n User ID<\/strong> – hit-scoped<\/em> Custom Dimension that collects the User ID that gets set when someone logs into your website<\/p>\n<\/li>\n<\/ol>\n <\/p>\n <\/a><\/p>\n<\/div>\n Why a hit-scoped Custom Dimension for User ID, you ask? Well, the whole privacy discussion around user tracking is complicated, and I would rather be poked repeatedly in the eye with a dead ferret than be drawn into it. By tracking User ID with a hit-scoped Custom Dimension, you\u2019ll only collect the data from logged-in users<\/em>. As soon as they log out, or if they re-enter the website having logged out, User ID will not be sent with the hits. If the Custom Dimension were session-scoped<\/em>, or even user-scoped<\/em>, you would be collecting User ID for potentially logged-out visitors as well, and that might be in the grey area privacy-wise.<\/p>\n The end result of combining all this information will be something like this:<\/p>\n <\/p>\n <\/a><\/p>\n<\/div>\n In this (fictional) data export, you can see 7 unique hits, made by two different client IDs (e.g. different browsers or devices), which span across three distinct sessions, but are all made by the same, logged-in user. With layered information like this, you can build intelligent models using GA data alone, but the possibilities it offers for data integration are remarkable as well.<\/p>\n The most difficult one of these solutions to implement, by far, is Client ID collection, so we\u2019ll start with that.<\/p>\n (UPDATE 5 April 2018<\/strong>: I recommend using this The difficulty with Client ID is that the You could<\/em> get the Client ID from browser cookies, but if it\u2019s a first-time visitor to your site, chances are that the cookie hasn\u2019t been set by the time the Page View Tag fires, and you\u2019ll miss this information. Also, if there are multiple trackers on the page, how do you know which So, I\u2019ve opted for a different approach. I\u2019m sending the Client ID using a non-interaction Event Tag<\/strong>, which fires when the page has loaded. This almost certainly guarantees that the This particular data collection method will require the following ingredients:<\/p>\n Session-scoped Custom Dimension<\/strong>, to collect the data in Google Analytics<\/p>\n<\/li>\n Custom JavaScript Variable<\/strong>, which gets the correct Client ID<\/p>\n<\/li>\n Window Loaded Trigger<\/strong>, which fires when the window has loaded if<\/em> the Custom JavaScript Variable returns a valid value<\/p>\n<\/li>\n Event Tag<\/strong>, which sends a non-interaction<\/em> event to Google Analytics when the Window Loaded Trigger fires<\/p>\n<\/li>\n<\/ul>\n This one is easy. Browse to the Google Analytics Admin of the web property you want to track to, select Custom Definitions -> Custom Dimensions<\/strong>, and create a new Custom Dimension that looks like the one below.<\/p>\n <\/p>\n <\/a><\/p>\n<\/div>\n The important thing is to choose Session<\/strong> as the scope, and to make note of the index assigned to it.<\/p>\n Note that if you wish, you could just as well scope this to User<\/strong> instead. The Client ID, by definition, is the same for the user throughout, so it might make sense to scope it accordingly. User-scoped dimensions are a bit questionable in terms of privacy, but in this case I don\u2019t see any issue, as you\u2019re just exposing a dimension that exists anyway. Thanks to Michael Hayes for pointing this out in the comments!<\/p>\n The Custom JavaScript Variable is named {{Get Client ID for current Tracker}}<\/strong>, and it needs the following code:<\/p>\n The function contents are wrapped in a The code is designed so that it cycles through all the GA trackers on the page. Once it encounters a tracker object which tracks to the property ID returned by the {{GA Tracking Code}}<\/strong> variable, it returns the Client ID associated with this object.<\/p>\n This means that you will need to have a variable called {{GA Tracking Code}}<\/strong>, which returns a valid property ID (UA-XXXXXX-X). On my website, for example, it\u2019s a Lookup Table Variable, which returns my main property ID for everyone else, but for me it returns a different property ID. This is because I use this secondary property for debugging implementations.<\/p>\n The Window Loaded Trigger is pretty simple. It\u2019s your basic Page View Trigger, where you set the Trigger Type to Window Loaded<\/strong>. However, you will need an additional condition in it.<\/p>\n <\/p>\n <\/a><\/p>\n<\/div>\n The condition Get Client ID for current Tracker<\/em> does not equal false<\/strong> ensures that the Trigger only fires if the Custom JavaScript Variable you just created returns a valid value.<\/p>\n The Event Tag is very basic, except for two customizations. First, you need set its Non-Interaction<\/strong> value to true<\/strong>. This prevents the Event from being calculated into interaction metrics like Session Duration and Bounce Rate.<\/p>\n Also, you\u2019ll need to add a Custom Dimension to the Tag, via More Settings -> Custom Dimensions<\/strong>. Add the index number of the dimension you created in step 1 to the Index<\/strong> field, and add the Variable reference you created in step 2 to the Value field.<\/p>\n <\/p>\n <\/a><\/p>\n<\/div>\n You can see how the fields should look from the image above.<\/p>\n If you did everything correctly, you should see your sessions populating with a new Custom Dimension that you can add to your reports, and pull out of GA via the API or via the reporting interface, if you wish.<\/p>\n <\/p>\n <\/a><\/p>\n<\/div>\n The dimension contains the Client ID of the visitor. You can then use this in your backend, for example, when you want to stitch hits sent from the same For Session ID, we\u2019re using a randomized string that is sent with each Pageview hit to Google Analytics. The string changes with each Pageview, but this doesn\u2019t matter. Because you\u2019re sending it to a session-scoped Custom Dimension, only the last value you send will be applied to the hits in the session.<\/p>\n The required components are:<\/p>\n Session-scoped Custom Dimension<\/strong>, to collect the data in Google Analytics<\/p>\n<\/li>\n Custom JavaScript Variable<\/strong>, which returns a valid Session ID string<\/p>\n<\/li>\n Small modification to your Page View Tag<\/strong>, so that the Session ID is sent to Google Analytics<\/p>\n<\/li>\n<\/ul>\n This is pretty much the same step you went through in the previous exercise.<\/p>\n <\/p>\n <\/a><\/p>\n<\/div>\n Just remember to make note of the index number, again.<\/p>\n The Custom JavaScript Variable is aptly named {{Random Session ID}}<\/strong>, and it has the following code:<\/p>\n This script creates a pretty unique, randomized session ID. It does it by taking the hit timestamp in Unix time, adding a period, and following with a random string of alphanumeric characters. Because of the timestamp (accurate up to milliseconds), it\u2019s very improbable that two similar session IDs are ever created.<\/p>\n An example of a session ID would be: 1427856715104.jdubr7umobt9<\/strong>.<\/p>\n In your Page View Tag, add a new Custom Dimension under More Settings -> Custom Dimensions<\/strong>. Set the index number you got from step 1, and set the value to the Variable reference {{Random Session ID}}<\/strong> you just created.<\/p>\n By using only the Page View Tag, you\u2019ll be sending the Session ID with each page load. Only the last Session ID you send will remain, however, and all the hits in the session will automatically be annotated with this ID, thanks to the session-scoped Custom Dimension. If this is confusing, remember to read up on Custom Dimensions<\/a>!<\/p>\n By virtue of the Custom Dimension, you now have an identifier with which you can stitch together arbitrary, discrete hits in Google Analytics.<\/p>\n <\/p>\n <\/a><\/p>\n<\/div>\n Together with the hit timestamp, you can start building realistic visit paths, if that suits your fancy.<\/p>\n Hit timestamp is something you should send with every single hit you send to Google Analytics. This means that you\u2019ll need to modify all your Google Analytics Tags, which might seem like a chore.<\/p>\n Accuracy is, of course, completely up to you, and you can opt to only send the timestamp with Pageviews and Transactions, instead.<\/p>\n What you\u2019ll need:<\/p>\n Hit-scoped Custom Dimension<\/strong>, to collect the timestamp in Google Analytics<\/p>\n<\/li>\n Custom JavaScript Variable<\/strong>, which returns a valid timestamp string<\/p>\n<\/li>\n Modification to all your tags<\/strong>, to which you want the timestamp to be attached<\/p>\n<\/li>\n<\/ul>\n There\u2019s nothing spectacular about this one. Create a new Custom Dimension in GA Admin, and set its scope to Hit<\/strong>.<\/p>\n <\/p>\n <\/a><\/p>\n<\/div>\n Remember to make note of the index.<\/p>\n The Custom JavaScript Variable needs to return the timestamp in String format. Now, there are many ways you could do this, for example:<\/p>\n Get timestamp in Unix time (milliseconds since Jan 1, 1970), adjusted for client timezone<\/p>\n<\/li>\n Get timestamp in Unix time, converted to UTC<\/p>\n<\/li>\n Get timestamp as an ISO string, adjusted for client timezone<\/p>\n<\/li>\n Get timestamp as an ISO string, converted to UTC<\/p>\n<\/li>\n Get custom string, adjusted for local time or converted to UTC<\/p>\n<\/li>\n Something completely different<\/p>\n<\/li>\n<\/ul>\n In my setup, I wanted the timestamp to be customized for my own tastes. That means that I\u2019m parsing it to resemble an ISO timestamp, but I\u2019m using client local time including the timezone offset, so I can see just which timezone the user is in. So, for example, if a visitor comes from Finland, which is GMT+3 (stupid daylight savings time), the hit timestamp might look like this:<\/p>\n\n
_ga<\/code> cookie gets a unique Client ID, which is how the Google Analytics backend knows that the same \u201cUser\u201d has visited the site multiple times. This Client ID is not exposed in any dimension you can query via the API.<\/p>\n<\/li>\n
\n X<\/span><\/p>\n
\n The Simmer Newsletter<\/strong>\n <\/p>\n1. The Method<\/h2>\n
\n
![\"Custom](\"https:\/\/www.simoahava.com\/images\/2015\/03\/Custom-dimensions.jpg#ZgotmplZ\")
<\/p>\n![\"Example](\"https:\/\/www.simoahava.com\/images\/2015\/04\/data-export.jpg#ZgotmplZ\")
<\/p>\n2. Client ID<\/h2>\n
customTask<\/code> method<\/a> instead for sending the Client ID in a Custom Dimension. The tracker object method outlined below is more complicated and far more unreliable. With customTask<\/code>, you can send the Client ID with any tag you want, with 100% accuracy each time.)<\/p>\nga<\/code> interface you use to retrieve the Client ID doesn\u2019t perform well in real time, especially for Tags firing very early in the page load sequence (e.g. your Page View Tag).<\/p>\n_ga<\/code> cookie to access? You don\u2019t.<\/p>\nga<\/code> interface is up and running, and I can use it to pull the Client ID for the correct tracker. The correct tracker is identified by querying the property ID (UA-XXXXXX-X) associated with the tracker object.<\/p>\n\n
2.1. Session-scoped Custom Dimension<\/h3>\n
![\"clientId](\"https:\/\/www.simoahava.com\/images\/2015\/04\/client-id.jpg#ZgotmplZ\")
<\/p>\n2.2. Custom JavaScript Variable<\/h3>\n
function<\/span>() {\n try<\/span> {\n var<\/span> trackers = ga.getAll();\n var<\/span> i, len;\n for<\/span> (i = 0<\/span>, len = trackers.length; i 1) {\n if<\/span> (trackers[i].get('trackingId'<\/span>) === {{GA Tracking Code}}) {\n return<\/span> trackers[i].get('clientId'<\/span>);\n }\n }\n } catch<\/span>(e) {} \n return<\/span> 'false'<\/span>;\n}\n<\/code><\/pre>\n<\/div>\ntry...catch<\/code> block, so any errors and problems with loading the ga<\/code> interface are gobbled up. If you want, you can add your own error debugging code into the catch<\/code> block. The key thing is to make sure the Event Tag doesn\u2019t fire if there\u2019s a problem with retrieving the Client ID. This means that you might miss some hits, but since we\u2019re sending the information to a session-scoped<\/em> Custom Dimension, you only need one successful hit sent during the session.<\/p>\n2.3. Window Loaded Trigger<\/h3>\n
![\"Window](\"https:\/\/www.simoahava.com\/images\/2015\/04\/window-loaded-trigger.jpg#ZgotmplZ\")
<\/p>\n2.4. Event Tag<\/h3>\n
![\"Event](\"https:\/\/www.simoahava.com\/images\/2015\/04\/event-tag.jpg#ZgotmplZ\")
<\/p>\n2.5. End result<\/h3>\n
![\"Client](\"https:\/\/www.simoahava.com\/images\/2015\/04\/client-id-as-dimension.jpg#ZgotmplZ\")
<\/p>\n_ga<\/code> cookie together in meaningful ways.<\/p>\n3. Session ID<\/h2>\n
\n
3.1. Session-scoped Custom Dimension<\/h3>\n
![\"Session](\"https:\/\/www.simoahava.com\/images\/2015\/04\/session-id.jpg#ZgotmplZ\")
<\/p>\n3.2. Custom JavaScript Variable<\/h3>\n
function<\/span>() {\n return<\/span> new<\/span> Date<\/span>().getTime() + '.'<\/span> + Math<\/span>.random().toString(36<\/span>).substring(5<\/span>);\n}\n<\/code><\/pre>\n<\/div>\n3.3. Modified Page View Tag<\/h3>\n
3.4. End result<\/h3>\n
![\"Session](\"https:\/\/www.simoahava.com\/images\/2015\/04\/session-id-table.jpg#ZgotmplZ\")
<\/p>\n4. Hit timestamp<\/h2>\n
\n
4.1. Hit-scoped Custom Dimension<\/h3>\n
![\"Hit](\"https:\/\/www.simoahava.com\/images\/2015\/04\/hit-timestamp.jpg#ZgotmplZ\")
<\/p>\n4.2. Custom JavaScript Variable<\/h3>\n
\n
![\"Timestamp](\"https:\/\/www.simoahava.com\/images\/2015\/04\/timestamp-in-tag.jpg#ZgotmplZ\")
<\/p>\n