\n<\/p>\n
If you follow me on Twitter<\/a>, then you know I sometimes complain about the current state of the industry \u2013 most notably centered around what passes for research and discussion these days. It feels like people want to be handed the fish \u2013 with little interest in learning how the person with the fish caught it. Seeking out debates and experience seems to have been replaced by wanting to be spoon fed blog posts \u2013 often laced with assumptions and misinformation hidden within a single-case graph or a slick graphic coupled with an impressive looking byline. <\/p>\n Separating fact from fiction becomes increasingly hard as the next generation of our industry raises themselves up by following rather than exploring<\/em>.<\/p>\n At SMX West<\/a> Google engineer Paul Haahr gave a presentation that gave some insight into how Google works. In my opinion, it’s the most transparent and useful information we’ve been presented with by Google in years. <\/p>\n I spent this morning taking a full set of notes on it \u2013 which I always do with something like this because I believe it helps me better retain the information. Within my notes, I make notations about questions I have, theories I come up with and conclusions I draw \u2013 right or wrong. This isn’t the sexy work, but it’s the necessary work<\/em> to be a formidable opponent in the game.<\/p>\n As I looked at the notes, I realized I missed the discussions, debates, and sharing of experience that used to surround analyzing information like this.<\/p>\n Some of what I feel limits that kind of discussion these days is needing to be seen as an infallible expert on all things Google. The industry has become so consumed with being an expert that it’s afraid to ask questions or challenge assumptions for fear of being proven wrong. Unlike many of the names within this industry, I’m not afraid to be wrong. I welcome it.<\/em> Being proven wrong means I have one more piece of concrete knowledge needed to win the game. Being questioned or challenged on a theory I hold gives me another theory to test.<\/p>\n So I’m publishing my notes \u2013 and personal notations \u2013 and am making a call for a real, exploratory \u2013 fuck it if I’m right or wrong<\/em> \u2013 search discussion. Whether you’re old school with massive experience or new school with untested theories and ideas \u2013 bring it to the table and let’s see what we can all walk away from it with.<\/p>\n Speaker \u2013 Paul Haahr<\/a> | Presentation Video<\/a> | Presentation Slides<\/em><\/p>\n To be clear, these are my notes from the presentation and not a transcription (notations in orange are comments made by me and not the speaker)<\/span>.<\/p>\n Timestamp: 3:38 \u2013 Link to timestamp<\/a><\/em><\/p>\n Haahr infers that this next bit of information is a 20 minute secret-sauce stripped version of the half-day class attended by every new Google engineer.<\/p>\n He starts by explaining the two main parts of the search engine:<\/em><\/p>\n 1. What happens ahead of time (before query):<\/p>\n 2. And query processing:<\/p>\n Timestamp: 8:49 \u2013 Link to timestamp<\/a><\/em><\/p>\n Timestamp: 10:10 \u2013 Link to timestamp<\/a><\/em><\/p>\n Timestamp: 12:00 \u2013 Link to timestamp<\/a><\/em><\/p>\n Metric optimization ideas and strategies are developed through an internal evaluation process that analyzes results from various experiments:<\/p>\n Timestamp: 12:33 \u2013 Link to timestamp<\/a><\/em><\/p>\n Example given for interpreting live experiments: Page 1 vs. Page 2<\/p>\n But in that scenario, was Algorithm A bette<\/em>r than Algorithm B? The second scenario should be a \u201cgood\u201d result because the user got a good answer \u2013 faster \u2013 from the snippet. But it’s hard for the algorithm to evaluate if user left the SERP because the answer they needed wasn’t there or if they left because they got their answer from a snippet.<\/em><\/p>\n This scenario is one of the reasons they also use human quality-raters.<\/p>\n Timestamp: 15:21 \u2013 Link to timestamp<\/a><\/em><\/p>\n Other notes:<\/p>\n \u2014 States they do it human quality-rater experiments for large query sets to obtain statistical significance and cites it as being similar to Mechanical Turk like processes<\/p>\n \u2014 Mentions that the published rater guidelines are Google’s intentions for the types of results they want to produce (notation \u2013 this is very different than a user rating a query based on personal satisfaction \u2013 instead they are told to identify if the results for the query meet Google’s satisfaction requirements and include the kind of results Google believes should be included \u2013 or not included. Quality rater guidelines are the results produced by the Google dream algorithm.)<\/span><\/p>\n \u2014 He says if you’re ever wondering why Google is doing something, it is most often them trying to make their results look more like the rater guidelines.<\/em> (notation \u2013 Haahr reiterated to me on Twitter<\/a> how important he believes reading the guidelines is for SEOs.)<\/span><\/p>\n \u2014 Slide showing human rater tools: Slides 33, 34<\/em><\/p>\n \u2014 Re mobile first \u2013 more mobile queries in samples (2x)<\/p>\n Timestamp: 19:04 \u2013 Link to timestamp<\/a><\/em><\/p>\n Are the needs as defined by Google<\/em> met?<\/p>\n Timestamp: 23:58 \u2013 Link to timestamp<\/a><\/em><\/p>\n The three most important concepts for quality:<\/p>\n The rating scale is high quality to low quality:<\/p>\n Timestamp: 25:28 \u2013 Link to timestamp<\/a><\/em><\/p>\n The primary goal for all features and experiments is to move pages with good ratings up and pages with bad ratings down. <\/em>(notation \u2013 I believe he means human ratings, but that was not clarified.)<\/span><\/p>\nNotes from the How Google Works presentation \u2013 SMX West 16<\/h2>\n
General opening remarks<\/h2>\n
\n
Life of a query<\/h2>\n
\n
\n
\n
\n
does this mean using a dedicated server can be a bonus? Should you check shared hosts for sites with similar topics? Confirms need for separate hosts for networked or related sites?<\/strike> this has been clarified by a former Googler, see this comment<\/a> for more detail. The tldr is that host clustering<\/em> is synonymous with domain clustering<\/em> (the more widely used term in the industry) and site clustering<\/em> and does not refer to host as in hosting.)<\/span><\/li>\nWhat engineers do<\/h2>\n
\n
\n
Key metrics for rankings<\/h2>\n
\n
\n
Optimizing metrics<\/h2>\n
Live experiments<\/h3>\n
\n
\n
\n
Human quality-rater experiments<\/h3>\n
\n
\n
Rating types<\/h3>\n
Assessing relevancy<\/h4>\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
Assessing page quality:<\/h4>\n
\n
\n
\n
\n
\n
\n
\n
Optimizing the metrics \u2013 the experiments<\/h2>\n
\n
\n
\n
\n
Two of the core problems they face in building the algorithm<\/h2>\n