Welcome back to AIMCLEAR‘s coverage of #SMX East 2011! Breaking News: Changing SEO elements of your website, e.g.: title tags, Meta descriptions, anchor text, interlinking structures, can have a tremendous effect on organic rankings. Okay, so maybe you don’t need spades of sophisticated know-how and online marketing agility to know that little tidbit. Pretty straight-forward: Update your site and take note if a ranking change occurs. One did? Great! Was it causal or correlative?
Was it what?
The ability to successfully distinguish between website updates that cause a change in rank and updates that merely correlate with change in rank is something deep marketers must possess in order to be a master craftsman craftswoman SERP-ninja. But hang on a tick… are there specific SEO changes that directly cause a change in your SERP position? Surely those are golden tickets every sitemaster should implement straight away!
Or is it more of an indirect effect? A mere, yet telling, correlation?
Fortunately for any frazzled marketers at #SMX East 2010, moderator Danny Sullivan, Q&A moderator Max Thomas, and speakers Eric Enge, Micah Fisher-Kirshner, Mitul Gandhi, Kristine Schachinger, and Tony Wright were set to take the stage and weigh in on Great Debate of Correlation vs. Causation. AIMCLEAR live-tweeted this session via @beebow. Fruits live after the jump.
Danny welcomed attendees and introduced speakers before next introducing the session itself: Causation vs. correlation was a big SEO theme from this year’s #SMX Advanced over in Seattle, earning it a spot at the table this go-round.
As a site owner, it’s crucial to be mindful of the difference between causation and correlation. Is X the cause of Y? Or X and Y are correlated?
Two presentations, delivered by Kristine Schachinger and Micah Fisher-Kirshner were about to tackle that very concept, after which the panelists would each get a turn to share thoughtful responses, and engage in a bit of a… well… debate.
Kristine was up first, ready to address, at a high-level, what correlation and causation are.
What is correlation? Correlation describes a relationship between two or more things that have a measurable effect on each other (things = variables).
What is causation? Causation is a relationship in which one action or event is the direct consequence of another.
In every causal relationship, there is a correlative relationship, but not always vice versa.
Oftentimes, correlation is mistaken for causation, meaning A is perceived to cause B, but it may or may not have a measurable effect.
In SEO terms, we could make an assumption based on old ideas or incorrect perception (a big no-no), for example, the effect of updated site code on rankings, pre-Google’s site speed announcement. When we have a skewed perception, we either have no correlation (think there’s one) or a spurious correlation.
A what?
Spurious correlation: (noun) a perception that A affects B, but in actuality a third hidden variable does. (Sneaky!)
Example:Â New York City might see a sudden spike in alcohol sales when oodles of SEOs are in town for SMX East. That spike, however, might be attributed to Fashion Week, which happens to coincide with SMX this year.
How to avoid such a slip-up?
- Test and measure so that you can fully understand the relationships.
- With causal relationships, you can directly measure that A does cause B.
- In correlative relationships, you can only measure the strength of the effect of A on B.
- Strength can be described by a factor known as a confidence interval, or confidence level, in the form of, for example, margin of error: + or – X%.
Returning to the original takeaway: You have to test everything to understand effect. As you test, make sure you are aware of the variables, sample size, outliers (do you control them?), your analysis method of choice (is it the proper one?), and the limitations of your conclusions, if they exist?
Kristine wrapped up by asking us the question we were all hoping to answer, namely: Is SEO correlative or causative?
Her answer: Most of it is correlative. Period.
There you have it. Micah was up next, ready to dig deeper into correlation vs. causation.
Perhaps one inspiration for this debate-style session at SMX East was the in-depth study SEOmoz CEO & Co-Founder, Rand Fishkin, presented attendees at SMX Advanced just three months back. That said, distinguishing good (accurate) vs. bad (inaccurate) public correlations presentations was the topic Micah kicked off with, just in case you’re interested in doing some hardcore testing on your own site with the hope of turning it into a presentation.
Elements of Good Correlation Presentations
- They’re shown with scatter plots. These help visualize the correlation & what you’re trying to prove / show.
- They use non-integer numbers. Generally, integers make it harder to visualize correlations
Some things to keep in mind and questions to ask if you’re considering compiling a preso of your own:
- Remind yourself that correlations are a place to start, not end.
- Sanity check the results with an SEO.
- Ask yourself, “Does this make sense? What if we factor for…”
- Continuously ask questions – examine data, dissect it, test it again. This is the only way you will get a better sense of what’s going on with your correlation.
- Ask yourself, “Do we have enough data to prove it is statistically significant? How long will data collection take to prove it?”
- Make sure you understand what the ROI is to collect and test data.
- Make sure you’re working with quality data. Do you trust where the data comes from? Do you have enough metrics that factor into algorithm?
- Does the correlation hold up by site type? Or is this a punishment-by-group type deal? Are how-tows, CSEs, marketplaces, news sources all being spanked?
When the time comes to test and understand data, be sure to look at metrics day over day, week over week, and stay mindful of seasonality, not to mention holidays or world events that can skew data.
Best of the tests focus on subdirectories, subdomains, and domains. Micah recommends finding ways to split your site in half along standard site architecture to run tests. If you don’t have these, use numbers in URL or even / odd.
Questions to Ask
Common correlating pitfalls can be other marketing channels. Or, it can all be internal. Ask yourself, did our brand team launch something? Did the UX team change things? Did something break? Did you launch another test on your site?
Extraneous online events can skew data, too. Did Google update its algorithm during your test? Did our analytics platform have a tracking events update?
“Proving requires digging.” –Micah Fisher-Kirshner
In case you didn’t glean this already: When looking at correlation presentations, question everything. Don’t just accept it. Drill deep. Ask questions. Keep digging. Did we mention, “Ask questions”?
With those pertinent words of advice, Micah wrapped up, and our remaining panelists got up to share their elevator speech about causation vs. correlation.
Here’s some high-level takeaways:
“A single statistic does not a trend make.” In other words: Test, test, test. Ask questions, ask questions, ask questions. –Mitul Gandhi
Tony Wright’s 4 Easy Steps to Better Rankings!
Do these things better than your competitor and you will outrank them, 95% of the time.
- Code. Have good code
- Content. Have good content.
- Connections. Have good inbound links.
- Conversations. Be social.
Do these 4 things cause better rankings? Tony can’t say, statistically, that they do. But do they help? Yep.  (Again… 95% of the time.)
Eric Enge shared some insight worth noting. To paraphrase, correlation and causation don’t matter – just do your best to build great stuff that causes both of those things to happen. When you think about what you’re going to do on a given day, think big picture. Think about what will cause all of those signals to work in your favor.
And that’s all she wrote! Thanks to the panelists for an awesome session. Stick around AIMCLEAR blog for more conference coverage coming atcha from #SMX East 2011.
