How Does Website Traffic Affect SEO Part 2

Googlebot Website Traffic Affect SEO

How Does Website Traffic Affect SEO Part 2Yes I remember the days when we would have to call and then we would basically pay $200 to have a human like go down the aisles like push the physical reset button on the server. So yeah those those times And then imagine you basically renting five servers somewhere in the data center Yeah, and that taking a week and then we come in and scoop up all your bandwidth Hey, we’re offline today because Google has its crawl day that that’s not what we want Yeah these days it’s more of like hacker news kind of moment waiting. Yeah, exactly So I feel like you have much more Considerate and yeah, we try to not overwhelm anyone and we respect the robots.txt. So that works within the crawl step as well And once we have the content, we can’t put strain on your infrastructure anymore. So that’s fantastic But modern web apps being mostly JavaScript We then put that in a queue.

And then once we have it we have the resources to render it We actually use another headless browser kind of thing. We call that the web rendering service then there’s other crawlers as well That might not have the capacity or the need to run JavaScript. This is like social media butts for instance They they come and look for metadata if that meta tag is coming into the JavaScript you usually have a bad time and they’re just like Sorry yeah, so that’s always been a big myth is and I remember when single page applications or SPAs really came into vogue a lot of People were really concerned. There’s a lot of FUD around if Crawlers in general don’t execute JavaScript, then they’re gonna see a blind page and how do you get around that? So so contextually within Googlebot it sounds like Googlebot executes JavaScript Even if it does do it at a later point.

Yes, so that’s Good, that’s good. But like is there anything that people need to be aware of beyond just oh well It’ll just run it and then it’ll see exactly the same thing as like a human with a phone or a desktop Let’s see. There’s a bunch of things that you need to be aware So the the most important thing is again, as you said, it’s deferred. It happens at a later point So if you want us to crawl your stuff as quickly as possible That also means we have to wait to find these links that JavaScript injects Wait, they’re basically we crawl we have to wait until javascript is executed Then we get the rendered HTML and then we find the link So there’s a nice little short loop that finds these links very relatively quickly right after crawling will not work right So we will only see the links after we render it and this rendering can take a while because the web is surprisingly big yeah, just a little bit like 30 trillion ducks in 2016 so I’ll say now there’s way more than that.

Yes more than that so so Robots.txt is very effective at being able to sort of tell much how to do a certain thing But in this scenario, like how do you tell that like, it’s Googlebot visiting your site? Yes question Yes So as we are basically using a browser in two steps one of the crawling and one is the the actual rendering At both of these moments we do give you the user agent header But basically there’s the string, it’s literally the string googlebot in there, that’s so right straightforward Yes, and you can actually use that to help with your SPA Performance as well. So okay as you can detect on the server side Oh, this is a googlebot user agent requesting you might consider sending us a pre-rendered static HTML Version and you can do the same thing for the others like all the other search engines and and social media Bots have a specific string saying that they are a robot Okay, so you can then basically go like oh in this case I’m not I’m not giving you the real deal that the single page app.

I’m giving you this HTML that we pre-rendered for you That’s called dynamic rendering we have ducks on that as well. The one thing that still doesn’t quite make sense to me is Does the Googlebot kind of have different contexts like Does it sometimes pretend that it’s sort like I I think of it as this little mythical creature that’s pretending to do certain things so like does it pretend to be on a mobile and then desktop like Are they different sort of I guess like user agents? Even though it still says Googlebot and can you differentiate between them you’re asking great questions because yes, we have different user agents So I’m not sure if you heard about more by first indexing being rolled out and happening I’ve heard that like it’s going to affect like how you’re ranked.

Potentially That’s two different things that get conflated so often. So mobile- first indexing is about us discovering your content using a mobile user agent and a mobile viewport. So we are using mobile user agents and and the user agent string says so if it says something about Android and the name And then you’re like aha. So this is the mobile Googlebot you have documentation on there There’s literally a Help Center article that lists all these things So we try to index mobile content to make sure that we have something nice to serve for people who are on mobile But we’re not pretending like random user-agents or anything that we stick to the user agent strings that we have documented as well And that’s more my first indexing where we try to get your mobile content into the index rather than the desktop content, huh? And then there’s mobile readiness or mobile friendliness If your page is mobile-friendly it makes sure that everything is within viewport and you have large enough tap targets and all these kind of lovely things and that just Is a quality indicator we call these signals we have over 200 of them That’s a lot So Googlebot collects all these signals and then stuffs them as metadata into the index And then when ranked we’re like, okay, so this user’s on mobile So maybe this thing that has a really good mobile friendliness Signal attached to it might be a better one and the thing where they have to like pinch zoom all the way out to be Able to read anything and then can’t actually deal with the different links because they’re too close to each other so that’s one of the many it’s not the signal it’s one of the many signals is one of the over 200 signals to to deal with I Had no idea.

They were 200 right? That’s like me I know that you’re not allowed to like share what they all out because like there has to be a certain mystique around it because I guess like a lot of SEOs abused that in the past. Yeah, yeah Unfortunately that is a game that is still being played and people are doing like weird stuff to try to game us And the interesting thing with this is with the 200 signals. It’s really hard to say which one gets you would like weights And they keep moving and they keep changing So it’s I love when people are like no let’s do this and then look my my rank changes like yeah for this one query But you lost on all the other queries because you did like really weird and funky stuff for that. So just Build good content for the users and then you’ll be fine I feel like that it feels like less effort as well and like constantly trying to yeah Yeah, but it’s not an easy answer, right? You’ll pay me to make you more successful on on Search engines and I come to you and say like so who are your users and what do they need and how could you express? That so that they know that it’s what they need That’s a hard one because that means I basically bring the ball back to you and you have to think about stuff and figure out strategically whereas if I’m like, okay, I’m just gonna you know Get you get you links, or do some funky tricks here and then you’ll be ranking number one.

That’s an easier answer It’s the wrong answer, but it’s the easier answer so people are like and links are the most important metric ever is I’m like, no, we have over 200 and it’s important, but it’s not that important and Chill out everybody, but this still happens. Yeah. I’m so glad it’s better now Like I feel I feel actually we’re at peace in general with SEO as well Suz thank you so much for being with me here and has been a great pleasure, you know Thanks for I like answering all of my weird and wonderful questions about the Googlebot. Did we bust some myths? I feel like we did Fantastic, I think that’s worth a high five. I say Thanks. Thanks join us again for the next episode of SEO myth-busting where Jamie Alberico and I will discuss if JavaScript and SEO can be friends and how to get there..

Googlebot Website Traffic Affect SEO

Leave a Reply