Thread: robots.txt file in classipress blocking google bot

robots.txt file in classipress blocking google bot

Hello,

My site has not yet been ranked by google, but already has an Alexa rank of 650,000ish after being online for 3 months.

For some reason google denied my first application for Adsense, due to a crawl issue. It was accepted this week after I re-submitted it.

Still no page rank though. I just used google site diagnostics and the report was that the crawl was blocked due to Robots.txt File. I'm not sure why my classipress is having these issues, and others aren't.

Googles instructions to fix this are: "If you would like to grant our crawler access to your pages, you can do so without granting permission to any other bots. Simply add the following two lines of text to the top of your robots.txt file:

User-agent: Mediapartners-Google
Disallow:

This change will allow our crawler to index the content of your site and provide you with the most relevant Google ads for that content."

So I have two questions for the classipress support crew.

1 - Is this the right fix for classipress and my site?
2 - Where do I find the robots.txt file?

Thanks

Re: robots.txt file in classipress blocking google bot

ClassiPress nor WordPress in general don't include a robots.txt file out of the box. You need to manually create one or your website already had one.

What's the url to your robots.txt file? It should be your domain name /robots.txt

Re: robots.txt file in classipress blocking google bot

Thanks for the reply.
The domain with the classipress installation is a sub-domain.

grey-bruce.boogiboard.ca

Re: robots.txt file in classipress blocking google bot

So here's your robots file which actually looks fine. You are even specifying a sitemap file which is great.

http://grey-bruce.boogiboard.ca/robots.txt

Your site has already been indexed by Google (249 pages as of writing this post) which is a good sign. It appears to be working fine so I'm not sure why the Google tool is telling you differently.

It does, however take time to rank high for any term when your site is brand new. Be patient and start accumulating inbound links and eventually Google will assign you some page rank love.

Re: robots.txt file in classipress blocking google bot

Thanks again.

I was thinking it may be the time thing. Just frustrating as a site owner, when you want to compare to other sites in your region (especially when your intent is the advertising $).

I appreciate your time.

egghead