I would recommend putting your robots.txt file in your root directory. You could get a little fancy, though, and put a redirect in place in your root .htaccess file.

Redirect robots.txt http://hamelife.com/wpblog/robots.txt

I would still put an XML sitemap in place as well using the plugin I specified in 1. An XML sitemap is in accordance with the Sitemap standard. This sitemap will rebuild with every post and automatically ping the search engines as well. It’s very good.

If you put in the sitemap and it’s in the wpblog directory as well, you can also put a redirect to it as well:

Redirect sitemap.xml http://hamelife.com/wpblog/sitemap.xml

Use caution and backup your htaccess file (it may be hidden so be sure to view hidden files with your FTP software) in case you hose it up!

Then be sure to add the sitemap line to your robots.txt file. If you’ve got the redirect line in you htaccess file, you could use either URI… I would probably default to the literal, though:
Sitemap: http://hamelife.com/wpblog/sitemap.xml ]]> http://www.marketingtechblog.com/2007/07/08/blog-tipping-7/#comment-51052 Rory Sun, 08 Jul 2007 21:19:41 +0000 http://www.douglaskarr.com/2007/07/08/blog-tipping-7/#comment-51052 I've just implemented your 404.php, Doug. Try this: <a href="http://hamelife.com/child/" rel="nofollow">hamelife.com/child</a> Now, <em>that's</em> a better 404 page! Again - Many Thanks. I’ve just implemented your 404.php, Doug.

Try this: hamelife.com/child

Now, that’s a better 404 page!

Again - Many Thanks.

Are you kidding, Doug? I am so thankful to you for casting a helpful eye over the site. I need all the advice I can get, and I really appreciate the pointers you have given.

First off, I am blown away by your compliments. Thank you so much - they are enormously encouraging.

Now, I wonder if you could help me with your further tips:
Point #1 - The M of hamelife takes a visitor to a sitemap. I used the Sitemap designed for Cutline. Do I have this configured correctly? Or should I move directly to Point #2 and configure my robots.txt correctly?

My robots.txt actually reads:

User-agent: *
Disallow: /wp-content/
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-
Disallow: /feed/
Disallow: /trackback/
Disallow: /cgi-bin/

and I have it located in hamelife.com/wpblog/robots.txt. Something doesn’t seem right with this - I feel like it should be located elsewhere, but I’m not sure how to do it. I could use the help.

My set up is to have the blog in hamelife.com/wpblog, so I don’t have it in the root directory. You can see how confused this is getting me!

Point #3 - Thank you for your suggestion, I’ll have to think more about it. I’m leaning more towards a second blog. I know it means keeping two lots of content, but I’m learning to be a bit more laid back about this blogging thing - not be so stressed.

Point #4 - I’m looking at your Wordpress 404 article as we speak. It is excellent advice and I will implement it as soon as possible.

Thanks again, Doug. Your help is much appreciated.