Jump to Navigation

We've moved! The new address is http://www.henriettes-herb.com - update your links and bookmarks!

You'll find a list of all my blog posts in the blog archive.

Site news

Blog categories:

'kay then, I've moved things to henriettesherbal.com.

On the 10th of November, somebody wrote to the ibiblio collection folks list, asking if anybody else had trouble accessing mysql.

On the 11th of November, I started to get emails telling me that the plant name search on my main site (the old ibiblio.org/herbmed setup) was wonky.

I asked ibiblio about it, and they said: "yep, it's excessive spidering of your content, crashing our mysql server". They also asked me to cache things, and told me that they'd disabled my mysql account and would enable it as soon as the excessive spidering was over and done with.

Bleh for rude robots (= spiders), says I.

Now, I have no idea how to cache things, unless you do it as explained in the mysql manual, by adding caching to the config files ... which the ibiblio admins have access to. I don't.

I've tried to block bad bots on ibiblio a few months ago, but you can't really block anything on ibiblio, cos they proxy the lot locally.

So I told them to add the .php scripts in my /pictures/ directory to their robots.txt: the content can be picked up by simply going from .htm file to .htm file.

No reply.

I then told them about the Crawl-delay: 90, which you can add to robots.txt, which means that spiders which honor this particular entry only will pick up one page every 90 seconds.

No reply.

They have access to the ibiblio robots.txt file, I don't.

On dreamhost (the host of this site), the blocking of bots works admirably, and I have my own robots.txt, which I can change as much as I like.

On the 11th and the 12th of November I copied my pics and the classic texts to henriettesherbal.com (22+ hours for the pics, 5+ hours for the classics...), just in case ibiblio was still wonky come Monday (November 14th).

My ibiblio mysql account is still closed today, Tuesday November 15.

So, without further ado:

Henriette's herbal pages are now found here.

The google search still points back to ibiblio; dunno if google will pick up things here, how long it'll take them, and so on, or if perhaps yahoo will spider this site more completely ... but at least the plant name search works now.

Please do tell me about all and any wonkiness on the new site. I might have missed a directory or two in my ftp'ing or in my changing of the files which needed changing.

Many thanks.

--
Update 1: Bummer, it looks as if the "Crawl-delay: 90" means that robots don't spider this site at all.
--
Update 2: Can't have google site search pointing to ibiblio from here, so I've changed the lot. Not a lot of results to be had yet ...

Comments

Hi Henriette

If you throw this between the head tags of your Ibiblio homepage, it will automatically redirect here.

Google indexing is all over the place (same with Alta Vista and Yahoo)....

Glad to hear the move's behind you. Dreamhost is a good choice.

Shayne

Thanks, Shayne.
I'm told that search engine spiders don't follow redirects to different domains, though.

Yep....sadly there's nothing we can do to wrangler the index spiders to any given location. I was thinking more for the user experience.

As always, thanks for the herbal dialogue!!

True, but a "Hey, where'd the site go?" might get people to change their bookmarks and links ...

Henriette,
TJ has some time and wants to work with you on optimising the database, making some static pages and dealing with spiders. This would be on ibiblio, of course. I think he's written you today. If not, give me a holler.

I feel so on top of things... already updated the link to your site from mine... now I'll have to go through a years worth of class handouts and see how many times in those I'll need to fix...

Thanks Paul, emailed.
Thanks jim, that was fast.



Main menu 2