SEMrush keeps taking down my site

My site is being absolutely bombarded by page requests originating from semrush.com. Its a bot that is indexing and downloading links to my entire site, which would be fine if it were a small website, except that I have content going back to 1996, which is rather a lot, and they are hitting the server with so many requests that Apache2 and/or MySQL Server 5.7 is crashing.

It’s utterly frustrating, and they have taken the site down 3 times in the past couple of hours. Their bot should be rate-limited so that it doesn’t cause issues like this! The site can cope with 100+ simultaneous visitors as it is running on an AWS T3 micro instance, which should be more than capable, but unfortunately, its not capable of dealing with the SEMrush bot.

In the coming hours I will be working on ways to permanently blacklist their services, and I suggest that other website owners do the same so that the same misfortune does not befall their site too.

WordPress: The Dreaded “Error Establishing Database Connection” message

My site has just been down for more than 24 hours due to a wordpress update, that in its wisdom the plugin developer assumed that the version of MySQL would be an older version, and corrupted the WordPress database that the site uses.

I’ve recovered the site from last nights backup, and thankfully there has been no data loss, but its still a timely reminder as to the importance of regular backups of both files and the database of even a small WordPress site.

Needless to say, panic averted, and automatic plugin updates are now turned off!

WordPress – Pinning posts to the home page

I was just asked by a local school how to make a post always appear on the home page – the answer is surprisingly simple, but if you don’t know where to look its rather difficult to find!

screen-shot-2016-11-14-at-10-42-20

On the publish box to the right of the wordpress post in the editor, click on the Edit link next to Visibility: Public and then tick the Stick this post to the front page box.

screen-shot-2016-11-14-at-10-43-46

Done! The post now stays at the top of the front/home page. All you have to remember to do is to un-stick it in the future!