SEMrush keeps taking down my site


My site is being absolutely bombarded by page requests originating from Its a bot that is indexing and downloading links to my entire site, which would be fine if it were a small website, except that I have content going back to 1996, which is rather a lot, and they are hitting the server with so many requests that Apache2 and/or MySQL Server 5.7 is crashing.

It’s utterly frustrating, and they have taken the site down 3 times in the past couple of hours. Their bot should be rate-limited so that it doesn’t cause issues like this! The site can cope with 100+ simultaneous visitors as it is running on an AWS T3 micro instance, which should be more than capable, but unfortunately, its not capable of dealing with the SEMrush bot.

In the coming hours I will be working on ways to permanently blacklist their services, and I suggest that other website owners do the same so that the same misfortune does not befall their site too.

1 Comment

Leave a Reply

  • You need to block their user agent via htaccess or apache level. This will deny them from consuming any resources at all. I did this on my site and load dropped from 100 to <1 and # of rewuests from 300/sec to 20/sec. Its day and night difference. Main offending user agents are m12bot semrush ahrefsbot blexbot (




Get in touch

I'm available on just about every social media platform out there as RMTWeb.