yetangitu/Oinktube
1
0
Fork 0
mirror of https://github.com/DanielnetoDotCom/YouPHPTube synced 2025-10-03 01:39:24 +02:00
Code Issues Releases Wiki Activity

Block Bad Website Bots and Spiders

Browse source 
Piped way to add bad crawlers to blocked list, that are slowing and consuming most of the traffic .
The other way would be the rDNS
This commit is contained in:
akhilleusuggo 2021-04-03 00:44:25 +03:00 • committed by GitHub
parent b247cafa36
commit 1503c9fa00
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 6 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

6
.htaccess
View file

@ -1,4 +1,10 @@
# BEGIN Expires Don't forget to enable mod_headers and mod_expires, you can do so by running a2enmod rewrite and a2enmod headers
#Block Bad Website Bots and Spiders.
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^.*(SCspider|BLEXBot|Textbot|s2bot|MJ12bot|CCBot|SemrushBot|SeznamBot|GrapeshotCrawler|Adsbot|PetalBot|Bytespider|ZoominfoBot|Nuclei|Barkrowler|Barkrowler|AhrefsBot|MegaIndex|CriteoBot|NetSystemsResearch|CensysInspect).*$ [NC]
RewriteRule .* - [F,L]
<ifModule mod_expires.c>
ExpiresActive On
ExpiresDefault "access plus 1 seconds"