Shortcuts

12. Bot blocking

12.1. Configuration

You should set enable_bots_blocking: True on the sc_pack config file, and with it: humanity_validator_host, humanity_validator_port, google_recaptcha_site_key, google_recaptcha_site_secret, and shimmercat/bots_views_dir. Have into account that google_recaptcha_site_key and google_recaptcha_site_secret comes from the reCAPTCHA sites you must create at https://www.google.com/recaptcha/admin. And also you should add the domain to the reCAPTCHA site those credentials belong to.

  • Create the consultant part for the bots on the devlove.yaml. The consultant key must be bots e.g:


    shimmercat-devlove:
      domains:  
        example.test.com:
          root-dir: www
            consultants:
                bots:
                    connect-to: 127.0.0.1:8088
                    application-protocol: http

Note that the current configuration is just an example file, for more details please see here.

  • Run the create command to build the bots blocking infrastructure:

    $ sc_pack create -f configuration/file/path/sc_pack.conf.yaml
    
  • If you have a specific bots blocking html customized, please remember to change the one we automatically created under: <views-dir>/.well-known/shimmercat/bot-blocking/

  • Restart the service:

    $ systemctl restart example_test.com
    

12.2. Mark and unmark an IP address as bot

You can block an IP address on a specific sc_pack instance with the command option mark_as_bot <ip_address>. For instance:

$ sc_pack mark_as_bot 127.0.0.9 -f configuration/file/path/sc_pack.conf.yaml

Unmarking an IP address as bot is similar as marking it, but the command name changes:

$ sc_pack unmark_as_bot 127.0.0.9 -f configuration/file/path/sc_pack.conf.yaml

12.3. Remove bots ip addresses

You can remove all the bots ip addresses from redis, which means that you are removing them as bots. You are able to remove all of them, or for a specific country specifying a country ISO code. When we refer to the term ‘country ISO code’ we are talking about the ‘ISO Alpha-2 code’ here.

To remove all the bots do:

$ sc_pack remove_bots all

To remove for a specific country, let’s say Sweden:

$ sc_pack remove_bots SE

It does not matter if the country code, or the all argument is in uppercase or not, we convert it automatically.