Shortcuts

2. Group Bot

This section groups opt_pipeline Bots handling.

2.1. BotsAccessPolicyView [/presentation-api/v1/bot-access-policy/]

2.1.1. BotsAccessPolicyView [POST]

Define the bots access policy. You can define policies per domain, or a general policy for all the domains. Below is an example on how to change the general bot policy to "Unreviewed bots can access the domain":

curl -X POST https://accelerator.shimmercat.com/presentation-api/v1/bot-access-policy/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json' -d '{"general_policy": {"unknown_bots_can_access": true}}'

When you define the domain policy, we use this policy to classify bots that access the domain when the bot is not classified automatically. If the domain policy has not been defined then we use your general policy to do the classification.

By default the domain policy is null, and the general policy is "Unreviewed bots can not access the domain". This means that unreviewed bots are blocked by default, and that they can not access your domains. We suggest that you take action over the bots that we can not automatically classify as good or bad, so that you can have more control over them. For this reason we can send out email notifications with bot information so that you can use the API endpoint. In general, for policy "Unreviewed bots can not access the domain" we send out a daily email report for you to review, and for policy "Unreviewed bots can access the domain" we send out a weekly email report for you to review.

On every request to the bot API endpoint you need to specify at least one of the objects: allow_access or deny_access, general_policy, per_domain_policy. The data structure for those objects is specified below. You can not repeat the same bot in both lists allow_access and deny_access. If that happens we will raise a validation error because it does not make sense to allow and deny the same bot at the same time.

Below is an example, using curl, on how to request the endpoint to allow access to Google bot on the domain www.shimmercat.tests.com:

curl -X POST https://accelerator.shimmercat.com/presentation-api/v1/bot-access-policy/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json' -d '{"allow_access": [{"domain": "www.shimmercat.tests.com", "bot": "Google"}], "general_policy": {"unknown_bots_can_access": true}, "per_domain_policy": [{"domain": "www.shimmercat.tests.com", "policy": {"unknown_bots_can_access": true}}]}'
  • Request (application/json)

      {
          "allow_access": [{"domain": "www.shimmercat.tests.com", "bot": "Google"}],
          "deny_access": [{"domain": "www.shimmercat.tests.com", "bot": "hacker-bot"}],
          "general_policy": {"unknown_bots_can_access": "true"},
          "per_domain_policy": [{"domain": "www.shimmercat.tests.com", "policy": {"unknown_bots_can_access": "true"}}]
      }
    
  • Response 200 (application/json)

      {
          "allow_access": [{"domain": "www.shimmercat.tests.com", "bot": "Google"}],
          "deny_access": [{"domain": "www.shimmercat.tests.com", "bot": "hacker-bot"}],
          "general_policy": {"unknown_bots_can_access": "true"},
          "per_domain_policy": [{"domain": "www.shimmercat.tests.com", "policy": {"unknown_bots_can_access": "true"}}]
      }
    

2.1.2. BotsAccessPolicyView [GET]

Get the bots access policy the customer has active. Below example using curl about how to request that endpoint:

curl -X GET https://accelerator.shimmercat.com/presentation-api/v1/bot-access-policy/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json'
  • Response 200 (application/json)

      {
          "general_policy": {"unknown_bots_can_access": "True"},
          "per_domain_policy": [{"domain": "www.shimmercat.tests.com", "policy": {"unknown_bots_can_access": "True"}}]
      }
    

2.2. ListCustomerUnreviewedBotsView [/presentation-api/v1/unreviewed-bots/]

2.2.1. ListCustomerUnreviewedBotsView [GET]

Get the list of unreviewed bots.

Below example using curl about how to request that endpoint:

curl -X GET https://accelerator.shimmercat.com/presentation-api/v1/unreviewed-bots/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json'
  • Response 200 (application/json)

      [
          {
              "domain_name": "www.shimmercat.tests.com",
              "domain_policy": "Unreviewed bots can access the domain",
              "bots": [
                  {
                      "name": "Google",
                      "description": null
                  }
              ]
          }
      ]
    

2.3. IsIPAddressBadBot [/presentation-api/v1/is-bad-bot/{ip}/]

  • Parameters

    • ip: 127.0.0.1 (string) - Ip address to verify.

2.3.1. IsIPAddressBadBot [GET]

Checks if the ip address was already registered in our database as a bad bot. An example of a request using curl:

curl -X GET https://accelerator.shimmercat.com/presentation-api/v1/is-bad-bot/127.0.0.1/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json'
  • Response 200 (application/json)

      {
        "is_bad_bot": true,
        "ip_address_or_range": "127.0.0.1"
      }
    

2.4. IsIPAddressRangeBadBot [/presentation-api/v1/is-bad-bot/{iprange}/]

  • Parameters

    • iprange: 69.208.0.0_31 (string) - Ip address range to verify.

2.4.1. IsIPAddressRangeBadBot [GET]

Checks if the ip address range was already registered in our database as a bad bot. An example of a request using curl:

curl -X GET https://accelerator.shimmercat.com/presentation-api/v1/is-bad-bot/69.208.0.0_31/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json'
  • Response 200 (application/json)

      {
        "is_bad_bot": true,
        "ip_address_or_range": "69.208.0.0/31"
      }
    

2.5. ListCustomerBadBotsView [/presentation-api/v1/list-bad-bots/]

2.5.1. ListCustomerBadBotsView [GET]

Get the list of bad bots reported by the customer. An example of a request using curl:

curl -X GET https://accelerator.shimmercat.com/presentation-api/v1/list-bad-bots/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json'
  • Response 200 (application/json)

      [
          {
              "name": "test",
              "ip_addresses": [
                  "127.0.0.1"
              ]
          }
      ]
    

2.6. MarkIpAsBadBotView [/presentation-api/v1/mark-as-bad-bot/{ip}/]

  • Parameters

    • ip: 127.1.1.1 (string) - Ip address to mark as bad bot.

2.6.1. MarkIpAsBadBotView [POST]

Mark the ip address as bad bot. You need to specify a domain which you want to add this bot to, and the bot name you will attach that ip address to. If a bot with that name does not exist for one of your domains we will add a new one with the specified name. An example using curl about how to request that endpoint to block the ip address 127.1.1.1:

curl -X POST https://accelerator.shimmercat.com/presentation-api/v1/mark-as-bad-bot/127.1.1.1/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json' -d '{"attach_to_bot": "test", "attach_to_domain": "www.shimmercat.tests.com"}'
  • Request (application/json)

      {
          "attach_to_bot": "test",
          "attach_to_domain": "www.shimmercat.tests.com"
      }
    
  • Response 200 (application/json)

      {
          "name": "test",
          "ip_addresses": [
              "127.1.1.1",
              "127.0.0.1"
          ]
      }
    

2.7. MarkIpRangeAsBadBotView [/presentation-api/v1/mark-as-bad-bot/{iprange}/]

  • Parameters

    • iprange: 69.208.0.0_31 (string) - Ip address range to mark as bad bot.

2.7.1. MarkIpRangeAsBadBotView [POST]

Mark the ip address range as bad bot. You need to specify a domain which you want to add this bot to, and the bot name you will attach that ip address range to. If a bot with that name does not exist for one of your domains we will add a new one with the specified name. An example using curl about how to request that endpoint to block all the ip address on the range 69.208.0.0/31:

curl -X POST https://accelerator.shimmercat.com/presentation-api/v1/mark-as-bad-bot/69.208.0.0_31/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json' -d '{"attach_to_bot": "test", "attach_to_domain": "www.shimmercat.tests.com"}'

Please notice that we use _ instead of / when sending the ip range on the URL.

  • Request (application/json)

      {
          "attach_to_bot": "test",
          "attach_to_domain": "www.shimmercat.tests.com"
      }
    
  • Response 200 (application/json)

      {
          "name": "test",
          "ip_addresses": [
              "127.1.1.1",
              "127.0.0.1",
              "69.208.0.0/31"
          ]
      }
    

2.8. RemoveIpFromBadBotListView [/presentation-api/v1/remove-ip-from-bad-bot-list/{ip}/]

  • Parameters

    • ip: 127.1.1.1 (string) - Ip address to remove from the bad bots list.

2.8.1. RemoveIpFromBadBotListView [POST]

Remove the ip address from the bad bots list. You need to specify a domain from which you want to remove this bot from, and the bot name you will remove that ip address from. An example using curl about how to request that endpoint to remove the ip address 127.1.1.1 from the black list:

curl -X POST https://accelerator.shimmercat.com/presentation-api/v1/remove-ip-from-bad-bot-list/127.1.1.1/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json' -d '{"remove_from_bot": "test", "remove_from_domain": "www.shimmercat.tests.com"}'
  • Request (application/json)

      {
          "remove_from_bot": "test",
          "remove_from_domain": "www.shimmercat.tests.com"
      }
    
  • Response 200 (application/json)

      {
          "name": "test",
          "ip_addresses": []
      }
    

2.9. RemoveIpRangeFromBadBotListView [/presentation-api/v1/remove-ip-from-bad-bot-list/{iprange}/]

  • Parameters

    • iprange: 69.208.0.0_31 (string) - Ip address range to remove from the bad bots list.

2.9.1. RemoveIpRangeFromBadBotListView [POST]

Remove the ip address range from the bad bots list. You need to specify a domain from which you want to remove this bot from, and the bot name you will remove that ip address range from. An example using curl about how to request that endpoint to remove the ip address range 69.208.0.0/31:

curl -X POST https://accelerator.shimmercat.com/presentation-api/v1/remove-ip-from-bad-bot-list/69.208.0.0_31/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json' -d '{"remove_from_bot": "test", "remove_from_domain": "www.shimmercat.tests.com"}'

Please notice that we use _ instead of / when sending the ip range on the URL.

  • Request (application/json)

      {
          "remove_from_bot": "test",
          "remove_from_domain": "www.shimmercat.tests.com"
      }
    
  • Response 200 (application/json)

      {
          "name": "test",
          "ip_addresses": []
      }
    

2.10. IsIPAddressVIPBot [/presentation-api/v1/is-vip-bot/{ip}/]

  • Parameters

    • ip: 127.0.1.1 (string) - Ip address to verify.

2.10.1. IsIPAddressVIPBot [GET]

Checks if the ip address was already registered in our database as a VIP bot. An example of a request using curl:

curl -X GET https://accelerator.shimmercat.com/presentation-api/v1/is-vip-bot/127.0.1.1/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json'
  • Response 200 (application/json)

      {
        "is_vip_bot": true,
        "ip_address_or_range": "127.0.1.1"
      }
    

2.11. IsIPAddressRangeVIPBot [/presentation-api/v1/is-vip-bot/{iprange}/]

  • Parameters

    • iprange: 69.209.0.0_31 (string) - Ip address range to verify.

2.11.1. IsIPAddressRangeVIPBot [GET]

Checks if the ip address range was already registered in our database as a VIP bot. An example of a request using curl:

curl -X GET https://accelerator.shimmercat.com/presentation-api/v1/is-vip-bot/69.209.0.0_31/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json'
  • Response 200 (application/json)

      {
        "is_vip_bot": true,
        "ip_address_or_range": "69.209.0.0/31"
      }
    

2.12. ListCustomerVIPBotsView [/presentation-api/v1/list-vip-bots/]

2.12.1. ListCustomerVIPBotsView [GET]

Get the list of VIP bots reported by the customer. An example of a request using curl:

curl -X GET https://accelerator.shimmercat.com/presentation-api/v1/list-vip-bots/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json'
  • Response 200 (application/json)

      [
          {
              "name": "test vip",
              "ip_addresses": [
                  "127.0.1.1"
              ]
          }
      ]
    

2.13. MarkIpAsVIPBotView [/presentation-api/v1/mark-as-vip-bot/{ip}/]

  • Parameters

    • ip: 127.1.2.1 (string) - Ip address to mark as VIP bot.

2.13.1. MarkIpAsVIPBotView [POST]

Mark the ip address as VIP bot. You need to specify a domain which you want to add this bot to, and the bot name you will attach that ip address to. If a bot with that name does not exist for one of your domains we will add a new one with the specified name. An example using curl about how to request that endpoint to white-list the ip address 127.1.2.1:

curl -X POST https://accelerator.shimmercat.com/presentation-api/v1/mark-as-vip-bot/127.1.2.1/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json' -d '{"attach_to_bot": "test", "attach_to_domain": "www.shimmercat.tests.com"}'
  • Request (application/json)

      {
          "attach_to_bot": "test vip",
          "attach_to_domain": "www.shimmercat.tests.com"
      }
    
  • Response 200 (application/json)

      {
          "name": "test vip",
          "ip_addresses": [
              "127.1.2.1"
          ]
      }
    

2.14. MarkIpRangeAsVIPBotView [/presentation-api/v1/mark-as-vip-bot/{iprange}/]

  • Parameters

    • iprange: 69.209.0.0_31 (string) - Ip address range to mark as VIP bot.

2.14.1. MarkIpRangeAsVIPBotView [POST]

Mark the ip address range as VIP bot. You need to specify a domain which you want to add this bot to, and the bot name you will attach that ip address range to. If a bot with that name does not exist for one of your domains we will add a new one with the specified name. An example using curl about how to request that endpoint to white-list all the ip address on the range 69.209.0.0/31:

curl -X POST https://accelerator.shimmercat.com/presentation-api/v1/mark-as-vip-bot/69.209.0.0_31/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json' -d '{"attach_to_bot": "test", "attach_to_domain": "www.shimmercat.tests.com"}'

Please notice that we use _ instead of / when sending the ip range on the URL.

  • Request (application/json)

      {
          "attach_to_bot": "test vip",
          "attach_to_domain": "www.shimmercat.tests.com"
      }
    
  • Response 200 (application/json)

      {
          "name": "test vip",
          "ip_addresses": [
              "127.1.2.1",
              "69.209.0.0/31"
          ]
      }
    

2.15. RemoveIpFromVIPBotListView [/presentation-api/v1/remove-ip-from-vip-bot-list/{ip}/]

  • Parameters

    • ip: 127.1.2.1 (string) - Ip address to remove from the VIP bots list.

2.15.1. RemoveIpFromVIPBotListView [POST]

Remove the ip address from the VIP bots list. You need to specify a domain from which you want to remove this bot from, and the bot name you will remove that ip address from. An example using curl about how to request that endpoint to remove the ip address 127.1.2.1 from the white list:

curl -X POST https://accelerator.shimmercat.com/presentation-api/v1/remove-ip-from-vip-bot-list/127.1.2.1/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json' -d '{"remove_from_bot": "test", "remove_from_domain": "www.shimmercat.tests.com"}'
  • Request (application/json)

      {
          "remove_from_bot": "test vip",
          "remove_from_domain": "www.shimmercat.tests.com"
      }
    
  • Response 200 (application/json)

      {
          "name": "test vip",
          "ip_addresses": []
      }
    

2.16. RemoveIpRangeFromVIPBotListView [/presentation-api/v1/remove-ip-from-vip-bot-list/{iprange}/]

  • Parameters

    • iprange: 69.209.0.0_31 (string) - Ip address range to remove from the VIP bots list.

2.16.1. RemoveIpRangeFromVIPBotListView [POST]

Remove the ip address range from the VIP bots list. You need to specify a domain from which you want to remove this bot from, and the bot name you will remove that ip address range from. An example using curl about how to request that endpoint to remove the ip address range 69.209.0.0/31:

curl -X POST https://accelerator.shimmercat.com/presentation-api/v1/remove-ip-from-vip-bot-list/69.209.0.0_31/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json' -d '{"remove_from_bot": "test", "remove_from_domain": "www.shimmercat.tests.com"}'

Please notice that we use _ instead of / when sending the ip range on the URL.

  • Request (application/json)

      {
          "remove_from_bot": "test vip",
          "remove_from_domain": "www.shimmercat.tests.com"
      }
    
  • Response 200 (application/json)

      {
          "name": "test",
          "ip_addresses": []
      }
    

2.17. MarkCustomIpRangeAsBadBotView [/presentation-api/v1/mark-as-bad-bot/{iprange}/]

  • Parameters

    • iprange: 69.208.0.0_4 (string) - Custom ip address range to mark as bad bot.

2.17.1. MarkCustomIpRangeAsBadBotView [POST]

Mark the custom ip address range as bad bot. It is a special case we implemented where the ip range will be translated a consecutive list of ip addresses e.g 69.208.0.0_4 represents 5 ip addresses: 69.208.0.0, 69.208.0.1, 69.208.0.2, 69.208.0.3, 69.208.0.4, so it is not on a CIDR notation, that’s why we call it “custom ip range”. You need to specify a domain which you want to add this bot to, and the bot name you will attach that ip address range to. If a bot with that name does not exist for one of your domains we will add a new one with the specified name. An example using curl about how to request that endpoint to block all the ip address on the range 69.208.0.0_4:

curl -X POST https://accelerator.shimmercat.com/presentation-api/v1/mark-as-bad-bot/69.208.0.0_4/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json' -d '{"attach_to_bot": "test", "attach_to_domain": "www.shimmercat.tests.com", "is_custom_ip_range": "True"}'
  • Request (application/json)

      {
          "attach_to_bot": "test",
          "attach_to_domain": "www.shimmercat.tests.com",
          "is_custom_ip_range": "True"
      }
    
  • Response 200 (application/json)

      {
          "name": "test",
          "ip_addresses": [
              "69.208.0.0",
              "69.208.0.1",
              "69.208.0.2",
              "69.208.0.3",
              "69.208.0.4"
          ]
      }
    

2.18. RemoveCustomIpRangeFromBadBotListView [/presentation-api/v1/remove-ip-from-bad-bot-list/{iprange}/]

  • Parameters

    • iprange: 69.208.0.0_4 (string) - Custom ip address range to remove from the bad bots list.

2.18.1. RemoveCustomIpRangeFromBadBotListView [POST]

Remove the custom ip address range from the bad bots list. It is a special case we implemented where the ip range will be translated a consecutive list of ip addresses e.g 69.208.0.0_4 represents 5 ip addresses: 69.208.0.0, 69.208.0.1, 69.208.0.2, 69.208.0.3, 69.208.0.4, so it is not on a CIDR notation, that’s why we call it “custom ip range”. You need to specify a domain from which you want to remove this bot from, and the bot name you will remove that ip address range from. An example using curl about how to request that endpoint to remove the ip address range 69.208.0.0_4:

curl -X POST https://accelerator.shimmercat.com/presentation-api/v1/remove-ip-from-bad-bot-list/69.208.0.0_4/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json' -d '{"remove_from_bot": "test", "remove_from_domain": "www.shimmercat.tests.com", "is_custom_ip_range": "True"}'
  • Request (application/json)

      {
          "remove_from_bot": "test",
          "remove_from_domain": "www.shimmercat.tests.com",
          "is_custom_ip_range": "True"
      }
    
  • Response 200 (application/json)

      {
          "name": "test",
          "ip_addresses": []
      }
    

2.19. MarkCustomIpRangeAsVIPBotView [/presentation-api/v1/mark-as-vip-bot/{iprange}/]

  • Parameters

    • iprange: 69.209.0.0_4 (string) - Custom ip address range to mark as VIP bot.

2.19.1. MarkCustomIpRangeAsVIPBotView [POST]

Mark the custom ip address range as VIP bot. It is a special case we implemented where the ip range will be translated a consecutive list of ip addresses e.g 69.208.0.0_4 represents 5 ip addresses: 69.208.0.0, 69.208.0.1, 69.208.0.2, 69.208.0.3, 69.208.0.4, so it is not on a CIDR notation, that’s why we call it “custom ip range”. You need to specify a domain which you want to add this bot to, and the bot name you will attach that ip address range to. If a bot with that name does not exist for one of your domains we will add a new one with the specified name. An example using curl about how to request that endpoint to white-list all the ip address on the range 69.209.0.0_4:

curl -X POST https://accelerator.shimmercat.com/presentation-api/v1/mark-as-vip-bot/69.209.0.0_4/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json' -d '{"attach_to_bot": "test", "attach_to_domain": "www.shimmercat.tests.com", "is_custom_ip_range": "True"}'

We also have a limit of 256 per request for the mask of the ip range.

  • Request (application/json)

      {
          "attach_to_bot": "test vip",
          "attach_to_domain": "www.shimmercat.tests.com",
          "is_custom_ip_range": "True"
      }
    
  • Response 200 (application/json)

      {
          "name": "test vip",
          "ip_addresses": [
              "127.1.2.1",
              "69.209.0.0/24"
          ]
      }
    

2.20. RemoveCustomIpRangeFromVIPBotListView [/presentation-api/v1/remove-ip-from-vip-bot-list/{iprange}/]

  • Parameters

    • iprange: 69.209.0.0_4 (string) - custom ip address range to remove from the VIP bots list.

2.20.1. RemoveCustomIpRangeFromVIPBotListView [POST]

Remove the custom ip address range from the VIP bots list. It is a special case we implemented where the ip range will be translated a consecutive list of ip addresses e.g 69.208.0.0_4 represents 5 ip addresses: 69.208.0.0, 69.208.0.1, 69.208.0.2, 69.208.0.3, 69.208.0.4, so it is not on a CIDR notation, that’s why we call it “custom ip range”. You need to specify a domain from which you want to remove this bot from, and the bot name you will remove that ip address range from. An example using curl about how to request that endpoint to remove the ip address range 69.209.0.0_4:

curl -X POST https://accelerator.shimmercat.com/presentation-api/v1/remove-ip-from-vip-bot-list/69.209.0.0_4/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json' -d '{"remove_from_bot": "test", "remove_from_domain": "www.shimmercat.tests.com", "is_custom_ip_range": "True"}'
  • Request (application/json)

      {
          "remove_from_bot": "test vip",
          "remove_from_domain": "www.shimmercat.tests.com",
          "is_custom_ip_range": "True"
      }
    
  • Response 200 (application/json)

      {
          "name": "test",
          "ip_addresses": []
      }