Define Specific Bots and Allow or Block them

If your website receives traffic from certain authorized automation tools or IoT devices, you can use the custom Bots module to allow these requests or block specific malicious behaviors.

Note: The custom Bots rules are applied after IP/Region Blocking, Custom Rules, Whitelist, DDoS Protection, and Rate Limiting. You can think of these rules as access control or custom rules under the Bot Management module. Typically, custom Bots are used to prevent certain automation tools (such as partners’ self-service ordering tools) from being identified as Bots and blocked. However, if these tools are abused to launch DDoS attacks, DDoS protection policies will still detect and handle them.

Steps

1. In the left navigation pane of the console, click Shared Configuration to access the shared configuration page.
2. Under the Custom Bots tab, click Add Rule.
3. Define matching criteria based on IP/IP range and User-Agent characteristics, and preset the actions for custom Bots.
4. Click the icon and select the Domain to which you want to apply this rule.
5. Click Confirm to submit the policy deployment task.

Specific Fine-grained Management

1. If you want to allow certain specific public Bots, and to prevent bypass through forged User-Agents, you can use a combination of matching conditions as shown below:

2. If you want to block certain specific AI Bots/public Bots/absolute Bots, you can directly use the User-Agent condition for matching.