FAQ

Reading time: About 6 minutes

1. What types of crawlers does Bot Management protect against?
Bot Management provides baseline protection through general-purpose policies such as AI Bots, Public Bots, Definite Bots, and Likely Bots, enhanced protection through scenario-based policies such as Web Bot Detection and Workflow Detection, and fine-grained protection through the Custom Bots policy.

It’s especially effective against the following types of crawlers and abnormal traffic:

• AI large-model crawlers: Automated collection behavior that scrapes site content in bulk for uses such as model training or retrieval augmentation.
• Common data-scraping crawlers: Script- or framework-based crawlers that perform high-frequency scraping, systematic traversal, or bulk downloading of pages/APIs.
• Disguised and adversarial crawlers: Advanced crawlers that evade detection by forging a User-Agent or request headers, simulating browser behavior, or using proxy pools and IP rotation.

2. What should I do if I find a false block?
A false block occurs when a normal user is mistakenly identified as a malicious Bot by a security rule and gets blocked. We recommend the following steps:

• Prioritize business availability first: Use policies such as Custom Bots, Custom Rules, or Whitelist to bring known legitimate traffic into your trusted scope — for example, fixed IPs or compliant User-Agent/request characteristics.
• Identify the root cause: Use logs and match records to confirm which policy was triggered and under what match conditions.
• Report and improve: Send the false-block sample along with the time window, URL, and client characteristics to our support engineers. We will optimize the relevant policy and model as quickly as possible to reduce the chance of future false blocks.

3. What should I do if I find a missed detection?
A missed detection occurs when a malicious Bot is not effectively identified and is allowed through to the origin. We recommend the following steps:

• Apply a temporary mitigation first (to reduce risk): If the attack has already affected your business, you can use existing capabilities right away — for example, enabling a Custom Rule to deny or rate-limit the traffic, or increasing the policy strength or applying a stricter action on the affected endpoints.
• Report and help us harden the policy: Send the sample and your analysis to our support engineers. We will incorporate stable attack patterns into our automated policies as quickly as possible.

4. Why is compatibility testing recommended before enabling Web Bot Detection?
Web Bot Detection primarily relies on embedding JavaScript code into Web/H5 pages to verify consistency between the browser environment and behavior. We recommend compatibility testing before enabling it, mainly because:

• Non-HTML scenarios cannot execute JS: For example, pure API endpoints, certain download/redirect scenarios, or specific rendering environments may be unable to complete verification, which can trigger an unintended action.
• Browser/engine differences affect coverage: Differences in JavaScript capability across browser versions, outdated browsers, or custom WebViews can lead to false positives or a degraded user experience.

Note: Compatibility testing helps reduce the risk of false blocks caused by differences in client environments.

5. Why isn’t an app SDK supported, and how can I effectively protect my app?

• Why isn’t it supported?

1. Integration complexity: It would require your team to embed an SDK and complete client-side integration testing, which increases development cost and can introduce compatibility issues.
2. App release delays: Mobile app releases require app store approval (especially on iOS), so urgent security updates cannot be rolled out quickly.
3. Privacy and compliance risk: An SDK may involve permission requests and data-collection scope declarations, increasing compliance review overhead.

• How can I protect my app effectively?

1. Definite Bots and Likely Bots policies: These use a combination of heuristic rules and machine learning models to identify and act on automation tools and abnormal black-and-grey market activity.
2. Workflow Detection policy: Detects and acts on abnormal mobile access behavior based on business workflow logic and request frequency.
3. Custom scenario-based policies: When general-purpose policies don’t provide sufficient coverage, you can design a targeted protection scheme based on your specific business scenario and attack pattern.

If you need further assistance, please contact our support team.