Labs refers to the Deflect Laboratory that provides early access to experimental features. Please note that these features may be unstable and are subject to change without prior notice.
Bots And AI Access Control #
This feature controls which bots and AI crawlers are allowed to reach your site. Each vendor gets its own row, and each row expands to show the individual bots that belong to it. For every vendor or bot, you choose one of three actions:
- Allow lets the bot through. Use this when you want a specific crawler to access and index your content, bypassing Deflect protection against bots.
- Block stops the bot from reaching your site by returning 403 forbidden. This is the one to use for AI crawlers you don’t want scraping your content.
- No Action leaves Deflect’s existing protections in charge. Nothing extra is applied for that bot, it’s handled the same way it would be without this feature.
Setting a policy at the vendor level applies it to every types of bot under that vendor. If you want different treatment for one specific bot, like blocking OpenAI’s GPTBot crawler (for model training) while leaving ChatGPT-User (execute user actions) alone, set the policy on that bot individually.
This feature sits on top of what Deflect already does, not in place of it. You’re fine-tuning protection for your own site, whether that means keeping AI crawlers out entirely or letting specific ones through.
Changes don’t take effect until you click “Apply” at the bottom of the page.
WebSocket #
Define paths to allow WebSocket connections for your web app. Paths are matched by prefix, wildcards (*) are not supported.
WebSocket facilitates realtime communication between your browser and web server. However, it necessitates a special configuration in a reverse proxy setting such as Deflect. By specifying WebSocket paths, you can configure Deflect to proxy straight to your origin server with specific headers, thereby enabling WebSocket connections.
Security implications of WebSocket on Deflect #
To establish a WebSocket connection, Deflect needs to be configured to directly route traffic along specified paths to your origin server. However, this reduces Deflect’s capacity to safeguard your web server from bots and DDoS attacks on these paths. Despite this, basic protection measures like rate-limiting remain in effect.
Consult Deflect customer service before you enable this feature, if you are concerned about security.
WebSocket Timeout #
Defines how long a WebSocket connection remains open before timing out (in seconds).
Disable SSL Session Reuse #
By default, Deflect reuses the SSL session in the connection pool on the identical origin server. However, if your origin server hosts multiple virtual hosts with different domain names, disabling SSL session reuse may solve the SNI mismatch error.
For more details, please refer to troubleshooting SNI issues.
