Web Bot Auth Policy
Authenticate bots using HTTP Message Signatures via the official web-bot-auth npm package. This policy allows you to specify friendly bots and block others based on configuration.
With this policy, you'll benefit from:
- Enhanced API Security: Protect your endpoints from unauthorized bot traffic while allowing legitimate bots
- Cryptographic Verification: Leverage HTTP Message Signatures to ensure bots are who they claim to be
- Flexible Bot Management: Easily configure which bots are allowed to access your API
- Detailed Request Context: Access bot identity information in subsequent policies or handlers
- Seamless Integration: Works with standard HTTP Message Signatures used by major bot providers
Configuration
The configuration shows how to configure the policy in the 'policies.json' document.
json
Policy Configuration
name
<string>
- The name of your policy instance. This is used as a reference in your routes.policyType
<string>
- The identifier of the policy. This is used by the Zuplo UI. Value should beweb-bot-auth-inbound
.handler.export
<string>
- The name of the exported type. Value should beWebBotAuthInboundPolicy
.handler.module
<string>
- The module containing the policy. Value should be$import(@zuplo/runtime)
.handler.options
<object>
- The options for this policy. See Policy Options below.
Policy Options
The options for this policy are specified below. All properties are optional unless specifically marked as required.
allowedBots
(required)<string[]>
- List of bot identifiers that are allowed to access the API.blockUnknownBots
(required)<boolean>
- Whether to block bots that aren't in the allowed list. Defaults totrue
.allowUnauthenticatedRequests
<boolean>
- Allow requests without bot signatures to proceed. This is useful if you want to use multiple authentication policies or if you want to allow both authenticated and non-authenticated traffic. Defaults tofalse
.directoryUrl
<string>
- Optional URL to a directory of known bots (for verification). Defaults tonull
.
Using the Policy
How It Works
The policy checks for HTTP Message Signatures in the request headers
(Signature
and Signature-Input
). These signatures are verified using the
web-bot-auth
npm package.
When a bot makes a request to your API, the policy:
- Checks if the request has signature headers
- Verifies the signature using the
web-bot-auth
library - Extracts the bot identity from the verified signature
- Checks if the bot is in the allowed list
- Either allows or blocks the request based on configuration
Configuration Options
Option | Type | Required | Default | Description |
---|---|---|---|---|
allowedBots | string[] | Yes | - | List of bot identifiers that are allowed to access the API |
blockUnknownBots | boolean | Yes | true | Whether to block bots that aren't in the allowed list |
allowUnauthenticatedRequests | boolean | No | false | Allow requests without bot signatures to proceed |
directoryUrl | string | No | - | Optional URL to a directory of known bots (for verification) |
Example Configuration
json
Bot Directory
If you specify a directoryUrl
, the policy will fetch the directory of known
bots from that URL. The directory should be a JSON object mapping bot
identifiers to their public keys in JWK format.
Example directory:
json
Request Context
When a bot is successfully authenticated, the policy adds the bot identity to
the request context. You can access this in subsequent policies or handlers
using the getBotId
helper function:
typescript
Error Handling
If a bot fails authentication, the policy returns a 401 Unauthorized response
with an error message. If a request doesn't have signature headers and
allowUnauthenticatedRequests
is false, the policy also returns a 401 response.
Cryptographic Verification
When a directory URL is provided, the policy performs cryptographic verification of the bot signatures:
- It fetches the bot's public key from the directory
- Imports the key using the Web Crypto API
- Verifies the signature against the request data
This provides strong cryptographic assurance that the bot is who it claims to be.
Implementation Details
The policy uses the web-bot-auth
npm package to implement HTTP Message
Signatures verification. The implementation:
- Uses the
verify
function from theweb-bot-auth
package to handle signature verification - Validates that the bot is in the allowed list
- Optionally verifies the signature against a directory of known bots
- Adds the verified bot identity to the request context
This implementation leverages the standard web-bot-auth library for bot authentication, ensuring compatibility and security across different bot providers.
Read more about how policies work