How to Whitelist the JTracking Crawler in Your Web Application Firewall (WAF)

Joanna Zhou

Joanna Zhou

— 2 min read

To ensure that JTracking can effectively crawl your website for designing tracking solutions and GTM configurations, it's sometimes necessary to whitelist its crawler in your Web Application Firewall (WAF). If the JTracking crawler is being blocked, please use the following instructions to allow its requests. The primary method involves creating a rule to allow requests where the User Agent string contains "jtracking".

1. Cloudflare WAF Configuration

To create a custom rule in Cloudflare to allow JTracking requests:

  1. Log in to the Cloudflare dashboard and select your account and domain.
  2. Navigate to Security > WAF > Custom rules.
  3. Select Create rule.
  4. In the Rule name field, enter a descriptive name (e.g., "Allow JTracking Crawler").
  5. Under the section If incoming requests match...:
    • For Field, select User Agent from the drop-down list.
    • For Operator, select contains.
    • For Value, enter jtracking.
  6. Under the section Then take action...:
    • From the Choose action drop-down list, select Allow. (Alternatively, some guides suggest "Skip" and then selecting specific security features to bypass).
  7. Select Deploy to activate the rule.

For more detailed information, please refer to the official Cloudflare documentation on creating custom rules: https://developers.cloudflare.com/waf/custom-rules/create-dashboard/

2. AWS WAF (for CloudFront) Configuration

If JTracking requests are being blocked by AWS WAF associated with your CloudFront distribution (often resulting in a "403 Error - The request could not be satisfied. Request Blocked."), you'll need to adjust your Web ACL rules:

  1. Open the AWS WAF console.
  2. In the navigation pane, under AWS WAF, choose Web ACLs.
  3. For Region, select Global (CloudFront) if your web ACL is for a CloudFront distribution.
  4. Select the Web ACL associated with your distribution.
  5. Review your rules and default actions:
    • If your Web ACL's default action is Allow: You need to identify if an existing rule with Action set to Block is inadvertently blocking requests containing "jtracking" in the User Agent. If so, modify that rule or create a new, higher-priority rule to specifically Allow requests where the User Agent contains "jtracking".
    • If your Web ACL's default action is Block: You must ensure that there is a rule with Action set to Allow that matches requests with "jtracking" in the User Agent. If no such rule exists, create a new rule:
      • Choose Add my own rules and rule groups.
      • For Rule Type, choose Rule Builder.
      • Provide a Name for the rule.
      • For If a request, select matches the statement.
      • For Inspect, choose Single header.
      • For Header field name, enter User-Agent.
      • For Match type, choose Contains string.
      • For String to match, enter jtracking.
      • For Action, choose Allow.
      • Add the rule and save your Web ACL.
  6. Test your configuration to ensure JTracking requests are no longer blocked.

For further troubleshooting and details, refer to the AWS WAF documentation on resolving "Request Blocked" errors and testing WAF protections. https://repost.aws/knowledge-center/cloudfront-error-request-blocked

3. Other WAF Services

For other WAF providers, the general approach is similar:

  • Access your WAF's management interface and find the section for rules or policy configuration.
  • Create a new rule or modify an existing one.
  • Define a condition that identifies JTracking requests. This is typically achieved by inspecting the User Agent header for a value that contains the string jtracking.
  • Set the action for this rule to Allow (or an equivalent term like Whitelist or Permit).

If you need assistance specific to your WAF solution, please consult your provider's official documentation or support resources, explaining the requirement to allow requests based on the User Agent containing "jtracking".

Read more