当前位置：首页 > news >正文

AWS WAF 新功能：给 AI 爬虫收费，在边缘层直接变现你的内容

news 2026/6/17 12:00:45

如果你运营一个内容网站，大概已经感受到了——AI 爬虫的流量比搜索引擎爬虫还猛。

AWS 的数据是：很多内容网站 50% 以上的流量来自 AI 爬虫，同比增长 300%+。这些爬虫把你的内容抓走训练模型、生成摘要，但不会给你带来回访流量。你承担了带宽和服务器成本，却得不到页面浏览量、广告曝光或订阅转化。

以前的选择很有限：要么 robots.txt 一刀切禁止，要么放任不管。

现在 AWS WAF 给了第三个选项：给它们收费。

这个功能干什么的

AWS WAF Bot Control 新增了 "AI traffic monetization" 能力。简单说：

你设定每个请求的价格（可以按路径、爬虫类型、验证等级分别定价）
AI Agent 访问你的内容时，在边缘层完成支付
支付通过稳定币到你指定的钱包
没付费的 AI 爬虫被拦截或降级

整个过程在 WAF 边缘完成，不需要改你的源站代码。

架构长什么样

AI Agent 请求↓
CloudFront / ALB↓
AWS WAF（Bot Control + AI Monetization）├── 验证 Agent 身份（verified/unverified）├── 检查支付状态├── 未付费 → 返回 402 Payment Required└── 已付费 → 放行到源站↓
你的内容服务器

配置示例

基本策略：按路径定价

{"Name": "AITrafficMonetization","Priority": 1,"Statement": {"ManagedRuleGroupStatement": {"VendorName": "AWS","Name": "AWSManagedRulesBotControlRuleSet","ManagedRuleGroupConfigs": [{"AWSManagedRulesAIMonetizationConfig": {"InspectionLevel": "TARGETED","EnableMonetization": true}}]}},"Action": {"Block": {}},"VisibilityConfig": {"SampledRequestsEnabled": true,"CloudWatchMetricsEnabled": true,"MetricName": "AIMonetization"}
}

分级定价规则

{"MonetizationPolicy": {"DefaultPricePerRequest": "0.001","Currency": "USDC","PaymentWallet": "0x...","Rules": [{"PathPattern": "/api/articles/*","PricePerRequest": "0.005","Description": "Full article content - premium pricing"},{"PathPattern": "/api/summaries/*","PricePerRequest": "0.001","Description": "Summary content - standard pricing"},{"PathPattern": "/public/*","PricePerRequest": "0","Description": "Public content - free tier"}],"BotCategoryOverrides": [{"Category": "verified_ai_search","Discount": 0.5,"Description": "Verified search engines get 50% discount"},{"Category": "unverified_scraper","Action": "BLOCK","Description": "Unverified scrapers blocked entirely"}]}
}

CDK 部署

from aws_cdk import (aws_wafv2 as wafv2,Stack
)
from constructs import Constructclass AIMonetizationStack(Stack):def __init__(self, scope: Construct, id: str, **kwargs):super().__init__(scope, id, **kwargs)web_acl = wafv2.CfnWebACL(self, "AIMonetizationACL",default_action=wafv2.CfnWebACL.DefaultActionProperty(allow=wafv2.CfnWebACL.AllowActionProperty()),scope="CLOUDFRONT",visibility_config=wafv2.CfnWebACL.VisibilityConfigProperty(cloud_watch_metrics_enabled=True,metric_name="AITrafficMonetization",sampled_requests_enabled=True),rules=[wafv2.CfnWebACL.RuleProperty(name="BotControlWithMonetization",priority=1,override_action=wafv2.CfnWebACL.OverrideActionProperty(none={}),statement=wafv2.CfnWebACL.StatementProperty(managed_rule_group_statement=wafv2.CfnWebACL.ManagedRuleGroupStatementProperty(vendor_name="AWS",name="AWSManagedRulesBotControlRuleSet")),visibility_config=wafv2.CfnWebACL.VisibilityConfigProperty(cloud_watch_metrics_enabled=True,metric_name="BotControl",sampled_requests_enabled=True))])