分享一篇python scrapy ip代理的设置实例

# encoding: utf-8import base64proxyServer = 代理服务器地址 ##我的是‘:9010’# 代理隧道验证信息  这个是在那个网站上申请的proxyUser = 用户名proxyPass = 密码proxyAuth = "Basic " + base64.b64encode(proxyUser + ":" + proxyPass)class ProxyMiddleware(object):    def process_request(self, request, spider):        request.meta["proxy"] = proxyServer        request.headers["Proxy-Authorization"] = proxyAuth在setting文件中添加

DOWNLOADER_MIDDLEWARES = {    '项目名.spider同级文件名.文件名.ProxyMiddleware': 543,}代码地址

<br/>