Python的请求在访问 etherscan.io 时触发了Cloudflare的安全性

Question

我正在尝试解析/抓取https://etherscan.io/tokens网站使用Python中的请求，但我得到以下错误：

etherscan.io

检查站点连接是否安全

etherscan.io需要在诉讼程序光线ID:73b56fc71bc276ed性能&担保人Cloudflare

现在，我在这里找到了一个解决方案：https://stackoverflow.com/a/62687390/4190159但是当我尝试使用这个解决方案时，我仍然无法阅读网站的实际内容，并得到下面所述的不同错误。

我的代码如下：

import requests
from collections import OrderedDict
from requests import Session
import socket
answers = socket.getaddrinfo('etherscan.io', 443)
(family, type, proto, canonname, (address, port)) = answers[0]
s = Session()
headers = OrderedDict({
'Accept-Encoding': 'gzip, deflate, br',
'Host': "grimaldis.myguestaccount.com",
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:77.0) Gecko/20100101 Firefox/77.0'
})
s.headers = headers
response = s.get(f"https://{address}/tokens", headers=headers, verify=False).text
print(response)

上述代码的错误如下：

Traceback(上次调用)：文件"Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/urlib3/connectionpool.py"；，第677行，在urlopen中chunked=chunked/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/urlib3/connectionpool.py"；，第381行，在_make_request中自我_validate_conn(conn)文件"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/urlib3/connectionpool.py"；，第976行，在validate_conn中conn.connect()文件"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/urlib3/connection.py"；，线路370，连接ssl_context=上下文，文件"Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/urllib3/util/ssl.py"；，第390行，在sslwrap_socket中return context.wrap_socket(sock)File"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py"；，线路407，在wrap_socket中_context＝self，_session＝session)文件"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py"；，第814行，在initself.do_handshake()File"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py"；，第1068行，在do_handshake中自我_sslobj.do_handshake()文件"Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py"；，第689行，在do_handshake自我_sslobj.do_handshake()ssl。SSLError:[SSL:SSLV3_ALERT_HANDSHAKE_FAILURE]SSLV3警报握手失败(_ssl.c:833)

在处理上述异常的过程中，发生了另一个异常：

Traceback(最近一次通话)：文件"Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/requests/adapters.py"；，第449行，发送中timeout＝timeout文件"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/urlib3/connectionpool.py"；，第725行，在urlopen中方法，url，error=e，_pool=self，_stacktrace=sys.exc_info()[2]文件"Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/urllib3/util/retry.py"；，第439行，增量引发MaxRetryError(_pool、url、error或ResponseError(cause))urlib3.exceptions.MaxRetryError:HTTPSConnectionPool(主机="72.67.8.107"，端口=443)：最大重试次数超过url:/tokens(由SSLError(SSLError(1，'[SSL:SSLV3_ALERT_HANDSHAKE_FAILURE]SSLV3警报握手失败(_ssl.c:833)')，))

在处理上述异常的过程中，发生了另一个异常：

Traceback(最后一次调用)：File"；label_ scraper.py"；，线16，英寸response=s.get(f"https://｛address｝/tokens"，headers=headers，verify=False).text文件"Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/requests/sessions.py"；，543行，在getreturn self.request('GET'，url，**kwargs)文件"Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/requests/sessions.py"；，第530行，请求中resp＝self.send(prep，**send_kwargs)文件"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/requests/sessions.py"；，第643行，发送中r＝adapter.send(请求，**kwargs)文件"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/requests/adapters.py"；，第514行，发送引发SSLError(e，request=request)requests.exceptions.SLError：HTTPSConnectionPool(主机="72.67.8.107"，端口=443)：最大重试次数超过url:/tokens(由SSLError(SSLError(1，'[SSL:SSLV3_ALERT_HANDSHAKE_FAILURE]SSLV3警报握手失败(_ssl.c:833)')，))Somdips MacBook Pro：下载somdipdey$python3label_cracpper.py Traceback(最近一次调用最后一次)：文件"Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/urlib3/connectionpool.py"；，第677行，在urlopen中chunked=chunked/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/urlib3/connectionpool.py"；，第381行，在_make_request中自我_validate_conn(conn)文件"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/urlib3/connectionpool.py"；，第976行，在validate_conn中conn.connect()文件"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/urlib3/connection.py"；，线路370，连接ssl_context=上下文，文件"Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/urllib3/util/ssl.py"；，第390行，在sslwrap_socket中return context.wrap_socket(sock)File"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py"；，线路407，在wrap_socket中_context＝self，_session＝session)文件"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py"；，第814行，在initself.do_handshake()File"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py"；，第1068行，在do_handshake中自我_sslobj.do_handshake()文件"Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py"；，第689行，在do_handshake自我_sslobj.do_handshake()ssl。SSLError:[SSL:SSLV3_ALERT_HANDSHAKE_FAILURE]SSLV3警报握手失败(_ssl.c:833)

在处理上述异常的过程中，发生了另一个异常：

Traceback(最近一次通话)：文件"Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/requests/adapters.py"；，第449行，发送中timeout＝timeout文件"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/urlib3/connectionpool.py"；，第725行，在urlopen中方法，url，error=e，_pool=self，_stacktrace=sys.exc_info()[2]文件"Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/urllib3/util/retry.py"；，第439行，增量引发MaxRetryError(_pool、url、error或ResponseError(cause))urlib3.exceptions.MaxRetryError:HTTPSConnectionPool(主机="72.67.8.107"，端口=443)：最大重试次数超过url:/tokens(由SSLError(SSLError(1，'[SSL:SSLV3_ALERT_HANDSHAKE_FAILURE]SSLV3警报握手失败(_ssl.c:833)')，))

在处理上述异常的过程中，发生了另一个异常：

Traceback(最后一次调用)：File"；label_ scraper.py"；，线16，英寸response=s.get(f"https://｛address｝/tokens"，headers=headers，verify=False).text文件"Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/requests/sessions.py"；，543行，在getreturn self.request('GET'，url，**kwargs)Somdips MacBook Pro：下载somdipdey$python3 label_charpper.pyTraceback(最近一次通话)：文件"Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/urlib3/connectionpool.py"；，第677行，在urlopen中chunked=chunked/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/urlib3/connectionpool.py"；，第381行，在_make_request中自我_validate_conn(conn)文件"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/urlib3/connectionpool.py"；，第976行，在validate_conn中conn.connect()文件"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/urlib3/connection.py"；，线路370，连接ssl_context=上下文，文件"Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/urllib3/util/ssl.py"；，第390行，在sslwrap_socket中return context.wrap_socket(sock)File"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py"；，线路407，在wrap_socket中_context＝self，_session＝session)文件"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py"；，第814行，在initself.do_handshake()File"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py"；，第1068行，在do_handshake中自我_sslobj.do_handshake()文件"Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py"；，第689行，在do_handshake自我_sslobj.do_handshake()ssl。SSLError:[SSL:SSLV3_ALERT_HANDSHAKE_FAILURE]SSLV3警报握手失败(_ssl.c:833)

在处理上述异常的过程中，发生了另一个异常：

Traceback(最近一次通话)：文件"Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/requests/adapters.py"；，第449行，发送中timeout＝timeout文件"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/urlib3/connectionpool.py"；，第725行，在urlopen中方法，url，error=e，_pool=self，_stacktrace=sys.exc_info()[2]文件"Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/urllib3/util/retry.py"；，第439行，增量引发MaxRetryError(_pool、url、error或ResponseError(cause))urlib3.exceptions.MaxRetryError:HTTPSConnectionPool(主机="72.67.8.107"，端口=443)：最大重试次数超过url:/tokens(由SSLError(SSLError(1，'[SSL:SSLV3_ALERT_HANDSHAKE_FAILURE]SSLV3警报握手失败(_ssl.c:833)')，))

在处理上述异常的过程中，发生了另一个异常：

Traceback(最后一次调用)：File"；label_ scraper.py"；，线16，英寸response=s.get(f"https://｛address｝/tokens"，headers=headers，verify=False).text文件"Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/requests/sessions.py"；，543行，在getreturn self.request('GET'，url，**kwargs)文件"Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/requests/sessions.py"；，第530行，请求中resp＝self.send(prep，**send_kwargs)文件"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/requests/sessions.py"；，第643行，发送中r＝adapter.send(请求，**kwargs)文件"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site packages/requests/adapters.py"；，第514行，发送引发SSLError(e，request=request)requests.exceptions.SLError：HTTPSConnectionPool(主机="72.67.8.107"，端口=443)：最大重试次数超过url:/tokens(由SSLError(SSLError(1，'[SSL:SSLV3_ALERT_HANDSHAKE_FAILURE]SSLV3警报握手失败(_ssl.c:833)')，))

如何解决此问题？

Answer 1

该网站处于cloudflare保护之下。所以您可以使用cloudscraper而不是请求来摆脱保护。现在运行良好。

示例：

from bs4 import BeautifulSoup
import cloudscraper
scraper = cloudscraper.create_scraper(delay=10,   browser={'custom': 'ScraperBot/1.0',})
url = 'https://etherscan.io/tokens'
req = scraper.get(url)
soup = BeautifulSoup(req.content,'lxml')
for tr in soup.select('table#tblResult tbody tr'):
d= list(tr.stripped_strings)
print(d)

输出：

['1', 'Tether USD (USDT)', 'Tether gives you the joint benefits of open blockchain technology and traditional currency by converting your cash into a stable digital currency equivalent.', '$1.005', '0.000042xa0Btc', '0.000534xa0Eth', '0.40%', '$56,223,000,912.00', '$67,594,315,221.00', '$40,022,235,444.23', '4,384,118', '0.003%']
['2', 'USD Coin (USDC)', 'USDC is a fully collateralized US Dollar stablecoin developed by CENTRE, the open source project with Circle being the first of several forthcoming issuers.', '$1.006', '0.000042xa0Btc', '0.000534xa0Eth', '0.44%', '$7,028,538,289.00', '$53,707,876,130.00', '$46,882,045,425.04', '1,470,173', '0.132%']
['3', 'BNB (BNB)', 'Binance aims to build a world-class crypto exchange, powering the futurenof crypto finance.', '$316.6558', '0.013290xa0Btc', '0.168135xa0Eth', '-0.87%', '$1,131,380,099.00', '$51,088,380,246.00', '$5,250,000,297.97', '322,395', '0.001%']
['4', 'Binance USD (BUSD)', 'Binance USD (BUSD) is a dollar-backed stablecoin issued and custodied by Paxos Trust Company, and regulated by the New York State Department of Financial Services. BUSD is available directly for sale 1:1 with USD on Paxos.com and will be listed for trading on Binance.', '$0.9976', '0.000042xa0Btc', '0.000530xa0Eth', '-0.30%', '$6,219,951,485.00', '$17,920,238,921.00', '$17,532,299,450.15', '125,632', '0.197%']
['5', 'HEX (HEX)', "HEX.com averages 25% APY interest recently. HEX virtually lends value from stakers to non-stakers as staking reduces supply. The launch ends Nov. 19th, 2020 when HEX 
stakers get credited ~200B HEX. HEX's total supply is now ~350B. Audited 3 times, 2 security, and 1 economics.", '$0.0626', '0.000003xa0Btc', '0.000033xa0Eth', '-5.10%', '$22,598,794.00', '$10,856,229,132.00', '$36,158,058,204.94', '308,035', '-0.040%']
['6', 'SHIBA INU (SHIB)', 'SHIBA INU is a 100% decentralized community experiment with it claims that 1/2 the tokens have been sent to Vitalik and the other half were locked to a Uniswap pool and the keys burned.', '$0.00', '0.000000xa0Btc', '0.000000xa0Eth', '-9.20%', '$2,070,477,368.00', '$9,155,756,506.00', '$15,489,873,503.95', '1,206,115', '0.088%']
['7', 'stETH (stETH)', 'stETH is a token that represents staked ether in Lido, combining the 
value of initial deposit + staking rewards. stETH tokens are pegged 1:1 to the ETH staked with Lido and can be used as one would use ether, allowing users to earn Eth2 staking rewards whilst benefiting from Defi yields.', '$1,844.28', '0.077404xa0Btc', '0.979260xa0Eth', '-2.26%', '$3,408,944.00', '$7,909,446,933.00', '$3,418,574,006.52', '94,316', '0.215%']
['8', 'Matic Token (MATIC)', 'Matic Network brings massive scale to Ethereum using an adapted version of Plasma with PoS based side chains. Polygon is a well-structured, easy-to-use platform for Ethereum scaling and infrastructure development.', '$0.9397', '0.000039xa0Btc', '0.000499xa0Eth', '-6.39%', '$529,032,596.00', '$7,551,024,649.00', '$9,397,310,544.00', '466,641', '0.083%']
['9', 'Dai Stablecoin (DAI)', 'Multi-Collateral Dai, brings a lot of new and exciting features, such as support for new CDP collateral types and Dai Savings Rate.', '$1.005', '0.000042xa0Btc', '0.000534xa0Eth', '0.36%', '$635,956,564.00', '$6,800,555,162.00', '$9,848,650,590.65', '479,078', '-0.008%']
['10', 'Wrapped BTC (WBTC)', 'Wrapped Bitcoin (WBTC) is an ERC20 token backed 1:1 with Bitcoin.nCompletely transparent. 100% verifiable. Community led.', '$23,983.00', '1.006567xa0Btc', '12.734291xa0Eth', '-1.26%', '$263,338,154.00', '$5,928,934,190.00', '$6,279,085,162.00', 
'51,459', '0.058%']

也是如此

云刮刀