当我尝试执行代码时
BeautifulSoup(html, ...)
它给出错误信息
TypeError:"Response"类型的对象没有len()
我尝试将实际的HTML作为参数传递,但它仍然不起作用。
import requests
url = 'http://vineoftheday.com/?order_by=rating'
response = requests.get(url)
html = response.content
soup = BeautifulSoup(html, "html.parser")
您得到的是response.content
。但它以字节(docs)的形式返回响应体。但是您应该将str
传递给BeautifulSoup构造函数(docs)。因此,您需要使用response.text
,而不是获取内容。
尝试直接传递HTML文本
soup = BeautifulSoup(html.text)
html.parser
用于忽略页面中的警告:
soup = BeautifulSoup(html.text, "html.parser")
如果使用requests.get('https://example.com')
获取HTML,则应该使用requests.get('https://example.com').text
。
您在"response"中只得到响应代码并始终使用浏览器标头以确保安全,否则你将面临许多问题
在调试器控制台网络部分"header"UserAgent 中查找标头
尝试
import requests
from bs4 import BeautifulSoup
from fake_useragent import UserAgent
url = 'http://www.google.com'
headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36'}
response = requests.get(quote_page, headers=headers).text
soup = BeautifulSoup(response, 'html.parser')
print(soup.prettify())
from bs4 import BeautifulSoup
import requests
url = 'your_url'
response = requests.get(url)
html = response.content
soup = BeautifulSoup(html, "html.parser")
print(soup)
它对我有效:
soup = BeautifulSoup(requests.get("your_url").text)
现在,下面的代码更好了(使用lxml解析器):
import requests
from bs4 import BeautifulSoup
soup = BeautifulSoup(requests.get("your_url").text, 'lxml')
您应该使用.text
来获取响应的内容
import requests
url = 'http://www ... '
response = requests.get(url)
print(response.text)
或与肥皂一起使用
import requests
from bs4 import BeautifulSoup
url = 'http://www ... '
response = requests.get(url)
msg = response.text
print(BeautifulSoup(msg,'html.parser'))
import requests
from urllib.request import urlopen
from bs4 import BeautifulSoup
import re
url = "https://fortnitetracker.com/profile/all/DakshRungta123"
html = requests.get(url)
soup = BeautifulSoup(html)
title = soup.text
print(title.text)
import requests
url = 'http://vineoftheday.com/?order_by=rating'
response = requests.get(url)
html = response.content
soup = BeautifulSoup(html.text, "html.parser")