BeautifulSoup 返回 typeError： 'NoneType' 的对象没有 len()

Question

我正在使用BeautifulSoup来抓取数据，并向我返回列表中所有div的列表，但它给了我这个错误：

Traceback (most recent call last):
File "C:UsersintelDesktopOne pagetest.py", line 16, in <module>
soup = BeautifulSoup(div.html,'html5lib')
File "C:UsersintelAppDataLocalProgramsPythonPython38libsite-packagesbs4__init__.py", line 287, in __init__
elif len(markup) <= 256 and (
TypeError: object of type 'NoneType' has no len()

这是我的代码：

from bs4 import BeautifulSoup
import requests as req
resp = req.get('https://medium.com/@daranept27')
x = resp.text
soup = BeautifulSoup(x, "lxml")

divs = soup.find_all("div")
#print(divs)
lst = []
for div in divs:
soup = BeautifulSoup(div.html,'html5lib')
div_tag = soup.find()
try:
title = div_tag.section.div.h1.a['href']
if title not in lst: lst.append(title)
except:
pass
print("n".join(lst))

Answer 1

尝试使用str(div)将div转换为str。这是完整的代码：

from bs4 import BeautifulSoup
import requests as req
resp = req.get('https://medium.com/@daranept27')
x = resp.text
soup = BeautifulSoup(x, "lxml")
divs = soup.find_all("div")
# print(divs)
lst = []
for div in divs:
soup = BeautifulSoup(str(div), 'html5lib')
div_tag = soup.find()
try:
title = div_tag.section.div.h1.a['href']
if title not in lst: lst.append(title)
except:
pass
print("n".join(lst))

输出：

/read-rosy/if-the-whole-world-is-compelled-to-forget-everything-cde200c0ad98
/wordsmith-library/seven-days-between-life-and-death-dffb639fb245
/an-idea/have-you-ever-encountered-a-fake-friend-if-so-try-these-simple-tips-to-overcome-it-d8473d755ab8