爬网 Web Python 时出错

Question

当我尝试运行下面的代码时，返回了此错误。如果有人能帮助指出我做错的地方，我将不胜感激。谢谢。

Traceback (most recent call last):
  File "web_crawler.py", line 26, in <module>
    links = get_all_links(page)
  File "web_crawler.py", line 14, in get_all_links
    url, endpos = get_next_target(page)
  File "web_crawler.py", line 2, in get_next_target
    start_link = page.find("<a href=")
TypeError: a bytes-like object is required, not 'str'
def get_next_target(page):
    start_link = page.find("<a href=")
    if start_link == -1:
        return None, 0
    start_quote = page.find('"',start_link)
    end_quote = page.find('"',start_quote+1)
    url = page[start_quote+1:end_quote]
    print(url)
    return url, end_quote
def get_all_links(page):
    links = []
    while True:
        url, endpos = get_next_target(page)
        if url:
            links.append(url)
            page = page[endpos:]
        else:
            break
    return links
import requests
url='https://en.wikipedia.org/wiki/Moon'
r = requests.get(url)
page = r.content
links = get_all_links(page)

Answer 1

response.content是请求的原始内容。他们没有解码它或任何东西，它只是原始字节。

您要使用的是 response.text 属性，该属性以字符串形式包含解码的内容。

(您可能还想使用像BeautifulSoup这样的html解析库，而不是当前的page.find方法(