我一直在尝试从新闻网站上抓取新闻标题。为此，我遇到了两个python库，即报纸和beautifulsoup4。使用这个漂亮的汤库，我可以从一个特定的新闻网站上获得所有指向新闻文章的链接。从下面的代码中，我可以从一个链接中提取新闻文章的标题。
from newspaper import Article
url= "https://www.ndtv.com/india-news/tamil-nadu-government-reverses-decision-to-reopen-schools-from-november-16-for-classes-9-12-news-agency-pti-2324199"
article=Article(url)
article.download()
article.parse()
print(article.title)
我想将来自两个库(即报纸和beautifulsoup4(的代码组合起来，这样我从beautifursoup库输出的所有链接都应该放在报纸库的url命令中，我就可以获得链接的所有标题。以下是beautfusoup的代码，我可以从中提取到新闻文章的所有链接。
from bs4 import BeautifulSoup
from bs4.dammit import EncodingDetector
import requests
parser = 'html.parser' # or 'lxml' (preferred) or 'html5lib', if installed
resp = requests.get("https://www.ndtv.com/coronavirus?pfrom=home-mainnavgation")
http_encoding = resp.encoding if 'charset' in resp.headers.get('content-type', '').lower() else None
html_encoding = EncodingDetector.find_declared_encoding(resp.content, is_html=True)
encoding = html_encoding or http_encoding
soup = BeautifulSoup(resp.content, parser, from_encoding=encoding)
for link in soup.find_all('a', href=True):
print(link['href'])
你的意思是这样的吗？
links = []
for link in soup.find_all('a', href=True):
links.append(link['href'])
for link in links:
article=Article(link)
article.download()
article.parse()
print(article.title)