使用python中的selenium网络驱动程序滚动网页

我目前正在使用selenium网络驱动程序来解析此网页(https://startup-map.berlin/companies.startups/f/all_locations/allof_Berlin/data_type/allof_Verified)使用Python提取所有启动URL。我尝试了这篇文章中提到的所有相关方法：如何在python中使用硒网络驱动程序滚动网页？以及其他在线建议。

然而，这个网站并没有成功。它只加载了前25家初创公司。一些代码示例：

from time import sleep
from bs4 import BeautifulSoup
from datetime import datetime
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
webdriver = webdriver.Chrome(executable_path='chromedriver')
# Write into csv file
filename = "startups_urls.csv"
f = open(BLD / "processed/startups_urls.csv", "w")
headers = "startups_urlsn"
f.write(headers)
url = "https://startup-map.berlin/companies.startups/f/all_locations/allof_Berlin/data_type/allof_Verified"
webdriver.get(url)
time.sleep(3)
# Get scroll height
last_height = webdriver.execute_script("return document.body.scrollHeight")
while True:
# Scroll down to bottom
webdriver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
# Wait to load page
time.sleep(3)
# Calculate new scroll height and compare with last scroll height
new_height = webdriver.execute_script("return document.body.scrollHeight")
if new_height == last_height:
break
last_height = new_height
htmlSource = webdriver.page_source
page_soup = BeautifulSoup(htmlSource, "html.parser")
startups = page_soup.findAll("div", {"class": "type-element type-element--h3 hbox entity-name__name entity-name__name--black"})
if startups != []:
for startup in startups:
startups_href = startup.a["href"]
startups_url = "https://startup-map.berlin" + startups_href
open_file.write(startups_url + "n")
else:
print("NaN.") 

f.close()
driver.close()

有什么建议吗？非常感谢。