根据Selenium结果创建熊猫的数据框

Question

m.webdriver.support.ui导入WebDriverWait从selenium.com.exceptions导入TimeoutException从时间导入睡眠来自硒.webdriver.support import expectedconditions as EC从selenium.webdriver.common.通过导入

driver = webdriver.Firefox()
driver.maximize_window()
driver.get('https://tradingview.com/screener/')
wait = WebDriverWait(driver, 35)
wait.until(EC.visibility_of_element_located((By.XPATH, "//div[@data-set='income_statement']"))).click()

# Scroll down 100 times
i  = 1
while i < 75:
print(i)
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
i += 1
sleep(1)

delay = 5 #seconds

# table updated, get the data
for row in driver.find_elements(By.CSS_SELECTOR,".tv-data-table__tbody tr"):
print(row.text) 

结果示例

AAPLD
APPLE INC
5.67USD 5.66USD 5.61USD 121.463BUSD 35.174BUSD 152.836BUSD 365.817BUSD 5.61USD 365.817BUSD 94.68BUSD
MSFTD
MICROSOFT CORP.
8.12USD 9.01USD 8.95USD 86.531BUSD 31.671BUSD 115.856BUSD 168.088BUSD 8.05USD 168.088BUSD 61.271BUSD     

Answer 1

1. 安装panda

pip install pandas

2. 创建一个文本数组，然后从中创建一个数据帧

tickers = []
names = []
amounts = []
for i, row in enumerate(driver.find_elements(By.CSS_SELECTOR,".tv-data-table__tbody tr")):
rest = i % 3
if (rest == 0):
tickers.append(row)
if (rest == 1):
names.append(row)
if (rest == 2):
amounts.append(row)


import pandas as pd
df = pd.DataFrame(dict(tickers=tickers, names=names, amounts=amounts))