ajax搜索和抓取巴士页面的价格//也许是硒

Question

我正试图在公共汽车页面上获取路线价格导入请求来自bs4进口BeautifulSoup进口再

popup_linkz= list()
p=range(1, 2, 1)
for i in p:





def get_headers(session):
res = session.get("https://new.turbus.cl/turbuscl/inicio-compra")
if res.status_code == 200:
print("Got headers")
return res.text

else:
print("Failed to get headers")



def search(session):
data = {
'origenInputModal': 'Santiago',
'destinoInputModal':'Calama',
'fechaRegreso': '03-04-2021',
'fechaIda': '31-03-2021',

}

res = session.post(
"https://new.turbus.cl/turbuscl/seleccion-itinerario", 
data=data) #not sure if this is the search link
if res.status_code == 200:
print("Search succeeded")
return res.text
else:
print("Search failed with error:", res.reason)
print(res.text)    

def get_popup_link(html):
soup = BeautifulSoup(html, "html.parser")


for t in soup.find_all('div', {'class': 'ticket_price-value'}):
precio = t.find('[class$="ticket_price-value"]').text
#cantidad = t.select_one('[id$="lblCantidad"]').text
#descripction = t.select_one('[id$="lblDescripcion"]').text
print(f"{precio=} {precio=}")

#print()                
return precio

def main():
with requests.Session() as s:
get_headers(s)
html = search(s)
popup_links = (get_popup_link(html))
print(popup_links)
# popup_linkz.extend(popup_links)
#print(popup_links)
#print(popup_linkz)
#download_html = get_download_html(s, popup_links)
# print(download_html)
#popup_linkz.extend(popup_links for i in range(0, 1, 1))
main()

#a = popup_linkz
#print(a)
enter code here

这是链接https://new.turbus.cl/turbuscl/inicio-compra

所以现在我可以找到搜索的输入框，但不确定是否要运行它

我收到这个错误ValueError: too many values to unpack (expected 2)

所以我不太确定自己到底失败了什么。

为了成功，你会试着激励我吗？

我一直在努力寻找一种新的硒方法来进行搜索。。。。

我现在做的是对的，还是我的第一个方法更好？

--编码：utf-8--

"quot"创建于2022年3月29日星期二21:04:05

@作者：克里斯蒂安·马科斯"quot">

# -*- coding: utf-8 -*-
"""
Created on Tue Mar 29 16:20:40 2022
@author: christian marcos
"""
from selenium import webdriver as wd
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from bs4 import BeautifulSoup as bs
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.keys import Keys
from pandas.io.html import read_html
import requests
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys

#select and fill firs field origin
driver=wd.Chrome('C:\chromedriver.exe')
driver.maximize_window()
driver.get('https://new.turbus.cl/turbuscl/inicio-compra')
driver.implicitly_wait(20)
driver.find_element_by_xpath('//*[@id="origen"]').click();
wait = WebDriverWait(driver, 30)
#select and fill firs field 
driver.implicitly_wait(10)
driver.find_element_by_xpath('//*[@id="modalOriginCity"]/div/div/div[2]/div[2]/ul/li[1]').click();

致问候，

Answer 1

所需的岗位数据不同。在这种情况下，您需要：

{
"fechaSalidaTramo": "31/03/2022",
"mnemotecnicoCiudadOrigenTramo": "stgo",
"mnemotecnicoCiudadDestinoTramo": "aric",
"horaSalidaTramo": 0,
"horaSalidaTramoMaxima": 0,
"codigoLinea": 90,
"numeroViaje": 0,
"numeroCuentaCorrienteCliente": 0,
"codigoIdaRegreso": 1,
"cantidadAsientos": 1,
"numeroRegistros": 0
}

链接为https://new.turbus.cl/turbuscl/recursos/vtwst76/web1。

在python中，它看起来是这样的：

import requests
HOST = "https://nclt.gov.in/"
LINK = "https://new.turbus.cl/turbuscl/recursos/vtwst76/web1"
DATA = '{"fechaSalidaTramo":"31/03/2022","mnemotecnicoCiudadOrigenTramo":"stgo","mnemotecnicoCiudadDestinoTramo":"aric","horaSalidaTramo":0,"horaSalidaTramoMaxima":0,"codigoLinea":90,"numeroViaje":0,"numeroCuentaCorrienteCliente":0,"codigoIdaRegreso":1,"cantidadAsientos":1,"numeroRegistros":0}'

HEADERS = {
"Content-Type": "application/json",
}
def get_route(origin, destination):
res = requests.post(LINK, data=DATA, headers=HEADERS)
if res.status_code == 200:
print("getting routes")
return res.json()
else:
print(res)

def main():
info = get_route("here", "there")
print(info)
if __name__ == "__main__":
main()

我是如何得到答案的：

1. 转到网站
2. 打开"网络"选项卡，以便查看请求
3. 进行搜索，找到匹配的请求
4. 将请求复制为curl请求，并将其导入poster
5. 删除标头，并查看在执行请求时是否出现错误。重复此操作，直到您只得到所需的标头
6. 复制所需的标头和数据，并使用请求进行测试