网页抓取问题下拉菜单在您检查它们时未显示

Question

我正在尝试从https://chanakyya.com/Election-Results?electionType=Assembly.我想争取北方邦的选票；Ajgara(SC(->投票站。我想刮去北方邦所有的投票站；Ajgara(SC(，但当我查看网站以获取类或id时，它只显示状态下拉菜单。到目前为止，代码只提供了北方邦的表格：

import requests
import pandas as pd
import choice
states = requests.get('https://chanakyya.com/Chanakya/states.json').json()
state_list = [x['stateDisplayName'] for x in states]
state_choice = choice.Menu(state_list).ask()
stateName = [x['stateName'] for x in states if x['stateDisplayName'] == state_choice][0]
url = f'https://chanakyya.com/Chanakya/{stateName}/{stateName}.json'
resultsData = requests.get(url).json()
tables = {}
for key, value in resultsData['ELECTION_DATA']['stateLevelData'].items():
table = pd.DataFrame(value)
tables[key] = table
for key, table in tables.items():
print(f'*** {key} ***')
print(table)

Answer 1

尝试：

import requests
import pandas as pd
url_1 = "https://chanakyya.com/Chanakya/UttarPradesh/AssemblyData/Ajagara_(SC).json?isNewDataFormat=true"
data = requests.get(url_1).json()
url_2 = (
"https://chanakyya.com/Chanakya/UttarPradesh/BoothData/"
+ data["ELECTION_DATA"]["boothFileName"]
)
data = requests.get(url_2).json()
df = pd.json_normalize(data)
print(df)

打印：

Party Assembly Cons No  Year Election Type                           Candidate Name Total rank                         Locality/Pooling Booth Vote polled
0      BJP              385  2014    Parliament                  DR MAHENDRA NATH PANDEY   731    1                               P.P.CHAMAV R.NO1         519
1       SP              385  2014    Parliament                                RAMKISHUN   731    2                               P.P.CHAMAV R.NO1          70
2      BSP              385  2014    Parliament                        ANIL KUMAR MAURYA   731    3                               P.P.CHAMAV R.NO1          56
3      AAP              385  2014    Parliament                                   IRSHAD   731    4                               P.P.CHAMAV R.NO1          23
4      INC              385  2014    Parliament   TARUN PATEL URF TARUNENDRA CHAND PATEL   731    5                               P.P.CHAMAV R.NO1           9
...and so on.

Answer 2

在inspect的网络选项卡中查看，您会发现这个url中有投票站列表的json

希望这能解决你的问题