如何使用Python请求将JSON与数据负载和标头POST

Question

我正试图使用GET和POST方法从网站上进行一些抓取，但现在我面临着一个新的挑战。

我正试图从信用模拟器中获取数据，我发现了这个葡萄牙网站(https://www.bancomontepio.pt/particulares/credito/pessoal/credito-pessoal-online)。

据我所知，我需要使用POST方法，但我必须指定数据(Amount值、Term…(。我通常通过创建字典结构来完成，但这不起作用。

公平地说，我有点迷失了方向，也许问题出在头球上。。。

这是我的代码：

import requests
import warnings
warnings.filterwarnings("ignore")
term=24
amount=5000
url = 'https://simuladores.bancomontepio.pt/ITSCredit.External/Calculator/ITSCredit.Calculator.UI.External/gateway/Calculator/api/Calculator/Calculate?hash=-1359629931'
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.122 Safari/537.36',
'Accept-Language': 'pt-PT,pt;q=0.9,en-US;q=0.8,en;q=0.7'}
payload = {'Amount': amount,'Term': term,'ProductCode':"26B1129900X"}
response = requests.post(url, headers=headers, data=payload, verify=False).json()

如果我去掉.json((，就会得到错误Response[410]。

目标是获得当项("Prazo"(或量("Montante"(值改变时改变的TAN或TAEG。

有什么想法吗？

[编辑]

headers = {'Accept': 'application/json, text/plain, */*' ,
'Accept-Encoding': 'gzip, deflate, br',
'Accept-Language': 'pt-PT,pt;q=0.9,en-US;q=0.8,en;q=0.7',
'Connection': 'keep-alive',
'Content-Length': '957',
'content-type': 'text/plain',
'Cookie': '_gcl_au=1.1.911606195.1646064658; OptanonAlertBoxClosed=2022-02-28T16:45:11.586Z; _ga=GA1.2.1147601977.1646064657; _ga_8WVEJF7X11=GS1.1.1646305654.3.1.1646309750.0; _ga_63QCVBV1V3=GS1.1.1646305679.1.1.1646309750.0; ASP.NET_SessionId=wlfbf2dx4oatlio0vl1ftinq; _gid=GA1.2.449121330.1646650093; calc-cookie=; OptanonConsent=isGpcEnabled=0&datestamp=Mon+Mar+07+2022+11%3A38%3A48+GMT%2B0000+(Hora+padr%C3%A3o+da+Europa+Ocidental)&version=6.30.0&isIABGlobal=false&consentId=6caccc97-6af1-4b55-9049-5694835d9f7a&interactionCount=2&landingPath=NotLandingPage&groups=C0001%3A1%2CC0002%3A1%2CC0003%3A1%2CC0004%3A1&hosts=H10%3A1%2CH20%3A1%2CH7%3A1%2CH8%3A1%2CH23%3A1%2CH11%3A1%2CH24%3A1%2CH13%3A1%2CH25%3A1&genVendors=&geolocation=ES%3B&AwaitingReconsent=false; _gali=slider-container; _gat_UA-186811106-6=1',
'Host': 'simuladores.bancomontepio.pt',
'Origin': 'https://simuladores.bancomontepio.pt',
'Referer': 'https://simuladores.bancomontepio.pt/',
'sec-ch-ua': '" Not A;Brand";v="99", "Chromium";v="90", "Google Chrome";v="90"',
'sec-ch-ua-mobile': '?0',
'Sec-Fetch-Dest': 'empty',
'Sec-Fetch-Mode': 'cors',
'Sec-Fetch-Site': 'same-origin',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36'}headers = {'Accept': 'application/json, text/plain, */*' ,
'Accept-Encoding': 'gzip, deflate, br',
'Accept-Language': 'pt-PT,pt;q=0.9,en-US;q=0.8,en;q=0.7',
'Connection': 'keep-alive',
'Content-Length': '957',
'content-type': 'text/plain',
'Cookie': '_gcl_au=1.1.911606195.1646064658; OptanonAlertBoxClosed=2022-02-28T16:45:11.586Z; _ga=GA1.2.1147601977.1646064657; _ga_8WVEJF7X11=GS1.1.1646305654.3.1.1646309750.0; _ga_63QCVBV1V3=GS1.1.1646305679.1.1.1646309750.0; ASP.NET_SessionId=wlfbf2dx4oatlio0vl1ftinq; _gid=GA1.2.449121330.1646650093; calc-cookie=; OptanonConsent=isGpcEnabled=0&datestamp=Mon+Mar+07+2022+11%3A38%3A48+GMT%2B0000+(Hora+padr%C3%A3o+da+Europa+Ocidental)&version=6.30.0&isIABGlobal=false&consentId=6caccc97-6af1-4b55-9049-5694835d9f7a&interactionCount=2&landingPath=NotLandingPage&groups=C0001%3A1%2CC0002%3A1%2CC0003%3A1%2CC0004%3A1&hosts=H10%3A1%2CH20%3A1%2CH7%3A1%2CH8%3A1%2CH23%3A1%2CH11%3A1%2CH24%3A1%2CH13%3A1%2CH25%3A1&genVendors=&geolocation=ES%3B&AwaitingReconsent=false; _gali=slider-container; _gat_UA-186811106-6=1',
'Host': 'simuladores.bancomontepio.pt',
'Origin': 'https://simuladores.bancomontepio.pt',
'Referer': 'https://simuladores.bancomontepio.pt/',
'sec-ch-ua': '" Not A;Brand";v="99", "Chromium";v="90", "Google Chrome";v="90"',
'sec-ch-ua-mobile': '?0',
'Sec-Fetch-Dest': 'empty',
'Sec-Fetch-Mode': 'cors',
'Sec-Fetch-Site': 'same-origin',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36'}
payload = {'CCRDCalculateInput':{},'MultifunctionsCalculateInput':{},'Device':{'Browser':'chrome','BrowserVersion':'90.0.4430.212','Device':'Desktop','Os':'windows','OsVersion':'windows-10','UserAgent':'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36'},'IsCustomer':'true','Amount':7500,'Term':60,'ConditionCode':'26B1129900X-01-I-129900-F','CreditDestinationCode':'129900','ProductCode':'26B1129900X','FinancedExpenses':'false','FrequencyTrancheCode':'null','GoalCode':'C006','GoalDescription':'PROJETOS PESSOAIS','FrequencyTypeCode':'M','FamilyCode':'CP','Proponents':[{'Position':1,
'Birthday':'1992-03-07T13:03:30.000Z','State':'true','EntityType':{'ID':1,'CompanyID':1,'Code':'P','Description':'Proponente','Value':'null','ValueString':'null','State':'true','Imported':'null'},'ExpenseCodes':['009']}],'Counterparts':0,'OptionalExpenses':[{'Code':'009','Factor':1}],'ResidualValue':0,'InterestOnly':0,'Deferment':0}

现在我得到一个空的json((。。。响应200，但我得到了这个结构：

{'Status': 'Unknown',
'Error': {'VisibleToHuman': False, 'Code': '0', 'Message': ''},
'Result': None}

据我所知，这种状态应该是"；OK"；以获取有关结果的一些信息。

干杯

Answer 1

看起来您需要扩展有效负载以包括更多(所有(参数(包括cookie，特别是ASP.NET_SessionId(。

import requests
import warnings
warnings.filterwarnings("ignore")
term=24
amount=5000
url = 'https://simuladores.bancomontepio.pt/ITSCredit.External/Calculator/ITSCredit.Calculator.UI.External/gateway/Calculator/api/Calculator/Calculate?hash=-1359629931'
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.122 Safari/537.36',
'Accept-Language': 'pt-PT,pt;q=0.9,en-US;q=0.8,en;q=0.7',
'Cookie':'ASP.NET_SessionId=fhkyn1vn5knlw3uhdnh50nii;'}

payload = {
"CCRDCalculateInput":{},
"MultifunctionsCalculateInput":{},
"Device":{
"Browser":"chrome",
"BrowserVersion":"96.0.4664.110",
"Device":"Desktop",
"Os":"windows",
"OsVersion":"windows-10",
"UserAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36"},
"IsCustomer":'true',
"Amount":amount,
"Term":term,
"ConditionCode":"26B1129900X-01-I-129900-F",
"CreditDestinationCode":"129900",
"ProductCode":"26B1129900X",
"FinancedExpenses":'false',
"FrequencyTrancheCode":'null',
"GoalCode":"C006",
"GoalDescription":"PROJETOS PESSOAIS",
"FrequencyTypeCode":"M",
"FamilyCode":"CP",
"Proponents":[{
"Position":'1',
"Birthday":"1992-03-10T13:03:24.000Z",
"State":'true',
"EntityType":{
"ID":'1',
"CompanyID":'1',
"Code":"P",
"Description":"Proponente",
"Value":'null',
"ValueString":'null',
"State":'true',
"Imported":'null'},
"ExpenseCodes":[
"009"]}],
"Counterparts":'0',
"OptionalExpenses":[{
"Code":"009",
"Factor":'1'}],
"ResidualValue":'0',
"InterestOnly":'0',
"Deferment":'0'}
jsonData = requests.post(url, headers=headers, json=payload, verify=False).json()
results = jsonData['Result']
mtic = results['MTIC']
installment = results['PeriodInstallment'][0]['Installment']
taeg = results['TAEG']
tan = results['PeriodInstallment'][0]['TAN']

print(f'Installment: {installment}nTAEG: {taeg}nTAN: {tan}nMTIC: {mtic}')

输出：

Installment: 224.5
TAEG: 14.8
TAN: 7.0
MTIC: 5708.2