httpclient提供不同的结果

Question

我在使用 Apache HttpClient 4.5.6 与 PowerSchool 系统交互时遇到问题。我省略了这个问题不需要的方法，但我 100% 确定它们按预期工作。

当使用Postman或 requests.py 时，使用完全相同的表单数据(我已经在python，java和Firefox中验证过它(，我会得到预期的HTML页面，其中包含所需的成绩和类信息，介于<!-- start student content -->和<!-- end student content -->注释之间(超过1000行(。但是，当在 Java 中使用相同的表单数据和标头时，生成的 HTML 页面仅包含以下内容：

<!-- start student content -->
<div id="quickLookup">
<tr>
<th class="right" colspan="19">Attendance Totals</th>
<th>0</th>
<th>0</th>
</table>
<table border="0" cellpadding="3" cellspacing="1" width="100%">

<tr>
<td align="center">Current Cumulative GPA (Q1): X.XXXX</td>
</tr>
<tr>
<td align="center"><a href="home.html?schoolid=XXXX&showdropped=true&91146885685933636948">Show dropped classes also</a></td>
</tr>
</table>
<tr>
<th class="right" colspan="10">Attendance Totals</th>
<th>0</th>
<th>0</th>
<th>0</th>
<th>0</th>
</table>
</div>
<!-- end student content -->

两个不同结果之间的脱节对我来说毫无意义，因为据我所知，Postman 和 requests.py 不执行 javascript。我希望HttpClient的结果是相同的。这是我的代码：

private static final BasicCookieStore cookieStore = new BasicCookieStore();
private static final HttpClient client = HttpClientBuilder.create().setDefaultCookieStore(cookieStore).build();
public static void main(String[] args) {
String baseUrl = "https://powerschoolinstallurl/";
String username = "username";
String password = "password";
try {
// get hidden data fields, calc hmac data
HashMap<String, String> result = getAuthCodes(baseUrl);
String dbpwField = getDBPWField(result.get("contextData"), password);
String pwField = getPWField(result.get("contextData"), password);
List<NameValuePair> form = new ArrayList<>();
form.add(new BasicNameValuePair("pstoken", result.get("pstoken")));
form.add(new BasicNameValuePair("contextData", result.get("pstoken")));
form.add(new BasicNameValuePair("dbpw", dbpwField));
form.add(new BasicNameValuePair("serviceName", "PS Parent Portal"));
form.add(new BasicNameValuePair("pcasServerURL", "/"));
form.add(new BasicNameValuePair("credentialType", "User Id and Password Credential"));
form.add(new BasicNameValuePair("account", username));
form.add(new BasicNameValuePair("pw", pwField));
form.add(new BasicNameValuePair("ldappassword", password));
UrlEncodedFormEntity requestEntity = new UrlEncodedFormEntity(form);
HttpPost postMethod = new HttpPost(baseUrl + "guardian/home.html");
postMethod.setEntity(requestEntity);
HttpResponse rawResponse = client.execute(postMethod);
System.out.println(rawResponse.getStatusLine().getStatusCode());
try {
String responseString = new BasicResponseHandler().handleResponse(rawResponse);
System.out.println(responseString);
} catch (HttpResponseException ignore) {}
System.out.println(cookieStore.getCookies().toString());
HttpGet getMethod = new HttpGet(baseUrl + "guardian/home.html");
// replicating headers, result is the same nontheless
getMethod.setHeader("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
getMethod.setHeader("Accept-Encoding", "gzip, deflate, br");
getMethod.setHeader("Accept-Language", "en-US,en;q=0.5");
getMethod.setHeader("Referer", "https://powerschoolinstallurl/public/home.html");
getMethod.setHeader("DNT", "1");
getMethod.setHeader("Connection", "keep-alive");
getMethod.setHeader("Cache-Control", "no-cache");
getMethod.setHeader("Host", "ps.install.domain");
getMethod.setHeader("Upgrade-Insecure-Requests", "1");
getMethod.setHeader("Pragma", "no-cache");
HttpResponse resp2 = client.execute(getMethod);
String responseString2 = new BasicResponseHandler().handleResponse(resp2);
System.out.println(responseString2);

} catch (IOException e) {
e.printStackTrace();
}
}

注意：没有任何身份验证问题，除了该详细信息外，页面通常会返回。

Answer 1

据我了解，

HttpPost postMethod = new HttpPost(baseUrl + "guardian/home.html");

主页.html不包含任何服务器站点脚本，您可以使用 JavaScript 或其他东西通过使用 Ajax 调用调用另一个 API 来填充您的数据。

所以从爪哇，

HttpResponse resp2 = client.execute(getMethod);

它正在获取 HTML 并响应 HTML，

Answer 2

我最终使用了Jsoup，遵循完全相同的逻辑并得到了我想要的结果。仍然不知道为什么HttpClient不起作用。