我正在尝试编写一个应该访问此网站的小机器人http://lsa.colorado.edu/cgi-bin/LSA-pairwise.html,在文本区域中输入一些文本,然后按提交按钮从提交中获取结果页面。这是一个语言学项目。然而,当我点击HtmlSubmitInput按钮时,返回的URL似乎格式错误,因为IncorrectnessListenerImpl通知我:
Apr 10, 2016 2:38:35 PM com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl notify
WARNUNG: Incorrect URL "http:/cgi-bin/LSA-pairwise-x.html" has been corrected
URL应该是
http://lsa.colorado.edu/cgi-bin/LSA-pairwise-x.html
这会导致以下堆栈跟踪(由于长度而缩短):
Exception in thread "main" java.lang.RuntimeException: java.net.UnknownHostException: cgi-bin: unknown error
at com.gargoylesoftware.htmlunit.WebClient.download(WebClient.java:2078)
at com.gargoylesoftware.htmlunit.html.HtmlForm.submit(HtmlForm.java:141)
at com.gargoylesoftware.htmlunit.html.HtmlSubmitInput.doClickStateUpdate(HtmlSubmitInput.java:90)
at com.gargoylesoftware.htmlunit.html.DomElement.click(DomElement.java:795)
at com.gargoylesoftware.htmlunit.html.DomElement.click(DomElement.java:742)
at com.gargoylesoftware.htmlunit.html.DomElement.click(DomElement.java:689)
at LSABot.submitInput(LSABot.java:30)
at Start.main(Start.java:8)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
[...]
我的猜测是HtmlUnit试图修复URL,但这只会导致"cgi-bin",当然这是错误的。我找了一遍又一遍,但没有找到任何与我的问题相关的东西。
我的LSABot类:
public class LSABot {
final WebClient webClient;
private HtmlPage mainPg, rsltPg;
private HtmlForm htmlForm;
private HtmlTextArea txtA;
private HtmlSubmitInput submitBt;
public LSABot () throws Exception {
this.webClient = new WebClient(BrowserVersion.CHROME);
this.webClient.getOptions().setJavaScriptEnabled(true);
this.mainPg = this.webClient.getPage("http://lsa.colorado.edu/cgi-bin/LSA-pairwise.html");
this.htmlForm = this.mainPg.getForms().get(0);
this.txtA = this.htmlForm.getTextAreaByName("txt1");
this.submitBt = this.htmlForm.getInputByValue("Submit Texts");
}
public void submitInput(String input) {
this.txtA.setText(input);
try {
this.rsltPg = this.submitBt.click();
this.webClient.waitForBackgroundJavaScript(30*1000);
} catch (IOException ioe) {
ioe.printStackTrace();
}
}
错误来自表单的html内容。action
属性应该是http://lsa.colorado.edu/cgi-bin/LSA-pairwise-x.html
而不是http:/cgi-bin/LSA-pairwise-x.html
。
试试这个代码,它应该工作:
LogFactory.getFactory().setAttribute("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.NoOpLog");
java.util.logging.Logger.getLogger("com.gargoylesoftware").setLevel(Level.OFF);
java.util.logging.Logger.getLogger("org.apache.commons.httpclient").setLevel(Level.OFF);
WebClient client = new WebClient(BrowserVersion.CHROME);
client.getOptions().setJavaScriptEnabled(true);
client.getOptions().setThrowExceptionOnScriptError(false);
client.getOptions().setThrowExceptionOnFailingStatusCode(false);
String url = "http://lsa.colorado.edu/cgi-bin/LSA-pairwise.html";
final HtmlPage page = client.getPage(url);
HtmlForm htmlForm = page.getForms().get(0);
HtmlTextArea txtA = htmlForm.getTextAreaByName("txt1");
txtA.setText("hello");
HtmlSubmitInput submitBt = htmlForm.getInputByValue("Submit Texts");
// change the form action attribute to the correct one
htmlForm.setAttribute("action", "http://lsa.colorado.edu/cgi-bin/LSA-pairwise-x.html");
HtmlPage page2 = submitBt.click();
System.out.println(page2.asText());