我在Android上写一个webcrawler
。我的代码是
public void parseHttp() {
AsyncHttpClient client = new AsyncHttpClient();
String url = "http://stackoverflow.com/questions/38959381/unable-to-scrape-data-from-internet-using-android-intents";
client.get(url, new AsyncHttpResponseHandler(Looper.getMainLooper()) {
@Override
public void onSuccess(int statusCode, Header[] headers, byte[] responseBody) {
String body = new String(responseBody);
System.out.println(body);
Pattern p = Pattern.compile("<h1(.*)<\/h1>");
Matcher m = p.matcher(body);
Log.d("tag", "success");
if ( m.find() ) {
String match = m.group(1);
Log.d("tag", match);
}
}
@Override
public void onFailure(int statusCode, Header[] headers, byte[] responseBody, Throwable error) {
Log.d("tag", "failure");
}
});
}
它是在一个字符串中找到h1
标签,这是使用regex
的web文档的响应。我可以通过使用Jsoup
库作为
tag
吗?try {
Document doc;
URL = requestString;
doc = Jsoup.connect(URL).timeout(20 * 1000).userAgent("Chrome").get();
Elements links = doc.select("h1");
responseMessage = links.text();
} catch (IOException e) {
responseMessage = e.getMessage();
}
我可以找到标签在Jsoup
使用AsynsHTTPResponceHandler
类吗?第四行是Elements links = doc.select("h1"); responseMessage = links.text();
如有任何帮助或指导,我将不胜感激。
Jsoup允许从字符串解析文档,而不是直接通过HTTP(S)加载。
Document doc = Jsoup.parseBodyFragment(body);