JSoup无法从html获取链接

Question

我试图从网站的html中获取链接，但无法使用Jsoup。

这是HTML:

<div class="anime_muti_link">
<ul>
<li><div class="doamin">Domain</div><div class="link">Link</div></li>
<li class="anime">
<a href="#" class="active" rel="1" data-video="example.com" ><div class="server m1">Server m1</div><span>Watch This Link</span></a>
</li>

<li class="anime">
<a href="#" rel="1" data-video="example.com" ><div class="server m1">Server m2</div><span>Watch This Link</span></a>
</li>

<li class="xstreamcdn">
<a href="#" rel="29" data-video="example.com">Xstreamcdn</div><span>Watch This Link</span></a>
</li>
<li class="mixdrop">
<a href="#" rel="7" data-video="example.com"><div class="server mixdrop">Mixdrop</div><span>Watch This Link</span></a>
</li>
<li class="streamsb">
<a href="#" rel="13" data-video="example.com">StreamSB</div><span>Watch This Link</span></a>
</li>
<li class="doodstream">
<a href="#" rel="14" data-video="example.com">Doodstream</div><span>Watch This Link</span></a>
</li>

</ul>
</div>

这是我写的似乎不起作用的android代码：

try {
Document doc = Jsoup.connect(URL).get();
Elements content = doc.getElementsByClass("anime_muti_link");
Elements links = content.select("a");
String[] urls = new String[links.size()];
for (int i = 0; i < links.size(); i++) {
urls[i] = links.get(i).attr("data-video");
if (!urls[i].startsWith("https://")) {
urls[i] = "https:" + urls[i];
}
}
arrayList.addAll(Arrays.asList(urls));
Log.d("CALLING_URL", "Links: " + Arrays.toString(urls));
} catch (IOException e) {
e.getMessage();
}

有人能帮我做这个吗？感谢

编辑：基本上，我正在尝试获得这6个链接，并将它们添加到我的列表中，以便在应用程序中使用。

编辑2:

所以我发现了另一种看起来更好的HTML：

<div class="heading-servers">
<span><i class="fa fa-signal"></i> Servers</span>
<ul class="servers">
<li data-vs="https://example.com" class="server server-active" style="display: block;" onclick="return loadIframe('ifrm', this.getAttribute('data-vs'));">Netu</li>
<li data-vs="https://example.com" class="server" style="display: block;" onclick="return loadIframe('ifrm', this.getAttribute('data-vs'));">VideoVard</li>
<li data-vs="https://example.com" class="server" style="display: block;" onclick="return loadIframe('ifrm', this.getAttribute('data-vs'));">Doodstream</li>
<li data-vs="https://example.com" class="server" style="display: block;" onclick="return loadIframe('ifrm', this.getAttribute('data-vs'));">Okstream</li>
</ul>
</div>

Answer 1

正如您所看到的，在这个li定义中，您包含了一个嵌套的div:

<li class="xstreamcdn">
<a href="#" rel="29" data-video="example.com">Xstreamcdn</div><span>Watch This Link</span></a>
</li>

这导致变量内容，即类为anime_muti_link的HTML片段，看起来像：

<div class="anime_muti_link"> 
<ul> 
<li>
<div class="doamin">
Domain
</div>
<div class="link">
Link
</div></li> 
<li class="anime"> <a href="#" class="active" rel="1" data-video="example.com">
<div class="server m1">
Server m1
</div><span>Watch This Link</span></a> </li> 
<li class="anime"> <a href="#" rel="1" data-video="example.com">
<div class="server m1">
Server m2
</div><span>Watch This Link</span></a> </li> 
<li class="xstreamcdn"> <a href="#" rel="29" data-video="example.com">Xstreamcdn</a></li>
</ul>
</div>

即使整理HTML，也会得到类似的结果。我使用了我以前的一个答案中的代码：

Tidy tidy = new Tidy();
tidy.setXHTML(true);
tidy.setIndentContent(true);
tidy.setPrintBodyOnly(true);
tidy.setInputEncoding("UTF-8");
tidy.setOutputEncoding("UTF-8");
tidy.setSmartIndent(true);
tidy.setShowWarnings(false);
tidy.setQuiet(true);
tidy.setTidyMark(false);
org.w3c.dom.Document htmlDOM = tidy.parseDOM(new ByteArrayInputStream(html.getBytes()), null);
OutputStream out = new ByteArrayOutputStream();
tidy.pprint(htmlDOM, out);
String tidiedHtml = out.toString();
// System.out.println(tidiedHtml);
Document document = Jsoup.parse(tidiedHtml);
Elements content = document.getElementsByClass("anime_muti_link");
System.out.println(content);

这就是为什么你只找到三个锚。

请尝试更正HTML或选择锚标记作为文档级别：

Document document = Jsoup.parse(html);
// Elements content = document.getElementsByClass("anime_muti_link");
// System.out.println(content);
Elements links = document.select("a");
String[] urls = new String[links.size()];
for (int i = 0; i < links.size(); i++) {
urls[i] = links.get(i).attr("data-video");
if (!urls[i].startsWith("https://")) {
urls[i] = "https://" + urls[i];
}
}
System.out.println(Arrays.asList(urls));

如果获得的结果包含不需要的链接，也许你可以尝试缩小所使用的选择器，比如：

document.select(".anime_muti_link a")

如果这不起作用，另一种可能的选择可能是选择具有data-video属性的锚元素，a[data-video]:

Document document = Jsoup.parse(html);
Elements videoLinks = document.select("a[data-video]");
String[] urls = new String[videoLinks.size()];
for (int i = 0; i < videoLinks.size(); i++) {
urls[i] = videoLinks.get(i).attr("data-video");
if (!urls[i].startsWith("https://")) {
urls[i] = "https://" + urls[i];
}
}
System.out.println(Arrays.asList(urls));

---

使用您的新测试用例，您可以使用非常相似的代码获得所需的信息：

String html = "<div class="heading-servers">n" +
"     <span><i class="fa fa-signal"></i> Servers</span>n" +
"     <ul class="servers">n" +
"      <li data-vs="https://example.com" class="server server-active" style="display: block;" onclick="return loadIframe('ifrm', this.getAttribute('data-vs'));">Netu</li>n" +
"      <li data-vs="https://example.com" class="server" style="display: block;" onclick="return loadIframe('ifrm', this.getAttribute('data-vs'));">VideoVard</li>n" +
"      <li data-vs="https://example.com" class="server" style="display: block;" onclick="return loadIframe('ifrm', this.getAttribute('data-vs'));">Doodstream</li>n" +
"      <li data-vs="https://example.com" class="server" style="display: block;" onclick="return loadIframe('ifrm', this.getAttribute('data-vs'));">Okstream</li>n" +
"     </ul>n" +
"    </div>";
Document document = Jsoup.parse(html);
Elements videoLinks = document.select("div.heading-servers ul.servers li.server");
String[] urls = new String[videoLinks.size()];
for (int i = 0; i < videoLinks.size(); i++) {
urls[i] = videoLinks.get(i).attr("data-vs");
if (!urls[i].startsWith("https://")) {
urls[i] = "https://" + urls[i];
}
}
System.out.println(Arrays.asList(urls));

最重要的部分是选择器的定义，该选择器应应用于已解析的文档，在我们的案例中为div.heading-servers ul.servers li.server。

我提供了一个包含许多片段的选择器，但根据实际使用的HTML，它可以用ul.servers li.server甚至li.server进行简化。