用于在 open-uri ruby 中查找 href 的正则表达式<a>

Question

我需要使用 ruby open-uri 找到两个网站之间的距离。用

def check(url)
    site = open(url.base_url)
    link = %r{^<([a])([^"]+)*([^>]+)*(?:>(.*)</1>|s+/>)$}
    site.each_line {|line| puts $&,$1,$2,$3,$4 if (line=~link)}
    p url.links
end

查找链接无法正常工作。任何想法为什么？

Answer 1

如果要查找a标签的href参数，请使用正确的工具，该工具通常不是正则表达式。更有可能的是，您应该使用 HTML/XML 解析器。

Nokogiri 是 Ruby 的首选解析器：

require 'nokogiri'
require 'open-uri'
doc = Nokogiri.HTML(open('http://www.example.org/index.html'))
doc.search('a').map{ |a| a['href'] }
pp doc.search('a').map{ |a| a['href'] }
# => [
# =>  "/",
# =>  "/domains/",
# =>  "/numbers/",
# =>  "/protocols/",
# =>  "/about/",
# =>  "/go/rfc2606",
# =>  "/about/",
# =>  "/about/presentations/",
# =>  "/about/performance/",
# =>  "/reports/",
# =>  "/domains/",
# =>  "/domains/root/",
# =>  "/domains/int/",
# =>  "/domains/arpa/",
# =>  "/domains/idn-tables/",
# =>  "/protocols/",
# =>  "/numbers/",
# =>  "/abuse/",
# =>  "http://www.icann.org/",
# =>  "mailto:[e-mail]?subject=General%20website%20feedback"
# => ]

Answer 2

我看到这个正则表达式有几个问题：

不一定是空格

• 必须在空标签中的尾部斜杠之前，但您的正则表达式需要它

• 您的正则表达式非常冗长和冗余

请尝试以下操作，它将从 标记中提取 URL：

link = /<a s   # Start of tag
    [^>]*       # Some whitespace, other attributes, ...
    href="      # Start of URL
    ([^"]*)     # The URL, everything up to the closing quote
    "           # The closing quotes
    /x          # We stop here, as regular expressions wouldn't be able to
                # correctly match nested tags anyway