如何在scrapy中为xpath编写正则表达式?



我是scrapy的新手,用它来在网页上查找问题和答案。我从这个页面开始了以下内容

我通过观察它们的 xpath 以这种方式尝试使用选择器:

sel = Selector(text=response.body)
spanList = (sel.xpath('//a/span').extract())

但是这样做时我得到了一些重复的内容,我以这种方式获得输出

"<span>How do I access my account online at Citibank Online?</span>",
"<span>What are the guidelines for creating an internet password?</span>",
"<span>I forgot my User ID for accessing my account online. How do I access my account online now?</span>",
"<span>How do I transfer funds to another bank account in India?</span>",
"<span>How do I transfer funds to my Rupee Checking Account from overseas?</span>",
"<span>How do I transfer funds from my Rupee Checking Account to my local bank account overseas?</span>",
"<span>How do I update my contact information?</span>",
"<span>I have not operated my Rupee Checking Account for a long time and I plan to visit India. Can I transact on my account when I visit India?</span>",
"<span>My Term Deposits with Citibank are due to mature soon. What do I need to do?</span>",
"<span>I would like to terminate my Term Deposits before maturity? Will I lose any money?</span>",
"<span>Why do I need to provide "Customer Profile Update" forms so often?</span>",
"<span>How do I access my account online at Citibank Online?</span>",
"<span>What are the guidelines for creating an internet password?</span>",
"<span>I forgot my User ID for accessing my account online. How do I access my account online now?</span>",
..................

如果您观察我发布的输出部分,则第一个和第三个跨度再次重复。

有没有办法编写一个好的正则表达式,无需重复即可获取内容。

我提到的页面中的问题样本 xpath 是

/html/body/div1/div[2]/div[3]/div[2]/div/div[2]/div/div[3]/div1/

div[3]/div1/a/span

/html/body/div1/div[2]/div[3]/div[2]/div/div[2]/div/div[3]/div1/div[5]/div[5]/div1/a/span

/html/body/div1/div[2]/div[3]/div[2]/div/div[2]/div/div[3]/div1/div[5]/div1/div1/a/span

看看这个,

points = response.xpath('//*[@class="ClsInnerDrop"]//span/text()').extract()
pointes = set(points)

最新更新