Golang Colly刮刮-网站Captcha抓住我的刮刮

Question

我确实为亚马逊产品标题制作了Scratching，但亚马逊captcha抓住了我的scratcher。我试了10次-go run main.go(8次抓住我-2次我刮到产品标题(

我对此进行了研究，但我没有找到任何解决方案来解决golang(只有python(，有什么解决方案吗？

---

package main
import (
"fmt"
"strings"0
"github.com/gocolly/colly"
)
func main() {
// Create a Collector specifically for Shopify
c := colly.NewCollector(
colly.AllowedDomains("www.amazon.com", "amazon.com"),
)
c.OnHTML("div", func(h *colly.HTMLElement) {
capctha := h.Text
title := h.ChildText("span#productTitle")
fmt.Println(strings.TrimSpace(title))
fmt.Println(strings.TrimSpace(capctha))
})
// Start the collector
c.Visit("https://www.amazon.com/Bluetooth-Over-Ear-Headphones-Foldable-Prolonged/dp/B07K5214NZ")
}

输出：

输入您在下面看到的字符对不起，我们只需要确保你不是机器人。为了获得最佳效果，请确保您的浏览器接受cookie。

Answer 1

如果你不介意使用不同的包，我写了一个包来搜索HTML(基本上是github.com/tdewolff/parse的薄包装(：

package main
import (
"github.com/89z/parse/html"
"net/http"
"os"
)
func main() {
req, err := http.NewRequest(
"GET", "https://www.amazon.com/dp/B07K5214NZ", nil,
)
req.Header = http.Header{
"User-Agent": {"Mozilla"},
}
res, err := new(http.Transport).RoundTrip(req)
if err != nil {
panic(err)
}
defer res.Body.Close()
lex := html.NewLexer(res.Body)
lex.NextAttr("id", "productTitle")
os.Stdout.Write(lex.Bytes())
}

结果：

Bluetooth Headphones Over-Ear, Zihnic Foldable Wireless and Wired Stereo
Headset Micro SD/TF, FM for Cell Phone,PC,Soft Earmuffs &Light Weight for
Prolonged Waring(Rose Gold)

https://github.com/89z/parse