如何将代理与 nodejs 一起使用

Question

我正在尝试抓取一个网站"https://shmoti.com"，但似乎他们已经阻止了我的 ip。所以我正在尝试使用带有节点的代理.js.

我正在使用node-fetch模块来获取网站的html正文。

如何将代理与node-fetch一起使用？我也不知道从哪里获得这样的免费代理.对此有任何帮助吗？

我需要使用获取模块本身，因为我正在以异步方式进行所有处理。

Answer 1

https://github.com/bitinn/node-fetch#options

它从他们的文档中说

{
// These properties are part of the Fetch Standard
method: 'GET',
headers: {},        // request headers. format is the identical to that accepted by the Headers constructor (see below)
body: null,         // request body. can be null, a string, a Buffer, a Blob, or a Node.js Readable stream
redirect: 'follow', // set to `manual` to extract redirect headers, `error` to reject redirect
// The following properties are node-fetch extensions
follow: 20,         // maximum redirect count. 0 to not follow redirect
timeout: 0,         // req/res timeout in ms, it resets on redirect. 0 to disable (OS limit applies)
compress: true,     // support gzip/deflate content encoding. false to disable
size: 0,            // maximum response body size in bytes. 0 to disable
agent: null         // http(s).Agent instance, allows custom proxy, certificate, lookup, family etc.

}

所以也许你可以试试

fetch('example.com',{
agent: new HttpsProxyAgent('http://127.0.0.1:8580')
}).then(function(res){
...
})