当尝试使用以下命令将项目部署到scrapyd时:
scrapyd-deploy test2 -p NOAA
或者,当尝试运行我创建的爬行器时:
curl http://localhost:6800/schedule.json -d project=test -d spider=myspider
得到以下输出:
Packing version 1478644677
Deploying to project "test" in http://localhost:6800/addversion.json
Server response (200):
{"status": "error", "message": "Use "scrapy" to see available
commands", "node_name": "osboxes"}
当我检查以下文件时,它们显示为空:
/var/log/scrapyd/scrapyd.log
/var/log/scrapyd/scrapyd.out
/var/log/scrapyd/scrapyd.err
我参考了这个答案:在部署鸡蛋文件在scrapyd服务器然后{"状态";error", "message";IndexError:列表索引超出范围"}
但这是一个不同的输出,他有一个索引问题。
与我的错误我认为什么是错误的蜘蛛本身?IT似乎可以部署,但它无法启动蜘蛛。
我按照指示安装了scrapyd。这是我的scrapy.cfg文件:
[settings]
default = NOAA.settings
[deploy:test2]
url = http://localhost:6800/
project = NOAA
运行ubuntu 14.04
找到解决方案了!
我/etc/scrapyd/conf.D/000-default(配置文件)由于某些原因是空的…
1)我用以下信息填充文件:http://scrapyd.readthedocs.io/en/stable/config.html#config
[scrapyd]
eggs_dir = eggs
logs_dir = logs
items_dir =
jobs_to_keep = 5
dbs_dir = dbs
max_proc = 0
max_proc_per_cpu = 4
finished_to_keep = 100
poll_interval = 5
http_port = 6800
debug = off
runner = scrapyd.runner
application = scrapyd.app.application
launcher = scrapyd.launcher.Launcher
[services]
schedule.json = scrapyd.webservice.Schedule
cancel.json = scrapyd.webservice.Cancel
addversion.json = scrapyd.webservice.AddVersion
listprojects.json = scrapyd.webservice.ListProjects
listversions.json = scrapyd.webservice.ListVersions
listspiders.json = scrapyd.webservice.ListSpiders
delproject.json = scrapyd.webservice.DeleteProject
delversion.json = scrapyd.webservice.DeleteVersion
listjobs.json = scrapyd.webservice.ListJobs
2)在运行scrapyd的终端窗口执行ctrl + c重新启动服务,然后输入命令scrapyd。
3)重新运行前面所述的命令(sudo很重要):
sudo scrapyd-deploy test -p NOAA
4)成功!
{"status": "ok", "project": "NOAA", "version": "1478702964", "spiders": 3, "node_name": "osboxes"}