如何在Google Colab上创建torch服务器并使用预测

Question

我试图在谷歌colab上创建一个torchserve，但加载花了很长时间，似乎我甚至无法连接到服务。这有可能在colab上创建一个torchserve吗？以下是当我尝试这样做时它显示的内容。

正在删除孤立pid文件。警告：sun.reflect.Reflection.getCallerClass不受支持。这将影响性能。2022-05-01T10:56:00634[信息]main org.pyarch.serve.servingsdk.impl.PluginsManager-正在初始化插件管理器。。。2022-05-01T10:56:00775[信息]main org.pytarch.serve.ModelServer-Torchserve版本：0.5.3TS主页：/usr/local/lib/python3.8/site-packages当前目录：/content临时目录：/tmpGPU数量：0CPU数量：2最大堆大小：3248MPython可执行文件：/usr/local/bin/Python配置文件：/content/drive/MyDrive/CoreferenceGerman/Config.properties推断地址：http://0.0.0.0:8000管理地址：http://0.0.0.0:8001指标地址：http://0.0.0.0:8002Model Store：/content/drive/MyDrive/COferenceGerman初始模型：droc_incremental_no_segment_dinstance.mar日志目录：/content/logs度量目录：/content/logsNetty线程：0Netty客户端线程：0每个型号的默认工作人员：2黑名单Regex:N/A最大响应大小：6553500最大请求大小：6553500限制最大图像像素：true首选直接缓冲区：false允许的URL：[file://.|http？：//.]允许模型的自定义python依赖项：false度量报告格式：prometheus启用指标API：true工作流存储：/content/drive/MyDrive/COferenceGerman型号配置：N/A2022-05-01T10:56:00784[信息]main org.pyarch.serve.servingsdk.impl.PluginsManager-正在加载快照序列化程序插件。。。2022-05-01T10:56:00824[信息]main org.pyarch.serve.ModelServer-加载初始模型：droc_incremental_no_segment_dstance.mar2022-05-01T10:56:13006[DEBUG]main org.pyarch.serve.wlm.ModelVersionedRefs-为模型droc_incremental_no_segment_distance添加新版本0.0.12022-05-01T10:56:13009[DEBUG]main org.pyarch.serve.wlm.ModelVersionedRefs-将型号droc_incremental_no_segment_distance的默认版本设置为0.0.12022-05-01T10:56:13010[信息]main org.pytarch.serve.wlm.ModelManager-已加载模型droc_incremental_no_segment_distance。2022-05-01T10:56:13011[DEBUG]main org.pytarch.serve.wlm.ModelManager-updateModel：droc_incremental_no_segment_distance，count：22022-05-01T10:56:13032[信息]main org.pyarch.serve.ModelServer-使用EpollServerSocketChannel初始化推理服务器。2022-05-01T10:56:13039[DEBUG]W-9000-droc_incremental_no_segment_dinstance0.0.1 org.pytoch.serve.wlm.WorkerLifeCycle-Worker命令行：[python3，/usr/local/lib/python3.8/site-packages/ts/model_service_Worker.py，--袜子类型，unix，--袜子名称，/tmp/.ts.sock.9000]2022-05-01T10:56:13051[DEBUG]W-9001-droc_incremental_no_segment_dinstance0.0.1 org.pytoch.serve.wlm.WorkerLifeCycle-Worker命令行：[python3，/usr/local/lib/python3.8/site-packages/ts/model_service_Worker.py，--袜子类型，unix，--袜子名称，/tmp/.ts.sock.9001]2022-05-01T10:56:13318[警告]W-9001-droc_incremental_no_segment_dinstance=0.0.1-stderr MODEL_LOG-追溯(最后一次调用)：2022-05-01T10:56:13327[警告]W-9001-droc_incremental_no_segment_dinstance=0.0.1-stderr MODEL_LOG-文件/usr/local/lib/python3.8/site packages/ts/model_service_worker.py"；，第17行，in2022-05-01T10:56:13329[警告]W-9001-droc_incremental_no_segment_dinstance=0.0.1-stderr MODEL_LOG-来自ts.MODEL_loader导入ModelLoaderFactory2022-05-01T10:56:13331[警告]W-9001-droc_incremental_no_segment_dinstance=0.0.1-stderr MODEL_LOG-文件/usr/local/lib/python3.8/site packages/ts/model_loader.py"；，第16行，in2022-05-01T10:56:13333[警告]W-9001-droc_incremental_no_segment_dinstance=0.0.1-stderr MODEL_LOG-来自ts.service导入服务2022-05-01T10:56:13333[警告]W-9001-droc_incremental_no_segment_dinstance=0.0.1-stderr MODEL_LOG-文件//usr/local/lib/python3.8/site packages/ts/service.py"；，第14行，in2022-05-01T10:56:13339[警告]W-9001-droc_incremental_no_segment_dinstance=0.0.1-stderr MODEL_LOG-来自ts.protocol.otf_message_handler导入create_predict_response2022-05-01T10:56:13340[警告]W-9001-droc_incremental_no_segment_dinstance=0.0.1-stderr MODEL_LOG-文件/usr/local/lib/python3.8/site packages/tsprotocol/otf_message_handler.py"；，第15行，in2022-05-01T10:56:13341[警告]W-9001-droc_incremental_no_segment_dinstance=0.0.1-stderr MODEL_LOG-导入火炬2022-05-01T10:56:13347[信息]main org.pytarch.serve.ModelServer-推断API绑定到：http://0.0.0.0:80002022-05-01T10:56:13349[信息]main org.pyarch.serve.ModelServer-使用EpollServerSocketChannel初始化管理服务器。2022-05-01T10:56:13350[警告]W-9001-droc_incremental_no_segment_dinstance=0.0.1-stderr MODEL_LOG-ModuleNotFoundError：没有名为"torch"的模块2022-05-01T10:56:13367[信息]main org.pytarch.serve.ModelServer-管理API绑定到：http://0.0.0.0:80012022-05-01T10:56:13367[信息]main org.pyarch.serve.ModelServer-使用EpollServerSocketChannel初始化度量服务器。2022-05-01T10:56:13369[信息]main org.pytarch.serve.ModelServer-Metrics API绑定到：http://0.0.0.0:80022022-05-01T10:56:13373[警告]W-9000-droc_incremental_no_segment_dinstance=0.0.1-stderr MODEL_LOG-追溯(最后一次调用)：2022-05-01T10:56:13377[警告]W-9000-droc_incremental_no_segment_dinstance=0.0.1-stderr MODEL_LOG-文件/usr/local/lib/python3.8/site packages/ts/model_service_worker.py"；，第17行，in2022-05-01T10:56:13377[警告]W-9000-droc_incremental_no_segment_dinstance=0.0.1-stderr MODEL_LOG-来自ts.MODEL_loader导入ModelLoaderFactory2022-05-01T10:56:13378[警告]W-9000-droc_incremental_no_segment_dinstance=0.0.1-stderr MODEL_LOG-文件/usr/local/lib/python3.8/site packages/ts/model_loader.py"；，第16行，in2022-05-01T10:56:13378[警告]W-9000-droc_incremental_no_segment_dinstance=0.0.1-stderr MODEL_LOG-来自ts.service导入服务2022-05-01T10:56:13378[警告]W-9000-droc_incremental_no_segment_dinstance=0.0.1-stderr MODEL_LOG-文件//usr/local/lib/python3.8/site packages/ts/service.py"；，第14行，in2022-05-01T10:56:13378[警告]W-9000-droc_incremental_no_segment_dinstance=0.0.1-stderr MODEL_LOG-来自ts.protocol.otf_message_handler导入create_predict_response2022-05-01T10:56:13379[警告]W-9000-droc_incremental_no_segment_dinstance=0.0.1-stderr MODEL_LOG-文件/usr/local/lib/python3.8/site packages/tsprotocol/otf_message_handler.py"；，第15行，in2022-05-01T10:56:13379[警告]W-9000-droc_incremental_no_segment_dinstance=0.0.1-stderr MODEL_LOG-导入火炬2022-05-01T10:56:13379[警告]W-9000-droc_incremental_no_segment_dinstance=0.0.1-stderr MODEL_LOG-ModuleNotFoundError：没有名为"torch"的模块2022-05-01T10:56:13400[信息]W-9001-droc_incremental_no_segment_dinstance=0.0.1-stderr org.pyarch.serve.wlm.WorkerLifeCycle-已停止扫描程序-W-9001-doc_incremental _no_se分段_dinstance0.0.1-stderr2022-05-01T10:56:13403[警告]W-9001-droc_incremental_no_segment_dinstance0.0.1 org.pytoch.serve.wlm.WorkerLifeCycle-terminateIOStreams()threadName=W-9001-dro_incremental-no-segment_dstance0.0.1-stderr2022-05-01T10:56:13400[信息]W-9001-droc_incremental_no_segment_dinstance=0.0.1-stdout org.pyarch.serve.wlm.WorkerLifeCycle-已停止扫描仪-W-9001-doc_incremental-no-segment_dinstance 0.0.1-stdout2022-05-01T10:56:13404[警告]W-9001-droc_incremental_no_segment_dinstance0.0.1 org.pytoch.serve.wlm.WorkerLifeCycle-terminateIOStreams()threadName=W-9001-dro_incremental-no-segment_dstance0.0.1-stdout2022-05-01T10:56:13428[信息]W-9000-droc_incremental_no_segment_dinstance=0.0.1-stderr org.pyarch.serve.wlm.WorkerLifeCycle-已停止扫描程序-W-9000-dro_incremental-no-segment_dinstance 0.0.1-stderr2022-05-01T10:56:13428[信息]W-9000-droc_incremental_no_segment_dinstance=0.0.1-stdout org.pyarch.serve.wlm.WorkerLifeCycle-已停止的扫描仪-W9000-droc_incremental_no_segment_Dinstance-0.0.1-stdout2022-05-01T10:56:13430[警告]W-9000-droc_incremental_no_segment_dinstance0.0.1 org.pytoch.serve.wlm.WorkerLifeCycle-terminateIOStreams()threadName=W-9000-dro_incremental-no-segment_dstance0.0.1-stderr2022-05-01T10:56:13430[警告]W-9000-droc_incremental_no_segment_dinstance0.0.1 org.pytoch.serve.wlm.WorkerLifeCycle-terminateIOStreams()threadName=W-9000-dro_incremental-no-segment_dstance0.0.1-stdout2022-05-01T10:56:13431[错误]W-9000-droc_incremental_no_segment_dinstance=0.0.1 org.pytoch.serve.wlm.WorkerThread-后端工作线程错误org.pyarch.serve.wlm.WorkerInitializationException：后端流关闭。网址：org.pytorch.serve.wlm.WorkerLifeCycle.startWorker(WorkerLifeCycle.java:139)~[model-server.jar：？]网址：org.pytorch.serve.wlm.WorkerThread.connect(WorkerThread.java:292)~[model-server.jar：？]网址：org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:179)[model-server.jar：？]位于java.util.concurrent.Executors$RunnableAdapter.call(Executitors.java:515)[？：？]位于java.util.concurrent.FFutureTask.run(FutureTask.java:264)[？：？]位于java.util.concurrent.ThreadPoolExecutiator.runWorker(ThreadPoolExecutiator.java:1128)[？：？]位于java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)[？：？]在java.lang.Thread.run(Thread.java:829)[？：？]2022-05-01T10:56:13408[错误]W-9001-droc_incremental_no_segment_dinstance=0.0.1 org.pytoch.serve.wlm.WorkerThread-后端工作线程错误org.pyarch.serve.wlm.WorkerInitializationException：后端流关闭。网址：org.pytorch.serve.wlm.WorkerLifeCycle.startWorker(WorkerLifeCycle.java:139)~[model-server.jar：？]网址：org.pytorch.serve.wlm.WorkerThread.connect(WorkerThread.java:292)~[model-server.jar：？]网址：org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:179)[model-server.jar：？]位于java.util.concurrent.Executors$RunnableAdapter.call(Executitors.java:515)[？：？]位于java.util.concurrent.FFutureTask.run(FutureTask.java:264)[？：？]位于java.util.concurrent.ThreadPoolExecutiator.runWorker(ThreadPoolExecutiator.java:1128)[？：？]位于java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)[？：？]在java.lang.Thread.run(Thread.java:829)[？：？]模型服务器已启动。2022-05-01T10:56:13759[警告]pool-3-thread-1 org.pytorch.serve.metrics.MetricCollector-worker pid尚不可用。2022-05-01T10:56:13833[信息]pool-3-thread-1 TS_METRICS-CPUUtilization。百分比：100.0 |#级别：主机|#主机名：649a8b344986，时间戳：16514025732022-05-01T10:56:13835[信息]池-3-线程-1 TS_METRICS-磁盘可用。GB：66.321777375 |#级别：主机|#主机名：649a8b344986，时间戳：16514025732022-05-01T10:56:13837[信息]pool-3-thread-1 TS_METRICS-磁盘用法。GB：41.377681732177734 |#级别：主机|#主机名：649a8b344986，时间戳：16514025732022-05-01T10:56:13838[信息]pool-3-thread-1 TS_METRICS-磁盘利用率。百分比：38.4 |#级别：主机|#主机名：649a8b344986，时间戳：16514025732022-05-01T10:56:13839[信息]池-3-线程-1 TS_METRICS-内存可用。兆字节：11904.3984375 |#级别：主机|#主机名：649a8b344986，时间戳：16514025732022-05-01T10:56:13840[信息]池-3-线程-1 TS_METRICS-已用内存。兆字节：794.34375 |#级别：主机|#主机名：649a8b344986，时间戳：16514025732022-05-01T10:56:13841[信息]pool-3-thread-1 TS_METRICS-内存利用率。百分比：8.3 |#级别：主机|#主机名：649a8b344986，时间戳：1651402573

当我试图预测它只是永远加载

知道我该怎么在可乐上解决这个问题吗？它在我的电脑上正常工作，但我需要它在colab上。

Answer 1

它表示存在错误

ModuleNotFoundError: No module named 'torch' 

即使你修复了它。在Google Colab上运行服务器也会很棘手，因为Google禁止使用Colab资源，并且他们禁用了ssh和其他可以将torchserve用作服务器的方式。然而，如果客户端进程也在colab上，我认为如果您修复了错误，它可能会工作。