是我的代码还是Sidekiq很慢,因为它不应该访问Rails模型?



我过去常常把很多代码封装在Thread.new {}中,然后开始偶然发现Sidekiq,使后台任务变得更容易一些。几周后,这很令人失望,我不知道这是我把代码输入sidekiq工作人员的方式,还是交易是什么。

例如,

[1] pry(#<DomainWorker>)> domain = Domain.find_by(unique_id: @unique_id)
[2] pry(#<DomainWorker>)> urls
=> [{:url=>"http://admin.example.com/"}, {:url=>"http://example.com/"}, {:url=>"http://www.example.com/"}, {:url=>"https://admin.example.com/"}, {:url=>"https://example.com/"}, {:url=>"https://www.example.com/"}]
[3] pry(#<DomainWorker>)> domain.domain_urls.build(urls)
=> [#<DomainUrl:0x0000557c97f2a3c8 id: nil, domain_id: 32, url: "http://admin.example.com/", created_at: nil, updated_at: nil>,
#<DomainUrl:0x0000557c97cc6e38 id: nil, domain_id: 32, url: "http://example.com/", created_at: nil, updated_at: nil>,
#<DomainUrl:0x0000557c97cadd70 id: nil, domain_id: 32, url: "http://www.example.com/", created_at: nil, updated_at: nil>,
#<DomainUrl:0x0000557c97c9c8e0 id: nil, domain_id: 32, url: "https://admin.example.com/", created_at: nil, updated_at: nil>,
#<DomainUrl:0x0000557c97c7ed18 id: nil, domain_id: 32, url: "https://example.com/", created_at: nil, updated_at: nil>,
#<DomainUrl:0x0000557c97c43df8 id: nil, domain_id: 32, url: "https://www.example.com/", created_at: nil, updated_at: nil>]

仅运行domain.domain_urls.build(urls)命令就需要大约15秒的执行时间。如果我打开rails c,这个命令会立即运行。

这是否违反了从sidekiq工作者或其他人那里操作模型的最佳实践?不知道为什么有时需要15-20秒,下次需要5秒,等等。这非常不一致。我甚至有一个MailWorker,有时发送电子邮件需要2秒,或者2分钟。不确定交易是什么。

这是我的config/sidekiq.yml配置文件:

development:  
:concurrency: 50
production:  
:concurrency: 50
:queues:
- default

是我的代码,还是Sidekiq不擅长做这些任务?我在这里错过了什么?

编辑

在我的开发环境中简单地禁用eager_load似乎会使取得一些的进展。

config/environments/development.rb:中

更改

config.eager_load = false

config.eager_load = true

它工作了一段时间,但在一堆任务排队并完成后,它又开始挂起。现在它又卡住了。

如果我在MySQL中运行show processlist,我会看到以下内容:

mysql> show full processlist;
+----+------------+-----------+-------------------+---------+------+----------+-----------------------+
| Id | User       | Host      | db                | Command | Time | State    | Info                  |
+----+------------+-----------+-------------------+---------+------+----------+-----------------------+
| 31 | app_dev | localhost | app_portal_dev | Sleep   | 1017 |          | NULL                  |
| 50 | root       | localhost | NULL              | Query   |    0 | starting | show full processlist |
| 55 | app_dev | localhost | app_portal_dev | Sleep   | 1059 |          | NULL                  |
| 57 | app_dev | localhost | app_portal_dev | Sleep   | 1028 |          | NULL                  |
| 58 | app_dev | localhost | app_portal_dev | Sleep   |  420 |          | NULL                  |
| 61 | app_dev | localhost | app_portal_dev | Sleep   | 1078 |          | NULL                  |
| 62 | app_dev | localhost | app_portal_dev | Sleep   |  425 |          | NULL                  |
| 63 | app_dev | localhost | app_portal_dev | Sleep   |  191 |          | NULL                  |
+----+------------+-----------+-------------------+---------+------+----------+-----------------------+
8 rows in set (0.00 sec)

此外,如果我向Sidekiq工作者发送终止TTIN信号,这就是输出:

2018-11-03T03:20:29.415Z 65977 TID-gsr3safel WARN: Thread TID-gsr4jqd7l processor
2018-11-03T03:20:29.415Z 65977 TID-gsr3safel WARN: /home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:68:in `select'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:68:in `rescue in _read_from_socket'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:64:in `_read_from_socket'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:56:in `gets'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:363:in `read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:263:in `block in read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:251:in `io'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:262:in `read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:118:in `block in call'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:232:in `block (2 levels) in process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:370:in `ensure_connected'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:222:in `block in process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:307:in `logging'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:221:in `process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:118:in `call'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:210:in `block in call_with_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:281:in `with_socket_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:209:in `call_with_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1172:in `block in _bpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:45:in `block in synchronize'
/home/myuser/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/monitor.rb:226:in `mon_synchronize'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:45:in `synchronize'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1169:in `_bpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1214:in `brpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/fetch.rb:36:in `block in retrieve_work'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq.rb:95:in `block in redis'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:65:in `block (2 levels) in with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:64:in `handle_interrupt'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:64:in `block in with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:61:in `handle_interrupt'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:61:in `with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq.rb:92:in `redis'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/fetch.rb:36:in `retrieve_work'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:89:in `get_one'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:99:in `fetch'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:82:in `process_one'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:71:in `run'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/util.rb:16:in `watchdog'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/util.rb:25:in `block in safe_thread'
2018-11-03T03:20:29.416Z 65977 TID-gsr3safel WARN: Thread TID-gsr4jqddx processor
2018-11-03T03:20:29.416Z 65977 TID-gsr3safel WARN: /home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:68:in `select'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:68:in `rescue in _read_from_socket'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:64:in `_read_from_socket'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:56:in `gets'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:363:in `read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:263:in `block in read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:251:in `io'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:262:in `read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:118:in `block in call'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:232:in `block (2 levels) in process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:370:in `ensure_connected'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:222:in `block in process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:307:in `logging'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:221:in `process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:118:in `call'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:210:in `block in call_with_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:281:in `with_socket_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:209:in `call_with_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1172:in `block in _bpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:45:in `block in synchronize'
/home/myuser/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/monitor.rb:226:in `mon_synchronize'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:45:in `synchronize'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1169:in `_bpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1214:in `brpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/fetch.rb:36:in `block in retrieve_work'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq.rb:95:in `block in redis'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:65:in `block (2 levels) in with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:64:in `handle_interrupt'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:64:in `block in with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:61:in `handle_interrupt'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:61:in `with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq.rb:92:in `redis'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/fetch.rb:36:in `retrieve_work'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:89:in `get_one'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:99:in `fetch'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:82:in `process_one'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:71:in `run'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/util.rb:16:in `watchdog'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/util.rb:25:in `block in safe_thread'
2018-11-03T03:20:29.416Z 65977 TID-gsr3safel WARN: Thread TID-gsr4jqdh1 processor
2018-11-03T03:20:29.416Z 65977 TID-gsr3safel WARN: /home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:68:in `select'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:68:in `rescue in _read_from_socket'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:64:in `_read_from_socket'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:56:in `gets'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:363:in `read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:263:in `block in read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:251:in `io'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:262:in `read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:118:in `block in call'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:232:in `block (2 levels) in process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:370:in `ensure_connected'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:222:in `block in process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:307:in `logging'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:221:in `process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:118:in `call'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:210:in `block in call_with_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:281:in `with_socket_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:209:in `call_with_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1172:in `block in _bpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:45:in `block in synchronize'
/home/myuser/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/monitor.rb:226:in `mon_synchronize'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:45:in `synchronize'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1169:in `_bpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1214:in `brpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/fetch.rb:36:in `block in retrieve_work'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq.rb:95:in `block in redis'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:65:in `block (2 levels) in with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:64:in `handle_interrupt'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:64:in `block in with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:61:in `handle_interrupt'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:61:in `with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq.rb:92:in `redis'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/fetch.rb:36:in `retrieve_work'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:89:in `get_one'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:99:in `fetch'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:82:in `process_one'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:71:in `run'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/util.rb:16:in `watchdog'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/util.rb:25:in `block in safe_thread'
2018-11-03T03:20:29.417Z 65977 TID-gsr3safel WARN: Thread TID-gsr4jqcuh processor
2018-11-03T03:20:29.417Z 65977 TID-gsr3safel WARN: /home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:68:in `select'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:68:in `rescue in _read_from_socket'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:64:in `_read_from_socket'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:56:in `gets'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:363:in `read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:263:in `block in read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:251:in `io'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:262:in `read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:118:in `block in call'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:232:in `block (2 levels) in process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:370:in `ensure_connected'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:222:in `block in process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:307:in `logging'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:221:in `process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:118:in `call'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:210:in `block in call_with_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:281:in `with_socket_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:209:in `call_with_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1172:in `block in _bpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:45:in `block in synchronize'
/home/myuser/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/monitor.rb:226:in `mon_synchronize'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:45:in `synchronize'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1169:in `_bpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1214:in `brpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/fetch.rb:36:in `block in retrieve_work'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq.rb:95:in `block in redis'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:65:in `block (2 levels) in with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:64:in `handle_interrupt'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:64:in `block in with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:61:in `handle_interrupt'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:61:in `with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq.rb:92:in `redis'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/fetch.rb:36:in `retrieve_work'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:89:in `get_one'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:99:in `fetch'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:82:in `process_one'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:71:in `run'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/util.rb:16:in `watchdog'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/util.rb:25:in `block in safe_thread'
2018-11-03T03:20:29.417Z 65977 TID-gsr3safel WARN: Thread TID-gsr4jqd4p processor
2018-11-03T03:20:29.417Z 65977 TID-gsr3safel WARN: /home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:68:in `select'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:68:in `rescue in _read_from_socket'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:64:in `_read_from_socket'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:56:in `gets'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:363:in `read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:263:in `block in read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:251:in `io'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:262:in `read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:118:in `block in call'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:232:in `block (2 levels) in process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:370:in `ensure_connected'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:222:in `block in process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:307:in `logging'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:221:in `process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:118:in `call'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:210:in `block in call_with_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:281:in `with_socket_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:209:in `call_with_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1172:in `block in _bpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:45:in `block in synchronize'
/home/myuser/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/monitor.rb:226:in `mon_synchronize'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:45:in `synchronize'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1169:in `_bpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1214:in `brpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/fetch.rb:36:in `block in retrieve_work'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq.rb:95:in `block in redis'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:65:in `block (2 levels) in with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:64:in `handle_interrupt'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:64:in `block in with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:61:in `handle_interrupt'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:61:in `with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq.rb:92:in `redis'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/fetch.rb:36:in `retrieve_work'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:89:in `get_one'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:99:in `fetch'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:82:in `process_one'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:71:in `run'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/util.rb:16:in `watchdog'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/util.rb:25:in `block in safe_thread'
2018-11-03T03:20:29.418Z 65977 TID-gsr3safel WARN: Thread TID-gsr4jqcn5 processor
2018-11-03T03:20:29.418Z 65977 TID-gsr3safel WARN: /home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:68:in `select'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:68:in `rescue in _read_from_socket'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:64:in `_read_from_socket'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:56:in `gets'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:363:in `read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:263:in `block in read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:251:in `io'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:262:in `read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:118:in `block in call'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:232:in `block (2 levels) in process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:370:in `ensure_connected'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:222:in `block in process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:307:in `logging'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:221:in `process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:118:in `call'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:210:in `block in call_with_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:281:in `with_socket_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:209:in `call_with_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1172:in `block in _bpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:45:in `block in synchronize'
/home/myuser/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/monitor.rb:226:in `mon_synchronize'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:45:in `synchronize'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1169:in `_bpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1214:in `brpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/fetch.rb:36:in `block in retrieve_work'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq.rb:95:in `block in redis'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:65:in `block (2 levels) in with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:64:in `handle_interrupt'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:64:in `block in with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:61:in `handle_interrupt'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:61:in `with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq.rb:92:in `redis'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/fetch.rb:36:in `retrieve_work'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:89:in `get_one'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:99:in `fetch'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:82:in `process_one'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:71:in `run'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/util.rb:16:in `watchdog'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/util.rb:25:in `block in safe_thread'

Rails开发模式的自动加载是单线程的。您可以将Sidekiq设置为任何您想要的并发性,但您仍然有一个巨大的互斥体来单线程处理您的作业,这样它们就可以安全地自动加载。打开eager_load或切换到生产模式将禁用自动加载。

并发:50太高了。不要将其设置为高于25。

也有可能您有大量的线程,但数据库连接量不同,请在config/database.yml中设置pool: 25

TTIN输出或数据库似乎没有任何问题。Sidekiq非常适合使用Rails和ActiveRecord。我们运行一个高负载的生产环境,有大约3万个并发线程(分布在几十个工作服务器上(,Sidekiq的表现非常出色。不要在开发模式中设置不必要的高并发性,尽管如果您只处理一个并不重要的任务。

尝试在本地调试Sidekiq工作程序-在方法的开头设置断点,并找到它变慢的地方。Sidekiq本身不太可能达到

相关内容

最新更新