斯芬克斯:拒绝许可/德尔塔合并时管道破裂

Question

当我启动创建和合并增量的批处理命令时:

D:Sphinxbinindexer.exe --config D:Sphinxprojectproduct.conf idx_product_delta --rotate
D:Sphinxbinindexer.exe --config D:Sphinxprojectproduct.conf --merge idx_product_main idx_product_delta --rotate

在search .log中发现这个错误，delta没有合并到main

[Fri Sep 25 15:34:42.549 2015] [ 2312] WARNING: rotating index 'idx_product_main': cur to old rename failed: rename D:Sphinxprojectdataproduct.spa to D:Sphinxprojectdataproduct.old.spa failed: Broken pipe

控制台输出为:

using config file 'D:Sphinxprojectproduct.conf'...
merging index 'idx_product_delta' into index 'idx_product_main'...
read 7.2 of 7.2 MB, 100.0% done
merged 11.5 Kwords
merged in 0.127 sec
ERROR: index 'idx_product_main': failed to delete 'D:Sphinxprojectdataproduct.new.spa': Permission deniedtotal 671 reads, 0.006 sec, 15.3 kb/call avg, 0.0 msec/call avg total 36 writes, 0.004 sec, 277.8 kb/call avg, 0.1 msec/call avg

我的product.conf是:

source src_product_main
{
    type = mysql
    sql_host = localhost
    sql_user = root
    sql_pass =
    sql_db = database
    sql_port = 3306 # optional, default is 3306
    sql_query_pre = REPLACE INTO sphinx_index_meta(index_name, last_update) 
                    VALUES('idx_prodotti_main', current_timestamp())

    sql_query_range = SELECT MIN(id),MAX(id) 
                      FROM product 
                      WHERE deleted = 0 AND visible= 1
    sql_range_step = 1000
    sql_query = SELECT id, text, last_update 
                FROM product 
                WHERE id>=$start AND id<=$end AND deleted = 0 AND visible = 1
    sql_attr_timestamp = last_update
}

index idx_product_main
{
    source = src_product_main
    path = D:Sphinxprojectdataproduct
    ondisk_attrs = 1
    stopwords = D:Sphinxprojectstopwords.txt
    min_word_len = 2
    min_prefix_len = 0
    min_infix_len = 3
    ngram_len = 1
}

source src_product_delta : src_product_main
{
    sql_query_range = SELECT MIN(id),MAX(id) 
                      FROM product 
                      WHERE deleted = 0 AND visible= 1
    sql_range_step = 1000
    sql_query = SELECT id, text, last_update 
                FROM product 
                WHERE id>=$start AND id<=$end AND deleted = 0 AND visible = 1
}

index idx_product_delta : idx_product_main
{
    source = src_product_delta
    path = D:Sphinxprojectdeltaproduct
    ondisk_attrs = 1
    stopwords = D:Sphinxprojectstopwords.txt
    min_word_len = 2
    min_prefix_len = 0
    min_infix_len = 3
    ngram_len = 1
}

indexer
{
    mem_limit = 128M
    max_iosize = 1M
}

searchd
{
    listen = 9312
    listen = 9306:mysql41
    log = D:Sphinxprojectlogsearchd.log
    query_log = D:Sphinxprojectlogquery.log
    read_timeout = 5
    client_timeout = 300
    max_children = 30
    pid_file = D:Sphinxprojectlogsearchd.pid
    seamless_rotate = 1
    preopen_indexes = 0
    unlink_old = 1
    workers = threads # for RT to work
    binlog_path = D:Sphinxprojectdata
}

我也试过Windows 7和Windows 8，都是稳定的2.2.10和测试版2.3.1-id64-beta (r4926)有相同的错误。

• 索引器使用cron (windows调度器)作为系统用户运行
• 搜索以SYSTEM用户运行的服务
• D:Sphinxprojectdata文件夹权限对SYSTEM有完全控制

如何解决这个问题

UPDATE for Eugene Soldatov answer

我也试过了(第一个命令less -rotate)

D:Sphinxbinindexer.exe --config D:Sphinxprojectproduct.conf idx_product_delta
D:Sphinxbinindexer.exe --config D:Sphinxprojectproduct.conf --merge idx_product_main idx_product_delta --rotate

但是在控制台输出中发现这个错误

Sphinx 2.2.10-id64-release (2c212e0)
Copyright (c) 2001-2015, Andrew Aksyonoff
Copyright (c) 2008-2015, Sphinx Technologies Inc (http://sphinxsearch.com)
using config file 'D:Sphinxprojectproduct.conf'...
indexing index 'idx_prodotti_delta'...
FATAL: failed to lock D:Sphinxprojectdeltaprodotti.spl: No error, will not index. Try --rotate option.
Sphinx 2.2.10-id64-release (2c212e0)
Copyright (c) 2001-2015, Andrew Aksyonoff
Copyright (c) 2008-2015, Sphinx Technologies Inc (http://sphinxsearch.com)
using config file 'D:Sphinxprojectproduct.conf'...
merging index 'idx_prodotti_delta' into index 'idx_prodotti_main'...
read 7.2 of 7.2 MB, 100.0% done
merged 11.5 Kwords
merged in 0.214 sec
ERROR: index 'idx_prodotti_main': failed to delete 'D:Sphinxprojectdataprodotti.new.spa': Permission deniedtotal 20136 reads, 0.071 sec, 30.9 kb/call avg, 0.0 msec/call avg
total 36 writes, 0.012 sec, 283.3 kb/call avg, 0.3 msec/call avg

在search .log中发现这个错误

[Wed Sep 30 09:09:29.371 2015] [ 4244] rotating index 'idx_prodotti_main': started
[Wed Sep 30 09:09:29.381 2015] [ 4244] WARNING: rotating index 'idx_prodotti_main': cur to old rename failed: rename D:Sphinxprojectdataprodotti.spa to D:Sphinxprojectdataprodotti.old.spa failed: Broken pipe
[Wed Sep 30 09:09:29.381 2015] [ 4244] rotating index: all indexes done

更新2

也尝试在两个命令之间插入sleep

D:Sphinxbinindexer.exe --config D:Sphinxprojectproduct.conf idx_product_delta --rotate
timeout /t 60
D:Sphinxbinindexer.exe --config D:Sphinxprojectproduct.conf --merge idx_product_main idx_product_delta --rotate

控制台输出

:

ERROR: index 'idx_prodotti_main': failed to delete 'D:Sphinxprojectdataprodotti.new.spa': Permission deniedtotal 20137 reads, 0.072 sec, 30.9 kb/c

UPDATE 3: Issue solved

sphinx家伙在这里解决了这个问题

http://sphinxsearch.com/bugs/view.php?id=2335

Answer 1

这种行为的原因是——rotate命令是异步的，所以当您运行第二个命令时:

D:Sphinxbinindexer.exe --config D:Sphinxprojectproduct.conf --merge idx_product_main idx_product_delta --rotate

first可以继续使用索引idx_product_delta:

D:Sphinxbinindexer.exe --config D:Sphinxprojectproduct.conf idx_product_delta --rotate

，所以它是锁定的

如果可能的话，删除第一个命令的——rotate选项。

更新:

似乎你需要——旋转选项在第一个命令。因此，您可以测量完成任务所需的平均时间，并在两个命令之间插入sleep。例如，对于30秒:

D:Sphinxbinindexer.exe --config D:Sphinxprojectproduct.conf idx_product_delta --rotate
timeout /t 30
D:Sphinxbinindexer.exe --config D:Sphinxprojectproduct.conf --merge idx_product_main idx_product_delta --rotate