如何配置elasticsearch使用基本授权的SSL

Question

我正在尝试在IBM Bluemix上部署一个使用Ramses (http://ramses.tech)的应用程序。不幸的是，应用程序在部署过程中崩溃了。在local.ini配置文件中，我做了如下设置:

# ElasticSearch
elasticsearch.hosts = xxxx.dblayer.com:9999
elasticsearch.http_auth = user:secret
elasticsearch.http.use_ssl = true
elasticsearch.verify_certs = true
elasticsearch.sniff = false
elasticsearch.index_name = my_api
elasticsearch.index.disable = false
elasticsearch.enable_refresh_query = false
elasticsearch.enable_aggregations = false
elasticsearch.enable_polymorphic_query = false

然而，当我尝试在ibm bluemix上部署我的应用程序时，我得到以下错误:

2016-10-16T13:40:52.226-0400[DEA/64]OUTStarting app instance (index 0) with guid 30875156-21f2-4e49-b115-882ec3efc41a
2016-10-16T13:41:01.121-0400[App/0]ERR2016-10-16 17:41:01,120 INFO [nefertari_mongodb.signals][MainThread] signals.setup_es_signals_for: setup_es_signals_for: <class 'nefertari_mongodb.documents.ESBaseDocument'>
2016-10-16T13:41:01.130-0400[App/0]ERR2016-10-16 17:41:01,129 INFO [nefertari.json_httpexceptions][MainThread] json_httpexceptions.includeme: Include json_httpexceptions
2016-10-16T13:41:01.130-0400[App/0]ERR2016-10-16 17:41:01,129 INFO [ramses][MainThread] __init__.includeme: Parsing RAML
2016-10-16T13:41:01.125-0400[App/0]ERR2016-10-16 17:41:01,124 INFO [nefertari][MainThread] __init__.includeme: nefertari 0.7.0
2016-10-16T13:41:01.186-0400[App/0]ERR2016-10-16 17:41:01,185 INFO [ramses.generators][MainThread] generators.generate_models: Configuring model for route `schools`
2016-10-16T13:41:01.191-0400[App/0]ERR2016-10-16 17:41:01,190 INFO [ramses.utils][MainThread] utils.resource_schema: Searching for model schema
2016-10-16T13:41:01.191-0400[App/0]ERR2016-10-16 17:41:01,190 INFO [ramses.models][MainThread] models.setup_data_model: Generating model class `School`
2016-10-16T13:41:01.184-0400[App/0]ERR2016-10-16 17:41:01,183 INFO [ramses][MainThread] __init__.includeme: Starting models generation
2016-10-16T13:41:01.198-0400[App/0]ERR2016-10-16 17:41:01,198 INFO [ramses.utils][MainThread] utils.resource_schema: Searching for model schema
2016-10-16T13:41:01.200-0400[App/0]ERR2016-10-16 17:41:01,199 INFO [nefertari_mongodb.signals][MainThread] signals.setup_es_signals_for: setup_es_signals_for: <class 'mongoengine.base.metaclasses.User'>
2016-10-16T13:41:01.199-0400[App/0]ERR2016-10-16 17:41:01,198 INFO [ramses.models][MainThread] models.setup_data_model: Generating model class `User`
2016-10-16T13:41:01.203-0400[App/0]ERR2016-10-16 17:41:01,203 INFO [nefertari_mongodb.signals][MainThread] signals.setup_es_signals_for: setup_es_signals_for: <class 'mongoengine.base.metaclasses.School'>
2016-10-16T13:41:01.206-0400[App/0]ERR2016-10-16 17:41:01,205 INFO [nefertari.elasticsearch][MainThread] elasticsearch.setup: Including Elasticsearch. {'chunk_size': 500, 'index.disable': 'false', 'verify_certs': 'true', 'host': 'https://xxxx.dblayer.com:9999', 'enable_refresh_query': 'false', 'enable_aggregations': 'false', 'port': '443', 'index_name': 'quartolio_api', 'enable_polymorphic_query': 'false', 'hosts': 'xxxx.dblayer.com:9999', 'sniff': 'false', 'http_auth': 'user:secret'}
2016-10-16T13:41:01.205-0400[App/0]ERR2016-10-16 17:41:01,204 INFO [ramses.generators][MainThread] generators.generate_models: Configuring model for route `users`
2016-10-16T13:41:01.218-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 94, in perform_request
2016-10-16T13:41:01.218-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/urllib3/connectionpool.py", line 384, in _make_request
2016-10-16T13:41:01.218-0400[App/0]ERR _stacktrace=sys.exc_info()[2])
2016-10-16T13:41:01.218-0400[App/0]ERR chunked=chunked)
2016-10-16T13:41:01.218-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/httplib.py", line 1136, in getresponse
2016-10-16T13:41:01.218-0400[App/0]ERRTraceback (most recent call last):
2016-10-16T13:41:01.218-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/httplib.py", line 453, in begin
2016-10-16T13:41:01.218-0400[App/0]ERR response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
2016-10-16T13:41:01.218-0400[App/0]ERR httplib_response = conn.getresponse(buffering=True)
2016-10-16T13:41:01.218-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/urllib3/util/retry.py", line 251, in increment
2016-10-16T13:41:01.218-0400[App/0]ERR version, status, reason = self._read_status()
2016-10-16T13:41:01.218-0400[App/0]ERR2016-10-16 17:41:01,216 WARNI [elasticsearch][MainThread] base.log_request_fail: HEAD http://xxxx.dblayer.com:9999/my_api [status:N/A request:0.011s]
2016-10-16T13:41:01.218-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/urllib3/connectionpool.py", line 594, in urlopen
2016-10-16T13:41:01.218-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/httplib.py", line 417, in _read_status
2016-10-16T13:41:01.218-0400[App/0]ERR response.begin()
2016-10-16T13:41:01.218-0400[App/0]ERR raise six.reraise(type(error), error, _stacktrace)
2016-10-16T13:41:01.218-0400[App/0]ERR raise BadStatusLine(line)
2016-10-16T13:41:01.218-0400[App/0]ERRProtocolError: ('Connection aborted.', BadStatusLine("''",))
2016-10-16T13:41:01.221-0400[App/0]ERR return command.run()
2016-10-16T13:41:01.221-0400[App/0]ERR File "/home/vcap/app/.heroku/python/bin/pserve", line 11, in <module>
2016-10-16T13:41:01.221-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
2016-10-16T13:41:01.221-0400[App/0]ERR return loadapp(app_spec, name=name, relative_to=relative_to, **kw)
2016-10-16T13:41:01.221-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 272, in loadobj
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/urlmap.py", line 31, in urlmap_factory
2016-10-16T13:41:01.221-0400[App/0]ERR2016-10-16 17:41:01,221 ERROR [nefertari.json_httpexceptions][MainThread] json_httpexceptions.create_json_response: 400 BAD REQUEST: {"explanation": "('Connection aborted.', BadStatusLine("''",))", "extra": {"data": "ConnectionError(('Connection aborted.', BadStatusLine("''",))) caused by: ProtocolError(('Connection aborted.', BadStatusLine("''",)))"}, "status_code": 400, "message": null, "timestamp": "2016-10-16T17:41:01Z", "title": "Bad Request"}
2016-10-16T13:41:01.222-0400[App/0]ERR val = callable(*args, **kw)
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/util.py", line 55, in fix_call
2016-10-16T13:41:01.221-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/pyramid/scripts/pserve.py", line 328, in run
2016-10-16T13:41:01.221-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/pyramid/scripts/pserve.py", line 58, in main
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 710, in create
2016-10-16T13:41:01.222-0400[App/0]ERR File "/home/vcap/app/quartolio_api/quartolio_api/__init__.py", line 14, in main
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 710, in create
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/pyramid/config/__init__.py", line 755, in include
2016-10-16T13:41:01.222-0400[App/0]ERR **context.local_conf)
2016-10-16T13:41:01.221-0400[App/0]ERR global_conf=vars)
2016-10-16T13:41:01.221-0400[App/0]ERR sys.exit(main())
2016-10-16T13:41:01.221-0400[App/0]ERRSTACK BEGIN>>
2016-10-16T13:41:01.222-0400[App/0]ERR return fix_call(context.object, context.global_conf, **context.local_conf)
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 350, in get_app
2016-10-16T13:41:01.222-0400[App/0]ERR return func(*args, params=params, **kwargs)
2016-10-16T13:41:01.221-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/pyramid/scripts/pserve.py", line 363, in loadapp
2016-10-16T13:41:01.222-0400[App/0]ERR return self.object_type.invoke(self)
2016-10-16T13:41:01.221-0400[App/0]ERR return loadobj(APP, uri, name=name, **kw)
2016-10-16T13:41:01.222-0400[App/0]ERR params=params)
2016-10-16T13:41:01.222-0400[App/0]ERR config.include('nefertari.elasticsearch')
2016-10-16T13:41:01.222-0400[App/0]ERR app = loader.get_app(app_name, global_conf=global_conf)
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 144, in invoke
2016-10-16T13:41:01.222-0400[App/0]ERR return context.create()
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/elasticsearch/transport.py", line 327, in perform_request
2016-10-16T13:41:01.222-0400[App/0]ERR cls.api.indices.exists([index_name])
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/nefertari/json_httpexceptions.py", line 78, in __init__
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/nefertari/elasticsearch.py", line 213, in create_index
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/util.py", line 55, in fix_call
2016-10-16T13:41:01.223-0400[App/0]ERR return loadapp(app_spec, name=name, relative_to=relative_to, **kw)
2016-10-16T13:41:01.222-0400[App/0]ERR name=name, global_conf=global_conf).create()
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/nefertari/json_httpexceptions.py", line 67, in exception_response
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/pyramid/config/__init__.py", line 755, in include
2016-10-16T13:41:01.222-0400[App/0]ERR c(configurator)
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/nefertari/elasticsearch.py", line 69, in includeme
2016-10-16T13:41:01.222-0400[App/0]ERR return STATUS_MAP[status_code](**kw)
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/nefertari/json_httpexceptions.py", line 26, in add_stack
2016-10-16T13:41:01.222-0400[App/0]ERR val = callable(*args, **kw)
2016-10-16T13:41:01.222-0400[App/0]ERR c(configurator)
2016-10-16T13:41:01.222-0400[App/0]ERR ES.create_index()
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/ramses/__init__.py", line 56, in includeme
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/nefertari/elasticsearch.py", line 60, in perform_request
2016-10-16T13:41:01.223-0400[App/0]ERR **context.local_conf)
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/elasticsearch/client/indices.py", line 225, in exists
2016-10-16T13:41:01.222-0400[App/0]ERR create_json_response(self, **kw)
2016-10-16T13:41:01.223-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 146, in invoke
2016-10-16T13:41:01.223-0400[App/0]ERR return loadobj(APP, uri, name=name, **kw)
2016-10-16T13:41:01.222-0400[App/0]ERR extra=dict(data=e))
2016-10-16T13:41:01.222-0400[App/0]ERR status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
2016-10-16T13:41:01.223-0400[App/0]ERR File "/home/vcap/app/.heroku/python/bin/pserve", line 11, in <module>
2016-10-16T13:41:01.222-0400[App/0]ERR msg += 'nSTACK BEGIN>>n%snSTACK END<<' % add_stack()
2016-10-16T13:41:01.223-0400[App/0]ERRTraceback (most recent call last):
2016-10-16T13:41:01.224-0400[App/0]ERR c(configurator)
2016-10-16T13:41:01.222-0400[App/0]ERRSTACK END<<
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 69, in _wrapped
2016-10-16T13:41:01.223-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/pyramid/scripts/pserve.py", line 328, in run
2016-10-16T13:41:01.223-0400[App/0]ERR global_conf=vars)
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/nefertari/json_httpexceptions.py", line 58, in create_json_response
2016-10-16T13:41:01.223-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 710, in create
2016-10-16T13:41:01.223-0400[App/0]ERR return command.run()
2016-10-16T13:41:01.223-0400[App/0]ERR val = callable(*args, **kw)
2016-10-16T13:41:01.223-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/pyramid/scripts/pserve.py", line 58, in main
2016-10-16T13:41:01.223-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/pyramid/scripts/pserve.py", line 363, in loadapp
2016-10-16T13:41:01.223-0400[App/0]ERR return self.object_type.invoke(self)
2016-10-16T13:41:01.223-0400[App/0]ERR return fix_call(context.object, context.global_conf, **context.local_conf)
2016-10-16T13:41:01.223-0400[App/0]ERR return context.create()
2016-10-16T13:41:01.223-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/util.py", line 55, in fix_call
2016-10-16T13:41:01.224-0400[App/0]ERR extra=dict(data=e))
2016-10-16T13:41:01.223-0400[App/0]ERR return self.object_type.invoke(self)
2016-10-16T13:41:01.223-0400[App/0]ERR File "/home/vcap/app/quartolio_api/quartolio_api/__init__.py", line 14, in main
2016-10-16T13:41:01.223-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/util.py", line 55, in fix_call
2016-10-16T13:41:01.224-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/pyramid/config/__init__.py", line 755, in include
2016-10-16T13:41:01.223-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 144, in invoke
2016-10-16T13:41:01.223-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
2016-10-16T13:41:01.223-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 710, in create
2016-10-16T13:41:01.224-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/ramses/__init__.py", line 56, in includeme
2016-10-16T13:41:01.224-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/pyramid/config/__init__.py", line 755, in include
2016-10-16T13:41:01.224-0400[App/0]ERR config.include('nefertari.elasticsearch')
2016-10-16T13:41:01.223-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 272, in loadobj
2016-10-16T13:41:01.224-0400[App/0]ERR ES.create_index()
2016-10-16T13:41:01.224-0400[App/0]ERR c(configurator)
2016-10-16T13:41:01.224-0400[App/0]ERRnefertari.json_httpexceptions.JHTTPBadRequest: ('Connection aborted.', BadStatusLine("''",))
2016-10-16T13:41:01.223-0400[App/0]ERR val = callable(*args, **kw)
2016-10-16T13:41:01.224-0400[App/0]ERR status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
2016-10-16T13:41:01.224-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 69, in _wrapped
2016-10-16T13:41:01.223-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 350, in get_app
2016-10-16T13:41:01.223-0400[App/0]ERR config.include('ramses')
2016-10-16T13:41:01.223-0400[App/0]ERR app = loader.get_app(app_name, global_conf=global_conf)
2016-10-16T13:41:01.224-0400[App/0]ERR params=params)
2016-10-16T13:41:01.223-0400[App/0]ERR name=name, global_conf=global_conf).create()
2016-10-16T13:41:01.224-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/nefertari/elasticsearch.py", line 69, in includeme
2016-10-16T13:41:01.224-0400[App/0]ERR return func(*args, params=params, **kwargs)
2016-10-16T13:41:01.224-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/elasticsearch/transport.py", line 327, in perform_request
2016-10-16T13:41:01.224-0400[App/0]ERR cls.api.indices.exists([index_name])
2016-10-16T13:41:01.224-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/nefertari/elasticsearch.py", line 60, in perform_request
2016-10-16T13:41:01.218-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/urllib3/connectionpool.py", line 643, in urlopen
2016-10-16T13:41:01.222-0400[App/0]ERR return self.object_type.invoke(self)
2016-10-16T13:41:01.223-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/urlmap.py", line 31, in urlmap_factory
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 146, in invoke
2016-10-16T13:41:01.218-0400[App/0]ERR2016-10-16 17:41:01,217 ERROR [nefertari.elasticsearch][MainThread] elasticsearch.perform_request: ('Connection aborted.', BadStatusLine("''",))
2016-10-16T13:41:01.222-0400[App/0]ERR config.include('ramses')
2016-10-16T13:41:01.223-0400[App/0]ERR sys.exit(main())
2016-10-16T13:41:01.224-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/nefertari/elasticsearch.py", line 213, in create_index
2016-10-16T13:41:01.262-0400[App/0]OUT
2016-10-16T13:41:01.315-0400[DEA/64]ERRInstance (index 0) failed to start accepting connection

基于错误，看起来像是试图连接到http://xxxx.dblayer.com:9999而不是https://xxxx.dblayer.com:9999。

我已经尝试了很多不同的配置:

1. 我尝试设置elasticsearch.host和elasticsearch.port我从Nefertari那里得到了Exception: Bad or missing settings for elasticsearch. 'hosts'而不是elasticsearch.hosts。

2. 我已经尝试设置elasticsearch.hosts与https://前缀，导致ValueError: too many values to unpack从Nefertari(看起来它试图用结肠作为一个字段名)。

3. 我试过设置elasticsearch.hosts为空白，或只是一个在设置elasticsearch.host时，冒号被空白包围和elasticsearch.port，得到ValueError: need more than 0 values to unpack

如何配置elasticsearch连接bluemix elasticsearch服务?

Answer 1

您需要在VCAP_SERVICES凭据中使用作为ca_certificate_base64提供的自签名证书。

base64编码。在使用密钥之前需要解码密钥，如示例应用程序https://github.com/IBM-Bluemix/compose-elasticsearch-helloworld-nodejs/blob/master/server.js#L56