CrateDB作为Django的时间序列数据库



我正在尝试使用CrateDB作为Django的时间序列数据库。我在docker上部署两个服务进行开发。

我从部署以下版本开始:

  • Django: 4.1.3
  • CrateDB: 5.1.0

然而,我得到兼容性错误:django.db.utils.NotSupportedError: PostgreSQL 11 or later is required (found 10.5) .

我已经将Django版本降级到3.0,在那里我没有得到这个错误,但是现在我得到了这个:

System check identified no issues (0 silenced).
Exception in thread django-main-thread:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 82, in _execute
    return self.cursor.execute(sql)
psycopg2.errors.InternalError_: Unknown function: pg_catalog.pg_table_is_visible(c.oid)
CONTEXT:  io.crate.exceptions.SQLExceptions.esToCrateException(SQLExceptions.java:164)
io.crate.exceptions.SQLExceptions.prepareForClientTransmission(SQLExceptions.java:151)
io.crate.protocols.postgres.Messages.sendErrorResponse(Messages.java:190)
io.crate.protocols.postgres.PostgresWireProtocol.handleSingleQuery(PostgresWireProtocol.java:795)
io.crate.protocols.postgres.PostgresWireProtocol.lambda$handleSimpleQuery$3(PostgresWireProtocol.java:748)
java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1187)
java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2309)
io.crate.protocols.postgres.PostgresWireProtocol.handleSimpleQuery(PostgresWireProtocol.java:748)
io.crate.protocols.postgres.PostgresWireProtocol$MessageHandler.dispatchMessage(PostgresWireProtocol.java:335)
io.crate.protocols.postgres.PostgresWireProtocol$MessageHandler.dispatchState(PostgresWireProtocol.java:325)
io.crate.protocols.postgres.PostgresWireProtocol$MessageHandler.channelRead0(PostgresWireProtocol.java:293)
io.crate.protocols.postgres.PostgresWireProtocol$MessageHandler.channelRead0(PostgresWireProtocol.java:277)
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:336)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:308)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)

The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/threading.py", line 980, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.9/threading.py", line 917, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/utils/autoreload.py", line 64, in wrapper
    fn(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/commands/runserver.py", line 121, in inner_run
    self.check_migrations()
  File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 486, in check_migrations
    executor = MigrationExecutor(connections[DEFAULT_DB_ALIAS])
  File "/usr/local/lib/python3.9/site-packages/django/db/migrations/executor.py", line 18, in __init__
    self.loader = MigrationLoader(self.connection)
  File "/usr/local/lib/python3.9/site-packages/django/db/migrations/loader.py", line 53, in __init__
    self.build_graph()
  File "/usr/local/lib/python3.9/site-packages/django/db/migrations/loader.py", line 220, in build_graph
    self.applied_migrations = recorder.applied_migrations()
  File "/usr/local/lib/python3.9/site-packages/django/db/migrations/recorder.py", line 77, in applied_migrations
    if self.has_table():
  File "/usr/local/lib/python3.9/site-packages/django/db/migrations/recorder.py", line 56, in has_table
    tables = self.connection.introspection.table_names(cursor)
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/base/introspection.py", line 52, in table_names
    return get_names(cursor)
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/base/introspection.py", line 47, in get_names
    return sorted(ti.name for ti in self.get_table_list(cursor)
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/postgresql/introspection.py", line 49, in get_table_list
    cursor.execute("""
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 98, in execute
    return super().execute(sql, params)
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python3.9/site-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 82, in _execute
    return self.cursor.execute(sql)
django.db.utils.InternalError: Unknown function: pg_catalog.pg_table_is_visible(c.oid)
CONTEXT:  io.crate.exceptions.SQLExceptions.esToCrateException(SQLExceptions.java:164)
io.crate.exceptions.SQLExceptions.prepareForClientTransmission(SQLExceptions.java:151)
io.crate.protocols.postgres.Messages.sendErrorResponse(Messages.java:190)
io.crate.protocols.postgres.PostgresWireProtocol.handleSingleQuery(PostgresWireProtocol.java:795)
io.crate.protocols.postgres.PostgresWireProtocol.lambda$handleSimpleQuery$3(PostgresWireProtocol.java:748)
java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1187)
java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2309)
io.crate.protocols.postgres.PostgresWireProtocol.handleSimpleQuery(PostgresWireProtocol.java:748)
io.crate.protocols.postgres.PostgresWireProtocol$MessageHandler.dispatchMessage(PostgresWireProtocol.java:335)
io.crate.protocols.postgres.PostgresWireProtocol$MessageHandler.dispatchState(PostgresWireProtocol.java:325)
io.crate.protocols.postgres.PostgresWireProtocol$MessageHandler.channelRead0(PostgresWireProtocol.java:293)
io.crate.protocols.postgres.PostgresWireProtocol$MessageHandler.channelRead0(PostgresWireProtocol.java:277)
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:336)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:308)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)

任何建议吗?

刚刚在CrateDB (https://github.com/crate/crate/pull/13223)中进行了更改,允许返回不同的PostgreSQL版本号,这将使我们通过第一个问题。

但是即使在最新版本的Django中,我们仍然会看到pg_table_is_visible调用的问题,这是一个在CrateDB中目前没有实现的函数,这是在https://github.com/crate/crate/issues/7988下跟踪的。这个特殊的问题可以通过编辑/usr/lib/python3/dist-packages/django/db/backends/postgresql/introspection.py和编辑SQL语句来绕过,删除对该函数的检查,如果您使用单个模式和具有足够权限的用户,这应该是ok的。

我希望这能有所帮助,但取决于你想做什么,你可能会遇到的另一个问题是自动增量列,默认情况下,Django使用的数据类型如serial(例如在django_migrations)和bigserial(自动字段),CrateDB是一个分布式系统,它目前没有这样的序列概念。可以使用uuid(参见https://code.djangoproject.com/ticket/32577)。

最新更新