尝试Apache Ignite集群快照过程(快照创建、恢复)，恢复操作不成功。
在恢复过程之前，我们已经销毁了我们想要恢复的缓存(根据官方文档)，但是在日志中仍然存在相同的错误消息/Java Exception。
Apache Ignite -快照创建/恢复过程:
./bin/control.sh --snapshot create 20230412_snapshot_test_v1
./bin/control.sh --cache destroy --caches CACHE_XYZ --yes
./bin/control.sh --snapshot restore 20230412_snapshot_test_v1 --start --groups CACHE_XYZ
Apache Ignite - Logs:
The snapshot operation will be aborted due to a handler error
class org.apache.ignite.IgniteCheckedException: The check procedure failed on 3 nodes.
The check procedure failed on nodes:
Node ID: 260115a7-95b4-48a3-bc98-af9d8af0df51 [192.168.XX.YY, 127.0.0.1]
Consistent ID: 547359f6-4501-4727-8563-a8b170a905f3
Exception: java.lang.NegativeArraySizeException
-935925367
Node ID: 33d84ca7-e4e2-4b6e-9c3c-63625c853dc1 [192.168.XX.YY, 127.0.0.1]
Consistent ID: aa7da35d-37b3-401d-8df3-f3c6e2761c85
Exception: java.lang.NegativeArraySizeException
-935925367
Node ID: d512f146-abb2-4e88-a22c-3a640e84be18 [192.168.XX.YY, 127.0.0.1]
Consistent ID: 0b6fafa5-a31f-4fa7-80c5-44c792af03ff
Exception: java.lang.NegativeArraySizeException
-935925367
...
Failed to restore snapshot cache groups
class org.apache.ignite.compute.ComputeUserUndeclaredException: Failed to reduce job results due to undeclared user exception [task=org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotHandlerRestoreTask@4c606da9, err=class org.apache.ignite.IgniteException: The check procedure failed on 3 nodes.
另一个奇怪的地方，由于某种原因--snapshot check无法连接到集群，但所有其他命令都正常工作。
Apache Ignite - Cluster Snapshot Verification:
./bin/control.sh --snapshot check 20230412_snapshot_test_v1 --verbose
...
INFO: Client TCP connection established: /127.0.0.1:11211
...
SEVERE: Failed to process selector key [ses=GridSelectorNioSessionImpl [worker=ByteBufferNioClientWorker [readBuf=java.nio.DirectByteBuffer[pos=0 lim=8192 cap=8192], super=AbstractNioClientWorker [idx=0, bytesRcvd=0, bytesSent=0, bytesRcvd0=0, bytesSent0=0, select=true, super=GridWorker [name=grid-nio-worker-tcp-client-0, igniteInstanceName=gridClient, finished=false, heartbeatTs=1682407593249, hashCode=245934772, interrupted=false, runner=grid-nio-worker-tcp-client-0-#5%gridClient%]]], writeBuf=null, readBuf=null, inRecovery=null, outRecovery=null, closeSocket=true, outboundMessagesQueueSizeMetric=null, super=GridNioSessionImpl [locAddr=/127.0.0.1:42122, rmtAddr=/127.0.0.1:11211, createTime=1682407593249, closeTime=0, bytesSent=702, bytesRcvd=78929, bytesSent0=702, bytesRcvd0=78929, sndSchedTime=1682407593249, lastSndTime=1682407593249, lastRcvTime=1682407593249, readsPaused=false, filterChain=FilterChain[filters=[GridNioCodecFilter [parser=GridTcpRestParser [marsh=JdkMarshaller [clsFilter=null], routerClient=false], directMode=false]], accepted=false, markedForClose=false]]]
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:192)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
at org.apache.ignite.internal.util.nio.GridNioServer$ByteBufferNioClientWorker.processRead(GridNioServer.java:1183)
at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.processSelectedKeysOptimized(GridNioServer.java:2508)
at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2273)
at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1910)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
at java.lang.Thread.run(Thread.java:750)
...
class org.apache.ignite.internal.client.impl.connection.GridClientConnectionResetException: Failed to perform request (connection failed): /127.0.0.1:11211
Apache Ignite - Logs:
[SnapshotPartitionsVerifyTask$VisorVerifySnapshotPartitionsJob] Verify snapshot partitions procedure has been initiated [snpName=20230412_snapshot_test_v1, consId=33303d92-f3db-4b33-be38-798167c2f726]
[IgniteSnapshotManager] Resolved directory for serialized binary metadata: /usr/share/apache-ignite/work/snapshots/20230412_snapshot_test_v1/db/binary_meta/node00-33303d92-f3db-4b33-be38-798167c2f726
... nothing else is printed, the service goes down ...
在快照验证失败后(如上所述的连接错误)，Apache Ignite服务立即在每个Linux服务器上停止，集群关闭，一切都需要手动重新启动。
这是一个三个节点的Apache Ignite集群(每个节点都在不同的服务器上)，配置了Apache ZooKeeper，我们在Apache Ignite 2.13.0和2.14.0上也有同样的问题，在Linux上运行OpenJDK 1.8.0。
恢复过程失败的原因是什么?
执行请求失败(连接失败):/127.0.0.1:11211
看起来，您试图连接到无效的地址。集群是否在本地主机上启动?
在启动节点尝试使用配置范围内的端口，如果所有节点都在同一主机上启动，那么控制脚本的端口可以是11211,11212，…如果使用端口11211的节点宕机，那么为了连接，您应该使用--port参数显式指定端口。
似乎Apache Ignite自动集群快照过程的问题已经在最新的Apache Ignite版本(v2.15.0)中得到修复。有关更多信息，请查看官方发布说明。