I am using concurrend append method from the class Core in Azure to store data to Azure Data lake.Below is the code and the exception which I got.I am getting this exception rarely not always.Could anyone guide me?...
public void invoke(String value) {
BitfinexSingletonClass obj = null;
try {
obj = BitfinexSingletonClass.getInstance();
} catch (IOException e1) {
slf4jLogger.info(e1.getMessage());
}
ADLStoreClient client = obj.getADLStoreClient();
byte[] myBuffer = (value + "n").getBytes();
RequestOptions opts = new RequestOptions();
opts.retryPolicy = new ExponentialBackoffPolicy();
OperationResponse resp = new OperationResponse();
slf4jLogger.info("" + value);
slf4jLogger
.info("...............Writing.........above......BITFINEX_DSHBTC_ORDER..Data............................ToADLake............");
Core.concurrentAppend(BITFINEX_DSHBTC_ORDER, myBuffer, 0, myBuffer.length, true, client, opts, resp);
slf4jLogger.info("...............BITFINEX_DSHBTC_ORDER...Data...Successfully....written.....to...AzureDataLake............");
if (!resp.successful) {
try {
throw client.getExceptionFromResponse(resp, "BITFINEX_DSHBTC_ORDER data is not written to ADL");
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
com.microsoft.azure.datalake.store.store.Adlexception：ConcurRentAppend操作失败，例外 在5次尝试[Java.net.UnknownHostException，Java.net.inknownHostexception，Java.net.inknownowerstexception，Java.net.SockettimeTimeException，Java.net.sockettemetexception]之后，最后一次遇到了例外。 在com.microsoft.azure.datalake.store.adlstoreclient.getExceptionfromresponse(adlstoreclient.java:1124( 在co.biz.yobit.sink.yobitltcbtctickeradlsink.invoke(yobitltcbtctickeradlsink.java:41( at org.apache.flink.streaming.api.operators.streamsink.processelement(streamSink.java:38( at rog.apache.flink.streaming.runtime.io.streaminputprocessor.processinput(streaminputprocessor.java:185( at rog.apache.flink.streaming.runtime.tasks.oneinputstreamtask.run(OneIniNputStreamTask.java:63( at rog.apache.flink.streaming.runtime.tasks.streamtask.invoke(streamtask.java:261( 请访问org.apache.flink.runtime.taskmanager.task.run(task.java:665( 在java.lang.thread.run(thread.java:748( 引起：java.net.sockettimeoutexception：阅读时间 在java.net.socketinputstream.socketread0(本机方法( at Java.net.socketinputstream.socketRead(socketInputStream.java:116( 在java.net.socketinputstream.read(socketInputStream.java:171( 在java.net.socketinputstream.read(socketInputStream.java:141( 在sun.security.ssl.inputrecord.read(inputrecord.java:465( 在sun.security.ssl.inputrecord.read(inputrecord.java:503( 在sun.security.ssl.sslsocketimpl.ReadRecord(sslsocketimpl.java:973( 在sun.security.ssl.sslsocketimpl.readdatarecord(sslsocketimpl.java:930( 在sun.security.ssl.appinputstream.read(appinputstream.java:105( 在java.io.io.bufferedinputstream.fill(BufferedInputStream.java:246( at Java.io.io.bufferedInputStream.Read1(BufferedInputStream.java:286( 在java.io.io.bufferedinputstream.read(BufferedInputStream.java:345( 在sun.net.www.http.httpclient.parsehttpheader(httpclient.java:735( 在sun.net.www.http.httpclient.parsehttp(httpclient.java:678( 在sun.net.www.protocol.http.httpurlconnection.getinputstream0(httpurlConnection.java:1569( 在sun.net.www.protocol.http.httpurlconnection.getinputstream(httpurlConnection.java:1474( 在java.net.httpurlconnection.getResponsecode(httpurlConnection.java:480( 在sun.net.www.protocol.https.httpsurlconnectionimpl.getResponsecode(httpsurlconnectionimpl.java:338( 在com.microsoft.azure.datalake.store.httptransport.makesinglecall(httptransport.java:292( 在com.microsoft.azure.datalake.store.httptransport.makecall(httptransport.java:91( 在com.microsoft.azure.datalake.store.core.concurrentappend(core.java:210( 在co.biz.yobit.sink.yobitltcbtctickeradlsink.invoke(yobitltcbtctickeradlsink.java:37( ... 6更多
上述错误通常是由于运行代码和Azure Data Lake Store的主机之间不可靠的网络条件的结果。正如评论中确认的那样，主机正在跨WAN连接的地理位置运行。因此，这些错误是可以预期的，如果您看到这些错误，您应该重试
建议将Flink群集与Azure Data Lake Store在同一区域的VM上运行。在该配置中，您将看不到这些网络错误。