使用共享资源网络重试上传到FTP的正确方法是什么?



尽管我使用 ftpClient.setControlKeepAliveTimeout(CONTROL_CONNECTION_KEEP_ALIVE_INTERVAL_SECONDS) 发送了保持活动请求,但我在处理服务器时遇到问题,该服务器会关闭我的 FTP 控制连接;此外,有时它只是从SocketException中消失,而不是正常的FTPConnectionClosedException。总而言之,FTP是我经常使用的非常狡猾的协议,我连接的每台服务器都需要一些调整,但是这个服务器给人带来了非常困难的时间。

我知道我可能做错了一百万件事,我的问题是,是否有一些解决方案已经在FTP上实施了重试,以防您失去控制连接,因为这不是应该令人震惊的事情(代理/防火墙有时只是随机丢失您的连接)。或者有没有一些更优雅的方法来解决这个问题。

我有这样的东西。

public void store(File fileToUpload) throws IOException, InterruptedException {
    String filename = fileToUpload.getName();
    int retries = 0;
    while (true) {
        try {
            ftpClient.storeFile(filename, inputStreamFactory.getInputStream(fileToUpload));
        } catch (FTPConnectionClosedException | SocketException exception) {
            LOGGER.debug("Control connection lost uploading {}, continuing.", filename);
        }
        // This sleep is because there's an anti-malware in the servers which makes the file not to appear
        // available immediately after an upload
        LOGGER.debug("Waiting {} milliseconds for anti-malware protection to process file", WAIT_AFTER_UPLOAD_MILLISECONDS);
        threadWrapper.sleep(WAIT_AFTER_UPLOAD_MILLISECONDS);
        if (!ftpClient.isConnected()) {
            connect();
        }
        LOGGER.debug("Checking if {} is already uploaded", filename);
        if (ftpFileChecker.isFileCompleted(listFiles(null), filename, fileToUpload.length())) {
            // Note this is likely to happen every time since their server will close the control
            // connection quite fast and FTPClient uses it at the end of storeFile
            LOGGER.debug("File {} was uploaded correctly", filename);
            break;
        } else {
            if (++retries > MAX_RETRIES) {
                throw new RemoteTimeoutException("Could not upload file, max retries exceeded");
            } else {
                LOGGER.info("File {} was not uploaded, retrying", filename);
            }
        }
    }
}
public void connect() throws IOException, InterruptedException {
    int retries = 0;
    while (true) {
        try {
            ftpClient = ftpClientFactory.createFtpClient();
            ftpClient.connect(server, FTP_PORT);
            if (!ftpClient.login(username, password)) {
                LOGGER.error("Login to FTP failed");
                throw new ConfigurationException("Login to FTP failed");
            }
            ftpClient.enterLocalPassiveMode();
            ftpClient.setControlKeepAliveTimeout(CONTROL_CONNECTION_KEEP_ALIVE_INTERVAL_SECONDS);
            ftpClient.setFileType(FTPClient.BINARY_FILE_TYPE);
            ftpClient.changeWorkingDirectory(uploadDir);
            break;
        } catch (FTPConnectionClosedException | SocketException exception) {
            if (++retries > MAX_RETRIES) {
                throw new RemoteTimeoutException("Could not upload file, max retries exceeded", exception);
            } else {
                LOGGER.info("Could not login, retrying");
            }
        }
        LOGGER.debug("Sleeping {} milliseconds before trying to reconnect", WAIT_BETWEEN_CONNECT_RETRIES_MILLISECONDS);
        threadWrapper.sleep(WAIT_BETWEEN_CONNECT_RETRIES_MILLISECONDS);
    }
}
public FTPFile[] listFiles(String directory) throws IOException, InterruptedException {
    int retries = 0;
    while (true) {
        try {
            return ftpClient.listFiles(directory);
        } catch (FTPConnectionClosedException exception) {
            LOGGER.debug("Control connection lost when listing files, continuing");
        } catch (SocketException exception) {
            LOGGER.debug("Socket exception when listing files, continuing");
        }
        if (!ftpClient.isConnected()) {
            connect();
        }
        if (++retries > MAX_RETRIES) {
            throw new RemoteTimeoutException("Could not list files, max retries exceeded");
        } else {
            LOGGER.info("Could not list files, retrying");
        }
    }
}

现在我在listFiles上得到了SocketException Broken Pipe,我再也想不通了,因为在我的本地服务器中它工作得很好,但是在这个特定的服务器中(提示它在Windows NT上运行 - :( - 它运行一些恶意软件保护,防止文件立即出现在服务器上,并且显然在一些非常奇怪的防火墙后面, 大约 5 秒后它会丢弃空闲连接,他们不会更改配置,因为他们是一家大公司并声称它适用于其他人)。

我尝试了 VFS 并调查了其他 FTP 客户端,但我发现似乎没有一个可以解决问题,甚至更无益,其中大多数(如 ftp4j)不在 maven Central 中,这真的让我无法尝试它们,除非有保证它会解决我的问题。

欢迎任何帮助。

编辑:给出的代码反映了它的开始复杂性,我当前的解决方案更稳定,复杂性更高,但它一点也不优雅,所以我把这个问题留给一个,以防有人愿意贡献一个不错的解决方案。

如果使用 Spring

,请考虑 Spring 重试。我相信最新的Maven版本是:

    <dependency>
        <groupId>org.springframework.retry</groupId>
        <artifactId>spring-retry</artifactId>
        <version>1.1.2.RELEASE</version>
    </dependency>

相关内容

  • 没有找到相关文章

最新更新