我的iOS应用程序应该能够从互联网下载大量文件（可以是FTP、REST服务器或任何其他类型的源，用户可以将文件放在那里）。由于一系列优势，我实现了从Dropbox下载以及其他可能性。我正在使用同步API与服务器同步数据。
我面临的问题是：一些用户在他们的Dropbox文件夹中有11800个文件和文件夹，应用程序会逐一下载。在下载了31-32%的文件之前，一切似乎都很好，也就是大约3600到3800个文件。然后Sync API库停止调用我的回调（观察者方法）并记录消息：
[WARNING] ERR: DROPBOX_ERROR_MISCSYSTEM: cfhttpbinding.c:353: CFHTTP Read Error: (NSPOSIXErrorDomain, 2)
[WARNING] ERR: DROPBOX_ERROR_NETWORK: cfhttpbinding.c:353: CFHTTP Read Error: (kCFErrorDomainCFNetwork, -72000)
[WARNING] ERR: DROPBOX_ERROR_NETWORK: cfhttpbinding.c:353: CFHTTP Read Error: (kCFErrorDomainCFNetwork, -72000)
[WARNING] ERR: DROPBOX_ERROR_NETWORK: cfhttpbinding.c:353: CFHTTP Read Error: (kCFErrorDomainCFNetwork, -72000)
等等
在调试它时，我注意到内存使用量增长到了600（六百！！）MB。我使用的是ARC，所以理论上不可能出现内存泄漏。尽管开发人员仍然可以保留许多强引用，但这将阻止内存被释放。当然，我不会忘记保留周期。我运行了"Allocations"工具，使用了xCode内置的Analyser工具，但仍然无法找到并解决这个问题。对我来说，内存使用没有明显的问题。
这是代码：
- (void) syncImages
{
if([NSThread isMainThread])
{
[self performSelectorInBackground:_cmd withObject:nil];
return;
}
DBPath *path = [[DBPath root] childPath:[SettingsManager dropboxImagesPath]];
DropboxManagerMetadata *metadata = [DropboxManagerMetadata new];
metadata.totalAmount = &m_uTotalAmountOfImages;
metadata.currentAmount = &m_uCurrentImage;
metadata.updateFiles = m_lstImageUpdateFiles;
metadata.continueBlock = ^(uint uTotalAmount, uint uCurrentAmount)
{
BOOL bContinue = !m_bImagesDownloadPaused;
return bContinue;
};
metadata.shouldProcessFileBlock = ^(NSString * const strRemotePath)
{
NSArray *lstPathComponents = [strRemotePath pathComponents];
// at first check if new file is related to images at all
// if we have path "/items/some_image.jpg, then we'll have the following path components
// 1. "/"
// 2. "items"
// 3. "some_image.jpg"
NSString *strRemoteFirstPathComponent = [lstPathComponents objectAtIndex:1];
BOOL bShouldProcess = [strRemoteFirstPathComponent compare:[SettingsManager dropboxImagesPath] options:NSCaseInsensitiveSearch] == NSOrderedSame;
return bShouldProcess;
};
metadata.willDownloadFileBlock = ^(NSString * const strRemotePath)
{
NSMutableString *strLocalPath = [[NSMutableString alloc] initWithString:strRemotePath];
[strLocalPath replaceOccurrencesOfString:[SettingsManager dropboxImagesPath]
withString:[SettingsManager appItemsDefaultPath]
options:0 range:NSMakeRange(0, [strLocalPath length])];
[strLocalPath setString:[SettingsManager itemPath:strLocalPath inDirectoryOfType:DirectoryTypeCaches]];
return strLocalPath;
};
metadata.fileProcessedBlock = ^(uint uTotalAmount, uint uCurrentAmount, NSString * const strFilePath)
{
[[NSNotificationCenter defaultCenter] postNotificationName:cstrNotificationDropboxImagesDownloadProgress object:nil];
};
metadata.fileUpdatedBlock = ^(NSString * const strLocalPath)
{
NSArray *lstPathComponents = [strLocalPath pathComponents];
if([lstPathComponents count] < 2) return;
NSString *strItemNo = [lstPathComponents objectAtIndex:[lstPathComponents count] - 2];
[[NSNotificationCenter defaultCenter] postNotificationName:cstrNotificationDropboxImageUpdated object:self userInfo:
[NSDictionary dictionaryWithObject:strItemNo forKey:cstrNotificationDropboxKeyItemNo]];
};
metadata.directoryProcessedBlock = ^(DBPath * const dbPath, NSArray * const lstContents)
{
NSFileManager *fileManager = [NSFileManager defaultManager];
NSMutableString *strLocalPath = [[NSMutableString alloc] initWithString:[dbPath stringValue]];
[strLocalPath replaceOccurrencesOfString:[SettingsManager dropboxImagesPath]
withString:[SettingsManager appItemsDefaultPath]
options:0 range:NSMakeRange(0, [strLocalPath length])];
[strLocalPath setString:[SettingsManager itemPath:strLocalPath inDirectoryOfType:DirectoryTypeCaches]];
NSMutableArray *lstLocalContents = [[NSMutableArray alloc] initWithArray:[fileManager contentsOfDirectoryAtPath:strLocalPath error:nil]];
if(lstLocalContents)
{
[lstLocalContents filterUsingPredicate:[NSPredicate predicateWithFormat:@"NOT (SELF BEGINSWITH[c] %@)", [ItemImagesAccessor previewImagePrefix]]];
[lstLocalContents filterUsingPredicate:[NSPredicate predicateWithFormat:@"NOT (SELF IN %@)", lstContents]];
for(NSString *strFileName in lstLocalContents)
[self deleteOldImagesAtPath:[strLocalPath stringByAppendingPathComponent:strFileName]];
}
};
[self calculateTotalAmountFilesAtPath:path withMetadata:metadata];
[[NSNotificationCenter defaultCenter] postNotificationName:cstrNotificationDropboxImagesDownloadProgress object:nil];
[self downloadContentsOfDirectoryAtPath:path withMetadata:metadata];
}
- (void) calculateTotalAmountFilesAtPath:(DBPath * const)dbPath withMetadata:(DropboxManagerMetadata * const)metadata
{
DBFilesystem *filesystem = [DBFilesystem sharedFilesystem];
if(!filesystem) return;
if(!metadata || !dbPath) return;
NSArray *lstContents = [filesystem listFolder:dbPath error:nil];
for(DBFileInfo *info in lstContents)
{
if(metadata.shouldProcessFileBlock && !metadata.shouldProcessFileBlock([info.path stringValue])) continue;
if(metadata.totalAmount)
*(metadata.totalAmount) = *(metadata.totalAmount) + 1;
if(info.isFolder)
[self calculateTotalAmountFilesAtPath:info.path withMetadata:metadata];
}
}
- (void) downloadContentsOfDirectoryAtPath:(DBPath * const)dbPath withMetadata:(DropboxManagerMetadata * const)metadata
{
DBFilesystem *filesystem = [DBFilesystem sharedFilesystem];
if(!filesystem) return;
if(!dbPath || !metadata) return;
NSFileManager *fileManager = [NSFileManager defaultManager];
DBError *error = nil;
NSArray *lstContents = [filesystem listFolder:dbPath error:&error];
if(error)
DDLogWarn(@"error while listing contents at path: %@. code: %u, description: %@", dbPath, [error code], [error localizedDescription]);
DDLogInfo(@"%u items found in directory: %@", [lstContents count], [dbPath stringValue]);
float fProgress = 0.f;
NSMutableArray *lstContentsFromDropbox = [NSMutableArray new];
for(DBFileInfo *info in lstContents)
{
if(metadata.continueBlock && !metadata.continueBlock(*(metadata.totalAmount), *(metadata.currentAmount))) break;
if(metadata.shouldProcessFileBlock && !metadata.shouldProcessFileBlock([info.path stringValue])) continue;
if(metadata.currentAmount)
{
*(metadata.currentAmount) = *(metadata.currentAmount) + 1;
if(metadata.totalAmount)
{
fProgress = (float)*(metadata.currentAmount) / (float)*(metadata.totalAmount) * 100;
DDLogInfo(@"processing %u item of %u total amount. progress is %.2f%%", *(metadata.currentAmount), *(metadata.totalAmount), fProgress);
}
}
NSString *strLocalPath = nil;
if(metadata.willDownloadFileBlock)
strLocalPath = metadata.willDownloadFileBlock([info.path stringValue]);
else
strLocalPath = [[NSMutableString alloc] initWithString:[info.path stringValue]];
[lstContentsFromDropbox addObject:[strLocalPath lastPathComponent]];
// check if new item is directory and create directory if needed
if(info.isFolder)
{
if(![fileManager fileExistsAtPath:strLocalPath])
[fileManager createDirectoryAtPath:strLocalPath withIntermediateDirectories:YES attributes:nil error:nil];
[self downloadContentsOfDirectoryAtPath:info.path withMetadata:metadata];
continue;
}
DDLogInfo(@"going to open file at path: %@", info.path);
DBFile *file = [filesystem openFile:info.path error:&error];
if(error)
DDLogWarn(@"error while opening file at path: %@. code: %u, description: %@", info.path, [error code], [error localizedDescription]);
// not nil value for status means, that file at server has some changes
if(file.newerStatus && [fileManager fileExistsAtPath:strLocalPath])
{
// will delete all related preview images if given "strLocalPath" is an image, or simply will delete given file otherwise
[self deleteOldImagesAtPath:strLocalPath];
}
if(file.newerStatus && !file.newerStatus.cached)
{
DBFile * __weak fileWeak = file;
[metadata.updateFiles addObject:fileWeak];
[file addObserver:self block:^()
{
if(fileWeak.newerStatus.cached || !fileWeak.newerStatus)
{
DBError *err = nil;
[fileWeak update:&err];
if(err)
DDLogWarn(@"error while updating file at path: %@. code: %u, description: %@", fileWeak.info.path, [err code], [err localizedDescription]);
NSData *data = [fileWeak readData:&err];
if(err)
DDLogWarn(@"error while reading file at path: %@. code: %u, description: %@", fileWeak.info.path, [err code], [err localizedDescription]);
[fileManager createFileAtPath:strLocalPath contents:data attributes:nil];
[fileWeak removeObserver:self];
[fileWeak close];
[metadata.updateFiles removeObject:fileWeak];
if(metadata.fileUpdatedBlock)
metadata.fileUpdatedBlock(strLocalPath);
@synchronized(self)
{
if(![metadata.updateFiles count] && metadata.finishProcessBlock && metadata.isSyncFinished)
metadata.finishProcessBlock();
}
}
}];
}
else
{
NSData *data = [file readData:&error]; // get the latest data
if(error)
DDLogWarn(@"error while reading file at path: %@. code: %u, description: %@", file.info.path, [error code], [error localizedDescription]);
// and create file at appropriate directory
[fileManager createFileAtPath:strLocalPath contents:data attributes:nil];
[file close];
}
if(metadata.totalAmount && metadata.currentAmount)
metadata.fileProcessedBlock(*(metadata.totalAmount), *(metadata.currentAmount), strLocalPath);
}
if(metadata.directoryProcessedBlock)
metadata.directoryProcessedBlock(dbPath, lstContentsFromDropbox);
}
代码和使用条件说明：
方法"calculateTotalAmountOfFilesAtPath:…"one_answers"downloadContentsOfDirectoryAtPath:…"是递归的，但递归从未深入到第一级（0，1，0，exit）。例如：/items/1000300/10000300.jpg。"/items/"路径作为初始参数，"1000300/"为级别0，"1000300-jpg"为级别1。
级别1总是只包含2-3个文件。例如，目录"/items/1000300"的内容总是类似于"1000300.jpg"、"1000300a.jpg"one_answers"1000300b.jp"。主数据集在级别0接收：在初始路径有很多子目录。例如，"/items/1000001"、"/item/100002"、。。。。。。。。"/项目/1007654"。
有一个本地变量"NSMutableArray*lstContentsFromDropbox=[NSMutableArray new]；"，它为特定目录的内容保留路径。它可能会导致0级的问题（见第2点），但我试图将其从算法中排除——没有显著的改进。
该错误是否是由Sync API本身的实现引起的？有什么变通办法吗？
谢谢。
附言：我更感兴趣的是从Dropbox Sync API中找到一些关于日志消息的信息，而不是如何调试内存问题。
问题是由高内存使用率（而非泄漏）引起的。应用程序必须在一个循环中执行类似的操作，该循环有超过11000次迭代。即使在堆上分配了一些内存，仍有其他内存在堆栈上分配。此外，我们不应该忘记自动释放的对象，它们仍然存在于ARC中
我通过将循环迭代封装到@autoreleasepool块中，部分解决了我的问题。这与非arc环境相同：即使你不再需要某个对象并释放对该对象的所有引用，如果它以前是自动释放的，你仍然不知道什么时候会从内存中清除它。
无论如何，我要重新实现这个算法，因为将11000个对象加载到内存中并将它们保存在列表中以在循环中进行处理并不是一个好主意。