r-导致内存泄漏的大型空间多边形数据帧



我有一个程序,它将一个大型SpatialPolygonsDataFrame对象(1.4Gb)加载到内存中,执行一些分析,然后尝试删除该对象。但是,使用系统命令free查看系统内存显示,在重置R会话之前,该对象一直保留。我可以使用rworldmaprworlxtra软件包制作一个世界地图的大列表,每个都是SpatialPolygonsDataFrame,然后尝试删除它们:

install.packages("sp")
install.packages("rworldmap")
install.packages("rworldxtra")
library(sp)
library(rworldmap)
library(rworldxtra)
these.maps.large <- lapply(1:100, function(x) assign(paste0("a_", x), getMap(resolution = "high")))
these.maps.smaller <- lapply(1:20, function(x) assign(paste0("a_", x), getMap(resolution = "high")))
# This frees the memory
rm(list="these.maps.smaller")
gc(reset=T)
# This fails to free the memory
rm(list="these.maps.large")
gc(reset=T)

编辑以下是在每个阶段之后调用system2("free")的输出。

Restarting R session...
> library(sp)
> library(rworldmap)
### Welcome to rworldmap ###
For a short introduction type :      vignette('rworldmap')
> library(rworldxtra)
> system2("free")
              total        used        free      shared  buff/cache   available
Mem:      131987656     1386468   118712292      540008    11888896   129731040
Swap:       4194300     3505464      688836
> these.maps.large <- lapply(1:100, function(x) assign(paste0("a_", x), getMap(resolution = "high")))
> system2("free")
              total        used        free      shared  buff/cache   available
Mem:      131987656     2708040   117390660      540008    11888956   128409404
Swap:       4194300     3505464      688836
> rm(list="these.maps.large")
> gc(reset=T)
         used (Mb) gc trigger  (Mb) max used (Mb)
Ncells 585803 31.3    9601876 512.8   585803 31.3
Vcells 711198  5.5   96623732 737.2   711198  5.5
> system2("free")
              total        used        free      shared  buff/cache   available
Mem:      131987656     2708428   117390424      540008    11888804   128409168
Swap:       4194300     3505464      688836
Restarting R session...
> library(sp)
> library(rworldmap)
### Welcome to rworldmap ###
For a short introduction type :      vignette('rworldmap')
> library(rworldxtra)
> system2("free")
              total        used        free      shared  buff/cache   available
Mem:      131987656     1386696   118711988      540008    11888972   129730744
Swap:       4194300     3505464      688836
> these.maps.smaller <- lapply(1:20, function(x) assign(paste0("a_", x), getMap(resolution = "high")))
> system2("free")
              total        used        free      shared  buff/cache   available
Mem:      131987656     1699628   118399100      540008    11888928   129417836
Swap:       4194300     3505464      688836
> rm(list="these.maps.smaller")
> gc(reset=T)
         used (Mb) gc trigger  (Mb) max used (Mb)
Ncells 702817 37.6    2564361 137.0   702817 37.6
Vcells 966452  7.4   21638748 165.1   966452  7.4
> system2("free")
              total        used        free      shared  buff/cache   available
Mem:      131987656     1699612   118399116      540008    11888928   129417852
Swap:       4194300     3505464      688836

有人知道为什么会出现这种情况吗?有人知道在不重置会话的情况下删除这些大型sp对象的方法吗?


R版本3.2.3(2015-12-10)平台:x86_64-redhat-linux-gnu(64位)运行于:Scientific Linux 7.2(氮气)

您不显示输出。我看到了:

> rm(list="these.maps.smaller")
> gc(reset=T)
            used  (Mb) gc trigger   (Mb)  max used  (Mb)
Ncells   7782803 415.7   14442815  771.4   7782803 415.7
Vcells 113371012 865.0  184235296 1405.7 113371012 865.0
> # This fails to free the memory
> rm(list="these.maps.large")
> gc(reset=T)
         used (Mb) gc trigger   (Mb) max used (Mb)
Ncells 524121   28   11554252  617.1   524121   28
Vcells 649283    5  147388236 1124.5   649283    5

这表明移除这些大的映射释放了大部分内存;该值几乎等于加载包之后在新会话中获得的值。

最新更新