我很高兴地运行了这段代码:
z=lapply(filename_list, function(fname){
read.zoo(file=fname,header=TRUE,sep = ",",tz = "")
})
xts( do.call(rbind,z) )
直到脏数据在一个文件的末尾出现:
Open High Low Close Volume
2011-09-20 21:00:00 1.370105 1.370105 1.370105 1.370105 1
这是在下一个文件的开头:
Open High Low Close Volume
2011-09-20 21:00:00 1.370105 1.371045 1.369685 1.3702 2230
所以rbind.zoo
抱怨重复
我不能使用类似的东西:
y <- x[ ! duplicated( index(x) ), ]
因为它们在不同的动物园对象中,在一个列表中。而且我不能使用 aggregate
,正如这里建议的那样,因为它们是动物园对象的列表,而不是一个大的动物园对象。而且我无法获得一个大对象'cos的重复项。第22条军规。
因此,当事情变得艰难时,艰难的黑客会把一些 for 循环组合在一起(请原谅打印和停止,因为这还不是工作代码):
indexes <- do.call("c", unname(lapply(z, index)))
dups=duplicated(indexes)
if(any(dups)){
duplicate_timestamps=indexes[dups]
for(tix in 1:length(duplicate_timestamps)){
t=duplicate_timestamps[tix]
print("We have a duplicate:");print(t)
for(zix in 1:length(z)){
if(t %in% index(z[[zix]])){
print(z[[zix]][t])
if(z[[zix]][t]$Volume==1){
print("-->Deleting this one");
z[[zix]][t]=NULL #<-- PROBLEM
}
}
}
}
stop("There are duplicate bars!!")
}
我难倒的一点是将 NULL 分配给动物园行不会删除它(NextMethod("[<-")中的错误:替换长度为零)。好的,所以我做了一个过滤器复制,没有违规项目......但我被这些绊倒了:
> z[[zix]][!t,]
Error in Ops.POSIXt(t) : unary '!' not defined for "POSIXt" objects
> z[[zix]][-t,]
Error in `-.POSIXt`(t) : unary '-' is not defined for "POSIXt" objects
附言虽然非常欢迎对我的实际问题"在动物园对象列表中重复行"的高级解决方案,但这里的问题专门是关于如何在给定 POSIXt 索引对象的情况下从动物园对象中删除行。
一小部分测试数据:
list(structure(c(1.36864, 1.367045, 1.370105, 1.36928, 1.37039,
1.370105, 1.36604, 1.36676, 1.370105, 1.367065, 1.37009, 1.370105,
5498, 3244, 1), .Dim = c(3L, 5L), .Dimnames = list(NULL, c("Open",
"High", "Low", "Close", "Volume")), index = structure(c(1316512800,
1316516400, 1316520000), class = c("POSIXct", "POSIXt"), tzone = ""), class = "zoo"),
structure(c(1.370105, 1.370115, 1.36913, 1.371045, 1.37023,
1.37075, 1.369685, 1.36847, 1.367885, 1.3702, 1.36917, 1.37061,
2230, 2909, 2782), .Dim = c(3L, 5L), .Dimnames = list(NULL,
c("Open", "High", "Low", "Close", "Volume")), index = structure(c(1316520000,
1316523600, 1316527200), class = c("POSIXct", "POSIXt"), tzone = ""), class = "zoo"))
更新:感谢G. Grothendieck提供的行删除解决方案。在实际代码中,我遵循了Joshua和GSee的建议,获得了xts对象列表而不是zoo对象列表。所以我的代码变成了:
z=lapply(filename_list, function(fname){
xts(read.zoo(file=fname,header=TRUE,sep = ",",tz = ""))
})
x=do.call.rbind(z)
(顺便说一句,请注意对do.call.rbind
的呼吁。这是因为rbind.xts
有一些严重的内存问题。见 https://stackoverflow.com/a/12029366/841830)
然后我删除重复项作为后处理步骤:
dups=duplicated(index(x))
if(any(dups)){
duplicate_timestamps=index(x)[dups]
to_delete=x[ (index(x) %in% duplicate_timestamps) & x$Volume<=1]
if(nrow(to_delete)>0){
#Next line says all lines that are not in the duplicate_timestamp group
# OR are in the duplicate timestamps, but have a volume greater than 1.
print("Will delete the volume=1 entry")
x=x[ !(index(x) %in% duplicate_timestamps) | x$Volume>1]
}else{
stop("Duplicate timestamps, and we cannot easily remove them just based on low volume.")
}
}
如果z1
和z2
是您的动物园对象,则在删除z2
中的任何重复项时rbind
:
rbind( z1, z2[ ! time(z2) %in% time(z1) ] )
关于删除具有指定时间的动物园对象中的点,上面已经说明了这一点,但一般来说,如果tt
是要删除的时间向量:
z[ ! time(z) %in% tt ]
或者如果我们知道tt
中只有一个元素,那么z[ time(z) != tt ]
.
rbind.xts
将允许重复的索引值,因此如果您先转换为 XTS,它可以工作。
x <- lapply(z, as.xts)
y <- do.call(rbind, x)
# keep last value of any duplicates
y <- y[!duplicated(index(y),fromLast=TRUE),]
我认为如果你先转换为xts
,你会有更好的运气。
a <- structure(c(1.370105, 1.370105, 1.370105, 1.370105, 1), .Dim = c(1L,
5L), index = structure(1316570400, tzone = "", tclass = c("POSIXct",
"POSIXt")), .indexCLASS = c("POSIXct", "POSIXt"), tclass = c("POSIXct",
"POSIXt"), .indexTZ = "", tzone = "", .Dimnames = list(NULL,
c("Open", "High", "Low", "Close", "Volume")), class = c("xts",
"zoo"))
b <- structure(c(1.370105, 1.371045, 1.369685, 1.3702, 2230), .Dim = c(1L,
5L), index = structure(1316570400, tzone = "", tclass = c("POSIXct",
"POSIXt")), .indexCLASS = c("POSIXct", "POSIXt"), tclass = c("POSIXct",
"POSIXt"), .indexTZ = "", tzone = "", .Dimnames = list(NULL,
c("Open", "High", "Low", "Close", "Volume")), class = c("xts",
"zoo"))
(comb <- rbind(a, b))
# Open High Low Close Volume
#2011-09-20 21:00:00 1.370105 1.370105 1.370105 1.370105 1
#2011-09-20 21:00:00 1.370105 1.371045 1.369685 1.370200 2230
dupidx <- index(comb)[duplicated(index(comb))] # indexes of duplicates
tail(comb[dupidx], 1) #last duplicate
# now rbind the last duplicated row with all non-duplicated data
rbind(comb[!index(comb) %in% dupidx], tail(comb[dupidx], 1))