R:主题模型,2个类似的文档,代码与一个使用,与另一个相似



我运行主题模型代码时会出现一个非常奇怪的错误。基本上,我有一个带有用户注释的.CSV文件。我想创建一个DTM,每个注释是一个文档。我取了一个8K评论,并在其中使用了以下代码:

> #LOAD LIBRARYS
> 
> library(tm)
> library(SnowballC)
> library(stringr)
> library(tictoc)
> tic()
> 
> #SET FILE LOCATION
> file_loc <- "C:/Users/Andreas/Desktop/first8k.csv"
> 
> #LOAD DOCUMENTS
> Database <- read.csv(file_loc, header = FALSE)
> require(tm)
> 
> #PROCEED
> Database <- Corpus(DataframeSource(Database))
> 
> Database <-tm_map(Database,content_transformer(tolower))
> 
> 
> Database <- tm_map(Database, removePunctuation)
> Database <- tm_map(Database, removeNumbers)
> Database <- tm_map(Database, removeWords, stopwords("english"))
> Database <- tm_map(Database, stripWhitespace)
> 
> 
> myStopwords <- c("some", "individual", "stop","words")
> Database <- tm_map(Database, removeWords, myStopwords)
> 
> Database <- tm_map(Database,stemDocument) 
> 
> 
> dtm <- DocumentTermMatrix(Database,          control=list(minDocFreq=2,minWordLength=2))
> 
> row_total = apply(dtm, 1, sum)
> dtm.new = dtm[row_total>0,]
> 
> removeSparseTerms( dtm, .99)
>
>>Outcome:DocumentTermMatrix (documents: 12753, terms: 194)
>Non-/sparse entries: 66261/2407821
>Sparsity           : 97%
>Maximal term length: 11
>Weighting          : term frequency (tf)
> 
> #TOPICMODELLING
> 
> library(topicmodels)
> 
>  
> 
> burnin <- 100
> iter <- 500
> thin <- 100
> seed <-list(200,5,500,3700,1666)
> nstart <- 5
> best <- TRUE
> 
>  
> k <- 12
> 
>
> ldaOut <-LDA(dtm.new,k, method="Gibbs", control=list(nstart=nstart, seed = seed, best=best, burnin = burnin, iter = iter, thin=thin))
> 

所以这很好。如果我将另一个8K注释的示例(也包括CSV文件,相同的格式等)。发生以下错误:

> library(tm)
> library(SnowballC)
> library(stringr)
> library(tictoc)
> tic()
> 
> #SET FILE LOCATION
> file_loc <- "C:/Users/Andreas/Desktop/try8k.csv"
> 
> #LOAD DOCUMENTS
> Database <- read.csv(file_loc, header = FALSE)
> require(tm)
> 
> #PROCEED
> Database <- Corpus(DataframeSource(Database))
> 
> Database <-tm_map(Database,content_transformer(tolower))
> 
> 
> Database <- tm_map(Database, removePunctuation)
> Database <- tm_map(Database, removeNumbers)
> Database <- tm_map(Database, removeWords, stopwords("english"))
> Database <- tm_map(Database, stripWhitespace)
> 
> 
> myStopwords <- c("some", "individual", "stop","words")
> Database <- tm_map(Database, removeWords, myStopwords)
> 
> Database <- tm_map(Database,stemDocument) 
> 
> dtm <- DocumentTermMatrix(Database,control=list(minDocFreq=2,minWordLength=2))
> 
> row_total = apply(dtm, 1, sum)
> dtm.new = dtm[row_total>0,]
> 
> removeSparseTerms( dtm, .99)
>
>>Outcome:DocumentTermMatrix (documents: 9875, terms: 0)
Non-/sparse entries: 0/0
Sparsity           : 100%
Maximal term length: 0
Weighting          : term frequency (tf)
> 
> #TOPICMODELLING
> 
> library(topicmodels)
> 
>  
> 
> burnin <- 100
> iter <- 500
> thin <- 100
> seed <-list(200,5,500,3700,1666)
> nstart <- 5
> best <- TRUE
> 
>  
> k <- 12
> 
> 
> ldaOut <-LDA(dtm.new,k, method="Gibbs", control=list(nstart=nstart, seed = seed, best=best, burnin = burnin, iter = iter, thin=thin))
>Fehler in obj[[i]][[which.max(sapply(obj[[i]], logLik))]] :
>attempt to select less than one element in get1index

我猜DTM的某些东西没有扭动,因为它说有9875个文档,但根本没有术语。但是我绝对不知道为什么这些代码适用于一个样本,而不是另一个样本。请告诉我,我是否在代码上做错了什么,或者您发现其他错误。

预先感谢!

项= 0这就是为什么您有prob

最新更新