我已经有了这个工作，但希望优化它。提取与此相关的文章数据需要很长时间，因为我的方法使用的是 for 循环。我需要逐行进行，运行每一行需要一秒钟多一点的时间。但是，在我的实际数据集中，我有大约 10,000 行，这需要很长时间。除了 for 循环之外，有没有办法提取全文？我对每一行都使用相同的方法，所以我想知道 R 中是否有类似于将列乘以一个超级快的数字的函数。
创建虚拟数据集：
date<- as.Date(c('2020-06-25', '2020-06-25','2020-06-25','2020-06-25','2020-06-25','2020-06-25','2020-06-25','2020-06-25','2020-06-25','2020-06-25'))
text <- c('Isko cites importance of wearing face mask, gives 10K pieces to 30 barangays',
'GMRC now a law; to be integrated in school curriculum',
'QC to impose stringent measures to screen applicants for PWD ID',
'‘Baka kalaban ka:’ Cops intimidate dzBB reporter',
'Is gov’t playing with traditional jeepney drivers? A lawmaker thinks so',
'PNP records highest single-day COVID-19 tally as cases rise to 579',
'IBP tells new lawyers: ‘Excel without sacrificing honor’',
'Senators express concern over DepEd’s preparedness for upcoming school year',
'Angara calls for probe into reported spread of ‘fake’ PWD IDs',
'Grab PH eyes new scheme to protect food couriers vs no-show customers')
link<- c('https://newsinfo.inquirer.net/1297621/isko-cites-importance-of-wearing-face-mask-gives-10k-pieces-to-30-barangays',
'https://newsinfo.inquirer.net/1297618/gmrc-now-a-law-to-be-integrated-in-school-curriculum',
'https://newsinfo.inquirer.net/1297614/qc-to-impose-stringent-measures-to-screen-applicants-for-pwd-id',
'https://newsinfo.inquirer.net/1297606/baka-kalaban-ka-cops-intimidate-dzbb-reporter',
'https://newsinfo.inquirer.net/1297582/is-govt-playing-with-traditional-jeepney-drivers-a-party-list-lawmaker-thinks-so',
'https://newsinfo.inquirer.net/1297577/pnp-records-highest-single-day-covid-19-tally-as-cases-rose-to-579',
'https://newsinfo.inquirer.net/1297562/ibp-tells-new-lawyers-excel-without-sacrificing-honor',
'https://newsinfo.inquirer.net/1297559/senators-on-depeds-preparedness-for-upcoming-school-year',
'https://newsinfo.inquirer.net/1297566/angara-calls-for-probe-into-reported-spread-of-fake-pwd-ids',
'https://newsinfo.inquirer.net/1297553/grab-ph-eyes-new-scheme-to-protect-food-couriers-vs-no-show-customers')
df<-data.frame(date, text, link)
虚拟数据集：
df
date text link
1 2020-06-25 Isko cites importance of wearing face mask, gives 10K pieces to 30 barangays https://newsinfo.inquirer.net/1297621/isko-cites-importance-of-wearing-face-mask-gives-10k-pieces-to-30-barangays
2 2020-06-25 GMRC now a law; to be integrated in school curriculum https://newsinfo.inquirer.net/1297618/gmrc-now-a-law-to-be-integrated-in-school-curriculum
3 2020-06-25 QC to impose stringent measures to screen applicants for PWD ID https://newsinfo.inquirer.net/1297614/qc-to-impose-stringent-measures-to-screen-applicants-for-pwd-id
4 2020-06-25 ‘Baka kalaban ka:’ Cops intimidate dzBB reporter https://newsinfo.inquirer.net/1297606/baka-kalaban-ka-cops-intimidate-dzbb-reporter
5 2020-06-25 Is gov’t playing with traditional jeepney drivers? A lawmaker thinks so https://newsinfo.inquirer.net/1297582/is-govt-playing-with-traditional-jeepney-drivers-a-party-list-lawmaker-thinks-so
6 2020-06-25 PNP records highest single-day COVID-19 tally as cases rise to 579 https://newsinfo.inquirer.net/1297577/pnp-records-highest-single-day-covid-19-tally-as-cases-rose-to-579
7 2020-06-25 IBP tells new lawyers: ‘Excel without sacrificing honor’ https://newsinfo.inquirer.net/1297562/ibp-tells-new-lawyers-excel-without-sacrificing-honor
8 2020-06-25 Senators express concern over DepEd’s preparedness for upcoming school year https://newsinfo.inquirer.net/1297559/senators-on-depeds-preparedness-for-upcoming-school-year
9 2020-06-25 Angara calls for probe into reported spread of ‘fake’ PWD IDs https://newsinfo.inquirer.net/1297566/angara-calls-for-probe-into-reported-spread-of-fake-pwd-ids
10 2020-06-25 Grab PH eyes new scheme to protect food couriers vs no-show customers https://newsinfo.inquirer.net/1297553/grab-ph-eyes-new-scheme-to-protect-food-couriers-vs-no-show-customers
用于获取每个链接的文章数据的代码：
now<-Sys.time()
for(i in 1:nrow(df)) {
test_article<- read_html(df[i, 3]) %>%
html_nodes(".article_align div p") %>%
html_text() %>%
toString()
text_df <- tibble(test_article)
df[i,4]<-test_article
print(paste(i,"/",nrow(df), sep = ""))
}
finish<-Sys.time()
finish-now
所以仅仅10篇文章，就花了10秒钟，我觉得这真的很长。看看是否有更快的方法来做到这一点。
您可以并行化循环：
#setup parallel backend to use many processors
cores=detectCores()
cl <- makeCluster(cores[1]-1) #not to overload your computer
registerDoParallel(cl)
now <- Sys.time()
result <- foreach(i =1:nrow(df),.combine=rbind,.packages=('dplyr','rvest') %dopar% {
test_article <- read_html(df[i, 3]) %>%
html_nodes(".article_align div p") %>%
html_text() %>%
toString()
data.frame( test_article = test_article, ID = paste(i,"-",nrow(df), sep = ""))
}
finish<-Sys.time()
finish-now
#stop cluster
stopCluster(cl)
请注意，不能从 foreach 循环内部写入原始数据帧，因为每个任务都在单独的环境中运行。