Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Split up data in CSV file and writing to a file in slices using R

I have a data in CSV file containing 956,678 rows. The following piece of code reads the file and splits the data in groups (each group having 65,000 rows and remainder rows go to last group) in R.

my_file <- read.csv("~myfile_path/file.csv")
grps <- (split(my_file, (seq(nrow(my_file))-1) %/% 65000))
for (i in grps)
{
write.csv(grps, paste("path/output_file", i, ".csv", sep=""))
}

Now, I would like to write these groups as CSV files to the disk. Can anyone suggest me how to do that?

EDIT1:

Based on the comments, I have modified the code and getting the following error:

Error in data.frame(0 = list(nih_addr_id = c(664L, 665L, 666L, 667L, : arguments imply differing number of rows: 65000, 46677

like image 702
khajlk Avatar asked Oct 15 '25 15:10

khajlk


1 Answers

Your write.csv in the loop is trying to write the list as a .csv file, rather than the dataframe element of the list.

Try:

my_file <- read.csv("~myfile_path/file.csv")
grps <- (split(my_file, (seq(nrow(my_file))-1) %/% 65000))
for (i in seq_along(grps)) {
    write.csv(grps[[i]], paste0("path/output_file", i, ".csv"))
}
like image 146
Scott Warchal Avatar answered Oct 18 '25 07:10

Scott Warchal



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!