Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to read a 2 column file and pass each column as a paired input for GNU parallel

I have a list of paired files in a directory that I put inside a file (using paste)

cat list.files.r12.txt | head -n3
./ID1_L2_R1_001.fastq.gz    ./ID1_L2_R2_001.fastq.gz
./ID2_L3_R1_001.fastq.gz    ./ID2_L3_R2_001.fastq.gz
./ID3_L2_R1_001.fastq.gz    ./ID3_L2_R2_001.fastq.gz

You can see that the pair is between R1 and R2.

This is just for testing as this is going to be passed to another command. The command below just paste line by line.

parallel -j 1 "echo {}" ::::  <(cat list.files.r12.txt | head -n3)

This one duplicates the files

parallel -j 1 "echo {} {}" ::::  <(cat list.files.r12.txt | head -n3)

This doesn't work:

parallel -j 1 "echo {1} {2}" ::::  <(cat list.files.r12.txt | head -n3)

Basically, what I want is that each line of the list.files.r12.txt file could be executed in parallel. This is the desired output:

program ... ./ID1_L2_R1_001.fastq.gz ./ID1_L2_R2_001.fastq.gz
program ... ./ID2_L2_R1_001.fastq.gz ./ID2_L2_R2_001.fastq.gz
program ... ./ID3_L2_R1_001.fastq.gz ./ID3_L2_R2_001.fastq.gz
like image 618
M. Beausoleil Avatar asked Feb 02 '26 11:02

M. Beausoleil


1 Answers

cat list.files.r12.txt | head -n3 |
   parallel --colsep '\t' echo
cat list.files.r12.txt | head -n3 |
   parallel --colsep '\t' echo one:{1} two:{2}

When you have paired end reads you can normally do:

parallel --plus echo read:{} paired:{/_R1_/_R2_} ::: *_R1_*

This way you avoid the pasting.

like image 73
Ole Tange Avatar answered Feb 05 '26 03:02

Ole Tange