Creating a gzip file using bash script

Question

I have a bash script that produces a bunch of files. However I wanted my files to be gzip. The way I've written my script, it produces files with *.gz extension. But, when I check whether it's gzip or not using the command

gzip -l hard_0.msOut.gz 

it says gzip: hard_0.msOut.gz: not in gzip format

My bash script is:

#!/bin/bash

#generating training data

i_hard=0
i_soft=0
i_neutral=0

for entry in /home/noor/popGen/sweeps/slim_script/singlePop/*
do
    if [[ $entry == *"hard"* ]]; then
        echo "It's there!"
        /home/noor/popGen/build/./slim $entry > /home/noor/popGen/sweeps/msOut/singlePop/hard_$i_hard.msOut.gz
        i_hard=$((i_hard+1))
    fi

    if [[ $entry == *"soft"* ]]; then
        echo "It's there!"
        /home/noor/popGen/build/./slim $entry > /home/noor/popGen/sweeps/msOut/singlePop/soft_$i_soft.msOut.gz
        i_soft=$((i_soft+1))
    fi
    if [[ $entry == *"neutral"* ]]; then
        echo "It's there!"
        /home/noor/popGen/build/./slim $entry > /home/noor/popGen/sweeps/msOut/singlePop/neutral_$i_neutral.msOut.gz
        i_neutral=$((i_neutral+1))
    fi

done

Can someone tell me how can produce gzip files using the bash script that I've made.

Answer 1

You're outputting the values to a file called something.gz, but that doesn't mean it's gzipped. That just means that you've chosen to have a file extension of .gz.

To gzip the output, add the following for example:

echo "compress me" | gzip -c > file.gz

The above will take the output of the echo and pipe it to gzip (-c will send to stdout) and redirect stdout to a file named file.gz

Your complete code:

#!/bin/bash

#generating training data

i_hard=0
i_soft=0
i_neutral=0

for entry in /home/noor/popGen/sweeps/slim_script/singlePop/*
do
    if [[ $entry == *"hard"* ]]; then
        echo "It's there!"
        /home/noor/popGen/build/./slim $entry | gzip -c > /home/noor/popGen/sweeps/msOut/singlePop/hard_$i_hard.msOut.gz
        i_hard=$((i_hard+1))
    fi

    if [[ $entry == *"soft"* ]]; then
        echo "It's there!"
        /home/noor/popGen/build/./slim $entry | gzip -c > /home/noor/popGen/sweeps/msOut/singlePop/soft_$i_soft.msOut.gz
        i_soft=$((i_soft+1))
    fi
    if [[ $entry == *"neutral"* ]]; then
        echo "It's there!"
        /home/noor/popGen/build/./slim $entry | gzip -c > /home/noor/popGen/sweeps/msOut/singlePop/neutral_$i_neutral.msOut.gz
        i_neutral=$((i_neutral+1))
    fi

done