Efficiently loop through a file with a fixed length record string using Perl

Question

I have a file that has a fixed length record with no newline characters.

Example: A file with 100 characters that has a fixed length record of 25 characters. (total of 4 records)

How can I read the file per record without having to store the data in a variable. (please see example below)

open my $fh, "<", "inputfile.txt" or die "can't open file\n";

my $data = <$fh>; # I would like to avoid storing the file contents in a variable

for (my $j = 0; $j < length $data; $j += 25 ) {

    my $record = substr($data, $j, 25) # Get one record
    print "$record\n";


}

2nd option:

I can also use $_ to capture the data in . Am I doing the same thing as above in terms of consuming additional memory?

open my $fh, "<", "inputfile.txt" or die "can't open file\n";

while ( <$fh> ) {

    for (my $j = 0; $j < length $_; $j += 25 ) {

        my $record = substr($_, $j, 25) # Get one record
        print "$record\n";
    }
}

The reason that I do not wan't to store it in a variable because I am concerned that if I am dealing with a very large file, it would consume twice as space as opening the file.

Am I making the correct assumption that I would be taking twice the space in memory as I did when I opened the file?

What would be the most efficient way to read the file wihtout having to consume a lot of memory?

Please correct me if my question does not make sense.

Thanks :)

Answer 1

You can use read to read a specific number of characters from a file handle.

Attempts to read LENGTH characters of data into variable SCALAR from the specified FILEHANDLE. Returns the number of characters actually read, 0 at end of file, or undef if there was an error (in the latter case $! is also set). SCALAR will be grown or shrunk so that the last character actually read is the last character of the scalar after the read.

Here's a short example.

while (read(\*DATA, my $record, 3)) {
    print $record, "\n";
}

__DATA__
foobarbazqrr

This will output

foo
bar
baz
qrr

---

If you read the whole file (as one line) at once, the space you would take up in memory would be at the size of the entire file. It would only be double the size of reading one record at a time if the file only has two very long records.