Is there a way in Python to read text file token-by-token?

Question

I need to process a BIG text file that contains space-separated float numbers in ASCII representation:

1.0012 0.63 18.201 -0.7911 92.2869 ...

How do I read these numbers one-by-one (not entire file and not line-by-line) using built-in Python tools? As sample, the C source code to solve this task looks like:

float number;
FILE *f = fopen ("bigfile.txt", "rt");
while (!feof (f)) {
    fscanf (f, "%f", &number);
    /* ... processing the number here ... */
}
fclose (f);

Answer 1

If a line-by-line solution is not viable (e.g. the file is just one massive line), you can read one character at a time using read(size=1).

You can do something like this:

current = ""
with open("file.txt") as f:
    while True:
        char = f.read(1)
        if char == "":
            # Reached EOF
            break
        elif char.isdecimal():
            current += char
        else:
            num = float(current)
            # process num however you like
            current = ""