Extract a specific text in between 2 characters in a line from a text file using Python

Question

Here is some of the content of my log file:

2021.08.09.11.32.35.7795, General, connected?, connected, operation state, connecting, laser state, Standby, pulse picker trigger, internal, Amplifier LD current in mA, 0, PP Frequency in kHz, 10000, System Reset, 65535, Bust Mode, 1, Modelock, no, Oscillator Ready, not ready, Front-end Ready, ready, current in mA, 0, temperature in deg C, 26, power in mW, 37, Pulse Picker Frequency in kHz, 0, Laser ON Runtime in hour, 1894, Laser Head Humidity in %, 42, Laser Head Temperature in deg C, 26, Two Photon Diode in mV, 12, Laser Power in mW, 34, Error Code, 0, Laser FW Version, 7.4.6, TcpModbus protocol version, 1.1.4, state, Null, On, FALSE, Good, FALSE, Fast Blank, FALSE, Treating, FALSE, Error, FALSE, Error Detail, (No Error)
2021.08.09.11.32.45.7868, General, connected?, connected, operation state, connecting, laser state, Standby, pulse picker trigger, internal, Amplifier LD current in mA, 0, PP Frequency in kHz, 10000, System Reset, 65535, Bust Mode, 1, Modelock, no, Oscillator Ready, not ready, Front-end Ready, ready, current in mA, 0, temperature in deg C, 26, power in mW, 34, Pulse Picker Frequency in kHz, 0, Laser ON Runtime in hour, 1894, Laser Head Humidity in %, 42, Laser Head Temperature in deg C, 25, Two Photon Diode in mV, 11, Laser Power in mW, 34, Error Code, 0, Laser FW Version, 7.4.6, TcpModbus protocol version, 1.1.4, state, Null, On, FALSE, Good, FALSE, Fast Blank, FALSE, Treating, FALSE, Error, FALSE, Error Detail, (No Error)
2021.08.09.11.32.51.0936, General, connected?, connected, operation state, connecting, laser state, Standby, pulse picker trigger, internal, Amplifier LD current in mA, 0, PP Frequency in kHz, 10000, System Reset, 65535, Bust Mode, 1, Modelock, no, Oscillator Ready, not ready, Front-end Ready, ready, current in mA, 0, temperature in deg C, 26, power in mW, 37, Pulse Picker Frequency in kHz, 0, Laser ON Runtime in hour, 1894, Laser Head Humidity in %, 42, Laser Head Temperature in deg C, 26, Two Photon Diode in mV, 11, Laser Power in mW, 34, Error Code, 0, Laser FW Version, 7.4.6, TcpModbus protocol version, 1.1.4, state, Null, On, FALSE, Good, FALSE, Fast Blank, FALSE, Treating, FALSE, Error, FALSE, Error Detail, (No Error)
2021.08.09.11.32.55.8040, General, connected?, connected, operation state, connecting, laser state, Standby, pulse picker trigger, internal, Amplifier LD current in mA, 0, PP Frequency in kHz, 10000, System Reset, 65535, Bust Mode, 1, Modelock, no, Oscillator Ready, not ready, Front-end Ready, ready, current in mA, 0, temperature in deg C, 26, power in mW, 34, Pulse Picker Frequency in kHz, 0, Laser ON Runtime in hour, 1894, Laser Head Humidity in %, 41, Laser Head Temperature in deg C, 26, Two Photon Diode in mV, 11, Laser Power in mW, 34, Error Code, 0, Laser FW Version, 7.4.6, TcpModbus protocol version, 1.1.4, state, Null, On, FALSE, Good, FALSE, Fast Blank, FALSE, Treating, FALSE, Error, FALSE, Error Detail, (No Error)

I want to extract the below data into a text file, based on the above data:

Date and Time               Amplifier LD current in mA   Two Photon Diode in mV
---------------             ---------------------------- ------------------------
2021.08.09.11.32.35.7795    0                            12
2021.08.09.11.32.45.7868    0                            11
2021.08.09.11.32.51.0936    0                            11 
2021.08.09.11.32.55.8040    0                            11

Also, I need to extract other parameters as well, like:

• current in mA
• temperature in deg C
• power in mW
• Laser Head Temperature in deg C
• Laser Power in mW
• Fast Blank

There may be as many as 10,000 lines of data in the text file. I would like to know the best way to use a Python script to extract this data. Here is a portion of my code I tried:

file_path = r"Laser-20210809-113225.log" 
outFile = open("result.txt", "w")

with open(file_path, "r") as f:
    content = f.read()
    size = len(content)
    start =0
    for line in file_path:
        start = content.find(" Two Photon Diode in mV,",start)
        start = start if start != -1 else size
        end = content.find(", Laser Power in mW", start)
        end = end if end != -1 else size
        print(content[start+1:end])
        outFile.write(content[start+1:end])
        outFile.write("\n")
        start = end + 1
outFile.close()

But I got stuck, when I tried to get the Date and Time data, which is towards the start of the line.

Answer 1

Your file is in the csv format. Python has a built-in library for helping read csv's: https://docs.python.org/3/library/csv.html

I just printed out the results but of course, you could write the results to a file.

import csv

with open(file_path, "r") as f:
    reader = csv.reader(f, delimiter=',')

    for row in reader:
        print(row[0], row[11], row[39])

You can also provide column headers to avoid the ugly numbers to access the correct columns. See https://docs.python.org/3/library/csv.html#csv.DictReader for details.

If you want additional features and to do more manipulation with the data, Pandas is a great library for this type of data. https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html would help you get the data in.