split with specific comma not within quotes in python

Question

i have this string and i want to split it with ","

x = 'a, b, c , d , "x,x,2" , hi'
x.split(',')

here is my real string

x = 'Outward   ,Supply , ,Tax Invoice ,IN9195212470,31/12/2019,VPS AGRO & AUTO PVT LTD ,311954,06AAACV9344F1ZA ,"VILLAGE KHANPUR KOLIAN, N.H. 1 ",6 K.M. FRO,KURUKSHETRA   ,HARYANA ,136131,VPS AGRO & AUTO PVT LTD ,311954,"VILLAGE KHANPUR KOLIAN, N.H. 1",6 K.M. FRO,KURUKSHETRA                             ,HARYANA             ,136131,503675,SM VAL. GENUINE DIESEL ENG. OIL 1/9 L   ,27101980,360,LTR,58204.04,9,5238.36,9,5238.36,0,0,0,0,0,0,0,68680.76,                    ,                    ,                              ,          ,          ,,               ,          ,06AAACW0287A1ZR   ,VALVOLINE CUMMINS PVT LTD-AMBALA        ,"KHASHRA NO-108/1/2,                                         ",          ,AMBALA                                  ,133004,HARYANA             ,                    ,                  ,                              , ,'

it returns this result

['a','b','c','d','"x','x','2', 'hi']

But I want to have this one

['a', 'b', 'c' , 'd' , '"x,x,2"' , 'hi']

how can do this in python

Help me

Answer 1

import shlex
lexer = shlex.shlex('a, b, c , d , "x,x,2" , hi')
lexer.whitespace += ','
print(list(lexer))

Result:

['a', 'b', 'c', 'd', '"x,x,2"', 'hi']

Here’s an updated solution for updated task:

x = 'Outward   ,Supply , ,Tax Invoice ,IN9195212470,31/12/2019,VPS AGRO & AUTO PVT LTD ,311954,06AAACV9344F1ZA ,"VILLAGE KHANPUR KOLIAN, N.H. 1 ",6 K.M. FRO,KURUKSHETRA   ,HARYANA ,136131,VPS AGRO & AUTO PVT LTD ,311954,"VILLAGE KHANPUR KOLIAN, N.H. 1",6 K.M. FRO,KURUKSHETRA                             ,HARYANA             ,136131,503675,SM VAL. GENUINE DIESEL ENG. OIL 1/9 L   ,27101980,360,LTR,58204.04,9,5238.36,9,5238.36,0,0,0,0,0,0,0,68680.76,                    ,                    ,                              ,          ,          ,,               ,          ,06AAACW0287A1ZR   ,VALVOLINE CUMMINS PVT LTD-AMBALA        ,"KHASHRA NO-108/1/2,                                         ",          ,AMBALA                                  ,133004,HARYANA             ,                    ,                  ,                              , ,'


import shlex
lexer = shlex.shlex(x)
lexer.whitespace = ','
lexer.whitespace_split = True
print([cell.strip() for cell in lexer])

Result:

['Outward', 'Supply', '', 'Tax Invoice', 'IN9195212470', '31/12/2019', 'VPS AGRO & AUTO PVT LTD', '311954', '06AAACV9344F1ZA', '"VILLAGE KHANPUR KOLIAN, N.H. 1 "', '6 K.M. FRO', 'KURUKSHETRA', 'HARYANA', '136131', 'VPS AGRO & AUTO PVT LTD', '311954', '"VILLAGE KHANPUR KOLIAN, N.H. 1"', '6 K.M. FRO', 'KURUKSHETRA', 'HARYANA', '136131', '503675', 'SM VAL. GENUINE DIESEL ENG. OIL 1/9 L', '27101980', '360', 'LTR', '58204.04', '9', '5238.36', '9', '5238.36', '0', '0', '0', '0', '0', '0', '0', '68680.76', '', '', '', '', '', '', '', '06AAACW0287A1ZR', 'VALVOLINE CUMMINS PVT LTD-AMBALA', '"KHASHRA NO-108/1/2,                                         "', '', 'AMBALA', '133004', 'HARYANA', '', '', '', '']

Answer 2

A solution using only split. Please note it uses f strings (python 3.6+) but the same behaviour can still be achieved in older versions. It is possible to achieve this without using regex as follows: I will comment the code for explanation:

# First split by double quote
x = x.split('"')
final_x = []
for i in range(len(x)):
    # We know that if the list element is even then it must be outside double quotes
    if i%2 == 0:
        # Split the list by commas and strip any whitespace
        x_element = x[i].split(',')
        x_element = [el.strip() for el in x_element]
        # extend the list
        final_x.extend(x_element)
    else:
        # This is an odd element of the list, therefore inside quotation.
        # put the string back into quotations
        x_element = f'"{x[i]}"'
        #append this to the final list
        final_x.append(x_element)
# filter out any white spaces left from the various splits         
final_x = [el for el in final_x if el !=''] 

Note the difference in appending the odd list elements and extending the even. This is because you are creating a new list with the split and we want to extend our output whilst for odd elements we want to add a new element to the list, hence we append.