How to split pandas column by a delimiter and select preferred element as the replacement

Question

I have the following pandas data frame:

import pandas as pd
df = pd.DataFrame({ 'gene':["1 // foo // blabla",
                                   "2 // bar // lalala",
                                   "3 // qux // trilil",
                                   "4 // woz // hohoho"], 'cell1':[5,9,1,7], 'cell2':[12,90,13,87]})
df = source_df[["gene","cell1","cell2"]]

It looks like this:

                 gene  cell1  cell2
0  1 // foo // blabla      5     12
1  2 // bar // lalala      9     90
2  3 // qux // trilil      1     13
3  4 // woz // hohoho      7     87

What I want to get is this:

   gene    cell1  cell2
0   foo       5     12
1   bar       9     90
2   qux       1     13
3   woz       7     87

Namely select 2nd element of the splited string by // as delimiter.

The best I can do is this:

df["gene"] = df["gene"].str.split(" // ")
df
Out[17]:
               gene  cell1  cell2
0  [1, foo, blabla]      5     12
1  [2, bar, lalala]      9     90
2  [3, qux, trilil]      1     13
3  [4, woz, hohoho]      7     87

What's the right way to do it?

Answer 1

Use the vectorised str.split this will be much faster than using apply on a large dataset:

In [13]:
df['gene'] = df['gene'].str.split('//').str[1]
df

Out[13]:
   cell1  cell2   gene
0      5     12   foo 
1      9     90   bar 
2      1     13   qux 
3      7     87   woz 

Answer 2

You can use regex and strip first and last spaces by strip:

df["gene"] = df["gene"].str.extract(r"\/\/([a-z ]+)\/\/")
df["gene"] = df["gene"].str.strip()

print df
  gene  cell1  cell2
0  foo      5     12
1  bar      9     90
2  qux      1     13
3  woz      7     87

\/\/([a-z ]+)\/\/ means:

    \/ matches the character / literally
    \/ matches the character / literally
    1st Capturing group ([a-z ]+)
        [a-z ]+ match a single character present in the list below
            Quantifier: + Between one and unlimited times, as many times as possible,
            giving back as needed [greedy]
            a-z a single character in the range between a and z (case sensitive)
             the literal character  
    \/ matches the character / literally
    \/ matches the character / literally

Or regex without strip:

df["gene"] = df["gene"].str.extract(r"\/\/\s*([a-z ]+)\s\/\/")

/\/\/\s*([a-z ]+)\s\/\// means:

    \/ matches the character / literally
    \/ matches the character / literally
    \s* match any white space character [\r\n\t\f ]
        Quantifier: * Between zero and unlimited times, as many times as possible, 
        giving back as needed [greedy]
    1st Capturing group ([a-z ]+)
        [a-z ]+ match a single character present in the list below
            Quantifier: + Between one and unlimited times, as many times as possible, 
            giving back as needed [greedy]
            a-z a single character in the range between a and z (case sensitive)
            the literal character  
    \s match any white space character [\r\n\t\f ]
    \/ matches the character / literally
    \/ matches the character / literally