Change dataframe value to NaN in Julia

Question

I have a dataset that uses ".." instead of NaN and I am currently trying to convert those strings into NaN. I have tried the following:

for i in 1:length(final_df[:,1])
    for j in 1:length(final_df[1,:])
        if final_df[i,j] == ".."
            final_df[i,j] = NaN
        end 
    end 
end

However I keep getting the following error: MethodError: Cannot 'convert' an object of type Float64 to an object of type String.

Here is a portion of the dataset for reference.

Any help is appreciated, thanks!

Answer 1

An alternative is to use broadcasting like this:

julia> using DataFrames

julia> df = DataFrame(rand(["..", 5.5, -3.9], 5, 5), :auto)
5×5 DataFrame
 Row │ x1    x2    x3    x4    x5  
     │ Any   Any   Any   Any   Any 
─────┼─────────────────────────────
   1 │ 5.5   5.5   -3.9  -3.9  ..
   2 │ -3.9  -3.9  ..    ..    5.5
   3 │ ..    -3.9  ..    5.5   5.5
   4 │ ..    ..    ..    5.5   5.5
   5 │ 5.5   -3.9  -3.9  ..    5.5

julia> ifelse.(df .== "..", missing, df)
5×5 DataFrame
 Row │ x1         x2         x3         x4         x5        
     │ Float64?   Float64?   Float64?   Float64?   Float64?  
─────┼───────────────────────────────────────────────────────
   1 │       5.5        5.5       -3.9       -3.9  missing   
   2 │      -3.9       -3.9  missing    missing          5.5
   3 │ missing         -3.9  missing          5.5        5.5
   4 │ missing    missing    missing          5.5        5.5
   5 │       5.5       -3.9       -3.9  missing          5.5

(note the :auto argument in constructor that generates column names automatically)

Here I give an example with missing because, as lungben noted NaN is not used in Julia to signal missingness (but you could have used it in this code equally well).

The benefit of using broadcasting is that you do not have to think about type promotion - it will happen automatically so you should not get errors.