Ordered insertion of data with parallel processing in ruby

Question

Here's my Ruby code

The method getLikes() fetches the Facebook likes in a hash format of the user with token t

def multi
    token = ["Facebook token 1","Facebook token 2","Facebook token 3",...]
    @data = []
    @threads = []

    token.each do |t|
        @threads << Thread.new{@data << getLikes(t)}
    end

    @threads.each do |th|
        th.join
    end


    render json: @data
end

The problem is that the data that @data's contents are not in the same order because of the parralization.

To fix this, I modified the first loop to

i = 0
token.each do |t|
    @threads << Thread.new{@data[i] = getLikes(t)}
    i = i + 1
end

But then the program does not wait for all the threads to finish. I get a few null values in the @data array.

What would be a good way to go about this?

Thanks

Answer 1

The issue is that your code is not thread-safe as it uses a shared-variable among threads without using mutexes. Hashes are not thread-safe in Ruby.

The solution is to return simple values in your threads, and aggregate the results in your main code. To keep the order, simply return the token along with the value:

def multi
  token = ["Facebook token 1","Facebook token 2","Facebook token 3",...]
  @threads = []

  token.each do |t|
    @threads << Thread.new{[t, getLikes(t)]}
  end

  data = Hash[@threads.map(&:value)] # this will block the main thread

  sorted_data = []
  token.each do |t|
    sorted_data << data[t]
  end

  render json: sorted_data
end