Why does this requests spec fail intermittently, or how can I debug it?

Question

This is the test that fails intermittently:

  context "as a regular_user" do
    before :each do
      @user = FactoryGirl.create(:user, :role => 'regular_user')
      visit new_user_session_path unless current_path == new_user_session_path
      fill_in "Email", :with => @user.email
      fill_in "Password", :with => @user.password
      click_button "Sign In"
    end

    it "removes thing from favorites while on thing index page" do
      @things = FactoryGirl.create_list(:thing, 5)
      @user.set_mark :favorite, @things.first
      @user.reload.favorite_things
      visit things_path unless current_path == things_path
      expect{
        first(:link, "Remove from favorites").click # Sometimes this fails
        @user.reload.favorite_things
      }.to change { @user.favorite_things.count }.by(-1)
      page.should have_content("Sort by")
    end
  end

I'm using the gem markable to provide the favorites feature.

There doesn't seem to be any pattern to the success/failure when I run the specs with rspec (it's not like once it fails it always fails, or vice versa). There might be a pattern when the specs are run through guard (spork is also being used) in that if I start guard and the test passes then it will always pass while that instance of guard is running ... Sames goes for failures - if I start guard and it fails, then all subsequent runs within that instance of guard will also fail.

Just now I ran rspec spec/requests/users_spec.rb:148 to run that one spec, and it failed:

1) Users as a regular_user removes thing from favorites while on thing index page
     Failure/Error: click_link "Remove from favorites" # Sometimes this fails
     Capybara::ElementNotFound:
       Unable to find link "Remove from favorites"
     # ./spec/requests/users_spec.rb:155:in `block (4 levels) in <top (required)>'
     # ./spec/requests/users_spec.rb:153:in `block (3 levels) in <top (required)>'

I ran rspec spec/requests/users_spec.rb:148 again right after that, and it succeeded:

Finished in 0.83754 seconds
1 example, 0 failures

I've tried adding "asdfasdf" within the <% @things.each do |thing| %> section of the index.html.erb view file, and checking for that with page.should have_content("asdfasdf") and this fails sometimes. I tried adding page.should have_selector('.favoritesblock', visible: true) just before the expect block (favoritesblock exists within the same @things block in the view), and this fails sometimes. It never fails to find text outside of the <% @things.each do |thing| %> loop, however.

I added save_and_open_page just before the first(:link, "Remove from favorites").click # Sometimes this fails line, and was able to produce an instance where the spec failed. The page only contained two things (my FactoryGirl call should be creating 5) and neither of them had a 'Remove from favorites' link (depending on if the thing is favorited or not, it displays "Add to favorites" or "Remove from favorites").

Based on that, I tried modifying the first two lines of the spec a bit, but it didn't help (still fails sometimes):

5.times { FactoryGirl.create(:thing) }
@user.set_mark :favorite, Thing.first

What I did notice is that when it succeeds, it does display the 'Remove from favorites' link for the first thing but it doesn't always display all 5 things.

So, I need to know why this spec fails intermittently or what else I can do to figure out what's causing the problem. Any ideas?

Answer 1

The way that I set up my Factory is the cause of the intermittent rspec failures.

The item has a boolean field, active, which either displays the thing on the site (if it's true) or does not display the thing on the site (if it's false).

I was clever with my Factory, and set that field as f.active { [true, false].sample }. So, sometimes the Factory creates active things and sometimes it creates inactive things. I purposefully created my Factories to behave this way because I want the default Factory to create as wide a selection of possible real-world things as possible. I just need to keep in mind that some aspects need to be manually set sometimes - like in this case, I need to create only active things.

The fix is to change my spec to only create active things:

@things = FactoryGirl.create_list(:thing, 5, :active => true)