Capybara + remote form request

Question

I have a form that I'm testing using Capybara. This form's URL goes to my Braintree sandbox, although I suspect the problem would happen for any remote URL. When Capybara clicks the submit button for the form, the request is routed to the dummy application rather than the remote service.

Here's an example app that reproduces this issue: https://github.com/radar/capybara_remote. Run bundle exec ruby test/form_test.rb and the test will pass, which is not what I'd typically expect.

Why does this happen and is this behaviour that I can rely on always happening?

Answer 1

Mario Visic points out this description in the Capybara documentation:

Furthermore, you cannot use the RackTest driver to test a remote application, or to access remote URLs (e.g., redirects to external sites, external APIs, or OAuth services) that your application might interact with.

But I wanted to know why, so I source dived. Here's my findings:

lib/capybara/node/actions.rb

def click_button(locator)
  find(:button, locator).click
end

I don't care about the find here because that's working. It's the click that's more interesting. That method is defined like this:

lib/capybara/node/element.rb

def click
  wait_until { base.click }
end

I don't know what base is, but I see the method is defined twice more in lib/capybara/rack_test/node.rb and lib/capybara/selenium/node.rb. The tests are using Rack::Test and not Selenium, so it's probably the former:

lib/capybara/rack_test/node.rb

def click
  if tag_name == 'a'
    method = self["data-method"] if driver.options[:respect_data_method]
    method ||= :get
    driver.follow(method, self[:href].to_s)
  elsif (tag_name == 'input' and %w(submit image).include?(type)) or
    ((tag_name == 'button') and type.nil? or type == "submit")
    Capybara::RackTest::Form.new(driver, form).submit(self)
  end
end

The tag_name is probably not a link -- because it's a button we're clicking -- so it falls to the elsif. It's definitely an input tag with type == "submit", so then let's see what Capybara::RackTest::Form does:

lib/capybara/rack_test/form.rb

def submit(button)
  driver.submit(method, native['action'].to_s, params(button))
end

Ok then. driver is probably the Rack::Test driver for Capybara. What's that doing?

lib/capybara/rack_test/driver.rb

def submit(method, path, attributes)
  browser.submit(method, path, attributes)
end

What is this mysterious browser? It's defined in the same file thankfully:

def browser
  @browser ||= Capybara::RackTest::Browser.new(self)
end

Let's look at what this class's submit method does.

lib/capybara/rack_test/browser.rb

def submit(method, path, attributes)
  path = request_path if not path or path.empty?
  process_and_follow_redirects(method, path, attributes, {'HTTP_REFERER' => current_url})
end

process_and_follow_redirects does what it says on the box:

def process_and_follow_redirects(method, path, attributes = {}, env = {})
  process(method, path, attributes, env)
  5.times do
    process(:get, last_response["Location"], {}, env) if last_response.redirect?
  end
  raise Capybara::InfiniteRedirectError, "redirected more than 5 times, check for infinite redirects." if last_response.redirect?
end

So does process:

def process(method, path, attributes = {}, env = {})
  new_uri = URI.parse(path)
  method.downcase! unless method.is_a? Symbol

  if new_uri.host
    @current_host = "#{new_uri.scheme}://#{new_uri.host}"
    @current_host << ":#{new_uri.port}" if new_uri.port != new_uri.default_port
  end

  if new_uri.relative?
    if path.start_with?('?')
      path = request_path + path
    elsif not path.start_with?('/')
      path = request_path.sub(%r(/[^/]*$), '/') + path
    end
    path = current_host + path
  end

  reset_cache!
  send(method, path, attributes, env.merge(options[:headers] || {}))
end

Time to break out the debugger and see what method is here. Sticking a binding.pry before the final line in that method, and a require 'pry' in the test. It turns out method is :post and, for interest's sake, new_uri is a URI object with our remote form's URL.

Where's this post method coming from? method(:post).source_location tells me:

["/Users/ryan/.rbenv/versions/1.9.3-p374/lib/ruby/1.9.1/forwardable.rb", 199]

That doesn't seem right... Does Capybara have a def post somewhere?

capybara (master)★ack "def post"
lib/capybara/rack_test/driver.rb
76:  def post(*args, &block); browser.post(*args, &block); end

Cool. We know that browser is aCapybara::RackTest::Browser` object. The class beginning gives the next hint:

class Capybara::RackTest::Browser
  include ::Rack::Test::Methods

I know that Rack::Test::Methods comes with a post method. Time to dive into that gem.

lib/rack/test.rb

def post(uri, params = {}, env = {}, &block)
  env = env_for(uri, env.merge(:method => "POST", :params => params))
  process_request(uri, env, &block)
end

Ignoring env_for for the time being, what does process_request do?

lib/rack/test.rb

def process_request(uri, env)
  uri = URI.parse(uri)
  uri.host ||= @default_host

  @rack_mock_session.request(uri, env)

  if retry_with_digest_auth?(env)
    auth_env = env.merge({
      "HTTP_AUTHORIZATION"          => digest_auth_header,
      "rack-test.digest_auth_retry" => true
    })
    auth_env.delete('rack.request')
    process_request(uri.path, auth_env)
  else
    yield last_response if block_given?

    last_response
  end
end

Hey, @rack_mock_session looks interesting. Where's that defined?

rack-test (master)★ack "@rack_mock_session ="
lib/rack/test.rb
40:          @rack_mock_session = mock_session
42:          @rack_mock_session = MockSession.new(mock_session)

In two places, very close to each other. What's on and around these lines?

def initialize(mock_session)
  @headers = {}

  if mock_session.is_a?(MockSession)
    @rack_mock_session = mock_session
  else
    @rack_mock_session = MockSession.new(mock_session)
  end

  @default_host = @rack_mock_session.default_host
end

Ok then, so it ensures it is a MockSession object. What's MockSession and how is its request method defined?

def request(uri, env)
  env["HTTP_COOKIE"] ||= cookie_jar.for(uri)
  @last_request = Rack::Request.new(env)
  status, headers, body = @app.call(@last_request.env)
  headers["Referer"] = env["HTTP_REFERER"] || ""

  @last_response = MockResponse.new(status, headers, body, env["rack.errors"].flush)
  body.close if body.respond_to?(:close)

  cookie_jar.merge(last_response.headers["Set-Cookie"], uri)

  @after_request.each { |hook| hook.call }

  if @last_response.respond_to?(:finish)
    @last_response.finish
  else
    @last_response
  end
end

I'm going to go right ahead here and assume @app is the Rack application stack. By calling the call method, the request is routed directly to this stack, rather going out to the world.

I conclude that this behaviour looks like its intentional and that I can indeed rely on it being that way.