End-to-end testing for API only Ruby on Rails apps with Cucumber syntax

In an earlier project, we had good test coverage with only unit tests produced following TDD practices. But despite our careful work, one unfortunate day, we broke a core functionanlity in production, even though our test suite was fully green. It turned out that the units under test encountered difficulties when working together outside the safe and isolated testing environment.

This experience taught me the importance of integration/acceptance tests. Following that event and after watching an interesting talk by Ian Cooper, I developed serious doubts about deep isolation for the system under test.

The following is my modest experience with testing API-only Rails apps.

My deepest desire

The current project I’m working on has tests in both RSpec and Minitest. They are good tools for unit testing, but as we move towards the integration level, I find them less convenient.

I have a deep desire to write end-to-end tests like this RSpec example:

describe 'User pays via debit card' do
  context 'when the transaction succeeds' do
    it 'creates a proper payment record'
    it 'stores the response of the payment gateway'
    it 'sends a payment.created metric'
    it 'registers the payment in XYZ service'
    ...
  end
end

This would serve as documentation and also verify that the happy path works for a feature.

However, one problem arises: each it block would run the entire scenario to test just one thing, which is not very performant. One solution could be to merge all the examples into one, but then we would lose the titles of the individual examples. We already have enough examples like it 'works' do followed by pages of less verbose expectations…

And what about testing intermediate states between bigger steps? We’ll come back to that later.

What? Did someone yell Cucumber from the back seat?

Cucumber and the gherkin syntax

As we explore behavior testing, it’s worth investigating tools like Cucumber.

But first, let’s enhance our previous payment scenario with a 3D secure 1.0 requirement. It adds an extra security layer to online card payments where the user has to prove their identity by entering a pin code received via SMS. This step usually happens on the website of the payment gateway, so we can think of it as an additional intermediate step.

With this in mind, I would write the previous test in Cucumber as follows. Note that this is only the list of steps; the step implementation would go into a different file.

  Scenario: Successful payment
    When the user starts a new payment process
      Then create a pending payment record
      And create a pending payment gateway request record
      And redirect the user to the offsite payment page
    When the offsite payment page redirects the user to our endpoint
      Then the user arrives at the successful page
    When we receive the payment callback
      Then update the payment record
      And store the response of the payment gateway
      And send a payment.created metric
      And register the payment in XYZ service
      And ...

The expressiveness of this format speaks for itself.

I must confess that we attempted to integrate Cucumber into our Rails API-only project, but we encountered some unexpected challenges, including mocking and VCR. As a result, we put the project on hold until we have a better understanding of how to handle them properly.

However, a few days ago, I stumbled upon the Turnip gem, which proudly advertises itself as a Gherkin extension for RSpec. It allows us to write Cucumber-like features in RSpec. This piqued my interest.

The Turnip gem

Driven by curiosity, I decided to give it a try, and what I discovered was even more fascinating.

A Turnip scenario is essentially a lightweight wrapper around an RSpec it block, which means the entire scenario runs within the scope of a single it block, offering all the advantages it provides.

For example, we can separate the expectation from the exercise when we expect to send a message to a class.

step 'the user requests a new payment' do
  allow(XYZService).to receive(:send_payment_created_event)

  post '/api/payments'
end

step 'register the payment in XYZ service' do
  expect(XYZService).to have_received(:send_payment_created_event)
end

Basic Turnip setup

By default, the request test helpers like post 'url' are not available in the step definitions (feature type tests). Fortunately, we can include the necessary mixin:

# spec/turnip_helper.rb
Dir[Rails.root.join('spec/acceptance/step_definitions/**/*.rb')].each { |f| require f }

RSpec.configure do |config|
  config.include RSpec::Rails::RequestExampleGroup, type: :feature
end

The feature files are similar to those used in Cucumber:

# spec/acceptance/user_pays_with_debit_card.feature

Feature: User pays with debit card
  Background:
    Given a user with an active debit card

  Scenario: Successful payment
    When the user initiates a new payment

The step definitions differ slightly:

# spec/acceptance/step_definitions/user_pays_with_debit_card_steps.rb
steps_for :user_pays_with_debit_card_steps do
  attr_reader :user

  step 'a user with an active debit card' do
    @user = create(:user)
    create(:debit_card, user: user)
  end

  step 'the user initiates a new payment' do
    request_params = {
      amount: '100'
    }

    headers = auth_headers_for(user)

    post '/api/user/payments', request_params, headers
  end
end

Exactly what we needed to continue with our end-to-end testing project.

End-to-end tests vs. Unit tests

In general, it is advised to avoid abusing end-to-end tests, because they take significantly more time than their unit level associates; the GUI tends to change frequently, resulting in broken tests, and so on.

However, in the case of an API-only app, we have a unique situation:

• The API changes much less than the GUI.
• We don’t need to render any HTML, except maybe in the case of emails.
• There is no HTML response, so there’s no need to emulate a browser.
• RSpec request tests (cover for Rails integration tests) only add a small overhead.

Our project has many unit tests and they run quite fast but our developers have had limited success in reading the tests and understanding how the application was supposed to behave on the business level.
This was the main reason that drag me toward end-to-end testing.

Case study

I built a fully-featured, detailed end-to-end Turnip test (similar to the example above) for a user successfully paying back their loan with a debit card applying 3D Secure authorization. It can be red and understood by business people as well. And it runs in under 400 milliseconds on my development laptop.

I’m very much willing to sacrifice this 400ms to have expressive and well-maintained documentation, and to ensure that the happy path works for a feature that contributes to our business’s profitability.

Closing thoughts

Our project had a tragic lack of end-to-end tests, but now we can cover our business-critical flows with high-level tests. This not only documents the functionality for both business and developers but also gives our team more confidence in deploying new changes to production and it reduces the cycle time for trying out new ideas.

I bet business people appreciate this last benefit, even if they have a low opinion of technical aspects like testing.

I hope it sparks curiosity and encourages further exploration into end-to-end testing strategies for your own projects.

Happy testing!