Behave: Writing a Scenario Outline with dynamic examples

Question

Gherkin / Behave Examples

Gherkin syntax features test automation using examples:

Feature: Scenario Outline (tutorial04)

  Scenario Outline: Use Blender with <thing>
    Given I put "<thing>" in a blender
    When I switch the blender on
    Then it should transform into "<other thing>"

    Examples: Amphibians
        | thing         | other thing |
        | Red Tree Frog | mush        |
        | apples        | apple juice |

    Examples: Consumer Electronics
        | thing         | other thing |
        | iPhone        | toxic waste |
        | Galaxy Nexus  | toxic waste |

The test suite would run four times, once for each example, giving a result similar to:

My problem

How can I test using confidential data in the Examples section? For example, I would like to test an internal API with user ids or SSN numbers, without keeping the data hard coded in the feature file.

Is there a way to load the Examples dynamically from an external source?

Update: Opened a github issue on the behave project.

Answer 1

I've come up with another solution (behave-1.2.6):

I managed to dynamically create examples for a Scenario Outline by using before_feature.

Given a feature file (x.feature):

Feature: Verify squared numbers

  Scenario Outline: Verify square for <number>
    Then the <number> squared is <result>

Examples: Static
  | number | result |
  |   1    |    1   | 
  |   2    |    4   |
  |   3    |    9   |
  |   4    |   16   |

  # Use the tag to mark this outline
  @dynamic
  Scenario Outline: Verify square for <number>
    Then the <number> squared is <result>

Examples: Dynamic
  | number | result |
  |   .    |    .   | 

And the steps file (steps/x.step):

from behave import step

@step('the {number:d} squared is {result:d}')
def step_impl(context, number, result):
    assert number*number == result

The trick is to use before_feature in environment.py as it has already parsed the examples tables to the scenario outlines, but hasn't generated the scenarios from the outline yet.

import behave
import copy

def before_feature(context, feature):
    features = (s for s in feature.scenarios if type(s) == behave.model.ScenarioOutline and
                'dynamic' in s.tags)
    for s in features:
        for e in s.examples:
            orig = copy.deepcopy(e.table.rows[0])
            e.table.rows = []
            for num in range(1,5):
                n = copy.deepcopy(orig)
                # This relies on knowing that the table has two rows.
                n.cells = ['{}'.format(num), '{}'.format(num*num)]
                e.table.rows.append(n)

This will only operate on Scenario Outlines that are tagged with @dynamic.

The result is:

behave -k --no-capture
Feature: Verify squared numbers # features/x.feature:1

  Scenario Outline: Verify square for 1 -- @1.1 Static  # features/x.feature:8
    Then the 1 squared is 1                             # features/steps/x.py:3

  Scenario Outline: Verify square for 2 -- @1.2 Static  # features/x.feature:9
    Then the 2 squared is 4                             # features/steps/x.py:3

  Scenario Outline: Verify square for 3 -- @1.3 Static  # features/x.feature:10
    Then the 3 squared is 9                             # features/steps/x.py:3

  Scenario Outline: Verify square for 4 -- @1.4 Static  # features/x.feature:11
    Then the 4 squared is 16                            # features/steps/x.py:3

  @dynamic
  Scenario Outline: Verify square for 1 -- @1.1 Dynamic  # features/x.feature:19
    Then the 1 squared is 1                              # features/steps/x.py:3

  @dynamic
  Scenario Outline: Verify square for 2 -- @1.2 Dynamic  # features/x.feature:19
    Then the 2 squared is 4                              # features/steps/x.py:3

  @dynamic
  Scenario Outline: Verify square for 3 -- @1.3 Dynamic  # features/x.feature:19
    Then the 3 squared is 9                              # features/steps/x.py:3

  @dynamic
  Scenario Outline: Verify square for 4 -- @1.4 Dynamic  # features/x.feature:19
    Then the 4 squared is 16                             # features/steps/x.py:3

1 feature passed, 0 failed, 0 skipped
8 scenarios passed, 0 failed, 0 skipped
8 steps passed, 0 failed, 0 skipped, 0 undefined
Took 0m0.005s

This relies on having an Examples table with the correct shape as the final table, in my example, with two rows. I also don't fuss with creating new behave.model.Row objects, I just copy the one from the table and update it. For extra ugliness, if you're using a file, you can put the file name in the Examples table.