BDD Automation with Robot Framework in Python

It aims to achieve the deliverable as per the requirements defined by business or product owners.

I define the business requirements as user stories with acceptance criteria. Clarifying the acceptance criteria among all stakeholders is pivotal for the success of a project. What if the acceptance criteria was not clear to our stakeholders? I can’t imagine the consequences of proceeding with the development without clarifying the acceptance criteria.

That’s where the evaluation of Behavior Driven Development (BDD) comes into existence. BDD is an extension of Test Driven Development (TDD). This involves a collaborative response from TDD for hurdles in maintaining stakeholder’s expectation and programmer’s approach for testing and coding.

Whereas BDD helps programmers to articulate their approach for testing and coding. I will be explaining more about developing a BDD solution with automated testing.

Why BDD?

BDD can enhance the agile process by involving all stakeholders in a very descriptive way, as BDD places a large emphasis on stakeholder’s conversation. It is the best practice to have these conversations closer to the moment when you plan to actually write test cases and code.

BDD seeks to qualify your acceptance criteria by having discussion around concrete examples. Concrete examples such as User stories should be detailed and full of context. Discovering the expectations of stakeholders and how they will be using the application in every possible situation.

I will be defining the acceptance criteria by describing scenarios drives the BDD development. This will help to understand the behavior of the required system under a specific initial condition.

Demo

In order to demonstrate the BDD, I would like to consider the TodoMVC application. I can list down different scenarios as described by TodoMVC and given-when-then scenarios.

Scenario: Empty list can have item added

Given an empty Todo list

When I add a Todo for ‘Buy cheese’

Then only that item is listed

And the list summary is ‘1 item left’

And the list’s filter is set to ‘All’ with ‘Completed’ & ‘Active’ unset

And ‘Clear completed’ is unavailable

Automation

Defined scenario clarifies the requirement in an well-descriptive manner. Let’s automate the scenario using Robotframework.

What do you need?

        $ brew install chromedriver --cask

2. Install Robot framework

        $ pip install robotframework

Writing test scenarios

Robot Framework provides some blocks of specification where we configure test prerequisites and test case definitions.

To configure the test automation, you may create a file “test_todo.robot” and follow the below snippets in a sequential manner.

First, we need to configure the settings section as mentioned below.

*** Settings ***
Library      SeleniumLibrary
Test Setup       Open TodoMVC Page Using Chrome Browser
Test Teardown    Close Browser

Settings section contains import statements. Additionally, we defines the Test Setup and Test Teardown with predefined commands from SeleniumLibrary or custom keywords. In our case, we want to open the TodoMVC page in chrome browser.

*** Variables ***
${MAIN_PAGE}               https://todomvc.com/examples/react/#/
${TODO_INPUT_FIELD}        //input[@class='new-todo']
${FILTER_ELEMENT}          //ul[@class='filters']/li/a

We defines few variables which will be referred at different places. Now let's create our custom keywords implementation.

*** Keywords ***
Open TodoMVC Page Using Chrome Browser
 Open Browser    ${MAIN_PAGE}    Chrome
 Maximize Browser Window

We defined the custom keyword as Open TodoMVC Page Using Chrome Browser which is responsible to perform the required actions to open the chrome browser with given URL and maximize the browser window. Similarly, we will be defining other keyword sets as per our need.

*** Keywords ***
Open TodoMVC Page Using Chrome Browser
 Open Browser    ${MAIN_PAGE}    Chrome
 Maximize Browser Window

Open TodoMVC Page Using Chrome Browser
 Open Browser    ${MAIN_PAGE}    Chrome
 Maximize Browser Window

An Empty Todo List
   Wait Until Element Is Visible         ${TODO_INPUT_FIELD}
   Wait Until Element Is Not Visible     //section[@class='main']

I add a todo for
 [Arguments]                         ${todo_item}
 Wait Until Element Is Enabled       ${TODO_INPUT_FIELD}
 Input Text                          ${TODO_INPUT_FIELD}       ${todo_item}
 Press Keys                          ${TODO_INPUT_FIELD}       RETURN

Only that item is listed
 [Arguments]                         ${todo_item}
 Wait Until Element Is Visible       //label[text()='${todo_item}']

The list summary is
 [Arguments]                         ${item_left}
 Wait Until Element Is Visible       //footer//span[contains(., '${item_left}')]

The filter is set to
 [Arguments]                         ${filter_name}
 Click Link                          ${FILTER_ELEMENT} [text()='${filter_name}']
 Wait until page contains element    ${FILTER_ELEMENT} [text()='${filter_name}'][contains(@class, 'selected')]

The filter is unset to
 [Arguments]                         ${filter_name}
 Wait until page contains element    ${FILTER_ELEMENT} [text()='${filter_name}'][contains(@class, '')]

Clear completed is unavailable
 Wait Until Element Is Not Visible     //button[text()='Clear completed']

Some keywords are defined with arguments signature. Arguments will be passed at the time of calling these keywords. Finally, we are going to call these keywords in our test scenario.

*** Test Cases ***
Empty list can have item added
 GIVEN An empty Todo list
 WHEN I add a todo for           Buy cheese
 THEN Only that item is listed   Buy cheese
 AND The list summary is         1 item left
 AND The filter is set to        All
 AND The filter is unset to      Completed
 AND The filter is unset to      Active
 AND Clear completed is unavailable

This is how our test case scenario is defined where we are calling all the custom keywords with required argument values. That's it, we are set to execute the test suite.

Running test

We can execute the test from command prompt using the robot command as mentioned below. Argument -d indicate the output directory to save the test results.

$ robot -d results test_todo.robot                                                           
==============================================================================
Test Todo                                                                     
==============================================================================
Empty list can have item added                                        | PASS |
------------------------------------------------------------------------------
Test Todo                                                             | PASS |
1 test, 1 passed, 0 failed
==============================================================================
Output:  /Users/abdulwahid/Workspace/todo-automation-testing/results/output.xml
Log:     /Users/abdulwahid/Workspace/todo-automation-testing/results/log.html
Report:  /Users/abdulwahid/Workspace/todo-automation-testing/results/report.html

Report

Test reports are generated as HTML files. You can view the generated test report.

Log

Execution Video

Other Scenarios

You can find other scenarios listed which have been automated on GitHub.

Conclusions

“The hardest single part of building a software system is deciding precisely what to build.” Fred Brooks

Hence, BDD enables a descriptive way for defining acceptance criteria. Whereas Robot framework helps us to automate the BDD tests by providing human-readable keywords capabilities. It allows QA engineers to write automated tests in generic format without any programming language intervention.