Specification by Example: Illustrating using examples, Validation & Green Tests
Natural languages play a trick on us. They leave space for interpretation, misunderstanding, and sometimes require knowledge of the subject area and / or specific slang in order to understand what is the point. A slight misunderstanding can lead to deadlines and a large number of edits or even rewriting entire modules. Instead of formulating requirements long and painfully, describe them with examples. Together with the client you can find key examples. The help of developers and testers is simply invaluable in this case, because they can identify potential problem areas and technical limitations in advance.
Here’s what a collaboration process on a specification might look like. For example, let’s take the online store “Horns and Hooves.”
Barbara – customer representative
Andy is a developer
Olivia – QA
Barbara: So, we have concluded an agreement with ABC Press publishing house that we will deliver books purchased from us for free.
Andy: All over USA?
Barbara: No, sure. Only in NY.
Olivia: And in our admin panel is the correct publishing house filled everywhere? You can give us a complete list of books, I’m afraid we can get it wrong, in the database the “publishing house” field is a line.
Barbara: Yes, of course, there are not many books there.
Andy: How many books can we deliver for free? Do we have any restrictions: 5-10? What happens if there are books from other publishers in the basket?
Barbara: It doesn’t matter, as long as there is at least one book in the basket, delivery is free.
Andy: Well, what happens if a client orders a book and a fridge? Such delivery will be expensive …
Barbara: Yes, we somehow didn’t think about it. Let’s write down and clarify, and we will discuss this in a week, is there enough information to start work?
Andy: Of course, I’ll start sketching architecture.
Olivia: And I’ll check what’s with the publishers.
In 3 days.
Barbara (by phone): We talked and decided that free shipping would be available only for books. If there is anything else in the basket, only normal delivery. And we decided to limit the number of books to no more than 10.
Andy: Ok.
As a result, we get key examples:
Customer type | Cart contents | Delivery |
VIP | 1 book | Free |
VIP | 10 books | Free |
VIP | 11 books | Standard |
Regular | 10 books | Standard |
VIP | 5 washing machines | Standard |
VIP | 1 washing machine | 5 books Standard |
Refining the specification
Through a joint discussion, you will be able to achieve a common vision of the goal. You can compare the previous stage with the fact that you mined a diamond. A diamond in itself is quite valuable, but after processing its value will increase many times over.
During the discussion, each team member will talk from his own point of view:
- Business representatives tend to pay too much attention to UI (because they are not specialists in the technical field, UI is the only thing they can “touch with their hands”).
- Developers tend to delve into implementation details: which framework to choose and which technology to apply.
- Testers will be paranoid in their search for bugs and vulnerabilities.
Thus, the original version is likely to be contradictory and redundant.
The purpose of this stage is to highlight the main point and provide the right amount of detail. The specification should become a single document for:
- Acceptance Criteria
- Acceptance tests
- Future regression tests
In order to minimize the chances of misunderstanding, we will write the scripts in terms of Given When Then. If you are not familiar with this form of writing yet, remember the math lessons at school: Given, Find, Solution. It’s about the same here.
Given – initial context (precondition)
When – event (what is a script trigger)
Then – the result we want to get
For our example with books, it will look like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
Feature: Free delivery In order to save money As a VIP customer I want the system to offer free delivery on certain items to me Scenario: Free delivery Given I am a VIP customer And I am on product detail page And There are only books in my shopping cart And There are <= 10 books in my shopping cart And I have added 'ABC Press' book to my shopping cart When I press 'Go to checkout' button And I have chosen 'NY' in 'Ship To' dropdown Then I can choose free delivery |
This form of recording, from the one side, remains readable for non-technical specialists (managers, client representatives, business analysts), and from the other side, it is strict enough to avoid ambiguity.
We can modify the script a bit to make QA happy.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
Scenario Outline: Free delivery Given I am a VIP customer And I am on product detail page And There are only books in my shopping cart And There are <bookQuantity> books in my shopping cart And I have added 'ABC Press' book to my shopping cart When I press 'Go to checkout' button And I have chosen 'NY' in 'Ship To' dropdown Then <deliveryType> is available Examples: |bookQuantity|deliveryType| |5 |Free | |10|Free | |11|Standard | |
In this case, the script can be used as a list of test cases.
To write this script, I used SpecFlow, a solution for the .NET platform. Similar tools are for Java, Ruby, PHP.
I specially focus on the instrument. How exactly to write tests using Given When Then is a too extensive topic, which we will discuss in the next article, but for now we will restrict ourselves to basic information. Tests are created in a declarative style using “steps”. Note that the parameters from the Examples table will be passed to the arguments of the methods.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 |
namespace ProjectName.Specification { public class FreeDeliverySteps { [Given("I am a VIP customer")] public void GivenIAmVipCustomer() { throw new NotImplementedException(); } [Given("I am on product detail page")] public void GivenIAmOnProductDetailPage() { throw new NotImplementedException(); } [Given("I have added \'ABC Press\' book to my shopping cart")] public void GivenIHaveAddedAbcPressBookToMyShoppingCart() { throw new NotImplementedException(); } [Given("There are only books in my shopping cart")] public void GivenThereAreOnlyBooksInMyShoppingCart() { throw new NotImplementedException(); } [Given("There are (.*) books in my shopping cart")] public void GivenThereAreBookQuantityBooksInMyShoppingCart(int bookQuantity) { throw new NotImplementedException(); } [When("I press 'Go to checkout' button")] public void WhenIPressGoToCheckoutButton() { throw new NotImplementedException(); } [When("And I have chosen 'NY' in 'Ship To' dropdown")] public void WhenIHaveChosenNYInShipToDropdown () { throw new NotImplementedException(); } [Then("Then (.*) is available")] public void ThenDeliveryTypeIsAvailable(string deliveryType) { throw new NotImplementedException(); } } } |
Automating validation without changing specification
The completed specification will serve both the technical task for the development of functionality and test scripts to verify the correctness of the work. Do not put off test automation for later. Tests will detect errors at an early stage of development, which means you do not have to redo the functionality again and again.
If when the requirements change in the future, you will not need to figure out which tests are no longer needed. Since the requirements have changed, then the specification also needs to be changed.
Tests that covered part of the functional code that “fell under the knife” will either turn red or stop working altogether (go into Ignored status), and the process will need to be repeated first: write tests, rewrite code, verify that all tests pass.
With this organization of work, you will change the specification and tests in one place. It means, the specification itself will become executable.
Validating frequently, evolving a documentation system
So, we have an executable specification (a specification with examples and related automated tests). Unfortunately, most of the team still cannot “feel” the result of the work done. Managers will not install the IDE with plug-ins, and deploy the entire system on their machine.
You need to integrate test execution into your build system (if there is no build server in the organization, maybe this is an occasion for it to appear?)
Once the tests are performed regularly, you will notice that automated tests are the most reliable source of information about the state of affairs in the project at the moment. The graph below shows an example of a successful development iteration. On the first day, was written a small part of the acceptance tests and all of them are red (which is understandable, because we do not have any functionality yet). Gradually, the number of tests increases to cover the entire development scope. In parallel, the number of green tests is growing – they began to develop functionality.
The number of green tests is the only reliable criterion for evaluating what has already been done and what remains to be done.
Unfortunately, practice shows that developers can report “yes, yes, tomorrow it will work” for months. The schedule will not lie: the feature is ready when all its tests are green.
Too many red tests, and release soon? Most likely, measures need to be taken, otherwise the functional will not be ready on time with the proper level of quality.
With the development of the product, the specification will grow, and the requirements will change after the requirements of the market. Even such a specification will have to be maintained and structured so that all information is quickly available.
We are ready to pay this price for the confidence that every next release will take place without deadlines, overtime, and the product will work like clockwork. Contact us to build your wow-product right now!
Related Posts
Leave a Reply Cancel reply
Service
Categories
- DEVELOPMENT (102)
- DEVOPS (53)
- FRAMEWORKS (25)
- IT (24)
- QA (14)
- SECURITY (13)
- SOFTWARE (13)
- UI/UX (6)
- Uncategorized (8)