Keep your eyes on the DSL prize with Twist

Lost and Confused Signpost

Introduction

Learning a language can be a challenging task.  The absorption of a lexicon takes time and patience.  Twist can help keep this task achievable for consumers of acceptance tests by allowing the definition of confirmation language to be natural.  This is assuming that the consumers will be business customers or other non-technical people.  If that is not the case, I would suggest using a pure code test framework such as TestNG to achieve the technical benefits of the Twist runner design.

Assuming the later statement is the case, Twist has the potential to facilitate communication in and around software delivery teams on a dramatic scale.  It can help give teams a highly accesible language to express acceptance tests that give common understanding, and enable rapid continuous delivery through automation.  However, it does take discipline to keep that goal in mind, and not to get side tracked on the technicalities of coding the automation solution.  It’s not that those things don’t need consideration.  On the contrary, it’s good to be thinking about the execution giving the fastest possible feedback, producing maintainable automation code, and ideally even test data abstraction.  The fact is that those things won’t matter if the language for communication is neglected as the focus.

Content Assist

Twist has an excellent set of features for managing a scenario focused language.  In particular, it allows the following:

  • Creation of method signatures and code from workflow steps (in natural language)
  • Rephrasing of workflow steps across scenarios and the use of searching by content assist
  • Abstraction of multiple workflow steps in to concepts or code implementation (the first is preferred for language clarity)
  • Data driving of workflow step parameters
  • … and an ever growing list

Limit The Deviation and Limit The Confusion

The experience I’ve had is that it is easy to create a massive amount of confusion (and also code) if you don’t keep to some basic patterns of usage around the language used and some ground rules.

Usage Patterns

  • Break down workflow fixtures by pages / areas / services to give them context
  • Before creating a new workflow step search for an existing match
  • When defining a new workflow step ensure it can be easily searched for by others

ground rules

  • To support the searching of workflow steps allow the context of the area of the system under test to be visible
    • e.g. “… on the home page” or “… using google search”
  • Keep to a basic set of action words against each area so that it can display the interactions it allows easily
    • e.g. “Verify search suggestions for ‘Simon Reekie’ using google search”
    • e.g. “Set language to ‘English’ on the home page”
    • e.g. “Set currency to ‘British Pounds’ on the home page”
  • If you are going to get something off one part of the system under test and verify it in another, then keep it visible within the scenario language
    • e.g. “Get and Store breakfast price as “Reported Breakfast Price” on the price page
    • e.g. “Verify breakfast price is equal to Stored “Report Breakfast Price” on the summary page

Conclusion

It is difficult enough communicating with many individuals to a common understanding, and keeping up with managing the complexity of that communication.  Give yourself a chance by building a DSL for Twist scenarios that keeps to a few understood rules, and works within the boundaries of the tool capabilities.  There is nothing wrong with adding or changing the rules, but do it knowingly and collaboratively with those using them.

For those that are looking for the technical implications of this approach, I can only really give you these figures as food for thought.

  • Before this approach was taken in my current workplace, and scenario workflow step language spaghetti was everywhere… we had:
    • Thousands of fixture and test implementation classes built up with duplication (I know… refactoring mindsets could have helped)
    • It took days in some cases for all types of people to write a useful automated scenario
    • The debugging of test failures out of CI took an hours and became a specialist skill
  • After this approach was embedded we had the same coverage plus much more, but we also had:
    • A couple of dozen fixture classes
    • A whole range to skill levels could build new scenarios to point of automation within minutes to an hour

1 thought on “Keep your eyes on the DSL prize with Twist

  • Hi Simon,

    i’m a little surprised that a comment hasn’t been left, so i feel duty bound!

    I think for me, with Twist, the big gotcha was English language.
    On our last project we had at least 57 different ways of navigating to a webpage.
    Martin Walton went to great lengths to try and harmonise the DSL, but it was always a struggle. A lot of the variety came from the developers, because when they implemented a new Twist scenario they would often think in terms of very terse technical steps, so we ended up with:
    User navigates to the HomePage
    User visits to the HomePage
    User is on to the HomePage
    User clicks through to to the HomePage
    User fetches the HomePage
    User lands on the HomePage
    in each of those cases the intent of the test was lost; what are we testing? The mechanism of getting to the homepage, the homepage being available, what? When we questioned the validity of the tests, and tried to determine what the intent was, we found that the homepage wasn’t even important, and it was just a pre condition for a test.
    I like the simplicity of the “Given When Then” format for writing tests because it is a simple way to describe, pre condition, action, post condition. aka Known condition, trigger, expected condition.
    Cucumber does a good job of allowing tests to be written in GWT stylee, but Twist doesn’t readily allow a test to be written like that (for reasons i wont go into here). At first we found this to be an obstacle, however later we discovered that by adopting a more English prose style of writing tests, it often afforded greater clarity above and beyond GWT.

    The intentions of Twist are very honourable. Non technical business users can write tests in in plain English in a wysiwyg interface. Techies wire them up, and they execute. Business users love watching test run automatically. However what is important is that those tests are clearly understood, and that the tests intent isn’t lost.

    Now, what we as testers need to do, is stop calling these things tests. They are just checks. If we only ever run the same automated checks they will only ever find the same defects. Because when you get right down to it, under the hood it is simply an assertion. Automated checks have value. But we have to stop calling them tests.

Leave a Reply

%d bloggers like this: