Git and Maven release:prepare

I don't know if it happens to me only due to my complex source control configuration (which uses SSH keys like Github), but it happens quite often to me that the release process gets stuck right when it's time to push changes.

When this happens you are left with almost no choice but to perform a release:rollback and start again, but today I've found a good solution to this problem: git reset.

So, if your release preparation process is interrupted for some reason and you want to restart it again, but when you try you get a message saying the git commit command failed, then just do the following:

git reset --soft HEAD~1

This command will remove the commit from your local clone leaving all the Maven changes intact, thus allowing you to recover the process where you left it.

Have fun!

Jacoco code coverage tool

Today I discovered with great pleasure a great project for gathering unit, integration and automation tests code coverage: Jacoco!

I was so excited that I decided to give it a try on my template Maven project and the POM went down 150 lines with greatly improved readability!

No more tricky profiles or instrumented artifacts you need to include explicitly and solely to gather coverage: everything works like a charm!

Obviously my previous post regarding Cucumber tricks is still totally valid.

Thanks Jacoco!

Gherkin syntax quicksheet

I found an interesting article containing a nicely presented and very well described Gherking syntax guide and I want to support it.

Embedding screenshots in Cucumber JVM

I was trying to improve our automated test suite and I thought that it could be useful to capture a screenshot of the browser whenever a test fails.
The current Cucumber JVM implementation highly simplifies this task, but the task is not achieved the way I thought, so this is the reason for this post.

Normally you would use a JUnit TestRule to augment all your test cases with a feature to take screenshots, but for Cucumber JVM it's much easier thankfully to the Execution Hooks:

 public class ScreenshotHook {  
     @Autowired  
     private WebDriver driver;
 
     @After  
     public void embedScreenshot(Scenario scenario) {  
         if (scenario.isFailed()) {  
             try {  
                 byte[] screenshot = ((TakesScreenshot) driver).getScreenshotAs(OutputType.BYTES);  
                 scenario.embed(screenshot, "image/png");  
             } catch (WebDriverException wde) {  
                 System.err.println(wde.getMessage());  
             } catch (ClassCastException cce) {  
                 cce.printStackTrace();  
             }  
         }  
     }  
 }  

As you can see I'm injecting the WebDriver instance (I'm using the Cucumber JVM Spring integration) and defining an embedScreenshot method which is annotated with cucumber.annotation.After: this is the important bit because methods annotated as such will be executed after each cucumber scenario.

For sake of completeness I want to tell you there is another hook cucumber.annotation.Before that can be used for things like logging into your application eliminating those repetitive log in steps.

UPDATE: Scenario was ScenarioResult in pre 1.0.0 versions of Cucumber JVM.

Agile and the Definition of Quality

If you are interested in software quality, if you want to improve your agile process output, if you like to have satisfied customers you want to read this article and, after reading it once, you might want to read it a second time: what he's saying is not obvious at all.

Selenium tests on Jenkins

It's not uncommon to have a CI server running tests based on Selenium and it's not uncommon to get in troubles with Linux headless (without a windows manager) servers.

Usually you get errors like:

Error: no display specified

or 

Error: cannot open display: :0.0

The following is what I think is a solution that should be portable and does not require a lot of configuration skills.
First of all you need to install the Jenkins Xvfb Plugin, which allows you to easily start a new virtual screen per  Jenkins job. Please do not forget to properly configure this plugin in the Jenkins System Configuration screen as described in the plugin page: if you don't know the path of the Xvfb executable you just run the following command in a shell on your CI server

 which Xvfb

Usually the value you will use as Directory in which to find the Xvfb executable is /usr/bin.

Once you have this you just need to activate the Xvfb plugin for your job and that should be it, without any modification in your POMs or any other hassle: the plugin will create a new virtual screen, set the DISPLAY environment variable and execute your build.

If you are using the FirefoxDriver, as I do, and you still have issues, but you can see in your output lines like the following ones, then your problems are not any longer related with being on a headless server and my suggestion is to ensure you are using the latest version of the selenium drivers:

Xlib:  extension "RANDR" missing on display ":32".
failed to create drawable
*** LOG addons.xpi: startup
*** LOG addons.xpi: Ignoring file entry whose name is not a valid add-on ID: /tmp/.../webdriver-staging
*** LOG addons.xpi: checkForChanges
*** LOG addons.xpi: No changes found

Please note the initial two lines are perfectly fine and not related to your issues!
I struggled with an error which actually went away right after I updated my selenium-java dependency to the current latest!

A few Cucumber tricks

I've been a fan of TDD since I discovered it a few years ago. Thankfully to my friend and colleague Augusto a few months ago I've discovered the beauty of BDD and ATDD: I got so fascinated by this practice I started an Open Source project for an Eclipse plugin featuring a rich editor for Cucumber feature files. It's called Natural and I think it's pretty cool if you want to check it out.

After having used Cucumber for a while and many, many mistakes I now have got some experience I wish to share with you.

One of the key features of feature files is that they represent a live project documentation, something developers write and maintain and everyone can understand. We believe so much in this concept that we wanted to give our product owner and business partners a nice view on those files, that's why we used Relish to publish them in a pretty colored format. At the same time we had to face some problems while practically use these files for automation testing.

After a few development cycles we were so excited by Cucumber and ATDD that our automation test suite counted hundreds of tests: it was consuming too much time to be completely executed on each commit by our CI system.
My suggestion is to track in your feature files the development cycle each test has been developed: this can be easily achieved by using a tag, something like @sprint-7.
This way you can instruct your CI system to execute the current sprint automation on each commit and schedule a full test nighttime (and launch time possibly).

Another issue was we were developing automation test before the functionality was implemented, so following TDD principles. By doing this our CI was reporting a build failure until the functionality was completely implemented. This is not a problem as it is expected to have failing tests during development, but the continuous failure messages received by the team members brought us to the point we were starting to ignore those messages, vanquishing the purpose of the automation suite and the CI system itself. The solution I suggest is to temporarily annotate the tests that are expected to fail with @future, instructing your CI to skip any test annotated as such: the developers can still execute those tests, but the CI will just ignore them. Once a feature is completed it will be developer's responsibility to remove such annotation.

With hundreds of tests we had problems in having the tests properly organized so here comes the folder structure and naming convention. Please consider those files not as test, but as your application documentation: as such you want to organize then in a manner they are easily accessible by a business person. My suggestion is to organize the tests in files describing different features, grouping the features into folders representing functional areas. Considering company search and report search capabilities I can imagine a search.feature and an advanced-search.feature, both inside a company folder and the same file names in a report folder.

Defect and user stories tracking, if desirable, can be once again achieved by using tags like @US1234 and @DE9876, but I would rather avoid cluttering the feature files with such information as they will tend to distract a business person from the real value of those files: application documentation.

Automation test coverage is a report we found business users find quite valuable, but it was tricky to achieve considering our application was a web one. Nevertheless we managed to have it running on our CI thankfully to Maven and its amazing set of plugins.

I've prepared a template Maven project for those reading this post to use as a starting point and guideline if you like the solutions we adopted: it will definitely help me in the future to avoid redoing all the steps from scratch! Please, read the README file before asking for clarifications :-)