Friday, 31 May 2013

Do You Like Your BDD Full Fat, Skimmed or Semi-Skimmed?

I've been having lots of fun writing micro services in nodejs and testing them with Yadda - my own BDD library. Before I started Yadda, I did an assessment of the other JavaScript BDD tools. It quickly became apparent there's a great deal of confusion about what BDD is, so I thought a blog post was in order to set things straight.

BDD stands for Behaviour Driven Development and is a term coined by Dan North. The story goes that Dan was running a TDD workshop, and noticed that the participants were getting distracted by the junit syntax, so he started expressing tests in terms of Given, When and Then steps.
Given 100 bottles of beer are sitting a the wall
When 1 falls
Then there should be 99 bottles of beer
This led him to write jbehave, a tool for mapping ordinary language steps to executable functions. Since then a host of other BDD tools such as CucumberRSpec, Concoordian, Specflow and Twist have sprung up.

There has been some argument that BDD is not a real thing, that it's just TDD done well. The counter argument is that the difference is the audience. If your tests are written in code, then the audience are developers. If your tests are written in an ordinary language, your audience can be anybody. I've been writing BDD tests for about five years now, but I've only once found a non developer with enough time, patience and discipline to co-author BDD tests. It was the most effective project I've ever worked on. So I think there is a difference, but it's rare that you'll be in a position to capitalise on it.

The BDD Sweet Spot

However, there is another compelling reason for using BDD tools. They soothe the pain of functional testing. Web testing especially. When you write web tests you are likely to encounter the following problems...
  1. The tests take a long time to write. This is due to what Rob Fletcher calls cadence. The cadence for functional tests will be higher than unit tests because the tests run much more slowly. Things quickly get much worse if you have to rebuild / restart your application whenever you change production or test code. It's usually hard to debug / step through functional tests too.
  2. The test suite takes a long time to run. Sometimes so long it's only run at night. This means the tests are usually broken in the morning and it's some poor souls job to fix all the problems that were introduced the day before. After a while the team gives up on writing functional tests.
  3. The tests are incomprehensible. They contain xpath expressions like '//table/tr[3]/td[4]' or css expressions like 'table#report tr.data td.total span'. Because the test data takes so long to set up, developers keep appending tests to each other.
  4. The tests are brittle. Any time you change your page markup you're likely to break numerous tests. This can limit peoples enthusiasm for refactoring HTML and CSS.
  5. The tests all break at once. Because most functional test tools don't provide an abstraction layer, and because often developers think of writing one until too late, and because of a "it's only test code" mentality, and because when testing is done last it's seen as a chore, it's common for those brittle xpath expressions to be copy and pasted like no tomorrow.
  6. The tests aren't run in a real browser. Because test suites are so slow (and even slower in IE) it's common to only run them in one browser. When that becomes slow, people look to running them in fake browsers. Thanks to improvements in JavaScript libraries this is not as bad an idea as it used to be, but it still a concern.

Since true BDD tools are written in an ordinary language, not only are they (hopefully) comprehensible - solving problem #3, but they also provide an effective abstraction layer, solving problem #5 and thereby greatly reducing the impact of problem #4. Once you've built up a respectable library of test steps, your abstraction layer will enable a high degree of reuse and you'll also partially mitigate problem #1.

This still leaves the problems of a slow development time for new test steps, slow test suites, and authenticity. This is where JavaScript comes in. Since these are functional tests, you can write them in any language you want. JavaScript is interpreted not compiled, which chips away at problem #1. Headless webkit browser bots like ZombieJS or CasperJS run like lightning compared to WebDriver. Goodbye problem #2. For a while you could even feel more confident because WebKit is the base of both Safari and Chrome. However recently Google announced they were forking WebKit and developing a new rendering engine called blink. I still think 5 out of 6 is pretty good.

So what's all this about Full Fat, Skimmed, Or Semi Skimmed?

Well, the problem is that in BDD terms the JavaScript community has got rather confused. The most popular tools, Jasmine, MochaVows  Chai, should.js are proclaimed as BDD, but they're not. Jasmine, Mocha and Vows, are excellent test frameworks, but from a BDD perspective they merely describe('some test function') with text. Chai and should.js don't even do this, they are fluid assertion apis, which attempt to simulate an ordinary language a.bit.like.this(); None of these tools pass the audience test. Worse still they don't provide the abstraction layer, meaning you've still got problems #1, #3, #4 and #5 to deal with.

The only true BDD tool I found during my search was CucumberJS. The problem with CucumberJS is that the Gerkin syntax is restrictive. I don't want to be limited to starting my sentences with Given, When, Then, And and But. I want to express myself naturally. I also want a test runner makes good decisions about which steps to run, instead of picking the first matching one it comes to. I want a tool that doesn't fail silently.  I want a tool that I can use synchronously or asynchronously. I want a tool that lets me plugin different step libraries so that I can test multiple interfaces (e.g. rest and HTML) with the same scenarios. I even want a tool that has nothing to do with testing and just maps ordinary sentences to functions so I could use it in a rules engine or build script. That's why I wrote Yadda.

Tuesday, 12 March 2013

Node.JS ChangeControl

I started an new job back in January instilling a healthy dose of Developer Anarchy into a large media organisation, using RubyNode, Clojure and Redis. One thing I found missing from Node was a change control tool similar to liquibase. This is not entirely surprising since you don't need to manage schema changes when you use a NoSQL datastore, but you might still want to bootstrap reference data, or automate some other just-before-startup tasks. To scratch this itch and also to learn more about to develop Node modules, I've written ChangeControl. Just think "liquibase" without the XML and SQL tasks.

Sunday, 23 December 2012

Yadda yadda yadda

Rob Fletcher's excellent Grails For Hipsters session at this year's Groovy and Grails exchange inspired me to have a play with CasperJS. It's a super fast, super slick, headless web test tool built on PhantomJS. Unfortunately there is a price to pay.

1. PhantomJS is a headless webkit implementation so your tests won't run against Firefox or IE.
2. There's no BDD support.

I can't do much about #1 but there was something I could do about #2. Back in the day I wrote an extension to Selenium IDE providing cucumber/gherkin like language support. It was pretty clunky, but functional (no pun intended). I dusted it off, removed the Selenium cruft, renamed it "yadda" and github'd it here. You can now write your CasperJS tests like this...

casper.start();
casper.yadda([
   "When I open Google's fr search page",
   "then the title is Google",
   "and the search form exists",
   "When I search for foo",
   "then the title is foo - Google Search",
   "and the search for foo was made",
   "and 10 or more results were returned"
]);
casper.run(function() {
   this.test.done(5);
   this.test.renderResults(true);
});


Since the yadda core is just a library for mapping steps (e.g. "When I search for foo"), to javascript functions, it will work without CasperJS and can hopefully be plugged into other JS testing frameworks without too much difficulty. Feel free to fork the project and send me a pull request if you do so.


Friday, 26 October 2012

I was wrong

I'm naturally a cynic. Sometimes this is a useful character trait, but it can also inhibit learning and dampen optimism. To combat this I try to replace my cynicism with genuine curiosity. It's a strategy that is paying off.

Last night I attended the 2012 Agile Awards. Before the event my cynicism meter was somewhere between DEFCOM 2 and 3. I had deep reservations about some of the shortlisted nominees and was concerned that the event would fall somewhere between an Agile nerd convention and an ego inflation exercise. Why would I want to attend something like that? The answer is curiosity - how would I know if my fears were legitimate unless I went and saw for myself. I'm very glad I did, because I couldn't have been more wrong.

Firstly all but one of the nominees I had reservations about had been nominated for work I was unaware of.

Secondly the nominees I knew and who did win were well deserving.

Third, the guests I spoke to were friendly and interesting. Not at all self inflated.

Fourth, I'd been dubious of the "Best Agile Newcomer" category. Unlike Most Valuable Player or Most Valuable Innovation, I didn't think that being "new" was a special cause for celebration. Once again I was wrong. The crowd's rousing cheer and Emma Hopkinson-Sparks obvious delight at winning was the highlight of my evening.

For the record, I was right about two things. The food and organisation were both excellent. I've known Simon Voice long enough to be confident that he wouldn't be so closely associated with anything that wasn't. It was however my first time for meeting his wife Julie, who was responsible for the event coordination (with credit to their daughter for choosing the music). So well done Julie, Simon and everyone else who worked so hard to put together a brilliant evening. I'm delighted to be proved so thoroughly wrong once again.

Friday, 5 October 2012

How Stages Of Learning Should Influence Coaching Style

The Japanese martial art concept of Shuhari identifies three stages required to reach mastery,
  1. Learn
  2. Detach
  3. Transcend
Another learning model, the Four Stages Of Competence, is defined as follows:
  1. Unconscious incompetence - The individual does not understand or know how to do something and does not necessarily recognise the deficit
  2. Conscious incompetence - Though the individual does not understand or know how to do something, he or she does recognise the deficit, as well as the value of a new skill in addressing the deficit.
  3. Conscious competence - The individual understands or knows how to do something. However, demonstrating the skill or knowledge requires concentration.
  4. Unconscious competence - The individual has had so much practise with a skill that it has become "second nature" and can be performed easily.
The Dreyfus model of skill acquisition (which I've selectively quoted for brevity) goes yet further with five stages,
  1. Novice - Rigid adherence to taught rules
  2. Advanced Beginner - Situational perception
  3. Competent - Formulates routines
  4. Proficient - Holistic view of situation
  5. Expert - Intuitive grasp of situations based on deep, tacit understanding
and some petition for inclusion of sixth stage, "Innovation".

There's clearly a degree of overlap between the models and I think it's important to be aware of these concepts when coaching. I've previously blogged about how autonomy is a key component for intrinsic motivation, and while I've not yet written on the subject am generally against Command and Control leadership styles. However I've begun to wonder if, considering the above learning models, whether an autocratic approach might be necessary for novices - those who require a prescribed process.

I suspect this might work well for individuals who are "Consciously incompetent" and may welcome a set of rules to follow, but less well for individuals who are "Unconsciously incompetent", and may feel undervalued or patronised.

It's also common for people to transition backwards from "Consciously incompetent" to "Unconsciously incompetent". They know enough to be dangerous as the saying goes. While in this state attempting to prescribe a process is likely to cause friction and harm morale. It might be better for the individual's personal development to let them fail. Not only will they hopefully revert to being "Consciously incompetent", but they may also acquire the valuable gift of humility.

Yet another possibility is that an individual may actually be competent, but lack confidence in their own ability, i.e. "Unknowingly competent". A skilled coach might recognise this, and assign some task that forces them out of their comfort zone, but I've also seen this backfire. It can cause the individual great anxiety and stress, so once again a culture that tolerates failure becomes important.

Once an individual reaches the level of "Conscious Competence" then I think the coaching style should focus on helping them detach from any previously prescribed process and ultimately lead to them transcending any formal process at all.

Sunday, 9 September 2012

In Your Face Build Alerts With Jinkies

I'm pleased to be presenting at this month's Energized Work Tek Talk on Wednesday the 12th September. During the session I'll discuss why I think audible build alerts are important and introduce Jinkies - an open source application for screaming at you when your build breaks and whispering sweet nothings when you fix it. I'll talk about our decision to develop the web interface on a set of RESTful services and how we implemented this in Grails. I'll take you through how our automated tests use Betamax to stub REST responses, and Cucumber for BDD. I'll also discuss the mistakes we made and what we'd do differently next time.

The session has already sold out, but it's not uncommon to have spaces open up last minute so it's still worth registering if you'd like to come.

Wednesday, 29 August 2012

A Case For Pair Programming - The Follow Up

A few days ago I presented a case for pair programming. Around the same time Gus Power forwarded Paul Graham's insightful article on the Maker's Schedule to the Energized Work mailing list. In response Kris Lander made the following statement...
This just adds to my growing belief that there are a lot of things we could do to improve the way we work both individually and as a team and that we might have to take a second look at a number of things we've been quite evangelical about. Pair Programming is one that I'm really starting to change my view on for example.
My ears pricked up. EW crew members are some of the best around, so when one of them makes a statement that appears to challenge my beliefs it's worth finding out more. I asked Kris to expand. He went on to explain while he valued pair programming, he thought it could be wasteful in some situations. He also suggested that when working alone, one may learn more effectively through trial and error than when working in a pair. It's well understood within Lean Software Development that it is not the fastest or biggest organisation that survives, but the one that adapts most quickly, hence great value should be place on learning, and this could potentially outweigh the benefits of full time pair programming. Furthermore learning (or mastery) is one of the key ingredients of intrinsic motivation, whereas working with someone always one step ahead of you can be demoralising. Finally Kris said he found that working in a full time pairing environment can lead to stagnation, since there is less opportunity to experiment. When working alone, you might permit yourself to briefly investigate something that catches your interest, even if it is not directly relevant to your story. Working in a pair this is less likely, and not everyone will have the time or desire to do this outside of office hours.

The discussion went on for days, with other Energized Workers supporting Kris' view. Simon Baker reminded us of what Kent Beck wrote in is original book on XP. I'm ashamed to say I don't have a copy to hand, but Simon summarised things nicely by saying...
He strongly advocates personal time for people to cut code, explore ideas, spike, make a mess, learn. Then to bring their ideas and discoveries back to the pair and cut the production code. This way people get a chance to at least have a crack at something once, then to do it again in a pair boosted by what they've tried previously, eliminated and learnt.
Kev Richards added that he's always found it hard to let other team members struggle over something, trying to get to grips with it and understand it. But the struggle is necessary - no pain no gain! By helping them understand, frequently he's getting in the way. However he also warned that the results of not pairing could be a nightmare.

Odette Power was reminded of a recent engagement where she learnt more in one day's pairing than in the first four weeks, but also said she often prefers to thrash a problem out on her own in order to really understand it.
In short, I think I'm with Simon (as Kent said) - pairing at the right time, and in a ready state of mind (fresh, new problem domain) is amazing. But not all the time. We know the what and the how- but let us not forget the WHY.
Tom Dunstan paid tribute to the benefits of pairing, but also believes that learning can be most effective when done alone...
Levelling up has to be personal trial and error. Even doing something badly and having someone point out a better way is still better than never having done it yourself.
Chris Pitts joined the party late,  but also agreed that pair programming should not be a full time activity...
When it works, it works beyond my wildest dreams, but it is fragile. Like a lot of agile techniques, it falls apart quickly if the wrong attitudes are present. The usual rules of honesty, humility, respect, and more than a bit of thick skin need to apply at all times. As always, the main problems are usually in the wetware. 
There does need to be downtime. Certainly I personally find it incredibly difficult to pair 100% of the time when there is uncertainty and doubt about the right route forward. When neither pair is familiar with a domain, or cannot see a solution to work towards (or cannot explain their logic!), I think that I can often learn much faster on my own; I can experiment faster, and rapidly try different solutions until I see something that is promising. Then I prefer to bring it back into the pair for validation and sanity checking.
Chris also expressed concern that pair programming (and Agile software development in general) requires a set of social skills that are not traditionally present in software developers.

Rob Fletcher backed up Chris' view that pairing is fragile and more suited to some tasks than others...
I've experienced it being incredibly productive and at other times felt like my pair or I (even both) are not involved. I'm not sure why it sometimes doesn't work. Sometimes it's the nature of the task, sometimes a personality clash, wrong mix of skills or abilities or just someone not in the right frame of mind. Even then I don't necessarily think pairing is _less_ productive than if those people were working individually. 
it's more suited to some tasks than others. I do a fair bit of coding in my spare time & find that I enjoy going solo when doing exploratory "spiky" stuff but find myself wishing for a pair when building code out from the results of such spikes. I miss the validation that I'm on the right track, the pressure to stay focused.
The discussion is still ongoing, but has made me realise that while presenting the benefits of pair programming, I had completely neglected to point out its problems or indicate in what situations it might not be appropriate. It's widely accepted that different people prefer different learning styles (but contested that teaching in the learning style preferred by the student is more effective). My preference is for learning by example, but I don't mind whether this example comes from a colleague, formal documentation, a blog post or my own trial and error. I wonder whether people with other learning styles find pairing more stressful and tiring than I do? It's worth considering.

I also agree that when spiking (the process of investigating one or more possible solutions to a design problem) can be more effective when pairs split up and work individually. I sympathise with Kris' point regarding stagnation. With two young kids, a long commute and a serious attempt to get fit before I hit 40, I'm finding it harder and harder to dedicate the time I would like to technology. Maybe this is another reason why organisations should adopt some variation of 20% time. If they did they might even end up inventing something as profitable as the post-it note. But that's another blog post.

Many thanks to the EW crew for drawing attention to pair programming's downsides and helping me level-up my thinking.