Automated Eclipse GUI testing the quick and simple way

November 3, 2007

We’re very test driven here at Cape Clear, we develop automated tests for everything we do. We’re not strict about writing our tests first, I for one write my tests with my code, iterating between one and the other in the course of realising a story (we follow Scrum, a lightweight wrapper process on agile/XP). I wouldn’t dream of writing code without some level of automated test coverage, to me it is meaningless – how do I know the feature code works if I don’t have something that proves it works now, and as I refactor the code base through iterations of the product? Writing tests makes me many orders of magnitude more productive as a developer. I still hear the “lack of time and resources to write automated tests” excuse from developers I know in other companies. Sometimes I argue with them, sometimes I just smile benignly: you don’t need more resources or time to write tests, writing tests gives you more resources and time, and of course results in far superior quality software. But the fly in the ointment for us has been automating the GUI testing of our Eclipse-based tools. We have extensive junit tests for the non-GUI parts of the tools, like our (WTP) facet install delegates, our builders, our models etc. Eighteen months ago we chose Eclipse TPTP, the GUI recorder and playback toolkit, to automate our GUI tests. Maybe others have had more success with TPTP than we have, but our experience was less than satisfactory. In the end we only achieved a tiny amount of coverage with it, and it is difficult to keep these kinds of tests passing and running continuously across multiple branches of the product. In general GUI recorder/playback tests are very brittle to even minor changes in the user interaction. Several things happened recently that made me realise we could and should drop that approach. We started to push our (PDE) junit tests up into the UI, specifically in relation to testing our GMF-based SOA Assembly Editor. We wrote tests that did things like clicking on all the tools in the palette and checking that the edit parts and model elements were created correctly. PDE junit tests run in the UI thread. It struck me that we already had 99% of what we needed to automate our GUI tests from junit. What we did not have was:

  • a test framework, test APIs, which read like a GUI test specification
  • the ability to automate the testing of blocking UI elements, namely wizards and dialogs

The first was easy, I took a couple of our WSDL-to-Java project wizard GUI test cases and prototyped the kind of APIs I wanted, they read just like we write our test specs. Then I implemented the APIs, most of which were very thin (but more test friendly) facades on existing APIs and existing test code. That left me with the wizards and dialog problem. When you launch an SWT wizard or dialog from a PDE junit test, it blocks because its waiting on input from the SWT event queue. The blocking happens in the open() method of org.eclipse.jface.window.Window, from which WizardDialog is derived. In an automated test, we want the input to come from the junit test code, not from the SWT event queue. Fortunatly, open() is public. I will resist going off on a tangent here about one of my pet gripes: the excessive marking of methods as private and classes as final etc. – let me decide how I want to specialise your code, you cannot see all ends and mostly I know what I’m doing. Anyway, back on topic, so now we have our own CcWizardDialog (which extends WizardDialog), and the code looks like this:

public int open() {
  if (this.cctest == null) {
    return super.open();
  }
  else {
    return doTestOpen();
  }
}
protected int doTestOpen() {
  Shell shell = getShell();
  if (shell == null || shell.isDisposed()) {
    shell = null;
    create();
  }
  shell = getShell();
  constrainShellSize();
  shell.open();
  ICcWizard ccWizard = new CcWizardImpl(getWizard(), this);
  cctest.testWizard(ccWizard);
  return getReturnCode();
}

The cctest member is an object that implements a simple callback interface, typically its the junit test itself. There is no difference in adding these kinds of test hooks to code and doing test-driven development. We write our code to be testable by code. Remember that we’re not trying to test SWT or core Eclipse platform components, we know they are well covered, stable, mature – basically: we know they work. And of course we do manually test and use our tools too. The ICcWizard interface looks like this:

public interface ICcWizard {
  ICcWizardPage getCurrentPage();
  ICcWizardPage next();
  ICcWizardPage back();
  void finish();
  void cancel();
  IWizard getWizard();
  boolean isNextEnabled();
  boolean isFinishEnabled();
}

And (an abbreviated) ICcWizardPage interface looks like this:

public interface ICcWizardPage {
  void setTextValue(...);
  void selectComboValue(...);
  void setCheckBox(...);
  void setRadioButton(...);
  ...
}

Now you can start to see how the junit test reads, just like you’d write a GUI test spec: launch the wizard, set a value in a text box, select a value in a combobox, go to the next page in the wizard, select a radio button, press finish. After which the call stack unwinds back to the junit test, which can then use project APIs to verify the results. I’ve been deliberately vague about ICcWizardPage. How that works is also quite interesting, and very simple. I will detail this and more in another posting. What I really like about the whole approach is that the tests are quick to write and simple to maintain.

Advertisements

5 Responses to “Automated Eclipse GUI testing the quick and simple way”

  1. Nate Says:

    I enjoyed reading your post, and look forward to seeing where these techniques can help with my own Unit Tests. About your “pet gripe” on excessive private/protected methods. You do realize that derived classes can make those methods more accessible (private could be made public in your derived class if you really need access). That is exactly what cloning does with the protected Object.clone() method. I don’t think that means it would always be the right thing to do, but at least it is possible.


  2. Hi David, interesting posting thanks! To get around the blocking of dialog.open(), in my jUnit test code I always use setBlockOnOpen(false), which is defined on the Window class. That way the open() method of my dialog doesn’t block and does not need to be modified to be testable, e.g.:

    public void testMyDialog() {
    MyDialog dlg = new MyDialog(…);
    dlg.setBlockOnOpen(false);
    dlg.open();
    // doesn’t block so now I can press buttons enter text in fields etc
    dlg.close(); // only needed if the dialog wasn’t closed by pressing OK / Cancel
    }

    Cheers, David Bosschaert

  3. David Black Says:

    Hi David,

    Using setBlockOnOpen(false) is certainly the better approach in the test scenario you’ve shown i.e. where the unit test is testing the dialog in isolation. It is good to test dialogs like that either way. But we also have a lot of tests which cause dialogs or wizards to be invoked via action delegates or dialogs that (for one reason or another) are invoked from deep behind an API. In this case the code that follows the open() is not test code, but feature code. So being able to pass a (possibly null) test hook object into these invocations means we can do “system level” UI tests where the only difference between real user interaction and test-driven interaction is the presence or absence of a test hook.

    regards
    David

  4. Doddsie Says:

    Hi, I enjoyed the article but am still having some problems like, … How do you handle confirmation dialogs.

    User enters some data and selects ok
    Confirmation dialog comes up
    Then what? Our tests are stopping there.
    Another example is if an error is thrown, a dialog appears and we would like to capture the error and continue testing. Report the test as failed ofcourse.

    Cheers,

    Dave


  5. Hello would you mind letting me know which webhost you’re working with?

    I’ve loaded your blog in 3 different browsers and I must say this blog loads a lot faster then most.

    Can you recommend a good web hosting provider at a honest price?
    Cheers, I appreciate it!


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: