Costello Tutorial

Purpose

Testing GUI applications by hand can be very labor-intensive. Costello allows you to write an automated test script for a GUI application without actually having to write it. By recording your actions, Costello writes a modifiable script that you can save and rerun whenever you want. This makes regression testing a snap.

Overview

In this tutorial, we will learn the basics of using Costello by creating and running a simple test script for a GUI application.

Walkthrough

Before we can go about creating our test script, we have to setup our work environment.

Setting up Costello

1. First we need to install Costello. Costello is part of the Abbot framework, so lets download the whole package here.
2. Next extract the zip folder to a chosen destination.
3. Now lets download a simple Java GUI program. For this tutorial we will use a simple GUI lines of code counter which can be downloaded from here.

Creating the test script

Now that we've got our test environment setup, we can start creating our test script.

1. First launch Costello. If the jar extension has been setup properly from Sun's JVM then you should be able to simple double click the Costello.jar file located in abbot's lib directory. If jar files are not correctly setup run the following command on the command line in abbot's download directory.
   

java -jar lib/costello.jar

You should see the following screen appear.

1. Click on File->New Script. This creates a new script with a default launch and terminate steps. Launch is where you will define a class and method to launch your GUI. Terminate disposes of the GUI components created from the launch step.
2. Edit the Launch line by clicking on "Insert your launch information here." A launch pane will appear where you can enter the name, target class, target method, arguments, and classpath of your GUI test.
   1. For this example, enter Launch swingui.LOC.main([]) under launch. This is the name that appears in the script window.
   2. Next enter swingui.LOC in the Target Class Name field. If the text appears in red, then you will have to specify the location of LOC.jar file in the classpath.
   3. Press the "+" button next to classpath and enter <download directory>/LOC.jar in the classpath (for this example C:/LOC.jar). You'll know you got it right if the text turns black and the Method field fills up with items.
   4. Next select main from the Method field. We don't have any arguments in this example so just leave the Arguments alone.
      
3. The finished Launch information should look like this:
   
   You can check if you got it right by using Test->Launch to display the GUI.

Record Actions

Next lets record some actions. Lets use LOC to count some lines of code.

1. Select Capture->All Actions or press Shift-F2. The GUI will appear if it is not already displayed.
2. Download the LOC.jar test file here.
3. Start recording actions by clicking Choose file and selecting the LOCtestcase1.txt file you just downloaded in the LOC GUI being run. As you interact with the GUI, your actions will be recorded in Costello. LOC should look like this:
   
   When you've finished your actions press Shift-F2 again. Expand the arrows next to the sequence that appear in Costello. Costello should now look similar to this:
   
   

Add an Assertion

Now that we recorded some actions, we need to check if we got the right output. We'll check to see if the lines of code are correctly displayed.

1. Hover over the "Total: 22" text label and press shift-F1 to add the value of the text label into Costello.
2. In Costello, select the "text" field in the properties window and press Assert equals. The assertion will compare the text label with the text contained in the label when the assertion was made. Make sure the assertion appears after your recorded actions. If it appears before simply drag and drop the assertion after your recorded actions in the script editor window. Costello should now look like this:
   
   

Running your script

Now that we've got our test let's see if we pass.

1. Select Test->Run or press Ctrl-R to run your test. If everything is fine you should see Costello trace through the recorded actions and finish at Terminate. Costello should look like this:
   
   Congratulations, we passed our test, but what happens when we don't?
2. Click on our Assert action in our Costello script. The Assert equals pane should appear again.
3. Change the Expected result to Total: 23.
4. Now run our test again. Costello should trace through the script but stop when it reaches the assert. You'll be able to tell what test failed because it will be highlighted in red and an error message will appear at the bottom of Costello like so:
   
   You've successfully written a test script in Costello. You can save your test script and run it again on future versions of the LOC program.

Exercises

As an additional exercise download the Hangman game from Fishbowl games found here and write a test script to check if the buttons become disabled after being pressed.

References

Abbot Quick Start Guide
JUnit Primer
Lines of Code Counter
Fish Bowl Games