Costello Tutorial

Purpose

Costello allows you to write an automated test script for a GUI application without actually having to write it. By recording your actions, Costello writes a modifiable script that you can save and reuse whenever you want. Why be a sucker and have to continuously go through the motions in testing the same module every time you change it when Costello can do it for you?

Overview

This tutorial will walk you through the basics when using Costello, which include:

  • Starting Costello
  • Loading your program
  • Creating a test script

There will also be an activity which gives you the opportunity to practice the walk-through.

Using Costello

Start-up

  1. Download the Abbot .zip file from their website.
  2. Unzip the file.
  3. From the terminal, change into the resulting directory (ie. abbot-1.0.1).
  4. Run Costello with the following command:

java -jar lib/costello.jar

Setting Up Your GUI

  1. Make sure you program is compiled.
  2. Go to "File" and select "New Script" (CTRL+N). A panel should appear on the right of the Costello UI.
  3. Under "launch" in the new panel, replace their default method with your main method to run your program off of (ie. Test.java is the main class, enter Test.main([]) ). If there are to be arguments in the method, include them as you would normally when calling method. This step lets Costello know what to run to start your program.
  4. Under "Target Class Name", enter the class that defines the method being used in "launch."
  5. Check if "main" is selected under "Method." If not, change it to "main."
  6. If your main method included an argument, list it in the "Arguments" section by inputing it in the provided text field. If you have more than one argument, input the first one, click on the "+" button to the right of the text field, and input the next argument in the resulting text field. Repeat this step until you have input all of your arguments.
  7. So your program can be found in the first place, set the Classpath to where your program is located relative to the directory you ran Costello out of (ie. Costello is ran out of C:\abbot-1.0.1 and your program is in C:\Test\, set to ..\Test).

Creating the Test Script

We will go over the two parts of creating a test script. First you record yourself interacting with your program then you use asserts on the different components of your program at specific points of your interactions.

Using Recording
  1. Go to "Capture" and click "All Actions" (SHIFT+F2). Your program will automatically be started and your interactions with it will be recorded.
  2. If you want to pause/stop recording, hit SHIFT+F2 or just click on the Costello UI.
Using Asserts
  1. Make sure your program is running, either through "Capture" or "Launch" (see Notes). If it is not running, Costello does not know what the components of it are, therefore you can't write an Assert for them.
  2. On the bottom left of the UI, a tree of the different components is shown. From there you can expand/collapse the branches to access the different components.
  3. Walk through the hierarchy of components and select the desired component.
  4. In the pane to the right of the tree, a list of the Properties (public methods) is shown. Scroll through the properties to find the method you want to test.
  5. Once you select the method, click the "Assert equals" button found below the pane.
  6. To modify the value to test for, go to the script pane (top left of the UI) and select the Assert.
  7. To the right, the properties of the Assert are shown. Modify "Expected Results" to test for a different value.

Running the Test Script

To run a test script, go to "Test" and click "Run" (CTRL+R) or click the "Run" button on the top right of the Costello UI. If an assert fails, the test script will be stopped.

Additional Notes

  • When saving a script, save it as a .xml file to maintain the standardized form of the scripts.
  • To start your program without recording, go to "Test" and click "Launch" (CTRL+L). This is useful if you want access to your program's components (for asserts) without starting the recording.
  • To read the entire stack trace when a test fails, left click on the bottom of the Costello where it states there's an error.
  • To remove a step in the script, go to "Edit" and click "Cut" (CTRL+X).
  • If you move your program while recording your actions, remove all of the "Move" calls except for the final one from that sequence. Otherwise you will have to wait for Costello to go through every single "Move" made on your program on its way to its ending location.

Activity

Through these steps, you will write a simple test script for Dr. Dalbey's LOC counter.

  1. Download the LOC.jar file.
  2. "Save Link As" the LOC test file.
  3. Write a script that records you choosing the test file, clearing the Total, unchecking "Count Comments", and choosing the test file again. Place asserts for the "Count Comments" checkbox and the "Total" label after each instance they change.

References

Abbot's Homepage