Friday, May 22, 2015

Unit Testing with scripts in EASE


In the company I work for we do lots of regression testing, mostly for interface testing of hardware devices. Therefore we needed a unit testbench similar to JUnit without the complexity of writing java code for our tests. So we implemented a simple Unit test framework for EASE.

Source code for this tutorial is available in the EASE scripts repository.

Step 1: Create a simple test script

Create a new Project in your workspace named JavaScript Unit Testing Tutorial. Create a new file Tests/Simple/01 Valid tests.js
// starts a simple test
startTest("empty", "an empty test case");
// ends a test
endTest();

// start another test
startTest("no test code", "a test containing no assertions");
print("Hi from valid test");
endTest();

// third test
startTest("valid assertions", "a test containing valid assertions");
// check
assertTrue(true);
endTest();

// code outside of a testcase
print('"' + getTestFile() + '" completed');
You might ask where these test functions come from as we do not load any modules. The unit test framework comes with a module called Unittest. It is hidden by default but will be loaded automatically when we execute EASE UnitTests. To unhide go to the Preferences/Scripting/Modules (Note: this is not necessary to use the module, it will only display it in UI components).

If you are familiar with unit testing in general you also know assertions and according methods like assertTrue(). In general these methods perform a check for validity. In case an expected result does not match with a current result, an error is generated for the containing test case.

So test cases are encapsulated between a startTest() and an endTest(). It is recommended to not use assertions outside test cases, yet not mandatory.

Step 2: Create and execute a test suite

To execute test files, we need to create a Test Suite, that sets up a test environment for us. Create a new Scripting/Script Testsuite named Testsuite.suite.  The editor will allow you to select test files you want to execute. Therefore we look for script files in the same project where the suite is located.

Select our previously created test file and hit the Run Test Suite button in the top right corner.



 A new view is opened that displays unit test results.



The top tree view reflects the file structure of our test files and displays decorators indicating the test execution result. If you select a test file, detailed results are displayed in the bottom section.

Step 3: Test errors

Create a new file Tests/Simple/02 Test errors.js.
// test containing assertions that fail
startTest("invalid assertions", "a test containing invalid assertions");
assertTrue(true);
assertTrue(false);
assertFalse(true);
assertFalse(false);
endTest();

// manually create a failure
startTest("failure", "test throwing a failure");
failure("test broken, stop execution here");

// an exception would also create a failure
throw new java.lang.Exception("code exception");
endTest();

startTest("never to be reached");
// not being reached as the failure above terminates test file execution
endTest();
Save the test, add it to your suite and run the suite right away. When an assertion fails, error markers are generated on your script files on the according code location. You will also see the most critical assertion reflected in the Script Unit Test view. Double clicking on errors will open the source editor on the corresponding location.



Similar to JUnit there exist two different types of things that can go wrong: errors and failures. Errors are assertions that do not meet their criteria. They are recoverable, meaning script execution of the test file continues.

Failures on the other hand are not recoverable, so script execution of that test file is terminated immediately.

Step 4: Test Suite configuration

Typically tests need to be adapted to a certain environment. In general you do not want test engineers to modify your script code. So the test suite supports setting of Variables that are automatically available in all test files. The Setup section allows to execute arbitrary code on the Test Suite start, Test File start, startTest() call and also for the corresponding teardown sections. As you may provide multiple suite files within a project, these secions are a good place to adapt tests according to your environment.

Optional: Test automation

Often tests need to be executed automatically, eg for a nightly test run. The Unittest module allows to run a testsuite from a script. Together with the headless application from EASE you can run a suite from the commandline and create a report (we currently support the JUnit xml format that can be picked up from Hudson/Jenkins).

Optional: Additional assertions

If you want to provide your own assertion methods, simply create your own module and let your wrapped method return an IAssertion. If you call your method from within a test script it will automatically validate your assertion method.

So why do I want to write scripted tests?

In the company I work for we use these kind of tests for interface verification of hardware devices. Therefore we provide our own modules that allow us to send stimuli and validate expected responses. We also use it in combination with analog measurement devices to verify that certain hardware parameters are within valid ranges.

Currently Script Unit Testing is supported for the Rhino script engine only.

Wednesday, April 8, 2015

Live charting with EASE

Recently we added charting support to EASE using the Nebula XY chart widget. Say you have a script acquiring some data you now may plot them as your measurement advances.

Step 1: Installation

For this tutorial you need to register the nebula update site:
http://download.eclipse.org/technology/nebula/snapshot
in Preferences/Install/Update/Available Software Sites.

Afterwards install EASE and add EASE Charting Support to install all required modules. This component depends on Nebula Visualization Widgets, therefore we added the nebula update site before. Currently you have to stick to the nighty update site of EASE as charting is not part of the current release.

Step 2: Some example plots

 
Try out this little script to get these sample plots:
loadModule('/Charting');

figure("Simple Charts");
clear();

series("linear", "m+");
for (x = 0; x <= 20; x += 1)
 plotPoint(x, x / 10 - 1);

series("1/x", "ko:");
for (x = 1; x <= 20; x += 0.2)
 plotPoint(x, 4/x);

series("sine", "bx");
for (x = 0; x < 20; x += 0.05)
 plotPoint(x, Math.sin(x) * Math.sin(3 * x));

figure("Advanced");
clear();

series("drop", "gd");
for (t = 0; t < 2; t += 0.01)
 plotPoint(t, Math.pow(Math.E, t * -3) * Math.cos(t * 30));

series("upper limit", "rf--");
plot([0, 2], [0.4, 0.4]);

series("lower limit", "rf--");
plot([0, 2], [-0.4, -0.4]);
Run the script either in the Script Shell or execute it using launch targets.

Step 3: Charting Module API

The API of the charting module tries to keep close to MatLab style, so the format string for a series is almost identical. You may even omit to create figures or series and start plotting right away and we create everything necessary on the fly.

For a better user experience we extended the XYChart a bit by allowing to zoom directly using the scroll wheel either over the plot or its axis. Use a double click for an auto-zoom.

Sunday, February 22, 2015

Reformatting multiple source files

Recently I had to apply a new code formatter template to a project. Changing 50+ files by hand seemed to be too much monkey work. Writing a custom plug-in seemed to be too much work. Fortunately there is this great scripting framework to accomplish this task quite easily.

Step 1: Detecting affected source files

I have already blogged about EASE and how to install it. So I went directly to the script shell to fetch all java files form a certan project:
loadModule('/System/Resources');

 org.eclipse.ease.modules.platform.ResourcesModule@5f8466b4
files = findFiles("*.java", getProject("org.eclipse.some.dedicated.project"), true)
 [Ljava.lang.Object;@1b61effc
Now we have an array of IFile instances. Easy, isn't it?

Step 2: The formatter script

Ideas how to format source files can be found in the forum. Taking the script and porting it to JS is simple:
function formatUnitSourceCode(file) {
 unit = org.eclipse.jdt.core.JavaCore.create(file);

 unit.becomeWorkingCopy(null);

 formatter = org.eclipse.jdt.core.ToolFactory.createCodeFormatter(null);
 range = unit.getSourceRange();
 formatEdit = formatter
   .format(
     org.eclipse.jdt.core.formatter.CodeFormatter.K_COMPILATION_UNIT
       | org.eclipse.jdt.core.formatter.CodeFormatter.F_INCLUDE_COMMENTS,
     unit.getSource(), 0, unit.getSource().length(), 0, null);
 if (formatEdit.hasChildren()) {
  unit.applyTextEdit(formatEdit, null);
  unit.reconcile(org.eclipse.jdt.core.dom.AST.JLS4, false, null, null);
 }

 unit.commitWorkingCopy(true, null);
}

Step 3: glueing it all together

Load the function into your script shell, fetch the list of files as in step 1 and then process all files in a loop:
for each (file in files) formatUnitSourceCode(file);
That's it. This short script uses your current source formatter settings and applies it to all selected files.

This little helper is available online in the EASE script repository.

Wednesday, January 14, 2015

EASE 0.1.0 - the very first release

I am proud to announce the very first release of the Eclipse Advanced Scripting Environment (EASE).

In case you have no idea what this is about, check out the project page. Afterwards you definitely need to install it right away in your IDE and start scripting.

Some facts: 
  • executes script code in your IDE, providing access to the running JRE and thus to the whole Eclipse API
  • supports JavaScript, Jython, Groovy and JRuby (see details)
  • allows to dynamically integrate scripts into toolbars and menus
  • extensible via script libraries (write your own)

This project started as an in house company solution some years ago. When I released it as open source back in 2013 I was quite astonished about the interest from the community. Soon this project moved into the e4 incubator which gave us a great chance to evolve and build up an infrastructure.
Last summer EASE was accepted as an Eclipse incubation project. With our first release we feel to be part of the eclipse community.

Let me take this opportunity to thank all the people who helped to make this happen. When I started out almost 2 years ago I did not expect to meet such an open minded community. So if you think of starting your own project I can just encourage you to take the first step. There are helping hands everywhere, trying to push you forward. Did I already say that I love this community?

XMLMemento: a simpe way to process XML files


Reading and writing simple XML files is a common task nowadays. Java comes with some powerful frameworks like SAX or Xerces to accomplish this task. But sometimes we look for something simpler. Today we will have a look at XMLMemento, a utility class by Eclipse that hides most of the complicated stuff from us.

Source code for this tutorial is available on googlecode as a single zip archive, as a Team Project Set or you can checkout the SVN projects directly. 

Preparations

We will use following XML file for this tutorial
<?xml version="1.0" encoding="UTF-8"?>
<root>
 <project name="org.eclipse.ease">
  <description>EASE is a scripting environment for Eclipse. It allows to
   create, maintain and execute script code in the context of the
   running Eclipse instance.</description>
  <releases>
   <release version="0.1.0" />
   <release version="0.2.0" />
  </releases>
 </project>
 <project name="org.eclipse.tycho">
  <releases>
   <release version="0.16.0" />
   <release version="0.17.0" />
  </releases>
 </project>
</root>

The code samples below use a slightly modified version of XMLMemento. To reuse it more easily in non-Eclipse projects I removed Eclipse specific exceptions, messages and interfaces. If you intend to use XMLMemento within Eclipse, simply stick to the original version contained in the org.eclipse.ui.workbench plugin. For pure Java projects simply copy XMLMemento as it is used in this tutorial.

Step 1: Writing XML

First we need to create a root node. Afterwards we create child nodes and set their attributes.
package com.codeandme.xmlmemento;

import java.io.File;
import java.io.FileWriter;
import java.io.IOException;

import org.eclipse.ui.XMLMemento;

public class WriteXML {

 public static void main(String[] args) {

  XMLMemento rootNode = XMLMemento.createWriteRoot("root");

  // first project
  XMLMemento projectNode = rootNode.createChild("project");
  projectNode.putString("name", "org.eclipse.ease");
  projectNode
  .createChild("description")
  .putTextData(
    "EASE is a scripting environment for Eclipse. It allows to create, maintain and execute script code in the context of the running Eclipse instance.");

  XMLMemento releasesNode = projectNode.createChild("releases");
  releasesNode.createChild("release").putString("version", "0.1.0");
  releasesNode.createChild("release").putString("version", "0.2.0");

  // second project
  projectNode = rootNode.createChild("project");
  projectNode.putString("name", "org.eclipse.tycho");

  releasesNode = projectNode.createChild("releases");
  releasesNode.createChild("release").putString("version", "0.16.0");
  releasesNode.createChild("release").putString("version", "0.17.0");

  // output to console
  System.out.println(rootNode);

  // write to file
  try {
   File tempFile = new File("XML sample.xml");
   FileWriter fileWriter = new FileWriter(tempFile);
   rootNode.save(fileWriter);

  } catch (IOException e) {
   e.printStackTrace();
  }
 }
}
Line 13: create the root node
Line 16: create a child node below root
Line 17: set an attribute on project node
Line 20: set text content of a node
Line 36: serialize XML to String (uses node.toString())
Line 42: write XML to File


Step 2: reading XML

Reading XML is also very easy. Just create a read root and start traversing child elements:
package com.codeandme.xmlmemento;

import java.io.FileReader;

import org.eclipse.ui.XMLMemento;

public class ReadXML {

 public static void main(String[] args) {
  try {
   XMLMemento rootNode = XMLMemento.createReadRoot(new FileReader("XML sample.xml"));

   System.out.println("Projcts:");
   for (XMLMemento node : rootNode.getChildren())
    System.out.println("\t" + node.getString("name"));

   System.out.println("EASE versions:");
   for (XMLMemento node : rootNode.getChildren("project")) {
    if ("org.eclipse.ease".equals(node.getString("name"))) {
     for (XMLMemento versionNode : node.getChild("releases").getChildren("release")) {
      System.out.println("\t" + versionNode.getString("version"));
     }
    }
   }

  } catch (Exception e) {
   e.printStackTrace();
  }
 }
}
LIne 11: parse input and extract the root node
Line 14: iterate over all child nodes
Line 15: read node attribute

Friday, December 5, 2014

EASE scripts conquer the UI

EASE just recently got extended to inject scripts into the UI. This means you may insert scripts into your view toolbars, menus and context menus. No need to deploy anything, just provide the script and add some keywords to it.

During this tutorial we will develop our script step by step. The final version is available from the EASE scripts repository.

To follow this tutorial you need to install EASE into your IDE/RCP.

Step 1: A very simple script

Our target will be to open a file browser on the current location selected in the Project Explorer view.

Create a General/Project in your workspace and a file Explore from here.js. To launch an external program we may use the Platform module:
loadModule('/System/Platform');
// runProcess('krusader');
runProcess('explorer.exe')
Now register your script location in the preferences: Scripting/Script Locations. Afterwards test your script by running it using one of the follwing methods:
  • double click in the Script Explorer view
  • right click in Project Explorer and select Run As/EASE Script
  • D&D script file to Script Shell view
Step 2: Bind to toolbar

As a next step we want to attach our script to the toolbar. So open your script file again and add following comment at the top of the file:
// ********************************************************************************
// name                 : Explore from here
// toolbar              : Project Explorer
// script-type          : JavaScript
// description          : Start a file browser using current selection.
// ********************************************************************************
The new toolbar button appears right after saving the script file. No need to restart eclipse! If not you probably did not set up your script locations in preferences correctly.
The toolbar keyword accepts view titles and view IDs if you know them. All supported keywords are listed in the wiki.


Step 3: Using the current selection

When starting the file browser we want to use the current selection as our start folder. it can be easily consumed by using getSelection() from the UI module:
loadModule('/System/UI');

var selection = getSelection();
if (selection instanceof org.eclipse.jface.viewers.IStructuredSelection)
 selection = selection.getFirstElement();

if (selection instanceof org.eclipse.core.resources.IFile)
 selection = selection.getParent();

if (selection instanceof org.eclipse.core.resources.IContainer)
// runProcess("/usr/bin/krusader", ["--left", getSystemProperty("user.home"), "--right", selection.getRawLocation()]);
 runProcess("explorer.exe", [selection.getRawLocation()]);
First we extract the very first element of the current selection. In case it is a file, we get its parent folder. Finally check that we really have a container, then open the file browser.


Step 4: Adding a context menu entry

We might also want to bind our script to any IResource context menu in our application. This is really simple, just add following keyword to the header:
// popup                : enableFor(org.eclipse.core.resources.IResource)

Step 5: Adding an image to the toolbar

Instead of having a text button in the Project Explorer toolbar we may replace it with an image. You guessed it, its just another keyword:
// image                : platform:/plugin/org.eclipse.ease.ui/icons/eobj16/folder.png
This URI reuses an icon stored within the ease.ui plugin. You may use relative paths, file system paths or even http URIs for your images.
When you start building up your own script library, consider pushing your scripts to our script sample repository.

Monday, November 10, 2014

Google Summer of Code - student experience

This year we received a great contribution from Martin, who worked on a GSoC project for EASE. Belows post sums up his experience with the GSoC program. So I leave the stage for Martin...




Hello everyone. First off let me quickly introduce myself. My name is Martin and I study software-engineering at Technical University Graz. This summer I participated in the Google Summer of Code program (GSOC) where I implemented a Jython debugger for the Eclipse Advanced Scripting Environment with the help of my mentor Christian who today is so friendly and "borrows" me his blog.

I wanted to write a blog entry for quite a while now but was never sure what to write about. I finally decided to simply tell my experience with this year's GSOC program and try to give a few pointers to anyone who is interested in participating next year. Please understand that these are just things that worked for me and there is no guaranty that you will be selected next if you do everything I did.

In general the most important part about applying is to have a pretty good plan what you are going to do. The application phase was mid of March this year and applicants where expected to submit a project proposal. This proposal should contain an outline of what you want to achieve, what your milestones and deliverables are and an estimation about your timeline. Based on this application your mentoring organization (the open source organization you are applying to) will choose who they take.

Eclipse has a list of projects for Google Summer of Code and so do several other organizations but if you think you have a potentially great idea just get in contact with them and they will likely tell you if this is something they are interested in.

Once you think you have an idea I would recommend getting in touch with the mentoring organization and see if they have anything particular in mind for your project or if they can give you some tips on how you could tackle the problem. I always like an open dialog about my projects and to hear what others think about it. It is hard to let go off a fixed plan in your head if you don’t get reasonable input from someone else. In my case my mentor helped me a lot in shaping the idea to an actual plan.

If you finally think you have a decent plan about what you want to do it is time to think about how you want to do it. This phase to me is the most important phase of the whole GSOC program. The better thought through your plan is the less work you’ll have to do in the actual implementation phase. Don’t get me wrong, you will still make amends as you go along but the better the initial plan the less derivations there will be and the less additional time you will spend planning.

Make a list of all deliverables (necessary and optional), order and rank them and create your deliverable plan. Once again the more time you spend here the less surprises you will face when doing the actual implementation. Should I even mention that you should give your planned timetable some thought and try to be as accurate as you can at this point of you project? I think you see where everything so far was going…

So one last time: Take your time when writing your project proposal, try to go into enough detail and do not just quickly hack together a proposal. It will help your chances of getting selected because you obviously show dedication and once you actually are taken it helps you bring your project to a satisfying end. The rest of the program – the actual implementation – is quite straight forward. I will still give you some last hints that helped me and hopefully help you too.

Communication is the most important part in the implementation phase. Try to align with your mentor regularly, try to get feedback from others working on your open source project (mailing lists, IRC, you name it) and try to communicate your progress with everyone involved. Your mentor can and will help you with your problems, that’s why they are mentors in the first place. I knew my mentor before so it is easy for me to say this but from what I’ve heard from others every mentor seems to be nice and willing to help you if you only ask. There are some shy people out there – which is totally okay – but they will probably have a harder time. As I said, I really like feedback and it helps me finish projects better than if I was working completely on my own. I know it is hard for software developers to change their minds once they have it set on an idea (believe me I’ve been there more often than I can count) but feedback will almost certainly help. Being more open minded about this is something I strongly believe most (including me) should always try to improve.

The other persons involved in your OS project will probably be the first ones to use your software and play the beta-testers. Listen to their feedback, try to be open to their ideas and wishes and by this you will get the most out of your project. For them as well as for you.

Lastly try to communicate your project’s progress regularly. This is something I could have done a little more and although it worked this time it is still something I would do differently in the future. Your mentor and everyone else involved will help you avoid the stress of having to work triple-shifts just to finish before the deadline because you did not see the stress coming. Most people working on an open source project have experience and have been in similar situations (be it at work or in private) and will help you anticipate these things.

For GSOC there are two deadlines, one in the middle of the implementation phase to show that you haven’t completely wasted the first one and a half months and one at the end where you have to deliver your final results.

I guess lots of people handle deadlines the way I used to do, which is procrastinating until it is almost too late. I understand that you work best under pressure and seek the thrill and challenge of having close to no time to do your job, but it is still something I try to avoid as much as possible (although it still happens from time to time). Your mentors will very likely give you a positive final evaluations if you do not complete everything but tried your best throughout the whole project. If you procrastinate and do not finish then I’m pretty sure you will fail your finals and (maybe even more important) will not get the payment.

These are all tips I can give you, maybe there are others out there that can help you even more. Still I believe if you keep these things in mind but still tackle the program in your way there will be nothing stopping you from participating and finishing next year. Believe me, it is worth it.

Have fun and who knows maybe we see each other during next year’s Google Summer of Code program. I will definitely try my luck again.