Posts Tagged ‘uiml.net’

Gummy UI improvements

Friday, March 21st, 2008

I am currently working together with Jan on improving the Gummy tool (website still under construction). We have come a long way since Jan wrote the first version for his Master’s thesis. I figured it might be interesting to share a few screenshots of different phases in the development:

Here’s the first version (June 2007):

Gumme screenshot of June 2007

A version with roughly the same UI but lots of architectural improvements (November 2007):

Gummy around the end of 2007

The current version with an improved UI (March 2008):

Gummy screenshot of March 2008

I think the most significant improvement is the new toolbox. Designers can now more easily distinguish between different widgets. In previous versions, some widgets were hard to distinguish. We added labels to each widget in the toolbox, and improved the inline rendering of the widgets. Notice that these images are still not predefined icons, but real widgets rendered to a bitmap. However, in the current version we chose to render them at their optimal size and then scale the images down.

Rendering widgets to bitmaps also forced us to migrate to Windows. We started our development on GNU/Linux using Mono but had to switch due to an annoying bug in Mono’s implementation of Control.DrawToBitmap. Future versions should be able to run in Mono again when this bug is fixed though.

Update: Jan sent me a screenshot of an even older version of Gummy, dated back to December 19, 2006:

Gummy screenshot of December 19, 2006

Full paper on Gummy accepted at AVI 2008

Thursday, February 21st, 2008

Our hard work before the holidays has paid off ;-) We just heard that our full paper submission for AVI 2008 has been accepted.

Gummy

Jan Meskens, Jo Vermeulen, Kris Luyten and Karin Coninx. Gummy for Multi-Platform User Interface Designs: Shape me, Multiply me, Fix me, Use me. To appear in Proceedings of AVI ‘08, the working conference on Advanced visual interfaces, Napoli, Italy, May 28-30, 2008.

In this paper we introduce a multi-platform user interface design approach, and Gummy, a design tool to support that approach. This work originated out of Jan Meskens’ Master’s thesis, in which he created a UIML GUI builder. While there are several tools for developing multi-platform user interfaces, these have a number of problems: (1) the resulting user interfaces often lack the aesthetic quality of manually designed interfaces; (2) the tools are not intuitive since designers have to deal with abstractions and do not directly manipulate the user interface design; and (3) designers can not accurately predict what the resulting user interface will look like. Our goal was to allow designers to reuse their skills of existing user interface design tools (such as GUI builders) as much as possible and try to maintain a high level of fidelity (unlike sketch-based design tools).

Gummy design process

We also had a short paper/poster about Gummy accepted to CHI 2008 Work-in-Progress. In this paper we explain how the tool can be used to involve domain experts in the user interface design process.

Gummy domain expert workspace

Kris Luyten, Jan Meskens, Jo Vermeulen and Karin Coninx. Meta-GUI-Builders: Generating Domain-specific Interface Builders for Multi-Device User Interface Creation. To appear in CHI ‘08 extended abstracts on Human factors in computing systems, Florence, Italy, April 5-10, 2008.

We received lots of input on the prototypes and early drafts of the papers, so thanks to everyone at our lab who contributed in one way or another :-) Additional thanks go to Karel Robert for creating the Gummy logo (have a look at his portfolio).

More information about the papers can be found at my publications page.

Pluggable typedecoders for Uiml.net

Friday, November 16th, 2007

I spent some time the last weeks to support type decoding plugins in Uiml.net. This is mainly useful when you want to interact with applications or web services that have their own types that need to be converted to something the widget set understands. Suppose for example a web service returns a set of Persons, which need to be represented in a list view. The renderer does not know how to transform a Person into an item of a list view, so you need to define a custom component that sits between the renderer and the web service, and can provide this conversion. However, since you don’t know which widget set is used, you have to do this for every possible widget set (e.g. System.Windows.Forms, Gtk#, System.Windows.Forms on the Compact Framework, etc.). Furthermore, it would be better to let the renderer manage this code.

So I created a type decoder plugin system and while I was at it, also cleaned up the code. This resulted in only one general TypeDecoder instance being created in the renderer, while we previously had one instance per backend. Now we have a container class in each backend to host widget set-specific type decoders. This container class get registered with the TypeDecoder, and is in fact also a plugin.

Instead of going into the implementation details, let’s have a look at an excerpt from the System.Windows.Forms container class (SWFTypeDecoders.cs):

37
38
39
40
41
42
43
44
45
46
47
using Uiml.Rendering;
using Uiml.Rendering.TypeDecoding;
 
public class SWFTypeDecoders
{	    
    [TypeDecoderMethod]
    public static System.Drawing.Point DecodePoint(string val)
    {
        string[] coords = val.Split(new Char[] {','});
        return new System.Drawing.Point(Int32.Parse(coords[0]), Int32.Parse(coords[1]));
    }

The only thing we have to do to define a type decoder method is add the [TypeDecoderMethod] attribute and support as a parameter the type we want to convert from. The return type is what we will convert to. In the above listing, the DecodePoint method converts a string to a System.Drawing.Point. The [TypeDecoderMethod] attribute is used to declare that the corresponding method is a type decoder. This way other auxiliary methods will not be registered and won’t pollute the type decoder registry.

To test the implementation, I created a simple class that connects to del.icio.us and gets all my tags. I use this class to show them in a Gtk# UIML GUI. To be able to convert between the XML document that is returned by del.icio.us and the user interface, I wrote a custom type decoder, and connected it to the renderer. I also have a short screencast showing its workings.

Pluggable typedecoders example

I have extended the Uiml.net type decoder to combine existing type decoding methods if direct conversion is not supported. In this example I created a type decoder to convert from a System.Xml.XmlDocument to a Uiml.Constant. But the Gtk.TreeView widget requires a Gtk.TreeModel. The renderer will therefore look for a conversion from a Uiml.Constant to a Gtk.TreeModel and apply the type decoders in sequence. Although we could have converted directly to this data type, this is not as flexible since it is widget set-specific. Although the interface will remain the same, I will probably change the underlying implementation to a graph with the types as vertices, and type decoders as edges to better support these indirect conversions.

Last year’s Uiml.net theses

Thursday, October 18th, 2007

In my previous post I wrote about this year’s Master’s thesis students that will be working on Uiml.net. However, I hadn’t blogged about what last year’s students accomplished yet.

Ingo Berben chose a Bachelor’s thesis to improve the standards compliance of our renderer. He eventually concentrated on the behavior section, and more specifically on supporting conditions other than events. The movie below shows how this can be used to support form validation. The renderer checks if every field is filled in, and displays a message accordingly.

You need to a flashplayer enabled browser to view this YouTube video

Rob Van Roey worked on support for multimodal user interfaces for his Master’s thesis. He implemented a new X+V backend for Uiml.net, which is thereby the first backend that renders to another XML document. The movie shows a multimodal user interface for controlling a smart home, in which Rob turns off the lights and turns on the alarm after saying the correct password.

You need to a flashplayer enabled browser to view this YouTube video

Finally, Jan Meskens created a UIML design tool on top of Uiml.net. The workspace of the tool is generated dynamically according to the loaded vocabulary. The movie shows the basic working of this tool. Jan joined our ranks after his studies, and will continue to improve the UIML designer.

You need to a flashplayer enabled browser to view this YouTube video

Ingo’s code is already available in a separate Uiml.net branch, and will soon be merged into the mainline. Jan is also working on integrating his work. When I find some time, I will probably merge Rob’s code as well.

New Master’s thesis students

Tuesday, October 16th, 2007

This year I am supervising the theses of Neal Robben (together with Geert Vanderhulst) and Tijl Lathouwers (together with Carl Bruninx).

Both theses are related to UIML and Uiml.net. Neal will be extending Uiml.net to support distributed user interfaces, while Tijl will be looking into adding metadata to user interfaces and using this metadata for adaptation purposes (e.g. personalization or device adaptation).

More information can be found on Neal’s blog and Tijl’s blog.

Annoyances while creating Visual Studio projects for Uiml.net

Thursday, June 28th, 2007

I am currently moving Uiml.net towards MSBuild files (a.k.a. Visual Studio project files) and experienced a few problems such as a restriction with Compact .NET’s OpenFileDialog.

After a bit of Googling, I finally know why I can’t seem to get the OpenFileDialog to look for files in \Program Files: it was just designed that way (do a quick search for OpenFileDialog on that page). Apparantly they wanted to help users organize their files, by restricting them to the \My Documents folder. Too bad they didn’t think of what developers might want to do … I want to allow people to try UIML examples when they click the “Select UIML file” button. Of course these files reside in the Uiml.net application directory itself, so I need to allow them to pick a file outside of the My Documents folder. If that wasn’t bad enough, it further restricts you to one level of subfolders within the My Documents folder!

I also noticed MSBuild has no nice way of copying a file to another path then the one it is being referenced from. I orginally wanted to copy the front-end files and vocabularies in the root directory where the Uiml.net executable is placed. Since I didn’t find a solution, I made dedicated directories for these files, and modified the code to look for them in those directories instead.

If I could tell MSBuild to copy a file to another path, I could also solve the OpenFileDialog problem just by copying the Uiml.net examples to the My Documents folder, while the rest of the application would stay in the Program Files folder.

So my (hackish) solution is now to copy all examples first into a direct subfolder of the user’s My Documents folder. Not very elegant, but it works.

Updated Uiml.net homepage

Tuesday, May 22nd, 2007

I spent some time yesterday to update the Uiml.net homepage:

New homepage for Uiml.net

We set up a wiki somewhere last year to provide more information about our UIML research. However, since Uiml.net is the main focus of our research efforts, I reorganized the wiki to be more of a homepage for Uiml.net, while still listing our publications.

I am hoping to migrate all the content of the old website (still located at Kris’ homepage) to the wiki soon.

Lately I have been enjoying Launchpad for hosting my projects and Bazaar branches. It has integrated specifications (_blueprints_), bug tracking, some kind of forum (_answers_). I have moved Uiml.net over to Launchpad as well. Launchpad’s integration of Bazaar branches will be useful for managing multiple experimental contributions from myself, Kris or our Bachelor’s or Master’s students.

Finally I’m hoping to put out a more or less stable release of Uiml.net by September. More information can be found at Launchpad.

Uiml.net and Cassowary.net now in Launchpad

Monday, December 18th, 2006

I just registered Uiml.net and Cassowary.net in Launchpad.

Launchpad automatically creates a Bazaar branch from the CVS repository, which allows me to maintain my work in Bazaar, while still importing changes from the Uiml.net CVS tree.

Uiml.net news

Tuesday, July 18th, 2006

It’s been a while since I reported on Uiml.net. We set up a wiki recently, which will eventually replace the old project page. There is still some content to be added though.

This year we also had some students working with or on our implementation:

A question we receive frequently is if we can send people the Visual Studio workspace. It would be good to upload that to the Wiki, and maybe do a formal release in the near future.

Modifying a UIML interface from the application logic

Monday, June 19th, 2006

As I posted in an earlier article, Uiml.net allows us to connect the application logic with the user interface.

I showed that it’s possible to create a callback that listens to certain events, optionally only coming from a specific part. In these callbacks we can examine the part that has sent the event using the Uiml.net API.

But is it also possible to modify that part?

Of course we could operate on the widget-set specific user interface object (e.g. a System.Windows.Forms.Button), but that would require different code for each target widget set. Here is how we would change a button’s label when it is clicked using this approach:

public class ModifyTest
{
  public ModifyTest()
  {
    ...
 
    // connect only to the hello button
    uimlDoc.Connect(this, "b_hello");
 
    ...
  }
 
  [UimlEventHandler("ButtonPressed")]
  public void OnButtonClicked(Part sender, UimlEventArgs e)
  {
    // get the concrete UI object
    System.Windows.Forms.Button b = (System.Windows.Forms.Button) sender.UiObject;
 
    // change its text to 'Do it again!'
    b.Text = "Do it again!";
  }
}

Uiml.net provides a property setter interface that can apply any property. We only need the renderer instance in order to access it. We could use this API to modify the user interface in a generic way, if we rely on a shared class and property among all widget set vocabularies. In our example, the Button class exists for all vocabularies that Uiml.net supports, and has a label property in each vocabulary.

Let’s have a look at the generic solution. We only list the important code:

public class ModifyTest
{
  IRenderer _renderer;
 
  ...
 
  [UimlEventHandler("ButtonPressed")]
  public void OnButtonClicked(Part sender, UimlEventArgs e)
  {
    // create a label property
    Property p = new Property();
    p.PartName = sender.Identifier;
    p.Name = "label";
 
    // set its value to 'Do it again!'
    p.Value = "Do it again!";
 
    // apply the property on the sender part
    _renderer.PropertySetter.ApplyProperty(sender, p); 
  }
}