Components, systems, subsystems, entities...*collapses*

So I've recently been reading into various articles and forum topics on Component-Based design, or Entity Systems (ES). This concept was probably first used in games when the first Dungeon Siege came out. Apparently it's closest neighbour is possibly that of Aspect-Oriented programming, which I know zero about. At first it seems like a complicated idea, but after I looked through a bunch of info on the net, I think I got the basic idea. Here's my attempt at giving an eagle-eye view of what it is, or rather, what I think it is in the context of a game:

You have three major concepts: entities, components, and systems.

Entity

An entity is a collection of components. These components describe how an entity functions. In terms of a game, an entity is a single instance of an object that exists in your game. So if you have two tanks of the same "type" in your game, you have two entities. Entities do not store data or contain logic. They are simply a means of identifying something.

Components

Components are the basic building blocks for entities. They store information and contain the functionality specific to their existence. For example, you might have a Renderable component in your game which contains all of the logic necessary for the rendering system to display an entity on the screen.

System

Systems have their own set of logic specific to the components that correspond to. I think it is technically possible to implement all of the logic in the system and the component simply be a data container, but I don't think I would like such a design. Often there is a [nearly] one-to-one correspondence between systems and components. You may have a RenderSystem to store all Renderable components and display them when necessary in your game loop. You could also have a PhysicsSystem, AnimationSystem, and so on.

That's my stab at defining those three concepts. The main task in designing your system/game/whatever is to follow the "separation of concerns" idea, each "concern" often mapping to a component/system duo.

So what do you get out of this? Well one thing, it's great for a large development team because, for example, the programmers can offer the game designers a set of components that they can freely use to construct a multitude of objects. These components can be added to an object at runtime to give a dynamic design.

For example, let's say you have a Targetable component which allows the player to select an entity as a target for his/her attacks. Well, let's say we have an NPC that is neutral to the player, but the player does something to provoke the NPC into a battle situation. All we have to do now is attach the Targetable component to this NPC and we're done. The logic surrounding what the player can and cannot attack is simply encapsulated in that component. In the classical OOP approach we would have to define an ITargetable interface that says whether or not the object described by the class is targetable. With ES the existence of a Targetable component on an entity implicitly tells us that the entity can be targeted. With the OOP style, we have to store a variable to say if that object is targetable or not at a specified time.

What one finds in this design is that the deep hierarchy often found in an OOP design is now almost completely flat. Object composition/aggregation is now one's guiding principle. Is this a good thing or a bad thing? One difficulty that often arises is inter-component communication, generally within entities. One could have these components store pointers/references to each other, register callbacks with each other, or use some messaging system to communicate. An example of such communication that seems to often arise is with a physics component. When updated, often animation/spatial components need to be notified of the result.

Well I'm really not sure at the moment whether this design is good or bad, but I think I'm going to give it a shot and document what I find. Things are slow-going with my game [engine] project because it's something I just pick at from time to time. I have no intention on getting a product out the door right away, so I use it to play with new ideas.

The reason I posted this entry was to look for other people's thoughts on this design style, and to give some concrete pros/cons from one's own experience. I'd love to hear about anything surrounding this topic, which includes other design patterns and styles that one found useful in one's own project.

Pointgrey Cameras

For anyone who doesn't know my research, I'm looking into stereo vision algorithms in an underwater camera array. What I'm hoping to do is a rough scene reconstruction that has improved results over blindly using an existing stereo algorithm.

Now on to what this post is about. I found a little subtle and, as far as I can tell, undocumented feature of the Pointgrey cameras. If you use the FlyCapture API then you would normally call flycaptureStart and flycaptureStop to start/stop grabbing images, respectively. For our purposes, we only need to grab single shots from the camera array, not streamed video. On top of that, we want the shots to be (for the most part) synchronized across the whole array.

Here's the twist, the start/stop calls aren't really what starts the "image grabbing" process, but simply powering up/down the camera (via the CAMERA_POWER register). The start/stop calls appear to only lock down/release a DMA context for the purposes of streaming. That means, if you have a firewire card with the TI chipset you can only start 4 cameras simultaneously.

So how do we grab a synchronized set of images using this knowledge? Well this is only applicable to the Grasshopper line of cameras since they have an on-board framebuffer that you can control. Here's what we do:

1. Place the cameras in "image hold" mode by setting bit 6 to 1 in the IMAGE_RETRANSMIT (0x12F8) register. This will store images on the camera.
2. Power up the cameras by setting bit 31 to 1 using the CAMERA_POWER register.
3. Simply wait around long enough for the frame buffers to fill up
4. Put each camera in trigger mode by setting bit 6 to 1 in the TRIGGER_MODE (0x830) register. What this does is prevent any more images from getting stored in the frame buffer.
5. For each camera
   1. Start the camera so you can transfer data over to your system
   2. Set bits 24-31 to 0x01 in the IMAGE_RETRANSMIT register to transfer an image.
   3. Stop the camera

This works great for using the cameras in a non-streaming context where you only have a single firewire bus/system to work with. If you want the images to be synchronized, be sure to set them all to the same FPS, and enable image timestamping (FRAME_INFO register, 0x12E8). Now all you do is find the set of images across all cameras which are closest together.

One other subtle thing I found with is that if you start the camera with the same dimensional parameters, but with a different bandwidth parameter, the on-board frame buffer will be flushed. Anyways, that's it for this post. I thought it would be nice to post this information just in case someone else out there has had, or will have, a similar issue. Cheers!

First Look

So what's this score editing application I've been talking about in my posts? We've decided to give a pre-beta teaser. Minus a couple of rendering issues, here's what she looked like a couple of days ago (click image for larger preview):



For our first release we focused on guitar tabs, and making sure the application as a whole works together properly. Unfortunately this release will be a private beta, so everyone will have to wait. I think we have a pretty good app for ~4 months of work (sparse work too, since we're both busy with school and work). Hopefully I've tweaked your interest!

Java Annotations: A [Somewhat] Brief Introduction

So in my last post where I described a messaging system we implemented, I also mentioned our use of annotations. I thought it would be appropriate to write a follow-up post with a brief introduction to them, so here it is. I'm going to talk about annotations in the Java sense, but a lot of this propagates to other [reflective] languages too.

Annotations are often defined as "notes of explanation or comments added to text". In Java, one can regard an annotation as metadata attached to code, a piece of code describing code. Well, the first question that one might ask is "Why do I need this when I can just use comments?" and indeed, this question makes for an excellent starting point when writing an introduction. Well, probably the main reason to have annotations over comments is that annotations are part of the language, with a specific syntax, which in turn allows parsers to easily understand them. Comments, on the other hand, could be in any form and would be a huge mess to understand by a compiler, unless a certain standard was specified.

"So what good are they for?" you might ask next. Well, they can let you do some pretty neat stuff. First I should describe the three flavors of annotations in Java:

source code

Present at the source code level only and will be discarded by the compiler during compilation. These guys are useful to give hints to compilers and programs as to the nature of the code itself. For example, one may have seen @Override or @Deprecated annotations when using Eclipse or other code. The former specifies that an error should be produced if the method doesn't override one in a super class, and the latter indicates a class, method, etc. as deprecated.

class

This is the default flavor. Compilers will produce bytecode containing these annotations, but you probably won't be able to access these during runtime. Useful if you are doing bytecode analysis of code.

runtime

To me, possibly one of the more useful flavors of annotations for developers. These annotations can be requested at runtime, which allows you to do neat tricks. The reason this is possible is due to the reflective nature of Java. You can access this through the Class.getAnnotations() method

So how do we create an annotation? Well, it's pretty straightforward. Note that I'll be using the messaging system from my previous post as an example. Here's what the Message annotation looks like:

@Retention(value = RetentionPolicy.RUNTIME)
@Target(value = {ElementType.FIELD})
public @interface Message {
    Class<?>[] signature() default {};
}

So the first line says we want to be able to retrieve this annotation at runtime. Note the use of the @ symbol here. This is the notation used for annotations. The second line says what type of things this annotation can be applied to. In the above example, it can only be applied to a field in a class, but not methods, or classes themselves. Note the @ symbol before the interface keyword in the third line. This is how we define an annotation. Finally, the fourth line specifies the one and only property in our annotation, and that's an array of classes that specify the signature of the message (i.e., the types for the data that will accompany a message). We specify the default signature to be an empty array. What's interesting in this example is that we used annotations to describe an annotation itself (@Retention and @Target describe @Message).

For our messaging system, the field itself is a static member that is a String, and that string defines the name of the message. For example,

@Message(signature = {String.class})
public final static String MYMESSAGE = "myMessageName";

describes a message with the name "myMessageName" which sends a String argument to all receiving functions. If we wanted to, we could have defined a second property in the annotation for the message name. In our message delivery class, we can then loop through all the fields in a class to register messages like this:

public void registerSender(Class<? extends MessageSender> sender) {
  MessageData msgData = getData(sender);
  for(Field field : sender.getDeclaredFields()) {
    if(field.isAnnotationPresent(Message.class)) {
      if((field.getModifiers() & Modifier.STATIC) == 0)
        continue;
   
      Message msg = field.getAnnotation(Message.class);
      msgData.addMessage(field.get(null).toString(), msg.signature());
    }
  }
}

Note that, for simplicity, I excluded the try/catch blocks and log messages in the above. A fairly straightforward piece of code: for each field in the class, if the field is static and has the Message annotation, we add the message to the set of messages we understand. This is far more convenient than having to register each individual message. For message receivers we have a ReceiverMethod annotation that I won't explain, but it looks something like this:

@Retention(RetentionPolicy.RUNTIME)
@Target(value = { ElementType.METHOD })
public @interface ReceiverMethod {
  // Special message name which allows catching all messages from a sender
  public static final String CATCHALL = "<<all>>";
 
  // Properties
  public Class senderClass();
  public String message();
}

We can then do something similar to the registerSender method above to register our receiver. So that's my quick introduction to annotations. Maybe you can find other interesting ways to make use of these little critters in your own programs.

Listening In

So it came to our attention recently that our application was making abundant use of the Observer/Listener pattern. For those not familiar with this pattern, you'd use this guy when you want the outside world to know about state changes in an object. This pattern is used often when developing with various architectural patterns, such as Model-View-Controller (MVC). Other examples, in Java, include many of the components in Swing, java.util.Observable, and java.beans.PropertyChangeListener.

I personally had issues with the extensive use of the listener pattern for us. Every time we added a new data class, we'd have to rewrite code for storing and notifying the listeners. Note that some of this could have been factored out into its own class. Nevertheless, if we ever changed the listener interface, change had to be made for everyone using that interface. Since we use Eclipse, this isn't too big of an issue, but it still bugged me a little. The final issue was, sometimes things needed to listen to events on EVERY instance of a data class, not just a specific instance. To make our lives a little easier, this required us to create a new type of listener that would be statically available (read: like a singleton).

My alternative was to create a messaging system. Before I started this messaging system, I thought about design so that I could generalize it so that it may be of use to others. A couple of the main design decisions that came up were:

• There should be a concept of a message sender, receiver, and a delivery system to coordinate message sending and receiving.
• There should be registration facilities to allow the system to become as type-safe and interface-like as possible.
• Java annotations (retention set to RUNTIME) will be used to define messages and receiver methods
• When registering a specific receiver instance to receive messages, weak references should be used so that a) the outside world doesn't have to concern themselves with unregistering the instance and b) considering (a), so that the garbage collector can destroy that instance (when necessary).
  

A test program I wrote up that uses this message system looks something like this. I personally find this a reasonably elegant system, for one that uses reflection. So what do we get out of this? I'll start with the cons (that I can think of) followed by (what I consider to be) the pros:

• Cons
  • We lose a lot of compile-time error checking
  • We introduce some overhead, mainly due to using the reflection API
• Pros
  • Adding or removing messages (generally) will require less work elsewhere in the code
  • Receivers only need to implement the messages they want to receive
  • Receivers are not required to name their methods as per an interface
  • Receivers can define what I call "catchall" methods, methods that accept all messages from a specified sender (this could also be done using the observable/listener pattern too, but I believe it would be a little less elegant)
  • Receivers define an accept method which allows them to dynamically control which instances they receive messages from

Currently I'm holding on to this until I feel it satisfies the needs of our score editing project completely, but after that I think I'll release it to the public so that someone else might find some use out of it.

Becoming a Jedi, err...JNI Master!

I myself am far from being a JNI master, since I only started doing my first JNI a few days ago. The problem we were having is that there is no way in Java to get some native behavior on OS X. For example, the closest to native you can get with Open/Save file dialogs is what you get from java.awt.FileDialog, which isn't very much.

Since we're developing in Swing it is key that if we do anything, it should be wrapped up in a neat little package that mimics existing Swing components. Not only that, but we want the behavior of any wrapper classes to be the exact same, or very similar to, the corresponding JComponent. Although it's far from being ready for public release, I currently have a pretty solid JNI library for the Open/Save dialogs in OS X. It's all wrapped up neatly in CarbonFileChooser, an extension to JFileChooser.

One issue that arose, at least for OS X, is that there are various threads that have to run independently: the AWT/Swing thread, and the AppKit thread. Hence, when one makes a native call that is going to be doing some GUI stuff or event-oriented callbacks to the AWT/Swing thread, one has to forward things along to the AppKit thread. This can be done via the performSelectorOnMainThread method. To achieve modality, I use the following code:

    public int showSaveDialog(Component parent) {
        dialogOpen = true;
        result = CANCEL_OPTION;
       
        cc_showSaveDialog(parent);
        while(dialogOpen) {
            try {
                Thread.sleep(100);
            } catch(InterruptedException e) { }
        }
       
        return result;
    }

What happens is that when the dialog is disposed, JNI calls are executed to set the dialogOpen variable to false, breaking the loop. I've been debating trying out wait() and using JNI to wake up the object instead, but for now the above code gets the job done.

Eventually I'm going to release this code library to the public so that anyone can use it, and eventually I'll open-source it (when I'm too lazy to keep maintaining it). The beauty of this, if one is using Swing, is that you really don't change your code at all. Just create the CarbonFileChooser class and you're good to go. Right now things are simple, so just some basic things can be done. You can do the following:

• Set the initial directory
• Add choosable filters, set the initial file filter, and get the chosen file filter once the dialog is disposed
• Get the selected file to open/save

And some things left to do:

• Multiple file selection and retrieval
• Show no filters if the "All Files" filter is the only one
• Setting the initial file name for save dialogs
• Update dialog to unselect files, as necessary, when filter changes

Java: not always so cross-platform

There's no doubting the fact that Java really makes one's life far easier in general when creating a piece of software that works on various platforms. But even with such power, there's always little things that really can make one's life a pain when creating a piece of software that is intended to be high quality.

One of the major sources of pain in Java is cross-platform GUIs. It's nice that Swing works "out-of-the-box" on most systems, but it often suffers from lacking in a "native" feel, even when using the system LaF. Also, it's sometimes hard to take advantage of platform-specific features that could really open up a world of possibilities for your app.

For our own project we decided to create plugins for various platforms that need more than the standard library offers. I'll talk about our plugin system in a future post. When we distribute platform-specific packages (e.g., a Mac OS X application bundle) we will include the appropriate plugin to extend the functionality of the app for that platform. We prefer this method over checking system properties so that a release never "breaks" when one of these properties magically changes. Anyways, some functionality we require includes:

• Querying what platform we're on. The application does not suffer when the platform is unknown, but we can provide specific features and an enhanced UI based on a known platform.
• Directory services. We want to know where the user stores their documents, where is their desktop, etc.
• Platform-specific GUI components.

So nothing too interesting to talk about in the first 2, but for the last one I can provide a little snippet of our default behaviour:

    public  T getSwingComponent(Class componentClass, Object... args) {
        //
        Class[] classes = new Class[args.length];
        for(int i = 0; i < args.length; ++i)
            classes[i] = args[i].getClass();
    
        //
        try {
            return componentClass.getConstructor(classes).newInstance(args);
        } catch (SecurityException e) {
        } catch (NoSuchMethodException e) {
        } catch (IllegalArgumentException e) {
        } catch (InstantiationException e) {
        } catch (IllegalAccessException e) {
        } catch (InvocationTargetException e) {
        }
        return null;
    }

So if we want a JToolBar we simply go getSwingComponent(JToolBar.class) and that's it. The implementation provides variable arguments behaviour to elegantly allow usage of all valid constructors. The only thing we lose here is compile-time error checking for incorrect constructor arguments, but this is not too big of an issue for us at the moment. Our platform-specific plugins can override this method to check for various Swing classes, and return an alternate instance of that class if necessary. For example, our OS X plugin returns a simple extension to JMenuBar that removes all icons from any menus added to it (to adhere to the Apple Human Interface guidelines). Another component I plan to work on for OS X a native file dialog (I don't like java.awt.FileDialog). This will probably have me diving into some JNI, which is good because I would like to play around with it. If I can get something decent working there, I'll definitely look into releasing the JNI code publicly.

A really simple technique but it really helps enhance our application and give an improved native feel over what Swing offers.

Beginnings

So I'm going to start my blogging off with an introduction to my project: a free cross platform musical score editor (which currently has no name). Our team consists of just myself and a classmate from my undergrad. We previously worked on a team project during our undergrad (a required course), so we were already familiar with team development.

For me, projects come in two flavors:

1. those I do simply for my own personal enjoyment, such as a game and,
   
2. those that fill a need for me, such as small scripts to get repetitive tasks done.

The musical score editor falls into category 2, but it is definitely an enjoyable project too. I personally found myself unhappy with existing free software for editing scores. Since I focus on guitar, I was looking at something with a simple interface to whip up a guitar tab and be on my way. Probably the best I could find is TuxGuitar, but it was far from a pleasurable experience. This established a need for me, one whose solution we will eventually share with others. So with a project idea, the next thing was to lay out some basic requirements:

• Cross-platform. I am an OS X user, and my friend is a Linux user.
• An interface that is both simple for the first-time user, but powerful for the more advanced users.
• Quick keyboard access to the most common commands to greatly improve throughput.
• Fully-featured. We want users to be able to do just about anything and everything they'd want to do with their musical scores. Clearly this will take time, but it is our goal.

With these requirements in mind, we decided that Java would make our lives far simpler. We chose Swing over SWT for our GUI library, since we both know and enjoy Swing. Our goal is to eventually bring this project to a level comparable to that of commercial software. It's a big goal, but we're extremely motivated and really enjoy this project. Anyways, some things I plan to blog about in the near future:

• Java: not always that cross-platform. Various topics on producing code and user-interfaces that feel more native.
  
• Working with JNI.
• Developing a flexible and easy-to-use plugin system.
• Object-based rendering systems: the pros and the cons.
• Other cool stuff!
  

We already have a highly functional and [mostly] stable version of our score editor internally, but we want our first public release to really be something amazing. We have many incredibly powerful features planned, some of which we have never seen before in the area of score editing. Hence, if you're reading this entry you should stay tuned for some good stuff! I'd post a teaser screenshot, but everyone likes a bit of suspense :)

Let's get 'er goin'

So it has been awhile since my last post, but I've been keeping myself busy with research and a large personal project that a friend and I are working on. It's a musical score editor written in Java. I'm keeping this post short but I think I will start blogging about that project, the issues we face, and the decisions we make to resolve these issues. I'll also talk about various design decisions we make, and maybe even some other stuff too, like our rendering system. Hopefully my future posts will provide useful insight and information on various topics surrounding development in Java.