Monday, 17 December 2007

Location based Java Micro Edition

My first encounter with location based services was in 2000 when working for a telco back when the internet bubble had not burst and life was good! Several months later the telco shelved the technology due to lack of enthousiasm of customers to pay for knowing where they were. And that is probably still the case today. Unless you are offering something fantastic (like on-board car navigation for example) nobody is going to pay for location based content.
Nonetheless the last year has seen a revival of location based chatter on the web and i don't seem to be able to escape from it. Last year we field tested location based java micro edition software on regular phones and bluetooth gps receivers. Then this year three seperate companies asked us help them with their location based software. All three were having difficulties with the technology so it's safe to say it's not as easy as some might portray.
First off, there is very little on the web in the way of working, open examples that you can take apart and examine. Countless sites have a code fragment that works for them, on their hardware. And then there were a bunch of authors who wrote about something they made in the Sun WTK emulator but never tested on a real phone. These 'look at me, my boss gave me time to read the tutorials and now i'm a guru' authors always annoy me because they always say things that are wildly wrong. For example at a local Java conference in the Netherlands there was someone lecturing on location based services who made the bold claim that 'Your mobile will determine which location based services are available and use them, so if you do not have a GPS receiver it will use the telecom networks positioning information. And yes you can actually use that right now!' not knowing that at the time there were no network operators that had that service switched on.

Anyway dragging myself back to the topic; part of the enthousiasm on the web has been due to the fact that a number of device manufacturers are starting to implement the location based API (JSR-179). It's been around for a while, but there used to be only two phones (guarded in finland by dragons) that had any sort of implementation. And needless to say they both had a different implementation all together. But Nokia and Sony Ericsson have been introducing enough models with a JSR-179 implementation to make it fairly widespread in the high-end segment of the phone market (in Europe anyway). This is a good development, but there are still a few problems with the Location API JSR-179 implementations:
  1. There are few phones that actually have an inbuilt method for location determination. The Nokia N95 must be the most popular, but most phones just do not have GPS hardware.
  2. The hardware that does have inbuilt GPS chips and antenna's tends to be exactly the hardware that is running an operating system that is hard to run java on. For example all the windows mobile devices that are popular for (among other things) in-car navigation. And what about all the Garmin, Becker and TomTom hardware that people are buying in busloads. Ever tried to run Java on one of those?
  3. A JSR-179 implementation dictates CLDC 1.1. So if you are developing for a number of devices here is another fragmentation issue. If you want your software to install on a CLDC 1.0 device (and there a tons of them) you cannot use JSR-179. So now you have two code-bases.
  4. The JSR-179 implementations are still new and therefore buggy. For example an implementation we tried automatically tried to search for GPS bluetooth devices the first time we tried it, but had no facility to change which GPS bluetooth device was selected. Great if you buy a new GPS receiver!

Generally speaking there are three strategies for getting location information in Java Micro Edition:

  1. Use the location API
  2. Use the bluetooth API (JSR-82) and bluetooth serial port protocol to read from a GPS bluetooth receiver.
  3. Use the serial port protocol to read from a COM port on the device.

Each strategy requires completely different code. I've tried em all, and they can all be done, but not on all hardware.

It's late... I'll get back to this soon.

Friday, 23 November 2007

What does MIDP, CDC, CLDC etc mean?

What are MIDP, CDC, etc.

Someone posted a question as to what MIDP, CDC, CLDC etc are. At the cost of getting very technical we need elaborate on the java architecture to explain exactly what the differences are.

Java Virtual Machines
Java programs are compiled into a machine independent pseudo-code, that is called bytecode. To run this bytecode you need a so called virtual machine. A virtual machine is a device specific program that takes your bytecode and translates it to instructions the hardware understands. This seperation makes Java portable between operating systems and machine types. Most software developers only encounter the garden variety of Java Virtual Machine (JVM); the type you run on a PC or server. But this is where it gets nastier for mobile developers.

The three musketeers
There are three varieties of JVM:

  1. Regular Java Virual Machine (JVM).Used by Java Enterprise Edition (J2EE), Standard Edition ( J2SE) and Connected Device Configuration (CDC). Allows you to access native code on the machine it is running and does cool stuff like verifying your java bytecode to make sure it doesn't do anything it shouldn't (like format your hard drive). The Connected Device Configuratio (CDC) is intended for high end PDA's/smartphones and contains a fair amount of the standard Java packages.
  2. Kilobyte Virtual Machine (KVM). Intended for small devices with crappy processors and a pathetic amount of memory. To get the virtual machine to run on these devices they decided to rip a lot of stuff out: Most notably lots of useful packages and classes and the bytecode verifier. So programmers have to run a seperate verifier to make applications for this virtual machine. Also they removed the capacity to run native code. Java configurations that run on the KVM are called Connected LIMITED Device Configurations (CLDC). If you want to start programming for CLDC beware; programmers are left with a bare minimum of classes - its java in 1998 all over again.
  3. Card Virtual Machine (Card VM)They ripped so much out of this virtual machine that it can run on a smart card (i.e. in the chip on your credit card or on your SIM card). I've never tried it, and hope i never have to. Supposedly you can do really cool stuff with RSA encryption.

In summary: CDC refers to java that runs on one type of virtual machine (JVM) and CLDC refers java that runs on another type of virtual machine (KVM)

In the embedded and mobile arena they (the organisations that lumbered us with Java Micro Edition - i always envision them as committees sitting around formica tables drinking coffee from plastic cups) decided to make life easier by introducing profiles. A profile is aimed at applications of a specific type. The main profiles are:

  1. Mobile Information Device Profile (MIDP). Translates as Java for cellphones. Manages the lifecycle of programs on the cellphone: installation, security, startup, running, pausing, stopping etc. The main entity in this profile is called a MIDlet. A MIDlet is the thing that you install, run etc. Security is writ big in this profile. Anything remotely interesting (like doing an http request to a server) will result in the cellphone asking the user to verify they want this to happen.
  2. Personal Profile (PP). Translates as Java for Windows Mobile / Palm devices. The main entity in this profile is called an Xlet. An Xlet is the thing that you run etc. Installation and security are not managed much in this profile. You can copy across your class files and run them using the Personal Profile virtual machine.

Wednesday, 14 November 2007

Installing IBM J9 on windows mobile

This article is for people who want to run java software on their windows mobile devices.
Most java applications for mobile devices are so called 'MIDlets'. Midlets can run on most mobile phones and those Windows mobile devices that have java support. Your device does not have java support which is why you are reading this article :-). The IBM J9 WEME java runtime environment works on Windows Mobile 2003 and above. The smartphone version will work on smartphones and PDA's.
The documentation by IBM is aimed at intermediate level software developers and definitely not at beginners or end-users.

The following assumes you want to run MIDlets on your Windows mobile device:

Installation Steps
  1. Download IBM J9 WEME for windows mobile smartphone edition. The page is here:, you need to register and then get the download.
  2. If your are not sure which processor / windows version you have, choose this one: Windows Mobile 5.0 Smartphone Edition, ARM, CLDC 1.1, MIDP 2.0, iMate SP5m, Dopod 577w. Its the last entry in the first table on the page. I tried it succesfully on three devices (HTC/QTec/MDA, HP iPaq and MIO). Once you have registered and got to the download bit of IBM's site you will most likely get this file: ibm-weme-wm50-sp-arm-midp20_6.1.0.20060727-102926.exe. (If there is a newer version on the site, check that the first 27 characters are the same; ibm-weme-wm50-sp-arm-midp20....). The file is about 80MB but don't be alarmed, you only install about 3MB on your device. Why IBM chooses to include 77 MB of stuff you are not interested in is a mystery to me.
  3. Run the file, and let the installer do its stuff. It will install a directory structure on your PC.
  4. Go to the installed directory (on my PC it looks like this: C:\IBM\WEME\runtimes\61\wm50-arm-sp-midp20) and open the zip file: It contains the runtime for windows mobile. Extract the directories bin, lib and examples to somewhere on your PC.
  5. Create a directory J9 on your windows mobile device. It can be on the device itself or on a storage card. Copy the directories bin, lib and examples to the J9 directory on your windows mobile device.
  6. Open file explorer on your device and go to J9/bin. Run the file emulator.exe. If you get an empty screen with 'Install' and 'Action' menu's at the bottom of the screen the installation was succesful.

Now run your first MIDlet

Run emulator.exe on your windows mobile device (see above) and choose 'Install'. Enter a URL (a what? i hear you say) that refers to your MIDlet. The syntax for installing files that are on your windows mobile device is: file://[path to file]. So to install the example MIDlet included in the installation enter file:///Storage Card/J9/examples/GolfScoreTrackerSuite.jad (this assumes you have installed the java runtime in a directory J9 on your storage card).

Thats it for now. A couple of things i will be writing about later on are:

  • How to create shortcuts that run a midlet directly.
  • How to tweak the runtime environment to get rid of the annoying security questions i.e. "XYZ Midlet wants to connect to a website, do you want this to happen?".
  • How to get midlets to access the file system.

Monday, 29 October 2007

Which Java Micro Edition Profile for windows mobile?

Which profile/ configuration
If you've just started with java micro edition you will most likely be confused by all the configurations and profiles that are available. How do you choose which profile / configuration to use? Which configurations are widely available and which can you use on your shiny new windows mobile device that you are hoping to deploy a java app on?

Step 1. Choose a profile / configuration.
If this is your first java micro edition project, choose the CDC1.1 configuration with Personal profile 1.1. This profile will allow you to use regular java programming skills (similar to desktop application programming). For example you can use SWT for the user interface.
If you are looking to deploy your application on non-windows-mobile devices then go with the CLDC1.0 configuration and MIDP2.0 profile. At the time of writing CLDC1.0 / MIDP2.0 is the most widely available configuration an profile and is pre-installed on over 95% of telephone models.

Step 2. Check your device has suitable java support
Before you start any development at all, get a demo application from somewhere and try deploying it to your target device. Documentation for many devices is often so hard to find that you only really know you can use the configuration/profile you want if you have seen it running on a real device. The Sun Wireless CLDC/MIDP2.x Toolkit comes with a set of demo applications which you can use to test for MIDP2.x compatibility.
Most mobile phones have support for java micro edition CLDC1.0/MIDP2.0 or higher. Many windows mobile devices have no java support at all. Fortunately you can install IBM's J9 WEME runtime on windows mobile devices.

Step 3. Install a runtime if necessary.
If not pre-installed, the only real option (i've raved about this earlier) for java on windows mobile is IBM's J9 WEME runtime. J9 comes in several flavours for windows mobile. The main distinction is the CDC/Personal profile flavour on the one hand and the CLDC/MIDP flavour on the other. If you chose CDC/Personal profile, you need to download two runtimes; one for the device (wm50-arm) and one for test and debugging on your desktop (win-x86).
I find many people get confused what to download and what its for. So here's an example:
  • Runtime for windows mobile 5 devices with ARM processor (HP, HTC, Qtec, MDA etc) is in: ibm-weme-wm50-arm-ppro11_6.1.1.20061110-161633.exe
  • Runtime for windows xp desktop is in: ibm-weme-win-x86-ppro11_6.1.1.20061110-161633.exe

Blink and you will miss the difference. They both have (different) installation documents with the same name 'install.pdf'. It boils down to installing both on your desktop, unzipping one of the installed zip files (different one for each runtime) and for the device runtime copying a few directories to the device. I'll be writing more on this later this week

Step 3. Forget the optional packages (for now)
A really really annoying part of development for java micro edition is the fact that a lot of sexy stuff is in optional packages. Optional in reality means one of the following:
  • you can't depend on it being on a device
  • two devices can implement it with non-trivial differences
  • the really sexy stuff is generally only implemented on 3 phones, which are kept in a dark loft in finland and guarded by dragons.

So for now forget the location based api, the multimedia stuff, bluetooth, web services and most other mouthwatering things. Come back to them when you are a seasoned dragon-slaying java micro edition veteran.

That's it for now.
When i have time i'll be writing about how to install and run IBM's J9 on a desktop and on a real device. This defeats a lot of people because the documentation doesn't provide any useful examples. Which is a pity because it is otherwise an excellent product. When i get round to it i'll also post how to deploy an application and run it (this may sound stupid, but it took me a whole day to clear this hurdle - and judging from the forums i'm not the only one).

Tuesday, 23 October 2007

KJAR has a virus

A few months ago i started using KJAR to compress jar files. It worked well, eliminating 10% space of a 110KB jar file. But it turned out to have a trojan horse virus in it which my anti-virus software eventually dealt with. So beware!

If anyone has an uninfected version i would like to hear from you.

Here is the site (which is otherwise excellent) where i got the infected KJar

Java Micro Edition on Windows Mobile

For one of our customers we were asked to make a mobile application on the hardware of their choice. In this case a hardened PDA with windows mobile on it. The devices are great, you can drive your car over them or drop them in the bath and they still work!
Anyway, examination of the devices revealed that they had no Java support. So did we decide to take the MS Visual Studio route? Of course not, we eat, drink and breath Java, so we're not going to let that happen are we....

A quick search of the net revealed some promising candidates for virtual machines that can be installed on windows mobile. Some time later we were narrowing it down to one (1) alternative: IBM's J9 Websphere Everyplace Micro Edition. That's all there is in this space at the moment. Nothing else. Zilch. All the others have disappeared, are limited time trials or been bought and are now kept under lock and key. Luckily you can download an unlimited trial version from the IBM website. Download it quickly because i have the feeling it won't be there for long. Register and search for "IBM J9 WEME". You should get a plethora of download options. More on that later.

Although i'm no microsoft expert there seem to be a definite advantage for software vendors to using java over the .net compact stuff:

  • The .net compact framework (or whatever it is that microsoft calls its application development stuff nowadays) seems to be slightly different on Windows CE, 2003, 2005 and 2006. We tried a demo application that was written for 2005 and it didn't work on 2003. So if your client decides they will be using windows mobile 2003 you have to change your software. Presumably this applies to windows mobile 2006 as well. Of course there are different java versions and compatibility problem in j2me apps, but you can fix that by upgrading to a new java VM which is nowhere near as drastic as upgrading the whole operating system.

We eventually got both the emulator and device runtime working for both MIDP 2.0 and Personal Profile and even got SWT running. Our application does network connections, uses the touchscreen, has a low level user interface and uses the Midlet lifecycle methods actively. All of these things were non-trivial in the IBM virtual machine.

I'm out of time right now. In the coming weeks i'll be writing on how to install it, how to run it (even harder) and what things don't work or work in a completely unexpected way.

Why this blog?

The last two years i've been looking for information about mobile web and mobile java.... and been quite frustrated about what i found, or rather didn't find.
My first encounter with all things mobile was in 2000 for a Vodafone / Vivendi startup. In those days it was really really hard to find any reliable information about mobile technology. So the company i was working for burned humungus amounts of cash to find things out by trial and error. Recently i started a company geared towards mobile web and applications and i was expecting the mobile software industry to have matured and opened up in the way enterprise software has in the last 5 years. But to my amazement this is not the case. It is a little easier to find your way around the mobile circus than it was in 2000, but i think it is still much too hard. While giving a course on mobile technology last month i realized that a lot of the stuff i was telling the students is excruciatingly hard to find on the net. Sure there are some forums you can look at, but if you work in mobile software development chances are you will be using several technologies, each equally obscure and undocumented. Wouldn't it be neat if the information you needed was bundled somewhere.

So here is my attempt to bundle useful information for software developers and users of mobile software. I will be adding stuff slowly but on a regular basis. I'm hoping it will save a lot of people a lot of time and frustration.

Happy reading,