Java Touch-enabled in S60 5th Edition
As somebody may have already noticed, Nokia has launched Nokia 5800 XpressMusic. It is the first S60 device with touch screen support. Today also the S60 platform release supporting touch UI called S60 5th Edition was announced.
But how does touch UI devices impact Java applications? On last JavaOne conference Görkem and I had a presentation on the impacts of touch user inteface to the Java ME application development. Our slogan was “Beware of the thing”. The living hand pet of Addams family called “The Thing” gave us an idea that similar things will soon start poking your applications and they are probably angry if the app won’t respond in a proper manner. The message was that Java ME developers should start taking touch devices into consideration as more and more touch enabled Java devices appear in the market.
Touch UI introduces a paradigm shift in mobile UI creation. New paradigms create always discontinuity. Good user interfaces are designed totally differently for touch and for keypad-only devices. In touch UI one can and should use direct manipulation: objects can be manipulated directly with a finger rather than operations selected via menus. Utilizing direct manipulation fully makes the application UI code very different from a code designed for non-touch. On the other hand if it is a necessity for an application development that the same code with little specializations is used both for touch and non-touch devices then the application’s usability is nearly always a compromise. Developers must make a choice how deep specialization they want to implement in the application UI for touch and non-touch devices.
We were now asked to create a very simply one-slider statement of what touch UI means for Java developers in S60 5th edition. Here’s the message we gave:
Java - touch-enabled with backward compatibility
- New tactile feedback and direct manipulation capabilities let Java developers explore the touch paradigm to the full.
- All Java User Interface (UI) components manage touch events automatically.
- Pointer events and tactile feedback enable creation of impressive touch sensitive 2D/3D graphics.
- A handy on-screen keypad smoothes transition to touch-driven UI design by providing backward compatibility for existing Java applications.
So basically this means that Java UI components will implement the touch support automatically in the way S60 UI style defines the touch UI interactions. As an example, developer does not need to care on how a List element is going to be selected (single tap or double tap), or how to do focus changes between widgets in containers from tap events. However, developers need to care for their custom components, e.g. using Canvas class, how interaction works therein.
In addition, S60 Java will provide a level of compatibility for existing applications. By default for MIDP LCDUI applications we provide an on-screen keypad for Canvas that allows users to use existing MIDlets designed only for keypad interactions. The keypad allows user to create key events pressing the navigation/select buttons on touch screen. User can also remove the keypad from application’s settings via Control Panel. There is a way for a MIDlet developer to get rid of the keypad via JAD/manifest attribute. And in fact it’s highly recommended that all applications that are really designed for touch UI and targeting touch enabled devices would remove the Canvas on-screen keypad. Then application can utilize the whole screen space but it must allow user to operate the application without support from keys. The S60 device just launched is for example one without any ITU-T keypad (only Send, End & Apps keys).
Following screen shot shows the Canvas on-screen keypad for compatibility:
One additional aspect of S60 5th Edition is the support for tactile feedback. The sensory feedback is an important aspect to think of when designing applications to touch devices since the devices are mostly used by interacting with a completely flat touch screen. Compared to real physical keypad keys, the flat screen doesn’t provide any sensory feedback via fingers. The idea in Tactile feedback is that it tries to provide some sensory feedback on touch operations via device’s piezo electronic motor. S60 platform UI supports some set of tactile feedback styles. The UI components in Java will provide the same tactile feedback as the same components in native applications. For applications custom components we have implemented a new TactileFeedback class to Nokia UI API. It allows application to register tactile feedback areas to the platform. The feedback engine attached to the window server will then generate the wanted feedback effects when user touches those areas of the custom component.
The upcoming Java Developer’s Library release for S60 5th Edition will document all of the above features and APIs in more detail. And of course touch support is also implemented to the S60 5th Edition SDK.
One thing should be remembered when developing in SDK’s S60 emulator and using desktop mouse to emulate touch events. A user’s hand is not the same as a mouse. The tip of the finger is very large and it cannot touch one pixel at a time. So application’s layouts need to take this into consideration. Applications need to reserve big enough touchable areas!
Hi!
That looks and sounds good. Do you think (and I hope) that in the future more devices, e.g. N-Series mobile computer, comes with this S60 5th editon release?
Kind regards,
Thorsten
Hi Aleksi,
Will it be the same java support in S60 5th editon release as in S60 3.2 editon release?
I couldn’t found support for eSWT at this page: http://www.forum.nokia.com/devices/5800_XpressMusic
Regards,
Ove
Thorsten, S60 releases often are used in more than one product, e.g. if you look now 3rd Edition FP 2 device list from Forum Nokia you see a lot of them already. S60 is a licensed technology so it’s not just Nokia which will be using it.
Ove, good catch this is an error in the Forum Nokia device specification. I contacted them to fix it. S60 5th edition of course is backwards compatible with 3rd Edition FP2 and thus eSWT UI toolkit and IAP Info APIs are also there in.
br,
-Aleksi
Will be added in the device spec after the weekend. Sorry for the omission.
For Edition level API support, see this wiki page
http://wiki.forum.nokia.com/index.php/Java_ME_API_support_on_Nokia_devices
Hartti
Aleksi, could you clarify the leftmost screenshot picture? Does that mean a midlet that is not in any way optimized for S60 5.0 can’t get full input capabilities (QWERTY), there is only those A,B,C and D buttons? Isn’t that quite bad?
Jukka: I’m sorry I have been away and had to focus to other stuff so replying late…The above picture is from LCDUI Canvas which is a low-level component where in keypad devices MIDlets get only low-level key events (keyPressed(int keyCode)). Then of course in devices without keys one doesn’t expect users to be able to deliver key events in Canvas. When MIDlets use normal editors like LCDUI TextField/TextBox or eSWT Text they will get automatically all the on-screen input capabilities that native applications have (QWERTY, HWR, Alpha-numeric entry) with all the locale specific input methods like Chinese predictive inputs in China, Japanese input in Japan, QWERTY keyboards based country/language specific layouts etc.. The Canvas API in MIDP specification itself isn’t designed to support such text input but works only on low-level key level and that’s the reason you don’t have text input support there.
However, since many existing MIDlets were designed keypad devices only in mind they rely entirely on key events in Canvas we decided to implement the on-screen keypad for Canvas which is on by default. This provides means for user to send the same navigation key events (and MIDP mandated Game Actions) from on-screen buttons as ITU-T devices do. We surely considered also full ITU-T key set in this compatibility on-screen keypad but found out quickly that it is a can of worms to do a good design for it. First of all we didn’t want to start emulating full text input in Java Canvas which isn’t anyway possible with the existing API as Canvas has only low-level key event callbacks and not high-level text input facilities. Full QWERTY is out of the question with physical display size what we have: if you look on the full QWERTY keyboard in editors you see that it takes full screen so there wouldn’t be any space for Canvas content itself, even the alphanumeric editor is full screen. We also didn’t want to have any floating windows on top of Canvas as they are really hard to use considering what’s our point: emulating key events for existing MIDlets.
Some MIDP applications have traditionally implemented their own text input facilities on Canvas which require full ITU-T key set to be available. This relies on assumption of device HW and these applications are broken in touch only devices without keyboards. These kind of applications have two choices: 1) use the editor APIs available in UI toolkits (LCDUI TextField/TextBox and eSWT Text), 2) do own text input emulation using touch screen. If you think 2) and what the apps are doing currently they are not far away from each other as currently the apps are emulating their own text input also in ITU-T devices.
Why apps are doing the text input emulation in the first place on Canvas? That is because there’s no good editor API in MIDP for Canvas. We have earlier provided a better toolkit called eSWT which solves this issue as there’s freely positionable Text editor widget. You can place that also on top of (eSWT) Canvas. Now we are also working on some new API extensions for LCDUI for text input but they take some time to appear. We’ve also proposed such to next MIDP spec version but I don’t know if will make it there.