S60 Open to new features

I recently had some time to take a look at some of the improvements we want to make for future version of Carbide. One that I'm really interested in is the ability to build and use externally pre-built SDK source file indexes (aka PDOMs). New APIs have been introduced based on work by Andrew Ferguson over at Symbian and the rest of the CDT-land developers. I plugged these APIs into the Carbide builder and pre-indexed the entire '\epoc32\include\' tree for a Symbian OS source base. Now when projects are imported and an index is created for the project, the indexer can skip over the includes have all ready been parsed: you get the time improvement with no impact on navigation ability.

Some early benchmarks look promising on a couple of test projects:

Other than saving valuable processing time for tasks other than indexing the same files over and over there are multiple other uses for pre-built PDOMs. We'll be investigating those over the next development cycle.

In this next post about some of the enhancements we made to the Carbide.c++ UI designer framework to better support UIQ and specifically, the control collection, I will discuss how the linking between the layout-specific control data and the shared control data was achieved. Our main design goal was that the user not be bothered with the details. The worst thing we could have done with our design was to force the user to make this link manually.

Before I explain how we automated this, I'll start with some more background about how the UI designer framework is organized. The editor, as I explained in a previous post, consists of various views working together - the main editor view, the outline view, the properties view, events view, and more. It does not know anything about UIQ or S60. It only knows how to manipulate a design document consisting of objects in the model. These objects get their characteristics from libraries of components. Everything about UIQ or S60 or any other framework we may support in the future is defined in the components themselves, or through a few other extension points supported by the editor.

Components specify everything about how an object will behave in the editor and these are basically what the user sees in the palette. The palette consists of creation tools which are specified by those components that are compatible with the current design. Components are defined in xml and can point to various bits of code (java or javascript) to allow them to render themselves at designtime, layout their children, and implement many other interfaces supported by the editor. They also specify how they will generate c++ and rss source code.

Many enhancements to the designer framework take the form of additional interfaces that components can implement. So, to allow extended creation behavior we added command extenders. These allow a component to extend a command such as the command for creation of new objects from the palette. Extending the command allows the extended behavior to be atomic with the original behavior, so if the original behavior is undone, the extended behavior can also be undone.

The UIQ components do a lot of extending of commands, and specifically, the creation of extra objects and their linkage is achieved using this facility. But our design goal of making the sharing of controls in the control collection as simple as creating a completely new control required an additional enhancement.

Controls are created from the palette. When this happens, an object is added to the layout and another is added to the control collection. But to share an object from the control collection, the user would have to somehow select it in the control collection and make a new object the in another layout and link them together. This was too complex to do manually. The better solution would be that existing objects in the control collection should also appear in the palette. But the palette only consisted of components, not actual objects that had already been created.

An additional interface was added to allow an object to specify creation tools in the palette, along with their creation behavior. This would allow the control collection object to specify its children as creation tools in the palette. Here is a screenshot of the palette with a special drawer containing the objects in the control collection.

By simply creating them from the palette, controls in the control collection can be shared between layouts as easily as new controls can be created since they use the same semantics in the editor. The user does not need to know anything about how to achieve the linkage because the components specifying the controls extend the creation commands and link themselves automatically. Our design goals are met. Hopefully, the control collection is a little less daunting to novice UIQ programmers while still a familiar friend to the expert user.

We are in the last weeks of testing Carbide 1.3: the code is frozen, the test team is rechecking everything, and the beta testers are trying out the release candidate build. This is usually the time when some bug that has been around all along makes a surprise appearance in a really troubling way.

Last week we started getting some reports of the entire workbench locking up. People who were really beating on the tools all day said that often after a paste or save operation the UI would be unresponsive. Of course we couldn't reproduce the problem but we had two interesting clues: some people said the UI started responding again after a while and one beta tester sent in a screenshot of the frozen workbench.

Everyone has their own personal "time out" setting: the amount of time they are willing to wait for an unresponsive application to start working again before they kill it or restart their computer or chuck it over the cube wall. So that some people said the workbench came back to life after a while probably indicated we had some lengthly process blocking the UI thread, not that Carbide was frozen forever.

The screenshot gave us a few more clues: the progress indicator said the CDT indexer job was just starting and that the text the user was pasting hadn't yet appeared in the editor. I started trying to reproduce the problem by forcing a project to reindex and then repeatedly pasting code.

After a lot of pasting we found that the problem had nothing to do with the paste operation but with the use of the control key (as in ctrl-v to paste or ctrl-s to save). With the control key down CDT was asked to produce a hyperlink. Normally this happens quickly because the file has already been indexed and the results cached, but when it's dirty it proceeds to do some indexing on the UI thread which leaves the workbench completely unresponsive.

It seemed like the right thing to do in this case was to just not do the indexing if the cache was stale. We turned to the CDT team to confirm this and within a few hours we had changes in place to prevent this problem and a couple other related issues too.

So now I'm starting this week like I did the last: hoping Carbide 1.3 can slide out the door without any serious issues popping up.

We build Carbide.c++ on top of a lot of open source components including the Eclipse platform and the C++ support from the CDT project. So I'm happy to announce that our team has added another committer to the CDT community: Warren Paul was voted in yesterday based on his record of contributions while developing Carbide. Anyone can contribute to Eclipse projects like CDT, but their patches have to be reviewed by one of the committers on the project. Committers have access rights to make changes to the code in the CVS repository and the responsibility to make sure the work is a good fit and high quality. They also help shape the architecture and steer the overall direction of the project. Warren joining the CDT team will help us contribute and keep Carbide in sync with the open source community.

One of the early decisions to adopt the eclipse platform revolved around the ease extending the eclipse platform and selecting from hundreds of free and commercially available plugins. However, I haven't seen a comprehensive list of plugin that developers have been plugging into Carbide.c++. Of course there's the usual version control systems that everyone uses, but what sort of cool additions are there for Symbian development.

One cool plugin I use frequently is ContextMenuPlugin for Eclipse. This one lets you quickly run an explorer context menu from a Carbide project. So there no needing to open another explorer window to do other tasks. I was only successful in getting this one running with Carbide 1.3, on Carbide 1.2.x it wouldn't load properly.

Another handy one that plugin directly into the Carbide menu is the Symbian Developer Certificate Request tool with Certificate Manager. Now you can handle all your developer certificate requests from within the Carbide.c++ IDE.

A plugin I haven't yet tried but looks very promising was done by a couple of students at the University of Applied Sciences in Hagenberg in Austria is PanicLookup. You can also get more information and a good summary from Andreas Jakl's Forum Nokia Blog.

And UIQ has their UIQ Cabride.c++ Configuration Plug-in over at the UIQ Developer Community.

Of course there are the multiple version control systems supported in eclipse and hundreds of other plugins to check out at Eclipse Plugin Central.

If you find anything else you find useful for Symbian-related development let me know! Or you can add it to the Forum Nokia Wiki where I've started a list.

And starting with Carbide.c++ 1.2 we have published APIs to easily integrate into Carbide projects to manipulate projects, settings and read any of the MMP or INF files. So integrating tools into Carbide is as easy as turning on, tuning in and pluggin-in!

--Tim

In my last post I began discussing the challenges that the Carbide.c++ UI designer team faced in deciding how to present the control collection to the developer in a way that would make sense to experienced UIQ developers but that would not burden novice developers with too much detail. Specifically, I talked about how we needed to separate the shared layout-agnostic control data from the layout-specific data while still maintaining a unity between them so they could be edited as one and in the most obvious place.

I mentioned that we created two different kinds of control types, and linked them together with a property in the layout control object that referenced its shared control collection control object. However, we did not want to burden the developer with having to think about these as two separate objects in the designer, so we made some enhancements to the framework to facilitate that design goal.

Here is a screenshot of the outline view showing a multipage view with one layout. The layout has one page and its container contains a label control. The layout-specific control is the one in the container. It represents in the outline the control that is visible in the layout part of the main editor. The label control with the shared data is the one in the control collection and it is only visible in the outline. To make obvious that the layout control references the control in the control collection, we use a default naming convention that initially names the layout control using the string "ref" but we can't rely on the initial name to show how these are linked together because the user is free to change these names.

Instead, we added an enhancement to the UI designer component system that we called "reference property promotion." Using this facility, a layout control component can promote the properties of its referenced control in the control collection. This means that the referenced control's properties will be editable as if they are the layout control's own properties. So although the layout control only declares layout-specific properties (plus the reference property - "control" - that points to the control collection control with the shared data), the properties of the referenced control are editable as if they belonged directly to the the layout control. This way, the user would not need to hunt down the node in the outline to edit those properties. Here is what the properties view shows when focused on the layout control (label1Ref1). The layout-specific data is editable alongside the shared data. As you can see, the layout-specific properties represent a very small portion of the data available to be edited. Without the promotion, most of these properties would only be available if the user had selected the control in the control collection in the outline.

So reference property promotion was the enhancement we added to make the shared data easily accessible. In my next post I will discuss several enhancements we added to the UI designer framework to make the creation and linking of the layout and control collection controls transparent to the user, as well as making the sharing of control collection controls among multiple layouts as easy as creating new controls.

Adding support for UIQ to the Carbide.c++ UI designer posed some challeneges for our existing architecture. Perhaps the most challenging aspect was how to present the control collection to the user in a way that was both meaningful and useful and not overly cumbersome. Having decided to generate source code that would share control data using the control collection, we now had to find a way to present this to developers within the graphical editor.

First, I should probably explain a little about the control collection for those readers not well versed in UIQ UI programming. The control collection is basically an array of layout-agnostic control data for the controls that are used in different layouts corresponding to the same view. For each control, there is an entry with its shared data in the control collection. In each layout, the shared data for these controls is referenced through a unique id.

For any view, there may be multiple layouts defined for different UI configurations (e.g., softkeys, pen, portrait, landscape, etc.). In a given application, the same control may exist in different layouts corresponding to different UI configurations. Because UIQ uses layout managers, different UI configurations will often all use the same layout, but they can have different layouts as well depending on the design of the application. A text editor for collecting some kind of textual data from the user may exist in both softkey and pen configurations, and the maximum number of characters it accepts may be the same, even if it is laid out differently in the different UI configurations. The maximum length of the text editor control is an example of data that can be expressed in the control collection. In different layouts, each control in the control collection is augmented with layout-specific data (this data is the layout manager settings that are specific to how that control is laid out in that specific layout).

For each control the developer adds to a design, the UI designer tool had to present two different sets of data: shared data and layout-specific data. We had several design goals in mind. The shared data had to be presented so that it could be easily accessed irrespective of which layout the developer was editing. We didn't want to make the single layout use case more cumbersome in order to support the multiple layout case. First, we needed to present the controls in the control collection so that they could easily be added to new layouts while still allowing the developer to create brand new controls. And second, we needed to present the shared data for each control in a way that was easily accessible, rather than make the developer have to hunt down where to edit these values.

In the UI designer tool, we have various eclipse views working together. There is the main editor. This has a layout area where controls are rendered as they would look in the running application. In the editor, there is a palette of control creation tools arranged in drawers by type (e.g., editors like text or numeric editors, controls like labels and images, etc.). Controls can be added to a design from the palette by selecting them in the palette and clicking or dragging them on the layout area. Once an object is created, it also appears in the outline view, and when selected, its properties can be edited in the properties view. There are additional views as well, but these provide the main editing environment.

In the S60 UI designer, we had no data sharing requirements, objects are simply created from the palette and there is no shared data separate from layout-specific data. We decided that to separate the shared data from the layout-specific data, we would make two kinds of control types. One kind would allow editing the control-specific data and the other would be a generic control type with layout-specific data. This seemed ok, but we needed to tie these together. Our framework already contained the concept of one object containing a reference to another object through a special kind of property, and this seemed exactly the facility to use to tie the two objects together. For source generation, the reference property would be sufficient. Each layout control would have a reference to a control in the control collection. But this referencing facility would have to be really beefed up to allow us to make the visualization and editing as intuitive as it would need to be.

In my next blog entry, I will discuss some of the reference property enhancements and other new facilities we added to the UI designer framework to support our stated design goals in a general and reusable manner.

(...or some not so obvious things in Carbide.c++ that will help you to cruise around your project resources and make you more efficient)

I just got back from a trip to Korea to visit with some customers that are migrating to Carbide.c++. I was tasked with teaching Carbide to new users in just a few short hours so that they would have enough information to begin evaluating the tool. I realized that you can't possibly teach everything Carbide can do in a few hours so I thought about some of the not-so-obvious things that make the eclipse platform and CDT a joy to work with (or a pain if you don't know them). These are things you don't realize from just following a simple HelloWorld tutorial or just playing around with the tool.

I thought about some of the most common questions that I've read from my experience on the Carbide Beta group and numerous other news groups. Things like, "It's so hard to find my sources in the Project view, I just have too many." Where the answer is, "Just use 'CTRL + Shift + R'".

I've also had my own, "Woah, eclipse can do that! Cool!". Something simple like commenting out several lines of source at one time (CTRL + /). And after two years I continue to discover cool and useful things that the eclipse platform can do.

So I took my own experience with source-related tasks that have saved me many hours a day and added those I show to others frequently. I've put all these on power point slides, but perhaps the best ones could be consolidated on single cheat sheet?

What do you find saves you time throughout the day when using Carbide?

Here's the presentation.

Happy Navigating!

--Tim

Now that the Carbide 1.3 beta period is winding down the rest of us on the team will be helping Mike out with the blog. I'll start with a story about one of the bugs we fixed in this release.

The Carbide 1.3 beta testers have been really positive while sending plenty of constructive criticism about things we need to improve. It's been very satisfying working with them to track down some of the bugs that escaped us last time. One was a report about debugger performance: some people said that when debugging they found that "stepping through code was really slow."

These kinds of problems can be really difficult to reproduce and fix because there are so many variables. Only a few people were reporting the problem and it wasn't consistent. We couldn't reproduce it at all.

A lot of things can impact debugger stepping performance including the speed of the debug connection and the number of variables displayed each time, but none seemed to be a factor in this case.

As people ran into the problem we started to get more specific information: we heard that not only was stepping in the debugger slow but the entire Carbide environment became really slow. Then someone noticed that everything on their computer was slow: every application, menu etc.

Next we heard from a developer on one of Nokia's software teams in Dallas. He could reproduce the problem but it only happened when debugging a particular project. I setup the same project and tried the same steps but couldn't reproduce the problem.

A trip to Dallas from our office here in Austin can easily occupy a long day, much of it spent in traffic, so first we tried a screen sharing session. That confirmed I was following the same repro steps with the project but getting different results: my debug session hummed along while my Dallas colleague's ground to a halt with the Java VM process sucking up 80 - 90% of the cpu. But the most interesting thing was that the application he was debugging (a music player) was sending a huge amount of text to the console view. The entire time a song was playing the console view was flooded with debug output. After playing a few songs the audio began to skip and the big slow down began.

I pulled a few long songs from my music collection and while my console view was also flooded with various debug messages I didn't see any performance hit. Time for a road trip.

When I got to Dallas we quickly reproduced the problem and all the evidence pointed to the console output. There is a setting that limits the amount of text in a console. My first theory was that we were overloading whatever method keeps the console under this limit. Maybe we were dumping in so much text it was thrashing about trying to keep the limit enforced? But after some more digging around I found that we were not using this setting properly and so didn't have any limit on the amount of text in a console. Over time all the text dumped into our console consumed more and more memory.

Limiting the amount of text in the console view was simple enough and has fixed the problem. I'm still not sure why the effect of the big memory leak on my Dallas colleague's computer was so severe while my similarly configured laptop wasn't bothered. In any event the whole experience shows how initial descriptions of bugs can be misleading and that you need work closely with the people using the tools all day in order to track down difficult to reproduce problems.

- Ken

In 2008 I have resolved to blog more - and it's taken me 5 weeks but here is my first entry.

Carbide.c++ v1.3 is getting ready to ship in about a month and it's a good time to start thinking about what is next for our product line.  One thing about development tools is that users generally line up into two groups.  Group 1 is the "your product rocks" fan club who could never imagine a better tool on the planet.  This group plans to name their first child male Carbide.johnny.  Group 2 is the "your product sucks" group and questions why in the world is Nokia wasting their time on Carbide when product X is so much better (product X could be MSVS, CodeWarrior, EMACS, or Commodore 64).  Both groups are correct (certain members of each group need to consider a different hobby) and Nokia is learning much from each.

We are making significant improvements to the tool and we do have a few basic things we still need to hammer out.  Here is a pretty honest assessment of where we are and where we are hoping to take the product next.

Positive Progress

• Stability has greatly improved.  Many users have noticed the difference and we have knocked out a number of stability-related bugs in this release.
• Build times - we found an nifty way to improve some of the common build methods without messing with the Symbian build system. 
• On-device debugging - we knocked out many bugs that affect the stability of this feature. We have also made it easier to debug nested DLLs.
• New features include:
• Static Analysis (CodeScanner)
• Dependecy Explorer - interactive tool showing caller/callee relationships in your project.
• UIQ - UI Designer support, and we added on-device debug support for SE devices.
• Global Search - we get hit hard that CodeWarrior as a good tool - we've added this in v1.3
• Platform Security tools - ability to search S60 code for Symbian capabilities and a tool to parse capabilities detected using the emulator. 

For our next product we will be focusing on new features of course, but will also want to maintain focus on improving the very basics of the product.  We have learned that Visual Studio handles the essentials (edit, build, debug) better than any tool and we are learning from this example.  Eclipse is more than happy to throw 50 options at the users - VS is more selective.  On Carbide we are leaning to the VS model.

Next week I will discuss the areas of improvement for our next product and conclude this series by providing a few ideas about new features.

Download the v1.3 beta if you have some time - learn more at http://groups.google.com/group/carbidecpp-beta

/Mike