[vcoleman]

Hi, I'm trying to put together a solid testing strategy for a BB application. It does not do any networking, except for invoking the browser to help pages and such. Right now the testing plan includes subscribing to a BES-HOST, and getting 5-6 different models with different Carriers. I've tried testing the APP with all the simulators but real BBs exhibit different behavior, such as timing for race-conditions, etc. Before embarking on a spending spree I'm wondering how other developers QA a BB app. Is a BES totally a waste (hosted option is pretty cheap) or does it present circumstances that a JDE with simulators can never replicate. What about different OS versions? Does QA involve compiling for different versions of the OS or just compiling once and running on several OS versions. What is customary in QA? Perhaps, it is common practice to test on just a couple of devices and cross your fingers... Do you have some general testing strategies that have worked?

Thanks,

Victor.

[baconismidog]

Just a thought, but you should make it work on the newest OS and one of the newest phones first before you spend piles of cash. Then, present it to the community as freeware so that people will try it out for you. Most people will be willing to help out in hopes that they will benefit from your software. Just include a caveat that you have not tested it on any BBerry other than the (your model here) and that feedback is greatly appreciated. When they download it, direct them to your FTP site and kindly request their email so you can contact them in a few days. People don't mind being contacted by the developer directly (I actually like to talk to writers), they do mind being contacted by a botnet or by someone selling xxxxxx...

Don't burn up too much time writing for old OS's, who do you know that is writing software for Windows 98 or even Windows 2000???

[eradis]

For unit testing you can look at http://j2meunit.sourceforge.net/

Ultimately your best bet is to test across the different carriers that you wish to support. The main reason is some carriers have MDS functionality built into their gateways and some do not (basically its TCP vs WAP). This may affect your browser component.

In terms of OS versions:

What we do is baseline a minimum OS version - 4.0.0. Anything compiled with respect to 4.0.0 is forward compatible (look out for deprecated things). There are really 3 families of device apis - 4.0.0, 4.0.2, and 4.1.0. My opinion is that if you are claiming it is compatible with the 3 families you should complie and test across the 3 families.

The RIM simulators are actually quite good at representing real device operation (compared to say the WTK simulators). But you can never be sure until it is actually tested on live devices.

[vcoleman]

Hi,

Thank you for your guidance. I'm still wondering though, if testing outside of a BES enclave is a wise decision? Should I setup a server or use a hosted-server option? Is there any value to that from a testing point of view? The application has no networking.

Thoughts?

Victor.

[arconsulting]

>> testing outside of a BES

It all depends on what your application does. It it uses networking, event injection or anything controlled by an IT Policy, you should test with a BES.

It terms of OS versions, we typically target v4.0. This means using JDE v4.0. For the most part, clients are running v4.0 on their handhelds and this maximizes the compatibility. Exceptions, however, do exist. OS v4.0.2 to support GPS comes to mind.