BlackBerry Forums Support Community               

Closed Thread
 
LinkBack Thread Tools
Old 01-25-2006, 11:09 AM   #1 (permalink)
New Member
 
Join Date: May 2005
Model: 7290
Posts: 11
Post Thanks: 0
Thanked 0 Times in 0 Posts
Default Solid Testing Strategy

Please Login to Remove!

Hi, I'm trying to put together a solid testing strategy for a BB application. It does not do any networking, except for invoking the browser to help pages and such. Right now the testing plan includes subscribing to a BES-HOST, and getting 5-6 different models with different Carriers. I've tried testing the APP with all the simulators but real BBs exhibit different behavior, such as timing for race-conditions, etc. Before embarking on a spending spree I'm wondering how other developers QA a BB app. Is a BES totally a waste (hosted option is pretty cheap) or does it present circumstances that a JDE with simulators can never replicate. What about different OS versions? Does QA involve compiling for different versions of the OS or just compiling once and running on several OS versions. What is customary in QA? Perhaps, it is common practice to test on just a couple of devices and cross your fingers... Do you have some general testing strategies that have worked?

Thanks,

Victor.
Offline  
Old 01-25-2006, 11:22 AM   #2 (permalink)
Thumbs Must Hurt
 
baconismidog's Avatar
 
Join Date: Jan 2006
Location: Los Angeles
Model: 8100
Carrier: Cingular
Posts: 192
Post Thanks: 0
Thanked 0 Times in 0 Posts
Default don't bury yourself

Just a thought, but you should make it work on the newest OS and one of the newest phones first before you spend piles of cash. Then, present it to the community as freeware so that people will try it out for you. Most people will be willing to help out in hopes that they will benefit from your software. Just include a caveat that you have not tested it on any BBerry other than the (your model here) and that feedback is greatly appreciated. When they download it, direct them to your FTP site and kindly request their email so you can contact them in a few days. People don't mind being contacted by the developer directly (I actually like to talk to writers), they do mind being contacted by a botnet or by someone selling xxxxxx...

Don't burn up too much time writing for old OS's, who do you know that is writing software for Windows 98 or even Windows 2000???
Offline  
Old 01-25-2006, 11:28 AM   #3 (permalink)
Talking BlackBerry Encyclopedia
 
Join Date: Sep 2004
Model: 8700r
Carrier: Rogers
Posts: 221
Post Thanks: 0
Thanked 0 Times in 0 Posts
Default

For unit testing you can look at http://j2meunit.sourceforge.net/

Ultimately your best bet is to test across the different carriers that you wish to support. The main reason is some carriers have MDS functionality built into their gateways and some do not (basically its TCP vs WAP). This may affect your browser component.

In terms of OS versions:

What we do is baseline a minimum OS version - 4.0.0. Anything compiled with respect to 4.0.0 is forward compatible (look out for deprecated things). There are really 3 families of device apis - 4.0.0, 4.0.2, and 4.1.0. My opinion is that if you are claiming it is compatible with the 3 families you should complie and test across the 3 families.

The RIM simulators are actually quite good at representing real device operation (compared to say the WTK simulators). But you can never be sure until it is actually tested on live devices.
Offline  
Old 01-26-2006, 01:25 PM   #4 (permalink)
New Member
 
Join Date: May 2005
Model: 7290
Posts: 11
Post Thanks: 0
Thanked 0 Times in 0 Posts
Default What about BES?

Hi,

Thank you for your guidance. I'm still wondering though, if testing outside of a BES enclave is a wise decision? Should I setup a server or use a hosted-server option? Is there any value to that from a testing point of view? The application has no networking.

Thoughts?

Victor.
Offline  
Old 01-29-2006, 11:18 AM   #5 (permalink)
Thumbs Must Hurt
 
Join Date: Aug 2004
Location: Metro NYC
Posts: 175
Post Thanks: 0
Thanked 0 Times in 0 Posts
Default

>> testing outside of a BES

It all depends on what your application does. It it uses networking, event injection or anything controlled by an IT Policy, you should test with a BES.

It terms of OS versions, we typically target v4.0. This means using JDE v4.0. For the most part, clients are running v4.0 on their handhelds and this maximizes the compatibility. Exceptions, however, do exist. OS v4.0.2 to support GPS comes to mind.
__________________
-- Aric Rosenbaum
BlackBerry consulting, BlackBerry development
www.arconsultinginc.com
BlackBerry consulting and development (RIM SI Partner)
Offline  
Closed Thread


Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On





Copyright 2004-2014 BlackBerryForums.com.
The names RIM and BlackBerry are registered Trademarks of BlackBerry Inc.