[jjnabc]

Hi All,

We are experiencing a strange problem in our BES & Domino Environment when performing Name Lookup from Blackberry Devices at certain locations and it is driving me mad!

Environment is Domino 6.5.5 Server and BES 4.1.2.20.

The server is physically placed in Denmark but when american users performs name lookups from their Blackberries to the server their response back to the devices can take up to 5-10 minutes. The server itself performs the lookup very fast (within miliseconds) but the response back to the device either times out on the way back to the device or something else goes wrong.

As I have understood the name lookup itself takes place on the BES. The names.nsf is full-text indexed and everything should be running as effectively as possible. The name lookup functionality works fine for other users at other location where response is sent back within seconds.

Can someone please give me a hint on what could be wrong and secondly does anyone have a clue of where to find documentation on what actually happens on the device and server when lookups are performed.

Hope to get a response from someone!

/Jesper

[noname]

if you could try, restart the Domino BES server and the next minute get an American user to do a lookup on the same name and see if it responds quickly. If yes, then it could be a hung thread on the BES, which is not able to respond to the lookup request. Then, your next step is to see if the server is "hungry" somewhere... e.g. memory... etc...

[jjnabc]

Hi

We have tried several things regarding when to do lookups and have not been able to see a pattern. The response times varies and as stated it works fine for rest of the world (or so I'm told) and tests prove that as well.

In my opinion the BES is always "hungry" for RAM. At the moment we have 500 users on one server 3.40GHZ processor with 4GB RAM and it still thinks it needs more and Domain Monitoring in Domino reports rating critical.

Everything else on the BES actually works perfect and response times for Mail distribution is excellent.

I am a happy customer apart from the little lookup problem

[noname]

hey jjnabc, i guess i overlooked that your BES is is Denmark and users are in USA. I guess it is inevitable in your case that the lookup will be delayed if your Americian users' mail dbs are on an American Domino server.

RIM white paper "Performance Characteristics - BlackBerry Enterprise Server Version 4.0 for IBM Lotus Domino" actually recommends that the BES be placed as close to the maill servers as possbile. If possible deploy multiple BES Domino servers close to host users close to their mail servers.

Meaning...
BES_Denmark serves all users on Denmark mail servers AND
BES_USA serves all users on American mail servers.

Having to get response over WAN for requests such as name lookup will definitely hit with performance issues. Just imagine how the data traffic move about over the WAN and you can have a rough idea on your issue.

Just my 2c worth.

[jjnabc]

Hey again,

Thanks for the input!

Basically you are absolutely right. Keep your users close to the BES but that is not always easy as we are a company that is spread all over the world and the majority of Blackberry users are actually within Europe and we are not planning on establishing an additional server right now.

To add some spice to the problem in US it is actually only parts of the country where the problem occurs i.e. California are hit really bad with response times in lookups.

Yesterday we tested Kansas (response time in lookup 4 seconds) Indianapolis (response time in lookup 4-5 seconds) Palo Alto in California (response time in lookup 5-10 Minutes!!)

It has been explained to me by RIM that a name lookup only goes to the BES and is not hitting your home Domino server at all therefore only a small delay in response to US (4-5 seconds) and the response time in Europe is about 1-3seconds which makes sense but response time in minutes is strange!

Have a nice weekend!

[amukhey]

jjnabc, I would follow noname's advice. Your configuration wont bring about exact results and give you problems down the line.

[iNeedThemes]

jjnabc, You mentioned that it appears user's in Cali have the longest delays. This could imply that it has less to do with activity over the WAN and more to do with their network coverage maybe? If you try sending one of these user's a P2P message how long does it take to get confirmation of delivery?

Also, do these user's in Cali report any delays with receiving mail? They may not notice anything overt so you can have them try sending themselves an email from their BB HH. The email should then go to the BES, to the user's mail file, get picked up by the BES and forwarded back to the mail file. This will let us know if there is a general carrier network delay.

[amukhey]

jjnabc, any updates on this issue and has it been resolved?

[jjnabc]

Hi All,

Sorry for the missing updates on this problem and thanks for the suggestions everybody.

As INeedThemes suggested I have been doing tests with the California users of sending mails from the BB to themselves and check response times. 2 tests were made and both seemed very slow. One took 3 minutes and the other took over 8 minutes. I checked mail queues in Domino and no delays or other things had an impact on the delivery.

I have still to make this test with other American users but it seems to me there is something different in California I just can't figure out what it is!

Since last the BES was upgraded to 7.0.2 FP1 but no other changes have been made.

Thanks!

[rsun]

We seem to be experiencing something similar, but it seems only our 8707's have this problem, users with other devices but based off the same remote mail servers (in relation to the BES), work very quick. Have you been able to figure out anything else?

[noname]

Do not think address lookup latency is specific to a device model, rsun. It could be a hung thread which was serving another transaction (e.g ScanForNewMail) and the same thread is trying to spawn a Domino thread to serve the lookup request and therefore is in a hung state. Have you try to recycle Domino and BES and try again?

[jjnabc]

Hi again,

I have RIM looking through logs for the particular area in California to check for reports of bad reception and so on.

We have come to the conclusion that the BES, Domino and device types have nothing to do with this problem.

Lookup functions like this:

Name Lookup performed on BB device
Traffic goes from RIM to BES
NBes on BES makes the lookup using Public Address Book or Directory
assistance on the BES!
Result returned from BES to RIM and then device

Have a nice weekend all!

[huckleberry]

We are having the same problem. However I am having issues w/ users based in the same office (also the same office/same subnet as the BES server). I myself can perform a lookup w/ results in 2 seconds, while others are taking a minimum of 5mins to get results (assuming that it does not fail). For the most part I have noticed some common trends among those that have painfully long lookup times

1. Larger mail files (2GB +)
2. Large State DB (10MB +)

As I stated above I can perform a lookup w/ instant results, my mail file is 300MB and my state DB is 2MB. I've called RIM and they blame AT&T, and of course they blame RIM. RIM said that once the server receives the request it processes instantly, which I agreed w/ however I've had devices side-by-side and of course one works in a respectable time frame and the other is terribly slow. I gone as far to swap devices, remove users from server completely and re-set up users on devices that worked great for others, however that did not help. Swapping SIM cards did not help either, and in a few instances I was actually able to setup using different providers. When performing lookups w/ devices side-by-side the server shows the lookups receives instantly for the device is performing a acceptable lookup, where as the other device that is taking 5+mins does not even make it to the BES for 5+mins, but once that is received and processed by the BES the results instantly appear on the BB which again leaves me to believe that the issue is some how related the user, their mail file, the state DB or some combination,

Dom Ver. 7.0.2 FP1 HF75, BES 4.1.3.22 (375 Users)

All of my users are AT&T,

[EnterpriseBBAdmin]

In my experience, after performing maintenance in the related Domino, still some users reported slow and poor performance when attempting to complete a Names Look Up task on their BlackBerry devices. The issue was experienced by some users and not others, even though their devices are being served by the same BES. Furthermore, the issue was not BlackBerry model, location of the users or service provider specific.

The issue stemmed from the overload on the current number of mail threads used by the BES. By default, the Registry setting of MaxTotalThreads has a decimal value of 40, as (DWORD). This key can be found on the BES at:


HKEY_LOCAL_MACHINE\SOFTWARE\Research In Motion\BlackBerry Enterprise Server\Agents

The optimal value depends on a combination of Domino servers on the environment and numbers of devices served by the BES. RIM assisted determining such value.

I gathered a BES log file for a full business day, plus a handful of PIN numbers of the BlackBerry devices in which the issue was observed and opened a case with RIM. I made such log and information available to RIM. A new suggested value was provided, along with a detailed breakdown of the mail threads and how many devices were being served by each of them.

With the value suggested by RIM, quitted the BES and Domino server, and then applied the new value. Restarted the server. Confirmed the Domino server and BES were running, then allowed the BES to run for 5-6 hours in order to catch up any backlogs due to this downtime.

The issue was resolved and no longer experienced since.

A word of caution: Please follow best practices to protect your servers, data and environment.

[Stu_Bee]

We experienced the same issue and solution as "EnterpriseBBAdmin" in the prior post.
Solution:
- Domino: Tell bes quit
- Registry change "MaxTotalThreads" from 40 to 80.
- Domino: load bes

Not sure the intial cause..however the affected users were on a mailserver that had just been patched with MS08-001 (TCPIP vulnerability). Possibly not related.