blackberry.net server unreachable from Exchange

[itadmin-uf]

My CEO is constantly breaking things and is having yet another issue that I can’t fix. He gets a delivery failure report about every 30 minutes to network@<hidden>. I can telnet to port 25 on the exchange box to etp1105.etp.na.blackberry.net and so on, but not etp11104.

I have resent the service books to his phone, rebooted the BES, and upgraded to the newest Desktop software but nothing has helped. I have frozen all messages to this server in exchange to stop the messages from going into his inbox. Has anyone seen a problem like this before? I have 9 other people on my BES that have no issues. The Delivery report is below.

From: System Administrator
Sent: Monday, July 27, 2009 2:43 PM
To: Technology inept CEO
Subject: Undeliverable: RIM_bca28a80-e9c0-11d1-87fe-00600811c6a2
Your message did not reach some or all of the intended recipients.
Subject: RIM_bca28a80-e9c0-11d1-87fe-00600811c6a2
Sent: 7/25/2009 2:35 PM
The following recipient(s) cannot be reached:
'network@<hidden>' on 7/27/2009 2:43 PM
Could not deliver the message in the time limit specified. Please retry or contact your administrator.
<mymailserver.com #4.4.7>

BES version is 4.1.3 (Bundle 37)
Exchange 2003 SP2 (Build 7638.2)

[TargetIT]

You're looking at a NDR because he's managed to set something up that is responding to RIM embedded messages (ie - delivery restrictions is my best guess).

In other words, he, or I should say Exchange on behalf of him, is responding to RIM when it shouldn't.

[itadmin-uf]

You're looking at a NDR because he's managed to set something up that is responding to RIM embedded messages (ie - delivery restrictions is my best guess).

In other words, he, or I should say Exchange on behalf of him, is responding to RIM when it shouldn't.

I don't think that is the issue. I checked his rules and he does not have any setup that reply to messages or any message delivery settings have changed on exchange. His mail worked fine for a month before this started happening. He is known for tinkering and not owning up to anything, so could it be something he did on his phone?

Can anybody check for me if you can connect to port 25 for etp1104.etp.na.blackberry.net? I see other connections to etp1105, 1102, 1109 and so on. I can connect to them just fine but not 1104.

Also why would only messages from him want to connect to etp1104.etp.na.blackberry.net? Does anybody know how that works?

[hdawg]

There is never a need for a BES / user to ever email network@<hidden> ... there is some configuration in place attempting to mail ... perhaps an out of office?

[TargetIT]

Well, that's not true - an email from the user is created after a successful activation. I kind of think it's broken myself - I do have logs dating back to successful messages to that address.

[penguin3107]

Well, that's not true - an email from the user is created after a successful activation. I kind of think it's broken myself - I do have logs dating back to successful messages to that address.

hdawg is correct.
What email are you referring to?

[TargetIT]

They send you a message, BES replies back. This is only during an activation. It's not a simple automated response. After the ETP.DAT file hits exchange and BES grabs it, it replies back. Here's a sample:

This message is used to carry data between the BlackBerry Handheld and an associated server. Please do not delete, move or respond to this message - it will be processed by the server.

BEGINETP 10
Awx0jPsAAAAAAQ!!
ENDETP 1660784311

It also has an ETP.DAT file.

I'm guessing it's a simple response saying, yup - that person belongs to me.

[hdawg]

BES doesn't reply back ... rather after receiving and processing the message it generates an encryption key.

Can you go through your SMTP logs and post something from a message being sent?

I won't hold my breath ...

[penguin3107]

BES doesn't reply back ... rather after receiving and processing the message it generates an encryption key.

Can you go through your SMTP logs and post something from a message being sent?

I won't hold my breath ...

+1 ^^

[TargetIT]

Guys - that's an ACTUAL email in response to the inbound email ETP activation message - I didn't just make it up. All email messages in and out of our company goes through another server - the message headers indicate the message is from our internal exchange server.

[hdawg]

Huh?

The BES injects items into the mailbox, yes ... but it doesn't do anything SMTP related during the EA ... I feel as though we're talking about two different things because I know what I'm talking about I am correct, and I know you wouldn't be talking out of your ass :)

[TargetIT]

What can I say - it didn't even occur to me that it was doing that until I checked the logs. Within 2-3 seconds of receiving the activation message from RIM, it sends out this email to the same email address that sent the original activation ETP message. As I said - it looks like an acknowledgment. But it doesn't make much sense since the BES could just send the response via 3101

[TargetIT]

Damn - what's going on with the date stamp in this thread?!

[itadmin-uf]

Didn't mean to start a fight, but if you knew my CEO it would totally make sense that anything involving him would get heated.

Because of space issues I had not been running message logging on my exchange box. I have that fixed now and just enabled it. All the messages that are sending to network(@<hidden>)etpXXXX.etp.na.blackberry.net are from my CEO and he is the one that is getting the bounce backs. Its almost as if he has redirector and BES setup on his phone. I asked him if he had the blackberry software on his home computer and he said no. He didn't have desktop manager installed on his work computer till I installed it 2 days ago to try to fix the problem.

Should I delete any service books and resend them? If so which ones? If he does have redirector setup on a computer somewhere how do I find that out and stop it?

[TargetIT]

I bet re-activating will help - if I'm right and it's simply responding to the inbound ETP message, a re-activation will most likely utilize a different email address and that response will get through.

[hdawg]

look in your SMTP logs to see which client is submitting messages.

[TargetIT]

It's irrelevent where the message is coming from - it's getting bounced by RIM and the NDR is being delivered to the mailbox.

[hdawg]

Ummm no, its quite relevant. Considering BES doesn't send email to that address and there is no need to ever do so knowing where it is coming from will help identify why this is happening.

[TargetIT]

I realize you think that, even I thought that, but it's simply not true - I have both the email and log files to back it up.

[hdawg]

Can you email me them? I would like to be proven wrong.

hdawg@<hidden>

[TargetIT]

Well I've already posted the email here - you can see that the ETP.DAT file/block is a lot smaller than an activation ETP.DAT. I'll activate my own phone again and post the SMTP logs - I don't have logs dating back past the last time I activated a phone.

[hdawg]

Excellent; thank you.

[TargetIT]

Here's the SMTP logs (I've only change my domain name and my name):

2009-07-30 16:39:54 192.168.195.103 OutboundConnectionResponse SMTPSVC1 EXCHANGE-X01 - 25 - - 220+smtp.Mycompany.com+ESMTP+MailMarshal+(v6.4.6.5922)+Ready 0 0 65 0 15 SMTP - - - -
2009-07-30 16:39:54 192.168.195.103 OutboundConnectionCommand SMTPSVC1 EXCHANGE-X01 - 25 HELO - exchange-x01.Mycompany.com 0 0 4 0 15 SMTP - - - -
2009-07-30 16:39:54 192.168.195.103 OutboundConnectionResponse SMTPSVC1 EXCHANGE-X01 - 25 - - 250+smtp.Mycompany.com+Hello+exchange-x01.Mycompany.com+(192.168.195.105) 0 0 83 0 15 SMTP - - - -
2009-07-30 16:39:54 192.168.195.103 OutboundConnectionCommand SMTPSVC1 EXCHANGE-X01 - 25 MAIL - FROM:<TargetIT@<hidden>> 0 0 4 0 15 SMTP - - - -
2009-07-30 16:39:54 192.168.195.103 OutboundConnectionResponse SMTPSVC1 EXCHANGE-X01 - 25 - - 250+sender+ok+<TargetIT@<hidden>> 0 0 43 0 15 SMTP - - - -
2009-07-30 16:39:54 192.168.195.103 OutboundConnectionCommand SMTPSVC1 EXCHANGE-X01 - 25 RCPT - TO:<network@<hidden>> 0 0 4 0 15 SMTP - - - -
2009-07-30 16:39:54 192.168.195.103 OutboundConnectionResponse SMTPSVC1 EXCHANGE-X01 - 25 - - 250+Recipient+ok+<network@<hidden>> 0 0 56 0 15 SMTP - - - -
2009-07-30 16:39:54 192.168.195.103 OutboundConnectionCommand SMTPSVC1 EXCHANGE-X01 - 25 DATA - - 0 0 4 0 15 SMTP - - - -
2009-07-30 16:39:54 192.168.195.103 OutboundConnectionResponse SMTPSVC1 EXCHANGE-X01 - 25 - - 354+send+the+mail+data,+end+with+. 0 0 34 0 15 SMTP - - - -
2009-07-30 16:39:54 192.168.195.103 OutboundConnectionResponse SMTPSVC1 EXCHANGE-X01 - 25 - - 250+B4a71ccda0000+Message+accepted+for+delivery 0 0 47 0 15 SMTP - - - -
2009-07-30 16:39:54 192.168.195.103 OutboundConnectionCommand SMTPSVC1 EXCHANGE-X01 - 25 QUIT - - 0 0 4 0 47 SMTP - - - -
2009-07-30 16:39:54 192.168.195.103 OutboundConnectionResponse SMTPSVC1 EXCHANGE-X01 - 25 - - 221+smtp.Mycompany.com+closing+connection 0 0 46 0 47 SMTP - - - -


As mentioned before, all outbound email relays through an intermediate server @<hidden> 192.168.195.103

[jibi]

We've had 217 messages sent through our infrastructure mail system to a network@<hidden> address in the last 24 hours. Only one of these persons, who was inactive for days until the few hours when he was reactivated, is on a BES server. The other 20-30 people are not on our servers. We have a plethora of activations each hour, so I can safely say that we do not send typically have these sent during activation, although there looks like there may be other reasons they're sent (perhaps Desktop Redirector).

[TargetIT]

We don't even deploy Desktop Manager to anyone - they get the BB and that's it. I find it odd that there's no documentation on it, but it's clearly doing it - and consistantly for me.

[jibi]

There is documentation on the workflow. I'd question some other potential unknown factor in your environment, to be honest. From the logs, can you gather the source of the messages (I'm not seeing it the log lines above)?

Process flow: Activating the BlackBerry device over the wireless network
The user receives or purchases a new BlackBerry® device.

1. The user contacts your organization's IT department to activate the BlackBerry device.

2. The administrator uses the BlackBerry Manager to create a temporary activation password for the user account and communicates that password to the user. The password applies to the user account only.

3. To activate the BlackBerry device over the wireless network, the user opens the activation application on the BlackBerry device and types the appropriate email address and the activation password.

4. The BlackBerry device sends an activation request message to the email account. The message contains information about the BlackBerry device, such as routing information and the public keys for the BlackBerry device.

5. The BlackBerry® Enterprise Server sends the BlackBerry device an activation response that contains routing information about the BlackBerry Enterprise Server and the public keys for the BlackBerry Enterprise Server.

6. The BlackBerry Enterprise Server and the BlackBerry device establish a master encryption key. The BlackBerry Enterprise Server and the BlackBerry device confirm knowledge of the master encryption key to one another. If the confirmation succeeds, the activation proceeds and further communication between the BlackBerry Enterprise Server and the BlackBerry device is encrypted.

7. The BlackBerry Enterprise Server sends the IT policy to the BlackBerry device. If the BlackBerry device cannot accept the IT policy, the activation does not complete.

8. The BlackBerry Enterprise Server sends the appropriate service books (for example, the messaging service book, wireless calendar service book, browser service book, and other service books) to the BlackBerry device. The user can now send messages from and receive messages on the BlackBerry device.

9. If the user is configured for wireless synchronization, and the BlackBerry device has wireless backup and wireless calendar synchronization turned on, the BlackBerry Enterprise Server sends user data to the BlackBerry device.

[nobody7290]

I bet, that someone/somthing tried to setup synchronisation using the Desktop redirector.