[MobileMind]

Our BES admins are told there is a problem with RIM over the early morning period of the 15th May 2009 1 am I missed a mail till about 10:25 am - so data services are not working in at very least Vodafone de or Vodafone Uk areas. Can people confirm and clarify whats going on. Also why are we suddenly having global outages after a period of good stability in last few months. Whats going on?

[MobileMind]

Originally Posted by MobileMind

Our BES admins are told there is a problem with RIM at moment - so data services are not working in at very least Vodafone de or Vodafone Uk areas. Can people confirm and clarify whats going on. Also why are we suddenly having global outages after a period of good stability in last few months. Whats going on?

RIM is back now - but can someone tell me what happened? The RIM Network Operations Centers were not reachable for a period of time. No official statement as yet

[skyman84]

Issue was effecting the whole of EMEA but should be resolved as of 9am GMT

[MobileMind]

Originally Posted by skyman84

Issue was effecting the whole of EMEA but should be resolved as of 9am GMT

Yes but whats going on? Why aren't people talking about this - is this the wrong forum or were people simply asleep during the down time? This follows incident of April 28th also.

Details on Yesterday’s EMEA BlackBerry Outage from Zenprise, BoxTone | CIO - Blogs and Discussion

RIM need to restore confidence for Companies that are starting to place or plan business critical apps on their Blackberry environments and who need some assurances about the long term reliability, stability, security and scalability of the environment.

We had 10 months of no incidents and then suddenly kapoww this year somethings going wrong. Is everyone trying to hack Obamas Blackberry or have RIM simply not got capacity to cope with all their extra users, or have they got people who cannot be relied on running the show. Whatever the reason FIX IT NOW!!!!

Wasn't WES 2009 all about High availability BES 5.0 and all that. Well this goes deeper. We need to know how serious the problem is with the RIM infrastructure itself right now and that are taking the necessary measures to fix this.

[Jadey]

Unfortunately you are unlikely to get that information anywhere before you'll get it from RIM - RIM are notoriously "quiet" about their major infrastructure outages. Most levels of t-support customers do not even get direct notifications. I have tried calling RIM t-support in the past, and the most info I get on T2 is "we are working on it". There are customers with a higher level of access to info with RIM, but many are bound by NDAs.

Bottom line - this site is not owned or affiliated with RIM. Until they release an official statement, we would all just be guessing what is wrong/why RIM have had 2 significant EMEA outages in past few weeks/whether it will happen again soon.

Also BES 5 high availability is not really relevant here. You can have as many HA configs as possible, but when the SRP link is down, it is game over.

[skyman84]

Right,
After you've taken your chill pill. We'll chat!


AFAIK the issues last week, and this one this morning are unrelated, but I've been known to be wrong in the past.

Originally Posted by MobileMind

Yes but whats going on? Why aren't people talking about this - is this the wrong forum or were people simply asleep during the down time?

Well, yes is the answer, most ppl were asleep, the issue started aprox 6am GMT and resolved by 9am GMT, bearing in mind most of the users in this forum are in the US A) they weren't effected (unless their BES is in the EMEA) and B) it was 1am over there.

As for us UK and EMEA users, there was little down time (well, 3 hours, which in my mind is not horrific). I'm not sure how much it's effected you, but of my 3,400 users, I have no calls about it!

Originally Posted by MobileMind

RIM need to restore confidence for Companies that are starting to place or plan business critical apps on their Blackberry environments and who need some assurances about the long term reliability, stability, security and scalability of the environment.

Well, that would be a job for RIM, give em a call, I'm sure they'd love to hear your comments.

This site is not in any way related to RIM, and it just a place for fans, users, admins, and experts to join together and share experiences and knowledge.

Originally Posted by MobileMind

Wasn't WES 2009 all about High availability BES 5.0 and all that. Well this goes deeper. We need to know how serious the problem is with the RIM infrastructure itself right now and that are taking the necessary measures to fix this.

Yeah, WES did focus very strongly on HA, however that's all on the client side, making sure if the client/admin messes up, then there's a failover to cover everything, unfortunaly RIM don't let many ppl poke around in their systems, so we can't tell you what kind of fallover they have.

Originally Posted by MobileMind

FIX IT NOW!!!!

I'm not sure RIM will, but I'm sure they're trying!!

[MobileMind]

Originally Posted by Jadey

Unfortunately you are unlikely to get that information anywhere before you'll get it from RIM - RIM are notoriously "quiet" about their major infrastructure outages. Most levels of t-support customers do not even get direct notifications. I have tried calling RIM t-support in the past, and the most info I get on T2 is "we are working on it". There are customers with a higher level of access to info with RIM, but many are bound by NDAs.

Bottom line - this site is not owned or affiliated with RIM. Until they release an official statement, we would all just be guessing what is wrong/why RIM have had 2 significant EMEA outages in past few weeks/whether it will happen again soon.

Fine and agreed we are all waiting on a word from RIM.

Also BES 5 high availability is not really relevant here. You can have as many HA configs as possible, but when the SRP link is down, it is game over.

I do not think I said that BES 5 was the problem and pointed to the deeper problem of the SRP link being down. So totally agree with you here.

I think you understand the fear/worry here. Blackberry is the best out there when it comes to large corporates and security/stability minded customers. But RIM have gone consumer in the last 6 months. Many of RIMs customers are desperate to know if they have sacrificed on quality to get the numbers in. WES2009 was all about restoring confidence in my view and I am very impressed with BES 5 and HA. But now it seems the real problems are somewhere else entirely- with RIMs infrastructure itself. RIM need to make a statement and they need to bring reassurance with it that they are investigating the actual issues and have solutions for them.

Mobile communications is the future but when these kinds of outages occur the confidence of those who want to start utilising this platform to improve, accelerate, localise and enrich their business processes is damaged. It's important that RIM do not simply shrug this off or dismiss it as being of no consequence but rather get the problem fixed.

[MobileMind]

Originally Posted by skyman84

Right,
After you've taken your chill pill. We'll chat!
http://static.pyzam.com/img/graphics/7/MJZ1634.gif

AFAIK the issues last week, and this one this morning are unrelated, but I've been known to be wrong in the past.


Well, yes is the answer, most ppl were asleep, the issue started aprox 6am GMT and resolved by 9am GMT, bearing in mind most of the users in this forum are in the US A) they weren't effected (unless their BES is in the EMEA) and B) it was 1am over there.

As for us UK and EMEA users, there was little down time (well, 3 hours, which in my mind is not horrific). I'm not sure how much it's effected you, but of my 3,400 users, I have no calls about it!



Well, that would be a job for RIM, give em a call, I'm sure they'd love to hear your comments.

This site is not in any way related to RIM, and it just a place for fans, users, admins, and experts to join together and share experiences and knowledge.



Yeah, WES did focus very strongly on HA, however that's all on the client side, making sure if the client/admin messes up, then there's a failover to cover everything, unfortunaly RIM don't let many ppl poke around in their systems, so we can't tell you what kind of fallover they have.


I'm not sure RIM will, but I'm sure they're trying!!

Drugs and drink don't work for me so I will pass on the chill pill. But take your point - get some perspective man!!! - it was not that long an outage and there were other reasons why I did not get that mail at 1am this morning!

Still - quickly sobering up and getting back to fear (Oh #### maybe my Blackberry is not perfect yet), anxiety (cause there is too much at stake here for RIM to #### this up) and desperate for an answer mode I do want that conversation with RIM!!!

[skyman84]

Originally Posted by MobileMind

Drugs and drink don't work for me so I will pass on the chill pill. But take your point - get some perspective man!!! - it was not that long an outage and there were other reasons why I did not get that mail at 1am this morning!

Still - quickly sobering up and getting back to fear (Oh #### maybe my Blackberry is not perfect yet), anxiety (cause there is too much at stake here for RIM to #### this up) and desperate for an answer mode I do want that conversation with RIM!!!

Yeah,
I think we all know RIMs BlackBerry product is the best at what it does out there, and that no technology is infallible.

I think we shall just have to see how the next 6/12 months pan out for them. You're right, they have tried to push into the consumer world more recently, (which is why BB is the number 1 smartphone for sales in the US at the mo) so you've got to give them a hand for keeping up with such an increase.

I'm confident personally that these are just little problems, and there will be some hickups along the road, but you've just gotta remember, the alternative is activesync!!!!

[MobileMind]

Originally Posted by skyman84

Yeah,
the alternative is activesync!!!!

Now that's funny!!!

[misterbulldog]

[QUOTE=Jadey;1383708]Unfortunately you are unlikely to get that information anywhere before you'll get it from RIM - RIM are notoriously "quiet" about their major infrastructure outages. QUOTE]

I had my first encounter of this type with RIM last week. We were having problems provisioning devices so I called the VZW. (VZW activations for us usually run very smooth). They said it wasn't them. I called RIM. They said it wasn't them. I called VZW again and one of the technicians said it was RIM and gave me a ticket number they had opened with RIM for the issue. I called RIM again and told the tech I had a VZW ticket number for the issue. He snickered and asked for the ticket number. He put me on hold and then came back and verified that there was a problem.

This was the first time I was aware of RIM withholding important information from their customers. At least now I understand the game.

[MobileMind]

[QUOTE=misterbulldog;1383819]

Originally Posted by Jadey

Unfortunately you are unlikely to get that information anywhere before you'll get it from RIM - RIM are notoriously "quiet" about their major infrastructure outages. QUOTE]

I had my first encounter of this type with RIM last week. We were having problems provisioning devices so I called the VZW. (VZW activations for us usually run very smooth). They said it wasn't them. I called RIM. They said it wasn't them. I called VZW again and one of the technicians said it was RIM and gave me a ticket number they had opened with RIM for the issue. I called RIM again and told the tech I had a VZW ticket number for the issue. He snickered and asked for the ticket number. He put me on hold and then came back and verified that there was a problem.

This was the first time I was aware of RIM withholding important information from their customers. At least now I understand the game.

Tried to phone RIM and got all the way to "service outage information" to receive an automatic information announcement "there are no outages at this time". This is of course true but does not help me find out what happened last night. Wrote email instead in the end.

Sometimes no information or deliberate cover up just creates suspicion about major #### ups.

[audit]

We do our best to have the outages posted to the Dataoutages News - Blackberry & Carrier Service Outage Information Center. site and also to the appropriate mail list when it happens to keep people up to date but we were short staffed at the time of the outage, I'm working on correcting that today by adding more people to update the site and the lists.

This outage was posted to the Bb-outage-intl Info Page last night as it happened though as you can see through the archives.

[MobileMind]

Originally Posted by audit

We do our best to have the outages posted to the Dataoutages News - Blackberry & Carrier Service Outage Information Center. site and also to the appropriate mail list when it happens to keep people up to date but we were short staffed at the time of the outage, I'm working on correcting that today by adding more people to update the site and the lists.

This outage was posted to the Bb-outage-intl Info Page last night as it happened though as you can see through the archives.

Fine so I was not the only one who experienced this and there are a list of witnesses on your site and rss feed.

Bb-outage-intl May 2009 Archive

But still no explanation from RIM