BlackBerry Forums Support Community               

Closed Thread
 
LinkBack Thread Tools
Old 05-15-2009, 03:27 AM   #1 (permalink)
Thumbs Must Hurt
 
Join Date: Oct 2008
Model: 9780
OS: 6.0.0.359
PIN: N/A
Carrier: Telekom.de
Posts: 79
Post Thanks: 1
Thanked 0 Times in 0 Posts
Default RIM Outage May 15th

Please Login to Remove!

Our BES admins are told there is a problem with RIM over the early morning period of the 15th May 2009 1 am I missed a mail till about 10:25 am - so data services are not working in at very least Vodafone de or Vodafone Uk areas. Can people confirm and clarify whats going on. Also why are we suddenly having global outages after a period of good stability in last few months. Whats going on?

Last edited by MobileMind : 05-15-2009 at 03:34 AM.
Offline  
Old 05-15-2009, 03:28 AM   #2 (permalink)
Thumbs Must Hurt
 
Join Date: Oct 2008
Model: 9780
OS: 6.0.0.359
PIN: N/A
Carrier: Telekom.de
Posts: 79
Post Thanks: 1
Thanked 0 Times in 0 Posts
Default

Quote:
Originally Posted by MobileMind View Post
Our BES admins are told there is a problem with RIM at moment - so data services are not working in at very least Vodafone de or Vodafone Uk areas. Can people confirm and clarify whats going on. Also why are we suddenly having global outages after a period of good stability in last few months. Whats going on?
RIM is back now - but can someone tell me what happened? The RIM Network Operations Centers were not reachable for a period of time. No official statement as yet

Last edited by MobileMind : 05-15-2009 at 03:29 AM.
Offline  
Old 05-15-2009, 03:38 AM   #3 (permalink)
CrackBerry Addict
 
skyman84's Avatar
 
Join Date: Sep 2008
Location: London, UK
Model: 9900
OS: 7.1
PIN: ask!
Carrier: O2 UK
Posts: 932
Post Thanks: 0
Thanked 1 Time in 1 Post
Default

Issue was effecting the whole of EMEA but should be resolved as of 9am GMT
__________________
Simon
http://www.twitter.com/simonjhardy
Offline  
Old 05-15-2009, 06:08 AM   #4 (permalink)
Thumbs Must Hurt
 
Join Date: Oct 2008
Model: 9780
OS: 6.0.0.359
PIN: N/A
Carrier: Telekom.de
Posts: 79
Post Thanks: 1
Thanked 0 Times in 0 Posts
Default

Quote:
Originally Posted by skyman84 View Post
Issue was effecting the whole of EMEA but should be resolved as of 9am GMT
Yes but whats going on? Why aren't people talking about this - is this the wrong forum or were people simply asleep during the down time? This follows incident of April 28th also.

Details on Yesterday’s EMEA BlackBerry Outage from Zenprise, BoxTone | CIO - Blogs and Discussion

RIM need to restore confidence for Companies that are starting to place or plan business critical apps on their Blackberry environments and who need some assurances about the long term reliability, stability, security and scalability of the environment.

We had 10 months of no incidents and then suddenly kapoww this year somethings going wrong. Is everyone trying to hack Obamas Blackberry or have RIM simply not got capacity to cope with all their extra users, or have they got people who cannot be relied on running the show. Whatever the reason FIX IT NOW!!!!

Wasn't WES 2009 all about High availability BES 5.0 and all that. Well this goes deeper. We need to know how serious the problem is with the RIM infrastructure itself right now and that are taking the necessary measures to fix this.
Offline  
Old 05-15-2009, 06:44 AM   #5 (permalink)
BBF War Game Mod
 
Jadey's Avatar
 
Join Date: Oct 2006
Location: Denver CO
Model: Z10
OS: 10010614
PIN: SEEKRIT innit
Carrier: AT&T
Posts: 4,294
Post Thanks: 9
Thanked 29 Times in 23 Posts
Default

Unfortunately you are unlikely to get that information anywhere before you'll get it from RIM - RIM are notoriously "quiet" about their major infrastructure outages. Most levels of t-support customers do not even get direct notifications. I have tried calling RIM t-support in the past, and the most info I get on T2 is "we are working on it". There are customers with a higher level of access to info with RIM, but many are bound by NDAs.

Bottom line - this site is not owned or affiliated with RIM. Until they release an official statement, we would all just be guessing what is wrong/why RIM have had 2 significant EMEA outages in past few weeks/whether it will happen again soon.

Also BES 5 high availability is not really relevant here. You can have as many HA configs as possible, but when the SRP link is down, it is game over.
__________________
Jadey : Groupware Infrastructure Architect, Denver CO
If I'm not here, I'm playing World's End on FaceBook. Mob/Mafia Wars are SOO last year
Offline  
Old 05-15-2009, 06:53 AM   #6 (permalink)
CrackBerry Addict
 
skyman84's Avatar
 
Join Date: Sep 2008
Location: London, UK
Model: 9900
OS: 7.1
PIN: ask!
Carrier: O2 UK
Posts: 932
Post Thanks: 0
Thanked 1 Time in 1 Post
Default

Right,
After you've taken your chill pill. We'll chat!


AFAIK the issues last week, and this one this morning are unrelated, but I've been known to be wrong in the past.

Quote:
Originally Posted by MobileMind View Post
Yes but whats going on? Why aren't people talking about this - is this the wrong forum or were people simply asleep during the down time?
Well, yes is the answer, most ppl were asleep, the issue started aprox 6am GMT and resolved by 9am GMT, bearing in mind most of the users in this forum are in the US A) they weren't effected (unless their BES is in the EMEA) and B) it was 1am over there.

As for us UK and EMEA users, there was little down time (well, 3 hours, which in my mind is not horrific). I'm not sure how much it's effected you, but of my 3,400 users, I have no calls about it!


Quote:
Originally Posted by MobileMind View Post
RIM need to restore confidence for Companies that are starting to place or plan business critical apps on their Blackberry environments and who need some assurances about the long term reliability, stability, security and scalability of the environment.
Well, that would be a job for RIM, give em a call, I'm sure they'd love to hear your comments.

This site is not in any way related to RIM, and it just a place for fans, users, admins, and experts to join together and share experiences and knowledge.


Quote:
Originally Posted by MobileMind View Post
Wasn't WES 2009 all about High availability BES 5.0 and all that. Well this goes deeper. We need to know how serious the problem is with the RIM infrastructure itself right now and that are taking the necessary measures to fix this.
Yeah, WES did focus very strongly on HA, however that's all on the client side, making sure if the client/admin messes up, then there's a failover to cover everything, unfortunaly RIM don't let many ppl poke around in their systems, so we can't tell you what kind of fallover they have.

Quote:
Originally Posted by MobileMind View Post
FIX IT NOW!!!!
I'm not sure RIM will, but I'm sure they're trying!!
__________________
Simon
http://www.twitter.com/simonjhardy

Last edited by skyman84 : 05-15-2009 at 06:54 AM.
Offline  
Old 05-15-2009, 07:02 AM   #7 (permalink)
Thumbs Must Hurt
 
Join Date: Oct 2008
Model: 9780
OS: 6.0.0.359
PIN: N/A
Carrier: Telekom.de
Posts: 79
Post Thanks: 1
Thanked 0 Times in 0 Posts
Default

Quote:
Originally Posted by Jadey View Post
Unfortunately you are unlikely to get that information anywhere before you'll get it from RIM - RIM are notoriously "quiet" about their major infrastructure outages. Most levels of t-support customers do not even get direct notifications. I have tried calling RIM t-support in the past, and the most info I get on T2 is "we are working on it". There are customers with a higher level of access to info with RIM, but many are bound by NDAs.

Bottom line - this site is not owned or affiliated with RIM. Until they release an official statement, we would all just be guessing what is wrong/why RIM have had 2 significant EMEA outages in past few weeks/whether it will happen again soon.
Fine and agreed we are all waiting on a word from RIM.

Quote:
Also BES 5 high availability is not really relevant here. You can have as many HA configs as possible, but when the SRP link is down, it is game over.
I do not think I said that BES 5 was the problem and pointed to the deeper problem of the SRP link being down. So totally agree with you here.

I think you understand the fear/worry here. Blackberry is the best out there when it comes to large corporates and security/stability minded customers. But RIM have gone consumer in the last 6 months. Many of RIMs customers are desperate to know if they have sacrificed on quality to get the numbers in. WES2009 was all about restoring confidence in my view and I am very impressed with BES 5 and HA. But now it seems the real problems are somewhere else entirely- with RIMs infrastructure itself. RIM need to make a statement and they need to bring reassurance with it that they are investigating the actual issues and have solutions for them.

Mobile communications is the future but when these kinds of outages occur the confidence of those who want to start utilising this platform to improve, accelerate, localise and enrich their business processes is damaged. It's important that RIM do not simply shrug this off or dismiss it as being of no consequence but rather get the problem fixed.
Offline  
Old 05-15-2009, 07:09 AM   #8 (permalink)
Thumbs Must Hurt
 
Join Date: Oct 2008
Model: 9780
OS: 6.0.0.359
PIN: N/A
Carrier: Telekom.de
Posts: 79
Post Thanks: 1
Thanked 0 Times in 0 Posts
Default

Quote:
Originally Posted by skyman84 View Post
Right,
After you've taken your chill pill. We'll chat!
http://static.pyzam.com/img/graphics/7/MJZ1634.gif

AFAIK the issues last week, and this one this morning are unrelated, but I've been known to be wrong in the past.


Well, yes is the answer, most ppl were asleep, the issue started aprox 6am GMT and resolved by 9am GMT, bearing in mind most of the users in this forum are in the US A) they weren't effected (unless their BES is in the EMEA) and B) it was 1am over there.

As for us UK and EMEA users, there was little down time (well, 3 hours, which in my mind is not horrific). I'm not sure how much it's effected you, but of my 3,400 users, I have no calls about it!



Well, that would be a job for RIM, give em a call, I'm sure they'd love to hear your comments.

This site is not in any way related to RIM, and it just a place for fans, users, admins, and experts to join together and share experiences and knowledge.



Yeah, WES did focus very strongly on HA, however that's all on the client side, making sure if the client/admin messes up, then there's a failover to cover everything, unfortunaly RIM don't let many ppl poke around in their systems, so we can't tell you what kind of fallover they have.


I'm not sure RIM will, but I'm sure they're trying!!
Drugs and drink don't work for me so I will pass on the chill pill. But take your point - get some perspective man!!! - it was not that long an outage and there were other reasons why I did not get that mail at 1am this morning!

Still - quickly sobering up and getting back to fear (Oh #### maybe my Blackberry is not perfect yet), anxiety (cause there is too much at stake here for RIM to #### this up) and desperate for an answer mode I do want that conversation with RIM!!!
Offline  
Old 05-15-2009, 07:13 AM   #9 (permalink)
CrackBerry Addict
 
skyman84's Avatar
 
Join Date: Sep 2008
Location: London, UK
Model: 9900
OS: 7.1
PIN: ask!
Carrier: O2 UK
Posts: 932
Post Thanks: 0
Thanked 1 Time in 1 Post
Default

Quote:
Originally Posted by MobileMind View Post
Drugs and drink don't work for me so I will pass on the chill pill. But take your point - get some perspective man!!! - it was not that long an outage and there were other reasons why I did not get that mail at 1am this morning!

Still - quickly sobering up and getting back to fear (Oh #### maybe my Blackberry is not perfect yet), anxiety (cause there is too much at stake here for RIM to #### this up) and desperate for an answer mode I do want that conversation with RIM!!!
Yeah,
I think we all know RIMs BlackBerry product is the best at what it does out there, and that no technology is infallible.

I think we shall just have to see how the next 6/12 months pan out for them. You're right, they have tried to push into the consumer world more recently, (which is why BB is the number 1 smartphone for sales in the US at the mo) so you've got to give them a hand for keeping up with such an increase.

I'm confident personally that these are just little problems, and there will be some hickups along the road, but you've just gotta remember, the alternative is activesync!!!!
__________________
Simon
http://www.twitter.com/simonjhardy
Offline  
Old 05-15-2009, 07:31 AM   #10 (permalink)
Thumbs Must Hurt
 
Join Date: Oct 2008
Model: 9780
OS: 6.0.0.359
PIN: N/A
Carrier: Telekom.de
Posts: 79
Post Thanks: 1
Thanked 0 Times in 0 Posts
Default

Quote:
Originally Posted by skyman84 View Post
Yeah,
the alternative is activesync!!!!
Now that's funny!!!
Offline  
Old 05-15-2009, 08:32 AM   #11 (permalink)
Thumbs Must Hurt
 
misterbulldog's Avatar
 
Join Date: Feb 2006
Location: D.C Metro Area
Model: 9630
OS: 5.0.0.975
Carrier: Verizon
Posts: 163
Post Thanks: 0
Thanked 1 Time in 1 Post
Default

[quote=Jadey;1383708]Unfortunately you are unlikely to get that information anywhere before you'll get it from RIM - RIM are notoriously "quiet" about their major infrastructure outages. QUOTE]

I had my first encounter of this type with RIM last week. We were having problems provisioning devices so I called the VZW. (VZW activations for us usually run very smooth). They said it wasn't them. I called RIM. They said it wasn't them. I called VZW again and one of the technicians said it was RIM and gave me a ticket number they had opened with RIM for the issue. I called RIM again and told the tech I had a VZW ticket number for the issue. He snickered and asked for the ticket number. He put me on hold and then came back and verified that there was a problem.

This was the first time I was aware of RIM withholding important information from their customers. At least now I understand the game.
__________________
BlackBerry® Certified Systems Administrator
Offline  
Old 05-15-2009, 08:55 AM   #12 (permalink)
Thumbs Must Hurt
 
Join Date: Oct 2008
Model: 9780
OS: 6.0.0.359
PIN: N/A
Carrier: Telekom.de
Posts: 79
Post Thanks: 1
Thanked 0 Times in 0 Posts
Default

[quote=misterbulldog;1383819]
Quote:
Originally Posted by Jadey View Post
Unfortunately you are unlikely to get that information anywhere before you'll get it from RIM - RIM are notoriously "quiet" about their major infrastructure outages. QUOTE]

I had my first encounter of this type with RIM last week. We were having problems provisioning devices so I called the VZW. (VZW activations for us usually run very smooth). They said it wasn't them. I called RIM. They said it wasn't them. I called VZW again and one of the technicians said it was RIM and gave me a ticket number they had opened with RIM for the issue. I called RIM again and told the tech I had a VZW ticket number for the issue. He snickered and asked for the ticket number. He put me on hold and then came back and verified that there was a problem.

This was the first time I was aware of RIM withholding important information from their customers. At least now I understand the game.
Tried to phone RIM and got all the way to "service outage information" to receive an automatic information announcement "there are no outages at this time". This is of course true but does not help me find out what happened last night. Wrote email instead in the end.

Sometimes no information or deliberate cover up just creates suspicion about major #### ups.
Offline  
Old 05-15-2009, 09:25 AM   #13 (permalink)
Whoever
 
audit's Avatar
 
Join Date: Apr 2005
Location: Michigan
Model: xxxx
Carrier: AT&T
Posts: 1,213
Post Thanks: 36
Thanked 0 Times in 0 Posts
Default

We do our best to have the outages posted to the Dataoutages News - Blackberry & Carrier Service Outage Information Center. site and also to the appropriate mail list when it happens to keep people up to date but we were short staffed at the time of the outage, I'm working on correcting that today by adding more people to update the site and the lists.

This outage was posted to the Bb-outage-intl Info Page last night as it happened though as you can see through the archives.
__________________
audit

Win or Lose... Everyone Has Their Fight
Offline  
Old 05-18-2009, 03:02 AM   #14 (permalink)
Thumbs Must Hurt
 
Join Date: Oct 2008
Model: 9780
OS: 6.0.0.359
PIN: N/A
Carrier: Telekom.de
Posts: 79
Post Thanks: 1
Thanked 0 Times in 0 Posts
Default

Quote:
Originally Posted by audit View Post
We do our best to have the outages posted to the Dataoutages News - Blackberry & Carrier Service Outage Information Center. site and also to the appropriate mail list when it happens to keep people up to date but we were short staffed at the time of the outage, I'm working on correcting that today by adding more people to update the site and the lists.

This outage was posted to the Bb-outage-intl Info Page last night as it happened though as you can see through the archives.
Fine so I was not the only one who experienced this and there are a list of witnesses on your site and rss feed.

Bb-outage-intl May 2009 Archive

But still no explanation from RIM
Offline  
Closed Thread

« no bis | OTA issues »

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On





Copyright © 2004-2014 BlackBerryForums.com.
The names RIM © and BlackBerry © are registered Trademarks of BlackBerry Inc.