BlackBerry Forums Support Community               

Closed Thread
 
LinkBack Thread Tools
Old 12-27-2006, 04:12 PM   #1 (permalink)
New Member
 
Join Date: Mar 2006
Model: 7100
Posts: 2
Post Thanks: 0
Thanked 0 Times in 0 Posts
Default Network Outage

Please Login to Remove!

Anyone hear anything about a RIM network outage today (12/27/06) 4PM EST? Multiple users have lost connectivity at our Washington DC facility.
Offline  
Old 12-27-2006, 11:31 PM   #2 (permalink)
Thumbs Must Hurt
 
aviationwiz's Avatar
 
Join Date: Jun 2005
Model: Storm
Carrier: Verizon Wireless
Posts: 160
Post Thanks: 0
Thanked 0 Times in 0 Posts
Default

I had an outage up until around 3PM CST from at least noon CST in Minneapolis, MN.
Offline  
Old 12-27-2006, 11:40 PM   #3 (permalink)
New Member
 
Join Date: Dec 2006
Model: 8700
Posts: 1
Post Thanks: 0
Thanked 0 Times in 0 Posts
Default

Asian Quake's Telecom Disruption
Exposes Global Networks' Fragility
By JASON DEAN
December 28, 2006

BEIJING -- An earthquake off Taiwan's coast disrupted communications across the Pacific region and Asia yesterday, underscoring a crucial vulnerability of an increasingly globalized economy.

International phone service was cut off or restricted in some regions, and Internet service slowed to a crawl in much of China after the magnitude 6.7 temblor struck late Tuesday, followed by several aftershocks, damaging as many as eight undersea cables. Service to BlackBerry mobile email devices and Bloomberg financial-information terminals was interrupted, and some transactions in currency and other financial markets were disrupted.
WALL STREET JOURNAL VIDEO

[video]1
WSJ's Ian McDonald explains how an earthquake off the coast of Taiwan may affect shares of Asian telecoms2.

The repercussions of a single natural event that, while killing at least two people, caused limited physical damage, highlights the continuing fragility of telecommunications at a time when businesses are ever-more reliant on far-flung operations and vendors. Even some of the system redundancies designed to back up primary undersea cable networks failed to work, experts say.

With more and more multinational companies relying on distant regions of the world for outsourcing and component manufacturing, the stability of communications has become a primary concern.

Asia, which has seen hundreds of billions of dollars of direct investment in recent years, is home to some of the world's most earthquake-prone areas, and there are fewer cables connecting Asian countries to each other and the rest of the world than linking the U.S. and Europe, making networks there more vulnerable.

Yet Asia also is the world's fastest-growing economic region, creating ever-increasing demand for communications traffic. Indeed, some of the world's biggest telecom carriers -- including Verizon Communications Inc. and AT&T Inc. -- are moving to build new trans-Pacific cables to keep up with the surging volumes of Internet and phone transmissions.

Companies in the region for the most part said they found ways to work around Tuesday's earthquake. Among the best off were companies that buy service from telecom carriers with redundant cables or backup satellite systems for these types of disasters. Stock markets were unfazed: In Tokyo, the Nikkei 225 Stock Average closed up 0.31% at 17,223.15 points, while Hong Kong's Hang Seng Index rose 2.1% to 19,725.73, a record high.
[Detail of AsiaNetcom's global subsea network.]
Asia Netcom
Detail of global subsea network in 2005 from Asia Netcom. View the full network (.pdf)3.

But some telecom carriers had to scramble to find other paths for their customers' phone calls and data transmissions. This caused congestion and slowdowns on the alternative routes, much as side roads become clogged with detoured cars when a major highway is closed because of an accident.

Also, some businesses found themselves out of touch with critical operations and customers. For example, Hong Kong-based Kingstar Shipping, which manages 10 ships around the region, was unable to reach clients in Japan or Korea and some in Singapore, according to KL Tam, managing director. "We have been trying other ways to communicate -- but nothing much works," he said. Most of Kingstar's ships today have cargo now, but "if this continued for several days, our operations overseas would essentially stop," he said.

Telecom companies said repairs to the damaged cables could take weeks to complete, although connections will improve as companies find alternative ways to deliver service. Repairing broken cables can be difficult. Special ships must be sent out to pull up the cable, find the break, and repair it. Asia Netcom, which operates undersea cable networks, said it was able to restore its services late last night. But Chunghwa Telecom Co., Taiwan's biggest phone company, said it could take as long as three weeks before service returns to normal.

Telecom companies for decades have been working to insulate their terrestrial and overseas networks from natural and man-made disasters. Most strategies are based on building redundancies into networks so if certain equipment fails or lines are cut, traffic is rerouted to backup systems. Many networks are designed as loops so when a cable is cut, traffic can reach its destination by going in the opposite direction.

Information can also be transmitted long distances by satellite, but such wireless systems can't provide the speed and capacity that cable systems do -- and satellite transit typically is more expensive. Satellites, too, are susceptible to service disruptions. Still, telecom carriers often use satellites as a backup: Chinese state television said China Telecom Corp., China's biggest land-line phone company, was contacting counterparts in the U.S. and Europe about using satellites to make up for the capacity shortfall, according to the Associated Press.

Without redundancy, even a minor incident can turn into a major outage. For example, Pakistan's Internet service was disrupted for nearly two weeks in 2005 after a fishing boat cut what was then the country's only undersea cable line.
[Map]

The undersea cable systems that connect countries in Asia are designed with redundancies. But the trans-Atlantic cable systems have more backups because there are more cable lines crisscrossing the ocean.

Verizon alone has six cable routes that connect the U.S. and Europe, forming a "mesh" network that can restore multiple cable paths in milliseconds in case of a cut line. "The more cable paths you have, the more survivability you'll have," says Ihab Tarazi, vice president of global network operations and technology at Verizon of New York.

At this point, experts say they don't know exactly what caused the extensive outages in Asia. But they say the cables damaged by the earthquake -- connecting Hong Kong and Southeast Asia with Japan and, ultimately, North America -- run through an especially congested pathway between Taiwan and Hong Kong; the same route above water serves as an important shipping lane linking North Asia with Southeast Asia. The capacity of these networks to handle data and phone traffic were greatly reduced, experts say.

Chunghwa Telecom, whose service was among the most severely disrupted, said its capacity was down to 40% of its normal long-distance call volume to the U.S., and less than 10% of normal volume to Southeast Asia. Later in the day, the company found alternate cables.

In Japan, KDDI Corp. began shifting traffic that normally would have been handled by the disrupted cables through the U.S. and Europe, a backup plan that had been formulated for dealing with connection outages.

This week's incident "raises the question of whether there needs to be a new wave of investment" in these cables, says Duncan Clark, chairman of BDA China Ltd., a telecom consulting firm in Beijing. More cabling between Asia and Europe would make it easier to reroute international traffic in the event of such breaks, he says.

Executives at some telecom companies say yesterday's problems indicated they may need to beef up their backup systems even more. "Our plan had been in place for a long time," said Maki Sato, a KDDI spokeswoman. "But we are now studying whether it should be strengthened."

More cable lines also are on their way. Verizon plans to use mesh technology in a planned $500 million Trans-Pacific Express cable system that it announced last week in concert with five Asian carriers. Mr. Tarazi says the technology will only work when there are multiple cable systems. But others, such as AT&T and Asia Netcom, also are planning trans-Pacific cable projects.

The problem yesterday flows partly from a slowdown of investment in new cables globally in recent years. During the telecom boom of the 1990s, companies laid huge amounts of fiber-optic cable both within and between countries in anticipation of an explosion of demand. When growth didn't happen as quickly as expected, companies were hammered financially -- with several going bankrupt -- and investment in new fiber-optic capacity slowed sharply.

In the intervening years, however, use of the Internet and international phone service has grown quickly in Asia, making capacity more tight and often technologically outdated. The number of Internet users in China, for example, soared to more than 123 million as of June, from about 8.9 million users at the beginning of 2000.

--Li Yuan, Andrew Morse, Evan Ramstad, Geoffrey A. Fowler and Laura Santini contributed to this article.
Offline  
Old 12-30-2006, 09:10 AM   #4 (permalink)
Knows Where the Search Button Is
 
scaredpoet's Avatar
 
Join Date: Feb 2006
Location: NJ
Model: iPhon
Carrier: AT&T
Posts: 47
Post Thanks: 0
Thanked 0 Times in 0 Posts
Default

Quote:
Originally Posted by kira
Service to BlackBerry mobile email devices and Bloomberg financial-information terminals was interrupted, and some transactions in currency and other financial markets were disrupted.
While I can understand how Blackberry service in Asia would be affected by the quake, what would it have to do with service in DC and Minneapolis? RIM's central NOC is located in Waterloo, ON, not Asia. Surely they aren't routing traffic from Ontario through Asia before reaching the US.
Offline  
Old 01-01-2007, 06:44 AM   #5 (permalink)
Thumbs Must Hurt
 
aviationwiz's Avatar
 
Join Date: Jun 2005
Model: Storm
Carrier: Verizon Wireless
Posts: 160
Post Thanks: 0
Thanked 0 Times in 0 Posts
Default

I contacted Verizon about the downtime and heard this back from them, regarding the outage I experienced on the 27th:
Quote:
At 12:06 pm, messages sent to hosted mailboxes were accepted, but may not be delivered to the handheld, and mail for external integrations may not be delivered to the handheld until after the outage is resolved. Customers may not be able to send or receive messages, view attachments or do 'more' until after the outage is resolved??

At 3:30pm, we received this notice from RIM, ****UPDATE**** RIM experienced a database corruption issue. The database was failed over, and the queue has been cleared. Please re-test with your customers at that time...? The issue has been officially resolved.
Offline  
Closed Thread


Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On





Copyright 2004-2014 BlackBerryForums.com.
The names RIM and BlackBerry are registered Trademarks of BlackBerry Inc.