Communications Company is a Hero at Marquee Sporting Event

Customer Profile

This communications company has over 25 years of experience blending exceptional service, high-caliber entertainment, and communications products. Through multiple acquisitions and continual investment in infrastructure, they combine technology and entertainment to deliver critical communication services to their customers.

One segment of their business includes partnering with large venues to provide wireless access to attendees at mega-sporting events, concerts, and other live entertainment events. These are time-sensitive and consumers expect “always-on” wireless availability and performance, just as they do everywhere they go, to enhance their experience.

The Challenge

When the communications company was engaged to manage the wireless access at a high-profile, multi-day sporting event with international media coverage, they knew it had to be a flawless experience for the event staff that had work obligations that depended on availability to on-line information and for attendees wanting to follow event details from their mobile devices. Should issues occur that would impact the staff and attendees, it would be critical to respond rapidly and fix any service issue. At previous events, a reported problem could go to the central NOC and become delayed in a troubleshooting queue, leading to disgruntled end-users who would complain to event management. The communications company could not risk user dissatisfaction at this large event, as it could impact their reputation and ability to secure future engagements. However, they lacked visibility across the entire wireless access network. Making the situation even more difficult, their existing tools did not alert them when there was an issue. They needed a solution for proactive monitoring of access and performance to quickly address user-impacting issues as they occurred during the event.

Solution in Action

IT decided to rely on NETSCOUT’s nGeniusPULSE to monitor the wireless infrastructure throughout the event. With visibility to the entire wireless infrastructure, combined with alerts of performance issues, they quickly identified two critical issues.

First, they found that the main switch had high utilization – up to 90%. Processes were being “hung” on that switch, adversely affecting performance. By reviewing the Syslog messages with nGeniusPULSE, they saw a relation to a specific packet process on the switch. With this insight, they were able to change code to resolve the issue within the first 2 days of the event. They carefully monitored the CPU throughout the remainder of the event, and received email alerts from nGeniusPULSE to ensure performance was maintained.

An even more critical issue became apparent when some users were unable to access wireless connections or had very slow response times. However, the problem was not consistent, making it difficult to identify the root cause. Syslog messages were continuously showing errors and with nGeniusPULSE, they had a way to visually see in the dashboard the volume of errors. The customer quickly correlated these errors to a known bug within the wireless LAN controllers. As the access point reached a certain number of users, it would reject any new users, and users would have degraded performance. Without nGeniusPULSE, they would have misdiagnosed the issues as network congestion. They did a “band-aid” repair in the code, closely monitored it for the rest of the event, and looked like heroes to the event management staff and attendees!

The Results

The successful engagement at this high-profile event, managing and improving wireless access, will help this communication company win future engagements at large venues and events, which provide an opportunity to boost revenue long term. They now have full visibility to the wireless networks at any of their events and a proven process to identify and resolve problems.

IT can now ensure quality end-user experiences at other events. When problems do occur, MTTR (Mean Time to Resolution) is reduced by using nGeniusPULSE. They currently monitor over 20,000 devices - all network devices and any new wireless devices. Their approach is to continuously monitor this infrastructure with additional focus and alerts during an event.

The company has improved relationships and collaboration with third-party vendors by providing reports and details from nGeniusPULSE that pinpoint the source of the problem. Again, this speeds the time to problem resolution and improves the end-user experience.