Assuring continuous availability for emergency services

Assuring continuous availability for emergency services

More people live in New South Wales than any other part of Australia, which leaves the state’s ambulance service with a very big job. In fact, the Ambulance Service of New South Wales (Ambulance) is the third largest in the world and responds to a request for assistance about every 28 seconds. That’s more than 1,133,000 emergency and non-emergency calls.

With lives and health under its care, Ambulance is uncommonly prepared for anything that might happen. The organisation has used a modern computer aided dispatch (CAD) system for many years. As daily demands for service increase—more than 22% in the last five years—Ambulance has upgraded and changed the system to keep pace. The most recent change involves hosting the CAD system — their most critical system — in a much more efficient way.
Ambulance decided to use a primary data centre to serve the four control centres that take calls and dispatch ambulances. In the past, the organization ran four separate instances of the CAD system at the Sydney, Charlestown, Dubbo and Warilla control centres.

Business Objectives
Hosting CAD at the four control centres had provided some resiliency because each site had its own independent system. The downside was the instances of the CAD system didn’t share the same database. The system performed extra steps to hand off information every time that a call taker needed to transfer an incident to another control centre. “We determined that we should look at an architecture whereby
there was just one single instance of the VisiCAD system that all four of our control centres accessed. There would be no need for this concept of transferring incidents from one centre to another, because they’re all accessing exactly the same data-base,” said Roger Hanssen, Director, Information Technology.

“So first we had to consider what would happen if that computer system goes down. It wouldn’t mean that just one control centre is out of action,” he added. The most critical service levels were required and continuity of ambulance dispatch operations in a consolidated data centre was to be provided. The system was specified to use the Microsoft Windows operating system and SQL Server database, in order to work with the CAD system and custom interfaces.

SDSI, a company which specialises in public safety software, recommended Stratus ftServer systems as the hardware platform for the new data center architecture. The Queensland-based firm provides application support and customisation to Ambulance, and is a distribution partner in Australasia for TriTech’s VisiCAD Command solution. Hanssen believes that maintaining continuity of operation is essential. “People don’t just ring for an ambulance during office hours," he explained. "It’s important that we have that 24/7/365 capability.”

Server Solution
“We looked at the Stratus ftServer as a platform that would give us internal redundancy to ensure that if something broke, the system kept going and could be repaired on the fly,“ he noted. Proven in the field to perform at the industry’s highest level of availability, Stratus ftServer systems prevent downtime
and data loss for Microsoft Windows-based applications and databases. The systems build in redundant components that serve as active spares, which keep the system running in the event of a problem. Self-diagnostic technology and proactive monitoring report potential problem conditions automatically and make troubleshooting fast.

In addition to the core VisiCAD Command software, the ftServer systems would also run critical interfaces, some custom-developed, that link computer aided dispatch to other, non-Stratus systems. The advantage of these interfaces is that data from the VisiCAD application and database gets presented to the other systems, and vice versa. These links include an interface to the mobile data terminals (MDTs) in the service’s 1,100 ambulances, which delivers information about each case to paramedics and lets them communicate with the CAD system.

Another interface connects CAD to the telephony system to pre-fill the call takers’ screens with details about incoming calls, such as call origin (automatic number identification/automatic location identification, or ANI/ALI). Additional interfaces include a link to a web-based booking system that doctors’ offices and hospitals use to schedule non-emergency transportation, such as taking a patient from a hospital to a nursing home. The newest is an interface to an electronic medical records system (on another server).

Paramedics use customised laptops in their ambulances to record patient information, which the ftServer-based VisiCAD system transfers to the hospital emergency department where the patient will be admitted.

Creating a disaster recovery site
“Another contingency plan was to establish a DR data centre which we didn’t have at that stage. Not only was there the internal redundancy within the Stratus platform, but we’d have the fall back of a mirrored system in another physical site,”
suggested Hanssen. “And that continuity plan is focused not just on the Stratus
platform, but the whole data centre. The data centre has uninterruptable power supplies, a backup generator, redundant air conditioning, and so on. Even so, if there was a major event at our primary data centre, we could quickly transfer to our DR data centre and continue operations,” he added.

Ambulance maintains its DR data centre as a warm standby. The facility runs the same system as the primary site, including six ftServer systems, VisiCAD Command software and all the CAD interfaces. Ambulance stays primed to respond under any circumstances. The primary and the secondary data centres are on opposite sides of Sydney harbour. These locations put them in separate telephone exchanges and on different parts of the electricity grid.

Business continuity planning is so thorough that personnel regularly test manual dispatch procedures, so they have practice in working with or without the CAD system.
Proactive monitoring saves time Hanssen said that Ambulance has been helped by scaling down the number of separate CAD instances, the IT staff needs to support, while having the always-on availability they need.

“The other advantage in going to the Stratus platform is the proactive monitoring. It’s very effective," he added. "So, if there’s something that’s not quite right, we’ll get a call from the service centre for Stratus to let us know.”

There has been an occasional need to replace a component in one of the ftServer systems. While the server continued to run normally using a redundant component, Stratus’ monitoring got the replacement component to the data centre’s door the next day.

Hanssen explained how the experience is different from using traditional server hardware. “In our previous scenario, if we had an equipment failure, it was up to us to identify that we had an equipment failure and then report it to the vendor. The proactive monitoring that Stratus provides is certainly a huge advantage, because it means we don’t have to do all that work on our own,” he said.

Business Impact
Ambulance rolled out the implementation in steps after proving the consolidated data centre concept could deliver the reliability. For several years now, the data centre and its Stratus ftServer systems have supported all four control centres in the state. Hanssen summed up the impact. “Our data centre architecture is simpler to manage yet we still have the redundancy. The Stratus platform ensures we can have continuity of our services. The servers’ fault-tolerant design means we have confidence in the system maintaining 24/7/365 operations.