Clarke Power Services

When Clarke Power’s server and storage infrastructures shut down, Logicalis and HP responded quickly to help bring company systems back online

Storage Volume Migration Expected to Go Off Without a Hitch
Clarke Power Services, a commercial vehicle maintenance provider based in Ohio and founded in 1964, offers services that keep transportation assets working and equipment lifecycle costs on track. The company supports fleets of all sizes and equipment types with capabilities ranging from complete outsourcing to individual truck repairs and preventative maintenance. Operating 27 full-service shops in nine states, Clarke is an authorized service center for Detroit Diesel, Mercedes-Benz truck engines, Sterling trucks and Allison Transmission.

Soon after upgrading its HP blade server firmware and VMware virtualization software, the IT team at Clarke Power Services spent a weekend upgrading the volumes of its HP EVA storage-area-network (SAN). Clarke had planned to replace the aging SAN within a few months, but the IT team was confident the migration to the new storage volumes would proceed with no interruptions to the business.

Corrupt Data Requires Emergency Response
Just as Vice President of Information Technology Mark Brinkman prepared to go home that Saturday night, two print servers failed. Given how late it was, Brinkman decided to wait until Sunday to bring them back up. When he came in Sunday morning, the email server was down. That’s when he knew he might have a serious problem.

“We now think that some of the data on the SAN volumes was corrupted due to past power failures when we experienced hard shut downs,” Brinkman explains. “But our data remained stable while the storage volumes were in their previous state. Once we moved the volumes, we think that’s when the corrupt data brought our servers down.”

At the time, Brinkman was not sure of the cause of the failures.He and his staff were able to bring the email server back online that Sunday, but by Monday morning, just as the work week started, the email server went down again. The team shut down the entire system for fear of a virus spreading throughout the server and storage infrastructures. 

Logicalis and HP to the Rescue
That’s when Brinkman called upon his two most trusted IT partners—Logicalis and HP—which collaborate frequently to design, deploy and support technology solutions for end-user customers. Logicalis has designed and deployed data center and infrastructure solutions for Clarke since 2003, including HP server and storage technology.

“Logicalis always brings in the right resources for anything we need,” Brinkman says. “They understand our environment, how the company operates, and how IT operates. We knew the Logicalis-HP combination would get us out of this jam.”

One of the key reasons Brinkman prefers to deploy HP technology is that he can also purchase VMware and other server and storage operating system software from HP. This allows him to turn to just one vendor for both hardware and operating-system issues.

“We don’t have to worry about the hardware provider and software provider pointing fingers at each other when problems arise,” Brinkman says. “And that’s critical when facing a catastrophic event like we were at this time.”

Full Back-Up Plays Key Role in Restoring Business Operations
Over the next few days, Logicalis and HP worked around the clock to bring the Clarke server infrastructure back online. HP kept the phone line constantly open, even as help desk personnel changed shifts. Multiple members of the Logicalis system engineering team and sales team came on-site to assist, bringing in meals and cots to sleep on overnight. Brinkman demonstrated his personal dedication to the company by staying on-site for four consecutive days without going home.

“Both Logicalis and HP demonstrated just how much they felt our pain,” Brinkman adds. “They knew that bringing our systems back online was critical and dived right into the trenches with us.”

A key factor in Clarke’s favor was that Brinkman and his team followed stringent protocols for backing up the server and storage infrastructure. They thus had a full backup right to the moment before the system shut down. With advice and help from Logicalis and HP, Brinkman’s team first installed an emergency server to operate basic business functions so Clarke could continue to take orders and provide services for its customers.

“With the full backup and the emergency server, we successfully kept the business open,” Brinkman says.

Temporary Solution Provides Flexibility for Future Use
Clarke then collaborated with Logicalis and HP to install a longer-lasting, more robust storage solution. The two companies expedited an order for an HP P2000 Modular Smart Array, which kept the company running at normal levels until the permanent solution could be deployed.

“We can use the temporary HP StorageWorks system for a planned future solution at one of our other locations, so it was not a total sunk cost,” Brinkman says. “This demonstrates how Logicalis helped us work through this emergency as cost-effectively as possible—as opposed to us having to buy temporary technology we could never use again.”

Once Logicalis deployed the new permanent solution—nine HP ProLiant BL460 blades supported by an HP BladeSystem c7000 chassis and 50+ terabytes of HP 3PAR StoreServ P7400 Storage—Clarke gained a much higher level of system resiliency. The newer technology and a four-controller architecture—as opposed to the previous SAN’s two controllers—provided Clarke with a more robust system.

Logicalis also designed the server chassis architecture to include HP Virtual Connect, which eliminates the need for SAN fabric by enabling flat SAN connectivity. With a direct connection from the blade servers to the storage system, Clarke benefits from increased availability and throughput.

Offsite Data Center Provides Complete System Redundancy
“We were then at the point where we were much more capable of withstanding one of our storage volumes experiencing a problem,” Brinkman says. “But we wanted to go further and ensure we could continue to function even if the main data center experiences a power outage or a disaster strikes our main building.”

To meet this need and provide Clarke with complete system redundancy and failover capabilities, Logicalis designed a solution to replicate the server and storage infrastructure to an identical infrastructure at an off-site co-location facility 120 miles from the main data center. Once fully deployed, the two systems will replicate data back-and-forth through a wide-area-network.

In addition to greater resiliency through the off-site data center, the new solution gives Clarke more robust application access to the storage and servers. End-users have noticed an improvement in general network performance, as the change to solid-state storage tiers and flash storage has improved application and query response times.

“Thanks to Logicalis and HP, we are now in much better shape both in terms of infrastructure performance and our ability to recover should a disaster strike again,” Brinkman says.