Increasing efficiency of disaster recovery processes

Brightsource’s IT team is responsible for the infrastructure supporting the company’s data processing solutions and automation services that enable clients to create compelling marketing collateral and drive successful campaigns.

Brightsource had been relying on a virtualised IT environment with HP hosts and EMC VNXe storage appliances at its Cheltenham and disaster recovery data centers, which served additional offices in London and Edinburgh. The company runs very large data processing jobs and requires the ability to quickly process runs of over six million records. Within this environment, the company was reaching the limit of both the CPU cycles the hosts were offering and the throughput of the SAN. The replication processes were also inefficient. A small modification to one field in the database would require the IT team to replicate the entire database utilizing all of the company’s bandwidth.

In addition, every time a change was made to any of the company’s replication configurations, the IT team had to shut down its entire virtual machine (VM) environment and restart the storage processes. This exercise would take upwards of 90 minutes requiring the IT team to complete the process outside of production hours.

These challenges led the company to look for a new storage solution. The IT team tested the Nimble Storage CS-series array with the goal to process six million records in 10 hours. “We put the Nimble array through our most rigorous testing without even pushing the Nimble array,” said Ed Rollinson, head of IT for Brightsource's corporate marketing division. “To be honest, after seeing the Nimble arrays in action, the product really sold itself.”
As a result of deploying Nimble’s hybrid storage solution, Brightsource now replicates all of its application data every hour, its VMs every 12 hours, work in progress (WIP) every three hours, and the company’s archive data every 48 hours. Rollinson added, “Previously, we were only able to look at one single data set with just one replication schedule. Now we can separate our data sets out and create different replication schedules depending on the urgency and importance of the job and recovery objectives for each category.”

The IT team also experienced a reduction in replication bandwidth consumption. While its previous storage environment processes were saturating Brightsource’s 20Mb link, Nimble’s WAN-optimised replication only consumes a small subset of the available bandwidth, reducing stress on other corporate services.

For performance and replication monitoring Brightsource utilises Nimble InfoSight, which uses sophisticated algorithms and powerful data analytics that enable trending, predictions, and forecasting for Nimble storage environments.

“With InfoSight, we can easily predict what we will need and how our systems will respond to various scenarios. I can pull up a performance report during meetings with our executives and say, ‘for this replication job, this is the bandwidth we are now using, and this is what we will need in the future.’ Having that information at our fingertips is an absolute dream for us. Nimble InfoSight showed us we were using only 0.6Mb of bandwidth to process our data,” Rollinson stated. “That’s just brilliant.”