Criteo expands capacity for Big Data services

Background

Founded in 2005, global technology company Criteo helps advertisers generate more sales through personalized performance advertising on a global scale. As of December 31, 2015, Criteo had 2,000 employees in 28 offices across the Americas, EMEA, and Asia-Pacific, serving over 11,000 advertisers worldwide and with direct relationships with 16,000 publishers. Its proprietary predictive algorithms are able to deliver an advertisement with the right product, to the right user, at the right time. And, by measuring return on post-click sales, Criteo helps to make ROI transparent and easy to measure.

At the core of its operation is the largest privately-owned Hadoop platform in the world. This is an open-source software framework for storing so-called “Big Data” and running applications on clusters of commodity hardware. It provides massive storage for any kind of data, enormous processing power, and the ability to handle virtually limitless concurrent tasks or jobs. For Criteo, it enables up to 85,000 jobs to be processed every day. It’s a highly data-intensive business supported by a network of nearly 17,000 servers.

Challenges

As online retailing continues to grow in popularity, Criteo has seen a huge growth in demand for its solutions. The resulting increase in the data it handles meant that, by January 2015, it needed to look at building a new Hadoop cluster. Matthieu Blumberg, Engineering Director for Infrastructure Operations at Criteo explains:

“What we discovered with Big Data is that the more data you have, the more you want and the more you need. For us, it meant that our existing Hadoop cluster was becoming too small for our needs so we decided to build a new one. It also gave us the opportunity to challenge the whole infrastructure stack and look at alternative hardware vendors.”

Solution

A Request for Proposal (RFP) for the additional servers was sent to six companies, including Huawei and the existing incumbent. Based on their responses, a list of three potential solutions was selected. This was the point when Huawei started to stand out as a serious contender. Matthieu Blumberg continues:

“We were very impressed with Huawei’s response to our RFP. They had clearly done their homework and demonstrated a deep understanding of what we were doing with Hadoop. Not only were all the answers given very relevant, alternative suggestions were made for the server’s power supply, which clearly showed that its people fully understood how a Hadoop cluster runs.”

The solution offered by Huawei was based on its FusionServer, a new generation 2U dual-socket rack-mounted server. It provides flexible resource expansion capabilities, as well as high computing performance, and is an ideal choice for Internet, Big Data, cloud computing, and key enterprise applications.

As part of the evaluation process, Criteo purchased a batch of ten servers from each of the three vendors and built a small Hadoop cluster on each group, so that the performance could be compared. This bench-marking exercise assessed performance, Linux capability, power usage, and driver support. Scoring highly across the range, Criteo selected Huawei. Matthieu Blumberg adds:

“FusionServer more than met all the criteria we had set. But what really impressed us about the company was its willingness to help and answer our questions. For example, we wanted to access the hard drive at a level other vendors were not prepared to allow. Within days it readily provided the tools we needed to do this.”

Benefits

FusionServer compared well with its competitors. It offered the highest hard-drive density, supporting sixteen per server, one more than its nearest rival. And at 20 per cent less, its power consumption also proved to be significantly better. Crucially, for Criteo, it delivered everything it was looking for at the right price.

“After a few months experience we decided that Huawei’s FusionServer should be made available as part of our worldwide catalogue, alongside the hardware offered by our main supplier. So, since the start of 2016 we have been buying Huawei servers for our US, Europe, and Asia-Pacific sites,” says Matthieu Blumberg.

Looking to the future, Criteo sees Huawei as a potential major vendor for the many things needed to support a modern data center and has already purchased additional networking equipment.

Matthieu Blumberg concludes:

“Huawei provides the whole range of products needed to build a powerful data center. You can buy the modular container, the servers, and the network switches all from one single vendor. That’s definitely a very compelling proposition.”

For more information, please visit: http://www.criteo.com/