Brilig Powers Smarter Advertising Using Big Data and GoGrid's Cloud Infrastructure
Founded in 2009, Brilig operates the world's first cooperative data marketplace for online display advertising. More than 100 co-op members—both buyers and sellers of data—participate in Brilig's open market for data. Brilig has essentially "de-siloed" thousands of databases with information about potential consumers of online display advertising. Advertisers can buy space in more than 7,500 consumer segments. Want to reach women earning $100,000 who pamper their dogs? No problem. It takes a lot of data to generate all those segments. In fact, it takes big data. Brilig relies on Hadoop to make its huge store of data useful and on GoGrid to host its big data infrastructure.
"Brilig wouldn't exist without the distributed data architectures, like Hadoop, associated with big data," says Paul Cimino, CEO of Brilig. "Just a few years ago, it would have cost millions of dollars to house and take advantage of data in the way our co-op members do inexpensively every day. Big data represents a profound change in the usefulness of the huge amounts of data we generate. The cloud is just as important. By running our solution on GoGrid's cloud infrastructure, we're able to process more than 300 million requests per day, without tying up our capital in database or server hardware."
The Big Data Challenge
From the very beginning, Brilig relied on infrastructure as a service (IaaS) to run its solution and to house its data. Even prior to launching, the company realized that its data-intensive business model would require ever more infrastructure. It didn't want to tie up its capital purchasing hardware or require its staff to spend their time and expertise maintaining a data center. Instead, Brilig launched in 2009 with an IaaS provider that used only virtualized hardware in its infrastructure. The approach worked well initially but, as Brilig grew, cost and performance issues led the company to explore other options.
"It's absolutely essential that our big data solution perform quickly," says Mark Kemna, Brilig's CTO. "Slow processes lead to slow ad serving, and no one—not ad networks, not web users, not publishers—wants that. We were concerned that an entirely virtualized environment couldn't keep pace with our big data solution. Looking at costs, we also wondered if the pricing suited us over the long term. Were we really using the best provider for us? We decided to find out."
A flexible network architecture for Big Data: Dedicated Servers provide high disk performance in a private environment and Cloud Servers offer elastic scalability for application and web servers in production and staging environments.
Brilig questioned whether an entirely virtualized environment was the best option because of two drawbacks inherent in the technology. In a virtualized environment, users share hardware. This approach works fairly well for serving static data, but not for the big data processing essential to the Brilig solution. Users in a virtualized environment also share more than just hardware—they compete for read/write bandwidth for database functions. Hadoop functions best when it has large amounts of bandwidth, however, which can lead to higher costs to gain acceptable performance.
GoGrid, in contrast, lets users take advantage of a hybrid environment, with virtualized servers for less read/write-intensive functions and dedicated hardware for big data processes. Kemna explains the advantages Brilig saw: "GoGrid dedicated hardware runs Hadoop efficiently, so the costs are as much as 50 percent lower. You also get a security advantage when using dedicated hardware to store data. But the biggest plus is performance. We get significantly better performance with GoGrid."
"We decided to make GoGrid our primary cloud infrastructure provider," adds Fahd Sultan, director of IT infrastructure for Brilig. "For redundancy, we chose to maintain a presence with our first provider as well. GoGrid has more than met our expectations. Compared to our first cloud provider, GoGrid is as much as four times faster for big data functions. We're achieving much better performance for a lower cost, too."
Service in Milliseconds
Brilig worked with the GoGrid team to plan its capacity needs, understand the working parameters of hybrid architectures, and set up its IaaS environment. Virtualized cloud servers run Brilig's key-value datastores, MySQL databases, and application and network management servers. Customized dedicated hardware runs Hadoop, with Hive (an open-source data warehouse solution designed for Hadoop) helping to facilitate queries. "The system answers 99 percent of all queries in less than 3 milliseconds, so the performance is there," says Sultan. "It's quite scalable, too. As our wealth of data and co-op grow, we can easily add cloud and dedicated capacity as needed."
Enhanced Service – For Less
Thanks to the lower cost of its current technology configuration, Brilig has been able to pass its savings along to customers in a number of ways. For example, Brilig offers a service that lets companies that collect offline information, such as direct mailers, add data to online datasets. In a sense, Brilig works to "cookie-fy" the data. When an ad network buys the data, the data provider sees a return. Brilig used to charge providers to process the data, but now it doesn't.
"Making offline data accessible took so much processing power in a virtualized environment that we had to charge a fee," reports Kemna. " The lower cost of running Hadoop on GoGrid makes that approach unnecessary. We hope to grow as much as tenfold every year. Not charging fees encourages organizations to join the co-op. We thrive on revenue share. So when our members win, we win."
The Cloud or Nothing
At publication, Brilig used the equivalent of 100 machines on the GoGrid hybrid infrastructure. With none of its capital going to infrastructure hardware, the company can focus its resources on attracting new members and enhancing its offering with innovative technology. The alternative—operating its own hardware—is almost inconceivable to leaders at Brilig.
"If we owned and operated our hardware, we'd need as many as three full-time network engineers to support it," explains Cimino. "As we grow, that number of engineers would grow, too. Our business model simply wouldn't work very well without the cost advantages of GoGrid. Plus, the high level of performance we get from GoGrid adds to the value."
Using GoGrid, Brilig has been able to:
- Support an annual tenfold increase in the use of its service
- Attract more data co-op members with low rates
- Process Big Data queries as much as 4x faster
- Reduce infrastructure costs by as much as 50%
"By running our solution on GoGrid's cloud infrastructure, we're able to process more than 300 million requests per day without tying up our capital in database or server hardware."
– Paul Cimino, CEO
OTHER CASE STUDIES
- Welcome to GoGrid!
- I'm a Cloud Infrastructure and Big Data Solutions expert.
- What questions do you have today?