Can Big Data Alone Keep Up With Ad Tech?

Ad tech is unique, with its own distinct requirements and constraints. Digital advertising is becoming increasingly transacted via programmatic means, which demands technology that can not only accommodate extreme data volumes, but that can also process the data at the pace of real-time digital business. The question is, can ‘big data’ alone suffice for all the needs of the ad tech industry?

The Holy Grail of digital advertising is to reach the right consumer, with the right message, at the right time and in the right place. Keeping track of return on investment of marketing budgets with the right attribution is also very important. The challenge is to identify the right technology to mine the data and efficiently process it into a sell-able asset; it is the refining process that makes the raw data valuable.

At PubMatic, we understand that value lies in the quality of the data refinement. We are committed to providing high-quality reporting and analytics to empower our clients to leverage data at every stage of a campaign to inform programmatic activity and make smarter, faster business decisions. In building our platform, we defined three areas where we had to perform.

1. Volume of Data

2. Instant Decisioning

3. Manageable Cost

Volume of Data

Successful customer engagement in the ad tech space demands lightning-fast queries on high volumes of complex data. We must be able to accommodate larger data sets and deliver more complex deals and services to largest clients. The deployment must be flexible enough to provide cost-effective and easy-to-consume services.We want to give our clients the ability to translate massive volumes of complex data into digital insight at unparalleled speed, with streaming data analysis and streamlined machine learning. To do this, we augmented our Big Data platform with a new class of technology focused on accelerated parallel computing. With Kinetica, a Graphical Processing Unit (GPU)-powered database that contributes high-speed data processing capabilities, PubMatic can empower our customers with real-time reporting and a sophisticated ad pacing engine.

Instant Decisioning

Advertising is the lifeblood of the internet, and digital advertising is increasingly transacted online programmatically, with eMarketer estimating that over 80% of digital display ads will be bought programmatically this year. Programmatic buying and selling of advertising uses real-time bidding to match marketers, who are trying to reach consumers across desktop, mobile, and over-the-top devices, with publishers and media companies that attract people with content. Digital advertising demand-side platforms (DSPs), sell-side platforms (SSPs), and centralized data management platforms (DMPs) and exchanges are dealing with a fire hose of real-time data that needs quick analysis to make advertising tick. At PubMatic, we needed to be able to sweep through vast volumes of complex streaming data in milliseconds, in order to create, target, and deliver ads with incredible speed and our signature precision. Technology-wise, we rely on the speed and parallel-processing power of Kinetica’s GPU engine to get the job done. Artificial intelligence powered by GPUs can optimize auctioning by discovering patterns and uncovering hidden insights in sub-seconds. By running ad decisioning algorithms, it’s easier for us to target the right audience and display the ads likeliest to appeal to them.

Manageable Cost

Programmatic trading operates at significant scale, with PubMatic generating over 400 terabytes of uncompressed data each day and processing over 10 trillion advertiser bids per month. However, the value of each individual transaction is relatively low compared with other industries. Therefore, the cost per transaction must be lower than many other industries; that means the infrastructure footprint has to be smaller. The ad tech industry leads in defining next-generation data platforms that can handle huge data sets with lower cost per byte requirements. We’re confident that adopting our technology criteria can only positively impact everyone’s bottom lines.

The key to success is getting the right tool for the problem. Digital advertising operates in the realm of extreme data, where it’s all about volume, speed, and cost. The increase in data volume is unpredictable but the costs can’t be. While it is easy to get stuck with familiar technologies, big data alone is not enough to keep up with the pace of ad tech.

Editor’s Note: This article originally appeared in Forbes on 8/17/18.

Vasu Cherlopalle is a data and technology veteran with over 20 years of experience in various sectors, including advertising, unified communication, cloud computing and security. As VP of big data and analytics at PubMatic, he is responsible for the company’s industry-leading real time and batch analytics platform for programmatic advertising. Vasu is actively involved in IAB activities, helping to define next generation ad tech protocols and specifications, and has received the IAB Service Excellence Award for his contributions. Prior to joining PubMatic, he was director of cloud engineering at Polycom Inc., where he was responsible for delivering cloud-based unified communication solutions.