When leading cybersecurity expert Neil Rerup commented, “True cybersecurity is preparing for what’s next, not what was last,” he hit the nail on the head. In the past, the security tools that organizations relied on were really more reactive than proactive, and they lacked the bandwidth needed to deal with massive amounts of data in real time.
Fast forward to today’s Extreme Data Economy, where companies must handle massive sets of complex and unpredictable data at unparalleled speed, requiring streaming data analysis, visual foresight, and streamlined machine learning capabilities just to maintain business in motion. Combine this challenging environment with increasingly sophisticated cybersecurity threats, and the stakes have never been higher. By December 20, the Identity Theft Resources Center (ITRC) had recorded 1,293 U.S. data breaches in 2017, exposing more than 174 million confidential records. In fact, cyber attacks are the fastest growing crime in the U.S., and they are increasing in size, sophistication, and cost.
Challenges to Cybersecurity
In this new Extreme Data Economy era, it’s clear that as cybersecurity attacks increase in both volume and sophistication, it’s becoming more and more difficult to protect systems, networks, and programs from digital attacks due to:
Increased data volume: Companies can easily generate gigabytes of data per second just from normal activities. In addition, you have sensors, cloud-based services, and mobile device data that are added to the mix. The more data points you have, the more potential for cyber attacks.
Traditional tools lack scalability: Tools and infrastructures that are solely CPU-based systems are very costly to maintain due to immense hardware sprawl required to achieve the level of compute needed with CPU cores. Modern cybersecurity solutions have to be able to scale to unprecedented levels given the growing volume and variety of data.
Data disorganization: Cybersecurity data is rarely organized so that it’s easy to retrieve and analyze.
A wide variety of security tools: Large companies typically use dozens of different types of security tools, and each one often has a different data and reporting format.
Discover Cyber Threats Faster with Kinetica’s Insight Engine
Performing real-time analysis on the volumes and disparate types of transaction, trade, and machine data necessary for fraud detection and cybersecurity is a challenge for even the most advanced data systems. Think of a cyber attack as a human disease, constantly morphing and evolving into something even more powerful. The reason a company gets attacked by a “virus” is because cyber attacks can pinpoint a security gap that the company didn’t account for and isn’t protected against, similar to a flu virus that attacks when our bodies are weakened and our immune systems are down.
Cyber data can be queried by any number of ways, including source IP, destination IP, packets, and past cyber events that have been logged. Kinetica’s insight engine, which is at the center of the Extreme Data Economy movement, is able to ingest streaming cyber data and simultaneously analyze the data in order to uncover relationships, anomalies, and patterns that signal potential fraud or suspicious activities and predict/prevent potential attacks before they happen!
In cybersecurity, because the payload is so fast, the data feeds are some of the hardest to process in real time. You’re literally ingesting data at the rate of 10-40 gigabits per second, with a wide array of different attributes. If your organization is experiencing a cyber attack, and you don’t know what to search for, it’s essential to have access to the brute force computational power of GPUs, so that you can perform real-time analytics and forensics on this live data.
Kinetica is able to do this by exploiting the power of GPUs and 1000s of cores. This makes Kinetica exceptional for the types of brute-force calculations needed for real-time analysis of cybersecurity data, and offers vast improvements when looking for anomalies in diverse streams of transaction and log data. Developers using common technologies are forced to create specific schemas and indexes, because they’re trying to optimize for certain common queries that users are going to be searching, filtering, and looking for. Kinetica eliminates the need for developers having to create clever schema optimizations or sophisticated indexes.
As a real-time discovery platform which can run a priori queries against the entire data corpus with extreme 10GbE-40GbE extreme data feeds being ingested, Kinetica enables companies to find that elusive “cyber threat needle in the haystack.”
Core capabilities include:
GPU-accelerated insight engine: Kinetica’s accelerated parallel computing brings thousands of GPU cores to address the unpredictability and complexity that result from extreme data.
Unparallel speed: Dramatically accelerate analysis with advanced parallel computing, featuring an in-memory insight engine with advanced GPU abstraction technology. Run SQL queries to process and analyze billions of rows in milliseconds. Kinetica enables simpler data preparation through distributed ingestion and extraction without the need to index.
Streaming data analysis: Ingest and process against the entire data corpus at the same time to get instant results. Run SQL queries on streaming and geospatial data, continuously collect, analyze, and ingest streaming data with historical data with Kinetica’s all-in-one solution to ingest, compute, visualize, and egress data. This enables cyber anomaly detection engine at scale.
Visual foresight: Instantly translate temporal, geospatial, and streaming data into visuals that reveal patterns and opportunities. Kinetica’s insight engine enables in-memory, distributed image processing and rendering, interactive visual discovery optimized for temporal and geospatial analysis, and an integrated visual dashboard.
Streamlined machine learning: Train models faster and predict more accurately with integrated machine learning workflows. Seamlessly integrate pre-existing code with Kinetica’s highly scalable solution. Simultaneously process and manage both BI and AI workloads, and integrate workflows with open source machine learning libraries. Use Kinetica to fuse multiple data feeds and detect anomalies, monitor multiple streams of global attack vectors, find security lapses, and mine system logs. Move from signature detection to behavioral analysis to better predict and prevent attacks. By applying machine learning and deep learning, companies can proactively and predictively guard their enterprise network rather than reactively.
Real-time Cybersecurity Case Study
We have a customer whose main objective was to identify a broad range of malicious activities in real time with confidence to protect users against current and emerging cyber threats. With a deep collection of all network traffic data, infrastructure data, and log data, Kinetica helped the customer provide additional device context and insights into user activities. By using Kinetica, the firm was able to perform deep packet and session-level analysis, and target the communications and activities of bad actors. Additionally, Kinetica’s advanced in-database analytics via user-defined functions (UDFs) allow data scientists to operationalize machine learning libraries such as TensorFlow, Caffe and Torch (or custom logic)—bridging the gap between data science, data analytics and business intelligence. Data scientists can deploy sophisticated AI models to mitigate the risks of potentially damaging zero-day threats (an attack that exploits a previously unknown security vulnerability).
With Kinetica’s insight engine, you can:
- Perform geospatial searches to look for denied network events.
- Execute any kind of SQL query that you normally would run in a traditional database.
- Use Kinetica’s Reveal visualization framework to slice and dice your data in multiple different ways. Enjoy point-and-click access to machine learning, deep learning, and OLAP on streaming data to act on threats. Or connect with any industry-standard visualization tools through ODBC/JDBC connectors.
- Conduct complex joins against data tables with billions of rows in sub-second time.
- Investigate deep packet inspection in real time in order to detect network traffic anomalies.
When extreme data and cyber attacks require your organization to act with unprecedented agility, Kinetica is there to power and protect your business in motion. Kinetica’s distributed data management architecture enables you to analyze cyber data and potential attacks in real time, so you can protect customers against current and emerging cyber attacks.
In the cybersecurity space, enterprises need a secure and resilient system to support their mission critical workloads and mitigate threats from all stages. Watch this brief demo to get a glimpse of how Kinetica’s insight engine can be used to detect and predict data breaches before they occur:
Want to learn more about the Extreme Data Economy? Register for the upcoming webinar and learn how your organization can thrive in this post-big data era:
The Extreme Data Economy: How Businesses Thrive in the Post-Big Data Era
Thursday March 22, 2018, 10:00am Pacific Standard Time
