The rise of big data and data management challenges

Research by IDC predicted that by 2025, the total volume of data stored globally will reach 175ZB. That’s a 5-fold increase from 2018 (33ZB). To put this into context, one zettabyte is equal to one trillion GB. Stored on DVD media, that would be a stack of disks spanning to the moon and back 12 times. But how does this data break down, and how much of this can be used to inform better security decisions to keep enterprises safe from targeted attacks?

Of the expected 175ZB, roughly 85 per cent is enterprise and public cloud data storage. More importantly, IDC predicts that by 2025 as much as 30 per cent of this data will be classified as ‘real-time, sensorised’ telemetry from endpoint and IoT devices. This presents an enormous challenge – as well as opportunity – for enterprises looking to improve their security posture by leveraging this abundant wealth of data.

It is worth noting that data alone is not useful, and that more data does not become more valuable by volume. Data must be contextualised and analysed to become information. By that same understanding, we know that information only becomes knowledge once we apply meaningful links between multiple information points, assembling the contextualised data into actionable results. Therefore, data without context tends to be superfluous, and our human brains quickly try to expel such unimportant bits of data.

Effective data management requires context

Today, most enterprises generate mountains of telemetry data for each entity, including the activity logs from users, devices, applications, and sensors. In this ‘age-of-observability,’ we can be confident that nothing important happens without a corresponding record of it having occurred.

Information security has taught us that even the most innocuous and banal data sets might somehow be relevant in the scope of an investigation or malicious detection. Frequently, we don’t know what we don’t know until long after a successful breach from a stealthy adversary. While most attacks can be thwarted by an effective endpoint detection and prevention platform, analysis of the breadcrumbs trail left behind is an effective means to identify the attackers’ TTPs (tactics, techniques, and procedures) and possible motivations, and the scope of an attack.

Analysing data

The volume of sensor data is not the only significant challenge facing enterprises today. The sheer number of telemetry sources, combined with the unique nature of each data source (different formats, content, and context), has created a challenging data problem for today’s enterprise. To effectively consume, parse, enrich, normalise, store and analyse this massive data set is not a cost-effective proposition for most organisations. As a result, most enterprises face the burden of selectively choosing which data sources to process based on the perceived value of each related to business process improvement or greater security efficacy.

Combining these disparate and quite unique sets of endpoint, cloud, network, and security data in one location is costly. The value realised is often difficult, if not impossible, to justify. As enterprise security architectures become more diverse, it is more important than ever that cross-vendor data analytic models become part of an effective detection and protection arsenal.

Rogue devices and shadow IT creates information blind spots

Attackers are opportunistic and will target any and all exposed devices – not just those known to the security operations team. As the enterprise attack surface expands due to IoT, cloud transformation, and BYOD), so too does the need to expand our sources of telemetry, minimising or eliminating any blind spots that inevitably exist.
Most organisations struggle to maintain an accurate inventory of connected devices, and fewer yet have the ability to identify when rogue or orphan devices appear on the network that could pose a potential security risk.

By harnessing the existing sensor grid – and the data collected from it – enterprises can more quickly identify gaps in security coverage to protect more of the attack surface. When event volumes from existing sensors change without a justified policy modification, security operations can be notified to ensure a configuration change – whether malicious or benign – hasn’t left the device in a state where logging is disabled or reduced.

Therefore, enterprises must adopt intelligent solutions that provide visibility into the entirety of their devise estate, enabling security operations teams to quickly identify unmanaged/at-risk devices, fingerprinting their characteristics, and highlighting those without protection capabilities. These solutions can then perform remote agent installation and policy enforcement of supported systems to reduce the enterprise attack surface and improve an organisation’s security posture. Overall, the solution to data management challenges is a strategy that democratises the data generated, collected, and analysed by an enterprise.

Tamer Odeh is the regional director at SentinelOne in the Middle East