splunk_escuHunting

Cisco Isovalent - Kprobe Spike

This analytic detects excessive kernel probe (kprobe) events in a Kubernetes cluster over a short period of time. Kprobes are a Linux kernel debugging and instrumentation mechanism that allows dynamic monitoring and tracing of kernel functions and system calls. In containerized or cloud-native environments, kprobes are occasionally used for legitimate low-level diagnostics; however, monitoring a spike in kprobe activity is important because malware or attackers may abuse this mechanism to gain insights into the kernel, attempt privilege escalation, or tamper with host processes. More than 10 kprobe events within 5 minutes may indicate suspicious activity, such as an attacker probing the kernel through repeated system calls (e.g., nsenter, mount, sethostname). Such abnormal volume and frequency of kprobe usage within application pods or on nodes can signal container escape attempts or low-level tampering with the host, thereby representing a potential security threat.

MITRE ATT&CK

T1068

Detection Query

`cisco_isovalent` process_kprobe.action!=""
| bin _time span=5m | rename process_kprobe.parent.pod.name as pod_name
| stats count as kprobe_count
        values(process_kprobe.function_name) as functions
        values(process_kprobe.process.binary) as binaries
        values(process_kprobe.args{}.string_arg) as args
  by pod_name _time
| where kprobe_count > 10 | `cisco_isovalent___kprobe_spike_filter`

Author

Bhavin Patel, Splunk

Created

2026-02-25

Data Sources

Cisco Isovalent Process Kprobe

References

https://docs.isovalent.com/user-guide/sec-ops-visibility/process-execution/index.html

Tags

Cisco Isovalent Suspicious ActivityVoidLink Cloud-Native Linux Malware

Raw Content

name: Cisco Isovalent - Kprobe Spike
id: 3df0e9a8-7d5e-4b2f-bcd7-bf93e671d1f2
version: 3
date: '2026-02-25'
author: Bhavin Patel, Splunk
type: Hunting
data_source:
    - Cisco Isovalent Process Kprobe
status: production
description: |
    This analytic detects excessive kernel probe (kprobe) events in a Kubernetes cluster over a short period of time.
    Kprobes are a Linux kernel debugging and instrumentation mechanism that allows dynamic monitoring and tracing of kernel functions and system calls.
    In containerized or cloud-native environments, kprobes are occasionally used for legitimate low-level diagnostics; however, monitoring a spike in kprobe activity is important because malware or attackers may abuse this mechanism to gain insights into the kernel, attempt privilege escalation, or tamper with host processes.
    More than 10 kprobe events within 5 minutes may indicate suspicious activity, such as an attacker probing the kernel through repeated system calls (e.g., nsenter, mount, sethostname).
    Such abnormal volume and frequency of kprobe usage within application pods or on nodes can signal container escape attempts or low-level tampering with the host, thereby representing a potential security threat.
search: |
    `cisco_isovalent` process_kprobe.action!=""
    | bin _time span=5m | rename process_kprobe.parent.pod.name as pod_name
    | stats count as kprobe_count
            values(process_kprobe.function_name) as functions
            values(process_kprobe.process.binary) as binaries
            values(process_kprobe.args{}.string_arg) as args
      by pod_name _time
    | where kprobe_count > 10 | `cisco_isovalent___kprobe_spike_filter`
how_to_implement: |
    Requires Cisco Isovalent Runtime Security with kprobe tracing enabled and logs
    forwarded into Splunk. Ensure that your Splunk Technology Add-on (TA) for Cisco
    Security Cloud parses the kprobe JSON correctly. Tune the threshold based on
    your workload baseline.
known_false_positives: |
    Busy or noisy pods may legitimately produce bursts of kprobe events during normal
    operation. Tune thresholds and filter by function_name to reduce false positives.
references:
    - https://docs.isovalent.com/user-guide/sec-ops-visibility/process-execution/index.html
tags:
    analytic_story:
        - Cisco Isovalent Suspicious Activity
        - VoidLink Cloud-Native Linux Malware
    asset_type: Endpoint
    mitre_attack_id:
        - T1068
    product:
        - Splunk Enterprise
        - Splunk Enterprise Security
        - Splunk Cloud
    security_domain: endpoint
tests:
    - name: True Positive Test
      attack_data:
        - data: https://media.githubusercontent.com/media/splunk/attack_data/master/datasets/cisco_isovalent/kprobe_spike.log
          source: not_applicable
          sourcetype: cisco:isovalent