Cisco Isovalent - Late Process Execution

name: Cisco Isovalent - Late Process Execution
id: 7f4b9b8e-5d6a-4a21-9e3f-0f1e8f2d1c3a
version: 3
date: '2026-03-10'
author: Bhavin Patel, Splunk
type: Anomaly
data_source:
    - Cisco Isovalent Process Exec
status: production
description: |
    Detects process executions that occur well after a container has initialized, which can indicate
    suspicious activity (e.g., interactive shells, injected binaries, or post-compromise tooling).
    The analytic compares the process start time to the container start time and flags processes
    launched more than 5 minutes (300 seconds) after initialization.
search: |
    `cisco_isovalent_process_exec` process_name="sh"
    | rename process_exec.process.start_time as ProcessStartTime
    | rename process_exec.process.pod.container.start_time as ContainerStartTime
    | eval ProcessStartTime=strptime(ProcessStartTime, "%Y-%m-%dT%H:%M:%S.%3Q")
    | eval ContainerStartTime=strptime(ContainerStartTime, "%Y-%m-%dT%H:%M:%S.%9Q")
    | eval ContainerTime5min=relative_time(ContainerStartTime, "+5m")
    | where ProcessStartTime > ContainerTime5min
    | table node_name cluster_name, pod_name, container_id, process_name, process_exec, process, ProcessStartTime, ContainerTime5min | `security_content_ctime(ProcessStartTime)`
    | `security_content_ctime(ContainerTime5min)`
    | `cisco_isovalent___late_process_execution_filter`
how_to_implement: The detection is based on process execution data generated by Cisco Isovalent Runtime Security. Ensure that Isovalent Runtime Security is deployed and configured in your Kubernetes environment to emit process_exec events. Configure the Cisco Security Cloud TA to collect these logs via HTTP Event Collector (HEC) and normalize them into the Splunk Common Information Model. This integration ensures that all relevant pod, container, and process activity is captured for monitoring and detection of suspicious behavior.
known_false_positives: This activity may be triggered by legitimate administrative scripts, container images, or third-party operators that use cron for scheduled tasks, so please investigate the alert in context to rule out benign operations.
references:
    - https://docs.isovalent.com/user-guide/sec-ops-visibility/process-execution/index.html
drilldown_searches:
    - name: View the detection results for pod - "$pod_name$"
      search: '%original_detection_search% | search pod_name = "$pod_name$"'
      earliest_offset: $info_min_time$
      latest_offset: $info_max_time$
    - name: View risk events for the last 7 days for - "$pod_name$"
      search: '| from datamodel Risk.All_Risk | search normalized_risk_object IN ("$pod_name$") starthoursago=168  | stats count min(_time) as firstTime max(_time) as lastTime values(search_name) as "Search Name" values(risk_message) as "Risk Message" values(analyticstories) as "Analytic Stories" values(annotations._all) as "Annotations" values(annotations.mitre_attack.mitre_tactic) as "ATT&CK Tactics" by normalized_risk_object | `security_content_ctime(firstTime)` | `security_content_ctime(lastTime)`'
      earliest_offset: $info_min_time$
      latest_offset: $info_max_time$
rba:
    message: Late process execution [$process_name$] detected in pod [$pod_name$]
    risk_objects:
        - field: pod_name
          type: system
          score: 20
    threat_objects:
        - field: process_name
          type: process_name
tags:
    analytic_story:
        - Cisco Isovalent Suspicious Activity
    asset_type: Endpoint
    mitre_attack_id:
        - T1543
    product:
        - Splunk Enterprise
        - Splunk Enterprise Security
        - Splunk Cloud
    security_domain: endpoint
tests:
    - name: True Positive Test
      attack_data:
        - data: https://media.githubusercontent.com/media/splunk/attack_data/master/datasets/cisco_isovalent/cisco_isovalent_process_exec_delayed_shell.log
          source: not_applicable
          sourcetype: cisco:isovalent:processExec