Can you explain this vulnerability to me?

This vulnerability affects Apache Spark versions before 3.5.7 and 4.0.1. It is a code execution flaw in the Spark History Web UI caused by overly permissive Jackson deserialization of event log data.

Specifically, the Spark History Server uses Jackson polymorphic deserialization with @JsonTypeInfo.Id.CLASS on SparkListenerEvent objects, which allows an attacker to specify arbitrary class names in the event JSON. This enables instantiation of unintended classes during deserialization.

An attacker with access to the Spark event logs directory can inject malicious JSON payloads into event log files. When the History Server loads these logs, it deserializes the payloads, triggering execution of arbitrary code on the host running the server.

For example, an attacker can inject JSON that causes the server to open a JDBC connection to a remote attacker-controlled server, demonstrating remote command injection capability.

Useful 0 Not Useful 0

How can this vulnerability impact me? :

If an attacker has write access to the Spark event logs directory, they can exploit this vulnerability to execute arbitrary code on the server running the Spark History Server.

This can lead to full system compromise, as the attacker can run commands or perform malicious actions on the host.

Useful 0 Not Useful 0

How does this vulnerability affect compliance with common standards and regulations (like GDPR, HIPAA)?:

I don't know

Useful 0 Not Useful 0

How can this vulnerability be detected on my network or system? Can you suggest some commands?

This vulnerability can be detected by checking for the presence of malicious or unexpected JSON payloads in the Spark event log files, especially those that specify arbitrary class names such as org.apache.hive.jdbc.HiveConnection.

Since the vulnerability involves deserialization of event log data, you can inspect the event log directory for suspicious JSON entries that might trigger the exploit.

Commands to detect this might include searching for suspicious JSON patterns in the Spark event logs directory. For example, using grep to find occurrences of the suspicious class name:

• grep -r 'org.apache.hive.jdbc.HiveConnection' /path/to/spark-logs

Additionally, monitoring network connections initiated by the Spark History Server to unexpected remote hosts or ports could indicate exploitation attempts.

Useful 0 Not Useful 0

What immediate steps should I take to mitigate this vulnerability?

The immediate mitigation step is to upgrade Apache Spark to version 3.5.7 or 4.0.1 and above, as these versions contain the fix for this vulnerability.

Additionally, restrict write access to the Spark event logs directory to trusted users only, preventing attackers from injecting malicious JSON payloads.

Consider monitoring and auditing the Spark History Server logs and network activity for any suspicious behavior until the upgrade is applied.

Useful 0 Not Useful 0