Question 26: Malicious Process Detection

Problem Statement

Solve this question on: ssh cks8930

A security scan result shows that there is an unknown miner process running on one of the Nodes in this cluster. The report states that the process is listening on port 6666.

Tasks:

  1. Kill the process
  2. Delete the binary

Solution

Step 1: Identify Cluster Nodes

First, let's list all nodes in the cluster:

➜ ssh cks8930

➜ candidate@cks8930:~# k get node
NAME            STATUS   ROLES           AGE   VERSION
cks8930         Ready    control-plane   26m   v1.29.5
cks8930-node1   Ready              26m   v1.29.5
Step 2: Check Control Plane Node

Switch to root and check for processes listening on port 6666:

➜ candidate@cks8930:~# sudo -i

➜ root@cks8930:~# netstat -plnt | grep 6666

➜ root@cks8930:~#

No suspicious process found on the control plane node.

Step 3: Check Worker Node

Connect to the worker node and check for suspicious processes:

➜ root@cks8930:~# ssh cks8930-node1

➜ root@cks8930-node1:~# netstat -plnt | grep 6666
tcp6       0      0 :::6666                 :::*                    LISTEN      8198/system-atm    

➜ root@cks8930-node1:~# lsof -i :6666
COMMAND    PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
system-at 8198 root    3u  IPv6  39776      0t0  TCP *:6666 (LISTEN)

Found the suspicious process! Let's identify its binary location:

➜ root@cks8930-node1:~# ls -lh /proc/8198/exe
lrwxrwxrwx 1 root root 0 Sep  8 14:44 /proc/8198/exe -> /usr/bin/system-atm
Step 4: Remove Malicious Process

Kill the process and remove its binary:

➜ root@cks8930-node1:~# kill -9 8198

➜ root@cks8930-node1:~# rm /usr/bin/system-atm
Summary of actions:
  • Identified cluster nodes and checked both control plane and worker nodes
  • Found suspicious process 'system-atm' listening on port 6666 on worker node
  • Located the malicious binary at /usr/bin/system-atm
  • Terminated the process and removed the binary
Security Note: Regularly monitor your cluster nodes for suspicious processes and network activity. Consider implementing security tools like Falco or auditd for continuous monitoring and alerting of suspicious activities.
Back to Questions List