Question 2: Runtime Security with Falco

Problem Statement

Solve this question on: ssh cks7262

Falco is installed on worker node cks7262-node1. Connect using ssh cks7262-node1 from cks7262. There is file /etc/falco/rules.d/falco_custom.yaml with rules that help you to:

Find a Pod running image httpd which modifies /etc/passwd.
Scale the Deployment that controls that Pod down to 0.
Find a Pod running image nginx which triggers rule "Package management process launched".
Change the rule log text after "Package management process launched" to only include:
- time-with-nanoseconds
- container-id
- container-name
- user-name
Collect the logs for at least 20 seconds and save them under /opt/course/2/falco.log on cks7262.
Scale the Deployment that controls that Pod down to 0.

Use sudo -i to become root which may be required for this question

Solution

Step 1: Investigate Falco Configuration

➜ ssh cks7262
➜ candidate@cks7262:~$ ssh cks7262-node1
➜ candidate@cks7262-node1:~$ sudo -i
➜ root@cks7262-node1:~# cd /etc/falco
➜ root@cks7262-node1:/etc/falco# ls -lh
total 132K
drwxr-xr-x 2 root root 4.0K Aug 19 13:18 config.d
-rw-r--r-- 1 root root  53K Sep  7 10:04 falco.yaml
-rw-r--r-- 1 root root   21 Aug 19 12:57 falco_rules.local.yaml
-rw-r--r-- 1 root root  63K Jan  1  1970 falco_rules.yaml
drwxr-xr-x 2 root root 4.0K Aug 19 13:18 rules.d

Step 2: Find httpd Pod Modifying /etc/passwd

➜ root@cks7262-node1:~# falco -U | grep httpd
Sat Sep  7 12:39:04 2024: Falco version: 0.38.2 (x86_64)
...
12:58:32.430165207: Warning Sensitive file opened for reading by non-trusted program (file=/etc/passwd gparent=containerd-shim ggparent=systemd gggparent= evt_type=open user=root user_uid=0 user_loginuid=-1 process=sed proc_exepath=/bin/busybox parent=sh command=sed -i $d /etc/passwd terminal=0 container_id=f86cd629e71c container_name=httpd)

Find the Pod and scale down:

➜ root@cks7262-node1:~# crictl ps -id f86cd629e71c
CONTAINER ID    IMAGE          NAME  ...   POD ID         POD
f86cd629e71c4   f6b40f9f8ad71  httpd ...   cab6dafd045d5  rating-service-5c8f54bd77-bgkh6

➜ root@cks7262-node1:~# crictl pods -id cab6dafd045d5
POD ID          CREATED       ...   NAME                              NAMESPACE    ... 
cab6dafd045d5   3 hours ago   ...   rating-service-5c8f54bd77-bgkh6   team-purple  ...

➜ root@cks7262-node1:~# k get pod -A | grep rating-service
team-purple     rating-service-5c8f54bd77-bgkh6             1/1     Running     0   ...

➜ root@cks7262-node1:~# k -n team-purple scale deploy rating-service --replicas 0
deployment.apps/rating-service scaled

Step 3: Find nginx Pod and Update Falco Rule

➜ root@cks7262-node1:~# falco -U | grep 'Package management process launched'
...
13:10:46.307338039: Error Package management process launched (user=root user_loginuid=-1 command=apk container_id=65338e61dc48 container_name=nginx image=docker.io/library/nginx:1.19.2-alpine)

Update the Falco rule in /etc/falco/rules.d/falco_custom.yaml:

# cks7262-node1:/etc/falco/rules.d/falco_custom.yaml
- rule: Launch Package Management Process in Container
  desc: Package management process ran inside container
  condition: >
    spawned_process
    and container
    and user.name != "_apt"
    and package_mgmt_procs
    and not package_mgmt_ancestor_procs
  output: >
    Package management process launched %evt.time,%container.id,%container.name,%user.name
  priority: ERROR
  tags: [process, mitre_persistence]

Step 4: Collect Logs

➜ root@cks7262-node1:~# falco -U | grep 'Package management process launched'
...
13:31:26.364958758: Error Package management process launched 13:31:26.364958758,65338e61dc48,nginx,root
13:31:31.356117694: Error Package management process launched 13:31:31.356117694,65338e61dc48,nginx,root
13:31:36.329307852: Error Package management process launched 13:31:36.329307852,65338e61dc48,nginx,root
...

Save logs to file:

# cks7262:/opt/course/2/falco.log
13:31:26.364958758: Error Package management process launched 13:31:26.364958758,65338e61dc48,nginx,root
13:31:31.356117694: Error Package management process launched 13:31:31.356117694,65338e61dc48,nginx,root
13:31:36.329307852: Error Package management process launched 13:31:36.329307852,65338e61dc48,nginx,root
13:31:41.338988597: Error Package management process launched 13:31:41.338988597,65338e61dc48,nginx,root
13:31:46.329154755: Error Package management process launched 13:31:46.329154755,65338e61dc48,nginx,root
13:31:51.308124986: Error Package management process launched 13:31:51.308124986,65338e61dc48,nginx,root
13:31:56.358522188: Error Package management process launched 13:31:56.358522188,65338e61dc48,nginx,root
13:32:01.360834976: Error Package management process launched 13:32:01.360834976,65338e61dc48,nginx,root
13:32:06.327657274: Error Package management process launched 13:32:06.327657274,65338e61dc48,nginx,root
13:32:11.342534392: Error Package management process launched 13:32:11.342534392,65338e61dc48,nginx,root
13:32:16.343746448: Error Package management process launched 13:32:16.343746448,65338e61dc48,nginx,root
13:32:21.303524240: Error Package management process launched 13:32:21.303524240,65338e61dc48,nginx,root
13:32:26.330027622: Error Package management process launched 13:32:26.330027622,65338e61dc48,nginx,root
13:32:31.364716844: Error Package management process launched 13:32:31.364716844,65338e61dc48,nginx,root

Step 5: Scale Down nginx Deployment

➜ candidate@cks7262:~# k get pod -A | grep webapi
team-blue       webapi-5499fdc5db-k4c7c                     1/1     Running      ...

➜ candidate@cks7262:~$ k -n team-blue scale deploy webapi --replicas 0
deployment.apps/webapi scaled

Back to Questions List