Lab 8: System and process monitoring¶

Objectives¶

After completing this lab, you will be able to:

view and manage processes
kill errant processes
change process priority

Estimated time to complete this lab: 60 minutes

Introduction¶

These exercises cover various topics related to monitoring and managing processes on a Linux systems. Topics covered include process identification and control, process priority management, signal handling, resource monitoring, and "cgroups" management.

Exercise 1¶

`ps` and /proc exploration¶

To explore and identify the first system process¶

Log in to the system as any user.
Find the name of the process with a process ID of 1 using /proc.
```
cat /proc/1/comm
```
Question

What is the name of the process with PID 1?
View the name and path to the executable behind the process with PID 1.
```
ls -l /proc/1/exe
```
Question

What is the path to the executable behind PID 1?
Use the ps command to find out the name of the process or program behind PID 1.
```
ps -p 1 -o comm=
```
Question

Does the ps command confirm the name of the process?
Use the ps command to view the full path and any command-line arguments of the process or program behind PID 1.
```
ps -p 1 -o args=
```
Question

What is the full path and command-line arguments for the process with PID 1?

Question

Why is the process with PID 1 important on a Linux system?

To display detailed process information using `ps`¶

The following steps show how to use ps to display basic process information.

Use the ps command to display a list of all processes in a tree structure.
```
ps auxf
```
Question

What is the structure of the process list, and what information is displayed?
Filter the list only to show processes associated with a specific user, e.g., the user "root."
```
ps -U root
```
Confirm that only the processes for the "root" user are displayed.
Show processes in a detailed format, including the process tree and threads. Type:
```
ps -eH
```
Question

What additional details are shown in this format?
Display the processes sorted by CPU usage in descending order.
```
ps aux --sort=-%cpu
```
Question

What process is consuming the most CPU?

Exercise 2¶

Managing processes with `kill`¶

To terminate a process using `kill`¶

Start a long running sleep process in the background and display the PID on your terminal. Type:
```
(sleep 3600 & MYPROC1=$! && echo PID is: $MYPROC1) 2>/dev/null
```
OUTPUT:
```
PID is: 1331933
```
Make a note of the PID for the new process on your system. The PID is also saved in the $MYPROC1 variable.
Send a termination signal (SIGTERM) to the sleep process.
```
kill $MYPROC1
```
Replace $MYPROC1 with the actual PID from step 1.
Check if the process has been terminated using ps and ps aux.
```
ps aux | grep -v grep | grep sleep
```

To terminate processes using `kill` signals¶

Start a new sleep process and make a note of its PID. Type:

(sleep 3600 & MYPROC2=$! && echo PID is: $MYPROC2) 2>/dev/null

OUTPUT:

PID is: 1333258

Send a different signal (e.g., SIGHUP) to the new sleep process. Type:
```
kill -1 $MYPROC2
```
Confirm that $MYPROC2 is no longer in the process table.

Start a new ping process and make a note of its PID. Type:

{ ping localhost > /dev/null 2>&1 & MYPROC3=$!; } \
    2>/dev/null; echo "PID is: $MYPROC3"

Use the kill command to send a SIGTERM signal to the ping process. Type:
```
kill -15 $MYPROC3
```
Replace MYPROC3 with the actual PID of the process on your system.
Start a long-running process using the cat command. Type:
```
{ cat /dev/random > /dev/null 2>&1 & MYPROC4=$!; } \
 2>/dev/null; echo PID is: $MYPROC4
```
Make a note of the PID for the process on your system.
Use kill to forcefully terminate the process by sending a SIGKILL signal.
```
kill -9 $MYPROC4
```
Confirm that the process is terminated.

Question

Explain the purpose of sending signals to processes using the kill command and the significance of different signal types.

Exercise 3¶

Monitoring System Resources with `top`¶

To monitor system resource usage with `top`¶

Launch the top command to view real-time system statistics.
```
top
```
Question

What information is displayed in the top interface?
Observe the CPU and memory usage of processes in the top interface.

Question

What processes are consuming the most CPU and memory?
Sort the processes in top by CPU usage (press P) and by memory usage (press M).

Question

What are the top processes consuming CPU and memory after sorting?

To monitor CPU and memory usage of specific processes using `top`¶

Create an arbitrarily large 512MB file that contains random data.
```
sudo fallocate -l 512M  ~/large-file.data
```
Start a resource-intensive process, such as a large file compression.
```
 tar -czf archive.tar.gz /path/to/large/directory
```
Open the top command to monitor the CPU and memory usage.
```
 top
```
Find and select the resource-intensive process in the top interface.

Question

What is the process ID and resource utilization of the intensive process?
Change the sorting order in top to display processes using the most CPU or memory (press P or M).

Question

What process is at the top of the list after sorting?
Exit top by pressing q.

To monitor processes and resource usage using `top`¶

Launch the top command in interactive mode.
```
top
```
Question

What information is displayed on the top screen?
Use the 1 key to display a summary of individual CPU core usage.

Question

What is the CPU core usage breakdown for each core?
Press u to display processes for a specific user. Enter your username.

Question

Which processes are currently running for your user?
Sort the processes by memory usage (press M) and observe the processes consuming the most memory.

Question

What processes are using the most memory?
Exit top by pressing q.

Question

Explain the significance of monitoring system resources using the top command and how it can help troubleshoot performance issues.

Exercise 4¶

Changing Process Priority with `nice` and `renice`¶

To adjust process priority using `nice`¶

Start a CPU-intensive process that runs with default/normal priority. Type:
```
bash -c  'while true; do echo "Default priority: The PID is $$"; done'
```
OUTPUT:
```
Default priority: The PID is 2185209
Default priority: The PID is 2185209
Default priority: The PID is 2185209
....<SNIP>...
```
From the output, the value of the PID on our sample system is 2185209.

The value of the PID will be different on your system.

Note of the value of the PID being continuously displayed on the screen on your system.
In a different terminal, using your PID value, check the process' default priority using ps. Type:
```
ps -p <PID> -o ni
```
Question

What is the running process' default process priority (nice value)?
Using the PID of the process printed, end the process using the kill command.
Using the nice command, relaunch a similar process with a lower niceness value (i.e. more favorable to the process OR higher priority). Use a nice value of -20. Type:
```
nice -n -20 bash -c  'while true; do echo "High priority: The PID is $$"; done'
```
Using your value of the PID, check the process' priority using ps. Type:
```
ps -p <PID> -o ni
```
Question

Has the process priority been successfully set?
Simultaneously press the Ctrl+C keys on your keyboard to kill the new high-priority process.
Using the nice command again relaunch another process but this time with a higher niceness value (i.e. least favorable to the process OR lower priority). Use a nice value of 19 Type:
```
 nice -n 19 bash -c  'while true; do echo "Low priority: The PID is $$"; done'
```
OUTPUT:
```
Low priority: The PID is 2180254
Low priority: The PID is 2180254
...<SNIP>...
```
Check the process's custom priority using ps. Type:
```
ps -p <PID> -o ni
```
Simultaneously press the Ctrl+C keys on your keyboard to kill the new low-priority process.
Experiment with altering the priority of different processes to higher and lower values and observe the impact on the process's resource usage.

To adjust the priority of a running process using `renice`¶

Start a CPU-intensive process, such as a lengthy mathematical calculation using the md5sum utility. Type:
```
find / -path '/proc/*' -prune -o -type f -exec md5sum {} \; > /dev/null
```
Use the ps command to figure out the PID of the previous find/md5sum process. Type:
```
ps -C find -o pid=
```
OUTPUT:
```
2577072
```
From the output, the value of the PID on our sample system is 2577072.

The value of the PID will be different on your system.

Make a note of the value of the PID on your system.
Use the renice command to adjust the priority of the running find/md5sum process to a lower niceness value (e.g., -10, higher priority). Type:
```
renice  -n -10 -p $(ps -C find -o pid=)
```
OUTPUT:
```
<PID> (process ID) old priority 0, new priority -10
```
Replace <PID> (above) with the actual PID of the running process.
Monitor the resource utilization for the find/md5sum process, using top (or htop). Type:
```
top -cp $(ps -C find -o pid=)
```
Question

Does the process now receive a higher share of CPU resources?
Change the priority of the find/md5sum process to a higher nice value (e.g., 10, lower priority). Type:
```
renice  -n 10 -p <PID>
```
OUTPUT:
```
2338530 (process ID) old priority -10, new priority 10
```
Replace the <PID> (above) with the actual PID of the running process.

Question

Explain how the nice command is used to adjust process priorities and how it affects system resource allocation.
Press the Ctrl+C keys simultaneously on your keyboard to stop the find/md5sum process. You can also use the kill command to accomplish the same thing.

Exercise 5¶

Identifying processes with `pgrep`¶

To find processes by name using `pgrep`¶

Use the pgrep command to identify all processes associated with a specific program or service, such as sshd.
```
pgrep sshd
```
Question

What are the process IDs of the sshd processes?
Verify the existence of the identified processes using the ps command.
```
 ps -p <PID1,PID2,...>
```
Replace "" with the process IDs obtained from step 1.
Use the pgrep command to identify processes with a specific name, e.g., "cron."
```
pgrep cron
```
Question

Are there any processes with the name "cron"?

Question

Explain the difference between using ps and pgrep to identify and manage processes.

Exercise 6¶

Foreground and background processes¶

This exercise covers managing processes with fg and bg

To manage background and foreground processes using `bg` and `fg`¶

Start a long-running process in the foreground. For example, you can use a simple command like sleep. Type:
```
sleep 300
```
Suspend the foreground process by pressing Ctrl+Z on your keyboard. This should return you to the shell prompt.
List the suspended job using the jobs command. Type:
```
jobs
```
Question

What is the status of the suspended job?
Bring the suspended job back to the foreground using the fg command.
```
fg
```
Question

What happens when you bring the job back to the foreground?
Suspend the job again using Ctrl+Z, and then move it to the background using the bg command.
```
bg
```
Question

What is the status of the job now?

Question

Explain the foreground and background process' purpose, and how they are managed using fg and bg commands.

To start a process in the background¶

The & symbol can launch a process that immediately runs in the background. For example, to start the sleep command in the background type:
```
sleep 300 &
```
Suspend the running process using Ctrl+Z.
List the status of all active jobs. Type:
```
jobs -l
```
Question

What is the status of the sleep 300 process?
Return the background process to the foreground using the fg command.
```
fg
```
Prematurely end the sleep process by sending it the SIGSTOP signal by pressing Ctrl+C.

To manage interactive processes using `bg` and `fg`¶

Start an interactive process like the vi text editor to create and edit a sample file text file named "foobar.txt". Type:
```
vi foobar1.txt
```
Suspend the running process using Ctrl + Z.

Use the bg command to move the suspended process to the background.
```
bg
```
Question

Is the process now running in the background?
Enter "Hello" inside foobar1.txt in your vi editor.
Suspend the running vi text editing session by pressing Ctrl+Z.
Launch another separate vi editor session to create another text file named "foobar2.txt". Type:
```
vi foobar2.txt
```
Enter the sample text "Hi inside foobar2.txt" in the 2nd vi session.
Suspend the 2nd vi session using Ctrl+Z.
List the status of all jobs on the current terminal. Type:
```
jobs -l
```
OUTPUT:
```
[1]- 2977364 Stopped       vi foobar1.txt
[2]+ 2977612 Stopped       vi foobar2.txt
```
You should have at least 2 jobs listed in your output. The number in the 1st column of the output shows the job numbers - [1] and [2].
Resume and bring to the foreground the 1st vi session by typing:
```
fg %1
```
Suspend the 1st vi session again using Ctrl+Z.
Resume and bring to the foreground the 2nd vi session by typing:
```
fg %2
```

Ungracefully terminate both vi editing sessions by sending the KILL signal to both jobs. Follow the kill command with the jobs command. Type:

 kill -SIGKILL  %1 %2 && jobs

OUTPUT:

[1]-  Killed                  vi foobar1.txt
[2]+  Killed                  vi foobar2.txt

Exercise 7¶

Process identification with `pidof`¶

To find the process ID of a running command using `pidof`¶

Let us pick a sample/common running process whose process ID we want to find. We will use systemd as our example.
Use the pidof command to find the process ID of the systemd. Type:
```
pidof systemd
```
Note the process ID(s) of systemd.
Verify the existence of the identified process using the ps command.
```
ps -p <PID>
```
Replace <PID> with the actual process ID obtained from step 2.

Question

Explain the difference between pgrep and pidof for finding the process ID of a running command.

Exercise 8¶

Exploring /sys filesystem¶

To explore the /sys filesystem¶

List the contents of the /sys directory. Type:
```
ls /sys
```
Question

What kind of information is stored in the /sys directory?
Navigate to a specific /sys entry, for example, the CPU information.
```
cd /sys/devices/system/cpu
```
List the contents of the current directory to explore CPU-related information.
```
ls
```
Question

What kind of CPU-related information is available in the /sys filesystem?

Question

Explain the purpose of the /sys filesystem in Linux and its role in managing system hardware and configuration.

Exercise 9¶

Killing processes by name with `pkill`¶

To terminate processes by name using `pkill`¶

Identify processes with a specific name, such as "firefox."
```
pkill firefox
```
Question

Have all processes with the name "firefox" been terminated?
Check the status of the processes you killed using ps.
```
 ps aux | grep firefox
```
Question

Are there any remaining processes with the name "firefox"?

Use pkill to forcefully terminate all processes with a specific name.
```
pkill -9 firefox
```
Confirm that all processes with the name "firefox" are now terminated.

Question

What is the difference between using kill and pkill to terminate processes by name?

Exercise 10¶

This exercise covers using the powerful exec command.

Process control with `exec`¶

To replace the current shell with another command using `exec`¶

Start a new shell session. Type:
```
bash
```
Run a command that does not exit in the new shell, such as a simple while loop.
```
 while true; do echo "Running..."; done
```
In the current shell, replace the running command with a different one using exec.
```
 exec echo "This replaces the previous command."
```
Note that the previous command is terminated, and the new command is running.
Confirm that the old command is no longer running using ps.
```
ps aux | grep "while true"
```
Question

Is the previous command still running?

Question

Explain how the exec command can replace the current shell process with a different command.

Exercise 11¶

Process management with `killall`¶

Like kill, killall is a command to terminate processes by name instead of PID. Some similarities can be observed between the usage of killall , kill, and pkill in process termination.

To terminate processes by name using `killall`¶

Identify processes with a specific name, such as "chrome."
```
 killall chrome
```
Question

Have all processes with the name "chrome" been terminated?
Check the status of the processes you killed using ps.
```
 ps aux | grep chrome
```
Question

Are there any remaining processes with the name "chrome"?
Use killall to forcefully terminate all processes with a specific name.
```
 killall -9 chrome
```
Confirm that all processes with the name "chrome" are now terminated.

Question

How does killall differ from pkill and kill when terminating processes by name?

Exercise 12¶

`cgroups` management¶

To manage processes using `cgroups`¶

List the existing cgroups on your system.
```
cat /proc/cgroups
```
Question

What are the cgroup controllers available on your system?
Create a new cgroup using the CPU controller. Name it "mygroup."
```
sudo mkdir -p /sys/fs/cgroup/cpu/mygroup
```
Move a specific process (e.g., a running sleep command) into the "mygroup" cgroup.
```
echo <PID> | sudo tee /sys/fs/cgroup/cpu/mygroup/cgroup.procs
```
Replace <PID> with the actual PID of the process.
Check if the process has been moved to the "mygroup" cgroup.
```
cat /sys/fs/cgroup/cpu/mygroup/cgroup.procs
```
Question

Is the process listed in the "mygroup" cgroup?

Question

Explain the concept of cgroups in Linux and how they can manage and control resource allocation for processes.

Exercise 13¶

Managing processes with `renice`¶

To adjust the priority of a running processes using `renice`¶

Identify a running process with a specific PID and priority using ps.
```
ps -p <PID> -o ni
```
Question

What is the current priority (nice value) of the process?
Use the renice command to change the priority (nice value) of the running process.
```
renice <PRIORITY> -p <PID>
```
Replace <PRIORITY> with the new priority value you want to set, and <PID> with the actual PID of the process.
Verify that the process' priority has changed using ps.
```
ps -p <PID> -o ni
```
Question

Is the priority now different?
Experiment with changing the priority to a higher and lower value and observe the impact on the process's resource usage.

Question

What happens to the process's resource consumption with different nice values?

Question

Explain how the renice command is used to adjust the priority of running processes and its effects on process resource utilization.

Author: Wale Soyinka

Contributors: Steven Spencer, Ganna Zhyrnova

Lab 8: System and process monitoring¶

Objectives¶

Introduction¶

Exercise 1¶

ps and /proc exploration¶

To explore and identify the first system process¶

To display detailed process information using ps¶

Exercise 2¶

Managing processes with kill¶

To terminate a process using kill¶

To terminate processes using kill signals¶

Exercise 3¶

Monitoring System Resources with top¶

To monitor system resource usage with top¶

To monitor CPU and memory usage of specific processes using top¶

To monitor processes and resource usage using top¶

Exercise 4¶

Changing Process Priority with nice and renice¶

To adjust process priority using nice¶

To adjust the priority of a running process using renice¶