Linux 9 - Basics of Linux

By Sheldon L Published at 2019-11-09 Updated at 2020-04-20

What is Linux

Linux History

The Boot Process, init and Services

# start|stop|restart|enable|disable
sudo systemctl start|stop|restart nfs.service
sudo systemctl enable|disable nfs.service

# Confirm
sudo ss -antlp | grep [service]

# List all available services
systemctl list-unit-files
lsof -i -P -n

Partitions and Filesystems

tree -aCd -L 1 /
tree -aCd -L 1 /usr
tree -aCd -L 2 /home
tree -aCd -L 1 /var

Mounting and Unmounting

mount | head -10            # mount: show all presently mounted filesystems
df -Th                      # df: disk free, T: filesystem Type
df -H
cat /proc/mounts

sudo mount /dev/sda5 /home  # /dev/sda5: device node, /home: mount point
# There are other ways to specify the partition other than the device node, such as using the disk label or UUID.

sudo umount /home

# To be automatically available every time the system starts up,
# edit /etc/fstab (filesystem table)
man fstab

NFS on the Server

sudo systemctl start nfs

cat /etc/exports

exports -av   # notify Linux about the directories you are allowing to be remotely mounted
sudo systemctl restart nfs
sudo systemctl enable nfs
# Note: On RHEL/CentOS 8, the service is called nfs-server, not nfs

NFS on the Client

cat /etc/fstab
sudo mount [servername|IP]:/projects /mnt/nfs/projects
cat /etc/fstab

man fstab

Debian Filesystem Hierarchy Standard (FHS)

The Filesystem in Debian

Comparing Files

<diff|cmp> -<c|r|i|w|q> [filename1] [filename2]
# c: context before and after the lines differing in content
# r: recursively compare subdirectories
# i: ignore the case of letters
# w: ignore differences in spaces and tabs (white space)
# q: quiet: only report if files are different without listing the differences

diff3 [MY-FILE] [COMMON-FILE] [YOUR-FILE]  # compare 3 files

# A patch file contains the deltas (changes) required to update an older version of a file to the new one.
# The patch files are actually produced by running diff with the correct options
diff -Nur [originalfile] [newfile] > [patchfile]

# To apply a patch, you can just do either of the two methods below:
patch -p1 < patchfile
patch [originalfile] [patchfile]

cd /tmp
cp /etc/group /tmp/group
dd if=/tmp/group of=/tmp/GROUP conv=ucase
# copy group to GROUP and convert to upper-cases, ucase: upper-case

diff -Nur group GROUP > group_GROUP
cat group_GROUP
patch --dry-run group group_GROUP
patch group group_GROUP
diff group GROUP

# comm
comm scan_a.txt scan_b.txt      # unique in a, unique in b, common in ab
comm -12 scan_a.txt scan_b.txt  # only common in ab

# vimdiff
vimdiff scan_a.txt scan_b.txt  # open vim with multiple files

Using the file Utility

file *

Backing Up Data

rsync [sourcefile] [destinationfile]

# copy files from one machine to another
rsync -r [sourcefile] [archive-machine]:[archives/destinationfile] -dry-run
# Note that rsync can be very destructive!
# using the -dry-run option to ensure that it provides the results that you want

# A good combination of options
rsync --progress -avrxH  --delete [sourcedir] [destdir]

Compressing Data

Command Usage
gzip The most frequently used Linux compression utility
bzip2 Produces files significantly smaller than those produced by gzip
xz The most space-efficient compression utility used in Linux
zip Is often required to examine and decompress archives from other operating systems
tar (tape archive) Is often used to group files in an archive and then compress the whole archive at once. 
gzip *               # Each file is compressed and renamed with a .gz extension
gzip -r [projectX]   # Compresses all files in the projectX directory recursively
gunzip foo           # De-compresses foo found in the file foo.gz
gzip -d foo          # Same as above

bzip2 *              # Each file is compressed and renamed with a .bz2 extension
bunzip2 *.bz2
bzip2 -d *.bz2

xz *                 # Each file is compressed and renamed with a .gz extension
xz foo               # Compresses foo into foo.xz using the default compression level (-6), and removes foo if compression succeeds
xz -dk bar.xz        # Decompresses bar.xz into bar and does not remove bar.xz
xz -dcf a.txt b.txt.xz > abcd.txt  # Decompresses a mix of compressed and uncompressed files to standard output
xz -d *.xz           # Decompresses the files compressed using xz

zip backup *         # Compresses all files in the current directory and places them in the backup.zip
zip -r backup.zip ~  # Archives your login directory (~) and all files and directories under it in backup.zip
unzip backup.zip     # Extracts all files in backup.zip and places them in the current directory

tar zcvf mydir.tar.gz mydir  # Create the archive and compress with gzip
tar jcvf mydir.tar.bz2 mydir # Create the archive and compress with bz2
tar Jcvf mydir.tar.xz mydir  # Create the archive and compress with xz
tar xvf mydir.tar            # Extract all the files in mydir.tar into the mydir directory
tar xvf mydir.tar.gz         # Extract all the files in mydir.tar.gz into the mydir directory
# Note: You do not have to tell tar it is in gzip format

# You can separate out the archiving and compression stages:
tar cvf mydir.tar mydir ; gzip mydir.tar
gunzip mydir.tar.gz ; tar xvf mydir.tar
# BUT this is slower and wastes space by creating an unneeded intermediary .tar file.

# Archiving (Backing Up) Home Directory
tar zcvf /tmp/backup_home.tar.gz ~
ls -lh /tmp/backup*           # h: human-readable

time tar zcvf mydir.tar.gz mydir
time tar jcvf mydir.tar.bz2 mydir
time tar Jcvf mydir.tar.xz mydir
du -shc mydir mydir.tar.gz mydir.tar.bz2 mydir.tar.xz mydir.tar.xz
# gz is the fastest, xz is the most compressed

Disk-to-Disk Copying (dd)

# Back Master Boot Record (MBR) (the first 512-byte sector on the disk
# that contains a table describing the partitions on that disk),
dd if=/dev/sda of=sda.mbr bs=512 count=1

# WARNING!
# DANGER!
dd if=/dev/sda of=/dev/sdb  # make a copy of one disk onto another

Software

Browsers and Network

Email

Office Applications

Development Applications

On other operating systems, these tools have to be obtained and installed separately, often at a high cost, while on Linux they are all available at no cost through standard package installation systems.

Sound Players

Movie Players

Movie Editors

Graphics Utilities

Install Source

Appearance

sudo apt install gnome-tweaks

Package Management Systems

Managing Processes

Process Types Description Examples
Interactive Processes Started by a user bash, firefox, top
Batch Processes Automatic processes scheduled from and then disconnected from the terminal, queued and work on a First-In, First-Out basis updatedb, ldconfig
Daemons Server processes that run continuously, wait for a user or system request httpd, sshd, libvirtd
Threads Lightweight processes, run under the umbrella of a main process, sharing memory and other resources, but are scheduled and run by the system on an individual basis, end without terminating the whole process, a process can create new threads at any time. firefox, gnome-terminal-server
Kernel Threads Kernel tasks that users neither start nor terminate and have little control over. These may perform actions like moving a thread from one CPU to another, or making sure input/output operations to disk are completed. kthreadd, migration, ksoftirqd
ID Type Description
PID Unique Process ID number
PPID Process (Parent) that started this process. If the parent dies, the PPID will refer to an adoptive parent; on recent kernels, this is kthreadd which has PPID=2
TID Thread ID number. This is the same as the PID for single-threaded processes. For a multi-threaded process, each thread shares the same PID, but has a unique TID. 
ps aux | grep [keyword] # ps aux: displays all processes of all users
ps aux | head -10
ps axo stat,priority,pid,pcpu,comm | head -10
ps -u     # display information of processes for a specified user
ps -ef    # displays all the processes in the system in full detail
ps -eLf   # displays one line of information for every thread
ps -fC [command_name]

pstree

kill -[signal] [pid]    # force kill
kill -SIGKILL [pid]
kill -9 [pid]           # same as above
term [pid]              # terminate gracefully

User and Group IDs

Priorities

ps lf    # show priority

# Setting the Priority when Starting Process
nice -n [nice_value_increament] /bin/[someSlowProcess]

# Changing the Priority of a Runing Process
renice [nice_value_increament] [PID]
renice +5 3077          # increase 5
sudo renice -5 3077     # decrease 5, must in sudo

genome-system-monitor   # check priority in monitor

Load Averages

w
# load average (for 1 min, 5 min, 15 min), a single-CPU everage

top
# Output:
# 1st line: time the system has been up, users logged on, load average as `w`
# 2nd line: total processes, the number of running, sleeping, stopped, and zombie
# 3rd line: CPU time divided between users (us) and kernel (sy),
  # and the percentage of user jobs running lower priority (ni),
  # idle mode (id) should be low if the load average is high,
  # the percentage of jobs waiting (wa) for I/O,
  # hardware (hi) vs. software interrupts (si),
  # steal time (st) is generally used with virtual machines, which has some of its idle CPU time taken for other uses
# Prameters of the table
  # Process Identification Number (PID)
  # Process owner (USER)
  # Priority (PR) and nice values (NI)
  # Virtual (VIRT), physical (RES), and shared memory (SHR)
  # Status (S)
  # Percentage of CPU (%CPU) and memory (%MEM) used
  # Execution time (TIME+)
  # Command (COMMAND).
# You can enter single-letter commands to change its behavior
  # t: Display or hide summary information (rows 2 and 3)
  # m: Display or hide memory information (rows 4 and 5)
  # A: Sort the process list by top resource consumers
  # r: Renice (change the priority of) a specific processes
  # k: Kill a specific process
  # f: Enter the top configuration screen
  # o: Interactively select a new sort order in the process list
  # 1: Show every single cpu

vmstat -a 2 1000  # all info, every 2 sec, run 1000 times

uptime

Background and Foreground Processes

# Ctrl + c : cancel process
# Ctrl + z : pauses the process and wait in background

jobs -l # jobs running in the background

fg %[job-number]  # restores a job to the foreground
bg %[job-number]  # restarted in the background
kill %[job-number]

[command] &
jobs -l

Scheduling Future Processes

$ at now + 2 days
at> cat file1.txt
at> <EOT>          # press Ctrl+D here


# CROND     #  driven by /etc/crontab (cron table), run every day/week/month...

crontab -e   # 0 10 * * * /tmp/myjob.sh
# OR
touch mycrontab
echo `0 10 * * * /tmp/myjob.sh` > mycrontab
crontab mycrontab

crontab -l  # list
crontab -r  # remove

touch /tmp/myjob.sh
echo '#!/bin/bash' > /tmp/myjob.sh
echo 'echo Hello I am running $0 at $(date)' >> /tmp/myjob.sh
chmod +x /tmp/myjob.sh


sleep [number]<s|m|h|d>      # suspends execution for at least the specified period of time

ln file1 file2      # Suppose that file1 already exists, a hard link, called file2, is created
ln -s file1 file3   # A soft link, file3 clearly points to file1 and has a different inode number
ls -li file?        # prints out in the first column the inode number

echo 'file1 content' > file1
cat file1
cat file2
cat file3

echo 'more content written in file2' >> file2
cat file1
cat file2
cat file3

echo 'more content written in file3' >> file2
cat file1
cat file2
cat file3

rm file1
ls -li file?
cat file2
cat file3

Piping and Redirection

## STDIN
# redirecting to new file
ls
echo "test"
echo "test" > redirection_test.txt
echo "test2" >> redirection_test.txt
echo "test3" 1>> redirection_test.txt  # syntax is the same as above
ls
cat redirection_test.txt
echo "Kali Linux is an open source project" > redirection_test.txt
cat redirection_test.txt

# redirecting to an existing file
echo "that is maintained and funded by Offensive Security" >> redirection_test.tx
cat redirection_test.txt

## STDOUT
# redirecting from a file
wc -m < redirection_test.txt
# redirection_test.txt is STDIN
# output (char count) is STDOUT

## STDERROR
ls ./not_exist 2>> error.txt          # 2: file descriptor
cat error.txt

ls ./not_exist > error.txt 2>&1
# A special shorthand notation can send anything written to file descriptor 2 (stderr)
# to the same place as file descriptor 1 (stdout)
cat error.txt

ls ./not_exist >& error.txt # an easier syntax for the above
cat error.txt

## Piping
cat error.txt | wc -m > count.txt
cat count.txt

Command Line

Text Terminal on the Graphical Desktop

Virtual Terminals (GNOME Terminal)

Turning Off the Graphical Desktop

sudo systemctl stop <gdm|lightdm|kdm>
# OR
sudo telinit 3

sudo systemctl start <gdm|lightdm|kdm>
# OR
sudo telinit 5

sudo

su    # enter sudo mode

# create the configuration file for 'student' under sudo mode
echo "student ALL=(ALL) ALL" > /etc/sudoers.d/student
chmod 440 /etc/sudoers.d/student

Getting Help - man, apropos, --help, info and Others

man ls
man passwd
man -k passwd       # -k: key word, will return section of a key word
man -k `^passwd$`
man 5 passwd        # 5: the section returned by above command
man -f passwd
man -a socket       # all pages with the given name in all chapters

apropos passwd      # equivalent to `man -k [pages]`
apropos partition

what is passwd      # equivalent to `man -f [pages]`

info make

uname --help

Other helps

Rebooting and Shutting Down

halt
poweroff
sudo shutdown -h 10:00 "Shutting down for scheduled maintenance."

reboot
sudo shutdown -r 10:00 "Reboot for scheduled maintenance."

Useful Commands

Text Editors

Users, Environment and Permissions

Network Operations

Bash Shell and Basic Scripting

Printing

Local Security Principles