Skip to content

linux.md

Latest commit

 

History

History
465 lines (313 loc) · 10.7 KB

linux.md

File metadata and controls

465 lines (313 loc) · 10.7 KB

Linux

Shell

Shell - the command line program with some scripting constructs that calls the binary programs in /bin, /usr/bin and similar directories.

Start with Bash which is the standard open source Linux shell.

Distributions & Lineage

  • Debian - the standard open source distribution

    • Ubuntu - more updated distro, originally Desktop focused then expanded into cloud server focus too - now the most widely used cloud distro

  • Redhat - Redhat Enterprise Linux (RHEL) and its clone CentOS used to the standard enterprise distro but has killed its open source credentials by strangling CentOS and consequently become legacy

    • Amazon Linux
    • Centos
    • Fedora
    • RockyLinux

  • Gentoo - l33t but takes time to compile.

  • Alpine - slim distribution designed for Docker

Debian stable is the most stable and reliable Linux out there since at least the 2000s, at the cost of lagging on package versions and its UIs are usually not as glossy. It is one of the top picks ever for servers for the reason of stability, rivalled only by commercial RHEL.

More people prefer Ubuntu for desktops for a bit more gloss and user focus, which was based on Debian and has been mature for many years - you can also pick Ubuntu Long Term Support releases for extra stability, people usually use those for servers - it’s been the number 1 cloud server distro for a decade+.

Fedora is ok for desktops, but it’s not as stable by design as Ubuntu LTS or the daddy Debian.

I think that Redhat after IBM acquisition has damaged the Redhat lineage by trying to squeeze out more profits by killing CentOS to force people to pay for their Enterprise Linux.

The history of Linux shows it’s hard to beat free over the long term.

Distribution Version

The generic way:

cat /etc/*-release

These files have different contents:

Distro File
Alpine /etc/os-release
/etc/alpine-release
Amazon Linux /etc/os-release
/etc/system-release
/etc/amazon-release
CentOS /etc/os-release
/etc/system-release
/etc/redhat-release
/etc/centos-release
Debian /etc/os-release
Gentoo /etc/os-release
/etc/gentoo-release
Redhat /etc/os-release
/etc/system-release
/etc/redhat-release
RockyLinux /etc/os-release
/etc/system-release
/etc/redhat-release
/etc/rocky-release
Ubuntu /etc/os-release
/etc/lsb-release

Cron

In RHEL 6

/etc/cron.allow

/etc/cron.deny

/var/spool/cron root:root 700

User Crons

Stored in /var/spool/cron/$USER.

crontab command is suid to allow user to manage it.

Opens the crontab in $EDITOR (default vi if $EDITOR environment variable is not set):

crontab -e

Reference:

man 5 crontab

Put this at the top of your user crontab file for easy reference:

# ┌──────── minute (0 - 59)
# │ ┌────── hour (0 - 23)
# │ │ ┌──── day of month (1 - 31)
# │ │ │ ┌── month (1 - 12)
# │ │ │ │ ┌─ day of week (0 - 7) (Sun=0 or 7)
# │ │ │ │ │
# * * * * * command

You can also use one of these timing shorthands:

@reboot
@yearly
@monthly
@weekly
@daily
@hourly

You may also find this site useful:

https://crontab.guru

Timezone

This affects the cron scheduling above and recorded dates of jobs eg. data loading and recording.

For modern Linux systems with systemd:

timedatectl list-timezones

Servers should usually be set to UTC for consistent easy comparison across international systems unless this affects data loading dates from cron above.

timedatectl set-timezone UTC

Networking

See Networking doc.

IPtables

Top for iptables, awesome!

iptstate

List rules with line numbers:

iptables -nL -line-numbers

DHCP

Install ISC DHCPd:

yum install -y dhcp

Edit config:

vim /etc/dhcp/dhcpd.conf

Enable it at boot:

systemctl enable dhcpd.service

Start the service:

systemctl start dhcpd.service

Test DHCP

Install dhcping tool:

yum install -y dhcping

Test DHCP response:

dhcping -s localhost

CGroups

Limit resource usage.

This is used by modern containerization like containerd and Docker.

Can limit:

  • CPU Time
  • CPU core assignments
  • Memory
  • Devices
  • Disk / Block I/O
  • Network bandwidth
yum install -y libcgroup
service cgconfig start
ls /cgroup
lscgroup

Create cgroup - /etc/cgconfig.conf:

group blah {
  cpu {
    cpu.shares = 400;
  }
}

service cgconfig restart

then add processes (tasks) into cgroups according to parameters in the file:

/etc/cgrules.conf:

<user> <subsystems> <control_group>
@<group> <subsystems> <control_group>
<user>:<command> <subsystems> <control_group>
eg.
*:firefox cpu,memory browsers/

service cgred start

Sysconfig services can instead add this to their /etc/sysconfig/<servicename> file

CGROUP_DAEMON="<subsystem>:<control_group>"

Disk Management

List disk space of mounted partitions:

df -h

List partitions:

cat /proc/partitions

Format a spare partition:

mkfs.ext4 /dev/sda2

Check and recover filesystem, replay journal, prompts for fixes:

fsck /dev/sda2

Mount a filesystem to the directory /data:

mount /dev/sda2 /data

/etc/fstab

Ensure the partition is:

  1. mounted by UUID as device numbers can change
  2. has nofail option set to make sure that a machine will attempt to come up to be able to SSH manage it otherwise you may end up in an AWS EC2 Disk Mount Recovery situation.

First inspect your /etc/fstab:

cat /etc/fstab

Back up /etc/fstab before editing it:

sudo cp -av /etc/fstab /etc/fstab.bak."$(date +%F_%H%S)"

Add the nofail option on any lines on which it does not exist:

sudo sed -i '/nofail/ ! s/defaults/defaults,nofail/' /etc/fstab

Inspect the changes:

cat /etc/fstab

Each line in the /etc/fstab should then look like:

UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx    /tmp    xfs    defaults,nofail    0    2

Validate your /etc/fstab by mounting using the short form of the mount command that reads and uses the /etc/fstab:

mount /tmp

Disable tmpfs

tmpfs stores /tmp files in a ramdisk, limited by the machine's RAM.

This is a trade off of memory vs /tmp files performance.

But this can also be a problem if you have a large volume of data going through /tmp, in which case you might want to disable it to avoid running out of RAM or /tmp space.

To disable on a systemd based Linux distro like RHEL-based variants, including Amazon Linux:

sudo systemctl stop tmp.mount
sudo systemctl mask tmp.mount

Output:

Created symlink /etc/systemd/system/tmp.mount → /dev/null.

DRBD

  • awesome disk replication, used this in the mid to late 2000s
  • mainline Linux kernel now
  • dual-primary (0.9+)mount
    • requires clustered filesystem (GFS, OCFS2)
  • mount -o ro to avoid complexity of dual primary cluster filesystems
  • sync + async repl options
  • get check_drbd nagios plugin to see how far behind replica is, automatically catches up, low maintenance once set up

Binaries Debugging

See the Binaries Debugging doc for commands to examine and work with binaries.

Linux Boot Process

Linux Boot Process

Linux Filesystem Layout

Linux Filesystem Layout

OOM Killer

Out of Memory Killer will kill -9 forcibly terminate the largest RAM consuming process when the OS runs out of RAM.

This is considered the better alternative than letting the whole OS crash.

Linux OOM Killer

eBPF

Kernel level for networking observability, tracing, and security

https://ebpf.io/

Meme

Headquarters

Headquarters

How I Sleep

But then you discover Macs and spend it anyway...

How I Sleep Paying No Licenses

Standardizing on Distro

Standardizing on Distro

Linux Cars

Linux Cars

Windows Problems vs Linux Problems

Mac falls somewhere in between the two, depending on the problem...

Windows Problems vs Linux Problems

Linux Inside

Linux Inside

Linux Users Switching Back After 10 Minutes of Using Windows

Linux Users Switching Back After 10 Minutes of Using Windows

Ported from various private Knowledge Base pages 2002+