How to Build Secure Linux Server? [Part 1]

Securing a Linux server goes beyond installation and setup. Every server is vulnerable to attacks, from brute-force login attempts to malware and misconfigurations. This guide offers essential steps to strengthen your Linux server’s security, complete with detailed steps and the reasons behind them. Let’s make your Linux server as resilient as possible!

Word From Our Sponsor

Kloudnative is committed to being a valuable resource for tech enthusiasts seeking the latest updates on cloud-native technologies. To support our work, you can visit the sponsored link below. So check our sponsors today!!!

Learn AI in 5 minutes a day

What’s the secret to staying ahead of the curve in the world of AI? Information. Luckily, you can join 800,000+ early adopters reading The Rundown AI — the free newsletter that makes you smarter on AI with just a 5-minute read per day.

Sign up to start learning.

1. Disable Root Login

Why?

The root user has unlimited access, which makes it a target for attackers. Disabling root login prevents attackers from attempting brute-force attacks directly on this powerful account. Instead, a user with limited permissions is used, reducing the risk.

How to Do It

Open the SSH configuration file:

sudo nano /etc/ssh/sshd_config

Find the line:

PermitRootLogin yes

Change it to:

PermitRootLogin no

Save and close the file.

Restart the SSH service to apply changes:

sudo systemctl restart sshd

2. Use Key-Based SSH Authentication

Why?

Password-based logins can be weak points in server security. SSH key pairs are much harder to crack than passwords, adding a strong layer of security.

How to Do It

Generate an SSH key on your local machine:

ssh-keygen -t rsa -b 4096

This creates a public-private key pair for secure login.

Copy your public key to the server:

ssh-copy-id username@server_ip

To disable password-based login, open /etc/ssh/sshd_config on your server:

sudo nano /etc/ssh/sshd_config

Set PasswordAuthentication no and restart SSH:

sudo systemctl restart sshd

3. Enforce Strong Password Policies

Why?

Strong password policies prevent weak, easily guessed passwords, reducing the likelihood of brute-force attacks.

How to Do It

Open the password policy configuration file:

sudo nano /etc/security/pwquality.conf

Set policies like minimum length and complexity:

minlen = 12 minclass = 3

• minlen requires at least 12 characters.

• minclass requires multiple character types (uppercase, lowercase, digits, etc.).

4. Keep the System Updated

Why?

Updates contain patches for known vulnerabilities. Failing to update leaves your server exposed to known attacks.

How to Do It

Run the update command:

sudo apt update && sudo apt upgrade -y   # Debian/Ubuntu 
sudo yum update -y                       # CentOS/RHEL

Enable automatic updates (on Ubuntu):

sudo apt install unattended-upgrades

5. Configure a Firewall

Why?

A firewall limits access to specific services and blocks unauthorized traffic, reducing the risk of intrusion.

How to Do It

For Ubuntu: Install and configure ufw:

sudo apt install ufw
sudo ufw allow 22    # Allow SSH
sudo ufw allow 80    # Allow HTTP
sudo ufw allow 443   # Allow HTTPS
sudo ufw enable

• This allows only SSH, HTTP, and HTTPS traffic while blocking other ports.

6. Install and Configure Intrusion Detection (Fail2Ban)

Why?

Fail2Ban protects your server from brute-force attacks by blocking IPs with too many failed login attempts.

How to Do It

Install Fail2Ban:

sudo apt install fail2ban

Configure Fail2Ban by editing /etc/fail2ban/jail.conf:

sudo nano /etc/fail2ban/jail.conf

Enable SSH monitoring with:

[sshd] 
enabled = true 
maxretry = 5 
bantime = 3600

• Blocks IPs after 5 failed attempts for one hour.

7. Disable Unnecessary Services

Why?

Running fewer services means fewer potential entry points for attackers, improving overall security.

How to Do It

List all active services:

sudo systemctl list-unit-files --type=service --state=enabled

Disable unneeded services:

sudo systemctl disable service_name

8. Set Proper File Permissions

Why?

Sensitive files like SSH and log files should have strict permissions to prevent unauthorized access or modification.

How to Do It

Restrict access to important files:

sudo chmod 600 /etc/ssh/sshd_config 
sudo chmod 640 /var/log/auth.log

9. Enable Logging and Monitoring

Why?

Logging provides a record of system events, helping you detect unusual activities and analyze incidents.

How to Do It

1. Use rsyslog to manage logs, or consider a centralized logging solution like ELK (Elasticsearch, Logstash, Kibana) for easier monitoring.

10. Implement Auditing with auditd

Why?

Auditing monitors critical files and actions, alerting you to unauthorized changes or suspicious activity.

How to Do It

Install and configure auditd:

sudo apt install auditd

Add rules in /etc/audit/audit.rules to track important files:

-w /etc/passwd -p wa -k passwd_changes

Restart auditd to apply:

sudo systemctl restart auditd

11. Secure SSH Configuration

Why?

Configuring SSH settings hardens your server against attacks by limiting login options.

How to Do It

1. Open /etc/ssh/sshd_config:

sudo nano /etc/ssh/sshd_config

Adjust settings:

Port 2222                  # Change the default SSH port
PasswordAuthentication no   # Disable password login
Protocol 2                  # Use SSH protocol 2 only

Restart SSH:

sudo systemctl restart sshd

12. Harden Kernel Parameters

Why?

Kernel hardening secures network settings and mitigates certain attacks by restricting network behaviors.

How to Do It

Open /etc/sysctl.conf and add settings:

net.ipv4.tcp_syncookies = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0

Apply changes:

sudo sysctl -p

13. Schedule Regular Backups

Why?

Backups ensure data is recoverable in case of a cyberattack, accidental deletion, or system failure.

How to Do It

1. Use rsync or tar for backups:

rsync -av /important_data /backup_location

14. Set Resource Limits

Why?

Resource limits help prevent denial-of-service (DoS) attacks by limiting user resource consumption.

How to Do It

Edit /etc/security/limits.conf:

sudo nano /etc/security/limits.conf

Set limits:

* soft nproc 4096 
* hard nproc 8192

15. Use Security Scanning Tools

Why?

Security scanners identify misconfigurations and vulnerabilities, helping you fix issues before attackers exploit them.

How to Do It

Install Lynis:

sudo apt install lynis

Run a system scan:

sudo lynis audit system

16. Protect Against Malware

Why?

Linux can still be vulnerable to malware, especially in environments with internet access or file sharing.

How to Do It

Install ClamAV:

sudo apt install clamav

Update and scan:

sudo freshclam 
sudo clamscan -r /directory_to_scan

17. Enable Multi-Factor Authentication (MFA)

Why?

MFA adds a second layer of verification, making it more difficult for attackers to gain access, even with a password.

How to Do It

Install Google Authenticator:

sudo apt install libpam-google-authenticator

Set up MFA:

google-authenticator

Enable MFA in PAM configuration:

sudo nano /etc/pam.d/sshd

Add:

auth required pam_google_authenticator.so

18. Implement Network Segmentation

Why?

Network segmentation limits traffic between different parts of your infrastructure, reducing the impact if an attacker gains access. By isolating sensitive services on private subnets or VLANs, you limit exposure and protect data.

How to Do It

1. On AWS or other cloud platforms, use Virtual Private Clouds (VPCs) and subnets.

2. With Firewalls, configure rules to separate traffic between different services.

3. On Linux: Configure iptables to create network segmentation by defining strict rules for each service or IP address range that’s allowed access.

Example:

sudo iptables -A INPUT -p tcp -s trusted_ip --dport 22 -j ACCEPT

19. Restrict sudo Access

Why?

Limiting sudo access minimizes the risk of privilege escalation. Only trusted users should have sudo privileges, as any commands they execute can affect the entire system.

How to Do It

Edit the sudoers file:

sudo visudo

Define specific permissions for each user or user group:

username ALL=(ALL) NOPASSWD: /path/to/specific_command

Regularly audit the sudoers file to ensure only necessary permissions are granted.

20. Enforce AppArmor or SELinux for Mandatory Access Control

Why?

AppArmor and SELinux are mandatory access control systems that add fine-grained permissions, confining processes to a limited set of resources and actions. This limits the impact if a process is compromised.

How to Do It

For AppArmor (Ubuntu/Debian):

• Check if AppArmor is enabled:

sudo apparmor_status

• Configure specific profiles for services in /etc/apparmor.d/.

For SELinux (CentOS/RHEL):

• Enable SELinux:

udo setenforce 1

• Use semanage to define policies:

sudo semanage fcontext -a -t httpd_sys_content_t "/web(/.*)?"

21. Use Port Knocking for SSH Access

Why?

Port knocking helps hide the SSH port by requiring a sequence of port “knocks” to open the SSH port, making it harder for attackers to detect your SSH service.

How to Do It

Install knockd on your server:

sudo apt install knockd

Configure port knocking in /etc/knockd.conf:

[openSSH]
sequence = 7000,8000,9000
seq_timeout = 5
command = /sbin/iptables -A INPUT -s %IP% -p tcp --dport 22 -j ACCEPT

Start knockd:

sudo systemctl start knockd

Now, only after knocking on ports 7000, 8000, and 9000 in that order will port 22 open for SSH.

22. Limit Open Ports to Reduce Attack Surface

Why?

Open ports represent entry points for potential attackers. Limiting them to necessary services reduces the risk of unauthorized access.

How to Do It

Use netstat or ss to view open ports:

sudo ss -tuln

Close unnecessary ports by disabling or firewalling services:

sudo systemctl stop service_name
sudo systemctl disable service_name

For example, if only SSH and HTTP/HTTPS are needed, ensure only ports 22, 80, and 443 are open.

23. Use File Integrity Monitoring (FIM)

Why?

File Integrity Monitoring (FIM) detects unauthorized changes to critical system files, helping identify potential compromises or malicious modifications.

How to Do It

Install an FIM tool like AIDE (Advanced Intrusion Detection Environment):

sudo apt install aide

Initialize the AIDE database:

sudo aideinit

Set up a cron job to run regular AIDE checks:

sudo crontab -e

Add:

0 3 * * * /usr/bin/aide --check

24. Implement Rate Limiting

Why?

Rate limiting protects against denial-of-service (DoS) attacks by limiting the number of requests or logins from a single IP address.

How to Do It

Use iptables to limit SSH connections:

sudo iptables -A INPUT -p tcp --dport 22 -m state --state NEW -m recent --set
sudo iptables -A INPUT -p tcp --dport 22 -m state --state NEW -m recent --update --seconds 60 --hitcount 4 -j DROP

• This restricts SSH connections to a maximum of 3 attempts per minute.

Alternatively, configure rate limits with Fail2Ban by adjusting the findtime and maxretry options in /etc/fail2ban/jail.conf.

25. Encrypt Sensitive Data

Why?

Encryption protects data in case of a security breach by making it unreadable to unauthorized users. This applies to data stored on disk and transmitted over the network.

How to Do It

Data at Rest: Use encryption tools like ecryptfs or LUKS to encrypt sensitive files and partitions.

sudo apt install ecryptfs-utils sudo ecryptfs-setup-private

Data in Transit: Ensure all data transfers use encrypted channels (e.g., HTTPS for web traffic, SFTP for file transfers).