Chapter 1: Programming / Scripting

The "Why": Why Does a DevOps Engineer Need to Code?

A common misconception is that DevOps is just "Operations" (managing servers) and that developers (Devs) do all the coding. This is completely false. DevOps is a *culture* and a *practice* that merges Development and Operations, and the "glue" that holds it all together is **automation**.

You cannot automate without programming. As a DevOps engineer, your primary job is to **automate everything**. You are a "Software Engineer for Infrastructure." You will write scripts and programs to:

• Provision Infrastructure: Automatically create (and destroy) 100 new servers on AWS, Azure, or GCP.
• Build & Test Code (CI): Create CI/CD pipelines that automatically build, test, and scan code every time a developer commits.
• Deploy Applications (CD): Automatically deploy the new version of the app to all 100 servers with zero downtime.
• **Monitor & Log:** Write scripts to check if all servers are healthy, aggregate logs, and send an alert to Slack if one goes down.
• Manage Backups: Write a script that automatically backs up your production database every night at 2 AM.
• Create Tools: Build internal command-line tools (CLIs) to make other developers' lives easier.

A developer writes code for the *product*. A DevOps engineer writes code for the *process*. Your code is the assembly line that builds and ships the product.

What to Learn: Bash vs. Python vs. Go

You don't need to learn all three, but you need to know *when* to use each one.

1. Bash (or another shell script): This is your **must-have**. It's the native language of every Linux server. You use it for simple, file-based tasks and for "gluing" other command-line tools together.
2. Python: This is your **workhorse**. When a task becomes too complex for Bash (e.g., it involves JSON, API calls, or complex logic), you use Python. It's the most popular language for DevOps automation.
3. Go (Golang): This is your **specialty tool**. Go is a compiled language that is extremely fast and excellent at concurrency (doing many things at once). You use it to build high-performance command-line tools and infrastructure components (like Docker and Kubernetes, which are written in Go).

This chapter will provide a deep dive into Bash and Python, as they cover 99% of DevOps tasks, and an introduction to Go.

Part 1: Bash Scripting (The Linux "Glue")

Bash (Bourne Again Shell) is the default command-line interpreter for nearly all Linux distributions (and macOS). A Bash script is simply a plain text file containing a series of commands that you would normally type into your terminal, one after another. It's the most basic and fundamental way to automate tasks on a server.

1. "Hello, World!" in Bash

Let's create our first script. Open a file named hello.sh.

#!/bin/bash

# This is a comment. The interpreter ignores it.
# 'echo' is the command to print text to the console.
echo "Hello, CodeWithMSMAXPRO!"

• #!/bin/bash: This is called the **"Shebang"**. It *must* be the very first line. It tells the operating system *which* interpreter (in this case, /bin/bash) to use to run this script. Other shells exist (like #!/bin/sh or #!/bin/zsh), but bash is the most common.
• Permissions: By default, new text files are not "executable." You must tell Linux that this file is a program.

  # Give the owner (u) permission to execute (x) the file
  $ chmod u+x hello.sh
  

• Running the Script: You run the script by typing its path.

  # './' means "in the current directory"
  $ ./hello.sh
  Hello, CodeWithMSMAXPRO!
  

2. Variables

In Bash, you create variables (all-caps by convention) and access them with the $ sign. **Important:** There are **NO spaces** around the = sign.

#!/bin/bash

# Assignment (NO SPACES)
NAME="MSMAXPRO"

# Accessing the variable
echo "Hello, $NAME"

# Use ${} to separate the variable from other text
echo "Hello, ${NAME}! Welcome."

# Wrong (with spaces):
# NAME = "MSMAXPRO"  <-- This will fail!

Command Substitution

You can store the *output* of a command in a variable using $(...).

#!/bin/bash

# Store the output of the 'date' command
CURRENT_DATE=$(date '+%Y-%m-%d')
echo "Today's date is $CURRENT_DATE"

# Store the list of .txt files
TXT_FILES=$(ls *.txt)
echo "Text files found: $TXT_FILES"

3. User Input (`read`)

You can ask the user for input using the read command. The -p flag adds a prompt.

#!/bin/bash

read -p "What is your name? " USER_NAME
read -s -p "Enter your password (it will be hidden): " PASSWORD

echo "" # New line
echo "Hello, $USER_NAME. We will not print your password."

• -p "Prompt": Displays the prompt text.
• -s: "Silent" mode, hides the user's input. Perfect for passwords.

4. Positional Arguments (`$1`, `$2`, ...)

You can pass arguments to your script when you run it. These are stored in special variables: $1 (first argument), $2 (second argument), and so on. $0 is the name of the script itself, and $# is the total number of arguments.

#!/bin/bash

# Run this script like: ./backup.sh /var/www /mnt/backups

SOURCE_DIR=$1
DEST_DIR=$2

echo "Backing up from $SOURCE_DIR to $DEST_DIR..."

5. Control Flow: `if` Statements (Deep Dive)

This is where Bash gets tricky. The syntax is old and very specific. There are two ways: the traditional [ ... ] (single brackets) and the modern [[ ... ]] (double brackets).

Rule: Always use double brackets [[ ... ]]. They are safer, more powerful, and prevent many common errors.

#!/bin/bash

read -p "Enter a number: " NUM

# Note: spaces are MANDATORY inside the brackets
if [[ $NUM -gt 10 ]]; then
    echo "Your number ($NUM) is greater than 10."
elif [[ $NUM -eq 10 ]]; then
    echo "Your number is exactly 10."
else
    echo "Your number is less than 10."
fi # 'if' is closed with 'fi' (if backwards)

Bash Comparison Cheatsheet

• Numbers: -eq (equal), -ne (not equal), -gt (greater than), -lt (less than), -ge (greater/equal), -le (less/equal)
• Strings (inside `[[ ... ]]`): == (equal, with pattern matching), != (not equal), -z $VAR (is empty), -n $VAR (is not empty)
• Files: -f $FILE (is a file), -d $DIR (is a directory), -e $PATH (exists)
• Logic: && (AND), || (OR), ! (NOT)

FILE_PATH="/var/log/syslog"
USERNAME="admin"

# Check if file exists AND username is "admin"
if [[ -f $FILE_PATH && $USERNAME == "admin" ]]; then
    echo "Admin user found and file exists."
fi

# Check if a string contains a substring (regex)
if [[ "Hello_World" == *World* ]]; then
    echo "It contains World"
fi

6. Control Flow: `for` and `while` Loops

Loops let you iterate over lists or run code until a condition is met.

#!/bin/bash

# 1. 'for' loop (over a list of strings)
SERVICES="web db cache"
for SERVICE in $SERVICES; do
    echo "Restarting $SERVICE server..."
    sleep 1
done

# 2. 'while' loop (e.g., wait for a server to come online)
COUNT=0
while [[ $COUNT -lt 5 ]]; do
    echo "Waiting... ($COUNT/5)"
    sleep 1
    COUNT=$((COUNT + 1)) # Arithmetic in bash
done
echo "Done waiting."

7. Functions

Functions let you group reusable code. Bash functions are simple. $1 is the first argument, $2 is the second, etc. $? holds the "exit code" of the last command (0 means success, non-zero means error).

#!/bin/bash

# Define the function
is_server_alive() {
    URL=$1
    echo "Pinging $URL..."
    
    # 'ping -c 1' sends one packet. 
    # '> /dev/null' hides the output
    ping -c 1 $URL > /dev/null
    
    if [[ $? -eq 0 ]]; then
        return 0 # Success
    else
        return 1 # Failure
    fi
}

# Call the function
is_server_alive "google.com"

# Check the exit code of the function
if [[ $? -eq 0 ]]; then
    echo "google.com is UP!"
else
    echo "google.com is DOWN!"
fi

8. Essential DevOps Commands (The "Glue")

The real power of Bash is not the language itself, but its ability to "glue" together other powerful command-line tools. These are your bread and butter.

grep (Find Text)

grep (Global Regular Expression Print) searches for a text pattern inside a file or input stream.

# Search for "error" in 'app.log'
grep "error" /var/log/app.log

# Search case-insensitively (-i) and show line numbers (-n)
grep -in "database connection" /var/log/app.log

# Show 3 lines *after* the match (-A 3)
grep -A 3 "FATAL_ERROR" /var/log/app.log

# Search for a pattern (Regex)
grep -E "error|warn" /var/log/app.log

Pipes: | (Chaining Commands)

The pipe | is the most powerful concept. It takes the **output** of the command on the left and uses it as the **input** for the command on the right.

# 1. 'ps aux' lists ALL running processes (hundreds of lines)
# 2. '|' pipes that output into 'grep'
# 3. 'grep "nginx"' filters the list to only show lines with "nginx"
ps aux | grep "nginx"

awk (Extract Columns)

awk is a tool for processing text, column by column. It's often used with grep.

# Find the 'nginx' process and print only the 2nd column (the Process ID)
ps aux | grep "nginx" | awk '{print $2}'

sed (Search and Replace)

sed (Stream Editor) is used to find and replace text inside a file.

# In 'config.txt', replace "DATABASE_URL=old" with "DATABASE_URL=new"
# '-i' means "in-place" (edit the file directly)
sed -i 's/DATABASE_URL=old/DATABASE_URL=new/g' config.txt

curl (Make Network Requests)

curl is your tool for making HTTP requests from the command line. You use it to test if your servers are online.

# Make a simple GET request to your website
curl https://codewithmsmaxpro.me

# Get only the HTTP status code (silent, no output)
curl -s -o /dev/null -w "%{http_code}" https://codewithmsmaxpro.me

# Make a POST request with JSON data
curl -X POST -H "Content-Type: application/json" \
     -d '{"username":"test"}' \
     https://api.example.com/login

jq (JSON Processor)

curl gives you raw JSON. jq is a command-line tool (like sed for JSON) that lets you parse and extract data from it.

# Get user data from an API, pipe it to jq, and extract *only* the email field
curl -s 'https://jsonplaceholder.typicode.com/users/1' | jq '.email'
"Sincere@april.biz"

Real-World Bash Script: Server Health Check

Let's put it all together. This script checks if a website is online. If not, it sends an alert (in a real world, this would send an email).

#!/bin/bash

SITE_URL="https://codewithmsmaxpro.me"
LOG_FILE="server_health.log"

# Get the HTTP status code from the server
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" $SITE_URL)

if [[ $HTTP_CODE -ne 200 ]]; then
    # Site is DOWN
    TIMESTAMP=$(date "+%Y-%m-%d %H:%M:%S")
    MESSAGE="[$TIMESTAMP] ERROR: Site $SITE_URL is DOWN! (Code: $HTTP_CODE)"
    
    echo $MESSAGE
    echo $MESSAGE >> $LOG_FILE # Append to log file
else
    # Site is UP
    echo "Site is UP and running (Code: $HTTP_CODE)"
fi

Part 2: Python (The Automation Workhorse)

Bash is great for simple scripts, but it has problems. The syntax is clunky, it can't handle complex data (like JSON) well, and error handling is difficult. When your script needs to be more than 100 lines long, it's time to use **Python**.

Python is the #1 language for DevOps and automation. Why?

• Easy to Read: The syntax is clean and looks like English.
• Powerful Libraries: It has battle-tested, free libraries for *everything*.
• Better Error Handling: Python's try...except blocks are far superior to Bash's.
• Cross-Platform: Your Python script will run on Linux, Windows, and macOS without changes. A Bash script will not run on Windows.

1. Python Environment Setup (Crucial)

You should *never* install packages (like boto3) to your main system Python. You must create an isolated **virtual environment** for *each* project.

# 1. Create a new directory for your project
$ mkdir my-python-script
$ cd my-python-script

# 2. Create a virtual environment named 'venv'
$ python3 -m venv venv

# 3. Activate the environment
# On Linux/macOS:
$ source venv/bin/activate
# On Windows (PowerShell):
$ .\venv\Scripts\Activate.ps1

# Your terminal prompt will now change to show (venv)

# 4. Install libraries. They only install inside 'venv'
(venv) $ pip install requests
(venv) $ pip install boto3

# 5. When you are done, just type 'deactivate'
(venv) $ deactivate

2. Python Basics (for DevOps)

Data Types (Lists and Dictionaries)

Python's lists and dictionaries (maps) are easy to use and essential for handling data.

# A List (like a MutableList in Kotlin)
servers = ['web-01', 'db-01', 'cache-01']
servers.append('web-02')
print(servers[0]) # 'web-01'

# Loop over a list
for server in servers:
    print(f"Processing {server}")

# A Dictionary (like a Map in Kotlin)
config = {
    'host': '192.168.1.10',
    'port': 8080,
    'retries': 3
}

# Accessing data
print(f"Connecting to {config['host']} on port {config['port']}")

Functions and Error Handling

Python functions are declared with the def keyword. Error handling is done with try...except.

def divide(a, b):
    try:
        result = a / b
        return result
    except ZeroDivisionError:
        print("Error: Cannot divide by zero!")
        return None
    finally:
        print("Division attempt finished.")

divide(10, 2)
divide(10, 0)

3. Essential Python Libraries for DevOps

os and sys (Interacting with the OS)

The os module lets you talk to the operating system (e.g., read environment variables). The sys module lets you handle arguments passed to your script.

import os
import sys

def main():
    # Get an Environment Variable (safer than hardcoding)
    api_key = os.environ.get('MY_API_KEY')
    if not api_key:
        print("Error: MY_API_KEY environment variable not set.")
        sys.exit(1) # Exit with an error code

    # Get arguments passed to the script
    # $ python my_script.py server1
    if len(sys.argv) < 2:
        print("Usage: python my_script.py ")
        sys.exit(1)
        
    server_name = sys.argv[1] # 'server1' (index 0 is the script name)
    print(f"Working on server: {server_name}")
    print(f"Using API key: {api_key[:4]}...")

if __name__ == "__main__":
    main()

subprocess (Running Shell Commands)

This is the *right* way to run Bash commands (like ls -l or docker ps) from *inside* your Python script.

import subprocess

def list_docker_containers():
    print("Running 'docker ps'...")
    command = ['docker', 'ps', '--format', '{{.ID}}\t{{.Names}}']
    
    try:
        result = subprocess.run(
            command, 
            capture_output=True, 
            text=True, 
            check=True # This will raise an error if docker fails
        )
        print("--- Docker Output ---")
        print(result.stdout)
    except FileNotFoundError:
        print("Error: 'docker' command not found.")
    except subprocess.CalledProcessError as e:
        print(f"Error running docker: {e.stderr}")

list_docker_containers()

json and requests (Working with APIs)

This is where Python truly beats Bash. You can easily make API calls with requests and parse the json response.

(First, run pip install requests in your terminal).

import requests
import json
import sys

def get_user_from_api(user_id):
    url = f"https://jsonplaceholder.typicode.com/users/{user_id}"
    
    try:
        response = requests.get(url, timeout=5) # 5 second timeout
        
        # Automatically check for errors (4xx or 5xx)
        response.raise_for_status() 
        
        # Parse the JSON text into a Python dictionary
        user_data = response.json()
        
        print(f"Username: {user_data['username']}")
        print(f"City: {user_data['address']['city']}")
        
    except requests.exceptions.HTTPError as e:
        print(f"API Error: {e.response.status_code}", file=sys.stderr)
    except requests.exceptions.ConnectionError:
        print("Network Error: Could not connect.", file=sys.stderr)
    except requests.exceptions.Timeout:
        print("Network Error: Request timed out.", file=sys.stderr)
    except json.JSONDecodeError:
        print("Error: Failed to parse JSON response.", file=sys.stderr)


get_user_from_api(1)

Real-World Python Script: AWS S3 Backup

Here is a script that uses the boto3 library (pip install boto3) to find all files in a local directory and upload them to an Amazon S3 (storage) bucket.

import os
import boto3
import sys
from botocore.exceptions import NoCredentialsError, ClientError

# Get config from Environment Variables
S3_BUCKET_NAME = os.environ.get('S3_BUCKET_NAME')
LOCAL_BACKUP_DIR = os.environ.get('LOCAL_BACKUP_DIR')

s3_client = boto3.client('s3')

def upload_backups():
    print(f"Starting upload to {S3_BUCKET_NAME}...")
    
    if not S3_BUCKET_NAME or not LOCAL_BACKUP_DIR:
        print("Error: S3_BUCKET_NAME or LOCAL_BACKUP_DIR not set.", file=sys.stderr)
        sys.exit(1)
        
    try:
        # Walk through the directory
        for root, dirs, files in os.walk(LOCAL_BACKUP_DIR):
            for filename in files:
                local_path = os.path.join(root, filename)
                s3_key = f"db_backups/{filename}"
                
                print(f"Uploading {local_path} to {s3_key}...")
                s3_client.upload_file(local_path, S3_BUCKET_NAME, s3_key)
        
        print("Backup complete!")
        
    except NoCredentialsError:
        print("Error: AWS credentials not found. Run 'aws configure'.", file=sys.stderr)
    except ClientError as e:
        print(f"AWS Client Error: {e}", file=sys.stderr)
    except Exception as e:
        print(f"An error occurred: {e}", file=sys.stderr)

if __name__ == "__main__":
    upload_backups()

Part 3: Go (Golang) - The Performance Tool

Go is a language you'll learn *after* you are comfortable with Python and Bash. It's a statically compiled language built by Google, which means it's extremely fast and produces a single binary file (e.g., my_tool.exe) that you can run anywhere without installing any dependencies.

It's used for high-performance infrastructure tools. **Docker, Kubernetes, Prometheus, and Terraform** are all written in Go. It is the true language of modern infrastructure.

Key Features

• Concurrency (Goroutines):** Go's main feature. It can easily run *thousands* of tasks at the same time, making it perfect for servers that handle many requests.
• Static Binary:** It compiles to *one single file* with no dependencies, making it trivial to deploy.
• Fast:** It's a compiled language, so it's nearly as fast as C or C++.
• Simple:** The language itself is very small, with only 25 keywords.

package main

import "fmt" // (fmt is the formatting/printing package)

func main() {
    fmt.Println("Hello from Go!")
}

package main
import ("fmt"; "time")

func say(s string) {
    for i := 0; i < 3; i++ {
        time.Sleep(100 * time.Millisecond)
        fmt.Println(s)
    }
}

func main() {
    // Start a new goroutine (background task)
    go say("World")
    
    // Run this on the main task
    say("Hello")
}

// Output (they run concurrently!):
// Hello
// World
// Hello
// World
// Hello
// World