genpolicy: add utility script for containerd pull

docs/how-to/setup-containerd-pull.md

@@ -0,0 +1,96 @@
+# Manual Steps for Setting Up Containerd and Docker for Genpolicy Tool
+
+This guide provides instructions for manually setting up your system to use the genpolicy tool with containerd and Docker. This setup is needed for managed identity-based authentication to private registries using an identity token.
+
+These are the steps performed by the script `src/tools/genpolicy/setup_containerd_docker.sh`
+
+## Prerequisites
+
+- Ensure you have `sudo` access on the system.
+- Determine the package manager used by your system (`apt-get` or `tdnf`).
+
+## Steps
+
+### 1. Install Containerd
+
+#### Using `apt-get` (for Debian/Ubuntu-based systems)
+
+```bash
+sudo apt-get update
+sudo apt-get install -y containerd
+```
+
+#### Using `tdnf` (for Photon OS-based systems)
+```bash
+sudo tdnf makecache
+sudo tdnf install -y containerd
+```
+
+### 2. Modify Containerd Configuration
+Open the containerd configuration file `/etc/containerd/config.toml`. Ensure the cri plugin is not disabled. Locate the line that looks like this (if present):
+
+```toml
+disabled_plugins = ["cri"]
+```
+
+Modify it to:
+
+```toml
+disabled_plugins = [] # leave other plugins if present
+```
+Such that `cri` is not disabled. Save and close the file.
+
+### 3. Restart Containerd
+```bash
+sudo systemctl restart containerd
+```
+
+If containerd is not running, start it:
+
+```bash
+sudo systemctl start containerd
+```
+### 4. Verify Containerd Status
+```bash
+sudo systemctl status containerd --no-pager
+```
+
+### 5. Find Containerd Socket File Location
+Locate the containerd socket file by inspecting the configuration file:
+
+```bash
+SOCKET_FILE_LOCATION=$(awk '/\[grpc\]/ {found=1} found && /address *= */ {print $3; exit}' /etc/containerd/config.toml | tr -d '"')
+```
+
+The socket file is usually `/run/containerd/containerd.sock` or `/var/run/containerd/containerd.sock`
+
+### 6. Fix Containerd Socket File Permissions
+
+This step is necessary so genpolicy tool can run without `sudo`
+
+```bash
+sudo chmod a+rw "$SOCKET_FILE_LOCATION"
+```
+
+### 7. Install Docker and jq
+Using apt-get (for Debian/Ubuntu-based systems)
+
+```bash
+sudo apt-get install -y docker.io jq
+```
+
+Using tdnf (for Photon OS-based systems)
+```bash
+sudo tdnf install -y docker jq
+```
+
+### 8. Adapt Docker Configuration
+
+Remove `credstore` key and value from `~/.docker/config.json` if present.
+
+### Conclusion
+The setup is now complete. Please use the socket file location found in the configuration when running the genpolicy tool. For example:
+
+```bash
+genpolicy -d=$SOCKET_FILE_LOCATION -y foo.yaml
+```

src/tools/genpolicy/genpolicy-advanced-command-line-parameters.md

@@ -57,17 +57,29 @@ To enable caching, use the `-u` command line parameter - e.g.,
 $ RUST_LOG=info genpolicy -u -y test.yaml
 ```
 
-# Use containerd to pull and manage images
-You may specify `-d` to use existing `containerd` installation as image manager. This method supports a wider set of images (e.g., older images with `v1` manifest). Needs `sudo` permission to access socket - e.g.,
+# Use containerd to pull and manage images (required for managed identity based authentication)
+
+Prereq: This features needs to run the following script to adapt your docker and containerd config (needs `sudo` access):
+
+```bash
+$ ./setup_containerd_docker.sh
+```
+
+Optional: Authenticate to private registry. For example in azure:
+```bash
+$ az acr login -n my_acr_name
+```
+
+You may specify `-d` to use existing `containerd` installation as image manager. This method supports a wider set of images (e.g., older images with `v1` manifest) - e.g.,
 
 ```bash
-$ sudo genpolicy -d -y test.yaml
+$ genpolicy -d -y test.yaml
 ```
 
 This will use `/var/contaienrd/containerd.sock` as default socket path. Or you may specify your own socket path - e.g.,
 
 ```bash
-$ sudo genpolicy -d=/my/path/containerd.sock -y test.yaml
+$ genpolicy -d=/my/path/containerd.sock -y test.yaml
 ```
 
 # Print the Policy text

src/tools/genpolicy/setup_containerd_docker.sh

@@ -0,0 +1,138 @@
+#!/bin/bash
+
+# Copyright (c) 2024 Microsoft Corporation
+
+# This script aims to adapt the system for the genpolicy tool to be able to use containerd pull function properly.
+# This is needed for managed identity based authentication to private registries using an identity token.
+
+# This script needs 'sudo' access. It will:
+# - install containerd if not installed already. Genpolicy tool uses containerd to pull image when using -d option
+# - ensure cri plugin is NOT disabled in containerd config. This is needed for policy tool to pull image
+# - restart containerd or start it if not running already. 
+# - print containerd socket file location. Use this when running genpolicy tool. Eg genpolicy -d=$SOCKET_FILE_LOCATION -y foo.yaml
+# - fix containerd socket file permissions if needed. This is so genpolicy tool is able to access containerd socket file without 'sudo'
+# - adapt docker config.json if needed. This is done so 'az acr login' command saves identity token to the docker config
+
+set -e -x
+
+# Function to check if a command exists
+command_exists() {
+    command -v "$1" > /dev/null
+}
+
+# Utility function for error messages and exit
+error_exit() {
+    echo "$1" 1>&2
+    exit 1
+}
+
+# Detect the package manager
+if command_exists apt-get; then
+    UPDATE_CMD="sudo apt-get update"
+    INSTALL_CMD="sudo apt-get install -y"
+elif command_exists tdnf; then
+    UPDATE_CMD="sudo tdnf makecache"
+    INSTALL_CMD="sudo tdnf install -y"
+else
+    error_exit "No supported package manager found. Please install either apt-get or tdnf."
+fi
+
+# Function to ensure a command is installed
+ensure_command_installed() {
+    if ! command_exists "$1"; then
+        echo "$1 could not be found, installing..."
+        $INSTALL_CMD "$1"
+        if ! command_exists "$1"; then
+            error_exit "Failed to install $1"
+        fi
+    else
+        echo "$1 is already installed"
+    fi
+}
+
+$UPDATE_CMD
+
+ensure_command_installed containerd
+
+# Modify containerd config if needed
+CONTAINERD_CONFIG_FILE="/etc/containerd/config.toml"
+
+if [ ! -f "$CONTAINERD_CONFIG_FILE" ]; then
+    error_exit "Containerd config file not found: $CONTAINERD_CONFIG_FILE. Please update CONTAINERD_CONFIG_FILE in this script to point to the correct containerd config file."
+fi
+
+if grep -qE 'disabled_plugins.*\["cri"\]' "$CONTAINERD_CONFIG_FILE" || grep -qE "disabled_plugins.*\['cri'\]" "$CONTAINERD_CONFIG_FILE"; then
+    echo "Modifying containerd config to enable cri plugin..."
+    sudo sed -i -E "s/disabled_plugins.*(\['cri'\]|\[\"cri\"\])/disabled_plugins = []/g" "$CONTAINERD_CONFIG_FILE"
+else
+    echo "CRI plugin is already enabled in containerd config"
+fi
+
+# Restart containerd using systemctl
+echo "Restarting containerd service..."
+if systemctl is-active --quiet containerd; then
+    sudo systemctl restart containerd
+else
+    sudo systemctl start containerd
+fi
+
+# Print containerd status
+echo "Containerd status:"
+sudo systemctl status containerd --no-pager
+
+# Print containerd socket file location found in config
+SOCKET_FILE_LOCATION=$(awk '/\[grpc\]/ {found=1} found && /address *= */ {print $3; exit}' "$CONTAINERD_CONFIG_FILE" | tr -d '"')
+
+if [ -z "$SOCKET_FILE_LOCATION" ]; then
+    error_exit "Socket file location not found in config"
+fi
+echo "Containerd socket file location: $SOCKET_FILE_LOCATION"
+
+# Wait for the socket file to be created
+echo "Waiting for containerd socket file to be created..."
+for i in {1..10}; do
+    if [ -e "$SOCKET_FILE_LOCATION" ]; then
+        echo "Containerd socket file found: $SOCKET_FILE_LOCATION"
+        break
+    fi
+    echo "Attempt $i: Socket file not found, waiting..."
+    sleep 1
+done
+
+if [ ! -e "$SOCKET_FILE_LOCATION" ]; then
+    error_exit "Socket file not found after waiting: $SOCKET_FILE_LOCATION"
+fi
+
+# Fix containerd socket file permissions
+echo "Ensuring containerd socket file permissions are set correctly..."
+sudo chmod a+rw "$SOCKET_FILE_LOCATION"
+
+if ! command_exists docker; then
+    echo "$1 could not be found, installing..."
+    sudo apt-get install -y docker.io
+    if ! command_exists docker; then
+        error_exit "Failed to install docker.io"
+    fi
+else
+    echo "docker is already installed"
+fi
+
+ensure_command_installed jq
+
+# Adapt Docker config if needed
+DOCKER_CONFIG_FILE="$HOME/.docker/config.json"
+echo "Checking Docker config file: $DOCKER_CONFIG_FILE"
+
+if [ -f "$DOCKER_CONFIG_FILE" ]; then
+    if grep -q '"credstore":' "$DOCKER_CONFIG_FILE"; then
+        echo "Modifying Docker config to remove 'credstore' key..."
+        jq 'del(.credstore)' "$DOCKER_CONFIG_FILE" > "$DOCKER_CONFIG_FILE.tmp" && mv "$DOCKER_CONFIG_FILE.tmp" "$DOCKER_CONFIG_FILE"
+    else
+        echo "'credstore' key not found in Docker config"
+    fi
+else
+    error_exit "Docker config file not found. Please update DOCKER_CONFIG_FILE in this script to point to the correct Docker config file."
+fi
+
+echo "Script execution completed. Please use $SOCKET_FILE_LOCATION as socket file location."
+echo "Eg. genpolicy -d=$SOCKET_FILE_LOCATION -y foo.yaml"