Post

Icon Server API Key Authentication

Understanding how Krill servers use API keys for client authentication and how to manage them

Server API Key Authentication

Overview

Krill uses API key authentication to secure communication between clients and the server. Each Krill server installation automatically generates a unique API key that clients must provide in their request headers to access protected endpoints. This document explains how API keys work, where they are stored, and how to manage them.

Architecture

Authentication Flow Diagram

sequenceDiagram
    participant Client as Krill Client
    participant Server as Krill Server
    participant KeyStore as API Key File

    Note over Server: Server starts and<br/>loads API key from file

    Client->>Server: HTTPS Request
    Note right of Client: Includes X-API-Key header

    Server->>KeyStore: Read stored API key
    KeyStore-->>Server: Return key value

    alt API Key Valid
        Server-->>Client: 200 OK + Response
    else API Key Missing/Invalid
        Server-->>Client: 401 Unauthorized
    end

Request Authentication Flow

flowchart TD
    A[Incoming Request] --> B{X-API-Key Header Present?}
    B -->|No| C[Return 401 Unauthorized]
    B -->|Yes| D{Key Matches Stored Key?}
    D -->|No| C
    D -->|Yes| E[Process Request]
    E --> F[Return Response]

Server-Side Implementation

API Key Generation

When you install Krill Server via the Debian package, the postinst script automatically generates a unique API key on every installation:

API Key File Location: /etc/krill/credentials/api_key

FileDescriptionPermissions
api_keyUUID-format API key0400 (owner read-only)

The API key is generated using:

  • Primary method: /proc/sys/kernel/random/uuid (Linux kernel random UUID)
  • Fallback 1: uuidgen command if available
  • Fallback 2: OpenSSL-generated pseudo-UUID

Directory Structure

1
2
3
4
5
6
7
8
9
/etc/krill/
├── credentials/          # Secure credentials directory (mode 0700)
│   └── api_key          # API key file (mode 0400)
├── certs/               # TLS certificates
│   ├── krill.crt
│   ├── krill.key
│   ├── krill.pfx
│   └── .pfx_password
└── version              # Package version

Security Permissions

The API key file uses restrictive permissions following Debian security best practices:

PathModeOwnerDescription
/etc/krill/credentials/0700krill:krillCredentials directory
/etc/krill/credentials/api_key0400krill:krillRead-only by krill user

Managing API Keys

Viewing the Current API Key

To view the current API key, SSH into your Krill server and run:

1
sudo cat /etc/krill/credentials/api_key

Regenerating the API Key

If you need to regenerate the API key (e.g., if it was compromised), follow these steps:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# SSH into your Krill server
ssh user@your-krill-server

# Generate a new UUID
NEW_KEY=$(cat /proc/sys/kernel/random/uuid)

# Securely overwrite the old key first
sudo dd if=/dev/urandom of=/etc/krill/credentials/api_key bs=64 count=1 2>/dev/null

# Write the new key
echo "$NEW_KEY" | sudo tee /etc/krill/credentials/api_key > /dev/null

# Set correct permissions
sudo chmod 0400 /etc/krill/credentials/api_key
sudo chown krill:krill /etc/krill/credentials/api_key

# Restart the Krill server to load the new key
sudo systemctl restart krill

echo "New API key: $NEW_KEY"

Setting a Custom API Key

If you want to use a specific API key (e.g., for integration with external systems):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# SSH into your Krill server
ssh user@your-krill-server

# Set your custom key
CUSTOM_KEY="your-custom-api-key-here"

# Securely overwrite the old key
sudo dd if=/dev/urandom of=/etc/krill/credentials/api_key bs=64 count=1 2>/dev/null

# Write the custom key
echo "$CUSTOM_KEY" | sudo tee /etc/krill/credentials/api_key > /dev/null

# Set correct permissions
sudo chmod 0400 /etc/krill/credentials/api_key
sudo chown krill:krill /etc/krill/credentials/api_key

# Restart the Krill server
sudo systemctl restart krill

⚠️ Warning: Custom API keys should be at least 32 characters long and contain a mix of letters, numbers, and special characters for security.

Restoring Correct Permissions

If permissions become corrupted, restore them with:

1
2
3
4
5
6
7
8
9
10
11
12
13
# SSH into your Krill server
ssh user@your-krill-server

# Fix directory permissions
sudo chmod 0700 /etc/krill/credentials
sudo chown krill:krill /etc/krill/credentials

# Fix API key file permissions
sudo chmod 0400 /etc/krill/credentials/api_key
sudo chown krill:krill /etc/krill/credentials/api_key

# Verify permissions
ls -la /etc/krill/credentials/

Expected output:

1
2
3
drwx------ 2 krill krill 4096 Jan 12 12:00 .
drwxr-x--- 4 krill krill 4096 Jan 12 12:00 ..
-r-------- 1 krill krill   37 Jan 12 12:00 api_key

Client Configuration

Using the API Key in Requests

Clients must include the API key in the X-API-Key header for all authenticated requests:

1
2
# Example curl request with API key
curl -H "X-API-Key: your-api-key-here" https://krill-server:8442/api/nodes

Protected Endpoints

The following endpoints require API key authentication:

  • All /api/* endpoints
  • Node management endpoints
  • Configuration endpoints

Unprotected Endpoints

Some endpoints remain accessible without authentication:

  • /trust - Certificate download for TLS trust establishment
  • /health - Health check endpoint

Reinstallation Behavior

When the Krill package is reinstalled or upgraded:

  • Reinstall: A new API key is generated, replacing the old one
  • Upgrade: The existing API key is preserved
  • Purge: The API key is securely deleted (overwritten with random data before removal)

Security Considerations

Best Practices

  1. Never share API keys over insecure channels
  2. Rotate keys periodically if you suspect compromise
  3. Use HTTPS to prevent key interception (Krill enforces TLS by default)
  4. Monitor access logs for unauthorized access attempts
  5. Backup your API key securely if you need to restore it later

Secure Deletion

When the Krill package is purged, the API key is securely deleted:

1
2
3
# The postrm script performs:
dd if=/dev/urandom of=/etc/krill/credentials/api_key bs=64 count=1 2>/dev/null
rm -f /etc/krill/credentials/api_key

This ensures the key cannot be recovered from disk after removal.

Troubleshooting

401 Unauthorized Errors

If you receive 401 errors when making API requests:

  1. Verify the API key is correct:
    1
    
    sudo cat /etc/krill/credentials/api_key
    
  2. Check the header format - ensure you’re using X-API-Key:
    1
    2
    
    curl -v -H "X-API-Key: $(sudo cat /etc/krill/credentials/api_key)" \
      https://your-server:8442/api/nodes
    
  3. Verify the Krill service is running:
    1
    
    sudo systemctl status krill
    
  4. Check the server logs:
    1
    
    sudo journalctl -u krill -f
    

Permission Denied Errors

If the Krill server can’t read the API key:

  1. Check file permissions:
    1
    
    ls -la /etc/krill/credentials/
    
  2. Verify ownership:
    1
    
    stat /etc/krill/credentials/api_key
    
  3. Fix permissions if needed:
    1
    2
    
    sudo chown krill:krill /etc/krill/credentials/api_key
    sudo chmod 0400 /etc/krill/credentials/api_key
    

API Key File Missing

If the API key file doesn’t exist:

1
2
3
4
5
6
7
8
9
10
11
# Regenerate it manually
sudo mkdir -p /etc/krill/credentials
sudo chmod 0700 /etc/krill/credentials
sudo chown krill:krill /etc/krill/credentials

NEW_KEY=$(cat /proc/sys/kernel/random/uuid)
echo "$NEW_KEY" | sudo tee /etc/krill/credentials/api_key > /dev/null
sudo chmod 0400 /etc/krill/credentials/api_key
sudo chown krill:krill /etc/krill/credentials/api_key

sudo systemctl restart krill

See Also

This post is licensed under CC BY 4.0 by the author.