Configuring SPIRE

How to configure SPIRE for your environment

To customize the behavior of the SPIRE Server and SPIRE Agent to meet your application’s needs you edit configuration files for the server and agent.

How to configure SPIRE

The SPIRE Server and Agent are configured in a file called server.conf and agent.conf respectively.

By default the Server expects the configuration file to reside at conf/server/server.conf, however the Server can be configured to use a configuration file in a different location with the --config flag. See the SPIRE Server reference for more information.

Similarly, the Agent expects this file to reside at conf/agent/agent.conf, however the Server can be configured to use a configuration file in a different location with the --config flag. See the SPIRE Agent reference for more information.

The configuration file is loaded once when the Server or Agent is started. If the configuration file for either is modified, the Server or Agent must be restarted for the configuration to take effect.

When running SPIRE in Kubernetes, it is common to store the configuration file in a ConfigMap object that is then mounted as a file into the container running the Agent or Server process.

The SPIRE Agent supports either HCL or JSON as the syntax for structuring configuration files. The examples below will assume HCL.

Configuring a trust domain

This configuration applies to the SPIRE Server and SPIRE Agent

The trust domain corresponds to the trust root of a SPIFFE identity provider. A trust domain could represent an individual, organization, environment or department running their own independent SPIFFE infrastructure. All workloads identified in the same trust domain are issued identity documents that can be verified against the root keys of the trust domain.

Each SPIRE server is associated with a single trust domain that must be unique within that organization. The trust domain takes the same form as a DNS name (for example, prod.acme.com), however it does not need to correspond to any DNS infrastructure.

The trust domain is configured in the SPIRE Server before it is first started. It is configured through the trust_domain parameter in the server stanza in the configuration file. For example, if the trust domain of the server should be configured to prod.acme.com then it would be set as:

trust_domain = "prod.acme.com"

Similarly, the Agent must be configured to issue identities to the same trust domain by configuring in the trust_domain parameter in the agent stanza of the Agent configuration file.

A SPIRE Server and Agent can only issue identities to a single trust domain, and the trust domain configured by an agent must match that of the server it is connecting to.

Configure the Port on which the Server Listens to Agents

This configuration applies to the SPIRE Server

By default, the SPIRE Server listens on port 8081 for incoming connections from SPIRE Agents; to choose a different value, edit the bind_port parameter in the server.conf file. For example, to change the listening port to 9090:

bind_port = "9090"

If this configuration is changed from the default on the server, then the configuration of the serving port must also be changed on the agents.

Configuring node attestation

This configuration applies to the SPIRE Server and SPIRE Agent

A SPIFFE Server identifies and attests Agents through the process of node attestation and resolution (read more about this in SPIRE Concepts). This is accomplished through Node Attestor and Node Resolver plugins, which you configure and enable in the server. 

Your choice of node attestation method determines which node-attestor plugins you configure SPIRE to use in Server Plugins and Agent Plugins sections of the SPIRE configuration files. You must configure at least one node attestor on the server and only one node attestor on each Agent.

Attestation of nodes running Kubernetes

To issue identities to workloads running in a Kubernetes cluster, it is necessary to deploy a SPIRE Agent to each node in that cluster that is running a workload (read more on how to install SPIRE Agents on Kubernetes).

Service Account Tokens can be validated using the Kubernetes Token Review API. Because of this, the SPIRE Server does not itself need to be running on Kubernetes, and a single SPIRE Server may support agents running on multiple Kubernetes clusters with PSAT attestation enabled.

Projected Service Account Tokens

At the time of this writing,  projected service accounts are a relatively new feature in Kubernetes and not all deployments support them. Your Kubernetes platform documentation will tell you whether this feature is available. If your Kubernetes deployment does not support Projected Service Account Tokens, you should enable Service Account Tokens instead.

Node attestation using Kubernetes Projected Service Account Tokens (PSATs) allows a SPIRE Server to verify the identity of a SPIRE Agent running on a Kubernetes Cluster. Projected Service Account Tokens provide additional security guarantees over traditional Kubernetes Service Account Tokens and when supported by a Kubernetes cluster, PSAT is the recommended attestation strategy.

To use PSAT Node Attestation, configure enable the PSAT Node Attestor plugin on the SPIRE Server and SPIRE Agent.

Service Account Tokens

In cases where workloads are running on Kubernetes but the Projected Service Account Token feature is not available for the cluster they are running on, SPIRE can establish trust between the Server and Agent using Service Account Tokens. Unlike when using Projected Service Account Tokens, this method requires that the SPIRE Server and SPIRE Agent both be deployed on the same Kubernetes cluster.

Because the service account token does not contain claims that could be used to strongly identify the node/daemonset/pod running the agent, any container running in a whitelisted service account can masquerade as an agent. For this reason it is strongly recommended that agents run under a dedicated service account when using this attestation method.

To use SAT Node Attestation, configure and enable the SAT Node Attestor plugin on the SPIRE Server and SPIRE Agent.

Attestation of nodes running Linux

SPIRE is able to attest the identities of workloads running physical or virtual machines (nodes) running Linux. As part of the attestation process it is necessary for the SPIRE Server to establish trust to a SPIRE Agent running on a Linux node. SPIRE supports a variety of Node Attestors depending on where the node is running that allow the use of different selectors when creating registration entries to identify specific workloads.

Join Token

A join token is a simple method of attesting a server to an agent using a single-use token that is generated on the server and supplied to the agent when the agent is started. It works on any node running Linux.

The SPIRE server can be configured to support join token attestation by enabling the built-in join-token NodeAttestor plugin, via the following stanza in the server.conf configuration file:

NodeAttestor "join_token" {
    plugin_data {
    }
} 

Once join token node attestation has been configured, a join token can be generated on the server using the spire-server token generate command. Optionally you can associate a particular SPIFFE ID with the Join Token with the -spiffeID flag. Read more about using this command.

When starting a SPIRE Agent for the first time with Join Token attestation enabled, the agent can be started with the spire-agent run command, and specifying the join token generated by the server using the -joinToken flag. Read more about this command.

The server will validate the join token and issue the Agent an SVID, and the SVID will be rotated automatically as long as it maintains a connection to the Server. On subsequent starts the Agent will use that SVID to authenticate to the server unless it has expired and not renewed.

To use Join Token Node Attestation, configure and enable the join token Node Attestor plugin on the SPIRE Server and SPIRE Agent.

To disable join token attestation on the server, comment out or delete this stanza from the configuration file before starting it.

X.509 Certificate

In many cases, particularly where nodes are provisioned manually (such as is in a datacenter) a node may be able to be identified by validating an existing X.509 leaf certificate that has previously been installed on the node, and uniquely identifies it.

Typically these leaf certificates are generated from a single common key and certificate (for the purposes of this guide, they will be called the root certificate bundle). The Server must be configured with the root key and any intermediate certificates, to be able to validate the leaf certificate presented by a particular machine. Only when a certificate that can be validated by the certificate chain to the server is found will node attestation be successful and workloads on that node be able to be issued SPIFFE IDs.

In addition attestor exposes the selector subject:cn which will match any certificate that is both (a) valid, as described above, and (b) whose common name (CN) matches that described in the selector.

To use X.509 Certificate Node Attestation, configure and enable the x509pop Node Attestor plugin on the SPIRE Server and SPIRE Agent.

SSH Certificate

In some environments each node is automatically provisioned with a valid and unique SSH certificate that identifies the node. SPIRE can use this certificate to bootstrap it’s identity.

Nodes attested via this method are automatically given a SPIFFE ID in the form of:

spiffe://<trust-domain>/spire/agent/sshpop/<fingerprint>

Where is a hash of the certificate itself. This SPIFFE ID can then be used as the basis of other workload registration entries.

To use SSH Certificate Node Attestation, configure and enable the sshpop Node Attestor plugin on the SPIRE Server and SPIRE Agent.

Attestation for Linux nodes on a Cloud Provider

Many cloud providers offer privaliged APIs that allow a process running on a particular node hosted by that provider to be able to prove which node it is running on. SPIRE can be configured to leverage these APIs for node attestation. This is particularly convenient for automation because an Agent starting for the first time on a new instance can automatically attest its identity to the SPIRE server without pre-existing certificates or join tokens being issued to it.

Google Compute Engine Instances

Google Compute Engine (GCE) node attestation and resolution allows a SPIRE Server to identify and authenticate a SPIRE Agent running on a GCP GCE instance automatically. In brief, it is accomplished through the following:

  1. The SPIRE Agent gcp_iit Node Attestor plugin retrieves a GCP instance’s instance identity token, and identifies itself to the SPIRE Server gcp_iit Node Attestor plugin.
  2. The SPIRE Server gcp_iit Node Attestor plugin calls a GCP API to verify the validity of the token, if the use_instance_metadata configuration value is set to true.
  3. Once verification takes place, the SPIRE Agent is considered attested, and issued its own SPIFFE ID
  4. Finally, SPIRE issues SVIDs to workloads on the nodes if they match a registration entry. The registration entry may include selectors exposed by the Node Attestor or Resolver, or have the SPIFFE ID of the SPIRE Agent as a parent.

To use GCP IIT Node Attestation, configure and enable the gcp_iit Node Attestor plugin on the SPIRE Server and SPIRE Agent.

Amazon EC2 Instances

EC2 node attestation and resolution allows a SPIRE Server to identify and authenticate a SPIRE Agent running on an AWS EC2 Instance automatically. In brief, it is accomplished through the following:

  1. The SPIRE Agent aws_iid Node Attestor plugin retrieves an AWS instance’s instance identity document, and identifies itself to the SPIRE Server aws_iid Node Attestor plugin.
  2. The SPIRE Server aws_iid Node Attestor plugin calls an AWS API to verify the validity of the document, using an AWS IAM role with limited permissions. 
  3. If the aws_iid Node Resolver plugin is configured, then SPIRE will use the verified identity of the node to look up additional information about the node. This metadata can be used as a selector in a registration entry.
  4. Once verification takes place, the SPIRE Agent is considered attested, and issued its own SPIFFE ID
  5. Finally, SPIRE issues SVIDs to workloads on the nodes if they match a registration entry. The registration entry may include selectors exposed by the Node Attestor or Resolver, or have the SPIFFE ID of the SPIRE Agent as a parent.

For more information on configuring AWS EC2 Node Attestors or Resolver plugins, refer to the corresponding SPIRE documentation for the AWS SPIRE Server Node Attestor and SPIRE Server Node Resolver on the SPIRE Server, and the SPIRE Agent Node Attestor on the agent.

Azure Virtual Machines

Azure MSI node attestation and resolution allows a SPIRE Server to identify and authenticate a SPIRE Agent running on an Azure VM automatically. SPIRE uses MSI tokens in order to attest the agent. The MSI tokens must be scoped to mitigate abuse if intercepted. In brief, it is accomplished through the following:

  1. The SPIRE Agent azure_msi Node Attestor plugin retrieves an an Azure VM’s MSI token, and identifies itself to the SPIRE Server azure_msi Node Attestor plugin.
  2. The SPIRE Server azure_msi Node Attestor plugin retrieves the JSON Web Key Set (JWKS) document from Azure–via an API call and uses JWKS information to validate the MSI token. 
  3. The SPIRE Server azure_msi Node Resolver plugin interacts with Azure to obtain information about the agent VM–such as subscription ID, VM name, network security group, virtual network, and virtual network subnet–to build up a set of attributes about the agent VM that can then be used as node selectors for the Azure node set.
  4. Once verification takes place, the SPIRE Agent is considered attested, and issued its own SPIFFE ID
  5. Finally, SPIRE issues SVIDs to workloads on the nodes if they match a registration entry. The registration entry may include selectors exposed by the Node Attestor or Resolver, or have the SPIFFE ID of the SPIRE Agent as a parent.

The default resource–assigned by the agent plugin–is scoped relatively widely; it uses the Azure Resource Manager(https://management.azure.com endpoint)’s resource id. For security reasons, consider using a custom resource id, to scope more narrowly. 

If you configure a custom resource ID in the agent configuration file, you must specify custom resource IDs for each tenant, in the NodeAttestor stanza of the server.conf configuration file. (See Configure the Server.)

For more information on configuring Azure MSI Node Attestors or Resolver plugins, refer to the corresponding SPIRE documentation for the AWS SPIRE Server Node Attestor and SPIRE Server Node Resolver on the SPIRE Server, and the SPIRE Agent Node Attestor on the agent.

Configuring workload attestation

This configuration applies to the SPIRE Agent

Whereas Node Attestation concerns how a SPIRE Server identifies a SPIRE Agent on a particular physical or virtual machine, Workload Attestation concerns how a SPIRE Agent identifies a specific process. The two are typically used in concert to identify specific workloads.

As with Node Attestation, Workload Attestation is accomplished through enabling the relevant plugins. Different plugins make different selectors available which can be used in registration entries to identify specific workloads. Unlike with Node Attestation, where only a single strategy can be used for any given workload at a time, Workload Attestation can involve multiple strategies for a single workload. A single workload may, for example, be required to run under a given Unix group, and be started from a specific Docker image.

Workload Attestation for workloads scheduled by Kubernetes

When workloads are running in Kubernetes, it is valuable to be able to describe them in terms of Kubernetes constructs such as the a namespace or service account or abel associated with the pod the workload is running under.

The Kubernetes Workload Attestor plugin works by interrogating the local Kubelet to retrieve kubernetes-specific metadata about particular process when it calls the Workload API, and uses that to identify workloads whose registration entries match those values.

For more information, including details of the exposed selectors, refer to the corresponding SPIRE documentation for the Kubernetes Workload Attestor plugin.

Workload Attestation for Docker containers

When workloads are running in a Docker container, it can be helpful to be able to describe them in terms of attributes of that container, such as the Docker image the container was started from, or the value of a particular envrionment variable.

The Docker Workload Attestor plugin works by interrogating to the local Docker deamon to retrieve Docker-specific metadata about a particular process when it calls the Workload API.

For more information, including details of the exposed selectors, refer to the corresponding SPIRE documentation for the Docker Workload Attestor plugin.

Workload Attestation for Unix processes

When workloads are running on Unix, it can be helpful to be able to describe them in terms of how that process is being managed by Unix, such as the name of the unix group it is running under.

The Unix Workload Attestor works by determining kernel metadata from the workload calling the Workload API by examining the caller of the Unix domain socket.

For more information, including details of the exposed selectors, refer to the corresponding SPIRE documentation for the Unix Workload Attestor plugin.

Configuring how to store server data

This configuration applies to the SPIRE Server

The data-store is where SPIRE Server persists dynamic configuration such as registration entries and identity mapping policies that are retrieved from the SPIRE Server. By default, SPIRE bundles SQLite and sets it as the default for storage of server data. SPIRE also supports other compatible data-stores. For production purposes, you should carefully consider which database to use, particularly when deploying SPIRE in a High Availability configuration.

The SPIRE Server can be configured to utilize different SQL-compatible storage backends by configuring the default SQL data-store plugin as described below. A complete reference for how this block is configured can be found in the SPIRE documentation.

Alternatively, SPIRE can be configured to use non-SQL compatible storage backends through third party datastore plugins. The guide on Extending SPIRE covers this in more detail.

Configure SQLlite as a SPIRE data-store

By default, the SPIRE Server creates and uses a local SQLlite database for backing up and storing configuration data. While convenient for testing this is generally not recommended for production deployments as it is difficult to share a SQLlite data-store across multiple machines, which can complicate backups, HA deployments and upgrades.

To configure the server to use a SQLlite database, enable the stanza in the configuration file that looks like this:

    DataStore "sql" {
        plugin_data {
            database_type = "sqlite3"
            connection_string = "/opt/spire/data/server/datastore.sqlite3"
        }
    }

There should be no other (un-commented) Datastore stanzas in the configuration file.

The database will be created in a path specified in the connection string, which is relative to the path of the spire-server binary.

Configure MySQL as a Datastore

In production it is recommended to use a dedicated database to back up and store configuration data. While installing and configuring a MySQL database is outside the scope of this guide, it is worth noting that SPIRE Server requires:

A dedicated database on the MySQL server for SPIRE Server configuration.

A MySQL user that has the ability to connect to any EC2 instance running the SPIRE Server and that can both INSERT and DROP tables, columns and rows from that database.

To configure the SPIRE Server to use a MySQL database, enable the stanza in the configuration file that looks like this:

    DataStore "sql" {
        plugin_data {
            database_type = "mysql"
            connection_string = "username:password@tcp(localhost:3306)/dbname?parseTime=true"
        }
    }

In the connection string above, substitute the following:

  • username for the username of the MySQL user that should be used to access the database
  • password for the password of the MySQL user
  • localhost:3306 for the IP address or hostname of the MySQL server, and port number
  • dbname for the name of the database 

Configure Postgres as a Datastore

In production it is recommended to use dedicated database to back up and store configuration data. While installing and configuring a Postgres database is outside the scope of this guide, it is worth noting that SPIRE Server requires:

  • A dedicated database on the Postgres server for SPIRE Server configuration
  • A Postgres user that has the ability to connect to any instance running the SPIRE Server and that can both INSERT and DROP tables, columns and rows from that database

To configure the SPIRE Server to use a Postgres database, enable the following stanza in the server configuration file:

    DataStore "sql" {
        plugin_data {
            database_type = "postgres"
            connection_string = "dbname=[database_name] user=[username]
                                 password=[password] host=[hostname] port=[port]"
        }
    }

The connection_string value is in a key=value format, however you may also use a connection URI (see 34.1.1. Connection Strings in the Postgres documentation for supported connection string formats).

The following list summarizes the connection string values you set:

  • [database-name] - the name of the database 
  • [username] - the username of the Postgres user accessing the database
  • [password] - the user’s password
  • [hostname] - the IP address or hostname of the Postgres server
  • [port] - port number of the Postgres server

Configuring how generated keys are stored on the Agent and Server

_This configuration applies to the SPIRE Server and SPIRE Agent

Both the SPIRE Agent and SPIRE Server generate private keys and certificates during normal operation. It is important to maintain the integrity of these keys and certificates to ensure the integrity of the issued SPIFFE identities is maintained.

Currently SPIRE supports two key management strategies on both the Agent and Server.

  • Store in-memory. In this strategy keys and certificates are only stored in-memory. This means that if the Server or Agent crashes or is otherwise re-started, the keys must be re-generated. In the case of the SPIRE Agent this typically requires the agent to re-attest to the Server upon restart. This strategy can be managed by enabling and configuring the memory key manager plugin for the SPIRE Server and/or SPIRE Agent.

  • Store on disk. In this strategy, keys and certificates are stored in a specified file on disk. An advantage of this approach is they survive a restart of the SPIRE Server or Agent. A disadvantage is that since they keys are stored in files on disk, additional precautions must be taken to prevent a malicious process from reading those files. This strategy can be managed by enabling and configuring the memory key manager plugin for the SPIRE Server and/or SPIRE Agent.

Alternatively, SPIRE can be configured to integrating a custom backend such as a secret store through third party key manager plugins. The guide on Extending SPIRE covers this in more detail.

Configuring which trust root / “upstream CA” your application will use

This configuration applies to the SPIRE Server

Each SPIRE Server uses a specific root signing key that is used to accomplish several important actions:

  • To establish trust by a SPIRE Agent to the SPIRE Server, as the agent holds a certificate that has been signed by that key (note though that trust from the server to the agent is established through attestation).
  • To generate X.509 SVIDs that are issued to workloads
  • To generate SPIFFE trust bundles (used to establish trust with other SPIRE Servers)
This signing key should be considered extremely sensitive as to obtain it would allow a malicious actor to impersonate the SPIRE Server and to issue identities on its behalf.

To help ensure the integrity of the signing key a SPIRE Server may either sign material itself using a signing key stored on disk, or delegate singing to an independent Certificate Authority (CA), such as the AWS Secrets Manager. This behavior is configured through the UpstreamCA section in the server.conf file.

For a complete server configuration reference, see the SPIRE Server Configuration Reference.

Configure an On-disk Signing Key

The SPIRE Server can be configured to load CA credentials from disk, using them to generate intermediate signing certificates for the server’s signing authority.

The SPIRE Server comes with a “dummy” key and certificate that can be used to simplify testing, however since this same key is distributed to all SPIRE users it should not be used for anything other than testing purposes. Instead an on-disk signing key should be generated.

If the openssl tool is installed, a valid root key and certificate can be generated using a command similar to the following:

sudo openssl req \
       -subj "/C=/ST=/L=/O=/CN=acme.com" \
       -newkey rsa:2048 -nodes -keyout /opt/spire/conf/root.key \
       -x509 -days 365 -out /opt/spire/conf/root.crt

This strategy can be managed by enabling and configuring the disk UpstreamCA plugin for the SPIRE Server.

Configure AWS Secrets Manager

The SPIRE Server can be configured to load CA credentials from Amazon Web Services Secrets Manager, using them to generate intermediate signing certificates for the server’s signing authority.

This strategy can be managed by enabling and configuring the awssecret UpstreamCA plugin for the SPIRE Server.

Configure AWS Certificate Manager

The SPIRE Server can be configured to load CA credentials from Amazon Web Services Certificate Manager Private Certificate Authority (PCA) them to generate intermediate signing certificates for the server’s signing authority.

This strategy can be managed by enabling and configuring the aws_pca UpstreamCA plugin for the SPIRE Server.

Configure another SPIRE installation

The SPIRE Server can be configured to load CA credentials from the Workload API of another SPIFFE implementation such as SPIRE. This enables a technique called “Nested SPIRE” that, as a compliment to HA deployments, allows independent SPIRE Servers to issue identities against a single trust domain.

A full treatment for Nested SPIRE is beyond the scope of this guide. However this strategy can be managed by enabling and configuring the spire UpstreamCA plugin for the SPIRE Server.

Export Metrics for Monitoring

This configuration applies to the SPIRE Server and SPIRE Agent

SPIRE allows you to export both Server and Agent metrics for the purposes of telemetry/monitoring. To configure which metrics collectors you export metrics data to, edit the “Telemetry configuration” section of the server and agent configuration files. SPIRE currently supports exporting of metrics to Prometheus, Statsd, and Datadog (DogStatsd format). You may configure multiple collectors at the same time. Statsd and DogStatsd both support multiple declarations in the event that you want to send metrics to more than one collector.

If you want to use Amazon Cloud Watch for metrics collection, review this document on retrieving custom metrics with the CloudWatch agent and StatsD.

Below is an example of a configuration block that exports telemetry to Prometheus, Datadog and StatsD:

telemetry {
        Prometheus {
                port = 9988
        }

        DogStatsd {
                address = "localhost:8125"
        }

        Statsd {
               address = "collector.example.org:8125"
        }
}

For more information, refer to the relevant section of the SPIRE Server reference guide, or SPIRE Agent reference guide.

Logging

This configuration applies to the SPIRE Server and SPIRE Agent

You can set the log file location and the level of logging for the SPIRE Server and SPIRE Agent in their respective configuration files. Edit the log_file value to set the log file location and the log_level value to set the level of logging. This can be one of DEBUG, INFO, WARN or ERROR.

By default, SPIRE logs go to STDOUT. However the SPIRE Agent and Server can be onfigured instead to write logs directly to a file by specifying the path to the file in the log_file attribute.

Where next?

Once you’ve configured your Server and Agents, consider reviewing the guide on Registering Workloads.