Hashicorp Vault docker installation and client testing

Vault Server Installation

Create one folder with 3 subfolders

mkdir -p vault/{config,file,logs}

Create vault configuration file

Create vault/config/vault.json

{
  "backend": {
    "file": {
      "path": "/vault/file"
    }
  },
  "listener": {
    "tcp":{
      "address": "0.0.0.0:8200",
      "tls_disable": 1
    }
  },
  "ui": true
}

Create docker-compose.yml

Create file vault/docker-compose.yml

version: '3.7'
services:
  vault:
    image: vault:latest
    container_name: vault
    ports:
      - "8200:8200"
    restart: unless-stopped
    volumes:
      -  ./logs:/vault/logs
      -  ./file:/vault/file
      -  ./config:/vault/config
    cap_add:
      - IPC_LOCK
    entrypoint: vault server -config=/vault/config/vault.json

Create container

Run docker-compose command in vault folder

cd vault
docker-compose up -d

Access WebUI

Access http://localhost:8200/ from browser

• Select 5 as Key shares, and 3 as Key threshold, and Initialize
• Download keys into a Json file
• Use 3 keys to unseal vault
• Use root token to login

Client installation

Ubuntu x86

• Add the HashiCorp GPG key

# curl -fsSL https://apt.releases.hashicorp.com/gpg | apt-key add -

• Add the official HashiCorp Linux repository

# apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"

• Install vault

# apt-get install vault

• Verify

# vault

Connect to vault

• Set environment

$ export VAULT_ADDR='http://127.0.0.1:8200'
$ export VAULT_TOKEN="<token>"

• Check status

$ vault status
...
Sealed          false
...

Secrets operations

Subcommand	kv v1	kv v2	Description
delete	x	x	Delete versions of secrets stored in K/V
destroy	x	Permanently remove one or more versions of secrets
enable-versioning	x	Turns on versioning for an existing K/V v1 store
get	x	x	Retrieve data
list	x	x	List data or secrets
metadata	x	Interact with Vault\'s Key-Value storage
patch	x	Update secrets without overwriting existing secrets
put	x	x	Sets or update secrets (this replaces existing secrets)
rollback	x	Rolls back to a previous version of secrets
undelete	x	Restore the deleted version of secrets

Example:

vault-getting-started:~# vault login root
Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.

Key                  Value
---                  -----
token                root
token_accessor       rSn3h08ikdez4zch5ghr4wYY
token_duration       ∞
token_renewable      false
token_policies       ["root"]
identity_policies    []
policies             ["root"]
vault-getting-started:~# vault kv put secret/hello foo=world
Key              Value
---              -----
created_time     2021-11-25T06:15:45.332182013Z
deletion_time    n/a
destroyed        false
version          1
vault-getting-started:~# vault kv put secret/hello foo=world excited=yes
Key              Value
---              -----
created_time     2021-11-25T06:15:48.808651794Z
deletion_time    n/a
destroyed        false
version          2
vault-getting-started:~# vault kv get secret/hello
====== Metadata ======
Key              Value
---              -----
created_time     2021-11-25T06:15:48.808651794Z
deletion_time    n/a
destroyed        false
version          2

===== Data =====
Key        Value
---        -----
excited    yes
foo        world
vault-getting-started:~# vault kv get -field=excited secret/hello
yes
vault-getting-started:~# vault kv get -format=json secret/hello | jq -r .data.data.excited
yes
vault-getting-started:~# vault kv delete secret/hello
Success! Data deleted (if it existed) at: secret/hello
vault-getting-started:~#

Secret Engine

The driver to save secret in different way, type of secret.

List

Every path has it's own secret type

$ vault secrets list

Path          Type         Accessor              Description
----          ----         --------              -----------
cubbyhole/    cubbyhole    cubbyhole_78189996    per-token private secret storage
identity/     identity     identity_ac07951e     identity store
kv/           kv           kv_15087625           n/a
secret/       kv           kv_4b990c45           key/value secret storage
sys/          system       system_adff0898       system endpoints used for control, policy and debugging

Enable

Set one path to specific secret type

$ vault secrets enable -path=kv kv

Success! Enabled the kv secrets engine at: kv/

or

$ vault secrets enable kv

Create secret

$ vault kv put kv/hello target=world
Success! Data written to: kv/hello

Get secret

$ vault kv get kv/hello

===== Data =====
Key       Value
---       -----
target    world

Delete secret

$ vault kv delete kv/hello
Success! Data deleted (if it existed) at: kv/hello

List

$ vault kv list kv/

Keys
----
hello

Disable

$ vault secrets disable kv/

Success! Disabled the secrets engine (if it existed) at: kv/

Dynamic Secrets

When using secret engine such as aws engine.

$ vault secrets enable -path=aws aws

Success! Enabled the aws secrets engine at: aws/

More Info: Dynamic Secrets

Authentication

Token

• Create token

$ vault token create
Key                  Value
---                  -----
token                s.iyNUhq8Ov4hIAx6snw5mB2nL
token_accessor       maMfHsZfwLB6fi18Zenj3qh6
token_duration       ∞
token_renewable      false
token_policies       ["root"]
identity_policies    []
policies             ["root"]

• Login

$ vault login s.iyNUhq8Ov4hIAx6snw5mB2nL

Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.

Key                  Value
---                  -----
token                s.iyNUhq8Ov4hIAx6snw5mB2nL
token_accessor       maMfHsZfwLB6fi18Zenj3qh6
token_duration       ∞
token_renewable      false
token_policies       ["root"]
identity_policies    []
policies             ["root"]

• Revoke

$ vault token revoke s.iyNUhq8Ov4hIAx6snw5mB2nL

Success! Revoked token (if it existed)

GitHub

• Enable

$ vault auth enable github

Success! Enabled github auth method at: github/

• Set organization

$ vault write auth/github/config organization=hashicorp

Success! Data written to: auth/github/config

• Configure the GitHub engineering team authentication to be granted the default and applications policies

$ vault write auth/github/map/teams/engineering value=default,applications

Success! Data written to: auth/github/map/teams/engineering

• List

$ vault auth list

Path       Type      Description
----       ----      -----------
github/    github    n/a
token/     token     token based credentials

• Set login method

$ vault login -method=github

GitHub Personal Access Token (will be hidden):
Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.

Key                    Value
---                    -----
token                  s.DNtKCjVQ1TxAzgMqtDuwjjC2
token_accessor         e7zLJuPg2tLpav66ZSu5AyDC
token_duration         768h
token_renewable        true
token_policies         [default applications]
token_meta_org         hashicorp
token_meta_username    my-user

• Login

$ vault login root

• Revoke all tokens generated the github auth method

$ vault token revoke -mode path auth/github

• Disable the github auth method

$ vault auth disable github

Success! Disabled the auth method (if it existed) at: github/

Policy

• Policy for token

The policy path secret/data/* is related to all secret path secret/*.
The policy path secret/data/foo is related to secret path secret/foo.

• Policy for approle

The policy path secret/approle/* is related to role_id + secret_id authentication.

Policy for token

• Create

$ vault policy write my-policy - << EOF
# Dev servers have version 2 of KV secrets engine mounted by default, so will
# need these paths to grant permissions:
path "secret/data/*" {
  capabilities = ["create", "update"]
}

path "secret/data/foo" {
  capabilities = ["read"]
}
EOF

• List

$ vault policy list

default
my-policy
root

• Show

$ vault policy read my-policy

# Dev servers have version 2 of KV secrets engine mounted by default, so will
# need these paths to grant permissions:
path "secret/data/*" {
  capabilities = ["create", "update"]
}

path "secret/data/foo" {
  capabilities = ["read"]
}

• Create token

$ export VAULT_TOKEN="$(vault token create -field token -policy=my-policy)"

• Check policy

$ vault token lookup | grep policies
policies            [default my-policy]

• Write success

$ vault kv put secret/creds password="my-long-password"

Key              Value
---              -----
created_time     2018-05-22T18:05:42.537496856Z
deletion_time    n/a
destroyed        false
version          1

• Write failed

$ vault kv put secret/foo robot=beepboop

Error writing data to secret/data/foo: Error making API request.

URL: PUT http://localhost:8200/v1/secret/data/foo
Code: 403. Errors:

* 1 error occurred:
  * permission denied

Policy for approle

• Enable

$ vault auth enable approle
Success! Enabled approle auth method at: approle/

• Create my-role link to my-policy

$ vault write auth/approle/role/my-role \
    secret_id_ttl=10m \
    token_num_uses=10 \
    token_ttl=20m \
    token_max_ttl=30m \
    secret_id_num_uses=40 \
    token_policies=my-policy
Success! Data written to: auth/approle/role/my-role

• Create role_id

$ export ROLE_ID="$(vault read -field=role_id auth/approle/role/my-role/role-id)"

• Create secret_id

$ export SECRET_ID="$(vault write -f -field=secret_id auth/approle/role/my-role/secret-id)"

• Login

$ vault write auth/approle/login role_id="$ROLE_ID" secret_id="$SECRET_ID"
Key                     Value
---                     -----
token                   s.Sh9h1wZ9ycATeSaASoOQvovr
token_accessor          xCgUIu6WWLM9opkEkAiNLsRc
token_duration          20m
token_renewable         true
token_policies          ["default" "my-policy"]
identity_policies       []
policies                ["default" "my-policy"]
token_meta_role_name    my-role

References

vault-docker
Install Vault

Troubleshooting Hashicorp Vault SSH Certificate Login

If can not login via SSH, normally can use -vvv as SSH option in SSH client command to verify, but it could be very long to read.

Another way is use systemctl status sshd command at server side to check the error.

For example, the output of systemctl status sshd got following lines,

Dec 12 00:40:37 example-host systemd[1]: Started OpenBSD Secure Shell server.
Dec 12 00:40:54 example-host sshd[22712]: error: Certificate invalid: expired
Dec 12 00:40:54 example-host sshd[22712]: Connection closed by authenticating user ubuntu 101.78.78.154 port 53369 [preauth]
Dec 12 00:41:12 example-host sshd[22716]: error: Certificate invalid: name is not a listed principal
Dec 12 00:41:12 example-host sshd[22716]: Connection closed by authenticating user ubuntu 101.78.78.154 port 53372 [preauth]

The first error shows vault signed certificate at client side had expired, need to rerun vault command to regenerate signed certificate.

The second error shows the user name was used in client is not listed in vault signed certificate, so need to use correct user name or configure a new role in the vault.