Cloud-Native | Luis Cacho

Trigger a scheduled job manually on Kubernetes

Thu, 22 Jun 2023 00:00:00 +0000

# Mockup command
kubectl create job --from=cronjob/<cronjob-name> <job-name> -n <namespace-name>

References:

¹How can I trigger a Kubernetes Scheduled Job manually?

https://stackoverflow.com/questions/40401795/how-can-i-trigger-a-kubernetes-scheduled-job-manually ↩︎

Delete Evicted Pods

Wed, 23 Nov 2022 00:00:00 +0000

oc get pods | grep Evicted | awk '{print $1}' | xargs kubectl delete pod

References:

¹ Gist

https://gist.github.com/ipedrazas/9c622404fb41f2343a0db85b3821275d ↩︎

ArgoCD cli login command

Wed, 17 Aug 2022 00:00:00 +0000

argocd login \
--insecure \
--grpc-web $(oc get route openshift-gitops-server -n openshift-gitops
-o jsonpath='{.spec.host}{"\n"}') \
--username admin \
--password $(oc get secret/openshift-gitops-cluster -n openshift-gitops -o jsonpath='{.data.admin\.password}' | base64 -d)

Understanding the Compliance Operator

Tue, 16 Aug 2022 00:00:00 +0000

Table of Contents

Compliance studies a company’s security processes. It details their security at a moment in time and compares it to a specific set of regulatory requirements. These requirements come in the form of legislation, industry regulations, or standards created from best practices.

Every company can protect its data accordingly if they follow Compliance frameworks and have quality security in place. To have proper protection, companies must understand that Compliance is not the same thing as security. However, security is a big part of Compliance.

Becoming secure and compliant means securing information assets, preventing damage, protecting it, eliminating potential attack vectors and decreasing the system’s attack surface.

Compliance Operator Introduction

The Compliance Operator was released in OpenShift Container Platform v4.6, and allows OpenShift Container Platform administrators describe the desired compliance state of a cluster and provides them with an overview of gaps and ways to remediate them. The Compliance Operator assesses compliance of both the Kubernetes API resources of OpenShift Container Platform, as well as the nodes running the cluster. The Compliance Operator uses OpenSCAP, a NIST-certified tool, to scan and enforce security policies provided by the content.¹

The Compliance Operator is available for Red Hat Enterprise Linux CoreOS (RHCOS) deployments only.

To secure your OpenShift cluster, it is necessary to consider both; platform (Kubernetes/OpenShift API) and host OS (RHCOS) perspectives, because Kubernetes is composed of control plane machines (master) and worker machines (node), then the Kubernetes services such as API Server, etcd, or controller manager run on the control plane to manage the workloads on the worker machines.

Platform and Host Perspective

The basic approach to perform hardening in a RHCOS host is described in the documentation and it is based in the RHEL 8 Security Hardening. For instance, the hardening consists in validate if the file has appropriately restrictive file permissions, if the file ownership is appropriately set, if required systemd services or processes are launched with appropriate arguments or parameters in the configuration, or if the appropriate kernel parameters are set. Then, hardening the control plane is specific to the Kubernetes services and includes the control plane components or the master configuration files. It should validate if the API server has started with restrictive arguments regarding allowing specific admission plug-ins, enabling audit logging, applying etcd server and peer configurations, and restricting RBAC, among others. For clusters that use RHCOS, updating or upgrading are designed to become automatic events from the central control plane, because OpenShift completely controls the systems and services that run on each machine, including the operating system itself through the Machine Config Operator.

Why Compliance Operator?

Why is the Compliance Operator needed to validate the hardening and apply changes in the configuration of the operating system and the platform? The Compliance Operator is defined as follows:

The Compliance Operator lets OpenShift Container Platform administrators describe the required compliance state of a cluster and provides them with an overview of gaps and ways to remediate them. The Compliance Operator assesses compliance of both the Kubernetes API resources of OpenShift Container Platform, as well as the nodes running the cluster. The Compliance Operator uses OpenSCAP, a NIST-certified tool, to scan and enforce security policies provided by the content.¹

In other words, the Compliance Operator checks the host and the platform to detect gaps in compliance by specifying profiles for scan and creates summary reports about security compliance so that you will be able to find if there is any configuration that violates the policy in the cluster. The reports also show which remediations are applied, so you can choose if you want to apply the recommended configuration by hand, step-by-step, or automatically. In short, the whole process goes like this: choosing a profile for scanning, specifying the scan settings, then initiating the scan, and generating the reports.

Compliance Operator Overview

The Compliance Operator leverages OpenSCAP, a NIST-certified tool, to scan and enforce security policies, and the security policies for the compliance checks are derived through SCAP content and built from the community-based ComplianceAsCode/content project. A bundle of security policies, or profiles created by default when the operator is installed and profiles scheduled include NIST 800-53 Moderate (FedRAMP), Australian Cyber Security Centre (ACSC) Essential Eight, CIS OpenShift Benchmark, and others, so far. You can also create or tailor your own profiles so that you can pick the rules you want to run other than the profiles provided by default.

How can I implement Compliance Operator in my cluster?

First of all, you will need to install the Compliance Operator, for that, you can follow the documentation steps

Great, the Operator is installed and functional, now lets check how to create, run and evaluate a compliance scan.

References:

https://docs.openshift.com/container-platform/4.8/security/compliance_operator/compliance-operator-understanding.html ↩︎ ↩︎

Delete OpenShift Operator

Mon, 15 Aug 2022 00:00:00 +0000

# Mockup commands

oc get subscription <operator> -n openshift-operators -o yaml | grep
currentCSV

oc delete subscription <operator> -n openshift-operators

oc delete clusterserviceversion <currentCSV> -n openshift-operators

# Example commands

oc get subscription jaeger -n openshift-operators -o yaml | grep currentCSV

oc delete subscription jaeger -n openshift-operators

oc delete clusterserviceversion jaeger-operator.v1.8.2 -n openshift-operators

Delete OpenShift Project

Mon, 15 Aug 2022 00:00:00 +0000

Issue, can’t delete a namespace/project and is stuck in a “Terminating” status

oc delete project fishbone

oc get projects fishbone
NAME DISPLAY NAME STATUS
fishbone Fishbone Terminating

Solution

Define the namespace/project to delete

export NAMESPACE=fishbone

Cleanup the finalizers section from the namespace definition and define a json file with that

oc get namespace $NAMESPACE -o=json | jq '.spec = {"finalizers":[]}' > $NAMESPACE.json
{
 "apiVersion": "v1",
 "kind": "Namespace",
 "metadata": {
 "annotations":
 "openshift.io/description": "",
 "openshift.io/display-name": "Fishbone",
 "openshift.io/requester": "system:admin",
 "openshift.io/sa.scc.mcs": "s0:c24,c14",
 "openshift.io/sa.scc.supplemental-groups": "1000580000/10000",
 "openshift.io/sa.scc.uid-range": "1000580000/10000"
 },
 "creationTimestamp": "2021-02-20T03:29:59Z",
 "deletionTimestamp": "2021-02-20T03:31:35Z",
 "name": "fishbone",
 "resourceVersion": "270664",
 "selfLink": "/api/v1/namespaces/fishbone",
 "uid": "885c4440-6f05-4e72-a103-28e61705406f"
 },
 "spec": {
 "finalizers": [
 "kubernetes"
 ]
 },
 "status": {
 "conditions": [
 .
 .
 .
 ],
 "phase": "Terminating"
}

Apply the json file from the previous step to delete the finalizers and therefore the deletion of the namespace/project

curl -k -H "authorization: Bearer $(oc whoami --show-token)" -H "Content-Type: application/json" -X PUT --data-binary @$NAMESPACE.json $(oc whoami --show-server)/api/v1/namespaces/$NAMESPACE/finalize

Verify that the namspace/project has been deleted

oc get projects $NAMESPACE
Error from server (NotFound): namespaces "fishbone" not found

References:

Kubernetes Advanced Topics¹
Kubernetes Namespace Stuck²
Red Hat article[^3]

How to Retrieve secrets values

Mon, 15 Aug 2022 00:00:00 +0000

oc get secret my-secret -n my-project --template={{.data.my-secret-key}} | base64 -d

oc get secret my-secret -n my-project -o go-template --template="{{.data.my-secret-key|base64decode}}"

oc extract secrets/my-secret -n my-project

oc get secret my-secret -n my-project -o jsonpath="{.data['tls\.key']}" | base64 -d

How to list the taints on kubernetes nodes

Sun, 14 Aug 2022 00:00:00 +0000

oc get nodes -o json | jq '.items[].spec.taints'
oc get nodes -o json | jq '.items[] | {node_name:.metadata.name,taints:.spec.taints[]?}

oc get nodes -o=custom-columns=NAME:.metadata.name,TAINTS:.spec.taints

get nodes -o=jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.taints}{"\n"}{end}'

References:

¹ StackOverflow

]: https://stackoverflow.com/questions/43379415/how-can-i-list-the-taints-on-kubernetes-nodes ↩︎

How to access an OpenShift 4 Node

Thu, 16 Sep 2021 00:00:00 +0000

In order to review the underline node, you need to follow the next steps:

Get the node name that is having issues or your need to login

oc get nodes

Run the oc debug command into the node

oc debug node/ip-10-0-129-255.us-east-2.compute.internal

Once you are logged in the node, you’ll need to run:

chroot /host

References:

¹ OCP4 Documentation

https://docs.openshift.com/container-platform/4.10/support/gathering-cluster-data.html#support-collecting-network-trace_gathering-cluster-data ↩︎

Get ServiceAccount token for MTC

Fri, 06 Aug 2021 00:00:00 +0000

oc sa get-token migration-operator -n openshift-migration

Command to perform a curl to a PodIP

Thu, 05 Aug 2021 00:00:00 +0000

curl $(oc get pod <pod-name> --template '{{ .status.podIP}}):8080

Approve pending CSRs in OpenShift

Wed, 04 Aug 2021 00:00:00 +0000

for id in $(oc get csr | grep Pending | awk '{print $1}'); do oc adm certificate approve $id; done

Create a Nooba Object Bucket Claim

Wed, 04 Aug 2021 00:00:00 +0000

noobaa obc create myobc

 oc get obc
 oc get obc myobc -o yaml
 oc get -n openshift-storage secret myobc -o yaml
 oc get -n openshift-storage cm myobc -o yaml

The ConfigMap and the Secret have the same name as the OBC.
The ConfigMap contains the S3 endpoint information for your application, and the Secret provides you with the S3 access credentials.

Get OpenShift Console URL

Mon, 02 Aug 2021 00:00:00 +0000

oc whoami --show-console

oc -n openshift-console get routes

Relaunch docker-compose deployment with a new image

Mon, 02 Aug 2021 00:00:00 +0000

docker-compose pull && docker-compose -f docker-compose.yml up -d

Istio debug commands

Sun, 01 Aug 2021 00:00:00 +0000

Istio Pods

kubectl get pods -n istio-system

Istio Logs

kubectl logs -n istio-system -l app=istiod

Istio Analize

istioctl analyze

Istio Proxy Status

istioctl proxy-status

Kubernetes debug commands

Sun, 01 Aug 2021 00:00:00 +0000

Kubernetes error messages

kubectl get events -n kube-system

Kubernetes pods logs

stern prox -n kube-system

Debug with busybox

kubectl run --generator=run-pod/v1 -i --tty busybox --image=busybox --restart=Never -- sh

CodeReady Containers commands

Tue, 20 Jul 2021 00:00:00 +0000

crc config view # View Actual config
crc config set autostart-tray false # Disable autostart
crc config set consent-telemetry false # Disable telemetry
crc config set enable-cluster-monitoring true # Enable Monitoring, needs more memory
crc config set host-network-access true # Enable TCP/IP connections
crc config set kubeadmin-password $MY_PASSW0RD # Define my own kubeadmin-password
crc config set memory 14336 # Increase Memory to 14GB
crc config set pull-secret-file $PATH_TO_PULL_SECRET_FILE # Path to a pull-secret file (download from https://cloud.redhat.com/openshift/create/local)
crc setup
crc start
crc ip
crc stop

Upload Image to a Registry with Podman

Tue, 20 Jul 2021 00:00:00 +0000

sudo podman pull docker.io/library/alpine
sudo podman push docker.io/library/alpine --tls-verify=false <KUBE_REGISTRY_ROUTE>/alpine

Podman remote client on MacOS using Vagrant

Tue, 23 Feb 2021 00:00:00 +0000

Table of Contents

Introduction

Podman is a daemonless, open-source, Linux native tool designed to make it easy to find, run, build, share and deploy applications using Open Containers Initiative (OCI) Containers and Container Images.

That been said, the core of podman only runs in Linux! To use podman on macOS, we need to implement the remote client to manage container using a Linux as a backend.

Brief Architecture

The remote client uses a client-server model. We need Podman installed on a Linux VM that also has the SSH daemon running. On our MacOS, when you execute a Podman command:

Podman connects to the server via SSH.
It then connects to the Podman service by using systemd socket activation.
The Podman commands are executed on the Linux VM.
From the client’s point of view, it seems like Podman runs locally.

Installation

Install podman on MacOS

To install podman remote client on MacOS, we use Homebrew

$ brew install podman

Create a new ssh-keys on MacOS

We will need to connect via ssh to our vagrant VM, in order to do it passwordless, we will create a ssh-key, the commands for that are:

$ ssh-keygen -t rsa -b 4096 -C "podman+vagrant"
Generating public/private rsa key pair.
Enter file in which to save the key (/Users/<USERNAME>/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /Users/<USERNAME>/.ssh/id_rsa.
Your public key has been saved in /Users/<USERNAME>/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:+pGx7Wcn9WdfRYKJrcdMiKEIPKFRW1lQ1MXP/8i0PLA podman+vagrant
The key's randomart image is:
+---[RSA 4096]----+
| ....=B+. ooo ..|
| . ++. ..+oo.+ |
| o .. o +oo =|
| .o+ |
| S ..|
| . = . . o|
| . + . B oo|
| . o E X o|
| . .+.= o.|
+----[SHA256]-----+

cat /Users/<USERNAME>/.ssh/id_rsa.pub
ssh-rsa AAAAB3NzaC1y ... O3JH8w== podman+vagrant

Create a vagrant VM

We will use a Virtual Machine based on Fedora 33,

To create the specified Vagrantfile, we need to follow the next steps:

$ mkdir my-fedora && cd my-fedora

$ echo "Vagrant.configure("2") do |config|
 config.vm.box = "generic/fedora33"
 config.vm.hostname = "my-fedora"
 config.vm.provider "virtualbox" do |v|
 v.memory = 1024
 v.cpus = 1
 end
end" >> Vagrantfile

Implementation

At this moment we have:

Podman installed
A ssh-key with no password created
A VM created with vagrant

Let’s start our implementation

Copy ssh-key from MacOS to Linux VM

We use the ssh-copy-id command, and it will ask us for the vagrant user password. The default one is: vagrant

$ ssh-copy-id -i id_rsa.pub vagrant@127.0.0.1 -p 2222

/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
vagrant@127.0.0.1's password:

Number of key(s) added: 1

Now try logging into the machine, with: "ssh -p '2222' 'vagrant@127.0.0.1'"
and check to make sure that only the key(s) you wanted were added.

You can verify connectivity with the command:

$ ssh vagrant@127.0.0.1 -p 2222
Last login: Tue Feb 23 07:33:45 2021 from 10.0.2.5
[vagrant@my-fedora ~]$

Configure the Linux VM

On this step we will:

Install the podman package and dependencies
Enable the podman service
Enable the sshd service

Installing podman

sudo dnf --enablerepo=updates-testing install podman libvarlink-util libvarlink

Enableling the podman service

We can enable and start the service permanently, using the following commands:

$ systemctl --user enable --now podman

Also, we will need to enable linger for this user in order for the socket to work when the user is not logged in.

sudo loginctl enable-linger $USER

You can verify that the socket is listening with a simple Podman command.

$ podman --remote info

Enableling the sshd service

In order for the client to communicate with the server you need to enable and start the SSH daemon on the Linux VM:

$ sudo systemctl enable --now sshd

Using the client

The first step in using the Podman remote client is to configure a connection. To do that, we need can add a connection by using the podman system connection add command.

$ podman system connection add <CONNECTION-NAME> ssh://vagrant@127.0.0.1:2222 --identity <SSH-KEY>
$ podman system connection add my-fedora ssh://vagrant@127.0.0.1:2222 --identity ~/.ssh/id_rsa.pub

We can verify that the connection is in place with the command:

$ podman system connection list
Name Identity URI
my-fedora* ~/.ssh/id_rsa.pub ssh://vagrant@127.0.0.1:2222/run/user/1000/podman/podman.sock

Now we can test the connection with the command:

$ podman info
host:
 arch: amd64
 buildahVersion: 1.18.0
 cgroupManager: systemd
 cgroupVersion: v2
 conmon:
 package: conmon-2.0.21-3.fc33.x86_64
 path: /usr/bin/conmon
 version: 'conmon version 2.0.21, commit: 0f53fb68333bdead5fe4dc5175703e22cf9882ab'
 cpus: 1
 distribution:
 distribution: fedora
 version: "33"
 eventLogger: journald
 hostname: my-fedora
 ...

Also, you can start running podman commands as you run them in docker:

$ podman images
REPOSITORY TAG IMAGE ID CREATED SIZE

Next steps

At this point we have installed everything that we need to start using podman on our MacOS, but podman only work if the Linux VM is up & running, otherwise you will receive an error similar to this:

$ podman images
Error: Cannot connect to the Podman socket, make sure there is a Podman REST API service running.: failed to create sshClient:
Connection to bastion host (ssh://vagrant@127.0.0.1:2222/run/user/1000/podman/podman.sock) failed.: dial tcp 127.0.0.1:2222: connect: connection refused

To avoid that behavior, what I implement is an automator workflow, here are the steps:

Get the vagrant VM id, to do that run:

$ vagrant global-status
id name provider state directory
--------------------------------------------------------------------------
894d683 my-fedora virtualbox running ~/vms/my-fedora

The above shows information about all known Vagrant environments
on this machine. This data is cached and may not be completely
up-to-date (use "vagrant global-status --prune" to prune invalid
entries). To interact with any of the machines, you can go to that
directory and run Vagrant, or you can use the ID directly with
Vagrant commands from any directory. For example:
"vagrant destroy 1a2b3c4d"

As you can see we are interested on get the id column, for this example: 894d683

Now, we need to open Automator, go to Launchpad -> search -> type automator, do click on the Automator Application

Launchpad + Automator

Then, we need to write an automator application, for this example I choose a workflow that it will run the command vagrant up <VM-ID>

$ vagrant up 894d683
Bringing machine 'my-fedora' up with 'virtualbox' provider...
==> default: Checking if box 'generic/fedora33' version '3.2.0' is up to date...
==> default: A newer version of the box 'generic/fedora33' for provider 'virtualbox' is
==> default: available! You currently have version '3.2.0'. The latest is version
==> default: '3.2.6'. Run `vagrant box update` to update.
==> default: Clearing any previously set forwarded ports...
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
 default: Adapter 1: nat
==> default: Forwarding ports...
 default: 22 (guest) => 2222 (host) (adapter 1)
==> default: Running 'pre-boot' VM customizations...
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
 default: SSH address: 127.0.0.1:2222
 default: SSH username: vagrant
 default: SSH auth method: private key
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
==> default: Setting hostname...
==> default: Machine already provisioned. Run `vagrant provision` or use the `--provision`
==> default: flag to force provisioning. Provisioners marked to run always will still run.

Automator Workflow

Save the Workflow and remember the location where you save it.
The final step is to add the workflow to our login items. Go to Systems Preferences -> Users & Groups -> Login Items and add the application that you save on the previous step.

Login Items Menu

And that is all, you will have a fully working podman command on your MacOS.

References:

Podman MacOS and windows install ¹
Detailed podman installation on MacOS ²
Automator Configuration ³

My journey to become CKA and CKAD

Thu, 26 Nov 2020 00:00:00 +0000

Hi, people that read me!

I usually don’t write about my achievements. Still, I’m proud of what I did, and in this post I want to share my journey to that back in July I passed two essential certifications for me, the CKA (Certified Kubernetes Administrator) and the CKAD (Certified Kubernetes Application Developer). Those certs are important to me because one of my professional goals is to apply all my knowledge as an SRE (Site Reliabilty Engineer), so I’m putting all my effort to get trained and gain experience as I believe that Kubernetes is and will continue to be widely used by the industry I want to be part of that.

I put a lot of hours/practice on my study lately, but my journey started probably 3 years ago, or even more time (don’t remember exacly). Still, I remember that I become interested when I was talking with my friends @yazpik and @tonyskapunk about the new stuff that is comming on tech, by that time they were running a Kubernetes Meetup at the Castle and I started to assist to each meetup.

Being honest, at the begginning, I barely understand all the concepts and projects that the speakers were talking about but once I started to get involved reading blog posts, practicing (Yeah I did the ‘Kubernetes the Hard way’ on minikube), also began to follow and test other exciting projects like Rancher. So after a while, I started to understand better what the speaker was trying to communicate. And that helps me a lot to understand not only the basics but some good projects that work in conjunction with Kubernetes.

With all the meetings that were held on Rackspace I get involved and I wanted to keep learning more so, I decided to assist to Kubecon North America 2018 that was held on Seattle, and I like to think that this set of conferences opened my mind for all the Kubernetes environment, and I feel that I need to keep learning even more about this awesome technology, that keeps me motivated to get my certifications too, in a nutshell I feel inspired about the conferences, the comunity, everything and I want to be part of it.

Talking about community, in that kubecon I met with some Latin-American friends that were interested in Kubernetes like @domix, @marKox,both from México, and @EdduMelendez from Peru, that from their trenches they are trying to grow the Kubernetes community on Spanish.

Mexicanos in Kubecon 2018

But returning to my certifications path, after a while I decided that I will have the certs by the end on this year, so back in March I get more serious studying and purchased the excellent course Certified Kubernetes Administrator (CKA) with Practice Tests a course by Mumshad Mannambeth that I highly recommend to reinforce the theory and also, becasue includes exercises similar to the exam that you can practice.

Once I feelt confident in me and my knowledge I get my coupon to present the CKA exam, and I passed!

My CKA Certification

I need tell you, it wasn’t easy, mostly because you need to be careful with the time/value of the questions and you need to weigh to reach the passing score, for this exam it was 74%.

Of course after I get the CKA certitification, I was super happy but on my train of thoughts, I keep thinking that I can use the CKA study as leverage to get the CKAD.

I already have the fundamentals of a CKA and just need some extra study to complement the CKAD curriculum, so I got commited (my wife too), and right away I got my CKAD exam coupon and I scheduled the test 2 weeks appart after I decided to get the cert (That’s 3 weeks appart after I took the CKA exam).

My approach works, I passed the CKAD certification!

My CKAD Certification

Now that I’m certified on Kubernetes I will look to be more involved in Cloud Native initiatives on my company, to apply the knowledge that I already have on current or new projects, and as always keep learning!

Kubernetes Contributor

Sun, 27 Oct 2019 00:00:00 +0000

Contributing to the Kubernetes project in different special interest groups, mainly focused on migration and modernization, documentation, community, and user/contributor experience.

Prow PR Dashboard

sig-docs

Contributor on the Spanish translation content at kubernetes.io in sig-docs.

kubernetes-docs-es

konveyor

Ansible Galaxy Roles

Fri, 25 Oct 2019 00:00:00 +0000

CircleCI Role: Ansible role that installs CircleCI CLI on your server/workstation.
Sonobuoy Role: Ansible role that installs sonobuoy on your server/workstation.

Docker Login the Right Way

Wed, 15 May 2019 00:00:00 +0000

Table of Contents

Hi again!

It is been a while since I wrote something here, as always, there is no much time for a hobby.

I’ve been working for a while with docker, not a production level, but for some applications that I use at work. And since the Docker Hub Data breach I put more atention on the security of my data/credentials, so I investigate a little about and found this official repository https://github.com/docker/docker-credential-helpers/ from Docker where are the supported credential helpers.

Credential Store

Docker keeps our credentials saved on a JSON file located on ~/.docker/config.json, but unfortunatelly credentials are just encrypted on base64, here is an articule/video where there is an explanation for the why it is a bad idea to just use base64 encryption.

The following is a diagram on how a plain text storage works:

Plain Text Storage

Here is an example on how ~/.docker/config.json looks like when is using plain text credentials:

cat ~/.docker/config.json
{
 "auths": {
 "https://index.docker.io/v1/": {
 "auth": "azRjaDA6c3VwZXJzZWNyZXRwYXNzd29yZAo="
 },
 "quay.io": {
 "auth": "azRjaDA6c3VwZXJzZWNyZXRwYXNzd29yZAo="
 }
 },
 "HttpHeaders": {
 "User-Agent": "Docker-Client/18.09.6 (linux)"
 }
}

After a successful docker login command, Docker stores a base64 encoded string from the concatenation of the username, a colon, and the password and associates this string to the registry the user is logging into:

$ echo azRjaDA6c3VwZXJzZWNyZXRwYXNzd29yZAo= | base64 -d -
luiscachog:supersecretpassword

A docker logout command removes the entry from the JSON file for the given registry:

$ docker logout quay.io
Remove login credentials for quay.io

$ cat ~/.docker/config.json
{
 "auths": {
 "https://index.docker.io/v1/": {
 "auth": "azRjaDA6c3VwZXJzZWNyZXRwYXNzd29yZAo="
 }
 },
 "HttpHeaders": {
 "User-Agent": "Docker-Client/18.09.6 (linux)"
 }
}

Docker Credential Helpers

Since docker version 1.11 implements support from an external credential store for registry authentication. That means we can use a native keychain of the OS. Using an external store is more secure than storing on a “plain text” Docker configuration file.

Secure Storage

In order to use a external credential store, we need a program to interact with.

The actual list of “official” Docker Credential Helper is:

docker-credential-osxkeychain: Provides a helper to use the OS X keychain as credentials store.
docker-credential-secretservice: Provides a helper to use the D-Bus secret service as credentials store.
docker-credential-wincred: Provides a helper to use Windows credentials manager as store.
docker-credential-pass: Provides a helper to use pass as credentials store.

docker-credential-secret service

On this post we will explore the docker-credential-secretservice and how to configure it.

We need to download and install the helper. You can find the lastest release on https://github.com/docker/docker-credential-helpers/releases. Download it, extract it and make it executable.

wget https://github.com/docker/docker-credential-helpers/releases/download/v0.6.2/docker-credential-secretservice-v0.6.2-amd64.tar.gz
tar -xf docker-credential-secretservice-v0.6.2-amd64.tar.gz
chmod +x docker-credential-secretservice
sudo mv docker-credential-secretservice /usr/local/bin/

Then, we need to specify the credential store in the file ~/.docker/config.json to tell docker to use it. The value must be the one after the prefix docker-credential-. In this case:
```
{
 "credsStore": "secretservice"
}
```
To facilite the configuration and do not make mistakes, you can run:
```
sed -i '0,/{/s/{/{\n\t"credsStore": "secretservice",/' ~/.docker/config.json
```

From now we are uning an external store, so if you are currently logged in, you must run docker logout to remove the credentials from the file and run docker login tostart using the new ones.

Let me know how this works for you.

References:

Docker Credential Helpers repository¹
Docker Credential Store Documentation²
Slides about this topic ³

Set a hugo blog on Kubernetes

Mon, 18 Jun 2018 00:00:00 +0000

Table of Contents

Overview

Since last year I been trying to become an SRE (Site Reliability Engineer), so I been involved with some emerging technologies, like ansible, docker and on this time with kubernetes.

This time, I will try to explain how I containerized my blog using:

Architecture

So, I take some ideas from here and I modify them and adapt the architecture described to my options.

The principal changes that I made are:

My Kubernetes cluster is running on 2 cloud server on Rackspace Public Cloud
The container registry that I’m using is Quay
Rackspace Public Cloud does not support a Kubernetes LoadBalancer service automatically, so I simulate that behavior adding a Cloud Load Balancer manually after the Kubernetes service provide me the port.

Architecture

Containerized

I use Hugo to deploy my blog, I used to do it as mentioned on this previous post (In Spanish).

Now, as a part of containerize the blog it make sense to me to create two stages as described here:

The first stage is a defined build environment containing all required build tools (hugo, pygments) and the source of the website (Git repository).
The second stage is the build artifact (HTML and assets), from the first stage and a webserver to serve the artifact over HTTP.

Dockerfile

Here is the Dockerfile that containerize the blog:

FROM ubuntu:latest as STAGEONE

# install hugo
ENV HUGO_VERSION=0.41
ADD https://github.com/gohugoio/hugo/releases/download/v${HUGO_VERSION}/hugo_${HUGO_VERSION}_Linux-64bit.tar.gz /tmp/
RUN tar -xf /tmp/hugo_${HUGO_VERSION}_Linux-64bit.tar.gz -C /usr/local/bin/

# install syntax highlighting
RUN apt-get update
RUN apt-get install -y python3-pygments

# build site
COPY source /source
RUN hugo --source=/source/ --destination=/public/

FROM nginx:stable-alpine
RUN apk --update add curl bash
RUN rm /etc/nginx/conf.d/default.conf
COPY modules/nginx.luiscachog.io.conf /etc/nginx/conf.d/luiscachog.io.conf
COPY --from=STAGEONE /public/ /usr/share/nginx/html/
EXPOSE 80 443

MAINTAINER Luis Cacho <luiscachog@gmail.com>

First Stage

Fetch the lastest Ubuntu image and name as STAGEONE
Install the last available hugo version from source.
Install pygments library to use it for highlighting.
Build the site with hugo and the output is set on /public as a build artifact.

Second Stage

Fetch the lastest stable nginx alpine image.
Update the image and install some utilities.
Delete the default nginx configuration file.
Copy the configuration files needed from the repository root directory.
Copy the build artifact /public from the previous stage (STAGEONE)

Cloud-Native | Luis Cacho

Trigger a scheduled job manually on Kubernetes

Delete Evicted Pods

ArgoCD cli login command

Understanding the Compliance Operator

Compliance Operator Introduction

Why Compliance Operator?

How can I implement Compliance Operator in my cluster?

Delete OpenShift Operator

Delete OpenShift Project

How to Retrieve secrets values

How to list the taints on kubernetes nodes

How to access an OpenShift 4 Node

Get ServiceAccount token for MTC

Command to perform a curl to a PodIP

Approve pending CSRs in OpenShift

Create a Nooba Object Bucket Claim

Get OpenShift Console URL

Relaunch docker-compose deployment with a new image

Istio debug commands

Kubernetes debug commands

CodeReady Containers commands

Upload Image to a Registry with Podman

Podman remote client on MacOS using Vagrant

Introduction

Brief Architecture

Installation

Install podman on MacOS

Create a new ssh-keys on MacOS

Create a vagrant VM

Implementation

Copy ssh-key from MacOS to Linux VM

Configure the Linux VM

Installing podman

Enableling the podman service

Enableling the sshd service

Using the client

Next steps

My journey to become CKA and CKAD

Kubernetes Contributor

sig-docs

kubernetes-docs-es

konveyor

Ansible Galaxy Roles

Docker Login the Right Way

Docker Login the right Way

Credential Store

Docker Credential Helpers

docker-credential-secret service

Set a hugo blog on Kubernetes

Overview

Architecture

Containerized

Dockerfile

First Stage

Second Stage