Cloud-Native | Luis Cacho

Trigger a scheduled job manually on Kubernetes

Thu, 22 Jun 2023 00:00:00 +0000

# Mockup command
kubectl create job --from=cronjob/<cronjob-name> <job-name> -n <namespace-name>

References:

¹How can I trigger a Kubernetes Scheduled Job manually?

https://stackoverflow.com/questions/40401795/how-can-i-trigger-a-kubernetes-scheduled-job-manually ↩︎

Delete Evicted Pods

Wed, 23 Nov 2022 00:00:00 +0000

oc get pods | grep Evicted | awk '{print $1}' | xargs kubectl delete pod

References:

¹ Gist

https://gist.github.com/ipedrazas/9c622404fb41f2343a0db85b3821275d ↩︎

ArgoCD cli login command

Wed, 17 Aug 2022 00:00:00 +0000

argocd login \
--insecure \
--grpc-web $(oc get route openshift-gitops-server -n openshift-gitops
-o jsonpath='{.spec.host}{"\n"}') \
--username admin \
--password $(oc get secret/openshift-gitops-cluster -n openshift-gitops -o jsonpath='{.data.admin\.password}' | base64 -d)

Delete OpenShift Operator

Mon, 15 Aug 2022 00:00:00 +0000

# Mockup commands

oc get subscription <operator> -n openshift-operators -o yaml | grep
currentCSV

oc delete subscription <operator> -n openshift-operators

oc delete clusterserviceversion <currentCSV> -n openshift-operators

# Example commands

oc get subscription jaeger -n openshift-operators -o yaml | grep currentCSV

oc delete subscription jaeger -n openshift-operators

oc delete clusterserviceversion jaeger-operator.v1.8.2 -n openshift-operators

Delete OpenShift Project

Mon, 15 Aug 2022 00:00:00 +0000

Issue, can’t delete a namespace/project and is stuck in a “Terminating” status

oc delete project fishbone

oc get projects fishbone
NAME DISPLAY NAME STATUS
fishbone Fishbone Terminating

Solution

Define the namespace/project to delete

export NAMESPACE=fishbone

Cleanup the finalizers section from the namespace definition and define a json file with that

oc get namespace $NAMESPACE -o=json | jq '.spec = {"finalizers":[]}' > $NAMESPACE.json
{
 "apiVersion": "v1",
 "kind": "Namespace",
 "metadata": {
 "annotations":
 "openshift.io/description": "",
 "openshift.io/display-name": "Fishbone",
 "openshift.io/requester": "system:admin",
 "openshift.io/sa.scc.mcs": "s0:c24,c14",
 "openshift.io/sa.scc.supplemental-groups": "1000580000/10000",
 "openshift.io/sa.scc.uid-range": "1000580000/10000"
 },
 "creationTimestamp": "2021-02-20T03:29:59Z",
 "deletionTimestamp": "2021-02-20T03:31:35Z",
 "name": "fishbone",
 "resourceVersion": "270664",
 "selfLink": "/api/v1/namespaces/fishbone",
 "uid": "885c4440-6f05-4e72-a103-28e61705406f"
 },
 "spec": {
 "finalizers": [
 "kubernetes"
 ]
 },
 "status": {
 "conditions": [
 .
 .
 .
 ],
 "phase": "Terminating"
}

Apply the json file from the previous step to delete the finalizers and therefore the deletion of the namespace/project

curl -k -H "authorization: Bearer $(oc whoami --show-token)" -H "Content-Type: application/json" -X PUT --data-binary @$NAMESPACE.json $(oc whoami --show-server)/api/v1/namespaces/$NAMESPACE/finalize

Verify that the namspace/project has been deleted

oc get projects $NAMESPACE
Error from server (NotFound): namespaces "fishbone" not found

References:

Kubernetes Advanced Topics¹
Kubernetes Namespace Stuck²
Red Hat article[^3]

How to Retrieve secrets values

Mon, 15 Aug 2022 00:00:00 +0000

oc get secret my-secret -n my-project --template={{.data.my-secret-key}} | base64 -d

oc get secret my-secret -n my-project -o go-template --template="{{.data.my-secret-key|base64decode}}"

oc extract secrets/my-secret -n my-project

oc get secret my-secret -n my-project -o jsonpath="{.data['tls\.key']}" | base64 -d

How to test the OpenShift Cluster and Machine Autoscaler

Mon, 15 Aug 2022 00:00:00 +0000

oc new-project work-queue


echo 'apiVersion: batch/v1
kind: Job
metadata:
 generateName: work-queue-
spec:
 template:
 spec:
 tolerations:
 - effect: NoSchedule
 key: node-role.kubernetes.io/infra
 value: reserved
 - effect: NoExecute
 key: node-role.kubernetes.io/infra
 value: reserved
 containers:
 - name: work
 image: busybox
 command: ["sleep", "300"]
 resources:
 requests:
 memory: 2Gi
 cpu: 4
 restartPolicy: Never
 parallelism: 50
 completions: 250' | oc create -f - -n work-queue

oc get jobs -n work-queue

oc get pods -n work-queue 

oc get machineset -n openshift-machine-api

oc get machines -n openshift-machine-api

oc get nodes

oc delete project work-queue

How to list the taints on kubernetes nodes

Sun, 14 Aug 2022 00:00:00 +0000

oc get nodes -o json | jq '.items[].spec.taints'
oc get nodes -o json | jq '.items[] | {node_name:.metadata.name,taints:.spec.taints[]?}

oc get nodes -o=custom-columns=NAME:.metadata.name,TAINTS:.spec.taints

get nodes -o=jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.taints}{"\n"}{end}'

References:

¹ StackOverflow

]: https://stackoverflow.com/questions/43379415/how-can-i-list-the-taints-on-kubernetes-nodes ↩︎

OpenID autodiscovery URL integration in OpenShift

Mon, 23 May 2022 00:00:00 +0000

# Discover the available URLs from your autodiscovery URL
curl https://<idp_host>/.well-known/openid-configuration

CLIENT_SECRET=$(oc get secret -n openshift-config my-id-secret my-id-secret -o jsonpath='{.items[0].data.clientSecret}' | base64 -D)

curl -s -X POST https://<idp_host>/idp/userinfo.openid -H 'content-type: application/x-www-form-urlencoded' -d "client_id=ocp4testawsuswest2QA&client_secret=${CLIENT_SECRET}&access_token=${ACCESS_TOKEN}" | jq

References: - ¹ https://access.redhat.com/solutions/4605141

https://access.redhat.com/solutions/4605141 ↩︎

How to access an OpenShift 4 Node

Thu, 16 Sep 2021 00:00:00 +0000

In order to review the underline node, you need to follow the next steps:

Get the node name that is having issues or your need to login

oc get nodes

Run the oc debug command into the node

oc debug node/ip-10-0-129-255.us-east-2.compute.internal

Once you are logged in the node, you’ll need to run:

chroot /host

References:

¹ OCP4 Documentation

https://docs.openshift.com/container-platform/4.10/support/gathering-cluster-data.html#support-collecting-network-trace_gathering-cluster-data ↩︎

Get ServiceAccount token for MTC

Fri, 06 Aug 2021 00:00:00 +0000

oc sa get-token migration-operator -n openshift-migration

Command to perform a curl to a PodIP

Thu, 05 Aug 2021 00:00:00 +0000

curl $(oc get pod <pod-name> --template '{{ .status.podIP}}):8080

Approve pending CSRs in OpenShift

Wed, 04 Aug 2021 00:00:00 +0000

for id in $(oc get csr | grep Pending | awk '{print $1}'); do oc adm certificate approve $id; done

Get OpenShift Console URL

Mon, 02 Aug 2021 00:00:00 +0000

oc whoami --show-console

oc -n openshift-console get routes

Relaunch docker-compose deployment with a new image

Mon, 02 Aug 2021 00:00:00 +0000

docker-compose pull && docker-compose -f docker-compose.yml up -d

Restart an Istio deployment

Mon, 02 Aug 2021 00:00:00 +0000

Error:

MountVolume.SetUp failed for volume "istiod-ca-cert" : configmap "istio-ca-root-cert" not found

Fix:

kubectl rollout restart deployment istiod -n istio-system

Istio debug commands

Sun, 01 Aug 2021 00:00:00 +0000

Istio Pods

kubectl get pods -n istio-system

Istio Logs

kubectl logs -n istio-system -l app=istiod

Istio Analize

istioctl analyze

Istio Proxy Status

istioctl proxy-status

Kubernetes debug commands

Sun, 01 Aug 2021 00:00:00 +0000

Kubernetes error messages

kubectl get events -n kube-system

Kubernetes pods logs

stern prox -n kube-system

Debug with busybox

kubectl run --generator=run-pod/v1 -i --tty busybox --image=busybox --restart=Never -- sh

CodeReady Containers commands

Tue, 20 Jul 2021 00:00:00 +0000

crc config view # View Actual config
crc config set autostart-tray false # Disable autostart
crc config set consent-telemetry false # Disable telemetry
crc config set enable-cluster-monitoring true # Enable Monitoring, needs more memory
crc config set host-network-access true # Enable TCP/IP connections
crc config set kubeadmin-password $MY_PASSW0RD # Define my own kubeadmin-password
crc config set memory 14336 # Increase Memory to 14GB
crc config set pull-secret-file $PATH_TO_PULL_SECRET_FILE # Path to a pull-secret file (download from https://cloud.redhat.com/openshift/create/local)
crc setup
crc start
crc ip
crc stop

Upload Image to a Registry with Podman

Tue, 20 Jul 2021 00:00:00 +0000

sudo podman pull docker.io/library/alpine
sudo podman push docker.io/library/alpine --tls-verify=false <KUBE_REGISTRY_ROUTE>/alpine

Wait for pod to be ready

Thu, 15 Jul 2021 00:00:00 +0000

export POD_NAME=hello-d8d8d7455-j9nzw
while [[ $(oc get pods $POD_NAME -o 'jsonpath={..status.conditions[?(@.type=="Ready")].status}') != "True" ]]; do echo "waiting for pod" && sleep 1; done

If you want to wait until all the pods with a common label you can do:

while [[ $(kubectl get pods -l app=hello -o 'jsonpath={..status.conditions[?(@.type=="Ready")].status}') != "True" ]]; do echo "waiting for pod" && sleep 1; done

OpenShift MigMigration

Mon, 01 Jan 0001 00:00:00 +0000

 oc get migmigrations.migration.openshift.io <mig-migration-plan> -n openshift-migration -o go-template="{{.status.phase}}{{ println }}"