This is a follow-up my previous post on Architecture considerations for stateful Kubernetes applications and is specific to VMWare’s Tanzu Kubernetes Grid (TKG) implementation of Kubernetes. In lieu of utilizing NFS pod to gain RWX (aka ReadWriteMany) access to vSphere volumes, I decided to go a different route. Important caveat #1 for TKG users: It is important to note that this is currently only recommended in test/dev environments. Portworx confirmed the Kernel headers issue is planned to be fixed in their v3 release. Important caveat #2 for TKG/vSphere users: You cannot generate a spec from PX […]
Volume is already exclusively attached to one node and can’t be attached to another Based on my previous post, this is meant to be succinct for others encountering the ‘Multi-Attach’ error. Kubernetes does not allow multiple nodes to mount (certain) volumes concurrently. A Kubernetes bug also exists that does not forcefully detach a pv from a node after the 6 minute timeout, causing multi-attach headaches. References: https://github.com/kubernetes-sigs/vsphere-csi-driver/issues/221 https://github.com/kubernetes/kubernetes/issues/65392 https://cormachogan.com/2019/06/18/kubernetes-storage-on-vsphere-101-failure-scenarios/ How it begins Hmm.. Pods are not restarting, what did I do this time? Primary issue found: How does one recover from this situation? There a few […]
While planning the architecture for our Tanzu Kubernetes Grid (TKG) deployment, one of the tests I wanted to perform was node + pod scaling to see if any issues arose. The application in question is Django/Python/JS/Bootstrap based, and I was trying to achieve the following goals: Migrate the application with as few architecture changes as possible, continuing to utilize the on-prem resources that have a couple of years worth of equipment + licensing to run out. Prepare the application for CI/CD onto cloud-native resources with IaC Utilize NFS RWX mounts to share media between the Django […]
In searching for ways to use an ‘insecure’ registry with Tanzu Kubernetes Grid, I found a post by William Lam explaining how. I wanted to build on this and figure out what was needed to deploy a customized cluster with tkg-cli, and skip deploying the kind cluster first. The files located in .tkg/bom/ are the key to this, as they are the templates that tkg-cli uses to bootstrap kind, and deploy the TKG cluster. With the same technique, you can perform additional customization as needed; possibly more on that in the future. Step 1. Locate and […]
Tip(s) #1 TKG / Photon OS 3.0 and Private Registry vSphere Integrated Containers / Harbor as private registry (link) Easy-to-deploy private registry that consumes native vSphere resources and integrate into an existing environment easily. It takes roughly 5-10 minutes to deploy a secured Harbor private registry integrated with (in my case) Active Directory. Custom certs can be provided at install time, or replaced easily after install. ErrImagePull: temporary failure in name resolution reg.corp.local Ensure your private registry is reachable on a domain other than .local. There are known issues with systemd-resolved There are some workarounds that […]
We use cookies to optimize our website and our service.