Author: Matt
-
Lessons in Managing my Kubernetes Cluster: Man Down!
I had a bit of a panic this week as routine tasks took me down a rabbit hole in Kubernetes. The more I manage my home lab clusters, the more I realize I do not want to be responsible for bare metal clusters at work. It was a typical upgrade… With ArgoCD in place, the…
-
Hitting for the cycle…
Well, I may not be hitting for the cycle, but I am certainly cycling Kubernetes nodes like it is my job. The recent OpenSSL security patches got me thinking that I need to cycle my cluster nodes. A Quick Primer In Kubernetes, a “node” is, well, a machine performing some bit of work. It could…
-
What’s in a (Release) Name?
With Rancher gone, one of my clusters was dedicated to running Argo and my standard cluster tools. Another cluster has now become home for a majority of the monitoring tools, including the Grafana/Loki/Mimir/Tempo stack. That second cluster was running a little hot in terms of memory and CPU. I had 6 machines running what 4-5…
-
A Lesson in Occam’s Razor: Configuring Mimir Ruler with Grafana
Occam’s Razor posits “Of two competing theories, the simpler explanation is to be preferred.” I believe my high school biology teacher taught the “KISS” method (Keep It Simple, Stupid) to convey a similar principle. As I was trying to get alerts set up in Mimir using the Grafana UI, I came across an issue that…
-
Walking away from Rancher
I like to keep my home lab running for both hosting sites (like this one) and experimenting on different tools and techniques. It is pretty nice to be able to spin up some containers in a cluster without fear of disturbing other’s work. And, since I do not hold myself to any SLA commitments, it…
-
Kubernetes Observability, Part 5 – Using Mimir for long-term metric storage
This post is part of a series on observability in Kubernetes clusters: Part 1 – Collecting Logs with Loki Part 2 – Collecting Metrics with Prometheus Part 3 – Dashboards with Grafana Part 4 – Using Linkerd for Service Observability Part 5 – Using Mimir for long-term metric storage (this post) For anyone who actually…
-
Getting Synology SNMP data into Prometheus
With my new cameras installed, I have been spending a lot more time in the Diskstation Manager (DSM). I always forget how much actually goes on within the Synology, and I am reminded of that every time I open the Resource Monitor. At some point, I started to wonder whether or not I could get…
-
Camera 1, Camera 2…
First: I hate using acronyms without definitions: NAS – Network Attached Storage. Think “network hard drive” SAN – Storage Area Network – Think “a network of hard drives”. Backblaze explains the differences nicely. iSCSI Target – iSCSI is a way to mount a volume on a NAS or SAN to a server in a way…
-
Tech Tips – Upgrading your Argo cluster tools
Moving my home lab to GitOps and ArgoCD has been, well, nearly invisible now. With the build pipelines I have in place, I’m able to work on my home projects without much thought to deploying changes to my clusters. My OCD, however, prevents me from running old versions. I really want to stay up-to-date when…
-
I’ll take ‘Compatibility Matrices’ for $1000, Alex…
I have posted a lot on Kubernetes over the past weeks. I have covered a lot around tools for observability, monitoring, networking, and general usefulness. But what I ran into over the past few weeks is both infuriating and completely avoidable, with the right knowledge and time to keep up to speed. The problem I…