Matt Gerega

Category: Technology

  • Building and deploying container applications

    On and off over the last few months, I have spent a considerable amount of time working to create a miniature version of what could be a production system at home. The goal, at least in this first phase, is to create an API which supports containerized deployment and a build and deploy pipeline to move this application through the various states of development (in my case, development, staging, and production).

    The Tools

    I chose tools that are currently being used (or may be used) at work to allow my research to be utilized in multiple areas, and it has allowed me to dive into a lot of new technologies and understand how everything works together.

    • Teamcity – I utilize TeamCity for building the API code and producing the necessary artifacts. In this case, the artifacts are docker images. Certainly, Jenkins or another build pipeline could be used here.
    • Octopus Deploy – This is the go-to deployment application for the Windows Server based applications at my current company, so I decided to utilize it to roll out the images to their respective container servers. It supports both Windows and Linux servers.
    • Proget/Docker Registry – I have an instance of ProGet which houses my internal nuget packages, and was hoping to use a repository feed to house my docker images. Alas, it doesn’t support Docker Registries properly, so I ended up standing up an instance of the Docker Registry for my private images.

    TeamCity and Docker

    The easiest of all these steps was adding Docker support to my ASP.NET Core 2.2 Web API. I followed the examples on Docker’s web site, and had a Docker file in my repository in a few minutes.

    From there, it was a matter of installing a TeamCity build agent on my internal Windows Docker Container server (windocker). Once this was done, I could use the Docker Runner plugin in TeamCity to build my docker image on windocker and then push it to my private repository.

    Proget Registry and Octopus Deployment

    This is where I ran into a snag. Originally, I created a repository on my ProGet server to house these images, and TeamCity had no problem pushing the images to the private repository. The ProGet registry feeds, however, don’t fully support the Docker API. Specifically, Octopus Deploy calls out the missing _catalog endpoint, which is required for selection of the images during release. I tried manually entering the values (which involved some guessing), but with the errors I ran into, I did not want to continue.

    So I started following the instructions for deploying an instance of Docker Registry on my internal Linux Docker Container Server (docker). The documentation is very good, and I did not have any trouble until I tried to push a Windows image… I kept getting a 500 Internal Server Error with a blob unknown to registry error in the log. I came across this open issue regarding pushing windows images. As it turns out, I had to disable validation in order to get that to work. Once I figured out that the proper environmental variable was REGISTRY_VALIDATION_DISABLED=true (REGISTRY_VALIDATION_ENABLED=false didn’t work), I was able to push my Windows images to the registry.

    Next Steps

    With that done, I can now use Octopus to deploy and promote my docker images from development to staging to production. As seen in the image, my current network topology has matching container servers for both Windows and Linux, along with servers for internal systems (including running my TeamCity Server and Build Agents as well as the Docker Registry).

    My current network topology

    My next goal is to have the Octopus Deployment projects for my APIs create and deploy APIs in my Gravitee.io instance. This will allow me to use an API management tool through all of my environments and provide a single source of documentation for these APIs. It will even allow me to expose my APIs for public usage, but with appropriate controls such as rate throttling and authentication.

    I will be maintaining my infrastructure files and scripts in a public GitHub repository.

  • Polyglot v2 and Docker

    Update: I was able to get this working on my Docker server. Check out the details here.

    Before you read through all of this:  No, I have not been able to get the Polyglot v2 server working in Docker on my docker host (Ubuntu 18.04).  I ended up following the instructions for installing on a Raspberry Pi using Raspbian.  

    Goal

    My goal was to get a Polyglot v2 server up and running so that I could run the MyQ node server and connect my ISY Home Automation controller to my garage door.  And since the ISY is connected through the ISY Portal, I could then configure my Amazon Echo to open and close the garage door.

    Approach #1 – Docker

    Since I already have an Ubuntu server running several docker containers, the most logical solution was to run the Polyglot server as a docker container.  And, according to this forum post, the author had already created a docker compose file for the Polyglot server and a MongoDB instance.  It would seem I should be running about as quickly as I could download the file and run the docker-compose command.  But it wasn’t that easy.

    The MongoDB server started, and as best I can tell, the Polyglot server started and got stuck looking for the ISY.  The compose file had the appropriate credentials and address for the ISY, so I am not sure why it was getting stuck there.  Additionally, I could not locate any logs with more detail than what I was getting from the application output, so it seemed as though I was stuck.  I tried a few things around modifying port numbers and the like, but to no avail.  In the interest of just wanting to get this to work, I attempted something else.

    Approach #2 – Small VM

    I wanted to see if I could get Polyglot running on a VM.  No, it would not be as flashy as Docker, but it should get the job done.  And it would still let me virtualize the machine on my server so I didn’t need to have a Raspberry Pi just hanging around to open and close the garage door.

    I ran into even more trouble here:  I had libcurl4 installed on the Ubuntu Server, but MongoDB wanted libcurl3.  Apparently this is something of a known issue, and there were some workarounds that I found.  But, quite frankly, I did not feel the need to dive into it.  I had a Raspberry Pi so I did the logical thing…

    Giving up…

    I gave up, so to speak.  The Polyglot v2 instructions are written for installation on a Raspberry Pi.  I have two at home, neither in much use, so I followed the installation instructions from the repository on one of my Raspberry Pi’s and a fresh SD card.  It took about 30 minutes to get everything running, MyQ installed and configured, and the ISY Portal updated to let me open the garage door with my Echo.  No special steps, just read the instructions and you are good.

    What’s next?

    Well, I would really like to get that server running in Docker.  It would make it a lot easier to manage and free up that Raspberry Pi for more hardware related things (like my home bar project!).  I posted a question to the ISY forums in the hopes that someone has seen a similar problem and might have some solutions for me.  But for now, I have a Raspberry Pi running that lets me open my garage door with the Amazon Echo.

  • TeamCity Plugin for Microsoft Teams Build Notifications

    TLDR;

    I wrote a TeamCity plugin that allows you to post notifications to Microsoft Teams.  Open source, based on tcSlackBuildNotifier.  Please use and contribute!

    GitHub Repository   Build Status

    If you really care…

    IRC is back, baby!  Well, not exactly.  Corporate messaging applications like Slack and Microsoft Teams are becoming fairly commonplace in companies big and small.  They allow for your standard one-on-one messaging, but their draw comes in the form of channels for hosting and storing long-running chatter (IRC!), forming one-off group chats for large and small projects or incidents, hosting video calls in both individual and group context, and generally being the “go to” application for communication within the company.  Additionally, most of these applications have a fairly robust API layer and plugin framework to allow those of us with the desire to do it a chance to integrate with the platform.

    A few years ago, my current company started with the free version of Slack.  As part of that, we used the tcSlackBuildNotifier plugin to allow TeamCity to post to Slack channels as a notification avenue.  The Slack implementation, however, was done through what could affectionately be referred to as “back channels:” A group signed up for it to use for their own purposes and it exploded from there.  It was not long before we had outgrown the free version and were looking to purchase licenses.

    We ran into two problems: Slack is expensive for our company size, and as part of our agreements with Microsoft (specifically, Office 365), we already had access to Microsoft Teams.  After a lot of futile arguments from some of those using Slack, we went with Teams as our official company communication platform.

    My problem was that I couldn’t find a similar notification tool for Teams that we had for Slack, and my team and I really liked having the build notifications posted to the channels.  It allowed us a quick notification when something failed or succeeded after failure.

    So, I took the tcSlackBuildNotifier and modified it to allow for posting to Microsoft Teams.  It is my first foray into Maven and what amounts to a re-introduction to Java, but it gets the job done.  I will not be terrible active in maintaining it:  if something breaks I will fix it, but I don’t anticipate new features coming out.  So, if you use TeamCity and Microsoft Teams, try it out!

     

  • First Reactions – ISY994i ZW/IR PRO Controller – Insteon, Z-Wave & IR + PLM

    As I posted last week, I started a journey down the home automation path with the purchase of a controller and some Insteon switches and sensors.  Given my current schedule with work and kids, it’s hard to find time to do much, but I was able to install the switches and one of the sensors and get a few scenes and programs setup, so I’ll run through a quick review.

    ISY994i ZW/IR PRO Controller – Insteon, Z-Wave & IR + PLM

    The controller, purchased from SmartHome.com, was extremely easy to install:  plugged in the PLM, connected the PLM to the controller and the controller to my switch, and then plugged in the power cable for the controller.  Configuration required getting a Java applet downloaded and running.  As a developer, the interface is fine for me, however, it lacks the ease of use that we have come to expect in our increasing mobile-centric world.  With some poking around, though, I was able to setup some scenes and a few programs.

    Of note:  I had to install Universal Devices “alpha” version of the 5.0 software in order to get compatibility with the Siren module, which isn’t exactly a new module.  From what I could tell, version 5 of the ISY Portal has been in development for at least two years now, so I’m not sure what the delay is about, but the installation of version 5 was pretty easy and I haven’t run into any major problems yet.

    The controller has a pretty substantial REST api which I believe I could use as the backend for a better API, either a mobile app or responsive web app.  That being said, that’s a lot of work.  If I was sure I would only be using Insteon products, I may have opted for an Insteon Hub, but I already know I want to interface with some other systems, so I’ll take my chances with this controller.

    Insteon Remote Toggle Switch

    Let me preface this review with the following disclaimer:  I am reasonable comfortable with residential electrical wiring, to the degree that I planned and installed all the wiring for lighting and switches when we finished our basement.  That said, the Insteon switches were pretty easy to install, but I do have a word of warning:  they are bulky.  Before you go investing a ton into them, it might be wise to buy one or two of the ones you want and test fit them in your home’s electrical work boxes.  I know for a fact that in a few of my switch boxes I may not have the room for them boxes without trimming existing wiring to make some room.

    Once installed, the controller did a great job of picking up the new hardware through a “device add” wizard.  From there, adding it to scenes and programs was pretty easy.  I was able to add a program to run my pool pump for 12 hours during the day and created a scene which turns on both the pool pump and lights.

    A word of caution, though:  if you plan on doing a large install and then having the controller pick up changes, make sure you write down the network address for each switch and where you installed it.  This will allow you to quickly name your switches for identification without doing what I did, which was turn one switch on and then check the controller and find the switch in the “On” state.

    Overall, I am impressed with the system.  The controller provides a good balance of features and usability, but as I mentioned, it leans more towards features and less towards a slick user interface.  It’s perfect for someone who wants all the features of home automation at a reasonable cost.  And with the APIs and a few third party software solutions, the UI issues can be addressed.

  • Go big or go home automation….

    First and foremost:  I know, the title really is terrible, but my wit is adjusting to the 30 degree temperature swing our little corner of the world experienced over the last 24 hours.

    Last fall, we put in a pool.  The crazy weather over the last six months, though, means that we still have a few more odds and ends to complete before we can officially call it done.  Two things in this project have led me to put a little investment into home automation over the last few days.

    There are some pretty fancy (and expensive) systems for controlling your pool.  Most of the Pentair systems start at around $600 for just the controllers and go way up from there.  While I would have loved to control all of that through the Pentair systems, I did not want to add another two thousand dollars to the cost of the pool, so we opted out of that one.

    While the electricians were here last week, I mentioned those systems, and the electrician said he would add switched for the pool lights and pump.  I asked him if I could replace the switches he installed with a few home automation-compatible switches, and he said sure.  So, with the right controller, I can control my pool lights and pump from a standard home automation system.  So I started thinking, which is always an expensive proposition.

    In addition to that, per our building code, we need to put an alarm on the back door, since our pool layout is such that the house forms one side of the barrier around the pool (our new fence is the other three sides).  Of course, I could have just purchased a $20 door alarm, but where is the fun in that?  So I went shopping and found a door sensor and alarm unit that will meet our building codes and let me add some additional monitoring to the house.

    As for the technical specs, I went with the Universal Devices controller with the PLM module from SmartHome.com.  The overall completeness of this package was a huge draw, not to mention a fairly substantial REST API interface which should allow me to tinker with integrating the controller with some other items in my home.

    I bought a few of the simple Insteon toggle switches and a four pack of the Insteon open/close sensors for the starter pack, as well as the Insteon Siren for the alarm sound.  My hope, however, is to tie all of this in to my Amazon Echo unit so that the Echo can generate the audio alerts for certain actions.

    So yea, I may have spent a little more than $20, but this little adventure into home automation is something that I have wanted to investigate for a few years now, and it seems like the right time to do so.  That, coupled with a convenient sale on Insteon products from SmartHome, let me to jump into this a bit more than I initially planned.

    Stay tuned for more updates as I get the system up and running.