Matt Gerega

Tag: Python

  • Spoolman for Filament Management

    “You don’t know what you go ’til it’s gone” is a great song line, but a terrible inventory management approach. As I start to stock up on filament for the 3D printer, it occurred to me that I need a way to track my inventory.

    The Community Comes Through

    I searched around for different filament management solutions and landed on Spoolman. It seemed a pretty solid fit for what I needed. The owner also configured builds for container images, so it was fairly easy to configure a custom chart to run an instance on my internal tools cluster.

    The client UI is pretty easy to use, and the ability to add extra fields to the different modules makes the solution very extensible. I was immediately impressed and started entering information about vendors, filaments, and spools.

    Enhancing the Solution

    Since I am using a Bambu Labs printer and Bambu Studio, I do not have the ability to integrate Bambu into Spoolman to report filament usage. I searched around, but it does not seem that the Bambu reports such usage.

    My current plan for managing filament is by weight the spool when I open it, and then weighing it again after each use. That difference is the amount of filament I have used. But, to calculate the amount remaining, I need to know the weight of an empty spool. Assuming most manufacturers use the same spools, that shouldn’t be too hard to figure out long term.

    Spoolman is not quite set up for that type of usage. Weight and spool weight is set at the filament level and cannot be overridden at the spool level. Most spools will not be exactly 1000g of filament, so the need to track initial weight at the spool level is critical. Additionally, I want to support partial spools, including re-spooling.

    So, using all the Python I have learned recently, I took a crack at updating the API and UI to support this very scenario. In a “do no harm” type of situation, I made sure that I had all the integration tests running correctly, then went about adding the new fields and some of the new default functionality. After I had the updated functionality in place, I added a few new integration test to verify my work.

    Oddly, as I started working it, I found 4 feature requests in that were related to the changes I was suggesting. It took me a few nights, but I generated a pull request for the changes.

    And Now, We Wait…

    With my PR in place, I wait. The beauty of open source is that anyone can contribute, but the owners have the final say. This also means the owners need to respond, and most owners aren’t doing this as a full time job. So sometimes, there isn’t anything to do but wait.

    I’m hopeful that my changes will be accepted, but for now, I’m using Spoolman as-is, and just doing some of the “math” myself. It is definitely helping me keep track of my filament, and I’m keeping an eye on possible integrations with the Bambu ecosystem.

  • Upgrades and Mermaids

    What I thought was going to be a small upgrade to fix a display issue turned into a few nights of coding. Sounds like par for the course.

    MD-TO-CONF

    I forked RittmanMead‘s md-to-conf project about 6 months ago in order to update the tool for Confluence Cloud’s new API version and to move it to Python 3.11. I use the new tool to create build pipelines that publish Markdown documentation from various repositories into Confluence.

    Why? Well, in the public space, I usually utilize GitHub Pages to publish HTML-based documentation for things, as I did with md-to-conf. But in the corporate space, we tend to use tools like Confluence or Sharepoint as spaces for documentation and collaboration. As it happens, both my previous company and my current one are heavy Confluence users.

    But why two places? Well, generally, I have found that engineers don’t like to document things. Having to have them find (or create) the appropriate page in Confluence can be a painful affair. Keeping the documentation in the repository means it is at the engineer’s fingertips. However, for those that don’t want to (or don’t have access to) open GitHub, publishing the documents to Confluence means those team members have access to the documentation.

    A Small Change…

    As I built an example pipeline for this process, I noticed that the nested lists were not being rendered correctly. My gut reaction was, perhaps the python-markdown library needed an update. So, I updated the library, created a PR, and pushed a new release. And it broke everything.

    I am no Python expert, so I am not really sure what happened, since I did not change any code. As best I can deduce, the way my module was built, with the amount of code in __init__.py, was causing running as a module to behave differently then running with the wheel based build. In any case, as I worked to change it, I figured, why not make it better.

    So I spent a few evenings pulling code out of __init__.py and putting it into it’s own class. And, in doing that, SonarCloud failed most of my work because I did not have unit tests for my new code. So, yes, that took me down the rabbit hole of learning about using pytest and pytest-mock to start to get better coverage on my code.

    But Did You Fix It?!

    As it turns out, the python-markdown update did NOT fix the nested list issues. Apparently, all I really needed to do was make sure I configured python-markdown to use the sane_lists extension.

    So after many small break-fix releases, v1.0.9 is out and working. I fixed the nested lists issue and a few other small bugs found by adding additional unit tests.

    Mermaid Support

    For Confluence, Mermaid support is a paid extension (of course). However, you can use the Mermaid CLI (or, in my case, the docker image) to convert any Mermaid in the MD file into an image, which is then published to Confluence. I built a small pipeline template that runs these two steps. Have a look!

    While it would be nice to build the Mermaid to image conversion directly in md-to-conf, I was not able to quickly find a python library to do that work and, well, the mermaid-cli handles this conversion nicely, so I am happy with this particular two-step. Just don’t make me dance.

  • Publishing Markdown to Confluence

    For what seems like a long time now, I have been using RittmanMead’s md_to_conf project to automatically publish documentation from Markdown files into Confluence Cloud. I am on something of a documentation kick, and what started as a quick fix ultimately turned into a new project.

    It all started with the word “legacy”…

    In publishing our Markdown files, I noticed that the pages all had the legacy editor text in Confluence Cloud. I wanted to move our pages to the updated editor, and my gut reaction was “well, maybe it is because the md_to_conf project was using the v1 APIs.

    The RittmanMead project is great, but it has not changed in about a year. Now, granted, once things work, I wouldn’t expect much change.

    So I forked the project and started changing some API calls. The issue is, well, I just did not know when to stop. My object-oriented tendencies took over, and I ended up way past the point of no return:

    • Split converter and client code into separate Python classes to simplify the main module.
    • Converted the entire project to a Python module and built a wheel for simplified installation and execution.
    • Added Flake8/Black for Linting
    • Added a GitHub workflow for building and publishing.
    • Added analysis steps to analyze the code in SonarCloud.io.

    It is worth noting that, at the end of the day, the editor value was already supported in RittmanMead/md_to_conf: You just have to set the version argument to 2 when running. I found that out about an hour or so into my journey, but, by that time, I was committed.

    Making a break for it

    At this point, a few things had happened:

    1. My code had diverged greatly from the RittmanMead project.
    2. Most likely, the functionality and purpose for which the original project was written had changed.
    3. I broke support for Confluence Server.
    4. I have some plans for additional features for the module, including the ability to pull labels from the Markdown.

    With that in mind, I had GitHub “break” the fork connection between my repository and the RittmanMead repository.

    Let me be VERY clear: the README will ALWAYS reflect the source of this repository. This project would not be possible without the contributors to the RittmanMead script, and whatever happens in my respository is building on their fine work. But I have designs on a more formal package and process, as well as my own functional roadmap, so a split makes the most sense.

    Introducing md-to-conf

    With that in mind, I give you md-to-conf (PyPi / GitHub)! I will be adding some issues for feature enhancements and work on them as I can, although, my first order of business will most likely be some basic testing to make sure I don’t break anything as I work.

    If you have a desire to contribute, please see the contribution guideline and have at it!