Introduction
Victron Energy↗ is a private Dutch, company that makes power inverters, solar charge controllers, and various other devices associated with off-grid energy systems. Their systems are often found in boats, RVs, vanlifer vans, and homes. They were one of the first companies that I discovered when I began looking into power inverters. After doing much more research, I decided to go with their produts to create a couple of energy backup systems for my home. A big part of the reason that I decided to go with Victron was that they have a philosophy close to my heart: they believe in making reliable products that last for decades. The other main reason was that I am rather obsessed with Home Assistant. My choice of system would have to integrate into Home Assistant, and Victron had a way to enable MQTT publishing. MQTT is a common way to integrate devices into Home Assistant, so I thought this would be an easy way forward. Unfortunately, that is not what I found.
Background
How did I get here? During a recent power outage, I noticed that one of my uninteruptable power supplies (UPS) shut down with the power. It was, of couse, the one powering my two servers and not my networking gear that is much more resistant to this sort of thing. As this is a UPS’s raison d’être, I looked into what new batteries would cost since I had had this device for a 5ish years and I knew the batteries had a shelf life. I didn’t like what I found. The solid lead-acid (SLA) batteries the UPS used were expensive, short lived, and it was difficult to find non-generics. I knew that battery technology had been improving dramatically recently, so after much searching, I figured out that I should be able to create my own UPS using off-the-shelf components for less than it would cost from one of the big vendors for a “proper”, enterprise UPS. I got to work creating a test system for another project that I had been meaning to tackle for years: a battery backup for my home’s sump pump. I did this first for a number of reasons:
- If it failed, it wouldn’t be failing for computer systems that need constant power. A sump pump will be fine for a couple of minutes without electricity in almost all cases.
- The damage done by not having a sump during torrential rains could easily be greater than my servers having a hard power off (i.e. cost of new basement > cost of 2 servers).
- The backup system would be able to backup my homes’s furnaces in the winter during a power outage. The previous winter had an extended (8 hour) outage during a particularly cold spell and my house can have pipes freeze when it’s well below freezing.
- There were fewer roadblocks in completing this sump backup system since I wanted to drywall the server closet before continuing.
My Cerbo-GX in its final position
Given that I had decided to go with a Victron Multiplus II inverter, I purchased a nearly-necessary device called the Cerbo GX for monitoring the system’s components. It is a little ARM computer with an ESP32 module for Bluetooth and WiFi running Victron’s custom, OpenEmbedded Linux distribution. Given that I was already going to spend too much money on the aluminum T-track for the cabinet, the breaker panel kit-out, and the somewhat pointless addressable RGBW lighting, I decided against one of their nice screens to go with the system. This obviously meant that I needed a good way to get this data into Home Assistant so that I could use it for my day to day monitoring. Victron’s documentation claimed that I could enable MQTT directly in the Cerbo GX device, so I thought all I would have to do was enable it and configure some annoying MQTT device YAML files in Home Assistant to digest the necessary topics. I had done similar things before and this didn’t seem any harder.
Victron’s MQTT Server
While I was testing the system on my workbench, I enabled MQTT and started trying to peek around at the topics to find the ones that I wanted to use. I quickly figured out that the Cerbo GX didn’t use an MQTT broker, it was an MQTT broker. It runs a Mosquitto broker server right on the device. This meant that I first got to figure out how to republish the Cerbo GX in my Home Assistant broker. This was complicated by the fact that I was testing the EMQX add-on at the time to see if I wanted to switch away from Mosquitto for some Erlang visibility goodness. After giving up with EMQX for “I hate this configuration” reasons and going back to Mosquitto for simplicity, I only had negative opinions form over what was going on. However, the Home Assistant broker was republishing the Cerbo GX broker, so I thought I was through the worst of it.
It was time to start exploring the available information in MQTT. The first program that I used was MQTTX from EMQ since I did like their Home Assistant add-on even if I didn’t end up using it. It was an Electron app, so when it bogged down and consumed all the memory on my machine (64GB by the way!), I just chalked it up to Electron and looked for something else. Next I tried MQTT Explorer, an older piece of software that I had used before with some success. It was annoyingly hard to install on Fedora Linux, but I think I eventually installed the Snap version, or AppImage, or Flatpak, I don’t remember or care. Anyway, that bogged down just as bad as MQTTX once I subsribed to the Cerbo GX’s MQTT topics. At this point, I finally figured out that the Cerbo GX was publishing on what seemed like thousands of topics every second. The MQTT exploring software just couldn’t keep up and the GUI would become unusable after a couple of seconds.
I was a little annoyed at this point. How was I going to figure out which topics to subscribe to? By guessing? By reading forums and hoping someone posted about the topic path? Hell no, I’m an engineer, I don’t do that. It was time to dig in and figure this out…so I decided to proceed with building the final system and to figure this out later.
A Second Attempt
A couple of weeks later, it was done, working, and I had to get back to this annoying topic so that I could set up my system in Home Assistant. I had decided that using the MQTT broker in the Cerbo GX wasn’t a good time, so I toyed with using Modbus over TCP/IP instead (seemed annoying), and eventually did find a good example↗ for integrating Victron MQTT into Home Assistant. I didn’t love it, but it worked…until I wanted the battery BMS’s state of charge in addition to the system state of charge (yes, they are different and I cared to know the difference). I, once again, didn’t know the topic name. Back to the same problem mentioned above, to try to find this topic name would be a painful, long process.
I had read a number of times that you could use Node-Red with the Cerbo GX. I knew of Node-Red through Home Assistant. I had glanced at it years ago as an alternative way to write automations and had decided at the time to try the Home Assistant buitin automations first. I never went back to it because the Home Assistant automations did everything I needed, albeit with some awful YAML configuration and templating at times. But hey, the automations worked, were well documented, and another layer of software to a system is adding many failure points.
Since I was stuck with the Victron MQTT topic finding problem, I decided to give Node-Red a try this time. I figured, worst case, I’d learn something. You have to actually install a different version of the Victron Venus OS (the “large” version) to use Node-Red, so the first thing that I had to do was get that installed. This part of the process was painless and everything on Victron’s side worked the first time. Everything could be done using the web-GUI of the Cerbo GX.
Since I had never used Node-Red before, I had to learn some more about it. I am a programmer by trade. I think about software like a programmer since I know what it must be doing under the hood. I spend more time in the weeds trying to figure out how things work, what’s atomic, and how to best structure data. My initial impressions weren’t good or bad. I really like the visual representation of the automation “flow” or structure, but also found it a little verbose. Despite that verbosity, it was limited in what it could express. Fortunately, Node-Red has a funcion node that allowed for writing straight Javascript, meaning that I could just write some code when I got stuck with the available features. Node-Red seemed like a good product, unsurprisingly, as it is used throughout the industry.
Next, I tried looking around in the Victron plugin that is already bundled in Venus OS for the available system values. This is where I got excited. The values were all there in the available nodes given by the Victron plugin. I would pick the type of value that I wanted (system, battery, etc), and a dropdown would appear with the available values. Almost every value that I would try would populate within Node-Red. This is exactly what I wanted: a way to see which values were available and to quickly check their current values. After playing around for a while, I decided that I had finally found a way forward that I liked.
The Architecture
I settled on this high-level architecture:
- Global state of all current Victron system values (that I care about)
- MQTT full-state updates occur at some interval (say 1 second)
- Home Assistant can purge repeated values
- Removed the need for separate topics for each, individual state
- MQTT device discovery command topic
- All sensors are configured out-of-the-box in Home Assistant
- Inverter mode and grid current limits can be set
- Switches are added to enable/disable automations
- System handles it’s own automations
- Created automation to cycle the batteries weekly
- Created automation to turn on cabinet lights when the cabinet door is opened
This leads to a clean implementation in Node-Red where I can easily see what is going on at any moment and add new data points with minimal configuration. I just have to add the new data to the global state and define it in the device config. The entire device is defined in one node in a JSON object within Javascript code. This might be the first time that I am happy to be programming in Javascript. The MQTT updates only update the data values that I care about and are limited to only once per second (or any other value that I like). This implementation so much cleaner than the Home Assistant MQTT file and keeps all of the configuation for the Victron battery backup within the system itself. It now has one of my favorite features of a system: self-configuration.
My current setup now has this flow of information: Cerbo GX dbus -> Node-Red -> MQTT broker -> Home Assistant. My only niggling concern with this setup is: what if Node-Red crashes? My answer after only a couple of months is that it has been rock solid. The only problems that I have encountered, so far, were self-inflicted, networking heartaches (I think I found a bug in Ubiquiti’s new policy firewall, but hell if I could reproduce it after fixing it). Only time will tell. The Cerbo GX is also web-accessible through my local network, so there is a builtin fallback monitor and I can restart Node-Red through there if needed. After struggling with this system for a couple of weeks, I am happy with this setup. All of my data is being logged in Home Assistant and basically every aspect of the data stream is configurable by me through Node-Red.
A quick note: I tried the Home Assistant Node-Red palette (plugin) and found it slow and cumbersome within Node-Red on the Cerbo GX. One explaination for this is the power of the Cerbo GX itself, I don’t know what’s in it other than an ESP32 chip for WiFi/Bluetooth. I also later found out that I had some firewall problems, so that might have been contributing. Once I figured that I could use MQTT directly in Node-Red and it didn’t require another plugin, I didn’t see the point of the Home Assistant plugin and removed it. It requires a token, a bunch of configuration, was a lot slower, and failed a lot more than the MQTT nodes buit into Node-Red. I’m not saying don’t use it, it’s probably the best way to add certain things without added configuration into Home Assistant, but I’d approach it skeptically since it seems a little flakier than just publishing on MQTT topics. I’ve now use this palette within the Home Assistant add-on since I’ve mostly moved to Node-Red there as well, and it works great in that context. It’s running on the same machine and pre-configured out of the box, so it’s just a better experience.
Final Recommendation
In summary, my position is now this: using Node-Red within the Cerbo GX to publish data though MQTT is a cleaner way to publish from the Victron system than trying to consume the buitin MQTT data directly. Using this method, we can have complete control of the information being published, can easily search what information is available, see the real-time state, create a device configuration for the system, and expand into other automation that only affect the Victron system. Node-Red is a quality product, and using it within the Cerbo GX allows for much greater and customizable expression of the Victron ecosystem. `