Wednesday, April 6, 2022

Planning for failure

 This week the new Home Assistant Core 2022.4.0 was released. As with all updated I was eager to update my system after reading the release notes to make sure there was no breaking changes I care about. Additionally I took the opportunity to update the OS to 7.6 from 7.2. That is where the problems started.

Once the updates were complete I noticed an issue, Zigbee and Zwave communication was not working. At this point I realized the most recent full backup I had was from just under a week ago. Normally that wouldn't be a problem, but in the last week I had updated camera configurations, scripts, automations, and added new hardware since the last backup. 

Once I noticed this issue I tried to fix forward by touching the Zwave config that was claiming to be broken  then when that failed I restored from the partial backup automatically created when the Core version was updated. This didn't help. Then I paused and decided I should probably create a back up in the bad state since at least it had my script/automation/device changes. If I failed in restoring at least I could get back to the current status.

I then restored to the last full backup hoping that at least I could regain control of my lights. The restore process worked, but the issue persisted. This left me very confused for a few minutes until I realized that the OS level hadn't changed. Apparently Home Assistant does not consider OS level part of a "full" backup. Luckily they do allow downgrading OS via a simple terminal command, so I was able to downgrade the OS to 7.2 and reboot. At this point my Zigbee/Zwave integrations were working again. 


The next step was to go through the restore process once more, but this time using the backup I had just created after recognizing there was a problem. Luckily Home Assistant allows you to complete a partial restore from a full backup, so I was able to restore everything accept the Zwave configuration that I had attempted to correct manually. 


I know people say not to plan for failure, but in tech you really do need to have appropriate plans in place or you might end up not being able to turn your light off/on. This experience has inspired me to be more patient when it comes to applying updates and to ensure that a backup is done anytime a significant change is done.  

Thanks to this nifty add-on (https://community.home-assistant.io/t/add-on-home-assistant-google-drive-backup/107928) I have automated backups that are stored in my google cloud account on a regular cadence. But that doesn't protect me from making foolish mistakes like I did in this post. If anything, I previously had more caution around changes/updates when I knew the only backups available were ones I manually created. 


No comments:

Post a Comment

Server Monitoring

I have a headless Linux server that I have previously written about for ripping media disks.  While I have had MQTT integrated into the bash...