gmaslowski.com

on software development

BLOG / PROJECTS / ABOUT
Posts /

My Domoticz deployment setup

Back to posts /
Twitter Facebook Google+
03 / Jan  2019

Disclaimer. I am using home automation software and hardware at home. But that’s not the only way for me to control my appliances. I always make sure that in the case of any home automation failure I am still able to manually control them.

Everyone interested in some home automation projects surely stumbled upon Domoticz at some point. In general there are other solutions as well, but I won’t describe them here.

I’m using Domoticz more than two years now and I’d like to share in this post how do I currently manage hardware and software in the scope of my simple home automation. I’ll also try to explain why my configuration is setup how it is. But before going into solutions, let me explain my requirements and reasons for them to exist.

What are my requirements?

Deployment automation

There were times, when I was deploying (or I should rather say managing deployment) manually. Since the beginning, Domoticz at my home was running on a RPI, which made the process look more or less like this:

This process was maybe not that hard, however it made me remember couple of things like IP addresses, scripts to run and backups to create. In order to automate this process I decided for using Docker images and Docker Swarm for easing up the process.

Docker abstraction

I started with searching for an already existing image for arm architectures. I cannot remember if I found something useful, but I had some criterias like for example cyclic/periodic updates. I’m sure I haven’t found any image which had this in place already. So I decided to use my doubtful skills to automatically create and publish a docker image for every newest published Domoticz version. With some help of Travis CI I setup a simple GitHub repository - gmaslowski/rpi-domoticz (post about building docker images with Travis CI can be found here). The artifact which is produced from it (always daily with newest Domoticz beta version, which is basically the current development and I think the only reasonable version to use) is published to DockerHub registry - gmaslowski/rpi-domoticz. That makes it really convenient to use. By default, with little help of Docker manifests, the image targets arm and amd64 architectures. The image is the core prerequisite to use dockerized version of Domoticz, which can be run manually with docker run, docker-compose or deploy into Docker Swarm.

Container orchestration

Having dockerized Domoticz, now came the time to deploy it. At home I run Docker Swarm. For some it might be an overhead, for some it might be weird to have a cluster at home… But I find it quite convenient to deploy software at home for purposes which I don’t want to have in the cloud. For example, file storage like Dropbox is too expensive (at least for me) and I try to avoid as much as possible storing private data, images, videos on the net. This makes me running Nextcloud. Another good reason for running Swarm is that I work in IT. I like to, and I have to be somehow up to date with technology. At work in current project we switched from Docker Swarm to Kubernetes already, so I find it convenient to run Docker Swarm at home.

Having a Docker Swarm cluster in place, and with some help of GitLab the configuration of the deployment looks like this:

version: '3.7'

services:
 
  domoticz:
    image: gmaslowski/rpi-domoticz:4.1030
    command:  [
      "/domoticz/domoticz",
      "-www", "8080",
      "-dbase", "/domoticzdb/domoticz.db"
      ]
    volumes:
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
    deploy:
      replicas: 1

With .gitlab-ci.yml:

stages:
  - software

.job_template: &home-job
  tags:
    - deployer
  only:
    - master
  when: always

software:
  <<: *home-job
  stage: software
  image: docker:18.09
  script:
    - docker stack deploy -c software/automation/automation.yml home

Here it can be seen that, updating my Domoticz deployment boils down to changing this line image: gmaslowski/rpi-domoticz:4.1030 to use the chosen Docker image version and push the change to master. GitLabCi and Gitlab Runner will make sure that new version gets deployed.

And that covers the topic of deployment automation.

Hardware placement and restrictions

As stated previously I have some RPIs which I’d like to use to run Domoticz. Basically what do I need? RPIs I already have, so the thing missing is making sure that the configuration (docker installed, user management etc.) is the same (or really similar) on every RPI and other hardware in the cluster. Additionally I’d like to avoid as much as possible any manual actions which require from me remembering any configurations. A fairly good example of that would be my router and AP configurations, when I really messed up a lot when after a factory upgrade my AP needed replacement. Because of no configuration stored anywhere but the AP I needed to recreate many settings (WiFi, accesses, QoS) from my head. A good point would be to automate that configuration and make applying it idempotent - I’m working on that and I think it can be a good option for another post. Let’s get back to the main topic.

Attic

The hardware runs in the attic. I have the tendency to keep my travelling ;) there limited as much as possible. I wan’t to be able to (to some extent) manage my hardware remotely. Of course, it’s not possible to fix a physical error remotely, but removing the need to have a physical access to hardware is the goal.

Ansible configuration

Before any more description, I’d like to made I thing clear, even for myself. For me it’s not about using Ansible - for me it’s about having infrastructure/configuration-as-a-code approach.

My tool of choice for the time beeing is Ansible. Why? Again, I’m in IT :D and:

I will not put all of my ansible configurations here, because the number of them is growing with every day - so much fun :). I will focus on what I configure with it:

High Availability

It happened to me couple of times that my home automation software was not fully working. The reasons were various. Let me enumerate them:

RPIs

In my experience, relying on one RPI for running Domoticz is not the way to go. Not only because of occasional failures, but also sometimes I’d like to detach a RPI from the cluster and use it for something else. So I already have 4 of them being able to run (amongst others) Domoticz. The only thing I need to make sure is that Docker Swarm places domoticz service only on them. That’s easy as applying a label onto each RPI Docker node i.e.: rpi=true:

greg@pc001:~|⇒  docker node ls -q | xargs docker node inspect -f ' []: '
ljvzdp36900i75poqgxww5bic [rpi001]: map[rpi:true]
qtrmc85j1dvw9w6jh7d1mlyn1 [rpi002]: map[rpi:true]
quu4g00qb669bxe3ols5jtuo5 [rpi003]: map[rpi:true]
al2orpsz28n0wnky2xxaa4tps [rpi004]: map[rpi:true]

And then configure the placement in the domoticz service docker descriptor file:

domoticz:
  image: gmaslowski/rpi-domoticz:4.1030
  command:  [
    "/domoticz/domoticz",
    "-www", "8080",
    "-dbase", "/domoticzdb/domoticz.db"
    ]
  volumes:
    - /etc/timezone:/etc/timezone:ro
    - /etc/localtime:/etc/localtime:ro
  deploy:
    replicas: 1
    placement:
      constraints:
        - node.labels.rpi == true

Storage

Remember that Ansible configured NFS? That’s good :). Docker has the option to mount a volume from a defined NFS storage. And that solves my storage availability problem. I have a server with mirrored disks for storing my private data. Additionally for the purpose of running Domoticz I setup a NFS on it (with Ansible of course). So now I don’t need to worry on which of the RPIs Domoticz starts - it will always use the same storage hence the same database. No need to synchronize or to copy data. How cool is that?

Here is the full snippet of my Domoticz deployment descriptor with NFS attached volume:

version: '3.7'

services:
 
  domoticz:
    image: gmaslowski/rpi-domoticz:4.1030
    command:  [
      "/domoticz/domoticz",
      "-www", "8080",
      "-dbase", "/domoticzdb/domoticz.db"
      ]
    volumes:
      - nfsdb:/domoticzdb/
      - nfsscripts:/domoticz/scripts/
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.labels.rpi == true
                                           
volumes:
  nfsdb:
    driver: local
    driver_opts:
      type: nfs
      o: addr=<nfs.ip>,nolock,rw
      device: ":/path/to/domoticzdb"
  nfsscripts:
    driver: local
    driver_opts:
      type: nfs
      o: addr=<nfs.ip>,nolock,rw
      device: ":/path/to/domoticz/scripts"

Besides I can remove (for other purposes) up too 3 RPIs from the cluster and still be sure that my home automation works. Yay!

Recap

Let’s try to summarize this post in one sentence.

I automatically deploy Domoticz in an easily recoverable RPI cluster with external RAID-1 storage.

Wow. That’s a really short TL;DR version. To visualize a little bit of what I was describing please have a look at this picture: Deployment Setup

What’s still missing?

I’m curious how people in generally deal with Domoticz, or any Home Automation solutions deployment, so that they keep working constantly. Do people care about HA, or just tackle failures whenever they appear?