rfid-tags for presence detection in smarthome

Wednesday, November 28. 2018

Pressence detection is vital in a smarthome setup, lights should go out when I leave and on when I come home.  But presence detection is not trivial, you can not rely on PIR motion sensors since they often do false positives. You can not rely on pinging the smartphone when it connects to the WLAN because phones have sleep modes to save energy and you'll find your phone regarded as absent when it sits next to you on your desk. Even owntracks and tasker provide solutions which work sometimes and fail later. .

And, more often then I like, I have to search for my keys before leaving the house. Now this is how I try to catch two flies with one stone:

I bought a RC522 rfid reader to integrate it into my smarthome IoT.
Most of the stuff you can find for those, both software and things, follows the idea of access control.
That is:

  • the thing with the reader waits in front of a door and controls a lock. User comes, holds a keycard at the reader, is denied or allowed, puts the keycard back in their pocket and enters...

My use case is different, more like what you may find in a hotel:

  • the thing with the reader waits behind the door and the user is supposed to insert the key card into the reader and leave it there. The reader identifies the key card and informs the smarthome controler that the owner of that key card is at home so it's time to turn on the lights and enable all the features that will be turned off again when the key card is taken out of the reader, later.

For this to work the key card (those blue drop shaped ones with a key chain ring you get with the reader) has to be in close range of the reader in a stable position, in other words, some sort of pocket is needed.

 

rfid-boxThe box is a remix of something I found searching thingiverse and sits at https://www.thingiverse.com/thing:3242266

It's printed in fast mode and with elements into all directions the slicer added lots of support which was hard to get off, you can see ht the box suffered from it.

Anyways, this cheap RC522 rfid reader which set me back by about 6€ sits in this box I printed, with a pocket to insert the chip and thus hold my keys at a defined place. The reader, connected to an Nodemcu V2 microcontroller identifies the chip, thus knows it's me who is at home and reports this to the smarthome controller. When I leave the house and take my keys the reader creates a new event and the system knows I'm out.

On the software side again there are plenty of example sketches for access control where the reader idles until you put a key card in its range, then the card is analysed, the result is given and the thing idles on. There is no event when the card is taken away.

So I had to write my own sketch for that, too. Again it is a remix.
It runs on an arduino clone and it creates an event when a known key card is entered, and another event when that card is taken out.

Since my smarthome is controlled by openHab2 which has a very usable rest api there is some code to report those events to it.
The sketch is part of the thingiverse file bundle linked above.

openHab2 startup chaos

Thursday, October 25. 2018

starting and stopping openhab2 is a messy affair, filling the logs with loads of Java's verbose exception messages and since the order of loaded models is basically random things may or may not work as intended when items have not been loaded or rules not executed that are assumed to be there.

As it was, manual corrections and reloads were needed after each restart.

There is an elegant workaround discussed at community.openhab.org/t/cleaning-up-the-startup-process-renaming-rules-windows-possible/38438/9 which uses systemd's ExecStartPre and ExecStartPost commands to deactivate all rules before starting openhab, and then reactivate them one by one when the system is up and has it's feet on the ground.

It did not work exactly like described there for me, but I still found a solution that has cleaned up the messy startup here.

A small bash script does the renaming. I put it in /etc/openhab2 next to openhab's configuration into a new folder /exec-scripts:

#!/bin/bash
#######################
#
# clean up the start process 
#  starting rules in a sorted manner after openhab2 got it's feet on the ground
#  called from systemd pre start 
#  to rename *.rules away initially 
#
#  synopsis: move_rules_at_start.sh org-extension new-extension (POST)
#  
#  first call is from
#  /etc/systemd/system/openhab2.service.d/override.conf
#
#  called from the running openhab again to rename them back one by one
#
# $3 allows to distinguish between pre and post action 
#
# REF: https://community.openhab.org/t/cleaning-up-the-startup-process-renaming-rules-windows-possible/38438/9


ORG=$1
NEW=$2
THX=$3
DUR=1
IGNORE=005_start.rules

for f in /etc/openhab2/rules/*.${ORG};
do
    CURRENT=$(basename $f)
    if [ "$CURRENT" == "$IGNORE" ]    
    then
        echo "ignoring $IGNORE"
    else
        OLDFILE=$f
        NEWFILE=${f%$ORG}$NEW
        mv "$OLDFILE" "$NEWFILE"
    fi
    # let some time between each load
    if [ "$THX" == "POST" ]
    then
        /bin/sleep $DUR
    fi
done

# some things left on startup
if [ "$THX" == "POST" ]
then
    /usr/bin/touch /etc/openhab2/things/tradfri.things
fi


Since the openhab2.service file is part of the package and thus is replaced with each update modifying the .service file would need to be repeated whenever a new version gets installed, but, systemd provides a way to override a service definition. systemctl edit openhab2.service creates an override dir and opens the editor to allow creating a service file that is not replaced by the next update.

Now don't try to copy the entire .service file, systemd will complain duplicated statements. My override.conf looks like this:

[Service]
ExecStartPre=/etc/openhab2/exec-scripts/move_rules_at_start.sh rules rules_

The third element is a .rules file in openhab2 that triggers the renaming of all the other .rules once the system is running. It is excepted from the renaming and I named it 005_start.rules. This works for me:


var rulesDelayed
var nada

rule "triggered by system start"
when 
    System started
then
    var duration = 1
    rulesDelayed = createTimer(now.plusMinutes(duration), [|
					logInfo("rulesDelayed", "Timer expired and start")
					nada=executeCommandLine("/bin/bash /etc/openhab2/exec-scripts/move_rules_at_start.sh rules_ rules POST ",90000)
                    logInfo("rulesDelayed", "result: "+nada)
					rulesDelayed = null

				])
    logInfo("rulesDelayed", "Timer set on "+duration+" min")
end


The bash script works the list of rules -files in an alphanumerical order so a naming scheme like praefixing all .rules with a three digit number finally allows to control the order of rules loading.

Startup is much faster now, less or no exceptions and a much cleaner log :-) Openhab is a Java application and those exceptions are super-ugly and come with a performance penalty.

You'll notice that the bash script supports an optional 3. parmeter to distinguish between pre- and post-action and that a certain config file is touched after the renaming. touch-ing triggers a reload of that config file and this was a workaround for a bug in the tradfri-addon, the bug may have been addressed by now.

TimeStamp in mosquitto.log

Monday, September 24. 2018

 mosquitto gibt jedem Log-Eintrag einen TimeStamp im für Maschinen recht praktischen Aekunden-seit-Epoche - Format. Für menschliche augen ist diese Angabe eher sperrig.

Mit perl und ccze (macht's bunt) kann man die Ausgabe etwas aufhübschen:

tail -n2000 -f  mosquitto/mosquitto.log |  perl -pe 's/(\d+)/localtime($1)/e'| ccze -m ansi

 

Presence Detection in WLAN

Sunday, September 9. 2018

 I was not content with the network/pingdevice based presence detection openhab2 offers. While it finds and detects smarphones connect to the wlan, it soon gets beaten by some energy-savong sleep modes, so the device does not ping. It may still respond to arping, though.

The following script is used on a linux router with several interfaces where a device miight be reachable. It can be configured from a settings.ini, reads it's sc an targets from a different ini file and reports it's findings via openHab's REST api.

settings.ini:

[settings]
baseUrl=http://koel.intern:8080/rest/items/
interval=30
interfaces=eth0,eth2


baseUrl points to the REST api of the openHab2 server, interfaces is a comma separated list of the interface/s that should get scanned and interval gives the number of seconds the scanner will sleep between to scans

scan.ini:

[feng]
item=presence_feng
MAC=C0:EE:FB:43:A8:ED

[goldAmmer]
item=presence_goldammer
MAC=192.168.123.11


[ente]
item=presence_ente
MAC=192.168.123.99


Each device has its own section with the section name acting as the unique id of the device, item is the openHab item (expected to be a Switch) that should get the result (ON or OFF) and MAC has the MAC or IP of the device 

The scanner is done in python (2.7)
arpscan.py:

#!/usr/bin/env python
#############################

import ConfigParser
import string
import os
import subprocess
import time
import sys

conf={}
devices={}
interfaces=""
scantext=""
grace=3

def get_script_path():
   return os.path.dirname(os.path.realpath(sys.argv[0]))
        
def LoadConfig(file, config={}):
   """
   returns a dictionary with keys of the form
   <section>.<option> and the corresponding values
   """
   config = config.copy(  )
   cp = ConfigParser.ConfigParser(  )
   cp.read(file)
   for sec in cp.sections(  ):
       name = string.lower(sec)
       obj  = { }
       for opt in cp.options(sec):
           obj[string.lower(opt)] = string.strip(
               cp.get(sec, opt))
       config[name]=obj
   return config

def ScanThings():
       scantext=""
       for interface in interfaces:
           p = subprocess.Popen(["arp-scan","-l","-q","-r","3", "-I", interface], stdout=subprocess.PIPE)
           output, err = p.communicate()
           scantext += output
                
       for key in devices:
           device=devices[key]
           url=conf['settings']['baseurl']+device['item']
           if device["mac"].lower() in scantext.lower():
               print key+" ja"
               val = "ON"
               device["tap"]=0
           else:
               print key+" nein"
               device["tap"] = device.get("tap", 0) + 1
               if device["tap"] <= grace:
                   print key+" tap: "+str(device["tap"])
                   val="na"
               else:
                   val="OFF"
                    
           if val != "na":         
               os.system('/usr/bin/curl --silent --header "Content-Type: text/plain" --request POST --data '+val+' '+url)


conf=LoadConfig(get_script_path()+"/"+"settings.ini", conf)
devices=LoadConfig(get_script_path()+"/"+"scan.ini",devices)

interfaces=conf['settings']['interfaces'].split(",")


while 1:
   ScanThings()
   time.sleep(float(conf['settings']['interval']))

 

           

 

           

The script is most useful when run as a daemon. You can easily do this with systemd which uses .service files found in /lib/systemd/system/

arpscan.service:

[Unit]
Description=Run arpscan device detection
After=multi-user.target
 
[Service]
Type=simple
ExecStart=/usr/bin/python /usr/local/bin/presence/arpscan.py
Restart=on-abort
 
[Install]
WantedBy=multi-user.target

The scanner reads settings and scan data, runs an arpscan on each configured interface and finally searches the specified MAC (or IP, just one of the two is needed) in the result.
it needs the linux arp-scan package to be installed, which you can test and confirm by running

arp-scan -l -I eth0

from the commandline. It needs to be run as root.

Some devices may show up oscillating between on and off, to get more stable results the script waits for a number of fails before the "OFF" result is sent. This can be finetuned by the value for grace. 

 

 Technologiewechsel, die Umstellung alter ISDN-Anschlüsse auf Voice over IP und damit verbunden von DSL auf VDSL rollt planmässig und wie eine Dampfwalze und nun war mein Anschluss auch dran. Ein paar mails, ein Brief, ein Päckchen mit der weissen Homebox-Flunder. Dann war der Tag da und obwohl mir der Abschluss der Ustellung erst für 17.30h angekündigt war, war um 10 schon alles über die Bühne. Kabel umgepfriemelt, einen ZUgangscode eingetippt, paar Minuten gewartet und dann war die flunder im Netz. 

Soweit so gut, nun zum Manual und die Kiste als Modem einrichten, wie es im Zyxel -Handbuch für das Teil beschrieben ist - bloss: die im Handbuch beschriebenen Menueoptionen gab es auf meiner Kiste gar nicht. Einfach nur das Modem austauschen und meine eigene Linux-Büchse dahinter als Router weiterlaufen lassen wie seit 20 Jahren schon - dass ging nicht. 
pppd passthrough, den die Flunder laut Manual anbietet, kann man nicht einstellen und so musste ich erst mal meine firewall-Konfig zerschiessen, um überhaupt meinem Lan den Weg ins Netz zu öffnen. Und an dem nun fehlenden event des allmorgendlichen pppd-Neustarts (um die Zwangstrennung zu einer mir genehmen Zeit stattfinden zu lassen) - das Update meines dynamischen DNS bei dns.he.net, das Update des IPv6 over Ipv4-Tunnels, überhaupt der Tunnel, Kollateralschäden. Für IPv6 hat die Kiste übrigens Einträge, aber die bleiben alle leer. Das ist Telefonica oder ihrem Vorleister Telekom noch zu neumodisch...

Dann verriet mir nmap, dass alle ports gefiltert seien. Die Kiste lässt mich aber die Einstellungen der Firewall weder ansehen noch ändern - alles was man machen kann, ist Port-Forwarding für TCP und UDP einzurichten. Dabei zeigt sich, dass 7170/TCP und 5060/UDP reserviert sind, auf letzterem läuft SIPund ersterer Port wird für Fernwartung genutzt bzw. freigehalten.

OK, damit waren nun die Services auf dem Gateway - der Proxy zu OpenHab für remote-Zugang zum SmartHome, der MQTT-Server, Da kleine tool auf dem Apache - wieder von Aussen zugänglich. Der IPv6-Tunnel blieb aber tot.

Das ist alles recht unerfreulich, ich kann schon den Nutzen davon sehen, so eine Homebox 'Omi-sicher' auszuliefern, aber auch erfahrenere Kunden so in einen weissen Käfig zu sperren - bei Hansenet (selig) oder auch Alice hätte es das nicht gegeben.

Recherche im Netz fand keinen workaround, keinen Trick zum Freischalten, sondern immer nur den Verweis auf die Fritzbox 7490. Schon bestellt.

Aber, nach all dem Gemecker: es gab auch eine erfreuliche Überraschung, und das ist die Bandbreite.Die war zu Alice-Zeiten mal bei stolzen 17,x MBit/sec im Download gewesen und in den letzten Jahren immer mehr verkümmert, letzthin war ich schon froh, wenn es 13 MBit/sec waren und nicht nur 10.
Ja, und nun kraxelte der Graph in meinem Bandbreiten-Check plötzlich bei 30 herum, und der Download knapp bei 10.

mit iPerf, scp und diversen Bandbreiten-Checkern nachgemessen (was bei letzteren Ergebnisse mit bis zu 54MBit anzeigten), openHab zeigte den Wechsel so:
2 bis 2,5 fachen Download und zehnfache Bandbreite im Upload. 

Man kann auch mal was gut finden..

bandbreit

Wie ich später merkte, war es tatsächlich der WLAN-Durchsatz, der den gemessenen Upload auf 35 kbit/s begrenzte, mit einem Lan-Kabel habe ich nun ~47, also auch im Upload das 3-4 - fache.

(Page 1 of 35, totaling 175 entries) » next page