I live at a place with a well developed public transport network, within 600 meters from my home there are 1 subway station, 2 city train stations, 3 bus-stops - so there are many alternatives and at times it is not easy to decide which way to turn when I leave my home to get the fastest connection.

They have those info boards at many stations with a count down until the next bus or train arrives. I wanted such a thing for myself, with all the nearby stations on it to check on a glance. And as I build my own smarthome I naturally wanted it to be a part of that.

Well, it can be done:

to build this it took:

  • an access key and some documentation to the database of public transport. VBB, the regional public transport org, gives it out for free if asked nicely. I assume other transport organisations will have something comparable.
  • a daemon running on a computer in the own network which fetches the data from the database, I did that in python
  • mosquitto running on a server in the home network to communicate the json to the smarthome
  • rules and items in openHab2 to receive, parse and format the connection data
  • HTML to format and display the connections into a habPanel page like the image above shows

 Now, the details:

  • The python code to fetch the data from the database:
     
#!/usr/bin/python
# -*- coding: utf-8 -*-

# 18.5.2019, v0.1

import RPi.GPIO as GPIO
import time
import os
from datetime import datetime
from datetime import timedelta
import urllib, json
import paho.mqtt.client as mqtt
import paho.mqtt.publish as publish
import sys
reload(sys)
sys.setdefaultencoding('utf-8')


baseUrl="https://demo.hafas.de/openapi/vbb-proxy/"
suffix="&accessId=get-it-from-vbb&format=json"


toScreen=1

duration=25

selection=[u'900060101N',u'900054105O', u'900054105U',u'900060107N',u'900060153N',u'900061153N',u'900060101S',u'900054105W',u'900060107S',u'900060153S',u'900061153S']


# mqtt
pubTopic="ext/vbb/timetable/state"
bannerTopic="ext/vbb/timetable/banner"
cmdTopic="ext/vbb/timetable/cmd"

mqBroker="192.168.150.1"
banner=""

boost=False
boostMin=5
noBoostMin=4
boostCount=0
slapCount=0
inProgress=False

def reset_halten():
        global halten
        halten={
        u'900060101N':{'lat': 52.47, 'lon': 13.340594, 'dist': 348, 'walk':5.0, 'dir':u'900600003', u'dep':[], u'to':'hin', 'name': u'S Friedenau (Nord)', 'extId': u'900060101'},
        u'900060101S':{'lat': 52.47, 'lon': 13.340594, 'dist': 348, 'walk':5.0, 'dir':u'900063101', u'dep':[], u'to':'her', 'name': u'S Friedenau (Süd)', 'extId': u'900060101'},
        u'900054105O':{'lat': 52.478099, 'lon': 13.342878, 'dist': 599, 'walk':8.0, 'dir':u'900054104', u'lines':'S42,S46,S45', u'dep':[], u'to':'hin', 'name': u'S Innsbrucker Platz (Ost)', 'extId': u'900054105'},
        u'900054105W':{'lat': 52.478099, 'lon': 13.342878, 'dist': 599, 'walk':8.0, 'dir':u'900044202', u'lines':'S41,S46,S45', u'dep':[], u'to':'her', 'name': u'S Innsbrucker Platz (West)', 'extId': u'900054105'},
        u'900054105U':{'lat': 52.478099, 'lon': 13.342878, 'dist': 599, 'walk':8.0, 'dir':u'900055101', u'dep':[], u'to':'hin', 'name': u'U Innsbrucker Platz', 'extId': u'900054105'},
        u'900060107N':{'lat': 52.47525, 'lon': 13.34313, 'dist': 280, 'walk':3.5, 'dir':u'900054105', u'dep':[], u'to':'hin', 'name': u'Cecilieng\xe4rten (Nord)', 'extId': u'900060107'},
        u'900060107S':{'lat': 52.47484, 'lon': 13.34302, 'dist': 230, 'walk':3.1, 'dir':u'900060104', u'dep':[], u'to':'her', 'name': u'Cecilieng\xe4rten (Süd)', 'extId': u'900060107'},
        u'900060153N':{'lat': 52.47086, 'lon': 13.34385, 'dist': 210, 'walk':2.8, 'dir':u'900054105', u'dep':[], u'to':'hin', 'name': u'Rubensstr. (Nord)', 'extId': u'900060153'},
        u'900060153S':{'lat': 52.46971, 'lon': 13.34377, 'dist': 350, 'walk':4.8, 'dir':u'900060104', u'dep':[], u'to':'her', 'name': u'Rubensstr. (Süd)', 'extId': u'900060153'},
        u'900061153N':{'lat': 52.47591, 'lon': 13.34091, 'dist': 550, 'walk':5.7, 'dir':u'900054105', u'dep':[], u'to':'hin', 'name': u'H\xe4hnelstr. (Nord)', 'extId': u'900061153'},
        u'900061153S':{'lat': 52.47531, 'lon': 13.33969, 'dist': 550, 'walk':5.7, 'dir':u'900061105', u'dep':[], u'to':'her', 'name': u'H\xe4hnelstr. (Süd)', 'extId': u'900061153'}
        }

        halten[u'callTime']=datetime.now().strftime('%Y-%m-%d %H:%M:%S')


############### MQTT section ##################

# when connecting to mqtt do this;

def on_connect(client, userdata, flags, rc):
        #print("Connected with result code "+str(rc))
        client.subscribe(cmdTopic)

# when receiving a mqtt message do this;

def on_message(client, userdata, msg):
        global boostCount,boost
        message = str(msg.payload)
        print(msg.topic+" "+message)

        if message == "boost":
                boost=True
                boostCount=0
                checkOnce()

def on_publish(mosq, obj, mid):
        pass
        #print("mid: " + str(mid))


def publishDict(obj):
        json_string = json.dumps(obj)
        client.publish(pubTopic,json_string,1)

def publishBanner(obj):
        json_string = json.dumps(obj)
        client.publish(bannerTopic,json_string,1)

def checkOnce():
        global inProgress
        if inProgress:
                return
        inProgress=True
        getAllTables()
        publishDict(halten)
        inProgress=False

def getAllTables():
        global selection,banners
        reset_halten()
        banners=""
        for halteKey in selection:
                getTableByHaltekey(halteKey)

def doNothing():
        pass

def getTableByHaltekey(halteKey):
        global halten, baseurl,suffix,duration,toScreen,banners

        halte=halten[halteKey]
        walk=halte[u'walk']
        extId=halte[u'extId']
        direction=halte[u'dir'] 
        reqTime=datetime.now() + timedelta(minutes=walk)
        datum = reqTime.strftime('%Y-%m-%d')
        zeit  = reqTime.strftime("%H:%M")
        datetimeFormat = '%Y-%m-%d %H:%M:%S'

        banner = "{} {} m {} min. Weg".format( halte[u'name'],halte[u'dist'],int(round(halte[u'walk'])) )
        #banners += banner + '\r\n'
        if toScreen:
                print "\n"+banner
                #print "{} {} m {} min. Weg".format( halte[u'name'],halte[u'dist'],round(halte[u'walk']) )

        # fuer die Ringbahn werden, wie im Witz, Richtungen nicht unterschieden...
        if u'lines' in halte:
                lines=halte[u'lines']
                uri = baseUrl+"departureBoard?extId={}&date={}&time={}&rtMode=FULL&duration={}&direction={}&lines={}".format(extId,datum,zeit,duration,direction,lines)+suffix
        else:
                uri = baseUrl+"departureBoard?extId={}&date={}&time={}&rtMode=FULL&duration={}&direction={}".format(extId,datum,zeit,duration,direction)+suffix

        # im debug die uri ausgeben
        if toScreen:
                pass
                #print uri

        response = urllib.urlopen(uri)
        dat = json.loads(response.read())

        if u'Departure' in dat:
                data=dat["Departure"]
                if len(data)==0:
                        return
        else:
                return 

        # kann sein, dass rtTime in einzelnen records fehlt, und manche records sind unplausibel
        delRecs=[]
        for recnum in range(len(data)):
                record=data[recnum]
                if u'rtTime' in record:
                        # verspätung als rtTime-time
                        planTime=record[u'date']+' '+record[u'time']
                        realTime=record[u'rtDate']+' '+record[u'rtTime']
                        lateSec=datetime.strptime(realTime, datetimeFormat)-datetime.strptime(planTime, datetimeFormat)
                        late=int(round(lateSec.seconds / 60.0) )
                        record[u'late']=late

                else:
                        # ha! record ohne rtTime, rtDate -> diese aus time, date kopieren
                        record[u'rtDate']=record[u'date']
                        record[u'rtTime']=record[u'time']
                        record[u'late']=0

                # ausgefallene Züge (?) bleiben im Suchergebnis stehen, obwohl ihre Zeit längs um wäre
                leaveTime=record[u'rtDate']+' '+record[u'rtTime']
                diff=datetime.strptime(leaveTime, datetimeFormat)-datetime.now()
                diffMin=int(round((diff.seconds / 60.0) - walk))
                if diffMin>100:
                        print "strange time: {}  rtTime: {} late: {} noch: {}".format(record[u'time'],record[u'rtTime'],record[u'late'],diffMin)
                        delRecs.append(recnum)
                if diffMin<-10:
                        print "spooky  time: {}  rtTime: {} late: {} noch: {}".format(record[u'time'],record[u'rtTime'],record[u'late'],diffMin)
                        delRecs.append(recnum)

                # Busse, die zu früh kommen, haben 1440 min offset
                if record[u'late'] > 1000:
                        record[u'late'] -= 1440

        for recnum in reversed(delRecs):
                del data[recnum]
        delRecs=[]

        dat=sorted(data, key = lambda i: (i['rtDate'], i['rtTime']))


        for pos in [0,1]:
                if pos==1 and len(dat)==1:
                        continue

                par=dat[pos]
                useDate=par["rtDate"]
                useTime=par["rtTime"]

                leaveTime=useDate+' '+useTime
                diff=datetime.strptime(leaveTime, datetimeFormat)-datetime.now()
                diffMin=int(round((diff.seconds / 60.0) - walk))
                # fix the occasional 1437 diffMin (instead of -3)
                if diffMin > 100:
                        diffMin -= 14400

                splitter = useTime.split(':')
                useTime = splitter[0]+':'+splitter[1]

                aline = "   {} {} {} {} noch: {} late: {}".format( par[u'name'].lstrip(),par[u'direction'],useDate,useTime,diffMin,par[u'late'])
                banner = banner + "
" + aline

                if toScreen:
                        print aline

                zug={u'name':par[u'name'].lstrip(),u'direction':par[u'direction'],u'date':useDate,u'time':useTime,u'left':diffMin,u'late':par[u'late'] }
                halte['dep'].append(zug)
                halte['banner']=banner
                #banners += banner +'
'
                banner=""



# mqtt
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.on_publish = on_publish
client.username_pw_set(username="user",password="password")
client.connect(mqBroker, 1883, 90)
time.sleep(4)
client.loop_start()


try:
        # loop
        while 1:
                if boost or (slapCount % noBoostMin == 0):
                        checkOnce()
                        #publishBanner({u'banner':banners})
                        boostCount+=1

                        if boostCount>=boostMin:
                                boost=False
                        print "boost: {} boostCount: {}".format(boost,boostCount)

                slapCount += 1
                time.sleep(60)

finally:
        print 'bye'

  • openHab items to receive or send data to/from mosquitto:
     
String mqVbb "incoming" { mqtt="<[mqttBroker:ext/vbb/timetable/state:state:default]" }  
String mqVbbCmd "outgoing" { mqtt=">[mqttBroker:ext/vbb/timetable/cmd:state:*:default]" } 
  • openHab items to obtain the parsed data; this is trivial and not shown
     
  • an openHab rule to parse and format the connection info into dedicated items (only the first part of it is shown)
     
rule "rule triggered  by mqVbb changed"
when 	
    Item  mqVbb changed 

then
    var String halte=""
    var String station=""
    var String zug1=""
    var String zug2="" 
    var String noch1=""
    var String noch2=""

    var String json = (mqVbb.state as StringType).toString
    
    halte="900060101N" // S Friedenau Nord
    
    station = transform("JSONPATH", "$['"+halte+"'].name", json) + " (" + transform("JSONPATH", "$['"+halte+"'].dist", json) + "m / " + transform("JSONPATH", "$['"+halte+"'].walk", json) + " min) "
    zug1= transform("JSONPATH", "$['"+halte+"'].dep[0].name", json) +" -> " + transform("JSONPATH", "$['"+halte+"'].dep[0].direction", json) +": " +transform("JSONPATH", "$['"+halte+"'].dep[0].time", json)+" ("+ transform("JSONPATH", "$['"+halte+"'].dep[0].late", json)+") "
    zug2= transform("JSONPATH", "$['"+halte+"'].dep[1].name", json) +" -> " + transform("JSONPATH", "$['"+halte+"'].dep[1].direction", json) +": " +transform("JSONPATH", "$['"+halte+"'].dep[1].time", json)+" ("+ transform("JSONPATH", "$['"+halte+"'].dep[1].late", json)+") "
    noch1=transform("JSONPATH", "$['"+halte+"'].dep[0].left", json)
    noch2=transform("JSONPATH", "$['"+halte+"'].dep[1].left", json)

    vbbHalte1.sendCommand(station)
    if (zug1.length()<100){vbbHalte1Zug1.sendCommand(zug1);vbbHalte1Noch1.sendCommand(noch1)}else{vbbHalte1Zug1.sendCommand(" ");vbbHalte1Noch1.sendCommand(" ")}
    if (zug2.length()<100){vbbHalte1Zug2.sendCommand(zug2);vbbHalte1Noch2.sendCommand(noch2)}else{vbbHalte1Zug2.sendCommand(" ");vbbHalte1Noch2.sendCommand(" ")}



    
    halte="900060101S" // S Friedenau Süd
    station = transform("JSONPATH", "$['"+halte+"'].name", json) + " (" + transform("JSONPATH", "$['"+halte+"'].dist", json) + "m / " + transform("JSONPATH", "$['"+halte+"'].walk", json) + " min) "
    zug1= transform("JSONPATH", "$['"+halte+"'].dep[0].name", json) +" -> " + transform("JSONPATH", "$['"+halte+"'].dep[0].direction", json) +": " +transform("JSONPATH", "$['"+halte+"'].dep[0].time", json)+" ("+ transform("JSONPATH", "$['"+halte+"'].dep[0].late", json)+") "
    zug2= transform("JSONPATH", "$['"+halte+"'].dep[1].name", json) +" -> " + transform("JSONPATH", "$['"+halte+"'].dep[1].direction", json) +": " +transform("JSONPATH", "$['"+halte+"'].dep[1].time", json)+" ("+ transform("JSONPATH", "$['"+halte+"'].dep[1].late", json)+") "
    noch1=transform("JSONPATH", "$['"+halte+"'].dep[0].left", json)
    noch2=transform("JSONPATH", "$['"+halte+"'].dep[1].left", json)

    vbbHalte2.sendCommand(station)
    if (zug1.length()<100){vbbHalte2Zug1.sendCommand(zug1);vbbHalte2Noch1.sendCommand(noch1)}else{vbbHalte2Zug1.sendCommand(" ");vbbHalte2Noch1.sendCommand(" ")}
    if (zug2.length()<100){vbbHalte2Zug2.sendCommand(zug2);vbbHalte2Noch2.sendCommand(noch2)}else{vbbHalte2Zug2.sendCommand(" ");vbbHalte2Noch2.sendCommand(" ")}


and finally some HTML to go into the template widget of a HabPanel page:

<div align="left" class "row">

  <div class="col-sm-6"><span style="color: red; font-size: 12pt">{{itemValue('vbbHalte1')}} </span>
    <ul>
      <li>  <span style="color: orange; font-size: 9pt">{{itemValue('vbbHalte1Zug1')}} </span><span style="color: cyan; font-size: 10pt">{{itemValue('vbbHalte1Noch1')}} </span></li>
      <li>  <span style="color: orange; font-size: 9pt">{{itemValue('vbbHalte1Zug2')}} </span><span style="color: cyan; font-size: 10pt">{{itemValue('vbbHalte1Noch2')}} </span></li>
    </ul>  
  </div>

  <div class="col-sm-6"><span style="color: red; font-size: 12pt">{{itemValue('vbbHalte2')}} </span>
    <ul>
      <li>  <span style="color: orange; font-size: 9pt">{{itemValue('vbbHalte2Zug1')}} </span><span style="color: cyan; font-size: 10pt">{{itemValue('vbbHalte2Noch1')}} </span></li>
      <li>  <span style="color: orange; font-size: 9pt">{{itemValue('vbbHalte2Zug2')}} </span><span style="color: cyan; font-size: 10pt">{{itemValue('vbbHalte2Noch2')}} </span></li>
    </ul>  
  </div>

Again this is just the start of it and those lines are repeated for all the stations and their items.

This is all just Alpha and there's room left for improvements, But 'm glad it works!

My little thing uses but a tiny subset of what functionality the api offers

Host your own doodles with jawanndenn

Thursday, March 14. 2019

For a website where event dates are offered to a group of users who need to find those dates that most users can attend, doodles are a good solution. There are well known providers for that sort of thing like doodle.com/, dudle.inf.tu-dresden.de/ or www.dfn.de/dienstleistungen/dfnterminplaner/, with the latter two offering better privacy for european users (no transfer of data to servers in the USA).

Still, hosting your very own service is an attractive alternative. For this there is jawanndenn.de/  which can be downloaded from github.com/hartwork/jawanndenn. It is a web application written in python by Sebastian Pipping, libre software.

Installation is easy with

pip install jawanndenn

and then run it from the command line with something like

jawanndenn --host your-domain.tld --port 23456  (the port is arbitrary).
navigate your browser to
https://your-domain.tld:23456 and create your first own poll.

You see the log entries written out on the console. This is nice for testing but to actually use jawanndenn you'll need something more stable than a manually started programm.

As my web server runs with apache this is how I did it:

- set up a subdomain dedicated for the application, like doodle.your-domain.tld

- get certificates for it from Let's Encrypt (i.e. using github.com/lukas2511/dehydrated )

- create a folder for document root of the new subdomain and put a file app.wsgi and a favicon.ico there, the former you'll find at github.com/hartwork/jawanndenn/blob/master/jawanndenn/app.wsgi , the latter can be done with touch as we won't see that icon anyways but it saves 404 log entries to have a file named like it.

- make sure that apache2 has mod_wsgi enabled

- set up a virtual host for it in apache2/sites-available. Example:

#
#
######################################
#  doodle.your-domain.tld
######################################
#
#

# subdomain dedicated for running the jawanndenn web application
# don't separate .conf and .common since this will lead to errors when both :80 and :443 try to set
# a WSGIDaemonProcess with the same name.
# Don't include processes=1 into the WSGIDaemonProcess definition for this will lead to errors "Single process needed"

<VirtualHost :80>
#       include sites-available/doodle.
your-domain.tld.common ## this won't do, see above
        ServerName doodle.
your-domain.tld


        WSGIDaemonProcess doodle user=web123 group=web123 threads=5
        WSGIScriptAlias / /var/www/web123/doodle/app.wsgi

    <Directory /var/www/web123/doodle>
        WSGIProcessGroup doodle
        WSGIApplicationGroup %{GLOBAL}
        Order deny,allow
        Allow from all
    </Directory>

   CustomLog /var/log/apache2/doodle.
your-domain.tld.access.log combined
   ErrorLog  /var/log/apache2/doodle.your-domain.tld.error.log

</VirtualHost>

<VirtualHost
:443>

ServerName doodle.your-domain.tld


    # supply a different name for the WSGIDaemonProcess on :443
    WSGIDaemonProcess doodler user=web123 group=web123 threads=5
    WSGIScriptAlias / /var/www/web123/doodle/app.wsgi

    <Directory /var/www/web123/doodle>
        WSGIProcessGroup doodler
        WSGIApplicationGroup %{GLOBAL}
        Order deny,allow
        Allow from all
    </Directory>

   CustomLog /var/log/apache2/doodle.your-domain.tld.access.log combined
   ErrorLog  /var/log/apache2/doodle.your-domain.tld.error.log


        SSLEngine on

        SSLCertificateFile      /etc/dehydrated/certs/
your-domain.tld/cert.pem
        SSLCertificateKeyFile   /etc/dehydrated/certs/
your-domain.tld/privkey.pem
        SSLCertificateChainFile /etc/dehydrated/certs/
your-domain.tld/chain.pem




</VirtualHost>

Now restart apache2, weed out typos and your done. You create new doodles from doodle.your-domain.tld,
make note of the handle of the new poll and embed it on a web site with an iframe like

<div class="dudel">
<iframe src="https://doodle.your-domain.tld/poll/a9108dcfa12006ce3c229d6f0110c5f976df14e963ddd9b7ffc8618013a0bd7e"
        width="100%" height="600" frameborder="0"
        allowfullscreen >
  <p> <a href="https://developer.mozilla.org/en-US/docs/Glossary">
    Fallback link for browsers that don't support iframes
  </a> </p>
</iframe>
<div>

 

I had no luck trying to use sandbox with the iframe, YMMV

 

Now, is that secure?
Depends.

Do a little exploration with the 'View Web Source' feature of your browser to see all the relevant URIs and handles visible in plain text. Who ever gets to that page with the doodles can do shyte like

  • create 'unofficial' doddle polls
  • post spam as the name of a doodle
  • what the heck

Thus, we do have the security feature of 'data never leave my own site'. Which is cool.
All the rest of your desired security you will have to provide on your own.
 

Now, can I edit a vote? As it comes jawanndenn does not support any editing of votes which is regarded a security feature but makes doodles pretty useless in real life cause people do change opinion all the time. You cannot manually edit the stored votes cause they are stored in a binary format that's hard to access.

What you can do is add that functionality by yoursel like modify the vote() method of the _Poll object to replace the prior vote with the new vote if the name is exactly the same. 

 

rfid-tags for presence detection in smarthome

Wednesday, November 28. 2018

Pressence detection is vital in a smarthome setup, lights should go out when I leave and on when I come home.  But presence detection is not trivial, you can not rely on PIR motion sensors since they often do false positives. You can not rely on pinging the smartphone when it connects to the WLAN because phones have sleep modes to save energy and you'll find your phone regarded as absent when it sits next to you on your desk. Even owntracks and tasker provide solutions which work sometimes and fail later. .

And, more often then I like, I have to search for my keys before leaving the house. Now this is how I try to catch two flies with one stone:

I bought a RC522 rfid reader to integrate it into my smarthome IoT.
Most of the stuff you can find for those, both software and things, follows the idea of access control.
That is:

  • the thing with the reader waits in front of a door and controls a lock. User comes, holds a keycard at the reader, is denied or allowed, puts the keycard back in their pocket and enters...

My use case is different, more like what you may find in a hotel:

  • the thing with the reader waits behind the door and the user is supposed to insert the key card into the reader and leave it there. The reader identifies the key card and informs the smarthome controler that the owner of that key card is at home so it's time to turn on the lights and enable all the features that will be turned off again when the key card is taken out of the reader, later.

For this to work the key card (those blue drop shaped ones with a key chain ring you get with the reader) has to be in close range of the reader in a stable position, in other words, some sort of pocket is needed.

 

rfid-boxThe box is a remix of something I found searching thingiverse and sits at https://www.thingiverse.com/thing:3242266

It's printed in fast mode and with elements into all directions the slicer added lots of support which was hard to get off, you can see ht the box suffered from it.

Anyways, this cheap RC522 rfid reader which set me back by about 6€ sits in this box I printed, with a pocket to insert the chip and thus hold my keys at a defined place. The reader, connected to an Nodemcu V2 microcontroller identifies the chip, thus knows it's me who is at home and reports this to the smarthome controller. When I leave the house and take my keys the reader creates a new event and the system knows I'm out.

On the software side again there are plenty of example sketches for access control where the reader idles until you put a key card in its range, then the card is analysed, the result is given and the thing idles on. There is no event when the card is taken away.

So I had to write my own sketch for that, too. Again it is a remix.
It runs on an arduino clone and it creates an event when a known key card is entered, and another event when that card is taken out.

Since my smarthome is controlled by openHab2 which has a very usable rest api there is some code to report those events to it.
The sketch is part of the thingiverse file bundle linked above.

openHab2 startup chaos

Thursday, October 25. 2018

starting and stopping openhab2 is a messy affair, filling the logs with loads of Java's verbose exception messages and since the order of loaded models is basically random things may or may not work as intended when items have not been loaded or rules not executed that are assumed to be there.

As it was, manual corrections and reloads were needed after each restart.

There is an elegant workaround discussed at community.openhab.org/t/cleaning-up-the-startup-process-renaming-rules-windows-possible/38438/9 which uses systemd's ExecStartPre and ExecStartPost commands to deactivate all rules before starting openhab, and then reactivate them one by one when the system is up and has it's feet on the ground.

It did not work exactly like described there for me, but I still found a solution that has cleaned up the messy startup here.

A small bash script does the renaming. I put it in /etc/openhab2 next to openhab's configuration into a new folder /exec-scripts:

#!/bin/bash
#######################
#
# clean up the start process 
#  starting rules in a sorted manner after openhab2 got it's feet on the ground
#  called from systemd pre start 
#  to rename *.rules away initially 
#
#  synopsis: move_rules_at_start.sh org-extension new-extension (POST)
#  
#  first call is from
#  /etc/systemd/system/openhab2.service.d/override.conf
#
#  called from the running openhab again to rename them back one by one
#
# $3 allows to distinguish between pre and post action 
#
# REF: https://community.openhab.org/t/cleaning-up-the-startup-process-renaming-rules-windows-possible/38438/9


ORG=$1
NEW=$2
THX=$3
DUR=1
IGNORE=005_start.rules

for f in /etc/openhab2/rules/*.${ORG};
do
    CURRENT=$(basename $f)
    if [ "$CURRENT" == "$IGNORE" ]    
    then
        echo "ignoring $IGNORE"
    else
        OLDFILE=$f
        NEWFILE=${f%$ORG}$NEW
        mv "$OLDFILE" "$NEWFILE"
    fi
    # let some time between each load
    if [ "$THX" == "POST" ]
    then
        /bin/sleep $DUR
    fi
done

# some things left on startup
if [ "$THX" == "POST" ]
then
    /usr/bin/touch /etc/openhab2/things/tradfri.things
fi


Since the openhab2.service file is part of the package and thus is replaced with each update modifying the .service file would need to be repeated whenever a new version gets installed, but, systemd provides a way to override a service definition. systemctl edit openhab2.service creates an override dir and opens the editor to allow creating a service file that is not replaced by the next update.

Now don't try to copy the entire .service file, systemd will complain duplicated statements. My override.conf looks like this:

[Service]
ExecStartPre=/etc/openhab2/exec-scripts/move_rules_at_start.sh rules rules_

The third element is a .rules file in openhab2 that triggers the renaming of all the other .rules once the system is running. It is excepted from the renaming and I named it 005_start.rules. This works for me:


var rulesDelayed
var nada

rule "triggered by system start"
when 
    System started
then
    var duration = 1
    rulesDelayed = createTimer(now.plusMinutes(duration), [|
					logInfo("rulesDelayed", "Timer expired and start")
					nada=executeCommandLine("/bin/bash /etc/openhab2/exec-scripts/move_rules_at_start.sh rules_ rules POST ",90000)
                    logInfo("rulesDelayed", "result: "+nada)
					rulesDelayed = null

				])
    logInfo("rulesDelayed", "Timer set on "+duration+" min")
end


The bash script works the list of rules -files in an alphanumerical order so a naming scheme like praefixing all .rules with a three digit number finally allows to control the order of rules loading.

Startup is much faster now, less or no exceptions and a much cleaner log :-) Openhab is a Java application and those exceptions are super-ugly and come with a performance penalty.

You'll notice that the bash script supports an optional 3. parmeter to distinguish between pre- and post-action and that a certain config file is touched after the renaming. touch-ing triggers a reload of that config file and this was a workaround for a bug in the tradfri-addon, the bug may have been addressed by now.

TimeStamp in mosquitto.log

Monday, September 24. 2018

 mosquitto gibt jedem Log-Eintrag einen TimeStamp im für Maschinen recht praktischen Aekunden-seit-Epoche - Format. Für menschliche augen ist diese Angabe eher sperrig.

Mit perl und ccze (macht's bunt) kann man die Ausgabe etwas aufhübschen:

tail -n2000 -f  mosquitto/mosquitto.log |  perl -pe 's/(\d+)/localtime($1)/e'| ccze -m ansi

 

(Page 1 of 36, totaling 177 entries) » next page