Introduction
Following on from using a Raspberry Pi to capture the electricity consumption of my home, I have over the last 2 years installed Solar Thermal Hot Water panels and very recently Solar PV. This meant that I had an opportunity to create something I had been looking to do for a while – an in house portable “Home Energy Centre”. This post documents the thought process behind what I did, the choices I made and also the problems and hurdles I had to overcome to get it all up and running. First let’s talk about the Home Energy Centre.
Hec-display in kitchen
This is built on a “Nook Simple Touch”, which I bought about a year ago during National Book Week for only £21!
Why did I choose this device? Several reasons really,
- it was cheap!
- It has built in Wi-Fi
- It is easily hackable (with Nook Manager)
- the battery life is good (about 1 week with wireless running all the time and the display refreshing)
- it has a touch screen
- it is small and can easily be mounted (either in a frame or on a fridge using magnetic adhesive tape)
- I can always re-use it as an e-book reader!
The “Home Energy Centre” (HEC) runs in the browser of the Nook. Because it runs in a browser it can also be displayed on any device with a browser, for example PC, Laptop, Mobile Phone.
The screen is split into 3 main sections.
The left hand section displays information on how the Solar PV panels are performing; telling me what Electricity is being generated now, if any is being exported, along with the totals for the day so far. This updates automatically reasonably frequently.
The middle column displays information on how the Solar Thermal Panels are performing, these provide hot water. So the display shows me, the current temperature of the panels, the current temperature of the water in the hot water tank; along with, the speed of the pump as a percentage and the number of hours the pump has run in total. The lower section tells you for how many hours the pump has run so far today, and importantly what the temperature gain in the Hot Water has been so far this day.
The right hand column displays the current weather for where I live.
Along the bottom are 3 “buttons”, for now only the left hand button works and this takes me to the graphs of electricity consumption I did a while ago (See this blog post). This information all in one place and readily visible has help us decide when to use electrical appliances in the house and when to or not heat the hot water using the central heating (Gas based) system we have in the house.
High Level Architecture
The Raspberry Pi performs all of these functions:
- Receives Solar Thermal output from the Resol pump controller via its VBUS data bus and the electronic circuit, into the tty0 serial port.
- Decodes the data calculates values and files it ready for processing
- Receives over the local network, via multicast, Solar and Electricity xml packets from the Owl Intuition Solar PV monitoring and files it ready for processing
- Runs the program to draw the Home Energy Display (python/pygame) as a picture, this program also fetches the latest weather information from Weather Underground via the internet.
- Runs tightvncserver (required by pygame)
- Runs lighttpd server to serve the image map / web page that is displayed on the Nook, or any web browser/mobile phone.
Solar Thermal Real Time Data Solution
Getting meaningful data from the Solar Thermal panels was definitely a challenge. I didn’t want to spend over £100 on a special LAN adapter they sell. My system has a Resol© BS/4 pump controller which has a VBUS© data bus output. The signal this bus produces is similar to RS485 in that it is a dual polarity signal, but not exactly the same. I designed the following circuit to process the signal and turn it into 9600 baud 0V, 3.3V serial data suitable for interfacing to a Raspberry Pi. Please note this circuit is not electrically isolated from the VBUS controller, which does switch a relay for the pump which runs from mains electricity. If an electrical fault occurs in the VBUS controller it could fry the circuit or the Raspberry PI. In a revised version I would incorporate an opto-isolator. The circuit is also designed only to receive data and not transmit it. If you want to use this circuit with a device that withstands 0,5V serial data, e.g. an Arduino©, then just remove the two resisters that form the voltage divider at the output. Remember to stop the Pi from starting the default tty0 terminal on the serial port. Now that we have some nice clean serial data coming into the Pi, we need to process it. The Resol protocol is quite unique in that it swaps MSB and LSB’s around so that only the starting byte of a sequence of data has its MSB set, &AA signifies the start of a sequence of data. Fortunately for me, someone far cleverer than me has written a C program to do this called “vbusdecode”. The next problem I had was that you need to know the XML format of the data for the controller you have in order to tell vbusdecode what to do. This information is not supplied in the documentation from Resol. After a lot of research and hunting and many false trails, I found an XML specification file in the installation files for some free software until recently given away buy Resol. This means that finally we can decode the data coming via the VBUS. I discovered that the BS/4 sends out three different types of data, I am only interested in one type and I want the data in CSV format, so I modified the source code for “vbusdecode” to do this. So every day a new file is created to store the Solar Thermal Data and every few minutes a new line of data is added. This file is used by the HEC display program as the source of its Solar Thermal data.
Useful Links & Credits
- vbusdecode (original information) Andy White
- Desrablog (where I started)
- Hobbytronixs (for clues on the electronic circuit) (in German)
Solar PV Solution
This solution was far simpler and one I got working in two evenings. The OWL© PV Intuition product comes with a device that spits out data in a multicast group on the local network. It also has a published API and can also be configured to push data to a certain IP address and port. I took the simple route and just used the multicast data. I have adapted an already created Python library/set of classes for receiving multicast data. Two types of XML datagrams are received approximately every 30 seconds, one for Electricity consumption and one for PV generation and Export. This program receives the XML data, decodes it and stores them in two files on the Raspberry Pi to process. The code is here.
HEC Display Program
This is main program that pulls all of the data sources together and creates the Home Energy Centre display. It is written in Python and Pygame. My original idea was to serve the display in an X-window using tightvncserver on the Raspberry Pi, and view it using the VNC client for Android. This proved problematic on the Nook, as in order to get the VNC client on the Nook I needed to install all of the google apps and the google play store app. This made the nook unstable. So I reverted to a different plan: just use the web browser on the nook, and change the main program to paint (save) a picture of the screen to an image file – which is web served to the browser. This works well in pygame and is easy to code. I have subsequently removed the majority of the Google apps from the Nook and it is much more responsive and stable. The main program is here.
Lighttpd Configuration and Webpage.
The Pi is running a standard install of lighttpd. It serves a very basic index.html web page like so:
This serves the “picture” of the HEC display as an image map, allowing mouse clicks or touch screen (on the nook) presses to be detected and appropriate actions taken. The page auto refreshes. There is another version of the page in a sub directory called nook, which is the same apart from having a slower refresh rate. This helps conserve the battery life of the nook.
Physical Installation
Everything is installed in a cupboard near to the pump controller for the Solar Hot water in a £ 3 Sandwich box!
Nook Modifications
The Nook has been rooted with NookManager. There are plenty of articles on how to do this. For example:
http://www.babblingengineer.com/how-to/how-i-turned-my-nook-into-an-e-reader-monster/
For information on installing NGAppsAttack look here.
I have also made the following modifications to remove the screen saver/timeout and slow down the screen refresh method to conserve battery life. Instructions for this can be found in this thread: http://forum.xda-developers.com/showthread.php?p=29113425
Known Problems/Irritations
The Pi sometimes locks up and is not reachable across the network. This was cured by removing the lid of the sandwich box!
Tightvncserver sometimes locks up.
Things to improve, things to do.
Start logging the Solar Thermal Data and the Solar PV data in RRD so daily graphs can be produced (similar to what I have done previously)
Tidy up the code for the Solar Thermal side.
Implement a 5 day weather forecast using the UK Met office’s data point service and implement the Forecast Button
Implement the Automation button (that is a topic for another time!)
Thanks for Reading!
I hope you have found this interesting and it inspires you to do something with a Raspberry Pi. Also thanks to the world wide web, all of the credits listed above and Simon, Stuart, Ashleigh and Mark for putting up with me and giving me encouragement on this project!
Pingback: Home Energy Centre using a #RaspberryPi and Nook Simple Touch | Raspberry Pi Pod
Thank you for your sample code – I plan to use a lot of it for my house!. I have a vbus/net interface to a Resol BS/3, and a SMA inverter on my PV array, so your code isn’t 100% what I need, but I’ll be able to make it work. I also will be using a kindle for my WWW browser, and might even make an interface for my smart TV 🙂
When I get the vbus/net and SMA protocol support added, if I send you patches, will you include them in your HEC package ? As I make patches/enhacements I’ll send them to you…if requested.
Hi Sean, I’m having problems leaving messages her so I am trying some direct replies. I’m going to build gazza’s circuit to try and download data from my Resol solar heating (hot water) controller. Would you be able to point me the direction of what files I need to load onto to raspberry pi. I assume its the ones in the Resol folder on Github.
Great goods from you, man. I have understand your stuff previous to and you’re
just too excellent. I really like what you’ve acquired
here, really like what you’re saying and the way in which you
say it. You make it entertaining and you still take care of to keep it smart.
I can’t wait to read much more from you. This is really a
tremendous website.
Wow thank you. I have been trying something like this with the Nook Touch Simple for ages, and just came across your blog with the xda link how to disable the screensaver!
How did you rotated your screen/browser?
Hi ps915, sorry for the delay in replying. Initially I could not remember! However after some digging I found the answer.
The Nook Manager software used to root the Nook, has in its home screen a settings icon (a cog). Note this is not the same as the built in settings screen for the Nook.
One of the settings option is called “Screen Orientation” and I chose the option “Force to Landscape”.
I hope this helps, regards Gaz99
Oh thanks,
will try that. 🙂
Hi
I use your circuit to get data from my RESOL DeltaSol BS Plus v2. I read data from DeltaSol in every minute. Sometimes I get CRC error. (Sometimes one error per month or 2-3 error per day. So the period of good work very different). I use in my circuit LM393P instead of LM339N and 5k resistor instead of 5k1 resistor. What is your opinion the problem is in two component or other thing? (eg. noise in vbus cable, i use siple two pair cable without shadow)?
When you plan to make the revision version (with the optoisolator)?
Thanks
Sorry my bad English
Hi Zero, I do not think that there is anything to worry about in the changes to the circuit you have made. One thought, I have twisted the two wires from the pump controller to the circuit, this helps to stop interference. Perhaps try this. Regards Gazz99
Hi, I have uploaded it to 123D circuits. It is called vbus-to-ttl-converter. Sorry, can’t add the full URL; antispam protection I suppose.
Corrections and ideas welcome. In particular, I’m not sure about the RJ-45 positions.
Also, what license would you like it to have?
Thanks.
happy for a share with anyone type of licence (Creative Commons?).
Hi, I ended up choosing a CC license with attribution.
I am awaiting for the printed boards now from OSHPark.com (as per 123DCircuits recommendation)
Hello,
I just wanted to drop a quick note to say that I finally built everything and works great.
I have documented what I did on GitHub:
https://github.com/alfredosola/vBus2IP
Hi Alfredo, well done that is great. It would be nice if you can reference my blog somewhere on you GitHub readme. good luck, Gazz99
Done!
Got any spare boards printed?
I have two spare boards.
I ended up purchasing three PCB’s from 123Dcircuits through OSHPARK.com. I will probably end up with a couple of PCB’s left over that I might part with. If anyone is interested let me know.
Francois, I would be interested in purchasing one of your spare circuits if you decide to part with them.
Dear Gaz, thank you for publishing such a really useful article. Earlier this year I installed solar panels to heat my hot water. I have a resol controller but as you correctly point out, the data logger is around 300 euro to buy. Fortunately I was gifted a Raspberry Pi as leaving present from work, so now I have the opportunity to start collecting data. I have ordered all the parts to make up the circuit on a breadboard but I’ve just noticed in your “sandwich box picture” that the circuit board has a power supply to it. This may seem a daft question but is this power supply a discrete feed or can it be taken from the Raspberry Pi? If it is discrete feed could you tell me what type of product I should be looking for in the electronic components store. Thanks.
hi David,
I used a separate 5v supply, but the circuit current draw is minimal, you should be able to use the same 5v supply as for the Pi. Make sure it is a 2A supply.
Hi Gary, one more question. In your article you mentioned a safety upgrade to the circuit using an opto isolator. I’m not any good with electronics but from googling it, would I be correct in saying that the opto isolator would be inserted into the VBUS input path so that;
vbus a connects to pin1
vbus b connects to pin 2
pin6 would connect to 5v
pin 5 would connect to where vbus a originally went
pin 4 would connect to where vbus b originally went.
I havent bought the opto isolator yet!
Thanks
hi, not quite. The VBUS signal is differential, and has to go through the comparitor first. I would put the opto-isolator between the output of the circuit and the input to the Raspberry Pi.
The purpose of the isolator would be to protect the Raspberry Pi, which is the most expensive part.
Have a look at the android app electricsign which takes a screencap of a website refreshed at set intervals. Can be used in conjunction with the nook default screens aver to display info like this without having the browser running / browser bar visible
Hi gazzat5, thank-you very much for this hint. I just tried the app, and it works great. Gaz99
On my nook I had it refresh every now and again and then go to the nook wallpaper screensaver with electric sign saving to the wallpaper directory.
I found this saved the most battery besides physically pressing the lock/sleep/on button on the back.
Unfortunately my customised nook and it’s books was stolen! 😥 I might eBay a replacement someday!
Thanks very much for your page about your experimentation with your resol hot water controller and logging the data, I have a setup with a Delta Sol B/S4 controller and wish to capture my own data in a similar way. I have managed to make a small circuit that converts the vbus data into serial, however I am stuck at the point of decoding the serial data. Are you able to provide any hints? I am trying to use an ardiuno board with one of those Chinese wifi chips to host the webpage on my LAN.
Thanks, Tom.
Hi Tom,
apologies for missing this comment left several months ago – sorry – took my eye off the ball! How good are you at decoding ‘c’ program? I used a c programme called Vbus decode written by Andy White to ‘decode’ the 8n1 serial data. his code is here https://code.google.com/p/vbusdecode/ . It should be easily possible in ‘Arduino’, Also see here in the docs folder. https://github.com/naylogj/hec-display/tree/master/hec/resol/docs . thanks to your comment I just realised one of my links in the blog article is now broken. I will fix it. thanks and good-luck.
No problems Gaz, Thanks for you feedback and reply. I actually just got my system working this morning! I ended up using an Arduino ESP-12e board which works a treat. My biggest problem was the code I was using to program my Arduino. But all sorted now.!!!!
That is cool. Would you be prepared to share a link to the code? i am sure others would be interested in doing this on a ESP. thanks.
I will check with the co-author of my esp code, to see if he has any issues with me posting it
Sorry Gaz99, My friend did not want to share the code from my Arduino ESP12e. The only code that I figured out was for the Vbus decoding which is pretty well all over the internet nowadays.
Hi Tom,
Check out my version at:
https://github.com/alfredosola/vBus2IP
Thanks, I have it working now!!!
Hi Gaz99. I got this working last night with a etched pcb and an arduino, thanks very very much for all the information. Because of my setup I would like to power the board with 3.3v instead of the 5v outlined in the circuit. I tried it briefly and this seems to work. Is there any reason you chose to power the circuit with 5v instead of 3.3v
Hi Rick – ‘because I had a ‘5v’ usb wall-wart lying around – no other reason!!! Good stuff, glad you got it working! Gazz99
Hi Gazz,
thanks for this blog. I got this working with an arduino with a deltasol c. However i’d like to power the circuit with 3.3v instead of 5. It seemed to work when i tried it. Is there a specific reason why 5volts is called out in your circuit diagram.
hi FatBeard – see my reply to Rick. I choose 5v because I had a ‘5V USB wallwart lying around spare’ – no other reason! Gazz99
I’m finally capturing data from my resol (Apricus) collector. If I dump the $data2 variable output of the log_solar command I get something like follows;
21:30,53.6,43.7,0,936,0,-4.7 , 0
Do these values look correct?
Is there an explication for each of these values?
F
Hi Francois,
if it is anything like my Resol© BS/4 controller it will be the following:
time hh:mm , Collector Temp (degrees C) , Water Store Temp (degrees C), Pump State (0=off, 1=on), Hours Pumped, Heat in KwH, Status (0=no error),
you have a 0 followed by -4.7 and a 0.
the “0, -4.7” is probably a double digit large number that is in reality positive. the last 0 is probably a status code [0 = no error]
i do not collect / use the kWh figure so I ignore it. Also have a look at
https://github.com/naylogj/hec-display/blob/master/hec/resol/spec/VBusSpecificationResolBS2009.xml
i hope this helps
Gazz99
Thanks, Ideally I’d also like to know when the controller kicks in the electrical element to boost the water temp….this doesn’t appear to be captured. I’ve managed to export this to the following URL under the extended tab (which is the right most button/square under the date)
https://pvoutput.org/intraday.jsp?id=11117&sid=9144
I’m only really interested in panel temp, tank temp, pump on/off and hopefully the external element switch on/off at some point. I’m going to strip this code back to and simplify it purely for use with pvoutput and get to github also.
Cheers
FP
Hi Francois,
Not sure if this is applicable to you, but I implemented a way to monitor my electric hot water heater element usage. I used a 240VAC to 5VDC mobile phone adapter. Connected one end of an extension cord to the heater element electrical connections, the other end plugged into the phone charger, the USB cable output from the phone charger was 5V, which I then passed through a voltage divider to interface with my Arduino ESP12e board (which runs on 3.3V, hence the need for the voltage divider). The Arduino simply recognised if there was 3.3V on the input (heater element on) or zero volts (heater element off).
Hope that helps.
Also, if that first column is the time, then my controller is way off. Is this time from the controller itself or is this code generating this?
I would guess this is the current time set on your controller. I do not know for sure as I only read the temps from my controller.
To be honest I have never really checked what the current time set on my controller is. I am using a Delta Sol BS/4
hi the BS/4 has its own clock, it does not adjust itself for daylight savings time. so it can be an hour out! it has to be manually changed.
I started pushing some data to ubidots to see if this would work…quick first attempt here
https://app.ubidots.com/ubi/public/getdashboard/page/uWOlJEza-oc8yGDs2uE0thv0i7o#/
F
Hi Francois, that looks nice. well done!
Pingback: Solar Hot Water data logging - Simon Law
Hello i used your circuit to build a datalogger for my resol system using arduino mega board , i tried the shematics on a breadboard and i wrote a skecth on arduino using the VBUSdecoder library, everything worked fine , however when i changed my computer and reinstalled the Arduino IDE something went wrong i also made a PCB after electricly testing it it worked, but now i dont get the problem it seems as the code bloks at the vbusreadsensor command !! any help !!
if (Serial2.available ());
{
data = Serial2.read ();
vb.readSensor();
Serial.println(“K”);
Serial.write (Serial2.read ());
on the serial monitor not even the K is written !
Hi Wassima, sorry for the delay in replying. I won’t bore you with the reasons why. Are you sure that you are using the correct serial port, i.e. Serial2 on the Mega is wired to the PCB? Did you get it working?
Pingback: Home Energy Centre using Raspberry Pi and Nook Simple Touch - Home Automation
Pingback: Controlling Hot Water production using Home Assistant Core | piamble
Pingback: Home Energy Centre using Raspberry Pi and Nook Simple Touch - Automations Center
Thanks for the very useful information. I know these posts are a few years old now but can anybody tell me if this can work with a raspberry pi pico W? Also if anybody can give me any tips on building the circuit for the Vbus interface I would be very grateful. I have little experience with electronics but I can solder and I know what most of the components are (except the decoupler and the opto-isolator)
Any info much appreciated
I’ve added a link to the PCB here https://oshpark.com/shared_projects/AjczSV3l
This was still stored here from when I had this printed. I now have leveraged this PCB along with https://github.com/danielwippermann/resol-vbus to collect readings into Home Assistant.