Post by triggerfish on Oct 9, 2018 23:20:43 GMT -8
Every now and then, after a reboot, I noticed my station is not reporting to wunderground. Then I realise I did not get the reboot mail... But the OLED is displaying data and the log on the pi is filling with info... When I look at the log, I see a sendmail exception in the top. For some reason python can not connect to the outside world. I am logged in on th epi from my work, so over the internet and the build in wifi, so it does communicate...
deviceID = 0x1050
GroveWeatherPi Solar Powered Weather Station Version 3.02 - SwitchDoc Labs
Program Started at:2018-10-10 08:51:06
---------------------- I2C Mux - TCA9545: Present BMP280: Present DS3231: Present HDC1080: Present HTU21DF: Not Present AM2315: Not Present ADS1015: Not Present ADS1115: Present AS3935: Present OLED: Present SunAirPlus: Present Sunlight Sensor: Not Present WXLink: Not Present
UseMySQL: Present Check WLAN: Present WeatherUnderground: Present ---------------------- ----------------- SunAirPlus Sampling ----------------- sendmail exception raised ----------------- Sometimes it works after a reboot, sometimes it works after killing the process and manually restarting it, but not always.
Post by triggerfish on Oct 10, 2018 9:15:41 GMT -8
Until now I rebooted, restarted, cursed and hit stuff, but the software refuses to send anything to the outside world. I have an ssh connection over the WiFi transmitter, so it does actually work, but I get the sendmail exception and stuff is not uploaded to wunderground anymore. I removed the fancy line that put the wind gusts in my graphs, but since that had worked for a couple of days, I don't expect anything form that. So how to debug this? Every time you think you get things solved some new problem pops up from nowhere
can you manually send yourself an email outside of the weatherpi? this would be the first step.
This is so freaking frustrating! I changed the exception code, to display an actual error in stead of just "sendmail exception" and, actually not, to my surprise all works like a charm again I *know* the change can not have anything to do with it, but the next time things fail, I possibly have an error...
The problem is that often after a reboot, I notice nothing is uploaded to wunderground. Then I also see there is no mail about the reboot. In the log it just says "sendmail exception". Sometimes a new reboot solves things, something I can reboot the pi, restart the software how much I want, without success. It just will not send the email, not will it upload to wundeground.
I set the exception to print an error an at first, off course, it worked fine, but tonight after the 48 hr reboot the problem started again. Because I changed the print statement, I now see:
getrandom() initialization failed. (_ssl.c:661)
I changed the code to:
except Exception as e: print(e)
I did similar with the upload code, but I get nothing from that at all in the log. Maybe there is a situation it does not even try?
There is something wrong with the way the Pi is establishing the WiFi connection. This is pretty clearly not part of GWP, but the boot up system.
Maybe it takes a long time to connect to the WiFi. Can you boot up by hand, login and start GWP by hand? Good long delay to test this thought.
Although I love explanations, this one has a problem... I already set the pi to wait for network before boot. I also set a sleep timer of about five minutes in /etc/rc.local before starting groveweatherpi, but nothing solved the problem. Another weak spot in the connection theory is the fact that when the problem arises, I can log into the pi over wifi and start the software manually still resulting in the error. In that case the pi has perfect wifi connection to my laptop at least. Having said that... I had two scheduled reboots by now, without the problem recurring, so it is very difficult to diagnose. I'll just wait and see what happens.
And to add... I reversed both changes, so they do not explain the current lack of problem.
Strange I just had this happen yesterday, I was going to put a loop in the beginning of the GWP (my version) to ping google, and wait for a response then only after I get a response, I will launch GWP. but I will try the sleep command for 5 min first. that is a quicker and easy way. Mine has only happened a few times. I was thinking it was with my hotspot, not the Pi. But yesterday I say it was connected to the hotspot per the hot spot screen, but no web interface. I don't know now. I may try to have a routine to stop and restart the APACHE, maybe that will help.
I tracked down the issue, it is 2 issues. my hotspot was using more power then the pi, so at night it would drain the main battery to keep it's own internal battery charged. I now have pulled out the battery, and am running it off the WittyPi2 now. so when it powers on with the Pi, it comes back to life. The free service I an using to redirect the Pi, web server to the rest of the world takes a while longer now to reconnect the pi to the world. But the above setting did help with it cutting out during the day. I suspect it was a power drain on the battery during the beginning part of the day until the solar panel caught up with the hotspot charging itself.
After a long silence, the error occurred again... I did some more searching and found an entropy issue... On boot, apparently it is possible there is possibly not enough entropy to generate safe hashes for ssl connections... The current hardware have on board pseudorandom number generators that can be addressed... I installed rng-tools and see at least on the command line an improvement in entropy. Have a look at hackaday.com/2017/11/02/what-is-entropy-and-how-do-i-get-more-of-it/ Npw wait and see what happens with the next reboots... Hopefully the error will stay away now.