berne
Junior Member
Posts: 85 
|
Post by berne on Nov 13, 2021 9:05:19 GMT -8
Because we have lots of power outages here and I am worried about SD card corruption I want to do daily backups (using a script file and crontab) of the RPi 3B I am using, including all of the SQL databases. I am trying to run mysqldump from the Raspbian command line and get:
pi@SwitchDocLabs:~ $ mysqldump --all-databases --user=root -p[password] > /home/pi/Berne-PC_Sata_SSD/SkyWeather2_SQL/RPi_Backup/mysql.sql
mysqldump: Got error: 1045: "Access denied for user 'root'@'localhost' (using password: YES)" when trying to connect
pi@SwitchDocLabs:~ $ mysqldump --all-databases --user=pi -p[password] > /home/pi/Berne-PC_Sata_SSD/SkyWeather2_SQL/RPi_Backup/mysql.sql
mysqldump: Got error: 1045: "Access denied for user 'pi'@'localhost' (using password: YES)" when trying to connect
pi@SwitchDocLabs:~ $ mysqldump --all-databases --user=pi -p[password] > /home/pi/Berne-PC_Sata_SSD/SkyWeather2_SQL/RPi_Backup/mysql.sql
mysqldump: Got error: 1045: "Access denied for user 'pi'@'localhost' (using password: YES)" when trying to connect
pi@SwitchDocLabs:~ $ mysqldump --all-databases --user=admin -p[password] > /home/pi/Berne-PC_Sata_SSD/SkyWeather2_SQL/RPi_Backup/mysql.sql
mysqldump: Got error: 1045: "Access denied for user 'admin'@'localhost' (using password: YES)" when trying to connect
(I have added access for user Pi using your default "password" as the password.)
Not a linux or SQL expert, what am I doing wrong? I can use phpmyadmin to do one off backups no problem. My webs searches so far haven't yielded a solution.
Thanks.
Sorry -p[password] seems to be incorrect, should be -ppassword. Works now.
|
|
berne
Junior Member
Posts: 85
|
Post by berne on Nov 15, 2021 8:07:27 GMT -8
So am making progress on this, but I have to give anyone wanting to do a full system backup a warning. SkyCam generates a lot of GB's. There is a possibility that in the process of making of making a backup on your local SD card, that you will max out the storage. Once this happens you will not be able to get into the desktop GUI. You will get a login screen but when you enter a password it will just return to the login. In my case I had to SSH into the RPi, which gives you a terminal on a win10 machine. (Google this) Then df to check disk utilization, and start getting rid of files or apps to free up storage. I got rid of wolfram, Oracle-Java and Scratch2 and Scratch. This gave me enough room to get into the gui thru RealVNC. Then I installed ncdu using apt-get to get a better picture of where all the GB are going. Huge GBs in the static directory so am using the tar --exclude option to avoid backing that up.
One thing for a software bug fix list. Why not use the lux reading to avoid taking skycam jpgs in the middle of the night? These are just black screens and waste a lot of GB's. Anyway Once I get into your software, I will be adding that if you haven't added it already. Also some info on how you manage all the jpgs would be useful. Do you delete files oLder than a certain date? What about the time lapses? Are they also deleted after a certain date? My preference would to kepp the timelapses (from daylight hours only, of course) and delete old jpgs that go into them to manage disk usage. Anyway just putting this out there for folks to help.
|
|
|
|
Post by doxidad on Nov 15, 2021 13:32:24 GMT -8
There are 2 jobs that run every day: cleanPictures which deletes pictures older than 14days and cleanTimeLapes that deletes timelapses older than 14 days. These are found in the module PictureManagement, You could change the number of days you want kept.
In the same module there is a function called buildTimeLapse that you could change the where clause on the SQL to limit the time range of the timelapse video created. That won't stop the pictures from being taken and stored every 50-60seconds though.
|
|
berne
Junior Member
Posts: 85
|
Post by berne on Nov 15, 2021 19:16:48 GMT -8
Thx. Will adjust the jpg's down and just keep the timelapses. I did a tar backup script with --exclude=*.jpg and *.mp4 which solved the time and massive size of the backup.
What about not taking pictures when lux = 0? Any advice on where in software that can be controlled. Also Seems like a bug fix to be entered on the log. (Taking pictures with no light is not a feature!)
I run this using crontab at 2 am, after a shutdown at 12:58 am followed by a smart outlet power cycle to get RPi running again (the Skyweather software is loaded as advised thru rc.local) and mounting the share after allowing time for acquiring the network. I was earlier using the smart outlet to cycle my Ourweather daily because otherwise was getting squirrely stuff happening. Ourweather has been very stable with daily reboot.
Already had one case of squirreliness with the RPi not reading the weather rack which was cured by a reboot.
|
|
|
|
Post by SDL on Nov 16, 2021 18:45:45 GMT -8
Thanks Doxidad. Sometimes I think you know this software better than I do!
SkyWeather2 is supposed to pick up the SDR not working and reboot. I'm adding that to the bug list. Maybe it has quit working.
BP
|
|
|
|
Post by doxidad on Nov 17, 2021 7:55:40 GMT -8
Thanks Doxidad. Sometimes I think you know this software better than I do! Nope, just have had 40+ years looking at other folks code to add/repair features (BTW - features are bugs wearing tuxedos  ). You start learning how to figure out code flow quickly when you have customers wanting stuff fixed/added before you even got on site and you never have seen the code before. Or, having 24X7 production line shut down in the middle of the night because of a problem in L1, L1+ or L2 code that again you may have never seen before. |
|
|
|
Post by SDL on Nov 17, 2021 14:15:04 GMT -8
Great stories! Tell more!
BP
|
|
|
|
Post by mauriceatkinson on Jan 10, 2022 1:30:27 GMT -8
Great backup script and just what I was looking for.
Dumb question...do the skyweather databases need to be offline before the backup scriopt is run? When I try to run this when skyweather is running get the error "permission denied" to the database backup command.
|
|
|
|
Post by SDL on Jan 10, 2022 9:06:38 GMT -8
Hmm. That's a new one on me. What happens if you stop SkyWeather2? Both the program and the dash_app?
Which Backup Script are you using?
BP
|
|
|
|
Post by mauriceatkinson on Jan 10, 2022 11:29:50 GMT -8
I stopped Skyweather and Dash and was able to run the mysqldump command. This was a step forward and I am now able to import the backup into excel.
I am sure there is an easier process that I need to investigate but at least I can do it. thanks
|
|
|
|
Post by doxidad on Jan 10, 2022 16:54:42 GMT -8
You can install phpMyAdmin. It has a very nice web interface for examing and doing maintenance on mysql databases. It also has a very nice extract/backup function. It allows you to export will all data and rebuild the database with the sql file it creates.
|
|
).