Sync private iCloud calendar with MagicMirror

Beh · Apr 13, 2020, 9:52 PM

@thisistheplace I never tried syncing VTODO. All I can do here, is to refer to the vdirsyncer docs regarding this topic: https://vdirsyncer.pimutils.org/en/stable/config.html?highlight=VEVENT#storage-caldav

Unfortunately you have to dig into that yourself, my time is very limted currently…

Beh · Apr 13, 2020, 9:56 PM

@elliot1996 I would need more information to help you debug this. Are you sure you used the right directory for the file? And you used the --user option when executing systemctl?

Everything you need to know about systemd unit files is here: https://www.freedesktop.org/software/systemd/man/systemd.unit.html

The file must be in one of the directories mentioned at the top of the page.

elliot1996 · May 12, 2020, 8:44 AM

@Beh I’m sure I have installed it in the right directory, I followed :curl https://raw.githubusercontent.com/pimutils/vdirsyncer/master/contrib/vdirsyncer.timer | sudo tee /etc/systemd/user/vdirsyncer.timer firstly under the MagicMirror Directory. Then activating using systemctl --user enable vdirsyncer.timer.

When looking at the status I get : Warning: The unit file, source configuration file or drop-ins of vdirsyncer.timer changed on disk. Run ‘systemctl --user daemon-reload’ to rel
● vdirsyncer.timer - Synchronize vdirs
Loaded: loaded (/etc/systemd/user/vdirsyncer.timer; enabled; vendor preset: enabled)
Drop-In: /home/smartm/.config/systemd/user/vdirsyncer.timer.d
└─override.conf
Active: inactive (dead)
Trigger: n/a

Apr 13 18:38:50 smartm-Latitude-E6230 systemd[915]: /home/smartm/.config/systemd/user/vdirsyncer.timer.d/override.conf:1: Assignment outside o
Apr 13 18:38:50 smartm-Latitude-E6230 systemd[915]: /home/smartm/.config/systemd/user/vdirsyncer.timer.d/override.conf:2: Assignment outside o
Apr 13 18:46:21 smartm-Latitude-E6230 systemd[915]: /home/smartm/.config/systemd/user/vdirsyncer.timer.d/override.conf:1: Assignment outside o
Apr 13 18:46:21 smartm-Latitude-E6230 systemd[915]: /home/smartm/.config/systemd/user/vdirsyncer.timer.d/override.conf:2: Assignment outside o
lines 1-12/12 (END)

Anything you could do to help would be amazing! Been trying to get it going all day, my mirrors close to perfect!

ShoeBox · Apr 25, 2020, 2:24 AM

@Beh this is a great tutorial. Thanks!

Jaunney · Nov 9, 2020, 9:01 PM

@Beh

Hi!

Thank you for your great solution. All went well till I tried to sync more than one iCloud Calendars.
Something is getting confused I think.
Could you please make an example for the right syntax in the following section:
collections = [“UUID1”, “UUID2”, “UUID3”]
Something like this?

I also have now problems to sync two non-iCloud Calenders after the great confusion…
My module (I use MMM-CalendarExt2) only shows one iCloud-calendar.

Thanks for your help.

BEst regards

Jaunney

duxnobis · May 10, 2020, 2:29 PM

Hi. Try to participate as well :-). I am totally new to the Raspberry world and so to MM. I have set up using the step-by-step guide. Thanks for this a lot. Now that I have set this up as expected, the result is not as expected. The other modules work well. Also 3rd party ones. But the calendar says “No upcoming event”. I have managed to obtain the according icloud calendar ID for my PRIVATE calendar. Any idea what I could potentially have wrongly set up or installed?

● vdirsyncer.timer - Synchronize vdirs
Loaded: loaded (/etc/systemd/user/vdirsyncer.timer; enabled; vendor preset: enabled)
Active: inactive (dead)
Trigger: n/a

pi@raspberrypi:~/.vdirsyncer $ vdirsyncer discover
Discovering collections for pair iCloud_to_MagicMirror
Mirror:

“14Exxxxx-xxxx-xxxx-xxxx-xxxxxxxxxEF2”
iCloud:
“14Exxxxx-xxxx-xxxx-xxxx-xxxxxxxxxEF2” (“Patrick”)
“73Fxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxB3F” (“Kalender”)
“tasks” (“Erinnerungen”)
“fc79xxxdf28cxxxxxxxxx42c0dxxxx7d6e3cf0xxxxxxcac0b2e4xxxxxxxxx” (“Peter”)
Saved for iCloud_to_MagicMirror: collections = [“14Exxxxx-xxxx-xxxx-xxxx-xxxxxxxxxEF2”]

config file:

CALDAV Sync

[pair iCloud_to_MagicMirror]
a = “Mirror”
b = “iCloud”
collections = [“14Exxxxx-xxxx-xxxx-xxxx-xxxxxxxxxEF2”]

Calendars also have a color property

metadata = [“displayname”, “color”]

[storage Mirror]

We need a single .ics file for use with the mirror (Attention! This is really slow on big amounts of events.)

type = “singlefile”

We’ll put the calendar file to a readable location for the calendar module

path = “/home/pi/MagicMirror/modules/calendars/%s.ics”

[storage iCloud]
type = “caldav”
url = “https://caldav.icloud.com/”

Authentication credentials

username = “xxxxxxxxxxx@mailprovider.com”
password = “xxxx-xxxx-xxxx-xxxx”

We only want to sync in the direction TO the mirror, so we make iCloud readonly

read_only = true

We only want to sync events

item_types = [“VEVENT”]

Can anyone of you help? Maybe you see what I did wrong or you know how I could check some points to see if I did something wrong.

Thanks for your inputs. Keep in mind, I am new in this ;-)
Patrick

duxnobis · May 12, 2020, 8:44 AM

@elliot1996 said in Sync private iCloud calendar with MagicMirror:

@Beh I’m sure I have installed it in the right directory, I followed :curl https://raw.githubusercontent.com/pimutils/vdirsyncer/master/contrib/vdirsyncer.timer | sudo tee /etc/systemd/user/vdirsyncer.timer firstly under the MagicMirror Directory. Then activating using systemctl --user enable vdirsyncer.timer.

When looking at the status I get : Warning: The unit file, source configuration file or drop-ins of vdirsyncer.timer changed on disk. Run ‘systemctl --user daemon-reload’ to rel
● vdirsyncer.timer - Synchronize vdirs
Loaded: loaded (/etc/systemd/user/vdirsyncer.timer; enabled; vendor preset: enabled)
Drop-In: /home/smartm/.config/systemd/user/vdirsyncer.timer.d
└─override.conf
Active: inactive (dead)
Trigger: n/a

Hi mate. I have the exact same issue. I have everything now up and running. I managed to have two private ical calendars showing up in the mirror. If I sync manually and reload it works fine. Unfortunately I dont manage the damn thing to do it automatically, ideally triggered on an event or at least all few minutes. My settings in the timer are 1min, 2min, 3min. Just because Beh said to avoid all 1min settings. Any Idea?

tobias789 · Jul 14, 2020, 2:33 PM

This post is deleted!

Feedy88 · Jul 31, 2020, 11:46 AM

@duxnobis @elliot1996 @Beh
Hey guys,
just checking in on this topic, as I also have issues with getting the timer to run while manual sync works fine.

Files are in /etc/systemd/user/ (vdirsyncer.service & vdirsyncer.timer)
Ran systemctl --user enable vdirsyncer.timer? (no error or any other feedback in Terminal)
Checked jorunalctl --user -u vdirsyncer with the result:
No journal files were found. -- No Entries --

I even went forward and tried to sync using a cronjob but this is unfortuately also not working:

error: Unknown error occured for iCloud_to_MagicMirror/9*******-****-****-****-************D: HTTPSConnectionPool(host='p65-caldav.icloud.com', port=443): Max retries exceeded with url: /1355377534/calendars/9*******-****-****-****-************D/ (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])")))
error: Use `-vdebug` to see the full traceback.
error: 1 out of 2 tasks failed.
Syncing iCloud_to_MagicMirror/9*******-****-****-****-************D

Anyone has an Idea?

Feedy88 · Aug 2, 2020, 2:45 PM

@elliot1996 @duxnobis not sure if anyone is still following, but I was able to get the automated sync with a work around through cronjob. Step-by-step guide below:

Create Shell script

First we need to create the script which is actually executed by the cronjob. To do this we open a terminal and type

nano ~/MagicMirror/modules/calendars/vdirsyncer.sh

Within the editor we just write a one-liner:

vdirsyncer -vinfo sync

Close by CTRL+X and save by confirming with Y

Make script executable.

Within terminal type

chmod +x ~/MagicMirror/modules/calendars/vdirsyncer.sh

Test the script

Run a test if your script works. Within terminal type

bash ~/MagicMirror/modules/calendars/vdirsyncer.sh

If working as intended you should receive the feedback

Syncing iCloud_to_MagicMirror/YOUR-CALENDAR-UUID

Checking/Changing the Version of vdirsyncer

This was actually my biggest issue. I could sync successfully when running the script manually but the actual cronjob failed. Looking into Debug-Logs I found that the manual run used Version 0.16.8 of vdirsyncer while the cronjob used 0.16.7. After manually chaning the versioning if worked as inteded. To ensue you are not running into that issue, let’s check the versioning from the get-go. Within Terminal type

sudo nano /usr/bin/vdirsyncer

An editor will show you the following code:

#!/usr/bin/python3
# EASY-INSTALL-ENTRY-SCRIPT: 'vdirsyncer==0.16.7','console_scripts','vdirsyncer'
__requires__ = 'vdirsyncer==0.16.7'
import re
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(
        load_entry_point('vdirsyncer= q=0.16.7', 'console_scripts', 'vdirsyncer')()
    )

Change all three instances of 0.16.7to 0.16.8, close via CTRL+X and save with Y.

Create the Cronjob

Cronjobs are a great thing. It is pre-installed with UNIX based systems, therefore also with our Raspberry Pi. More information about Cronjobs can be found here. To set up a new cronjob, within Terminal type

crontab -e

The result will be the following code:

# Edit this file to introduce tasks to be run by cron.
#
# Each task to run has to be defined through a single line
# indicating with different fields when the task will be run
# and what command to run for the task
# 
# To define the time you can provide concrete values for
# minute (m), hour (h), day of month (dom), month (mon),
# and day of week (dow) or use '*' in these fields (for 'any').
# 
# Notice that tasks will be started based on the cron's system
# daemon's notion of time and timezones.
# 
# Output of the crontab jobs (including errors) is sent through
# email to the user the crontab file belongs to (unless redirected).
# 
# For example, you can run a backup of all your user accounts
# at 5 a.m every week with:
# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
# 
# For more information see the manual pages of crontab(5) and cron(8)
# 
# m h  dom mon dow   command

Type a new line at the bottom with the cron syntax. Here is an overview for the timings:

# ┌───────────── minute (0 - 59)
# │ ┌───────────── hour (0 - 23)
# │ │ ┌───────────── day of the month (1 - 31)
# │ │ │ ┌───────────── month (1 - 12)
# │ │ │ │ ┌───────────── day of the week (0 - 6) (Sunday to Saturday;
# │ │ │ │ │                                   7 is also Sunday on some systems)
# │ │ │ │ │
# │ │ │ │ │
# * * * * * <command to execute>

As for us it makes senst to run vdirsyncer every X minutes (I am going with 5 minutes in my case) the syntax looks as follows:

*/5 * * * *

followed by the script to execute:

bash ~/MagicMirror/modules/calendars/vdirsyncer.sh >> ~/vdirsyncer.log 2>&1

The first part should be familiar to you. What follows after the >> is a log we are creating. This will be helpful in case anything does not work as inteded. The full cronjob should look as follows:

# Edit this file to introduce tasks to be run by cron.
#
# Each task to run has to be defined through a single line
# indicating with different fields when the task will be run
# and what command to run for the task
# 
# To define the time you can provide concrete values for
# minute (m), hour (h), day of month (dom), month (mon),
# and day of week (dow) or use '*' in these fields (for 'any').
# 
# Notice that tasks will be started based on the cron's system
# daemon's notion of time and timezones.
# 
# Output of the crontab jobs (including errors) is sent through
# email to the user the crontab file belongs to (unless redirected).
# 
# For example, you can run a backup of all your user accounts
# at 5 a.m every week with:
# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
# 
# For more information see the manual pages of crontab(5) and cron(8)
# 
# m h  dom mon dow   command
*/5 * * * * bash ~/MagicMirror/modules/calendars/vdirsyncer.sh >> ~/vdirsyncer.log 2>&1

Close again by CTRL+X. You are now ready and have everything set up. The cronjob will start automatically on every startup and run the sync in the interval you have set it up.