Sync private iCloud calendar with MagicMirror

mdefrancesco

So i’m not sure if anyone is still watching this thread but I’ve been banging my head against the wall trying to sync. The tutorial is awesome and clearly there was a ton of work put in, so I don’t ask questions lightly.

I followed the directions to the letter but i can’t seem to control the sync. one time it took 2.5 mins and the next time it took 9 mins.

here is my timer…
GNU nano 2.7.4 File: /etc/systemd/user/vdirsyncer.timer

[Unit]
Description=Synchronize vdirs

[Timer]
OnBootSec=1m
OnUnitActiveSec=1m
AccuracySec=1m

[Install]
WantedBy=timers.target

that’s the only thing i have changed. it’s like i can’t get the sync to activate.
I have maybe 5 different modules running but nothing crazy (weather, clock, jeopardy, etc)

if i manually vdirsyncer sync and then reload the mirror AWESOME, no problems. just cant get the damn timer to work

any help or suggestions would be greatly appreciated…I’m still new to all this so thanks

Beh

@mdefrancesco Sorry for the late reply.

Hmm… that’ weird. Did you get it working by now?

If not:
Did you use the --user flag when activating the timer? And what does systemctl --user status vdirsyncer.timer say?
And did you try different times in the timer file? I could imagine that setting everything to 1 minute could cause problems…

cagey

@Beh first and foremost, thank you very much for setting out clear and detailed instructions. I’m a complete newbie but by following your instructions it works perfectly.

I saw that, when I run “vdircyncer discover”, I receive a list of all my calenders and, in addition, my iCloud reminders. The reminders seem to have similar UUIDs to the calendars.

I tried following the same process for my “Family” reminders by creating a CALDAV sync (in the vdirsyncer config file) called “pair iCloudTD_to_MagicMirrorTD”, including the Family UUID in the “collections” section and creating a %.ics file in /MagicMirror/modules/calendars/%s.ics. I have items_types as “VEVENT”, “VTODO”.

I also inserted the %.ics file as a calendar entry in the “calendar” module in MagicMirror.

However, the to do list does not appear in MM (it just says “No upcoming events”). The calendar however works perfectly.

I’m guessing it’s not possible to import iCloud reminders in the same way as iCloud calendars but I thought I’d ask before I spend more time reading through the vdirsyncer documentation and other Google searches.

Is there a config setting I’m missing or is it not possible at all? I’m struggling to understand the difference between VTODO and reminders.

thisistheplace

@cagey I tried that too and asked for help in september last year.

Somewhere in this thread @Beh said it should work with “VTODO” but the ics files are empty after syncing. Thats why nothing is shown in your calendar.

Would be really happy to have the icloud reminders on my mirror.

elliot1996

@Beh Hey, great tutorial! Im having the same issues as above, timers just not working. Followed your troubleshooting, still no luck, just says ; Unit vdirsyncer.timer could not be found.

Even though I can see it does exists and its editable in nano

Any ideas?

elliot1996

This post is deleted!

Beh

@thisistheplace I never tried syncing VTODO. All I can do here, is to refer to the vdirsyncer docs regarding this topic: https://vdirsyncer.pimutils.org/en/stable/config.html?highlight=VEVENT#storage-caldav

Unfortunately you have to dig into that yourself, my time is very limted currently…

Beh

@elliot1996 I would need more information to help you debug this. Are you sure you used the right directory for the file? And you used the --user option when executing systemctl?

Everything you need to know about systemd unit files is here: https://www.freedesktop.org/software/systemd/man/systemd.unit.html

The file must be in one of the directories mentioned at the top of the page.

elliot1996

@Beh I’m sure I have installed it in the right directory, I followed :curl https://raw.githubusercontent.com/pimutils/vdirsyncer/master/contrib/vdirsyncer.timer | sudo tee /etc/systemd/user/vdirsyncer.timer firstly under the MagicMirror Directory. Then activating using systemctl --user enable vdirsyncer.timer.

When looking at the status I get : Warning: The unit file, source configuration file or drop-ins of vdirsyncer.timer changed on disk. Run ‘systemctl --user daemon-reload’ to rel
● vdirsyncer.timer - Synchronize vdirs
Loaded: loaded (/etc/systemd/user/vdirsyncer.timer; enabled; vendor preset: enabled)
Drop-In: /home/smartm/.config/systemd/user/vdirsyncer.timer.d
└─override.conf
Active: inactive (dead)
Trigger: n/a

Apr 13 18:38:50 smartm-Latitude-E6230 systemd[915]: /home/smartm/.config/systemd/user/vdirsyncer.timer.d/override.conf:1: Assignment outside o
Apr 13 18:38:50 smartm-Latitude-E6230 systemd[915]: /home/smartm/.config/systemd/user/vdirsyncer.timer.d/override.conf:2: Assignment outside o
Apr 13 18:46:21 smartm-Latitude-E6230 systemd[915]: /home/smartm/.config/systemd/user/vdirsyncer.timer.d/override.conf:1: Assignment outside o
Apr 13 18:46:21 smartm-Latitude-E6230 systemd[915]: /home/smartm/.config/systemd/user/vdirsyncer.timer.d/override.conf:2: Assignment outside o
lines 1-12/12 (END)

Anything you could do to help would be amazing! Been trying to get it going all day, my mirrors close to perfect!

ShoeBox

@Beh this is a great tutorial. Thanks!

Jaunney

@Beh

Hi!

Thank you for your great solution. All went well till I tried to sync more than one iCloud Calendars.
Something is getting confused I think.
Could you please make an example for the right syntax in the following section:
collections = [“UUID1”, “UUID2”, “UUID3”]
Something like this?

I also have now problems to sync two non-iCloud Calenders after the great confusion…
My module (I use MMM-CalendarExt2) only shows one iCloud-calendar.

Thanks for your help.

BEst regards

Jaunney

duxnobis

Hi. Try to participate as well :-). I am totally new to the Raspberry world and so to MM. I have set up using the step-by-step guide. Thanks for this a lot. Now that I have set this up as expected, the result is not as expected. The other modules work well. Also 3rd party ones. But the calendar says “No upcoming event”. I have managed to obtain the according icloud calendar ID for my PRIVATE calendar. Any idea what I could potentially have wrongly set up or installed?

● vdirsyncer.timer - Synchronize vdirs
Loaded: loaded (/etc/systemd/user/vdirsyncer.timer; enabled; vendor preset: enabled)
Active: inactive (dead)
Trigger: n/a

pi@raspberrypi:~/.vdirsyncer $ vdirsyncer discover
Discovering collections for pair iCloud_to_MagicMirror
Mirror:

“14Exxxxx-xxxx-xxxx-xxxx-xxxxxxxxxEF2”
iCloud:
“14Exxxxx-xxxx-xxxx-xxxx-xxxxxxxxxEF2” (“Patrick”)
“73Fxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxB3F” (“Kalender”)
“tasks” (“Erinnerungen”)
“fc79xxxdf28cxxxxxxxxx42c0dxxxx7d6e3cf0xxxxxxcac0b2e4xxxxxxxxx” (“Peter”)
Saved for iCloud_to_MagicMirror: collections = [“14Exxxxx-xxxx-xxxx-xxxx-xxxxxxxxxEF2”]

config file:

CALDAV Sync

[pair iCloud_to_MagicMirror]
a = “Mirror”
b = “iCloud”
collections = [“14Exxxxx-xxxx-xxxx-xxxx-xxxxxxxxxEF2”]

Calendars also have a color property

metadata = [“displayname”, “color”]

[storage Mirror]

We need a single .ics file for use with the mirror (Attention! This is really slow on big amounts of events.)

type = “singlefile”

We’ll put the calendar file to a readable location for the calendar module

path = “/home/pi/MagicMirror/modules/calendars/%s.ics”

[storage iCloud]
type = “caldav”
url = “https://caldav.icloud.com/”

Authentication credentials

username = “xxxxxxxxxxx@mailprovider.com”
password = “xxxx-xxxx-xxxx-xxxx”

We only want to sync in the direction TO the mirror, so we make iCloud readonly

read_only = true

We only want to sync events

item_types = [“VEVENT”]

Can anyone of you help? Maybe you see what I did wrong or you know how I could check some points to see if I did something wrong.

Thanks for your inputs. Keep in mind, I am new in this ;-)
Patrick

duxnobis

@elliot1996 said in Sync private iCloud calendar with MagicMirror:

@Beh I’m sure I have installed it in the right directory, I followed :curl https://raw.githubusercontent.com/pimutils/vdirsyncer/master/contrib/vdirsyncer.timer | sudo tee /etc/systemd/user/vdirsyncer.timer firstly under the MagicMirror Directory. Then activating using systemctl --user enable vdirsyncer.timer.

When looking at the status I get : Warning: The unit file, source configuration file or drop-ins of vdirsyncer.timer changed on disk. Run ‘systemctl --user daemon-reload’ to rel
● vdirsyncer.timer - Synchronize vdirs
Loaded: loaded (/etc/systemd/user/vdirsyncer.timer; enabled; vendor preset: enabled)
Drop-In: /home/smartm/.config/systemd/user/vdirsyncer.timer.d
└─override.conf
Active: inactive (dead)
Trigger: n/a

Hi mate. I have the exact same issue. I have everything now up and running. I managed to have two private ical calendars showing up in the mirror. If I sync manually and reload it works fine. Unfortunately I dont manage the damn thing to do it automatically, ideally triggered on an event or at least all few minutes. My settings in the timer are 1min, 2min, 3min. Just because Beh said to avoid all 1min settings. Any Idea?

tobias789

This post is deleted!

Feedy88

@duxnobis @elliot1996 @Beh
Hey guys,
just checking in on this topic, as I also have issues with getting the timer to run while manual sync works fine.

Files are in /etc/systemd/user/ (vdirsyncer.service & vdirsyncer.timer)
Ran systemctl --user enable vdirsyncer.timer? (no error or any other feedback in Terminal)
Checked jorunalctl --user -u vdirsyncer with the result:
No journal files were found. -- No Entries --

I even went forward and tried to sync using a cronjob but this is unfortuately also not working:

error: Unknown error occured for iCloud_to_MagicMirror/9*******-****-****-****-************D: HTTPSConnectionPool(host='p65-caldav.icloud.com', port=443): Max retries exceeded with url: /1355377534/calendars/9*******-****-****-****-************D/ (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])")))
error: Use `-vdebug` to see the full traceback.
error: 1 out of 2 tasks failed.
Syncing iCloud_to_MagicMirror/9*******-****-****-****-************D

Anyone has an Idea?

Feedy88

@elliot1996 @duxnobis not sure if anyone is still following, but I was able to get the automated sync with a work around through cronjob. Step-by-step guide below:

Create Shell script

First we need to create the script which is actually executed by the cronjob. To do this we open a terminal and type

nano ~/MagicMirror/modules/calendars/vdirsyncer.sh

Within the editor we just write a one-liner:

vdirsyncer -vinfo sync

Close by CTRL+X and save by confirming with Y

Make script executable.

Within terminal type

chmod +x ~/MagicMirror/modules/calendars/vdirsyncer.sh

Test the script

Run a test if your script works. Within terminal type

bash ~/MagicMirror/modules/calendars/vdirsyncer.sh

If working as intended you should receive the feedback

Syncing iCloud_to_MagicMirror/YOUR-CALENDAR-UUID

Checking/Changing the Version of vdirsyncer

This was actually my biggest issue. I could sync successfully when running the script manually but the actual cronjob failed. Looking into Debug-Logs I found that the manual run used Version 0.16.8 of vdirsyncer while the cronjob used 0.16.7. After manually chaning the versioning if worked as inteded. To ensue you are not running into that issue, let’s check the versioning from the get-go. Within Terminal type

sudo nano /usr/bin/vdirsyncer

An editor will show you the following code:

#!/usr/bin/python3
# EASY-INSTALL-ENTRY-SCRIPT: 'vdirsyncer==0.16.7','console_scripts','vdirsyncer'
__requires__ = 'vdirsyncer==0.16.7'
import re
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(
        load_entry_point('vdirsyncer= q=0.16.7', 'console_scripts', 'vdirsyncer')()
    )

Change all three instances of 0.16.7to 0.16.8, close via CTRL+X and save with Y.

Create the Cronjob

Cronjobs are a great thing. It is pre-installed with UNIX based systems, therefore also with our Raspberry Pi. More information about Cronjobs can be found here. To set up a new cronjob, within Terminal type

crontab -e

The result will be the following code:

# Edit this file to introduce tasks to be run by cron.
#
# Each task to run has to be defined through a single line
# indicating with different fields when the task will be run
# and what command to run for the task
# 
# To define the time you can provide concrete values for
# minute (m), hour (h), day of month (dom), month (mon),
# and day of week (dow) or use '*' in these fields (for 'any').
# 
# Notice that tasks will be started based on the cron's system
# daemon's notion of time and timezones.
# 
# Output of the crontab jobs (including errors) is sent through
# email to the user the crontab file belongs to (unless redirected).
# 
# For example, you can run a backup of all your user accounts
# at 5 a.m every week with:
# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
# 
# For more information see the manual pages of crontab(5) and cron(8)
# 
# m h  dom mon dow   command

Type a new line at the bottom with the cron syntax. Here is an overview for the timings:

# ┌───────────── minute (0 - 59)
# │ ┌───────────── hour (0 - 23)
# │ │ ┌───────────── day of the month (1 - 31)
# │ │ │ ┌───────────── month (1 - 12)
# │ │ │ │ ┌───────────── day of the week (0 - 6) (Sunday to Saturday;
# │ │ │ │ │                                   7 is also Sunday on some systems)
# │ │ │ │ │
# │ │ │ │ │
# * * * * * <command to execute>

As for us it makes senst to run vdirsyncer every X minutes (I am going with 5 minutes in my case) the syntax looks as follows:

*/5 * * * *

followed by the script to execute:

bash ~/MagicMirror/modules/calendars/vdirsyncer.sh >> ~/vdirsyncer.log 2>&1

The first part should be familiar to you. What follows after the >> is a log we are creating. This will be helpful in case anything does not work as inteded. The full cronjob should look as follows:

# Edit this file to introduce tasks to be run by cron.
#
# Each task to run has to be defined through a single line
# indicating with different fields when the task will be run
# and what command to run for the task
# 
# To define the time you can provide concrete values for
# minute (m), hour (h), day of month (dom), month (mon),
# and day of week (dow) or use '*' in these fields (for 'any').
# 
# Notice that tasks will be started based on the cron's system
# daemon's notion of time and timezones.
# 
# Output of the crontab jobs (including errors) is sent through
# email to the user the crontab file belongs to (unless redirected).
# 
# For example, you can run a backup of all your user accounts
# at 5 a.m every week with:
# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
# 
# For more information see the manual pages of crontab(5) and cron(8)
# 
# m h  dom mon dow   command
*/5 * * * * bash ~/MagicMirror/modules/calendars/vdirsyncer.sh >> ~/vdirsyncer.log 2>&1

Close again by CTRL+X. You are now ready and have everything set up. The cronjob will start automatically on every startup and run the sync in the interval you have set it up.

sdetweil

@Feedy88 and if u edit your script and make the 1st line
#!/bin/bash

when the script is invoked it will launch the script using bash so all the bash symbols, redirection etc can be moved into the script, and you just use the script and to.launch it

/home/pi/MagicMirror/modules/calendars/vdirsyncer.sh

also, cron typically executes commands as root, so ~/filename will try to write to /root/filename, but not where u intended. so use the full path

/home/pi/MagicMirror/modules/calendars/vdirsyncer.sh

Feedy88

@sdetweil Thanks for the notes, but can you explain a bit more into detail. You mean, if I add

#!/bin/bash as first line to the script, I can also add the logging part and remove it from the cron?

To be honest, I’m quite a noob to Linux based systems and scripts. Nonetheless, hope my answer will help some people to get the sync going who have issues with the timer service from vdirsyncer.

sdetweil

@Feedy88 yes, if you use the exec command in the script you can capture stdout & stderr and pipe them to a file

LOG_FILE=filename

Open standard out at `$LOG_FILE` for write.

This has the effect

exec 1>$LOG_FILE

Redirect standard error to standard out such that

standard error ends up going to wherever standard

out goes (the file).

exec 2>&1

normal rest of your script

jlnxaer

Hello Guys,

I’m fairly new to the whole Raspberry Pi Thing.
I got the whole Calendar up and running. My only problem is, that it is not syncing automatically…
Do you guys know any fix for this problem?

The fix from @Feedy88 didn’t work for me unfortunately.

Cheers!