Never mind, I found the answer. If anyone is searching for this in the future, in /boot/config.txt, set

hdmi_force_hotplug=1
hdmi_group=2
hdmi_mode=82

More info: https://www.raspberrypi.org/forums/viewtopic.php?f=91&t=19600