Grub cryptomount taking longer and longer

Why does Grub's cryptomount keep taking longer and longer after I enter the password? Is this a disk corruption issue? It takes at least 5 minutes to decrypt a volume now.

Could it be your filesystem? Try a defrag if you filesystem on spinning rust, heck even try a defrag on flash if you ignore the fake danger.

How long did it take when you first started using it? Compared to how long it takes now?
What changes did you make to the system which might affect it, if any?
Have you used fsck to do a filesystem check if it is a Linux filesystem?
How old is the drive? Have you run any diagnostics on it?

I'm pretty sure it's not an issue with the drives. It's a GRUB issue.
I can mount the LUKS version 2 drives perfectly quickly with cryptsetup, but GRUB's cryptomount takes a few minutes.
I updated from i386-pc GRUB 2.06 to x86_64-efi GRUB 2.12 today, but still GRUB's cryptomount takes a few minutes to open the drives.

GRUB bug #59408 led me to discover this thread, which states that the issue is that SSE registers are disabled in GRUB, so the pbkdf iterations take much longer than cryptsetup with SSE enabled.

How do I do enable/disable SSE in GRUB? I don't understand the vague instructions here.