Ok, I went through the payment page again, and apparently the issue was that my browser pre-filled the country, and even if I re-select the correct country, then it would still not update the State / Region dropdown. I reloaded the page and manually entered my details, and now it works as intended.
Thanks @budrun for showing that it actually updates when the country is selected.