I did this as a test with Radarr and it completely reset everything.
Had to redo mounts, set the new API Key everywhere etc. Bit annoying and unfortunate, won't be doing it with Sonarr etc as I did a ton of stuff in there unless it's absolutely necessary.
The UI wants me to update the configs for Sonarr, Jellyfin, Radarr (done this one as it's the least used of my curated apps) and Prowlarr.