0x80070002 Error with Bash on Windows following Creators Update

After installing the Windows Creators update I found that "Bash on Ubuntu on Windows" stopped working - when running from the Start Menu the window instantly closed, when running from CMD with normal user permissions it came back with the 0x80070002 error.

This appears to be because I installed using my Administrator account, although it's not possible to follow the official instructions without elevating permissions. It may've also been affected by using a Roaming Profile.

Here are the steps I took to fix, (this completely wipes any previous data), it may include irrelevant steps as well;

I ran the Administrator Command Prompt and typed:

lxrun /uninstall /full /y

Then deleted %localappdata%\lxx using my normal account.

After rebooting I ran CMD with my normal Windows Account and uninstalled

lxrun /uninstall /full /y

I then installed from the same command prompt

lxrun /install

This prompted "Downloading from the Windows Store..." and finished by getting me to set up my new UNIX account, Bash on Ubuntu on Windows has now appeared in the Start Menu as expected.