404 on all subpages after upgrading Optimizely CMS to 11 latest?

For some solutions upgrading Optimizely to past version 11.20.11 will be a rather strange experience. Start page works, login to Optimizely CMS works, everything inside edit mode works, but browsing subpages gives you the famous 404 Not Found. A faulty setup and a bug fix are to "blame".

Up until Optimizely CMS 11.20.11, everything works as expected, but from 11.20.12 and upwards, the 404 Not Found code might show up. After asking around, and reading the release notes from 11.20.12, I've found my problem. There was a bug fix in version 11.20.12 of EPiServer.CMS.Core named Language mapping for wildcard host affect other sites. This fix could affect your solution if it's not correctly set up.

In the solution I was working on, the setup for default language was missing in site information in Optimizely admin. Old setup:

Old site information - notice no Culture set on localhost:9267

This has been working up and until CMS 11.20.11, but after the earlier mentioned bugfix, subpages now will give a 404:

Trying to load a page with old site information after updated CMS

Setting the culture correctly, and trying again:

Fixing issue with missing culture
Request to page will no longer give 404 Not Found

Remember to keep your site information and languages set up correctly!