Soo I guess I figured it out. Since my machine is running Fedora 38, it comes with SELinux enabled, which does not work well with docker volumes. Doing exactly what I wrote in my previous post on some ubuntu VM I had on my hands brought me straight to the database setup.
So for whoever may have the same issue: Look at your SELinux config or do it on a different machine 
And thinks to Christian for your support!