I personally do not like that a part of the system path is displayed when the password location is shown.
Of course, it is possible to hack-fix that in the Neos code. But exposing on production the setup page also does not look well, so disabling or hiding the setup in a reversible manner seems to be a better choice.
I’ve tried composer remove neos/setup (but after the Neos was initialized) and moving Packages\Application\Neos.Setup out followed by flow package:rescan. I did not like the results in both cases, as I got “nothing noticeable” and “Error 500” with creation of a NEW file(that’s BAD - think about inodes limit on VPS) on the filesystem respectively.