Setup process error with custom PHP environment

andri · February 28, 2019, 12:32pm

Hi there!

On our latest webserver generation, we’re using a flexible approach to configure PHP versions and extensions based on environment variables (entry in PATH points to the appropriate PHP version, PHP_INI_SCAN_DIR is used to select custom global and local configuration).

This works fine so far, however we discovered a problem during the Neos setup. After step 2 (“Configure database”), Neos will call itself through PHP’s exec function:

github.com

neos/flow-development-collection/blob/master/Neos.Flow/Classes/Core/Booting/Scripts.php#L694


 * @return boolean true if the command execution was successful (exit code = 0)
 * @api
 * @throws Exception\SubProcessException if execution of the sub process failed
 */
public static function executeCommand(string $commandIdentifier, array $settings, bool $outputResults = true, array $commandArguments = []): bool
{
    $command = self::buildSubprocessCommand($commandIdentifier, $settings, $commandArguments);
    $output = [];
    // Output errors in response
    $command .= ' 2>&1';
    exec($command, $output, $result);
    if ($result !== 0) {
        if (count($output) > 0) {
            $exceptionMessage = implode(PHP_EOL, $output);
        } else {
            $exceptionMessage = sprintf('Execution of subprocess failed with exit code %d without any further output. (Please check your PHP error log for possible Fatal errors)', $result);


            // If the command is too long, it'll just produce /usr/bin/php: Argument list too long but this will be invisible
            // If anything else goes wrong, it may as well not produce any $output, but might do so when run on an interactive
            // shell. Thus we dump the command next to the exception dumps.
            $exceptionMessage .= ' Try to run the command manually, to hopefully get some hint on the actual error.';

On this call, any environment variables are lost and the exec is not able to execute itself due to missing modules (those are loaded in some specific php.ini in PHP_INI_SCAN_DIR):

Exception #1355480641 in line 143 of /home/neos/Data/Temporary/Production/Cache/Code/Flow_Object_Classes/Neos_Setup_Step_DatabaseStep.php: An exception occurred in driver: could not find driver

I see that this is not a problem related to Neos in particular but to our setup, however i try to find more background infos about this issue:

any insights about the whole mechanism to understand why Neos is calling itself like this
any hints to circumvent this
does anyone else use a setup like this
will you be willing to accept a path which would load local environment variables if defined (something like prepending test -f ~/.profile && source ~/.profile to the call, but a little more sophisticated and portable i guess)

Thanks,
Andri

sebastian · March 1, 2019, 3:18pm

Hey Andri,

Neos/Flow is pre-compiling PHP classes (generating intermediate proxy classes) to implement advanced features like Aspect-Oriented Programming or Dependency Injection in a very peformant way. In order to do this, we must load the classes to inspect them (using reflection). As it is not possible to un-load a class again and load a generated proxy instead, we need to do this in a sub request. (That’s the so-called “compile step” of Flow).

You can not circumvent this. What you can do, is influence this behavior via Settings.yaml, see these lines in the config.

You could even take environment variables and put them in there, e.g.:

Neos:
  Flow:
    core:
      subRequestEnvironmentVariables:
        MY_ENV_VAR: '%env:MY_ENV_VAR%'

All the best, hope this helps
Sebastian

andri · March 1, 2019, 3:44pm

Hi Sebastian,

Thanks for your clarifications. Let me try if i can adopt the local environment settings as you proposed. I’ll let you know about the outcome.

Thanks,
Andri

christianm · March 1, 2019, 4:07pm

Have a look at the called `Scripts:: buildSubprocessCommand`` as well as you can see how we construct the call including the known env.

andri · April 5, 2019, 2:53pm

Hey Guys

Thanks for your help. I took some time to test everything today, and ended up with the environment variable defined in Settings.yaml as proposed by Sebastian.

Our managed server documentation is updated accordingly, so we’re good to go for the moment.

I don’t know if others use such “special” setups as well, if yes, you might wanna consider to loop through all existing environments variables to the subprocess - don’t know about the drawbacks which exist for sure though.

Keep up the good work.

Cheers,
Andri