Logging in a containerised setup

Using Neos & Flow
1

Hello Neos-Community & Team,

i’m wondering if i can change/configure the neos logging output to stdout. For this i’ve found the post from @sorenmalling back in 2019. Unfortunately i couldn’t find any setting named logFileURL within the Settings.yaml.

My goal is to basically ship the logfiles to an OpenSearch instance with e.g. fluent-bit or/and Data Prepper. Did anyone of you did that or can share some knowledge about it?

That would be great :slight_smile:

Thanks a lot!

2

Both the ConsoleBackend (GitHub Source) and AnsiConsoleBackend (GitHub source) logs to STDOUT

Beware, that the first hit in the browser will most likely fill your browser with logs (due to STDOUT being the browser in that first compile request cycle).

Configuration goes like this:

Neos:
  Flow:
    log:
      psr3:
        'Neos\Flow\Log\PsrLoggerFactory':
          systemLogger:
            default:
              class: Neos\Flow\Log\Backend\ConsoleBackend
          securityLogger:
            default:
              class: Neos\Flow\Log\Backend\ConsoleBackend
          sqlLogger:
            default:
              class: Neos\Flow\Log\Backend\ConsoleBackend
          i18nLogger:
            default:
              class: Neos\Flow\Log\Backend\ConsoleBackend
3

Hey @sorenmalling, thanks a lot for your input!

I’ve tried to update the Settings with this and it was throwing me lots of

Building DI proxy for [...] messages as i run node:repair.

But it seems like it is only showing the logs from flow, nothing from /Data/Logs/*.

4

As expected, since you are running it in the console, that being STDOUT

/Data/Logs is the default path configured when using the FileBackend, look in the source here

Since you changed the class to ConsoleBackend, all logs that would go to Data/Logs text files, is now being echoed to STDOUT

5

@Benjamin Did it help you?

6

Hey @sorenmalling! Thanks for your reply :slight_smile:

Unfortunately the logs we are looking for are not coming to stdout. I’ll tell you step by step what i did to trigger an exception:

I’ve created the Settings.yaml within the path /Configuration/

Neos:
  Flow:
    log:
      psr3:
        'Neos\Flow\Log\PsrLoggerFactory':
          systemLogger:
            default:
              class: Neos\Flow\Log\Backend\ConsoleBackend
          securityLogger:
            default:
              class: Neos\Flow\Log\Backend\ConsoleBackend
          sqlLogger:
            default:
              class: Neos\Flow\Log\Backend\ConsoleBackend
          i18nLogger:
            default:
              class: Neos\Flow\Log\Backend\ConsoleBackend

Then i was running a node:repair, which caused quite some logs such as:

[...]
INFO      A configuration file has been changed, refreshing compiled configuration cache
INFO      The configuration has changed, triggering an AOP proxy class rebuild.
INFO      File Monitor "Flow_ConfigurationFiles" detected 1 changed files and 0 changed directories.
INFO      Aspects have been modified, therefore rebuilding all target classes.
DEBUG     Built AOP proxy for class "Neos\ContentRepository\Domain\Model\ContentObjectProxy".
DEBUG     Built AOP proxy for class "Neos\ContentRepository\Domain\Model\Node".
DEBUG     Built AOP proxy for class "Neos\ContentRepository\Domain\Model\NodeData".
DEBUG     Built AOP proxy for class "Neos\ContentRepository\Domain\Model\NodeDimension".
DEBUG     Built AOP proxy for class "Neos\ContentRepository\Domain\Model\NodeType".
[...]

Afterwards i’ve removed a part of my Policy.yaml (within a Site-Package), to cause an exception. So i’v commented out 'Neos.Flow:Anonymous':.

In the frontend, i saw the following exception:

image

But by checking the logs via docker:

docker container logs 074a5ea87032 -f:

There was only this as an output:

[Wed Aug 23 06:36:50 2023] 172.18.0.1:34054 Closing
[Wed Aug 23 06:36:55 2023] 172.18.0.1:45962 Accepted
[Wed Aug 23 06:36:55 2023] 172.18.0.1:45962 Closing
[Wed Aug 23 06:36:55 2023] 172.18.0.1:45974 Accepted
[Wed Aug 23 06:36:55 2023] 172.18.0.1:45974 Closing

However, inside Data/Logs/Exceptions/ i’ve found the file 20230823063655e944c0.txt

With:

cat 20230823063655e944c0.txt
Exception #1391894094 in line 97 of /app/Packages/Framework/Neos.Flow/Classes/Configuration/Source/YamlSource.php: A parse error occurred while parsing file "/app/Packages/Sites/mb.Manufaktur/Configuration/Policy.yaml". Indentation problem at line 623 (near "  'Neos.Flow:Everybody':").
[...]

But this exception, neither the content from System_Development.log appeared to stdout :frowning:

7

@Benjamin the exception logging is managed differently, that is configured under Neos.Flow.Log.throwables and uses the Neos\Flow\Log\ThrowableStorage\FileStorage class.

This is not as easy to route to stdout or stderr as you will need to implement your own ThrowableStorageInterface, or you could try reconfiguring the storage path which would look something like:

Neos:
  Flow:
    log:
      throwables:
        optionsByImplementation:
          'Neos\Flow\Log\ThrowableStorage\FileStorage':
            storagePath: 'php://stderr'