There are more and more situations, when I create features for Neos, where I think: What we really need here is a job queue!
For example, right now I am implementing a mechanism which allows for analyzing assets when they are uploaded, to detect features like faces and landmarks, and then adapt the cropping area accordingly. It would be possible, but not very clever, to run the analysis as part of the same request which is responsible for uploading / creating the asset. Not only because the response will take too long for the user, but also when something goes wrong. An error in the analysis would also cause the upload to fail. Therefore I’d like to put this operation into a queue and process it asynchronously.
I know that we have discussed this topic here and there, but I’m not sure about the current state and if there’s a consensus:
I propose that we add the Job Queue common package to the Neos distribution by default. As a Neos core dev, you should be able to rely on that there is some kind of job queue available. Certainly, you need to be able to use different backends and so on, but it shouldn’t be possible to remove the Job Queue common package.
What we’d need as well is a default backend which works without further configuration. That could be a PDO backend and then some kind of worker. This worker should be run in CLI mode, so it shouldn’t be backed by some kind of web-request-triggers-worker mechanism.
Before we go into details: What do you think about that? Has it been discussed or even decided elsewhere already?