I have a web application which has some quite expensive tasks which should be triggered by the user interface, e.g. “create bills” (in batch), “send notifications” etc.
I’m using the Flowpack Job Queue with Beanstalkd. This works well so far, yet I have one requirement that isn’t covered by the implementation: I need to trace whether a job was (successfully) executed or not. In technical terms, my requirement is as follows:
The user triggers a job from the frontend.
The backend triggers a deferred method (e.g. “createBills”) and returns the ID of the job to the frontend.
The frontend polls very five seconds to check if the job was executed.
As soon as this is the case, the frontend can show success/continue/offer a download etc…
The easiest way to achieve this would be to have a “Job” model in my application, creating a Job object when the job was triggered, return the UUID and poll for it. At the end of the deferred method, the job’s status is set to “complete”.
But I’m not sure if this could be a general need and possible addition for the Job Queue package. A new annotation TraceableDefer which basically does the same thing, but within a package:
/**
* @Job\TraceableDeferred(queueName="my-queue")
*/
public function doSomething()
If this annotation is set, a TraceableDeferredJobException containing the messageId is thrown when this method is called. At the same time the job is stored in the DB and marked as executed at the end of the method (using AOP)?
And maybe we have a little REST-like API to call if a job is still queued or was executed.
Hello!
Sorry, can’t provide a real solution, but I would try AOP in combination with signal/slot mechanism (just as keywords for further investigations).
thanks for the detailed description.
I can totally see your usecase but I’m not 100% whether (and how exactly) this should be added to the core package… If we add it, I’d prefer some callback or signal/slot over a “missused” exception.
But you can already achieve the same with todays tools by turning the batch process into an entity (for example InvoiceExport).
That entity persistes all it needs to know about its task (e.g. the invoice ids to export) and could have a status, i.e. initializing, ready and failed that you then update from the async job handler.
It sounds more complex than it actually is and you have the advantage of custom failure handling and tooling if you want.
Thanks for you reply. I went with the own entity TraceableJob containing a status, possible message (failure reason etc.) and a small API that can be polled anytime (api/job-status/1479298f-5de0-40a6-85ce-e4c2068c61cc) to check for the status of a job.
This works fine. Since it’s in an integrated package, I cannot share the whole code, but if anyone needs it, please request it here so I can greate a Gist containing all relevant files.
I did not share the JavaScript part because it’s all based on React and hard to extract the respective parts. But I think it’s clear: When triggering the action to be executed using beanstalkd, the TraceableJob object is passed along. Use its identifier to ping the job-status API on a regular interval. As soon as you have something else than pending as status, handle it. We use the $result property of the TraceableJob class to pass along URIs, e.g. when the deferred action produces an export file.
The code is based on Flow 3.3, so you might need to adjust all calls to the new class names.
Hi @lorenzulrich
Thank you very much! That looks just how i wanted to tackle our problem! (and contains some Flow stuff that i had no idea about yet^^)
(and contains some Flow stuff that i had no idea about yet^^)