What do you think about flow as daemon?

Using Neos & Flow Flow App Development
1

Hi forumers!

My actual job is to implement a daemon.
The basics are done and that works so far but it is implemented classically as “proof of concept” only.
Now I have to combine this with our flow project: the daemon have to get and put
informations to it.

My consideration is, to implement the daemon in flow too.

What do think about that?
Does anyone have experiences with this subject?

Every comment and/or hint is welcome!!

2

Hi Frank,
in order to give valuable feedback it would be helpful to know what exactly you want to achieve by using Flow as a deamon

1 Like
3

Hello Bastian!

in short words its a kind of push server which shall distribute infomations in our complex flow application.

4

Not sure what kind of “push” you plan to use/build :smile:

But we’ve built a “daemon-like” thingie with Flow that runs a regular command-controller-action via supervisor-daemon -> this means the command controller-action is being restarted again and again, every time it exits.

Upon start of the command, it loops in itself:

  • In each loop, it checks a message queue whether something has to be done or not
  • If a task is in the queue, it picks it up and processes it, then exits the command controller (and get’s restarted by supervisor daemon)
  • If there’s nothing to do, it just waits and loops again (and exits after let’s say 1’000 loops to get restarted by supervisor).

This can even work with multiple “runners” in parallel: Supervisor can start 1…n instances of the command controller in parallel, which then process a queue pretty fast.

In our case it’s part of a “config-management” application where new instances can be requested via a Web-GUI (leading to multiple single tasks per order that are placed in the job-queue), where two workers take care of executing the requested tasks from the task-queue as fast as possible (instead of having to wait for e.g. a cronjob that works every 1min or less often).

Hope this helps :smile:

5

Hello Mario,

thank you for your input!

The “task queue” is something I’ve considered too.
I doubt about the permanent polling of the database but I havn’t another idea actually.

To be more precise of the meaning of “push”: its a webSocket implementation
(javascript only on client side).

6

Hi Frank

Yes, we had the same thoughts. To reduce the impact of this polling, we’re having the queue in Beanstalk instead of the MySQL database. With this, we didn’t see any issues when the two workers are polling e.g. once a second or so.

Regarding your project and websockets, I’m out of ideas (no experience with that). But hopefully someone else can/will shed some light on this, I’m curious :slight_smile:

Cheers,
Mario

7

Hi @mexoona

as @mrimann mentioned you should switch to Beanstalkd instead of MySQL (for performance reasons - no manual polling needed with Beanstalkd). Flow can handle asynchronous processing and running commands as workers (daemons) very well.

Just have a look at theses packages that allow you to send tasks to the message queue by adding a simple annotation:

And the supervisor package is great to manage your workers/daemons (statistics, stop, start, autorestart if something goes wrong…):

We are using this in production for processing resource intensive asynchronous tasks in the background while keeping the web application as fast as possible (e.g. rendering PDFs, sending emails,…)

Best,
Martin

1 Like