Does somebody know, if there is an detailed description of the flow validation process?
I want to know, what goes on exactly from users http post over property mapping and vlaidation to finially persistence to DB.
Background of my questetion is: we have extremly performance problems during “saving” a dataset.
The underlining model ist quite complex with several dependencies like m:n relations.
At this moment there are only about 2000 datasets in DB: it takes about 1 min form user pressing "save"
to see a result.
I’ve monitored sql request and it says there are about 5000 selects around the update statement.
Now I want to know, where exactly preformance loss is located…
Guess its vailidation… Due to this, I want to understand this process.
So I’m going to go a bit more into the technical and design details that you should consider to deal with the performance problems you are suffering.
First of all, you stated that you have a pretty deep model hierarchy with lots of m:n relations.
This is a common scenario that starts up performance problems and can be dealt with a bit more undestanding of DDD, Doctrine specifics and thought into how to build your domain model.
In case you haven’t read yet, take a short look at the definition of an “Aggregate” and “Aggregate Root”: http://flowframework.readthedocs.io/en/stable/TheDefinitiveGuide/PartI/ConceptsOfModernProgramming.html#aggregates
The core idea around Aggregates is, that they should encapsulate the behaviour of a business concept as a single unit. This unit may contain multiple objects with relations, but from a functional and consistency POV it should be considered a whole.
Now what easily happens, unwantedly or unconciously, is that you design a very huge Aggregate root (deep hierarchy with broad relations). This is a general problem, because that means that whenever a single unit within that aggregate changes, the whole aggregate needs to be validated/persisted. An Aggregate is a (transactional) “consistency boundary”.
What you should try to achieve is to create small Aggregates. (A good read to Aggregate design is Vaugn Vernons Series http://dddcommunity.org/library/vernon_2011/)
Within Flow that means:
think of where entities in your model are an entry point for such a boundary, then make that entity an Aggregate Root by creating a Repository for it. Flow by default cascades all persistence operations, unless the related entity is an Aggregate Root.
try to minimize relations between Aggregate Roots, as normally that is a sign for either just using an access shortcut with unwanted side-effects, or your Aggregate Roots not being thought out correctly yet (this is a learning process inside your domain, so it’s ok if it isn’t perfect from start).
also try to minimize bidirectional relations, an aggregate root only delegates behaviour downward and sub-entities should mostly be unaware of the whole aggregate they belong to and instead focus on their part of the job. Especially avoid bidirectional One-To-One associations (https://tideways.io/profiler/blog/5-doctrine-orm-performance-traps-you-should-avoid#4) in order to avoid countering the next suggestion:
With all this in place, you can take a look at the validation performance with a better starting point. Flows Validation is triggered on every action call that passes an entity that is annotated, as well as again during persistence. Yes, Flow validates your objects twice, because your objects may change inside your controller and get into an inconsistent state. There are ways to work around this, but this shouldn’t be your first concern (read about Validation Groups if you’re still interested).
However there is still one issue with Flows Validation, that it eagerly traverses (and loads!) the whole object tree every time, so if you can not seperate your aggregates, this becomes an issue. There is currently a PR in place that provides a small counter-measure that works well with small aggregates, that could be of benefit: https://github.com/neos/flow-development-collection/pull/334
It prevents validating related aggregates that have not been fetched from persistence (ie. lazy loaded and untouched).
If you need further advice or have questions on what I wrote, just ask.
