Hydration of Lazy annotated collection causes memory_limit error

Using Neos & Flow Flow App Development
1

Hi :slight_smile:

Foreword

This weekend I have been putting the last touches in a project. In short terms, the project is about

  • A endpoint collect a bunch of data
  • It is saved directly to a database - no Flow involved here
  • In my Flow app, i annotate a model class with @ORM\Table(name='data') and @Flow\Entity(readOnly=true)
  • This model has a OneToMany relation to another table, that holds follow-up data via another (non-flow) rest service

I then give the registered user, a possibility to query these data set with parameters like

  • Date start
  • Date end

To create a range

and environments which is a “context”, or a “group”, these data are saved in.

The issue

Some of the these data points can have thousands of relation in the @OneToMany relation, to follow up data.

I try to use PaginateViewHelper and pass on my QueryResultInterface object, from my custom created qurey in my repository.

I of course annotate my @OneToMany with a @Flow\Lazy but at the time I start to render the datapoints in a list, the persistence layer, start hydrating all the objects, including the relations. Remember, there can be thousands in each of them. This causes a memory_limit error, and in our case, it gives the :fire: White Screen Of Death :fire:

I’m looking for suggestions on how to handle this properly, some ideas I’ve came up with my self would be

  • Remove @OneToMany annotation and instead make the queries directly with it’s own Repository - eventually injected into the model
  • Go away from models and create custom queries, returning the column values, and then create a bunch of viewhelpers to make the template rendering smooth and easy.

All ideas and questions are welcome :slight_smile:

2

Hi @sorenmalling,

could you add a little bit of context describing your actual usecase?
a “[…] model has a OneToMany relation to another table, that holds follow-up data via another (non-flow) rest service” is very hard to reason about :wink:

(if you don’t want to share the projects domain you can maybe come up with a similar example?)

3

Have you tried using EXTRA_LAZY for the relation? http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/tutorials/extra-lazy-associations.html

If you are using slice() to only show part of the collection per page, this should work relatively good (at least in theory :sweat_smile:)