Funding request: [Flow Light] Resource management: Create a resource storage that doesn’t need database

This is one of 3 tasks in a combination to create a Flow Light

Since there is a direct dependency on the Doctrine persistence layer inside the current Resource Management this funding request will only make sense if the Split Doctrine Dependency from Neos.Flow is accepted`

What is the purpose of this project?

To add a new resource handling mechanism, that can work without the database dependency.

This will move all work around resources away from “PersistentResource” as a entity, but still keep that as the object being passed around.

This should become a basic possibility of handling resources, stored in a folder. The today-known resource management including database etc. is a addon, that will conflict with this package and hence these two can not be installed at the same time

What does it take to get started?

We have the knowledge and concept at hand.

What will you be doing

Split the persistence resource management out of the Neos.Flow package and created additional packages for the

  • basic resource management
  • persistent resource management (known today)

At first they will not be able to be installed both at the same time, due to conflicting ways of “persisting data”.

When is it considered done?

It’s considered done when:

  • All tests passes
  • We can run known Flow applications with the new storage concept

How long does it take?

Since there is a direct dependency on the Doctrine persistence layer inside the current Resource Management this funding request will only make sense if the Split Doctrine Dependency from Neos.Flow is accepted`

I’m very cautious with this estimate. But somewhere between 40-50 hours. For the sake of this funding request I will go for the largest (worst case)

How mouch money are you asking for?

Not knowing the hourly rate this is calculated with a hourly rate of €100

50 hours * €100 = €5000

I do think this is another good step in the direction of decoupling Flow from the doctrine/persistence in itself. Though quite a bit more work than the routing/objectpathmapping move, I think this also can be done independently!

Things to consider:

  • what is the persistence aspect of resource management used for?
    • iterating resources by collection
    • find resources by similarity (sha1)
    • find existing resource by content match (sha1) [similar to above]
  • what are the non-functional requirements for that?
    • it needs to be able to handle thousands and even hundreds of thousands of resources performantly

I think it’s neither easy, nor useful to not depend on this functionality in ResourceManagement. So a “Basic Resource Management” should IMO still provide those functionalities, but just not depend on storing the required information in a database. Maybe, just maybe, this can also be handled with the Caching layer - though I’m not sure it’s a good idea to use the Caching layer as a pseudo-database in all places. Maybe we can just store the meta information in a structured way side by side at the resource storage location. Those are things that need to be investigated.

Then, what I would imagine is to basically provide alternative implementations, or underlying abstractions (that need to be introduced) for the PersistentResource and the ResourceRepository. Every place using ResourceManagement should still use those two classes as the API for handling Resources (ResourceRepository only internally of course), so those places don’t have to change. Then we’d have a Neos.Flow.ResourceManagement.DoctrineStorage and a Neos.Flow.ResourceManagement.CacheStorage or Neos.Flow.ResourceManagement.FileStorage package, that provide those implementations, wich the user can configure to be used as resource metadata storage.

1 Like

I agree to everything @aberl said. And especially:

I would also suggest to turn this around:
Have some ResourcePersistenceStrategy and provide adapters for the Caching Framework & Doctrine

1 Like

I like the idea.