Opinions on an architecture

Using Neos & Flow
1

I am planning a new project and am interested in your opinion:

It is a big archive database with CRUD forms of 300k+ records, some of them are structured in a tree and some additional features like a good search for example.
Current plan is to use Flow as backend service and a vue.js application based on vuetify as frontend.
Today i set up a prototype consisting of a plain Flow installation (without Neos) with the Neos.ContentRepository and Wwwision.GraphQL a slightly modified (removed the requirements to neos/neos) Wwwision.Neos.GraphQL and Flowpack.Elasticsearch.ContentRepositoryAdaptor. This gives me a working graphQL endpoint for the Neos.ContentRepository with elasticsearch index.

I am now thinking about what would be best:

  • Use this setup (maybe in the future replace the CR with the eventsourced one)
    • Con: not sure how the CR handles 300k + nodes
    • Con: relations are not as “clean” as doctrine models, elasticsearch is a must to get relations on the reverse side, not sure how long it takes to rebuild an index og 300k nodes
    • Pro: get history, undo and stuff easily integrated when switching to the eventsourced CR in the future
    • Pro: tree structure out of the box
  • Use plain doctrine models instead of the CR
    • Pro: Clean classic DB structure
    • Pro: No big unknowns
    • Pro: Also Elasticsearch integration available
    • Con: Boring (as in not as exciting as the other options)
    • Con: Tree-Structure, History and other stuff needs to be build from scratch
  • Use the eventsourcing package
    • Con: No elasticsearch integration out of the box
    • Con: Not sure about the “stability” and performance with 300k records
    • Pro: Big potential (good base for history, undo and that kind of stuff)
2

My favorite is solution #1:

  • as the current CR is very straight implemented 300k+ records (node entries) shouldn’t show any problems. The migration path to ESCR is clear.
  • #2 is really boring. It obsoletes flow.
  • #3 für such a huge project? Too risky for me for now

One question for my understanding: why GraphQL as midware? Is there a demand for complex queries to the CR?

3

What do you need to build from scratch? I have a number of similar usecases from a current project, that I can easily open source and share findings about :partying_face: Doctrine Extensions have a lot of it done - and if it doesn’t fit you exact case, it can be “taken out” and yoiu can adapt it.

I’m moving fast these days with Route (websupply/route-annotation - Packagist) and Policy annotation (topic in Slack - not a composer package yet) a event “raised events” in my model (dispatched using GitHub - wbsply/event-dispatcher: PSR-14 Event Dispatcher for Neos Flow - Symfony EventDispatcher with autodiscover) , and on the read side (For larger outputs) I go directly to the injection of the Connection and query directly with SQL.

Boring? No! Fast pace? Yes! Supporting the community and project? Big time! <3

Ping me about the usecaess, let’s see what we have around :slight_smile:

1 Like
4

Thanks for your replies!

@tdausner There are definitely some more complex queries. I have not much experience with graphql but the concepts seem to make it easy to request sumsets of data (which will come in handy to improve performance - we dont need to fetch all references in everty case). Also the mutations provide a lot of power and structure.

@sorenmalling I guess you are right. Doctrine provides a lot of infrastructure too. Nice approach by the way with the route-annotations.

The prototype i have build (with the old CR) seems promising. Also this will be a great use case to test out the isolated usage of the new CR later on. So i think i will stick to this approach for now and concentrate a bot more on the frontend while i get a better feeling about it all.
I used a modified Ttree.ContentRepositoryImporter (hat to remove Neos.Neos dependencies in the code) to import 200k nodes which is no problem at all. Will keep you posted about further development…

1 Like