I believe we could write the Neos backend with both Ember 2.x and React; both would certainly be doable – and I’d even expect the code to look somehow similar, as Ember 2 (same as e.g. Angular 2) has been heavily inspired by data-down-actions-up of React.
However, when we have migrated to Ember 2 and Ember CLI (which is some route to go still), I see most benefits because Ember 2 has strong conventions (the way Flow has strong conventions as well); and React e.g. does not tell you which file-structure you should use for writing your modules…
I feel that this discussion in the end it boils down to a “library” (=react) or “framework” (=ember) approach.
However, I’d suggest to move this discussion (e.g. which FE framework to use) out of this thread… And rather focus on our goals and not so-much on how we want to archive them.
I decided to read some discussions here this morning and I must say I’m pretty much delighted by reading this thread
My 2 cents:
I pretty much agree to the visions above
Flow with CQRS / Event sourcing / DDD would be awesome.
I would really love to change Flow from being a full framework to a layer on top of Symfony. Even if that means a hell of a job. If we manage to put our vision / the current strong points of ‘the way Flow works’ in such a layer it would open a lot of possibilities… (and the codebase would probably be smaller too )
I would like to discuss moving the UI from Neos to a separate package. I’m getting tired of 2 things: the lack of a solid API and the constant framework discussions for the ajvascript part. We use Neos now only as a content repository and find that the strong points of Neos are easy to sell that way. Selling the interface is a bit harder for us though…
Thanks for that writeup Robert. Since I’m not really involved with Neos, but only with Flow for the time being, I only respond to that part:
Considering what I hear from the community and am passionate about myself, I think we should strengthen Flow and position it as the Domain-Driven Design framework for PHP. We could take all the experience we made together and work on Flow 4.0 which provides excellent support for Domain-Driven Design, CQRS, Event Sourcing and so on. During the last couple of weeks I have experimented a lot with Flow + CQRS + ES and I really see an opportunity there.
I can not express enough how much I would love to go and push that direction. Flow (or rather extbase) was my introduction into the DDD world and I barely grasped what I was actually doing from the start, but it all just felt right. After some years now, I have a better understanding of what DDD is and what Flow already delivers in that regard (and also what not, like clean Value Object support or Domain Events). I really love the way of DDD and only last year entered the world of CQRS + ES, which just totally set my (dev-)world on fire, hence I would love to make Flow the (opinionated) DDD Framework in the PHP world.
Regarding the idea of others to just wrap a layer around Symphony I’m not sure about that, but most likely because I’m not deep enough into Symphony to really know if that would be feasible. If so, then I wouldn’t mind to have a solid base to work on, though I’m not sure how that would restrict us from changing that base to the needs. We would basically have a huge dependency on the Symphony Team to help us get our required changes into the core then.
Great, would love to do that! The question is if we can get that right with the little experience we have and how much of a rework that will mean which is not a lot of benefit for the average user right away.
I see a lot of risk in that. The CR is the major player in the backend for Neos and with the above mentioned lack of experience with CQRS / ES we are bound to make mistakes in our first implementations. Starting with something that big seems to me very risky with little gains for Neos users. That said I would love to have it. But I see it rather as a vision than a goal for short (or maybe even mid term).
I think we positioned ourselfs and the project as a future project, that means we need to move forward technologically so I see rewrites and changes to libraries as something we constantly need to do to stay at the front of the pack. We shouldn’t sacrifice a whole lot of stability but we need to move. Fear of change will drive us down a bad path and also code that is working fine should be touched from time to time to get it up to date and make it easier to handle when the time comes that it needs to be touched. IMHO we should strife for constant renewal with as less disruption for users as posible and I think that is possible. Something to make that easier is obviously more testing. If we increase our test coverage we can much more easily deal with changes to libraries and the code base as we can be sure that for users it’s still the same.
And we need to stay shippable and be able to progress while we refactor
Yes absolutely, we should never be blocked by a major refactoring on the way in master, but shippable master is still something I don’t agree with 100%, as mentioned in other places there must be a place for mistakes, hasty decisions and second thoughts. Obviously this is the code review, still we are humans and also 4 and 6 eyes might not see everything, so master it is and I think it’s fine that errors end up in master, as long as they are fixed right away.
Decouple & Modularize
Same for Flow, saying that for ages and yes definitely needed!
About the symfony base, I had a closer look at it and I don’t feel so strongly that we should and easily can do it TBH. I still think we should decouple flow and then start to replace parts of it with existing components, but the full stack symfony is a very different thing. Symfony components do not offer much we need right now…
So far my 2cc, I guess at large I agree with the direction.
As it’s not that clear so far I would like to add:
Gradually improving experience with your technology.
What I mean is that you should be able to run a Neos on classical LAMP with some features (about what we have now) but be able to ramp up your stack (with redis, ES, Postgresql, whatever we decide is good) and gain benefits from it (speed and features)
I think with the right decoupling and thoughts going into the architecture that is feasible and allows us to keep the existing target groups while at the same time opening up new user groups in the more “high end” sector.
(as posted in Slack already) I worked on an idea for a future architecture for Neos. I see that as the underlying foundation of the product strategy / vision because it might ultimately change the way Neos as a product is defined.
I’ll write about the details in a separate topic, but the main idea for me is to define boundaries for individual services and have a real and more radical separation between them. This should pave the way to select the best fitting technology, release cycle, team structure for each service. Following the idea of microservices, the technical architecture and organisational structure influence each other here.
What does that practically mean? Neos itself should be a suite of individual components / services that can be composed for content management solutions. It should be possible to replace these components by a different implementation depending on requirements and the project. It should also be possible to build a headless Content API server using Neos, or a Content API solution with a structured editing UI. Or having an editing interface and using Neos as a static site generator to publish for static hosting. Building a two stack CMS solution should be directly supported by the architecture.
So the scenarios where Neos can be used could be magnitudes higher and especially more complex projects / SaaS solutions benefit from the added flexibility.
It’s nice to see such a passionate discussion regarding the project. Especially when the vision seems to be very similar among everybody participating. Really brings the a shared feeling of joy for the project.
This is a topic I’ve spent a lot of time thinking about as well. I think Sebastian is spot on that it’s better to focus on where we want to go than how we get there.
Half a year ago I tried to summarize:
Where is Neos today?
CMS with limited feature set that gives a simple & intuitive editing experience that editors and developers love to use.
Allows for building websites with application functionality seamlessly together using a powerful application framework.
What will Neos be in the future?
A modern and fully customizable content application platform with strong conventions, good architecture and great user experience for editors & developers that can be used for any web solution purpose that creates and/or manages content for any type of output and strong integration with other applications.
Thus I really agree with the vision of a flexible content application platform. That requires modularization/decoupling/boundaries, clear interaction points (APIs) and composition flexibility. The possibility of seamless integration between websites and applications is a strong factor, so we should ensure it remains that way, and even improve it further. Contrary to Rens, the UI is a key selling point in my experience, so I’d like to keep a strong focus on that. However it depends on the projects, so it surely makes sense making it easier to remove that part if it’s not needed. Another key factor is the developer satisfaction, which we need to maintain and hopefully improve along the way.
+1 for getting rid of web centric focus
+1 for a optional composable UI
+1 for focussing on structured content
+1 for focussing on non hierarchical data in combination with hierarchical data
+1 for keeping a smooth as possible (ASAP ;)) migration path
+1 for focussing on handling of content (CR, dimensions, workspaces, CQRS)
+1 for focussing on professional developers
All this would as an example enable Neos being able to use Neos as an open source Contentful/GatherContent.
Personally I’m a bit torn between keeping the LAMP stack compatibility versus requiring modern server setups as I see both sides being valid. However I do agree with Christian regarding being an innovator and we should strive to keep it that way, since most of us are motivated by that aspect. So there’s a fine line between the two we have to find. So we keep innovating and avoid getting far behind since we didn’t adopt new approaches and still keep stability/backwards compatibility.
Also agree with Dominique regarding there being quite some way to be able to call Flow the DDD framework. The lack of encouraging bounded contexts, read/write models etc. which is what makes DDD really beneficial shows that. However going in that direction would be really great.
I somewhat feel the same. However I do think we can improve this part by changing the way we work. If we start developing bigger/riskier changes in separate branches with PRs or direct pulls to those branches and have a strict merge policy where we need more than usual amount of reviews before merging to master, we should be able to improve this without making the ongoing review process a challenge.
Hey all.
So cool to have this discussion here!
A lot of things have been mentioned already and I fully agree on the general direction. So I’ll restrain myself to the Flow-part for now (with a strong focus on the needs of a more decoupled Neos that you all seem to agree to).
I agree that we need adjusted tutorials and communication in general. I don’t think that Flow lacks many features for that though as DDD is mostly a matter of application design.
IMO there are currently three major parts where Flow underperforms in those regards:
It should be really easy to serve and consume (Hypermedia) APIs
It’s a pain to create proper REST APIs in Flow and that gotta change. It’s not a simple task due to the lack of accepted standards and the inherent dependencies of domain model, -logic and routing. But it could be done in a completely separate package and I already have some rough ideas.
You don’t need to go CQRS/Event Sourcing to be “Domain-Driven”, but it should be an option
…at least for parts of your application.
It’s a tough one because of the risks of over-engineering like it happened with things like SOAP or almost all Java frameworks
Everything should be composable
I share Alexanders and Christians concerns regarding the feasibility and benefit of rebuilding Flow on top of Symfony – at least I can’t imagine how that should work in practice.
But I think that we really need to start thinking more decoupled when creating new behavior. Most features can be created in completely separate packages. And our task should be to adjust the core where that’s not the case (e.g. provide additional hooks/APIs).
In addition we should strive for decoupling Flow subpackages into more general-purpose packages (and Christian has already started with that!)
One last thing regarding:
The great guys from yeebase (you might now them as t3n) generously support us kickstarting a microsite for flow. The plan is to setup a (neos) website as quickly and friction-free as possible. Please get in touch if you have any concerns regarding that approach and/or follow the #guild-website channel on slack for updates.
I do too in some regard. Though I feel like I need to make bold statements regarding using Symfony as a foundation as in my opinion this was opted about 2 years ago. By then people felt it might be a good choice and worth some research. @christianm did some research and found that components would be easily usable but there are not that many feasible components. The Symfony team actually said they’d be willing to discuss things we would need for making this possible and see if they could support that in the new version. To make a long story short we all shift it to the low priority / too much work list and the end result is (and will probably be): we keep maintaining our own code in our own little community world.
While I didn’t really do an in depth check how such change could be done I do really fear we are not putting enough effort into it and as such throw it in. If (and keep in mind I’m not 100% sure if it would be technically possible at all) we could manage to put the stuff that creates Flow (AOP, conventions, Fluid, and so on) into Symfony components we would be forced to decouple. But hey, this is all highly hypothetical
Just to prevent any misunderstandings: I’m all for decoupling and to adapt other libraries where possible!
I never understood this as rebuilding Flow “as a slim layer on top of Symfony”, but maybe the end result is the same: make it easier to use 3rd party libraries in Flow and at least as importantly allow our components to be reused outside of Flow.
I just doubt that it’s feasible to do this in one large project but IMO we should start by
adapting common (PSR) interfaces and moving subpackages into (optional) external packages, leaving only a slim adapter in Flow
go the extra mile to create external packages when adding new major features/behavior and put Flow magic like AOP etc. on top of it within Flow or a separate adapter package
Do you agree to that, @radmiraal or do you have other ideas on how to approach this?
I’m just quietly reading most of the posts here but one thing that came to my mind when reading about Flow as a “Symfony layer”: can this be done in a way that existing packages/projects can be updated?
I think there are a bunch of bigger Flow projects out there and it could cause some serious problems for these companies if there’s a point (event if it’s a few years away) where we need to tell them to rebuild the whole thing.
I agree with you that upgrading strategies and ways in which backwards compatibility must be kept is extremely important – I think that basically all of us agree here
To be honest I’m shouting out ideas without concrete solutions in mind I’m still in the problem phase so to say. I feel like we in general do agree that we should open up / make our code shareable and usable to others. But somehow it never leads to really concrete steps or so.
Adapting common interfaces would indeed have a +100, and far better separation of features and behavior in packages would be lovely.
Maybe this product vision thread would be an interesting place to collect what such a ‘layer on top of symfony’ would mean? As that could help to extend what was written above already
Regarding the “Flow on Symfony” part: I agree with @bwaidelich here - I don’t think that it’s feasible or would make sense for our existing community to start from scratch and implement the Flow feature set on top of Symonfy. Also I don’t think that we should - right now - look further into making Flow a Symfony application (that would mean, compatibility on bundle level).
That being said, we should definitely look into modularizing Flow more and also looking into replacing certain parts of Flow with third-party components. Typical examples would be logging, the HTTP stack and so on. We also should look into taking advantage of Symfony’s tooling, like the profiler.
What’s important to me in that regard is:
we should maintain and further improve the typical Flow developer experience, which means convenience features like DI (property injection …), automatic registration of certain components (command controllers and their documentation for example)
we should make it more natural for experienced Symfony developers to find their way around in Flow
we should replace existing code with third party components which is not part of the special Flow developer experience and is probably better maintained elsewhere (HTTP, our virtual browser, loggging, …)
we should strive for making Flow more lightweight (e.g. an option to use Flow without AOP, look for a solution without proxy class generation, make it possible to realistically create PHAR applications with Flow)
we must make sure to provide a realistic upgrade part for existing Flow projects
I think we should create technical story-tickets (possibly with sub-tasks) for those, explaining the desired long-term outcome in order to prevent situations like:
Well, if you modularize you actually create the possibility to use one of the existing 14 standards… You just need to resist the urge to create the 15th at that point.
Yes, you’re right of course, but facing reality it’s often not as easy I’m afraid.
Take the HTTP subpackage for example: If we’d create an adapter to PSR-7, we’d have to think of a solid migration path including proper documentation or else people will end up using n+1 ways of dealing with it.
This might be very obvious but we’ve seen it failing multiple times in the past and new ways of solving things actually increased confusion because they a) increased complexity, b) weren’t properly documented, c) the previous way wasn’t properly deprecated (not calling names here ;))
we should strive for making Flow more lightweight (e.g. an option to use Flow without AOP, look for a solution without proxy class generation, make it possible to realistically create PHAR applications with Flow)
I don’t think that’s necessary, nor really feasible. A lot of the power of flow lies within the proxying/aop and building a flow distribution without that would feel like an akward attempt to achieve mainstream usability at the probable cost of functionality/performance. Sure, the proxying/aop is not perfect and has it’s drawbacks (everyone at least once needed to double flush caches in order to get things to work as expected again), but I regard it as an integral part of Flow (not the cache flushing ;)).
)
