RFC: JS Refactoring Process of Backend

Thanks for the update. Sounds reasonable. However it does hint that it’s turning into a discussion about Ember vs. React instead of a framework or no framework as it was previously in this thread. Is that correct or an misinterpretation?

Like we discuss during the sprint at Rosenheim, I think we “need” to follow the ember way, because we have a lots of experience with the framework, and version 2.0 solve a least a big part of the current pain (virtual dom, get rid of the script tag, …).

I’m not against a react prototype, but the beast is always in the details, and a prototype does not contains those details. So the prototype can look and work nice, but it doesn’t lower the risk for building the complete application IMHO.

For getting ride of require, I agree that other solution can be nice, but Alhoa use require too, so take care of this point, or switch to ckeditor in the same step :wink:

Not Exactly, because Flux/React are just an example for an alternative architecture. Since React is more like a view layer and Flux is more like a pattern than an actual framework, even going for these two would mean to actually not have a framework. Plus there are other, even more lean solutions that follow the same principles.

I guess we all agree, that the heart and soul of any future foundation would be the unified data model - which is something that could be designed totally independent from anything that has been discussed so far. Maybe we should start a discussion soon on this too.

My impression is, that we currently disagree on having either a framework and reduce it’s feature set to the bare minimum needed for the backend or to create that bare minimum from scratch with more lightweight libraries.

Imho the latter approach leads to better maintainability, because smaller, dedicated, one-purpose libraries bring a natural interchangability that causes less trouble on updates, breaking changes etc. A whole lot of things (like Ember.RSVP and such) should imho even be handled via the growing number of native browser APIs and polyfilled when needed.

3 Likes

Flux can be achieved without React as well though. In general Flux might be a better approach for what we need, however just because it’s promoted by React doesn’t mean we have to use React to follow the same pattern. We already don’t use ember-data and don’t intend to, so using Ember with Flux should be perfectly achievable.

Thanks for the response Wilhelm, thinking more about it from a non-developer point of view I’m not sure the current plan is such a good idea.

I must admit it sounds like we’re trying to create the best architectural solution and not refactoring/migrating the existing solution. This is a very risky thing to do and one I’m not sure I can support, regardless of how great the technical foundation would become. Reason being that complete rewrites are toxic. They take very long time to do and change so much that it takes even longer to stabilize, regardless of test coverage. We must not forget just how much work went into the current implementation. Creating the whole thing over again will take a long time (probably 1-2 years) and I doubt we would be able to sustain the development for so long. From a product owner view, it’s a complete no go. The gain does not out weight the risk / workload (not even closely). Sorry to be on the fence on this one, but it’s a very important decision and thus we need to be professional / realistic. I see migrating step by step, by keeping existing code until refactoring, is the only real option we have. This has nothing to do with being pro Ember, I’d love to use React personally, but we chose it and now we have to stick to it. Additionally the arguments for not using Ember are pretty vague, if you ask me. Feel free to convince me otherwise though, but I’d like to see estimates on the amount of work needed and create a viable plan how it could work out in reality. The business part, not the architectural part, is the most important thing here.

And maybe at some point we have limited the dependency on Ember so much we can start considering replacing it with independent libraries.

Regarding the risk of updating, that’s probably not a big problem. Ember 2.0 is a pure sign of a framework maturing (it’s 1.x without deprecated functionality), so I’d wouldn’t expect them to change a whole lot of things in the future. More likely are smaller incremental improvements, but those should be possible to follow if we see benefit in doing so.

TL;DR: A refactoring is doable, not a complete rewrite.

I’m not completely sure about that, I’d love to give it a spin for the “new” Ember architecture prototype. It also matured pretty much towards the 2.0 release and has full support for JSON API baked in. I also have rather good experiences with it in an ongoing project. In effect you can have a Flux architecture with routes as the dispatcher, Ember Data stores as the - well - store and one way / read only data propagation to the components to always reflect the current state.

Ah okay, my bad. Assumed so since we decided against it in back then. Would be good to start collecting ideas for how that could work. So far Orbit.js and ember-data have been suggested, other suggestions?

This is more or less my POV too. But before being too quick in a decision where we don’t understand the alternatives fully, we thought about having a prototype to demonstrate the architectural idea of the alternative of staying with Ember.

One point where I can’t agree is that the Neos backend needs only a small feature set for the upcoming years. I would have an easier time to explain that if we had a nice shared vision on what Neos should become. But I really don’t see Neos only as an augmented website (and a site builder / editor). If we want to be less web-centric and lay the foundation for COPE or just more output agnostic publishing we need to consider not rendering a website in the first place. IMHO the Neos backend needs to consist of different modules (pages) that are configured with components (adjustable by an integrator / site developer) so it can be adapted to different requirements for content creation / editing / preview&publishing. And having a mixed architecture with app-style JS combined with server-side rendered modules and sprinkle-style JS (like we have now with the media browser) is really something that keeps us from providing the user experience we want/need to have.

2 Likes

Hey, missed the meeting too and would’ve loved to be kept in the invite loop a bit more.

Overall I’d like to shortly explain an idea I had. We’re starting to develop an application in which we need frontend editing and part of the components of Neos. We can not use the full interface of Neos though because it simply doesn’t fit our needs and leads to too many problems.

This basically means we have to write a UI anyway, which would imho be component based and in many parts similar to Neos. We would make the code opensource and could easily accept feedback, code, improvement ideas, and so on. What if we, as a preparation and way to gain experience, start working on that package (that would contain the most ideal UI structure for an application that talks to a REST API of the content repository) and open it up to the Neos team to squeeze everything out of it?
Depending how this all evolves we could decide if the package would be a solid foundation for Neos. Also I’d love to have more eyes on the package anyway so we can discuss and use it together.

Furthermore I’m (reckless as I am) in favor of a rewrite because I just don’t believe that we can patch the code into a clean cope-able thing. But I definitely see the risk of it, so project wise I tend to be with @aertmann here. Let’s move with caution and stay away from the risk to take too much work at our shoulders by completely rewriting Neos UI stuff inside the package now.

Maybe “bare mininum” was a bit misleading. What I actually meant was the exact set of features that are required by Neos, so without the overhead that is inevitably introduced by a full framework (which would lead to the custom ember build, @sebastian mentioned earlier). That set is of course not necessarily small. And I absolutely agree, that the backend itself should have a rich feature set.

I think we’re on the same page here (a stronger outlined vision wouldn’t hurt though :smile: ) and I’m also interested in helping Neos to get pushed into that direction (while I assume you don’t mean to drop the current editing experience entirely though, but merely to embed it into a greater COPE-oriented context, right?).

I do also agree with @aertmann on the business concern. Proceeding on a step-by-step basis, might be the way to go here. Nevertheless, since we all pretty much agree on dropping vie, create and aloha obviously, there is still quite some stuff inevitable to be rewritten imho. That does not mean, that we actually need to rewrite the whole thing, but it does give us opportunity to have a closer look on what can be improved along the way.

You’re right, we didn’t manage to clarify our idea yet. And I’m under the impression that we’re dealing with a lot of misunterstandings due to this. But @inkdpixels, @akoenig and I are currently putting our heads together to write up a detailed outline of what we have in mind alongside with all the concerns we have right now.

Actually I see a lot of common ground and that we’re not that far apart. I do believe that we can develop a mutual vision of this.

1 Like

Absolutely! I think the editing proved to be one of the strongest selling points for Neos so far. It just should be one of many ways to create / curate content. And for some projects it might not be suitable, or just for defined use-cases (e.g. by node type / editor role / task).

3 Likes

Here is something on the use if iFrames I just found (again): https://forum.typo3.org/index.php/t/203717/ Might contain some helpful info, it seemed promising back then.

Yes, that prototype actually looked very promising. But we never got any feedback from the original author again. Maybe he’s still around somewhere?

Hey everybody,

at the Code Sprint in Frankfurt we had a partly-remote and partly-local meeting with the following participants:

  • Aske Ertmann
  • Christian Müller
  • Rens Admiraal
  • Sebastian Kurfürst
  • Dmitri Pisarev
  • Dominique Feyer
  • (some more people where I cannot attach faces to names yet :-1: ) )

In a nutshell, we summed up the current state, and then directly deep-dived into the Ember vs non-Ember Discussion.

Basically I had a discussion with @dimaip on Ember vs React where we dissected the technical parts, where the others were listening in. The main points are:

  • We all agree that Data-Down-Actions-Up is a great thing we should follow.
  • We all agree that a componentized structure is a great idea.
  • We agree that Ember is a more “monolithic” framework and React/Flux/NPM modules is more a mixing-different-libraries approach.
    • I (Sebastian) favor the more monolithic Ember approach; Dmitri was in favor of the mixing-different-libraries approach.
    • Interestingly, what the one sees as benefit, the other sees as drawback in this regard. So we agree on the facts, just tend to weight them differently.
  • We agreed that we would be able to build Neos with both ways generally.

After the technical ping-pong between Dmitri and myself, more people chimed in and brought other (more non-technical topics) onto the table, the main one being:

  • We think it is not feasible to start-from-scratch, but rather we need a way to mangle our current architecture into the new direction without rewriting everything. In the view of (at least) Rens, Aske, Dominique and myself, this is the main reason we’d like to use Ember 2.0.

That’s why in order to move forward, we settled the discussion and decided to use Ember 2.0. @dimaip said he’s also OK with the decision, and he sees the points being made. Aske said something like “for personal project or if I’d start from scratch, I’d probably go the React way, but the existing codebase and the incremental migration possibility is the big argument in favor of Ember”.

The roadmap on top of the RFC is still generally correct; I’m just updating some details to reflect this decision.

All the best,
Sebastian

7 Likes

the two guys standing around where @alexfruehwirth and me :smiley:
We will gladly participate in this project, altough i have to commit that we don’t have a whole lot experience in Ember (2.0), but we’re very eager to learn.

If you throw us bits that we can chew i’m very positive that we can certainly help you with the refactoring.

6 Likes

Since i got some experience with Ember 2+ i could try to help.
First step would be to move the current work to Github (PR) and discuss changes over there?
I also got some experience in migrating an ember project from requirejs to ember-cli so i’m trying to work on that too.

7 Likes

Hey Philipp,

would be great if you could explain a little how this process works :smile:

Thanks so much,
Sebastian

Hey @gerdner,

We’re now working with ember-cli too, and I was thinking about how to apply that to the current Neos code. But if you already have experience there I’d love to have some kickstart :wink:

Maybe we should see if we can plan some hours that we work on the topic?

1 Like

I imported initial work from gerrit into github.
https://github.com/neos/neos-development-collection/pull/116
(i can not post more than 2 links :))

First step is to finish and test this work. At the moment, ember 2.beta ist commited. We should update to the last stable version 2.1 and start testing. I updated to Ember 1.12 in the past and only small problems in the build process appeared.

“We propose to have intermediate steps which can be merged directly in Master; trying to avoid a long-running fork.”

So after the update the could merge the result.

Next step is to migrate to the new folder structure from ember-cli.
I recommend to use the ember-cli pod structure since we want a component driven approach in the future.

  • rename files so the ember-cli-resolver can resolve them (pods?!)
  • copy files by type (views, controller, components etc.) to their pod folder
  • migrate to ES6 Module loader (busywork) (i got not much difficulties in the past with that part)
  • introduce package management via bower for browser dependencies
    • list all current dependencies with their current version and put them into a bower.json
  • migrate the build process to broccoli (perhaps very difficult - i don’t now the current gruntfile so i will have a look on that)
  • have a look on the requirejs parts one the php side (yaml-config etc.)

Migrating to ember-cli is a very big step. So perhaps we could takle that on a code-sprint or something like that?

My first thoughts so far :smile:

@radmiraal yep! :slight_smile:

1 Like

@gerdner: I agree that having something like a codesprint would be best, but I also think it isn’t a topic where a lot of people can work on so probably having a small group makes sense. What about a virtual sprint in which we work on the topic for for example 2 days and see how far we get? Or try to find a place somewhere half way where you live and my place and meetup?

2 Likes