Project proposal: Rewriting the UI using React

:+1:

I vote to continue moving in the technical direction laid by this prototype (not just React vs. Ember thing) to fulfil the shared Neos product vision.

1 Like

Voted in favour of pursuing the React idea, assuming it will be done in parallel until we can “switch”. And please, whoever works on it: do it well and fast. :smile:

Also, keep in mind we must maintain the current UI as long as we have supported versions using it out there. Alternative way out: Make the new UI a fully drop-in replacement from the user’s perspective and ship it with “old” releases. Good luck…

5 Likes

:+1: from me. But please let’s plan and proceed with this project cautiously, as Sebastian pointed out. Don’t just rewrite the UI with a new technology.

2 Likes

:+1: while keeping in mind that:

  • IMHO the curent Ember UI should be separated from Neos first (to allow parallel development)
  • IMHO the rewrite requires a clean server side API too (maybe also in a separate package to allow flavors)
4 Likes

:thumbsup: assuming:

  • Ember Ui should be separated
  • done parallel
  • in general as Sebastian last post pointed out
1 Like

:+1: in general. But IMO we should thoroughly evaluate a decoupled architecture (like initiated by @christopher) before we decide about too many details

1 Like

:+1: assuming like most others:

  • Ember UI separation (anyway a good idea)
  • Parallel development
  • Having a good business value we can communicate to the outside
1 Like

I can’t really vote based on a technical point of view but based on what I heard and read it seems to be quite some work to update Ember as well and is quite realistic to switch to React. And as some people are really interested in doing it with React and we could use some kind of hype around that I think it makes sense to switch to React. (Assuming that we will go the parallel path).

I abstain - don’t have enough technical insight :smile:

1 Like

:+1: from my side. My main point would be to make sure the server side API will be redone as well and we make sure the UI will be really separate by a clean (REST-)API. Will say, it has to be possible to replace the UI package by something else, if needed.

2 Likes

Everybody here,

as just mentioned in the retrospective, I wondered if you guys followed the post “selecting a client-side framework for Drupal” by Dries. It mentions licensing conflicts with GPL in case of Angular and a rather stupid patent clause by Facebook regarding React.

See also this issue in the Automattic issue tracker and this discussion in the React issues tracker.

Please do read that article if you haven’t and let’s discuss here if it makes a difference for us.

1 Like

My 2cts …

  1. I don’t like the patent clause, it’s not really in the Open Source spirit
  2. I don’t think that it makes any difference for us, also because the Apache license has a similar clause which wouldn’t affect any of us or using the software in practice. Let’s be grit our teeth and ignore it. Not nice, but hey.

Still I think it’s worth mentioning that Dries / Drupal seem to rather tend choosing Ember than React :wink:

2 Likes

Thanks for bringing this topic up @robert.

As you also stated, I think unless we also want to submit a patent for the Neos UI, this whole discussion does not affect us in any way.

Still I agree with you that this is not in the spirit of OSS in general, but I acknowledge the fact that the react core dev’s are also in the process with facebook-legal to remove this clause from the project, I think there is a fair chance that the clause will be removed. :+1:

2 Likes

I closed the voting above, I see eleven positive votes in the comments below. I guess that the result we expected and so we go ahead.

3 Likes

Hey guys,

Thanks for the responses. Since there was a clear preference towards starting from scratch, I focused on my time off.

Anyway I wanted to summarize a few consequences of this decision and share some thoughts, insights and advice.

  • During my involvement with this project I have really come to respect the extent of the solution we’re trying to build. This is where most of my reservation from doing a rewrite comes from. It has taken thousands of hours to achieve the current state, and there’s still so much to improve. Doing a rewrite means most of that work has to be redone with pretty limited resources.
  • There seem to be a very optimistic view of how much effort it will take to achieve the desired goal and estimates being made that have almost no backing. That’s very concerning and a sign of a disastrous project from my point of view. The fact is that we’ve more or less built this product already and we have a fairly good idea how long it takes to do. I’d be happy to help answering how long different aspects have taken. The first thing one should do is to get a clear idea of what one is trying to built, which no one seems to have done so far. Without it you will only have a vague idea how far you are and even if you’re building the right thing. I already started doing it during the Ember upgrade, and I suggest to finish it to have a clear scope of the project. See Neos UI overview - Google Docs
  • As I mentioned earlier I see two outcomes of this project, project extends beyond deadline or a less featurerich or potentially less stable solution replaces the existing one. Therefore I suggest to be realistic and accept to aim for the less featurerich outcome. Focus on the important aspects first and stabilize those. This is something we’ve been pretty poor at in the past, partly due to development being motivation based. We should have limited scope and focussed much more on delivering solid solutions on fewer things. My first suggestion would be to skip components like the edit/preview panel, inspector validation, data views, keyboard hotkeys, tab key navigation, code editor and similar for the first version.
  • During the development of the current UI, development stagnated. Around the time of releasing 1.0 we reached a point where it was mainly stability, handling edge cases and small improvements that was important. This wasn’t very motivating work to do and only few relied on the product. This was a pretty frustrating time lasting a whole year and took a lot of perseverance to get through. This maturing phase is likely to happen for the rewrite as well. Guess is how to prevent it from happening?
  • The success of this project is largely tied how much time people end up investing in it. It’s great to see Tyl and Williams drive, but the desired deadline is certainly more than they can achieve themselves. Looking at the past activity I must say I have my doubts if there will be the necessary support. I hope that it will motivate some who wouldn’t otherwise be, since there’s a popular stack and the chance of architecting things.
  • Most of the existing code base can’t be used for anything. Of course a lot of experience has been gained, which can be useful in a rewrite, but that’s very little benefit achieved from it. I hope that the existing code is taken into account seriously when rewriting to ensure all functionality is duplicated/tackled. Of course most of it will be rewritten in a new way causing different problems, but the functionality of the components remain very similar.
  • A lot of effort was saved previously by using jQuery plugins. A lot of complex functionality now potentially needs to be implemented ourselves unless React compatible components exists. This includes trees, drag & drop, sliders, date picker, cropping & upload. There’s a big chance of falling into the trap of not invented here syndrome. Hopefully there’s suitable components available.
  • Regardless of introducing tests, a new solution will have to go through it’s maturing phase. Tests help reduce regressions and approaching testing more methodically. However many of the stability issues Neos have had in the past, came mostly from not covering all aspects/use cases and the complexity of interacting components. Not from not being tested properly/not having tests or regressions.
  • The maturing phase will take time and effort, and requires a user base willing to accept it. I personally doubt many existing users will be interested in that (at least my customers won’t), since we’ve had a poor history in that regard. I do hope there will be new users or some that value the improvements the new solution has to offer will be available.
  • Attention to detail has been a sparse resource in the past. And something that’s required to deliver a user experience similar to the existing one. Therefore it’s important to ensure it’s a focus area covered by the people involved. Improving code quality doesn’t include this in any way.
  • It puts existing effort (WIPs) into limbo and it’s likely that most development on the current UI will cease. Meaning introducing new features are less likely, since implementing them in the rewrite is likely a big risk and difficult due to it’s progress phase. Not sure how we should tackle this.
  • It is certainly one the biggest and riskiest decision being made while I’ve been involved with the project and the first one I strongly disagree to. And I firmly believe it would be better achieved by refactoring our existing code base with much less effort and risk and a similar outcome.

While that has been said, I hope everyone involved takes ensuring the success of this project as a challenge and makes it happen. There’s is certainly a lot of good outcomes that can come from it, but it requires a lot of effort working together to achieve.

Technical alternations/suggestions:

  • When building the node state handling, take having the same node in different context into account. This is a current weakpoint and makes split screen difficult. Related to this is a bug where the same node exists twice on the same page through a reference, causing one of the two not to work.
  • There are some of the UX concepts that should be improved when rewriting instead of rebuild in the same way.
  • Streamline headers and collapsing of panels (documents, content, inspector) and collapsing of them – https://jira.neos.io/secure/attachment/12718/Neos-UI.png
  • Replace tabs in the inspector with dropdown – https://jira.neos.io/browse/NEOS-1790
  • Combine “Menu” and “Navigate” to tabs – https://jira.neos.io/browse/NEOS-1174
  • Implement a multi-functional tree – https://jira.neos.io/browse/NEOS-1789
  • Building the tree, take different sorting/filtering into account not just based on node hierarchy as well as lots of nodes on the same level
  • Property validation is currently based in the model, however only visible in the inspector editors. It should however also be applicable to inline editable properties.
  • URL handling should be decoupled from the website view. Instead of using the node URL, the identifier should be used to avoid making a website focused application. E.g. replace http://neos.dev/en/download@user-admin;language=en_US.html with neos ⚛ Universal Compiler and Bytecode JIT.
  • Give some thought how other modules should be bootstrapped and work in relation to the content module. There’s a need to reuse components across the two.
  • When implementing the select editor, consider making it the basis for the reference(s) and file/images editor. They all represent a list and will benefit from search, add/remove, reordering, search/select.
  • There are some areas with some pretty tricky CSS solutions which took a long time to achieve. These will be completely hidden and requires a lot of manual testing to get right. Therefore I recommend looking at the existing CSS to understand why it’s build like it is, regardless of the undesirable class structure. Some of these areas are the tree (overflow/scrolling/ellipsis), inspector fields (overflow/scrolling/ellipsis, modified/error borders, error messages), select2/chosen (overflow), edit/preview modes panel.

Lastly a have some questions / concerns about the new prototype:

  • The CSS approach might not be the best approach. Since we currently have both a JS and PHP rendering, re-using styles for components on PHP generated pages wouldn’t be possible. This comes back to the point I made regarding the how these two play together.

@sebastian: Completely agree with everything in your post, well put.

Cheers

2 Likes

Forgot one point. We’ve wanted to introduce theming for quite some time to better support poor screen, projector and bright light conditions. This is much easier to done when starting out than do later on. So I’d suggest to spent a little bit of theme thinking how to do that before it’s too late. Initially a light theme would be the goal, see https://dribbble.com/shots/2009015-Neos-white-theme

1 Like

Hey Aske, thanks a lot for your detailed feedback, it’s really valuable for the rewrite project at this point.
We will have a discussion tonight to review which points of UI we can change during the rewrite and which we can rewrite as is, and we’ll base it on the list that you brought up.
I think the focus for now would getting minimal working UI that could be installed alongside the current UI.
Will post the updates here soonish.

Hey everybody,

I just summed up the TODOs from the meeting @dimaip has been referring to:

@aertmann: I definitely understand your concerns; and as written previously, to me the main concern is that we cannot archieve the quality / bug-free-ness we need right from the start. I am willing to bet on this, as React/Redux state handling is a lot better understandable, simple and predictable (to what I see so far); hopefully reducing race-conditions etc.

Furthermore, I’d just like to point a very huge THANK YOU to you Aske for coming up with such a detailed posting even if you generally dislike the decision and feel it is very risky. Looking forward to hopefully see you soon again :slight_smile:

All the best,
Sebastian

hey Aske,

even though I can’t add much to discussion from a technical standpoint, I’d like to at least let you know that I really appreciate the way you handle this difficult decision. I’m glad that we can discuss this difficult topic in a constructive way, and even though I do agree on some on your points (especially the underestimation part), I also decided to fully stand behind this decision. Let’s give it a shot - sensing the spirit here at the sprint I think that there’s a good chance that this will really work out.

Cheers!
r

2 Likes

Hi Aske,

thanks for your detailed feedback. First off, let me say, that I really feel bad for seemingly having decreased your trust in the project, since this certainly was not my intention. Therefore I am just as thankful for you taking the time to bring in your perspective, experience and advice. And though I disagree with some of your points, I cannot emphasize enough the importance of your view on this entire topic and how much I value your advice. Now, I hope that I can say a few words, that hopefully help to give you a better feeling on what we are doing:

Estimation is without any doubt a problem, that we can’t solve for this project. The only thing we can do, is to time-box the effort and to frame hard criteria, after which to stop the project entirely. This is what we did at the sprint. The project is constrained now regarding the time it should take and if it bends its limits, the consequence will be its failure. I am strongly convinced that it will succeed wihtin these boundaries, since they are pretty reasonable, but of course I can’t foresee the future.

Feature-wise there is consensus not to re-implement everything, especially those features that are already planned to be improved. Dominique took the time to introduce Tyll and me to the UX improvements as suggested by Christoph Ono (https://jira.neos.io/browse/NEOS-1173), that are now basis of all reconsiderations on the way. We had a fruitful discussion at the sprint that resulted in four major improvements that should definitely be part of the rewrite (documented here: https://github.com/PackageFactory/PackageFactory.Guevara/wiki/Project-Planning). I agree to concentrate on the core experience of the UI first, though. We should aim to break the usual workflow as little as possible, except for explicitly improved features. Stability should be our primary goal.

Regardless of the resulting featureset, the project will - as you’ve said above - certainly go through a maturing phase. And while I do not believe that this is preventable at all, I had some thoughts on how this maturing phase will actually look like. Since we won’t just switch off the current UI but will allow a parallel installation at least during an alpha and beta phase, we can profit from the grown project and user base that Neos has by now. The UI has a chance to hit tons of edge cases before its final release.

Regarding the participation, the resonance during the code sprint was very positive and I hope the involvement of all those new contributors will sustain. Still I remain realistic about the progression of such projects. Participation tends to get more during the sprints and less between them, but I am certainly willing to keep working on this and I know, so many more are too. I also really hope that someday you will join the project as well, since I am certainly not the only one to consider not only your opinion and experience valuable, but who would also love to work together with you on this.

Finally, I totally agree with all of your technical suggestions. So let me just respond with a few headwords:

  • The handling of nodes is already designed with multiple, cross-dimension content views in mind
  • As said above, we will take the UX improvements you’ve mentioned into account
  • The tree is currently more a projection of the available nodes in the redux store, which can be freely sorted and filtered. Dominique even suggested to make the tree replaceable with different views, that would allow for a less web-centric editing experience - which wouldn’t be that hard given what we have implemented so far. I think the hardest part here will be to reason about server-side solutions e.g. for searching nodes.
  • Validation will take place at a different level. Currently our implementation makes no difference to the source of a change. Changes to a node (e.g. property change, creation of a node, deletion of a node) are handled as separate objects, that also could be validated separately. We haven’t spoken about this so far, but imho that would be the place to do validation.
  • Nice idea for the URL handling :slight_smile: I took the freedom to create an issue for that: https://github.com/PackageFactory/PackageFactory.Guevara/issues/88

CSS:

Using css modules is a controversial decision that without any doubts results to the drawbacks you’ve mentioned. We would have the option to switch to a modular BEM-driven CSS approach or to somehow find a way to let our server side rendering understand css modules. Luckily the component-driven approach gives us some time to think this through, though we shouldn’t postpone this for too long. As you’ve said, we also should take theming into account while re-visiting the css approach.

Now, to close this, I just want to express my hopes, that despite your reasonable doubt, there is a way to do this together.

Best wishes,
Wilhelm

3 Likes