Problem with pages with a lot of content elements and Aloha initialization

mad · August 2, 2016, 8:06am

Hi together,
maybe someone had also problems with editing pages with a lot of content elements combined with Aloha initialization. I don’t know why, because my skills in javascript-debugging too small for ui with such many external dependencies and frameworks. But would be great, someone could pointing in the right direction to find a solution or work-around …
###Challenge:
The Backend is loading incredible slow for pages with a lot of content elements.

###crazy facts:

Loading-time «hole backend» (reload with chrome dev-tools enable for disable caching): ~90 seconds
Loading-time «website within backend without Navigate - / Structure - / Inspector - Area’s (Appears before all the other stuff is loading after on top) : less than 1 second
Loading-time «hole backend» (Apply-Button in Inspector to save a change) : ~119 seconds and longer
Loading-time «structure-Area» (reload-button*): ~1.5 second
Loading-time «Navigate-Area» (reload-button*): ~1.5 second

*

I thought the many nodeElements would cause the summary-time in structure because Neos has to run through, but in same consideration I couldn’t explain myself, why then the structure-tab has to lacy-load nested elements if I moving to a deeper level…
But the reply

let me suspect that this problem might not have to do with Neos and the many nodeElements itself, but rather with Aloha.

Also the loading-time for «Structure» and «Navigate» above sounds as the problem is not the many of nodeTypes for Neos its own. Maybe Aloha ist struggling?

Question:

Has anyone (maybe @dimaip ) stumble over the same problems and maybe found a solution or a work-around for Aloha or Neos?

dfeyer · August 2, 2016, 8:36am

There is to area where performance can be improved:

Cache (the reload after publish, trigger a lots of check/flush in the cache) what kind of cache backend did you use ?
JS and here it’s way more complex to improve the current situation

If you use the FileBackend, it’s slow espacialy when the cache use tags, and this is the case for the ContentCache.

And other important informations, is how many nodes you have in this page ?

If you check the Chrome inspector is this wait time fully related to JS or some backend API HTTP request ?

mad · August 2, 2016, 1:17pm

I use the default «FileBackend» but the longest loading-time is not the «reload after publish». It is the «Apply-Button-process» after change some small value in inspector …

Unfortunately chrome dev-tools has problem to handling, most of the time (currently > every try) it close without messages: maybe it crashes.
##Apply:
Firefox’s Network monitor for «Apply»:

Runtime Analysis (Laufzeitanalyse) message > «buffer-capacity is 100% full. Old value will be overwritten» (free translation DE->EN) and turns crazy. Think this will also the problem for chrome dev-tools. I have to kill the firefox-process…

#Reload:

Is not a reload after publish for ~90 seconds; It is a simple reload of browser-window:
So I guess no cache flush and recreate …
And the website with all the nodes is loading in less then 1 second, before it drive crazy with the backend-panel-building …

Firefox’s Network monitor for «browser-reload»:
tons of really-short loading-processes > than gap > get-child-nodes-for-tree > than big gap > workspaces > than more of really short loading-processes
Could it have to do something with workspaces?

With Firefox and the reolad-browser-window-process I get a Run time analysis (Laufzeitanalyse):
There are tons and tons of «Kleine GC» and «Nicht-inkrementelle GC» in the red-bar in the overview in top: I print-Out only a part of.

…
…
On the other end of «Kleine GC», «Nicht-inkrementelle GC» and a few «Stil Neuberechnung» is this:

Between: 99% «Kleine GC» and «Nicht inkrementelle GC» and maybe 1% «Stil-Neuberechnung»
So I save this as *.json file and count the
«MinorGC» => 7839
«nonincrementalReason» => 93

I don’t know what the Nursery-Collection- or MinroGC-Problems (maybe Garbage Collection) are triggering. But If there would be canceled (red-bar in top-overview would disappear) the loading-time would be around 102’400 ms shorter. I think it would be less than 10 seconds (because I can’t start and stop the analysis immediately) left: That would be more than OK!

Maybe it is the same cause for the extremely long «apply»runtime-process. But I can’t do an analysis. Did someone know how I could enlarge the «buffer-capacity» for runtime-analysis?

How many nodes:

If I do the right query with «SELECT count(*) FROM typo3_typo3cr_domain_model_nodedata WHERE path LIKE ‘%node-577df83673007%’» («node-577df83673007» is Name under Inspector>Additional info «Name»)
Result: 1839
Or do I have to query something else or with flow or can I find it in Neos?

Hope this will help to do one step closer to the bottleneck.

dimaip · August 2, 2016, 11:04pm

Just wondering, are you using the latest version of Neos?

mad · August 3, 2016, 6:11am

Hi @dimaip,
yes, version «typo3/neos» 2.2.5.

mad · August 3, 2016, 11:48am

Hi,
I have found a solution to do an Runtime-Analysis for «apply» with a less-many-node-test-website:

Node-Elements for this site: 674
#Apply:
(was enable to do an runtime-analysis with less nodeElements)
Result in Firefox:

firefox-apply-analysis.png1466×244 67 KB
Picture further down looks similar

If I’m right:

Total Loading-Time: ~50’000ms
HttpRequest: (1478+695+1574) 3747ms
Differenz: ~46’000ms for JS

in exportet json-File of analysis:

MinorGC => 2’332
nonincrementalReason => 24
OUT_OF_NURSERY => 2’300

Hope someone will see more than I in this analysis or in the incredibly frequent MinorGC/OUT_OF_NURSERY …

dimaip · August 3, 2016, 11:51am

There were some horrible problems with editables (introduced by me!) in 2.0 fixed in 2.1 I think so that probably is not your case.

We suffer from this problem too, especially on pages with tons of content. I don’t know how to solve this issue without dropping Aloha, that’s why I actively participate in the UI rewrite project, which performs times better (it uses CKeditor).

mad · August 3, 2016, 11:59am

Hi @dimaip,
nice to hear!

Thought I was the only one and do something very simple fundamentally wrong; Or have built some nodeTypes wrong.
Thanks!

Have you found a temporary work-around to reduce a lot of elements from type row>columns>htmlTag, or did you found a solution to break apart the elements or to speedUp the Backend in some other ingenuousness ?

Nevertheless, hope the suffering will redeemed, maybe with changes in Aloha or with the switch to CKeditor.

dimaip · August 3, 2016, 12:40pm

I try to break large pages apart for editing, e.g. I edit each section of single-page website on a separate page, and then just render them together for frontend visitors.
My hopes are with the new UI, but if you’ll find a quick way to fix performance of the current UI, that would be great!

mficzel · August 3, 2016, 1:24pm

Imho the most expensive cache operation for the typoscript caches is the tag-based cache invalidation that happens during publish since the now invalid caches are thrown away immediately. The next reload takes a bit longer because some caches are rebuilt but not as long as the flushing if there are lots of documents.

You can easily validate this by running flow:cache:flush … if the system runs much better afterwards especially when publishing you have a peformance problem with cache invalidation.

I had that several times for sites with many nodes and usually solved it with redis.

# Use Redis Caching in Caches.yaml
TYPO3_TypoScript_Content:
  backend: TYPO3\Flow\Cache\Backend\RedisBackend
Flow_Mvc_Routing_Resolve:
  backend: TYPO3\Flow\Cache\Backend\RedisBackend
Flow_Mvc_Routing_Route:
  backend: TYPO3\Flow\Cache\Backend\RedisBackend

If redis is not available you could try sqlite like here @christianm did in his experiment here Try this please, Cache configuration

aertmann · August 3, 2016, 6:04pm

674 node elements definitely sounds like a lot. I seriously wouldn’t want to be editing a page with that many elements. You mean there’s literally 674 nodes elements in the content structure tree right?

I assume most of them don’t actually have any editable properties right?

I would think even a more performant UI would still choke on so many elements, thus you should consider if you can reduce them somehow.

You mention having row -> columns -> html tag, I assume that means three different node elements right there? It’s a bit hard to tell, since you not really saying what you’re doing and why, but normally that would sound like a wrong approach. If it’s for total flexibility, then you should consider making it more strict and provide specific node types that solve common needs instead. Also you can probably skip the columns node type and incorporate it into the row node type.

As for improving the performance in current the UI itself, it might be possible to find some bottlenecks in the initialization of content elements, like it’s being done multiple times and maybe it’s applying Aloha to elements that don’t even have inline editable properties.

mad · August 4, 2016, 9:29am

Hello @aertmann,

I did break apart the website and comparison-part and create a nodeType to collect them together only in frontEnd. But at the end I guess there are all of them in the content structure tree (didn’t count them one by one):
1 day later,

the less-many-node-test-website got 1’013 in 6 parts.

the test-with-many-elements 1’834 in 8 parts.

Approach

Yes, thats also the tendency in my suspicion . I torment still myself, and think, how could I simplify it???
####neos.demo

The approach in demoSite or neos in general with grids, is for lots of websites OK - for sure. But what’s about websites within columns don’t behavior only different in medium (col-sm-6, or col-sm-3)? Than it looks like an oversimplification. Unfortunately I can’t design a useful and more flexible derivation out of them – Because:
####more flexible task?
What about the situation if the first col have to be 12 in small, 4 in medium and 6 in large, and simultaneously «col 2,3,4,5» has to be different behavior depend on viewport-width (3 small, 4 medium, 2 large). Or some other a more complex requirements.

possible solution

So I guess:
In this complex case and with your recommendation for more specific nodeTypes, I have to create a nodetype with fix 5 columns, and hardcoded classes, because of to many variants. All of them push in one dropDown select-box would neither simple for template-layout creation nor self-explanatory in inspector.
And the same website has at least 5 totally different grid-divisions. So I would need to create for each one a particular nodeType, but this will mess-up my «create new …»-popup and it would be really hard to pic the particular nodeType.

But maybe I try it in future to have a comparison.
If you or someone-else have a good idea to simplify, I would be thrilled!

current solution

For this website I create a foundation-6 approach on dimaip’s f5-grid. It is perfect for flexibility, but at the end, there are long pauses while Aloha is building «?what ever?». Maybe because of this approach? I don’t know.

Maybe Aloha is too lovely and says «hello» to everyone

Strangely the backend-website with all the stuff for Aloha (in dev-tools the source-Code has all the additionally Divs filled full with Aloha-stuff) appears immediately. But then there is a extremely long waiting-time for (I think Inspector). But I don’t know why, if Inspector would only have connection with the current selected element?
The structure-panel do lazy-loading for next deeper grid-nesting. So I guess this should not go nuts.
Big question mark in my head?!

####What I’m doing and why

Sorry for all the read-stuff, but you are asking for … I don’t like to «not really saying»!
In this particular website I have to show a comparsion. This should use in grid for responsive design.
In large the layout should be like in picture further down. In small, the group-headers (Fringile, Trisique …, Cras …) should show 12 column, but the first column in sub-group (medium-grey) should go to 12, the others 3. In medium quite the same, with a smooth split between col 3 and 4.
If the user click on Group-Header, the sub-items will shown. If the user click on sub-line, the «additional Info» will appear (violet bordered: Ipsum Magna Pe…).

at the end:

for each Group-header: 4 nodeElements ( X 6 other groups) = 24 nodeElements
for each comparsion-line: 19 nodeElements (X 10 comparsion-items on average X 6 groups) 1140 nodeElements
A few of the «additional Info» has also images and links grouped in grid (= more nodeElements).
And then, all the other website-Layout-Elements = more nodeTypes/nodeElements

It rocks in frontEnd-Website. All the challenges are well resolved and the ui works smoothly.

I would like…

to reduce the nodeElements, and change the approach. But with the default-neos grid, I have problem to do not oversimplification with big draw-backs for layout and responsiveness - I guess.
But I’m more than interested in, if someone found a cool genius solution!

Elements that don’t even have inline editable properties == HOT-POINT ?

Thats also a point, that I would like to optimize. Because of row- and column-nodeTypes have no inline-editing inside. So I could delete the «inlineEditable: TRUE» or set it on FALSE. But for some reason, if I delete the Line 16 or set on «FALSE» , in backend > this edit-area is hurtling down further and the scroll area is growing immediately gigantic.
Maybe I have to set an other value? Or how I could command Aloha, don’t say hello to Elements without «inline-editable-values»?

Hope this really saying: how, what and why

Would be great if we could find some better solution for simplification in backend-structure and for Aloha less go berserk!