SpeedUp Neos-Backend?

Hi,
I love to work with Neos. Most of the time it is a great pleasure to create or change websites and content.
But on my current Site I have problem with Neos. The system is slowing down more and more.

The site is creating a table in the frontend, and I’m really happy about:

But in the Backend, if I change something and press «Apply», Neos needs a lot of time! More then 30 seconds!

I create a nodeType to define checked or crossed icons with SVG. All the other stuff is only «text» and «headlines» very occasionally a «picture».
I guess it is the nesting of the elements the Neos proposes in the neck. Because the site also uses a lot of foundation f6 grid. No problem on all the other websites with same grid nodetypes and also with a lot of f6 grids. And absolutely no problem in Frontend with this site.
But with more and more little stuff, the site is slow down more and more in editor-mode.

###Maybe someone has the same behavior/problem and knows a simple and easy solution to speed up the backend? Would be great.

  • Possibly there is a way to cache some elements or to hide others or to set some options in the js-script.
  • Or knows someone a good and speedup solution for a lot of small data-pieces fit in gridSystem?

No matter which web browser I use, average load-time is quite similar.
Also I guess, there are not too many elements in the typo3_typo3cr_domain_model_nodedata: Only 728 records.

Maybe the ContentModule-built.js Script or it’s called children-scripts
is the brakeman, because of Chrome’s Timeline:


same in Firefox:

I guess, the source-code as it own is not really messy. It should be easy to render it. But maybe it is the amount of code-lines:

<div class="row near up-close">
    <div class="columns">
        <dl class="smarter-description first-level" id="">
            <input type="checkbox" id="m96b9fde6-f9c6-43d9-bc80-ecc921fdb3ba" checked="">
            <dt><label for="m96b9fde6-f9c6-43d9-bc80-ecc921fdb3ba">Fringilla</label></dt>
            <dd>
                <dl class="smarter-description" id="">
                    <input type="checkbox" id="m851b598e-eea9-4653-8fb0-a9c0a25e6834">
                    <dt><label for="m851b598e-eea9-4653-8fb0-a9c0a25e6834">
                        <div class="row near">
                            <div class="columns small-12 large-p24">
                                <p>Sollicitudin Nibh</p>
                            </div>
                            <div class="columns small-3 large-p19">
                                <p>225</p>
                            </div>
                            <div class="columns small-3 large-p19">
                                <p>112</p>
                            </div>
                            <div class="columns small-3 large-p19">
                                <p>531</p>
                            </div>
                            <div class="columns small-3 large-p19">
                                <p>222</p>
                            </div>
                        </div>
                    </label></dt>
                    <dd>
                        <div class="row near up-close">
                            <div class="columns small-12 medium-8 large-6 medium-centered large-centered">
                                <h2>Egestas Mattis Sem Aenean</h2>
                                <p>Donec ullamcorper nulla non metus auctor fringilla. Vivamus sagittis lacus vel augue laoreet rutrum faucibus dolor auctor. Vivamus sagittis lacus vel augue laoreet rutrum faucibus dolor auctor. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus.</p>
                            </div>
                        </div>
                    </dd>
                    <input type="checkbox" id="mff137646-5a42-48db-a7d7-ac364bebb71e">
                    <dt><label for="mff137646-5a42-48db-a7d7-ac364bebb71e">
                        <div class="row near">
                            <div class="columns small-12 large-p24">
                                <p>Risus Amet</p>
                            </div>
                            <div class="columns small-3 large-p19">
                                <p></p>
                                <p>19</p>
                            </div>
                            <div class="columns small-3 large-p19">
                                <p></p>
                                <p>632</p>
                            </div>
                            <div class="columns small-3 large-p19">
                                <p></p>
                                <p>548</p>
                            </div>
                            <div class="columns small-3 large-p19">
                                <p></p>
                                <p>71</p>
                            </div>
                        </div>
                    </label></dt>
                    <dd>
                        <div class="row near up-close">
                            <div class="columns small-12 medium-8 large-6 medium-centered large-centered">
                                <h2>Bibendum Magna</h2>
                                <p>Morbi leo risus, porta ac consectetur ac, vestibulum at eros. Maecenas sed diam eget risus varius blandit sit amet non magna. Vestibulum id ligula porta felis euismod semper. Maecenas faucibus mollis interdum.</p>
                            </div>
                        </div>
                    </dd>
                    <input type="checkbox" id="mc727eba5-f03e-4337-8282-c195645f0af6">
                    <dt><label for="mc727eba5-f03e-4337-8282-c195645f0af6">
                        <div class="row near">
                            <div class="columns small-12 large-p24">
                                <p>Etiam Purus Fringilla</p>
                            </div>
                            <div class="columns small-3 large-p19">
                                <p></p>
                                <p>Amet-Egestas</p>
                            </div>
                            <div class="columns small-3 large-p19">
                                <p></p>
                                <p>Pellentesque-Ornare</p>
                            </div>
                            <div class="columns small-3 large-p19">
                                <p></p>
                                <p>Risus Porta</p>
                            </div>
                            <div class="columns small-3 large-p19">
                                <p></p>
                                <p>Ligula-Nullam</p>
                            </div>
                        </div>
                    </label></dt>
                    <dd>
                        <div class="row near up-close">
                            <div class="columns small-12 medium-8 large-6 medium-centered large-centered">
                                <h2>Ipsum Magna Pellentesque Purus</h2>
                                <p>Donec ullamcorper nulla non metus auctor fringilla. Donec id elit non mi porta
                                    gravida at eget metus.<br>Lorem ipsum dolor sit amet, consectetur adipiscing elit.
                                </p>
                            </div>
                        </div>
                    </dd>
                    <input type="checkbox" id="m7ec8ce8b-989f-4f76-9834-7acd42417a53">
                    <dt><label for="m7ec8ce8b-989f-4f76-9834-7acd42417a53">
                        <div class="row near">
                            <div class="columns small-12 large-p24">
                                <p>Ultricies</p>
                            </div>
                            <div class="columns small-3 large-p19">
                                <span class="icon-wrapper"><svg class="icon"><use
                                        xmlns:xlink="http://www.w3.org/1999/xlink"
                                        xlink:href="//localhost:3000/_Resources/Static/Packages/Vendor.Site/Images/sprite.svg#ja"></use></svg></span>
                            </div>
                            <div class="columns small-3 large-p19">
                                <span class="icon-wrapper"><svg class="icon"><use
                                        xmlns:xlink="http://www.w3.org/1999/xlink"
                                        xlink:href="//localhost:3000/_Resources/Static/Packages/Vendor.Site/Images/sprite.svg#ja"></use></svg></span>
                            </div>
                            <div class="columns small-3 large-p19">
                                <span class="icon-wrapper"><svg class="icon"><use
                                        xmlns:xlink="http://www.w3.org/1999/xlink"
                                        xlink:href="//localhost:3000/_Resources/Static/Packages/Vendor.Site/Images/sprite.svg#ja"></use></svg></span>
                            </div>
                            <div class="columns small-3 large-p19">
                                <span class="icon-wrapper"><svg class="icon"><use
                                        xmlns:xlink="http://www.w3.org/1999/xlink"
                                        xlink:href="//localhost:3000/_Resources/Static/Packages/Vendor.Site/Images/sprite.svg#ja"></use></svg></span>
                            </div>
                        </div>
                    </label></dt>
                    <dd>
                        <div class="row near up-close">
                            <div class="columns small-12 medium-8 large-6 medium-centered large-centered">

                                <h2>Condimentum Ornare Cras</h2>


                                <p>Duis mollis, est non commodo luctus, nisi erat porttitor ligula, eget lacinia odio sem nec elit. Etiam porta sem malesuada magna mollis euismod. Integer posuere erat a ante venenatis dapibus posuere velit aliquet. Etiam porta sem malesuada magna mollis euismod. Vivamus sagittis lacus vel augue laoreet rutrum faucibus dolor auctor. Fusce dapibus, tellus ac cursus commodo, tortor mauris condimentum nibh, ut fermentum massa justo sit amet risus. Morbi leo risus, porta ac consectetur ac, vestibulum at eros.</p>
                            </div>
                        </div>
                    </dd>

                    <!-- 1000-lines more like above -->
                    <!-- ... -->
                    <!-- ... -->
                </dl>
            </dd>
        </dl>
    </div>
</div>

Would be great if someone could get me to the right direction.
Thanks and cheerio Martin

Sounds like the content cache is causing the slow down. It’s file based by default and gets slower and slower over time. An easy way to check is to flush the cache and see if the backend becomes quick again.

See https://speakerdeck.com/aertmann/making-friends-with-pagespeed-and-webpagetest-web-performance-with-insights-for-neos?slide=34 for how to solve it using a proper cache backend.

1 Like

Thank you for your help and link @aertmann .
Unfortunately an ./flow flow:cache:flush --force doesn’t solve the problem. Same long load-time as before.

But if I do a change in field «Hide after» or «Hide» the «Apply»ing is quite fast. But «_hidden» and «_hiddenAfterDateTime» in «TYPO3.Neos:Hidable» or «TYPO3.Neos:Timeable» is without «reloadIfChanged» Flag.

It’s just a feeling, but maybe I have produced the bottleneck myself in one of my nodeTypes. For only a few times, the bottleneck is no problem. But with more and more, and the nesting it get closer and closer.
For example

  • I have devined a description field for my DL|DD|DT- container. with this, it is more clear in backend where the customer have to change some values, because of this nodeDescription is shown as label. In .yaml-File I use this maybe “not efficient” code:
label: ${'[DL]: ' + (q(node).property('nodeDescription') ? '[' + q(node).property('nodeDescription') + '] ' : '') + (q(node).property('value') ? q(node).property('value') + ' ' : 'SmarterDescription - (root-Element)')}
  • Same in the Grid-nodeType. There I have to add classes for small|medium|large in f6 foundation I do it within the .ts2 file:
### Grid with prototype for Column
prototype(Vendor.Site:SmarterGridColumn) < prototype(TYPO3.TypoScript:Tag) {
	attributes.class = ${'columns' +
		(String.isBlank(q(node).property('small')) ? '' : ' ' + q(node).property('small')) +
		(String.isBlank(q(node).property('medium')) ? '' : ' ' + q(node).property('medium')) +
		(String.isBlank(q(node).property('large')) ? '' : ' ' + q(node).property('large')) +
		(String.isBlank(q(node).property('smallOffset')) ? '' : ' ' + q(node).property('smallOffset')) +
		(String.isBlank(q(node).property('mediumOffset')) ? '' : ' ' + q(node).property('mediumOffset')) +
		(String.isBlank(q(node).property('largeOffset')) ? '' : ' ' + q(node).property('largeOffset')) +
		(String.isBlank(q(node).property('additionalClass')) ? '' : ' ' + q(node).property('additionalClass')) +
		(q(node).property('useWaypoint') && (!node.context.inBackend) ?  ' waypoint' : '' ) +
		(String.isBlank(q(node).property('waypointAnimationType')) && (!node.context.inBackend) ? '' : ' ' + q(node).property('waypointAnimationType')) +
		(q(node).property('scrollMeElement') ?  ' scrollme' : '' )

		}
		content = TYPO3.Neos:ContentCollectionRenderer

		# We have to define contentElementWrapping, because we do not inherit from TYPO3.Neos:Content
    	@process.contentElementWrapping = TYPO3.Neos:ContentElementWrapping
}

Maybe it is to much of work or messy code-style, so I comment-out both with «#» in .yaml the label-line and the superClass mixin-inclusion, in ts2 all the lines for attributes.class. Layout looks horrible, so I guess, the commented-out code is not in work, but:
->NO CHANGE in speed.

I tried also with a new ./flow flow:cache:flush --force
But: ->NO CHANGE in speed.

I work with MAMP-Pro on osx 10.11.5 so I did also «try&error» with:
php 5.5.10
with/without xdebug
without cacheModule/ with APC or XCache or OPcache
php 7.0.0
with/without xdebug
without cacheModule/ with APC or XCache or OPcache
But: ->NO CHANGE in speed.


Maybe I did an other «no-go/bad solution» but can’t see it :cry:.

##My 3 questions:
- Is there a documentation/tutorial to find bottlenecks in own Code, or for find integration-problems in NEOS?
I read your speakerdeck @aertmann looks great, but with the «flush->no speedChange» I guess it is maybe some problem with my code in general.

- I work with phpStorm and xDebug so maybe there is a possibility to do a profiling with? What I have to do for, because of Neos extensive cache-mechanism?

-Is there a easy tool|plugin to do profiling&debugging for also «NO-Neos-Cracks»? I found this : “Profiling and debugging TYPO3 Flow and Neos (T3CON12DE)” but it looks like it is past away. Do know someone more about?

Maybe I’m on completely wrong path. Suggestions are very welcome!

Thanks for help
Martin

@aertmann I read also your «second speakerDeck».

My challenge: I don’t like do muscle up my developing-server and struggle later on with server(s) out in the wildlife from mittwald, proserver (from @daniellienert), domainfactory, or something much more crazy like 1&1 or some other mass hoster.

Would be great to find a more lightweight solution; because Redis or Memcache force an managed or root server most of the time (says my quick research). And if it is at all possible, I would like to lighten the load on my and certainly the server’s neck.
Stay as as flexible as possible and not become root or server-admin is rahther my main online-life goal :relaxed:

  • At the beginning, with only 1or 2 main-parts -> with approximate 7 second-level-parts and within each time 5 third-level-parts the load-average-time was normal/quite fast.
  • Now with 5 main-records -> 7 second each-time and 5 to 10 third-level the load-average-time is annoying slow. And there are at least 3 main-parts to do :persevere:.

##Thoughts/Questions:
####Could it be a solution/workAround to create some “hidden-in-menu”-sub-sites with only one main-part each? This 8 or more subsites, will stay fast and easy to edit and then concat for frontend-only with EEL/typoscript. Or will each of these concats only be an even closer bottleneck?

####Does anyone have experience with a lot of tag-elements nested and solved the slowdown => speedUp without heavy Hardwares/SoftwareTools on server-side?

Hi Martin,

lets try to dive into this. At least I would guess that Redis or Memcache will not substantially help you, another option would be Pdo backend and database, but still I doubt that will fix it. Some of the screens you posted, suggest that this is solely JavaScript related. Which probably means debugging is best done in the browser with dev tools.

If I don’t forgot I will try and read a bit more into the details you posted later on.

Hey Christian,
would be really cute if you could find time and a helping hand.

There are only two javascripts in:

  • app.min.js with own simple Menu and animation stuff in.

  • script.min.js with some regular stuff like modernizr in.
    Both jshint’ed minimized, and concat’ed. Without errors in DevTools Console from chrome.
    In Firefox there are only :

  • getPreventDefault() sollte nicht mehr verwendet werden. Verwenden Sie stattdessen defaultPrevented. ContentModule-built.js:422:59052

  • document.createAttribute() sollte nicht mehr verwendet werden. Verwenden Sie stattdessen element.setAttribute(). translator.js:2658:11

  • setAttributeNode() sollte nicht mehr verwendet werden. Verwenden Sie stattdessen setAttribute(). translator.js:2666:13

  • Das nodeValue-Attribut von Attributen sollte nicht mehr verwendet werden. Verwenden Sie value stattdessen. ContentModule-built.js:422:849651

To have certainty, I deleted both js-scripts from the template, disabled the IM-Translator , flushed the flow cache, restart the browser and double checked in devTool => NO “own” JS in town:
But also -> NO CHANGE in speed.

Only 2 warnings in Firefox left:

  • Das nodeValue-Attribut von Attributen sollte nicht mehr verwendet werden. Verwenden Sie value stattdessen. ContentModule-built.js:422:849651
  • getPreventDefault() sollte nicht mehr verwendet werden. Verwenden Sie stattdessen defaultPrevented. ContentModule-built.js:422:59052

So, the base for find the bottleneck should be relatively clean I hope. Deeper instructions very welcome.

Hi

Yes it could (50% faster ca. ~10sec.), but with a big drawBack =>
Change in OriginalNode will only shown in Frontend (linked Stuff), if I flush the cache…
So good for fill a lot of values first time as knower developer, but not practical after goLive, with average Editors out there…

But learned a lot with :wink:.
Would be great to find the js-bottleNeck.

In regards to the cache you should be able to tag the entries on the other page with these nodes and it should take care of flushing correctly.

@christianm Thanks!

It works (I guess):

  • If I don’t change stuff in splitting backend-PartPage => load-time from FrontEnd-Summary-Page is quite speedy.
  • If I change something => longer loading time and changed value in FrontEnd-Summaray-Page.

But looks a bit weard and mess with all the nested flowQueries further down!

Unfortunately, I have to get the particular SmarterContainer otherwise the double Ancestor «[DL]: Nachschlagewerk …» would break the layout in FrontEnd-Summary-Page, on other hand I need this [DL]-tag in Backend for display correctly the reduced list-part.

So I checked the value of indentifier with:

#SmarterCollector.html

{namespace ts=TYPO3\TypoScript\ViewHelpers}
<f:if condition="{blnLinked}">
    <f:debug>{linkedNode.identifier}</f:debug>
<ts:render path="CCRenderer" context="{node: linkedNode}" />
</f:if>

output:

I guess this matched value in linkedNode.identifier (with Identifier-String in Backend) is the right value to use for docu’s entryTags documentNode.Identifier-String-part.


This linkedNode.identifier is the Identifier for «the other page» or in this context «the linked part of split page-part» to speedUp (at least a bit) the Backend-Editing with this splitting in smaller parts.

#SmarterCollector.ts2

### SmarterCollector to use nodeContent in other Neos-places
prototype(Vendor.Site:SmarterCollector) < prototype(TYPO3.Neos:Content) {

    CCRenderer = TYPO3.Neos:ContentCollectionRenderer

    blnLinked = ${q(node).property('internalLink') != NULL && q(node).property('internalLink') != '' ? TRUE : FALSE}
    
    linkedNode = ${q(q(node).property('internalLink')).find('[instanceof Sd.SmarterDesign:SmarterContainer]').get(0)}

    @cache {
        mode = 'cached'

        entryTags {
            # Whenever the linkedSplitPagePart>nodeType SmarterContainer changed => this change/flush the SmarterCollector's cache            
            1 = ${'Node_' + q(q(node).property('internalLink')).find('[instanceof Sd.SmarterDesign:SmarterContainer]').get(0).identifier}
            # Whenever an descendant of the linkedSplitPagePart>nodeType SmarterContainer changed => this change/flush the SmarterCollector's cache             
            2 = ${'DescendantOf_' + q(q(node).property('internalLink')).find('[instanceof Sd.SmarterDesign:SmarterContainer]').get(0).identifier}
        }
    }
}

###Questions:

  • is there an easier way, to get the identifier from the specific nodeType SmarterCollector, defined in yaml as «type: reference»? (without the mess of nested flowQueries)
  • Can anyone show me some storage-solution (like reusable property/variable concept from js/php) for TypoScript, to define the identifier-string once and reuse for property «linkedNode» and also for concat the «entryTag-property» both-times?
  • Did someone knows some Tool/Helper/«f:debug»-solution to check the cache easy and reliable: like this idea?