TemplateRootPathes overriding in Views.yaml

Creating Neos & Flow Code
1

I recently stumbled over an inconsitent or at least unexpected behavior of the Views.yaml that occurs if multiple TemplateRootPathes are defined and overridden. (LayoutRootPathes and PartialRootPathes behave the same)

If multiple requestFilters match they are sorted by weight and afterwards the configurations of all matching filters are merged with the lowest weight first. The problem is that during that process keys that are added to ‚templateRootPaths‘ in the higher weighted requestFilters end up last in the merged configuration. That leads to the situation that the first templateRootPaths of the highest weighted configuration is used as last.

Example requestFilters in Neos Core

-
  requestFilter: '__default__'
  options:
    templateRootPaths:
      'TYPO3.Neos': 'resource://TYPO3.Neos/Private/Templates/Media'
      'TYPO3.Media': 'resource://TYPO3.Media/Private/Templates'

Additional requestFilters in third party package

-
  requestFilter: '__higher_weighted_matcher__'
  options:
    templateRootPaths:
      'Vendor.Site':  'resource://Vendor.Site/Private/Templates/Media'

I that case i would expect the path from the Vendor.Site Package to be evaluated before the core pathes but since the merging is not taking the order into account this is the merged configuration.

-
  requestFilter: '__higher_weighted_matcher__'
  options:
    templateRootPaths:
      'TYPO3.Neos': 'resource://TYPO3.Neos/Private/Templates/Media'
      'TYPO3.Media': 'resource://TYPO3.Media/Private/Templates'
      'Vendor.Site':  'resource://Vendor.Site/Private/Templates/Media'

As you can see the Vendor.Site path is added as last key and thus never used. This behavior makes it impossible to add single pathes to the top of the list to make sure they are evaluated first. Also adding the original pathes to the overriding configuration does not alter this situation.

Possible Solutions

In general i see some possible approaches to solving this problem. I already tried the two promising aproaches and created pr’s for that.

  1. Make ArrayMergeRecursiveOverrule ordering aware … i think this would lead to many unexpected effects and should be avoided

I personally consider approach 3 to be not breaking and a usefull feature.

-
  requestFilter: '...'
  options:
    templateRootPaths:
      'Vendor.Package':
        'position': 'before Vendor.OtherPackage'
        'value': 'resource://Vendor.Package/Private/Templates'

Questions:

  • Which additional approaches are possible to make adding TemplateRootPathes in third party packages usable?
  • Which one of those approaches should go to the core?
1 Like
2

As 3 changes the configuration syntax it should be considered breaking as well I guess. Maybe a fallback can be done though. So I guess 2 or 3 it is. One particular problem I have with hte implementation is that IMHO the fluid view shouldn’t need to know about this. It receives an array of paths and uses them in order. The sorting should happen beforehand.
In that regard option 2 could be easier to implement. I think the basic use case is to be able to override view options from the outside, maybe even multiple times, which would be fulfilled with 2 and 3 IMHO as long as you manage to create a __higher_weighted_matcher__ to override the previous one. I am not sure about that, but that’s a story for another day.

3

The new syntax is optional and can be mixed with the old one. That’s why i consider it non breaking.

If we agree to change the behavior we should really consider option 4. Avoiding merging solves the problem here and removes some unexpected behavior … at least to me it was unexpected.

4

Thanks for bringing this over here

As mentioned in the PR:

We just can’t know what the correct behavior should be if the more specific configuration just sets one array item to an existing array.
In the case of TemplatePaths you probably want to add it to the beginning but there might be other options where you just want to append this.

…

I like that solution because it can be implemented completely backwards compatible.
On the other hand it makes the configuration pretty “bloated” making the “merging” not really worthwile… that’s why:

would be the most consistent and easy to comprehend…
It would also solve another issue that we currently have:
If you change the template implementation in your Views.yaml The View will complain about any base options that are not supported by that view…

I think we could live with that different behavior if we communicate it in the release notes and provide a core migration that adjusts any existing Views.yaml

5

I don’t think that core migrations are possible because the filters are evaluated at runtime. Nearly impossible to find all filters that could match the same requests as a given filter. At least with reasonable effort.

I would prefer option 4 nevertheless.

6

Option 4 destroys a whole world of extensibility though. I think there are a lot of edge cases in which this will break and prevent implementations.

7

I personally find the current flexibility bit too much and would prefer easily understandable behavior.

In which cases is the current behavior necessary? Maybe i’ll get the point then :wink:

8

I can’t present one, but will check in my projects. But from experience in previous changes that also made things simpler, people will have use cases and will be unhappy.

That said: I am of course all for making things simpler, but I also want to make sure people are not made unhappy :wink:

1 Like
9

The more I think about it the more I root for the explicit solution, even though it might make some people unhappy for a while. still:

I don’t think that core migrations are possible because the filters are evaluated at runtime

Really? I do think it could work like this:

  1. collect all request filters
  2. for each filter find all other filters that “overlap”*
  3. copy options from the less specific to the more specific filters

*) This is the tricky part, but it is deterministic whether a filter matches a subset of another one or not

In any case we could warn about every Views.yaml configuration during the migration

Which one? Everything is still possible

10

ping

11

I would prefer 3, but if you want to go 4, that’s fine by me, it’s super clean and simple.

12

I would prefer option 4 because of it is clean and predictable. I could do a pr for that but if we want a core-migration for old filters i would abstain. I fear such a migration would take a real high effort to get it right with very little benefit.

The PR for Option 3 is already there so if that is preferred i can clean this up.

13

That would be great!

You’re right, it will probably be quite a hassle to get that working.
What we should do though is to display a generic warning for any existing Views.yaml – we did similar things previously and I can take care of the respective core migration.

14

Finally found the time to create a pr: https://github.com/neos/flow-development-collection/pull/690

1 Like