Composer dependencies: declare transitive dependencies?

Creating Neos & Flow
1

The question

Should we declare requirements (again) that are a transitive dependency of another dependency already?

Yesterday I came across a PR that added some new dependencies to a composer manifest. My immediate reaction was: that is redundant!

After all, we don’t declare most of the Doctrine packages, since they are required by Doctrine ORM anyway. Makes for an easy to understand composer manifest. Then the thing got me thinking, and I decided to check what others think. Composer is not the only, nor the oldest dependency manager, after all. So these two are worth reading IMHO:

And there is even a book dealing with this topic (among others, of course): Maven: The Definitive Guide

A conclusion?!

Now, after reading this, I tend to believe two things:

  1. We should declare any library/package that is actually used in the code
  2. This is a ton of work now and later

Sigh

So much work, so little time

Maybe now is the point where trying to be a good citizen in the Composer ecosystem comes back to haunt us. Because, yes, we should explicitly declare any use of our utility packages instead of just relying on the fact that Flow requires them anyway.

And of course we must explicitly require (at least) Commons and DBAL of Doctrine, since we use them actively in our code. Oh, and the Annotations package. Probably more, we’d need to check. Also other vendors, possibly.

And when we are done, we need to keep track of all those dependencies, and update them whenever needed and/or possible. Thinking about the work we already have with keeping all pour dependencies working, that makes me cringe.

Another conclusion?

So, opinions on that? We could continue with our policy of “no transitive dependencies are re-required”. If a dependency really “goes away”, we will notice. Maybe the extra work of debugging such a failure is a good tradeoff against having to work with those extra dependencies?

2

The dependencies are only transient as long as they are (only) required by a required package. If they are actively used in the package, they should be declared as a direct dependency as that is what they are now.
That’s the reason I declared them in the first place.

I wouldn’t want to dive into the dependencies of the other packages to look if this dependencies are already required there.

But on the other side we could argue about dropping transient dependencies within the ``Neos namespace (so f.e. doctrine would only be required by Flow). Dropping the requirements in one package (in this example Flow), would result into having to check the packages depending on Flow for dependencies to the dropped required package (doctrine in this example) and since this would probably be the case anyway…

3

Just found this one: https://packagist.org/packages/maglnet/composer-require-checker

I’ll check that out, could be very, very helpful…

4

Sure sounds like a nightmare on top of our existing dependency nightmare. It do agree it makes sense though, but as Raffael suggest, probably limit doing it for the core package of Flow/Neos.

Btw. so far we haven’t experienced any issues in this regard right? Because then it’s kinda one of those if it ain’t broken, don’t fix it things.

Also would those dependencies be tied to specific versions or just any version and let the other packages decide which version is compatible? Because if using specific versions, it’s an even bigger nightmare.

5

Right now it ain’t broken. The question arose because some PRs added those “superfluous” dependencies…

And if added, it must be specific versions (or ranges, rather), because otherwise nothing is gained. Basically, never do unconstrained dependencies.

6

Ok, let’s collect my thoughts on this:

If a package needs a certain dependency directly then that’s important to know, also to gather at a glance how the dependency network really is. Imagine I refactor eg. symfony/yaml out of Flow just to learn later that Neos still uses it for something.
We can (at most) control our internal dependencies (eg. Neos depends on Flow), any external dependency (and depdency change can change any time without further notice).

These two thouhgts lead me to the following points I like to make about the topic:

  • We SHOULD declare transitive dependencies, also to be a good example for people writing their own packages.

  • We MUST declare them for third party dependencies that are not under control AND for packages that are not in the core (like the Flowpack package you choose as example)

  • We CAN skip transitive dependencies when the immediate dependency is under tight control (eg. IMHO we can skip the Files Utility package as Neos dependency as we depend on Flow and that is not changing anytime soon and if it were we can easily coordinate moving the dependency over).

To make another point: Going with NOT declaring transitive dependencies means we have to treat any removal of a dependency as breaking change as another package could depend on it without declaring so, that’s obviously bad. And if this is followed as best practice by developers problems are obviously going to follow. Also even if that’s the nastiest part of maintaining those, it protects you and other developers against problems with version bumps of dependencies. You see immediately that composer won’t install it until you raised the dependency version in every depending package which raises awareness.

So tl;dr; it’s a PITA but somewhat important and should be done.

2 Likes
7

Thanks for putting this together. I agree to your list but I would not make a distinction between packages being under “tight control” and others.
I suggest that: If we go that route (and after reading this thread I’m all in) we should go through the packages and add all dependencies whether they are under our control or not

8

So, doing this “go through the dependencies and declare as needed” topic, would there be people to do that for Neos 3.1? You have two weeks… I can recommend https://github.com/clue/graph-composer as a tool to help with visualizing the existing dependencies and already mentioned https://packagist.org/packages/maglnet/composer-require-checker earlier in this topic.

So… anyone? :wink:

9

I can probably have a go at it.

1 Like
10

Thanks @christianm - for those interested, this is the first result: https://github.com/neos/neos-development-collection/pull/1455

11

Transient dependencies should only be declared if the package has version-specific requirements on them. Otherwise, it can be argued that the dependency exist only on the intermediate module. If that module gets a change that breaks your code because of a change on it’s dependencies that just means that your package is not compatible with the new version of the module. If that happens, then adding the dependency to the transient’s module correct version is the next logical step. No need to state transient dependencies when they are implied by the module you are importing as long as that is actually the case.