Release cycle / version numbers / breaking changes

Creating Neos & Flow
1

Continuing the discussion from Creating a pull request:

Semantic Versioning

Since a number of projects and companies rely on Neos and Flow now, we need to establish a more predictable and reliable release concept. The first point we need to agree upon is, if we commit to the concept of Semantic Versioning. Although I don’t see a real alternative, we should make sure that we agree on this.

We might have been negligent regarding Semantic Versioning in some cases in the past, but in my opinion that does not imply that we should or can squash it all together. We need to have clear rules related to this and form a group of people who look after its enforcement (that could be a “Quality Assurance Community of Practice”).

Predictability and Investment Protection

When someone starts using Neos for a new project, she usually wants to know for how long she can upgrade to new versions without having to invest into bigger refactorings of her site configuration or code. Especially bigger organisations with bigger projects need this kind of investment protection because a major upgrade could potentially mean a large invest.

If we follow Semantic Versioning, there is of course no point in releasing one major version after another without a reasonable amount of minor releases in between. That means we will have to take an extra look on changes which break backwards-compatibility. It is always annoying and demotivating if you worked on a bugfix or improvement and then, when it’s ready to be submitted as a Pull Request, the team mates refuse to merge it - for example because it contains breaking changes. Therefore we also need a better way to plan ahead for our own sake - if we know when the next major version is being scheduled, we can either postpone backwards incompatible features to a later version or maybe find a creative way to make the change compatible.

Release Cycle

Here’s a proposal for a release cycle:

  • we follow a simple and predictable scheme: 1.0, 1.1, 1.2, 1.3 (LTS), 2.0, 2.1, 2.2, 2.3 (LTS), 3.0, 3.1, 3.2, 3.3 (LTS) …
  • from our experience so far, we need about half a year for one release; let’s just schedule versions according to it
  • each major / minor release receives bug fixes for 1 year and security fixes for 1 additional year (2 in total)
  • each Long Term Support version receives bug fixes for 2 years and security fixes for 1 additional year (3 in total)
  • we might open a new major branch (e.g. 3.0) before the last minor version of a branch (2.3) is released in order to make larger refactorings and bigger features possible

In practical terms that could mean for Neos:

  • 2.0: August 2015
  • 2.1: December 2015
  • 2.2: August 2016
  • 2.3: December 2016
  • 3.0: August 2017

Definition of Done / Definition of Breaking

We need to discuss and agree upon a Definition of Done (the prerequisite for merging a Pull Request, see also “Creating a Pull Request”) and that must contain a definition of when a change is considered “breaking”.

My take on the latter is the user’s point of view:

Every change in the core which requires an action by the user, integrator or developer which can’t be easily executed as an automated task during deployment is a breaking change.

According to that definition, code migrations are breaking, but database schema upgrades or node migrations (if they can really be executed during a deployment) are okay.

Side note: @kdambekalns found an interesting aspect in the versioning policy of PostgreSQL:

This policy will be followed on a best-effort basis. In extreme cases it may not be possible to support a release for the planned lifetime; for example if a serious bug is found that cannot be resolved in a given major version without significant risk to the stability of the code or loss of application compatibility. In such cases, early retirement of a major version may be required.

So, what’s your take on all this?

1 Like
2

IMHO these discussions should be separated from one another, but well, I will try to only comment/talk about breaking changes here.

And I would actually start with the definition of breaking, that is a hard nut to crack…

This does not suffice for me, even with the explanation below. IMHO just because we can easily execute the doctrine:migrate command doesn’t mean it’s reasonable or a good idae to automate that. Since some time I am arguing that every migration should be marked breaking just because we cannot know the details of the majority of projects out there. A DB migration that strictly only adds DB fields could be okayish, but I still don’t feel 100% confident about it, because it still relies on a deployment process that actually runs migrate.

So for me any change that includes:

  • API modification
  • interface modification
  • frequently used/extended methods modification
  • change of expected behavior
  • migrations of any sort

(probably I forgot stuff here) is to be marked breaking.
This naturally would result in more major releases yes, but we could still differentiate and predict/plan how difficult an ugprade would be (Investment Protection). There are changes with less impact and with more impact…
So from my perspective we have been ignoring SemVer in the past.

And now the big point from my side being one of the people nearer to the code overall I think we are not at the point to realistically stop (minor) breaking changes for a longer time. Sorry, I don’t see that. Just because many basic cases do work fine I see a lot of technical debt, need to change, bugs, edge cases that need to be worked on and I can definitely see some more or less breaking changes coming for that. Waiting over a year for some of those changes now will be a blocker from my perspective because “why work on/fix/change Foo if that was so much better done after Bar is changed which would be breaking”.

I think the PostgreSQL paragraph makes absolute sense and is something we definitely should adopt.

Whatever we agree on, I think it makes sense to have a marker for changes that should stand out but are not strictly breaking (would be for migrations if we decided they are not breaking).

3

Quickly looking at the change log already reveals ~5 changes marked as breaking from various people (including all of us) so seems the next version will be a major version anyway? :wink:

4

what’s your suggestion for a release cycle then?

5

we need to take that serious, go through the changes again (after we agreed on what “breaking” is) and then probably revert changes which are definitely backwards-incompatible.

6

As said, that’s a different discussion to lead. I would collect the topics I speak about, try to get as much of it done as possible for another major release (maybe having a minor in between). This woudl mean a major release without much features (basically the Nov Sprint originally planned) and then talk about more plannable release cycles.

7

The definition of breaking etc. are separate topics indeed, and I didn’t mean to discuss them here so thoroughly. What I’m aiming for is a consensus on the general approach – for example: a fixed schedule with predictable version numbers.

I think I don’t understand what you mean with

major release without much features

– do you mean that we should focus on refactorings / breaking changes for a major release? Because traditionally major releases are those who do carry a plethora of new bigger features.

8

I am saying that we are not in a state to talk about release cycles but should get the topics that potentially require breaking changes over with (and I am mostly talking about refactorings and technical debt) and do a major release afterwards.
And then after that we can talk about release cycles.

9

Thanks, I understand now.

However, I disagree: I’m pretty sure that we will find enough technical debt or refactorings which would lead us to say “we’re not ready yet for a real release cycle” for the next years. But we can’t afford doing that. Now that Neos is in the wild in significant projects, we are not as free with our changes as we used to be. It’s a compromise, but if we don’t commit to it, we will loose trust at our users.

1 Like
10

Sure I am aware of that and it’s a thin line to walk, so it’s just a question of where to draw the line. From a technical perspective I am pretty convinced we should take a bit more time especially as we already agreed last year that we need refactorings but then instead crammed more features in.

Sure that argument can be made all the time, but we also need to take care to be up to standards and ready for future extension to keep the users trust. I don’t think we are necessarily in a bad spot currently but I definitely draw the line in a different place right now.

11

I can only talk about Flow really, but I guess it’s not much different from Neos, so here’s my 5 cents:

Semantic Versioning?

Yes!

PostgreSQL paragraph?

Makes total sense, so yes!

So from my perspective we have been ignoring SemVer in the past.

Maybe not ignoring, but violating. The question is, whether those violations were reasonable. And yes, I argue that there are cases, where SemVer just can’t be held upon, without either holding back important bugfixes/improvements or introducing hackish code in an effort to stay 100% backwards compatible. Both are things I don’t consider healthy for the project as a whole. Again, there are cases were either is plausible.

I am saying that we are not in a state to talk about release cycles

But when is that point? I don’t think there’s a specific line of code or commit, which will mark the line of “ready for stable release cycle”, so IMO it makes no sense to wait or work towards that first. Start deciding on a release cycle you want to achieve and then work towards that, step by step. If you can’t fulfill one release cycle just yet and a release is pushed back, take it as is, try to get as close to the set release date as possible, if absolutely necessary even skip a release, but try to get the best out of it for making the next release hold better to the cycle.
From all I know, the road to success is steady improvement rather than instant perfection (Kaizen).

the definition of breaking, that is a hard nut to crack…

Indeed, and I would go a similar route with that: decide on strict rules (towards christians suggestion), but leave room to break the rule (ie. still add a supposedly “breaking” change to a minor version), when it makes sense, because otherwise the whole project would suffer. Those rule breaks need to be openly and clearly communicated on the release of course, but that is still better than holding back important fixes/improvements. Users can then at least decide themself if they want to do the minor version upgrade with a little bit of manual work, but giving them a big benefit from that one breaking change. If you’d strictly stay to the rules, the customer would have to wait for the next major release (with possibly even more breaking features, hence more work for him to upgrade), and that may not be an option. This in turn would lead to customers having to place their bet on own branches or use tools like beard, which I don’t think is something that neos/flow should depend on.

So as a summary:
IMO it makes sense to decide on strict rules and a plan early, but decide on a per case basis to break the rules/violate plan, if there are good enough reasons. And what is good enough reason is a team decision in the end.
Such breaks/violations are not bad per se, they are just neccesities to keep the project going forward within the given resources, so try to embrace them instead of trying to avoid them for dogmatic reasons. That’s basically exactly what the Postgre Paragraph is about after all.

tl;dr: Be strict with setting up your rules, but don’t be dogmatic about them. Start & deliver early, and improve steadily towards the set goal.

1 Like
12

Great to actually discuss these topics so we can reach a consensus for the project since there are clearly different incompatible views on these topics.

These topics are difficult and can influence each other and thus make sense discussing individually and together. I’ll try to add some points to the discussion to get a better view of the big picture.

Assumptions

Some assumptions are made in this post, would be good to back that up with some actual feedback from users on this matter. Are projects not using Neos due to a lack of SemVer or no defined release dates or fear of investment protection due to deprecated versions?

What are the problems with the way we’ve done it so far and are how big are they?

Definition of how we’re done it so far for Neos (as seen from my perspective)

Minor releases (1.1, 1.2) included a small/medium amount of breaking changes that wouldn’t require major changes in project code to upgrade and small/medium new features. Major release (2.0) required bigger changes in project code to upgrade and included large new features. Very important breaking bugfixes were included in patch versions. Everything has been based on “sentimental versioning”, where every breaking changes were taking into consideration.

API

Following SemVer requires a clearly defined API, which cannot be said for Flow and especially not for Neos.

What defines the public api? The @api annotation? Something being documented? Interfaces? Settings?
This needs a clear definition and be put into practice before we can even think about applying SemVer.

Flow
The @api annotation can work for most cases in Flow, but doesn’t cover things like configuration, policies and routing. Additionally the definition of the @api has never really been done thoroughly AFAIK.

Neos
Even fewer things are defined as API in Neos, and thus far if something was documented it became API. Which is not really a good practice. Since there’s no clear definition, we really don’t have an API we can use to follow SemVer with. That doesn’t mean we don’t have an API, but it’s hard to make rules about not breaking it.

In general SemVer works well for libraries with limited scope and a clear public API. However if you have a large surface area, which I’d definitely say about Neos, it becomes hard to follow.

What makes/justifies major releases?

A time schedule? Enough breaking changes? Bigger features requiring breaking changes? Marketable features?

Semantic versioning

One point about SemVer is that it’s based on breaking the public API. One important point is that you should not fear major releases, which works well for libraries, but poorly for things that need marketing. A new marketable feature might not require breaking changes, but unmarketable refactorings might. So either following SemVer puts constraints on the marketing or not following it puts constraints on SemVer. It is possible these things can work together, but it would be far from optimal and one would likely often weight more than the other.

It’s also highly likely than that the immature (API) we now have will either require more major releases or hinder progress.

Breaking changes

I see breaking changes as basically everything that can break project code, since we don’t have a clearly defined API.

Additionally automated migrations can also break project code, and thus I don’t agree with the given definition.

So far everything (in most cases due to different approaches) that potentially could affect end users have been marked as breaking. This have been assumption based though. To me this is actually very useful, because I can easily check the list of breaking changes to see if any of them apply to my project. Also helps if you’re using the development version of a branch.

As long as we don’t have a clearly defined API, I think this is our only sensible option.

Release cycle

What exactly are we solving by setting specific dates on releases? Plenty of projects don’t rely on that without any problems. It does put a lot of pressure on us as a project, which I really don’t think is worth it. Additionally it’s not agile in any way. If a release makes sense, then a release is due. I find it much more important that we strive for smaller releases and release more often than trying to stick to a predictable schedule.

If the problem it’s solving is investment security, then I’d argue we can solve it better by promising certain lifetimes for releases in terms of bugfixes and security fixes. This doesn’t put pressure on the releases, but secures investment. If you want to have new features you can upgrade if there’s a new version out.

Since predictability puts constraints on progress and quality, consider what has more value: predictability or progress and quality? Personally I prefer delivering a good product when it’s ready over a forced release according to a schedule.

When a new version is out then you can consider if you want to upgrade or not based on the estimated amount of work required depending on the release.

And basing a decision on our experience about half a year per release should be avoided since in reality those deadlines have primarily been met due to select individuals putting in unhealthy amounts of effort to finalize them due to broken promises on release dates. Which is why I don’t think we should set those dates. Rather be agile, make smaller scoped releases more often and improve our development and delivery process. Feel like we’re discussing Optimizing release development workflow for Neos/Flow here again.

My take

In general I see Neos in a maturing state without a clearly defined API. This doesn’t mean that it’s not perfectly usable, but following SemVer at this state doesn’t make much sense on my opinion. It still has broken or flawed concepts and even missing important ones, that require maturing and doesn’t have a clearly defined API. I’d suggest we keep following a more pragmatic and agile approach and adopt SemVer at a later state. This should be combined with trying to keep things backwards compatible and keep an easy migration path. I think our user base will be better off with that in the end.

Flow could adopt SemVer, but even there are quite some things that still need basic plumbing (just see the changes @christianm is pushing lately) and a clearly defined API. In Flows case I don’t see any good reason not to release major releases more often, since it’s not marketed and serves more as a library.

  • -1 for time scheduled releases
  • +1 for promising release lifetimes
  • -1 for SemVer for Neos (now)
  • (+1) for SemVer for Flow (now)
  • +1 sentimental versioning for Neos (now)
  • Release major when migration path becomes too difficult.
  • DoD: all changes potentially breaking project code are breaking until a clear API is defined

Ps. an interesting post and discussion regarding semantic versioning.

13

Thank you for posting your thoughts.

I think it’s most important to get this straight before we get to the conclusions. From my side of the table it has been always clear that in PHP only and exactly those methods and classes which are marked as @api are our public API. I have propagated, written and acted according to that throughout all the years (see also this blog post from 2011). I also assumed, until now, that this is basically the case, because if it’s not we must solve this as soon as possible.

The major reason why I started writing Flow and Neos was that TYPO3 did not have a clearly defined public API.

So, I don’t see it as negative like you do. Regarding the other parts:

  • PHP: I assume that we mostly defined the API with @api. If something is missing we need to fix it now.

  • configuration: all configuration provided by our packages is considered part of the public API

  • routes: that needs to defined, but in general there should be few interaction with third-party packages

  • JavaScript: no defined public API yet, so I consider that to be non-public at the moment

14

I agree to Aske, @api is not well defined in Neos. Too many things in there do not have @api that I would be hesitant to change.

Quick example \TYPO3\TYPO3CR\Domain\Service\Context::getDimensions()
I still guess it would be pretty bad to change it. I can probably dig up a whole lot more things…
Generally Neos (including Media, TypoScript, CR etc.) has a very wide open front that is in most places not @api, sure we can say “only use @api” but I think we all know from project experience that that’s not enough…

But of course I would like to make @api THE thing to follow in PHP, it’s just not there right now (I think).
Just as a short answer.

Re SemVer I am torn here. I see the cons to SemVer especially in Neos but still think that the concept has something to it.

15

I agree with this example. But it’s not that we don’t have an defined public API. I don’t think that we’ll find hundreds of places which are missing but rather dozens. And for these we simply create a list (next week?), add the @api tag and then we start to support it officially.

I don’t see that a couple of missing @api tags could challenge the whole concept.

Regarding this I suggest that someone comes up with a list of PHP methods which are not yet API but need to be. If that turns out to be more than hundred, we need to discuss that again.

16

Two quick answers to this:

I came up with discussing this whole topic of fixed release cycles mainly for the reason that I have been approached by customers (usually bigger corporations) and people I met at the PHP conference and Meet Neos with exactly that question. It usually goes along the lines of: “okay, we’re pretty convinced of the product, but how safe is our investment? Will the core team be there in 5 years from now? When I start this 400,000 € project with Neos today, how long can I use it until the next major release? Do you offer LTS? Do you offer SLAs?”

I am convinced that medium to large projects need the confidence of a version roadmap and LTS releases.

Secondly, we also need a plannable release cycle for development (see my original post). If you put a feature with a breaking change into the queue, today you simply can’t tell if it will be part of a release next month, next year or in three years. So either we will just keep releasing versions with breaking changes all the time in order to fit in bigger new features or bigger features simply won’t happen because a customer who might sponsor it can’t know when he can actually start using the feature.

1 Like
17

Thanks for the reasoning behind it, would be good to know if the community are also asking for the same. Haven’t seen or heard a anyone asking about it in the community other than occasionally without much backing, which is why I’d like to validate the importance of this. It’s important to weight this against other things we could focus on in the project at this point.

Do you think version roadmap and planned LTS releases is the only way or best way to solve that need? As said I think there are different ways to achieve that, which would work better for the project. Do you have a comment on that?

I agree we need to figure out how to deal with breaking changes, but having a distinct plan is agin not the only solution to that problem. A more agile approach would be to use RFCs before starting features that will be breaking and make a decision based on the scenario rather than a rule. If a feature can’t be justified making a breaking release, a feature branch can easily live until it can be merged and rebased periodically if preferred.

18

Flow
My perception of the current state is that it’s possible to find many areas that aren’t marked @api in Flow, which should be. The fact that there’s a big lack of documentation for many parts in general support that. There’s not even any PHP API documentation for Flow. I might be wrong or pessimistic here, but I think there are plenty parts of the core, which weren’t designed with a defined public API in mind. I fully support the effort to go through all the concepts on the core and ensure the API is clearly defined, however I think that’s a project and not just something you can do quickly.

I could imagine these steps to be taken before deciding on SemVer:

  1. Make clear rules about what defines the API
  2. Create an overview of the API or lack thereof (same for documentation)
  3. Decide on each missing or unfinished part if it should become public API
  4. Add/update the API for the parts (mention those that aren’t public as well)
  5. Add PHP API documentation (APIGen)
  6. Start following SemVer

Could easily be this year, if the necessary effort is put into it.

Otherwise my argument would be that we’re making a false promise, which is even worse than not following SemVer and claiming we do.

Neos
Same argument about false promise is even stronger for Neos, since there the situation is even worse. Following SemVer when the API is only a fraction of the surface area used in project. I cannot support to just say that the whole JS, FlowQuery, EEL, TS and Media parts are simply not public API to claim we follow SemVer, when in reality that is the API although not clearly defined.

Neos is simply not far enough in it’s development and wasn’t built with a public API in mind as I see it. What’s there now is often more random than a designed API. We’re planning quite some improvements to lay a good foundation for the future and to solve the missing general pieces.

We can apply same plan as suggested for Flow.

It’s natural that Flow is better suited for SemVer now, than Neos is. Give it a bit more time and we should get to the same state with Neos I’m sure, but it takes more work. Maybe a 4.0 can be the first SemVer version and the same for Neos? That I find realistic, but it doesn’t happen automatically unless we work towards that goal so we need to start now.

19

I agree with @aberl with a +1000, we need to try to be as sticky to semver as possible, since now for Flow and Neos. But we need, based on team discussion, to be breaking times to times. Don’t be dogmatic on this, just focus on stability, but keep the possibility to be breaking because we know that we need some serious refactoring and the we can not release a major version for every release. Breaking changes need to be documented clearly, this is a requirement for merging.

20

Just wanted to comment on the release cycle topic here from a non-technical/practical approach… a step back and look at it from the end an user’s point of view… oh wait, that’s me for the most part :slight_smile:

Really, Neos version numbers aren’t as important as seeing consistent accountability and quick time-to-market with features. At this point in the game Neos is still young and has a lot of competition out there. The last thing we need is for version numbers to hold back progress or get in the way… it should instead be used to help with productivity as everyone has consistent deadlines they are working toward… accountability. I don’t like deadlines any more than the next guy, but a very large percentage or people are not motivated to produce without one. Those on the outside of a core team may also have difficulty knowing if things are really getting done or if things are just being put off and put off because people are working on their own pet projects… yes I have my own too :slight_smile:

A major release doesn’t have to be feature packed for us to be happy. For a point of reference, we have all seen mature products like TYPO3 CMS hold things back forever that could be released much sooner and start the refining process, which slows progress. (To their credit though they have really been rocking it lately!)

The comment from @robert below is where I went, "finally someone is getting what I and possibly many other companies are looking for. I am not a medium or large company but maintaining like 50 websites is not a small investment when it comes to making major technology shifts/commitments. I do support and work with a number of medium businesses though and they trust me to make good long term decisions for them. Glad to hear you’re keeping your ear to the ground and listening to what some of your end users are most interested in.

TL;DR version:

  1. A properly implemented regular release cycle will have the effect of increasing productivity and accountability.
  2. See number point number one.
  3. Don’t let release numbers hold back progress. Whatever happens, I suspect it would be wise err on the side of rapid prototyping at this point in the game since Neos is still a fledgling product that needs some more feathers put on it.
3 Likes