Funding request: Support an index per language and Elasticsearch 6/7

The Neos Project Finances & Fundraising
1

With ~70k installations, the Flowpack.ElasticSearch.ContentRepositoryAdaptor is one of the most common used 3rd party packages for Neos. It is not only used for implementing search, it is also crucial to retrieve CR Nodes on large instances.

The current package version supports Elasticsearch Version 5.6 only. This version of Elasticsearch is already EOL since 3/2019 (https://www.elastic.co/de/support/eol). So an update to support ES 6+ is required.

What is the purpose of this project?

To support Elasticsearch 6+, a bigger refactoring will be necessary, as Elastic dropped the support for types starting from ES6. I outlined the steps to refactor to an explicit type field here as a replacement.

Second, there is a PR from Dominique Feyer to let the package create one index per ContentDimension. This is great, as it allows to configure the index language-specific. The PR needs to be rebased, some adjustments are necessary, bugs needs to be fixed and tests need to be adjusted.

I would like to get that PR merged before refactoring to typeless indexes to not make it un-mergeable (again).

Although maintaining the package is normally doable in my free time or while implementing projects. I fear, these two changes are too big to fit in soon.

When is it considered done?

How much money are you asking for?

I guess 30-40 hours will get it done-done.

5 Likes
2

Hey there,

I want to give you a brief update what has been done so far.

I took over https://github.com/Flowpack/Flowpack.ElasticSearch.ContentRepositoryAdaptor/pull/314 from @dfeyer, rebased it onto master, did some cleanup and refactoring, made it testable and got the tests green.

I also did some performance testing and optimizations and managed to bring the indexing time down from 20 minutes to 6 minutes in my test project (where it took 10 minutes with the released version 6) - you find detailed statistics within the PR.

Some of you already use the branch of Dominique in projects, as far as I know. It would be great If you find the time to have a look into my PR for reviewing and testing.

I will now start to work on the changes towards ES 6 / 7 compatibility upon this branch.

Cheers,
Daniel

2 Likes
3

Hey Daniel

Thank a lot for you work so far, I am glad this fork makes it into the package as we rely in quite a few projects on it.

I would love to help testing the result. At the moment this is not so easy because we dont have a Neos 5 project so far and that will not change untill Neos 5 LTS is released. We have a strict LTS policy in our company (for good reasons).

Again, thank you for picking up this fork.

Peter

4

After working on it for nearly a year now, I’m thrilled to announce you today, that the new major versions of the Neos Elasticsearch packages finally have been released!

Among the support for Elasticsearch 6 and 7 they bring a ton of new features and improvements. I wrote a blog post which gives you an overview of what has been changed: https://www.neos.io/blog/elasticsearch-package-releases.html

Released Packages:

I love to hear your thoughts and feedback on these changes and also some of your use cases that may bring the packages even further with next versions.

4 Likes
5

Great work, Daniel. Thanks a lot for your efforts!

1 Like