Simple search and packages-repository

mad · April 21, 2016, 2:28pm

Hi,
my next «basic-question»:

For big enterprise solutions, elastic search is quite important, I guess. But is there also an easy search solution for small/medium Sites?

I found «Search» on Github from Lelesys (https://github.com/lelesys/Lelesys.Plugin.IndexedSearch). But there are some big important notes:
###- … was done when TYPO3 Neos was at alpha3/4
###- This package does database query for search result and not indexed yet.

Questions:

Is there a package for simple search «out in the box» in Neos. If, how I can activate or find it in the demo?
Is the packages above good for the 2.1.6 version? Does anyone have experience with it?
Does I have to Index the Site? And How? Sounds all in all difficult … Is there some easy understandable doc out there?

Packages in general:

Is there a central point to find Neos-Packages and reviews? (Under https://packagist.org/ the finding is quite difficult -> for IndexedSearch Lelesys no match. Google/Github is by chance and without helping reviews)

Tanks for help
Martin

daniellienert · April 21, 2016, 3:35pm

There is a kind of “drop in replacement” for elasticSearch which you can find here: GitHub - Flowpack/Flowpack.SimpleSearch: A simple php/sqlite search engine for generic data. / GitHub - Flowpack/Flowpack.SimpleSearch.ContentRepositoryAdaptor: Adaptor for the Neos CR, based on the SimpleSearch package

Packagist is a good start. Updates, Downloads and Stars in the corresponding github repository gives you an idea how “good” that package might be.
We are working on something only neos-related. Stay tuned

Cheers
Daniel

mad · April 21, 2016, 9:24pm

Sounds great! Will try it next week.

Ingenious thing: clean, powerful and easy to grasp. Thanks so far for Neos, all work, documentation and the no less important tools to extend and specialize it …

… and both suggestions
Martin

fgrell · April 22, 2016, 8:46am

The problem with the SimpleSearch Implementation is, that it’s based on SQLite, which locks the database while updating, so you are unable to perform searches while the index is building. Depending on your site this can take a while.
I had SimpleSearch installed but switched to Elasticsearch since then (which is not without it’s own challenges ).

stolle · April 23, 2016, 8:46pm

Check out the brand new Package Repository at Neos CMS Extension packages & plugins - Search all packages available in the Neos universe - Neos.io

mad · April 24, 2016, 1:57pm

Great. Thats really helpfull!
Maybe something on the Pagination is broken, but with the search, all works nice.

_{Although, if I click on the page 2, 3, …the Get-parameters [current page] = changes, but the packages list seems to remain the same, also the highlighted number in the pagination-overview. It looks like the get-Params are damn long.

But I don’t know enough so far about Pagination-Issues in Neos.

I’m quite sure it will be in work to change this behavoir.}

stolle · April 25, 2016, 4:42pm

That is fixed with https://github.com/neos/Neos.MarketPlace/commit/06506e21965739de88e57ff4423bf7a70b8beae9 and will be available as soon as https://github.com/neos/Neos.NeosIo/pull/98 is merged

mad · April 27, 2016, 4:31pm

Hey,
after installing Flowpack.SimpleSearch and Flowpack.SimpleSearch.ContentRepositoryAdaptor the flow command

./flow nodeindex:build
Workspace "live" without dimensions done. (Indexed 37 nodes)
Workspace "user-martin" without dimensions done. (Indexed 38 nodes)
Finished indexing.

So it seems to work. I know, that’s not really much stuff to search trough . But for beginning not to bad I think/hope.
###But
I have to lern a lot more stuff, I think. Because I can’t figure out what I should do now.
Maybe someone has good links to learn more about:

Flowpack.SimpleSearch
Using SQLite instead of MySql
do some simple query in Neos

Would be great

stolle · April 27, 2016, 4:33pm

@mad i can recommend the documentation the packages provide. https://github.com/Flowpack/Flowpack.ElasticSearch.ContentRepositoryAdaptor/blob/master/README.md provides a bunch of examples. If you can use Elasticsearch i’d suggest to go for it!

mad · April 28, 2016, 2:31pm

Hi again,
thank you, I go through the readme.

So I could install Flowpack.SimpleSearch and it works. Looks good.
I guess, Elastic Search is a really interesting tool. Currently, the less steep learning curve of simple search seduce, I think.
But steep enough for beginners like me: I have to lern at least in two regions:

##First :
Unfortunately I’m quite unskilled in database queries, especially with sqlite. I read here and there and have downloaded an sqliteBrowser.
But (shame on me) I have not even found the sqlite file. I read, sqlite packed all stuff in one file. Where can I find this? I searched on the hole harddrive. Only other matches for *.sqlite. Maybe because the file.extension could be all kind of (so I’ve read), and look trough sourcecode I can’t find the clou …

Perhaps it is an absolutely stupid question. Nevertheless, maybe someone can help me. Where is the associated mysql-file for simple search?

Second:

I found the Prototype for searchResults under
«Flowpack.SimpleSearch.ContentRepositoryAdaptor/Resources/Private/TypoScript/Root.ts2»
I guess I could overwrite this for adapt the search-query for my need, because the search-Query seams to match to exact.

In Database is a node with «Wasserballon» as content.
If I search in formfield for:

Wasserballon -> MATCH
Wasser -> no-match
Wasser* -> MATCH
*asserballon -> no-match
ball -> no-match
-*ball* -> no-match

I tried to change the prototype to ->
searchResults = ${Search.query(site).nodeType('TYPO3.Neos:Document').log().fulltext("%"+request.arguments.search.word).execute()+"%"}-> NO MATCH
With escaped “%”+…"%" or “*” and also “\*”-> NO MATCH

Maybe sql statements have to be different with other wildcards than MySql. Or I have misunderstood the prototype or typoscript-string-concatenation, or something else.

But search for «ball» with fulltext(“Wasser”+request.arguments.search.word+“on”) -> MATCH. So i guess, concat is working…
###I can’t find a working solution:
Did someone have good links or direkt information about
overwrite the Search Prototype to match less exact ?
more information about TypoScript and Prototype in Neos like https://learn-neos.com/blog/hitchhikers-guide-to-typoscript-2-part1.html and part 2 and inAction 1+2 ?
build queries in sqlite ?

stolle · April 28, 2016, 2:48pm

Hey @mad,

github.com

Flowpack/Flowpack.SimpleSearch/blob/master/Configuration/Objects.yaml#L10


      
          Flowpack\SimpleSearch\Factory\IndexFactory:
            scope: singleton
          
          Flowpack\SimpleSearch\Domain\Service\IndexInterface:
            className: Flowpack\SimpleSearch\Domain\Service\SqLiteIndex
          
          Flowpack\SimpleSearch\Domain\Service\SqLiteIndex:
            arguments:
              2:
                value: '%FLOW_PATH_DATA%Persistent/Flowpack_SimpleSearch_SqLite/'
          
          Flowpack\SimpleSearch\Domain\Service\MysqlIndex:
            arguments:
              2:
                value: 'mysql:host=127.0.0.1;dbname=<set-db-name>;charset=utf8mb4'
            properties:
              username:
                value: '<set-user-name>'
              password:
                value: '<set-password>'

To your second question:
First of all you don’t create custom sql queries. Instead you use the ${Search.query(site). This search query wraps to this class:
https://github.com/Flowpack/Flowpack.SimpleSearch/blob/master/Classes/Flowpack/SimpleSearch/Search/SqLiteQueryBuilder.php
You can check there what fulltext() does. Maybe a like() would be the way to go for you? You could of couse write another Implementation and provide some custom functions.

stolle · April 28, 2016, 2:50pm

You would override this setting with your custom class:

github.com

Flowpack/Flowpack.SimpleSearch/blob/master/Configuration/Objects.yaml#L5


      
          Flowpack\SimpleSearch\Factory\IndexFactory:
            scope: singleton
          
          Flowpack\SimpleSearch\Domain\Service\IndexInterface:
            className: Flowpack\SimpleSearch\Domain\Service\SqLiteIndex
          
          Flowpack\SimpleSearch\Domain\Service\SqLiteIndex:
            arguments:
              2:
                value: '%FLOW_PATH_DATA%Persistent/Flowpack_SimpleSearch_SqLite/'
          
          Flowpack\SimpleSearch\Domain\Service\MysqlIndex:
            arguments:
              2:
                value: 'mysql:host=127.0.0.1;dbname=<set-db-name>;charset=utf8mb4'

Make sure to implement IndexInterface. The SqLiteIndex should give you a hint how to build queries