Could we build a version of Flow without doctrine orm / database requirement

Thanks for this, you really put some good research into this and that’s really awesome :slight_smile:

Could the object path mapping go to a cache instead of a entity?

The main issue there are the two queries inside the ObjectPathMappingRepository, findOneByObjectTypeUriPatternAndPathSegment and findOneByObjectTypeUriPatternAndIdentifier. Those would need to implemented through simple key-value lookups in order to be able to use e.g. the Cache framework for persistence.

Perhaps introduce a Database cache backend

There is already the PdoBackend Cache Backend, but that still only provides the normal Cache Backend interface of a key-value store.

Current state of affairs

Basically, the whole Persistence layer is already supposed to be pluggable with different implementations, and the original “generic” persistence implementation is still lying around, but pretty much unmaintained. So there is already some core principle to detach Flow from the massive Doctrine dependency in place. That of course would still leave you with the Persistence-Layer dependency.

So what I would see as a next step towards that goal of getting around the hard Doctrine dependency, would be to make the Doctrine implementation of the Persistence layer an own package, which could then be stripped from a custom (light) Distribution.

If the dependency to any Database is an issue, a Generic Persistence Layer backend could maybe be provided that works without a Database. BUT: That would mean we again have the huge effort to maintain a custom ORM layer and an underlying backend implementation.

Otherwise I guess the choice will always be: Want Security and/or ResourceManagent? Then you need the Persistence Layer (with a Database).

Yes, let’s slice up the elephant!

Just to split some hairs: Since a few versions, a database connection is no longer required by Flow.
But, yes, all the doctrine dependencies are.

Probably makes sense. Instead or in addition to that we should finally introduce some AccountInterface and code against that for the cases that need security (probably the large majority) but don’t want to use a db.

yes, could be done. But it got to be a persistent cache – you don’t want to lose those mappings if you flush the cache.
The lookup should be doable if we tag the cache entries.

I’d like to be able to use resources without a db. I mean in general it should be possible to store files just … in the filesysten (obviously you miss some of the metadata, but for the basics that should work IMO)

1 Like

Exactly, been seeing that comment inside the core. So yes, have a interface in the core and let developers implement that.

Can we create a project on github and try to collect issues without messing up the rest of the Flow issue tracker? Or should I just fork the flow collection/distributioni (?) repository and start collecting issues there ?

@aberl I was wondering if we could isolate the IdentityRoutePart to a own package aswell? Neos.Routing.IdentityRoute - but then I came across the DynamicRoutePart that requires the persistencemanager interface to get the identity of a object (used for ex. a “single view” route. So the splitting of the RoutePart classes might also be larger than expected, since everything excecpt static routes (controller/action based, no persisted object arguments) requires some sort of Persistence layer.

But of course we could use the Generic persistence implementation for that, as it already implements a generic way of getting that identifier. Or split the routing into different parts - you can still do routing with out having to pass around persisted object.

Which brings me to another topic: Are we touching other parts now aswell? TypeConverters, PropertyMapper ? Or are they good to leave and are not affected by these changes (more likely the other way around)

since everything excecpt static routes (controller/action based, no persisted object arguments) requires some sort of Persistence layer.

And Persistence(Manager)/entityManager is also part in a lot of other components, like MVC Views, PropertyMapping, Validation, Object Serialization, etc.

As much as I’d like to just separate a few components and make them optional to build a minimal Flow Distribution that works without Persistence, I think it’s a pretty huge effort and you either end up with many micro-components that need to be maintained, or a single big “Persistence component”, that contains component addons for pretty much every other component and feels like a full framework in itself.

Not trying to argue against it, I just want to show the amount of work that will be involved to reach the final goal. Of course starting to separate some components, like Security Account and Resource Management as you suggested would be a good thing to do in any case, even if it doesn’t fully resolve the dependency to a persistence layer.

As stated, getting rid of the Doctrine dependency is probably pretty doable. But for making Flow truly independent of a Database, I only see these two approaches:

  • factor out/rewrite all Parts that currently depend on the Persistence Layer, then make that Layer an optional part. A rewrite would surely be possible for Routing and Resource Management, but for example for Entity Privileges probably not so much.

  • write a Persistence Layer implementation that works without a database (ugh)

So, I took the time to collect a few weeks of thougt about this topic here, to create some sort of picture of the task

People with larger core knowledge, all comments are appreciated.

And if someone could give me a suggestion on how to organize a fork and a composer.json to read these new packages and similar (this is kind of the same thing when the FluidAdaptor was created - splitting up, and putting together), that would help me and avoid that I spend the first couple of days on that :slight_smile:

Sorry for not replying earlier. Hope you didn’t already spend the days to get the setup. If not, to setup your fork, it’s generally enough to add your own repository in the composer.json like this: https://getcomposer.org/doc/05-repositories.md#loading-a-package-from-a-vcs-repository

Basically just make flow-development-collection point to your own github fork. Afterwards any composer install/update will pull from there.

Anyway, thanks for the writeup! It’s highly appreciated! I’ll try to have a look and give more concrete feedback ASAP.

Thanks @aberl - I think I got it started :smiley:

And created a Neos.Security repository and added a bunch of issues

Two things I could really use some core developer insight on is the following (@aertmann @bwaidelich @christianm)

Perhaps I’m doing this wrong… Instead of creating a new giant package with the Security subcontext, would it be a better idea to just remove the entities and move that and database stuff to a new package.

that would also be easier for backward compatibilty

@sorenmalling Thank you for investing time and thought into this!

I see it as two different topics:

1. Extract the Security SubPackage

Following @christianm’s example we could create a new package neos/flow-security (or a similar name) and move the classes from Neos\Flow\Security there.
I’m always in favor of splitting up optional parts from the core. But in this case I’m not sure whether it’s feasible because security is so deeply integrated in many parts of Flow (but I might be wrong here!)

2. Make Authentication database independant

This is probably the low(er) hanging fruit here.
We should introduce a new AccountInterface and adjust all the places that currently refer to Account.
The existing Account entity could still stay in the core (for the start at least) – it doesn’t do any harm, does it?
At some point we could move all DB-related classes (PersistedUsernamePasswordProvider, Account, …) to a separate package, but that would require some B/C layer probably

BTW: I can offer to take care of replacing the Routing ObjectPathMapping implementation with a persistent cache

Thanks @bwaidelich

So, I might rethink stuff and create some smaller iteration.

First one being:

Introduce AccountInterface to Flow and make the Account entity implement that.

The interface should contain

  • accountIdentifier
  • authenticationProviderName (?)
  • credentialsSource (password, for sake simplicity and understanding)

Question: Supporting multiple AccountInterface implementations

Since now, the provider classes uses the generic AccountRepository. Could a second iteration make a “GenericUsernamePasswordProvider” class, that you can extend, and then simply set your own RepositoryClass as a property, and then “it just works”?

Just replying myself… Could a providerOption like “repository” or “implementaionClassName” or similar, be a great and simply solution. And have a default value to the Account class. that would also make it backward compatibile! <3

@sorenmalling If we introduce a new Interface I’d like to take the chance to use more ValueObjects from the start (only if it doesn’t produce a lot more work of course).
This is the interface I could imagine:

interface AccountInterface
{

  public function getIdentifier(): AccountIdentifier

  public function getProviderName(): AuthenticationProviderName

  public function getCredentialsSource(): CredentialsSource

  public function getRoles(): Roles

  public function hasRole(Role $role): bool

}

note: I renamed getAccountIdentifier() and getAuthenticationProviderName() to keep b/c with the current implementation (that doesn’t return ValueObjects)

And for the expiration-fields maybe a separate, more specific, interface along these lines:

interface ExpirableAccountInterface extends AccountInterface
{

    public function hasExpirationDate(): bool

    public function getExpirationDate(): ?\DateTimeInterface
}

note: At first I had the isActive() in that second interface, but IMO that logic should be moved out of the implementation

Not sure what to do with creationDate but all the mutating methods should be removed (set*()) and the whole failedAuthenticationCount / lastSuccessfulAuthenticationDate should never have been part of the entity in my opinion.

First we should think of reasons why you would want to have a different implementation. IMO we need to support some kind of “transient” Account implementation that is not persisted but will be returned from the provider. This will allow for db-less-authentication and for easier “augmented” authentication, i.E. using a 3rd party auth service.

I can’t think of a use case that would require a different DB-based account implementation

Could we define a CredentialsSourceInterface as well? I guess most projects would like to have individual control over validation here.

Not meant as a “different database” but meant as a different repository than AccountRepository - for example CustomerRepository if I treat my customer as someone with access to something.

That’s a great idea, especially in terms of validation and such.

meant as a different repository than AccountRepository - for example CustomerRepository if I treat my customer as someone with access to something.

But that promotes/opens the door for mixing up Account and User, which are two very different concepts. I always liked Flow making this explicit and hence forcing users to think about this (I myself had the two in a single mental concept until I learned better through Flow).

1 Like

Not when I look at it. A customer can be a Account of a system, and have a property (ex. the Email Address, customer number) that serves as a unique identifier (accountIdentifier). I’m not a fan of the decoupling today, because I have to take care of the account identifier 2 places, if I use the email address of a Customer as account identifier.

If you try to look at it a tiny little bit differently, you’ll realize that it is much easier to work with (at least in the Flow world).

You don’t need to “decouple” it. If you really have that 1:1 mapping between Customer and Account, just make it a 1:1 relation. For example like this:

/**
 * @Flow\Entity
 */
class User
{
    /**
     * @var Account
     * @ORM\OneToOne
     */
    protected $account;

    public function getEmailAddress()
    {
        return $this->account->getAccountIdentifier();
    }

    // ....

Wouldn’t that work for you?

Just a small example: we use signup via E-Mail, Google, Facebook, Twitter. So the identifier might be a social media ID or just the E-Mail. This also allows the user to connect multiple solutions (e-mail and facebook - because that gives more options like “what your friends liked” etc.) We also use the Neos.Party Package and have the E-Mail address saved there. Yes we need to see that they don’t split apart for the E-Mail Login but i really love how easy it was to implement with Flow.

1 Like

And the first commit for this is in review