RFC: PIM x Property Graph: EAV, Reference Types, Constraints, Neos 9.3 and all the good stuff

Hello, fellow Neosians,

as we are currently implementing a PIM based on the regular Neos UI, we’re considering a few additions to the CR that have been on my mind for some time now anyway.

The Concept

The challenge is to make the attribute schema of a product editable, both which attributes exist and what values they can have but without falling into the same traps that other systems did and model everything as their regular entities, which would be nodes in our case.

I’ll use the GQL/Cypher syntax for this:

(node {properties})
-- [reference {properties}] ->
(node {properties})

My idea ist to use our property graph model as follows:

(product)
-- [value {?value: ..., ?valueReference: ...}] ->
(attribute {?options: [...]})

Let’s have a look at general attributes in order of complexity:

Scalar values
e.g. length: 3.14m
It does not make sense to manage attribute options here, but we need to make sure we get a certain type of relation: it must declare a float value. Scope of the value can be NodeAggregate.

(product) 
-- [value:FloatQuantitativeValue {value: float}] ->
(length:Dimension {name: string, dimension: string, unit: string})

Product specific values in general
similar to scalar values, but with other types. Scope of the value can be anything, whatever makes sense.

Value lists
e.g. size (one of S,M,L). Managed as options on the attribute node. Translatable on the attribute node. Must be written to the reference as a valueReference, meaning it must have an id for referencing:

StringOption: {id: string, value: string}

(product)
-- [value {valueReference: optionId}] ->
(size:StringOptionProvider {name: string, options: StringOption[  ]})

Scope of the value can be NodeAggregate, scope of the option id must be NodeAggregate, scope of the option value can be anything.

Required extensions to Neos

References must be typed (similar to nodes) :
DistributionPackages/Acme.Catalog/ReferenceTypes/QuantitativeValue.yaml

Acme.Catalog:QuantitativeValue:
    properties:
       value:
           type: float
       minValue:
           type: float
       maxValue:
           type: float

Untyped references must either be migrated or fall back to something similar to unstructured (there should not be too many references with properties around yet)

Nodes must be able to limit how they can be referenced

'Acme.Catalog:Content.DimensionAttribute':
    [...]
    constraints:
        referenceTypes:
            '*': false
            'Acme.Catalog:QuantitativeValue': true

Nodes must be able to declare the type of references to be used:

'Acme.Catalog:Document.Product'
    references:
        attributes:
            constraints:
                referenceTypes:           
                    '*': false
                    'Acme.Catalog:QuantitativeValue': true
                    'Acme.Catalog:SimpleValue': true

We need the reference editor for this: WIP: FEATURE: New reference editor (DD Sprint) by mhsdesign · Pull Request #4021 · neos/neos-ui · GitHub

The reference editor must respect the reference constraints of the target node (similar to how the creation thingy respects the child constraints).

The reference editor must be able to evaluate the target node: If the target attribute has options S,M,L, those must be selectable by ID in a selectbox editor (could be UI configuration)

Target Version

As this should be implementable in a non-breaking way, target version would be 9.3

WDYT?

The model could improve a bit with moving the constraints to the reference types themselves:

Acme.Catalog:FloatQuantitativeValue:
    constraints:
        sourceNodeTypes:
            '*': false
            'Acme.Catalog:Document.Product': true
        targetNodeTypes:
            '*': false
            'Acme.Catalog:Content.FloatDimensionAttribute': true
    properties:
       value:
           type: float
       minValue:
           type: float
       maxValue:
           type: float
2 Likes

I totally like this approach.

Further Considerations:

Just constraining source and target node types may not be enough; I think we must constrain to combinations of both.

Given we have the following references:

# The default reference without properties
Neos.ContentRepository:Featureless: {}

Acme.Catalog:FloatQuantitativeValue:
  properties:
   value:
     type: float
   minValue:
     type: float
   maxValue:
     type: float

and the following node types:

Acme.Catalog:Content.FloatDimensionAttribute:
  superTypes:
    Acme.Catalog:Content.Attribute: true

Acme.Catalog:Document.Product:
  references:
    attributes:
      constraints:
        nodeTypes:
          '*': false
          'Acme.Catalog:Content.Attribute': true
    relatedProducts:
      constraints:
        nodeTypes:
          '*': false
          'Acme.Catalog:Document.Product': true

Acme.Catalog:Content.ProductComparison:
  references:
    # The products to compare
    products:
      constraints:
        nodeTypes:
          '*': false
          'Acme.Catalog:Document.Product': true
    # The attributes to compare above products in
    attributes:
      constraints:
        nodeTypes:
          '*': false
          'Acme.Catalog:Content.Attribute': true        

Then we must be able to enforce that

  • only FloatQuantitativeValue references may be used to connect Product and FloatDimensionAttribute
  • Featureless references may be used to connect ProductComparison to Product and Attribute

The strictest we can be in the current model is

Neos.ContentRepository:Featureless:
  constraints:
    sourceNodeTypes:
      '*': false # this may already cause problems
      'Acme.Catalog:Content.ProductComparison': true
      'Acme.Catalog:Document.Product': true
    targetNodeTypes:
      '*': false # this may already cause problems
      'Acme.Catalog:Content.Attribute': true
      'Acme.Catalog:Document.Product': true

Acme.Catalog:FloatQuantitativeValue:
  constraints:
      sourceNodeTypes:
        '*': false
        'Acme.Catalog:Document.Product': true
      targetNodeTypes:
        '*': false
        'Acme.Catalog:Content.FloatDimensionAttribute': true
  properties:
   value:
     type: float
   minValue:
     type: float
   maxValue:
     type: float

which allows us to connect product and attribute either via FloatQuantitativeValue or Featureless, with the latter unwanted. We cannot further constrain it though, because removing Product from its sourceNodeTypes would prevent us from referencing anything from products and removing Attribute from its targetNodeTypes would prevent us from referencing attributes in ProductComparison.

A more fine-grained solution could be

Neos.ContentRepository:Featureless:
  constraints:
    productComparison: # name of the use case
      sourceNodeTypes:        
        '*': false
        'Acme.Catalog:Content.ProductComparison': true
      targetNodeTypes:
        '*': false
        'Acme.Catalog:Content.Attribute': true
        'Acme.Catalog:Document.Product': true
    similarProducts:
      sourceNodeTypes:        
        '*': false
        'Acme.Catalog:Document.Product': true
      targetNodeTypes:
        '*': false
        'Acme.Catalog:Document.Product': true

Acme.Catalog:FloatQuantitativeValue:
  constraints:
    attribute:
      sourceNodeTypes:
        '*': false
        'Acme.Catalog:Document.Product': true
      targetNodeTypes:
        '*': false
        'Acme.Catalog:Content.FloatDimensionAttribute': true
  properties:
   value:
     type: float
   minValue:
     type: float
   maxValue:
     type: float

It might still be rather hard to keep Featureless off the lawn where it does not belong

OPGM Considerations

(as I will be using it for the project and it tends to bring a lot of clarity into things by disallowing any kind of array soup)

Attribute references

We cannot declare complete reference types as declared above as OPGM objects as they are resolved via ContentSubgraphInterface::getReferences() and ::getBackReferences, so they only ever contain the reference properties and either source or target node. Which is totally fine but needs a few decisions:

In the OPGM world, reference relations change meaning depending on perspective. Let’s consider the above example:

#[NodeTypeDeclaration]
public readonly class FloatValueDimension implements Attribute
{
    public function __construct(
        public string $name,
        public string $code,
        public string $unit,
    ) {
    }
}

public readonly class FloatAttributeValue implements AttributeValue, ReferenceRelation
{
    public function __construct(
        public ?float $value,
        public ?float $minValue,
        public ?float $maxValue,
        public FloatValueDimension $attribute,
    ) {
    }
}

/** @implements \IteratorAggregate<AttributeValue> */
public readonly class AttributeValues implements \IteratorAggregate
{
    [...]
}

#[NodeTypeDeclaration]
public readonly class Product
{
    public function __construct(
        public AttributeValues $attributeValues,
    ) {
    }

    public function getLength(): ?float
    {
        foreach ($this->attributeValues as $attributeValue) {
            if (
                $attributeValue instanceof FloatAttributeValue
                && $attributeValue->attribute->code === UNCEFACT::CODE_LENGTH
            ) {
                return $attributeValue->value;
            } 
        }

        return null;
    }
}

We can now see that FloatAttributeValue is directly related to FloatValueDimension and can derive that part of the reference type declaration from that class. What’s still missing is the back reference part; We know that products in general reference attribute values, but that’s not quite enought yet to make a general assumption.

If we consider the length attribute of type FloatValueDimension, the back reference has a different shape because we now want to know the related products instead of the attributes:

#[BackReferenceRelation(referenceRelation: FloatAttributeValue::class)]
public readonly class FloatProductValue implements ProductValue
{
    public function __construct(
        public ?float $value,
        public ?float $minValue,
        public ?float $maxValue,
        public Product $product,
    ) {
    }
}

/** @implements \IteratorAgggregate<FloatProductValue> */
public readonly class FloatProductValues implements \IteratorAggregate
{
    [...]
}

#[NodeTypeDeclaration]
public readonly class FloatValueDimension implements Attribute
{
    public function __construct(
        public string $name,
        public string $code,
        public string $unit,
        public FloatProductValues $productValues,
    ) {
    }
}

This way, we get both directions, have a type-safe connection between BackReference and Reference via the attribute and can constitute the complete ReferenceRelation declaration.

General references

If we want to implement our product comparison model, we can do so without declaring any kind of reference type because we don’t need any additional information (with Products and Attributes being simple typed collections):

#[NodeTypeDeclaration]
public readonly class ProductComparison implements Content
{
    public function __construct(
        #[ReferenceRelationDeclaration]
        public Products $products,
        #[ReferenceRelationDeclaration]
        public Attributes $attributes,
    ) {
    }
}

The relations between ProductComparison and Product or Attribute don’t need a type at all, which hints to the type of a reference being nullable if they don’t carry any properties. They are never resolved themselves, only the nodes are relevant.

Conclusion

This could be used to simplify our model: Neos.ContentRepository:Featureless does not exist and does not need to be constrained. If explicit reference types are available for source and target, the generic, empty reference is disallowed. We might get rid of the use cases as well by defining explicit back/reference types instead (traits 4tw).