We work on a generic package to have entity based JSON API:
https://github.com/ttreeagency/JsonApi
Adding CR support will be doable.
We work on a generic package to have entity based JSON API:
https://github.com/ttreeagency/JsonApi
Adding CR support will be doable.
Hi Stephan,
sorry I hadn’t got the time to get back to you about this earlier. Did you already start with a prototype for this approach?
I totally like the idea of using DTOs around Flow entities (and pure objects) to specify the JSON API mapping. We should definitely use a whitelist approach (when exposing properties) for security here.
But I think we need further discussion around how to process / create the actual response. I see that this is very performance critical and must be done in a pretty efficient way. Especially with nodes / entities forming a graph it can happen that a single request results in a lot of SQL queries on the server.
Caching responses in Flow is another story that might be an option, but my first thought is, that the request processing in Flow might be too slow if the response has lots of relations.
One of the advantages of specific @include@ and @fields@ requests is, that the server doesn’t need to process additional relations / fields because traversing relations (and deep serialization) is the major performance bottleneck here.
I’d suggest to have the least abstraction that we need here and start with pretty specific custom views to collect the needed data (and with a very slim controller). We had a good experience in another project with this lightweight approach. I’ll post something about that in another topic later. We could think about going to the HttpComponent level here, that might give us a performance benefit (especially with read-only operations).
Anyway, thanks for your input and ideas here, it’s perfectly fine if you just implement a prototype following your initial idea and we see how it works out.
@christopher Check this library https://github.com/neomerx/json-api/ my package is based on this one. Framework agnostic, and nicely done.
Indeed, looks good as the base to generate the JSON API response and pretty similar to what we developed here.
Regarding a futur public API for the CR (not only for Neos Services), i think it should be really nice to use the same API as Contentful (like Mattermost did, use the Slack API and improve where possible, but basically have the future CR API compatible with Contentful can be nice. Not sure if it’s doable.
I like this idea to not re-invent the API wheel where possible.
I’m not deep into the topic, so Just wondering, why do we choose to follow Contenful and not Prismic or Garhtercontent?
We don’t choose anything currently, it’s just an example. We have other options, like this really pragmatic article Your CMS REST API should be WordPress Compatible … we can even support multiple API.
I will write an RFC next week about this topic.
The idea to adapt / improve an existing API has come up yesterday too when discussing the JS concepts / Neos architecture. What we’ll have to balance out is an API that works efficiently for the UI (JS client) and is generic enough for all the other use-cases. I’d not like to support multiple APIs from the start to keep things simpler (formats are okay, but I mean fundamentally different).
Hey there.
I had quite some time now to think about it and give it a little try.
That’s basically what I came up with:
The example data package only contains a bunch of classes:
I think the best way to get my idea is to look at the example data package first and just keep the jsonapi.org document structure in mind.
Basically every document structure element is implemented as JsonSerializable, so exposing json_encode($resource) or json_encode($topLevel) is just enough for the read api.
What I described as DTO
in my post from dec. 22 just became the Resource
class described in jsonapi.org document structure. A Resource
, according to jsonapi.org, is the element having an identity, attributes and relations. So using that exaclt class as DTO seemd to be a good idea to me.
What I described as DtoConverter
in my post from december got a ResourceInformation
since it doesn’t only convert from Entity to Resource but it also provides endpoint information builds endpoint URIs.
Opposed to my former idea of using sub requests, and since I implemented a TypeConverter to convert the jsonapi.org resource array to a PHP Resource object by its type and id property anyway, I used that converter for creating the expected result of sub requests, too.
I didn’t completely drop the idea of having sub requests. But since my first attempt of using real curl based HTTP request for that was literally ten times slower than mapping the default setup uses mapping.
Regards,
Stephan.
Hey again.
I don’t want to amend my former post another time but keep it as a description what I did differently compared to my post from december.
This one is, in contrast, to explain what I did in general, since this came up currently. I think I need to go deeper into
I basically implemented every object defined in the document structure providing the actual schema structure and required/optional attributes as an individual class. No schema object is dynamically, implicitly constructed or by just knowing array keys . Of course when it comes to e.g. the Meta
object or the Links
this seems a bit overengineered, but dropping existing classes, using arrays instead and reducing the footprint can be done at any time I guess.
Every schema object in my implementation can json_encode() itself.
The actual question asked was: How does an exposed relationship know about its self
link and thus can create that URI in the “links” structure? More to that later.
TopLevel
object.The most important result parts of the TopLevel object is data
which represents the business content of a particular request and optional field links
, that can contain the currently request URI as well as e.g. next and last to allow pagination. So the TopLevel more or less represents meta structure of a request. It does not directly contain any payload data.
The pretty awesome third important but unfortunately optional (in terms of "may not even be implemented on client side or server side) feature of the TopLevel object is the included
array.
The TopLevel
is the desired response type for listAction
, showAction
and showRelatedAction
, so basically every action that is meant to result not meta data or structures but business data.
The data
field of a TopLevel can either be a single Resource
in case a single result is expected (e.g. from the showAction
. But no matter if the TopLevel
object returns an array or a single one, a Response
object is always constructed the same way.
A resource
object brings its id
and type
to be identified uniquely, as well as attributes
, links
and relationships
.
This means: The resource
needs to know about a single business data object, which makes it the level I choose as DTO.
The attributes
property of a Resource
object is an Attributes
object.
According to the jsonapi.org specification, this one is only a key => value
store.
BUT: I implemented it as an Attributes
class knowing its Resource
object. So every getOffset()
and setOffset()
call (in terms of \ArrayAccess
) is internally passed to the Resource
, which makes the Attributes
object not contain any payload data itself but only a mapper object for creating the jsonapi.org hierarchy.
The relationships
property of a Resource
object is an Relationships
object.
Just like the Attributes
object, the Relationships
object does not contain any payload data but just acts as a wrapper to access its parent Resource
object.
When being json_encoded, its result is not plain key => value
but a rather simple object exposing a self
link (links to the metadata of the relationship), a related
link (links to the business data being meant to be exposed by that relationship) and optional data
which embeds the id
and type
of one or more linked data.
The Relationships
response is used to provided the output data for the showRelationshipsAction
. Not the whole thing is exposed but $resource->relationships[$relationshipName]
falling into the \ArrayAccess getter.
I added a class called ResourceInformation
.
It’s one and only purpose is: Know a single implementation of the abstract class ResourceInformation
to handle and which combination of Package, Subpackage, Controller, Action and Arguments have to be used to create a public URI for that resource type (the self
link of that Resource
class) or the public related URI for relationships of that resource type (the related
link of a relationship of that Resource
class).
This means:
ResourceInformation
by using an UriBuilder
with proper arguments.self
links of Resources
is initiated right inside of the Resource
class, considering the payload of the Resource
.self
links and related
links of Relationships
are initiated right inside of the Relationships
, considering both, the payload of the parent Resource
class as well as the relationship name.There is obviously room for improvement.
The NodeResourceInformation
I added to my ExampleData package shows the fact that the ResourceInformation
object is the one thing to know controller argument names. If a controller does not use custom argument names but sticks to $resource
just like the abstract ApiEndpoint
starts with, I could think about a GenericResourceInformation
object getting its mapping from a Settings.yaml instead of object attributes.
My JsonApiOrg
package brings an abstract ApiController
that only reroutes indexAction()
calls to other actions, depending on available request arguments and the given request method.
I intended to use a distinct controller class for every Resource
Type handled through this API.
First reason: This allows for security constraints through Policy.yaml
based on resource types.
Second reason: Meaningful routes are easily set up.
Every Resource
of the jsonapi.org schema always comes with its type
attribute. If a single controller is implemented to take every abstract Resource
instead of a distinct extension of it, there is no mechanism in place that prevents e.g. Account
objects to be added where Node
objects are meant to arrive. The first situation such a misuse of an API endpoint catches and hopefully fails would be the NodeRepository
, hence the persistence. But making such a thing only based on the framework internals throw proper type mismatch exceptions doesn’t seem right to me.
So I strongly suggest to create distinct, routable, policy-protectable and “one concern only” endpoints.
I tried to keep my ExampleData as short as possible.
Domain\Model\Node
would be the business model, so that’s not a requirement for the jsonapi.org structure but the business data one would expose.Controller\NodeController
only takes JsonApiOrg schema data and implements every GET, POST, PUT and DELETE request the object type should support.Domain\Model\Dto\NodeResourceInformation
points to that NodeController
Domain\Model\Dto\NodeResource
provides attribute names and relationship names to be exposed by the API.Objects.yaml
, the internal class Netlogix\JsonApiOrg\ExampleData\Domain\Model\Node
is always mapped to the string nodes
when exposing the type
property of the Resource
according to the the jsonapi.org schema.Routes.yaml
, all routes target the indexAction.I’d be glad to receive your feedback, especially since this is kind of a completely different approach to that jsonapi.org idea than @dfeyer implemented.
Regards,
Stephan.
Hello folks,
i’ll try to put my own two cents as understandable as possible.
By reading the posts here didn’t get “aggregate root” keyword.
So i’m thinking on Flow api highly dependent on jsonapi.org with Flow magic for aggregate roots.
So as perfect solution for Flow developers i see defining action controllers for agregate roots only. The JsonApiAggregateRootControllers generate JSON-Api compatible JSON.
So non-aggragate Entities MUST be contained in the JSON data (NOT in “relationships”) in associative way. Using PATCH to manipulate properties in the aggregate tree by URL
PATCH /blog/flow-is-avesome/post/45 HTTP/1.1
Content-Type: application/vnd.api+json
Accept: application/vnd.api+json
{
"data": {
"type": "post",
"id": "45",
"attributes": {
"title": "DDD in JSON API",
"content": "………………………………………………………………………………………"
}
}
}
or as PATH in the JSON
PATCH /blog/flow-is-avesome HTTP/1.1
Content-Type: application/vnd.api+json
Accept: application/vnd.api+json
{
"data": {
"type": "blog",
"id": "flow-is-avesome",
"attributes": {
"posts": {
45: {
"title": "DDD in JSON API",
"content": "………………………………………………………………………………………"
}
}
}
}
}
Relationships, which are aggragates MUST be contained in “relationships” and CRUD manipulations follow Json-API standard.
Routing: Defining routing only for Aggregates.
Decision about Read/Write accessing could be configurable but using setter/getter/add/remove/etc. from Domain Model classes should be default.
So i think on AbstractAggregateRootApiController with some service classes, which provides JSON-Api functionalities with possibility to override magic methods like in repository.
I hope you got my idea.
If we can spend sometimes during the next code sprint to discuss all those API topics, could be nice to write a first RFC Draft, and continue the discussion based on that.
Hi folks and sorry for digging out this old thread.
As a little disclaimer: The reason to use HTML as Hypermedia format originated in an idea of Jon Moore (one of the Hypermedia gurus out there).
I still find the idea strikingly persuasive but I also see some drawbacks at least with todays constraints.
In the meantime I played a lot around with different APIs and had the chance to test different media types in complex scenarios.
Recently I had a close look at JSON API
. I had high expectations that this would finally end the “API format wars” given it’s written by some very clever REST experts.
I have to admit: I didn’t get along with it very well (and I really tried!).
To cut a long story short: My favorite is currently still HAL as it seems to hit a good balance between pragmatic and over-sophisticated.
Regarding hypermedia controls: Yes, HAL is missing a notion of HTTP method for links. But so is JSON API. And I don’t think it’s important – at least not until clients get so smart that they really only need the base URL of the API and explore all of their logic from there.
And there are extensions allowing for specifying forms including method and payload definition if that’s needed.
Having written all this I think that our main issue & challenge is to find a good abstraction level for our APIs.
I’d especially suggest not to fall into the CRUD-trap that will lead to high coupling and very complex code (as we already experience today).
I liked the idea of combining REST APIs with a CQRS-kind-of-style, described in these slides: http://de.slideshare.net/fatmuemoo/cqrs-api-v2
A (very simplified) API for the CR could look a bit like this:
{
"_links": {
"self": {
"href": "/api/nodes"
},
"curies": [
{
"href": "https://neos.io/rels/{rel}",
"name": "neos",
"templated": true
}
],
"neos:create": {
"href": "/api/nodes",
"rel": "command",
"title": "Create new node"
}
},
"_embedded": {
"neos:node": [
{
"id": "36a63168-2042-8ef7-3787-4a215f4dff75",
"type": "Some:Node.Type",
"properties": {
"text": "Some Text"
},
"_links": {
"neos:hide": {
"href": "/api/nodes/36a63168-2042-8ef7-3787-4a215f4dff75/hide",
"rel": "command",
"title": "Hide this node"
},
"neos:move": {
"href": "/api/nodes/36a63168-2042-8ef7-3787-4a215f4dff75/move",
"rel": "command",
"title": "Move this node"
}
}
}
]
}
}
The nice thing with this is that we could just omit the “command links” on the server if the current user hasn’t got the required privileges. The client / UI wouldn’t have to know about those, it would only show the “Hide”-Button if the neos:hide
relation is found…
@bwaidelich hey, what do you think of the API written by @wbehncke for the new Neos UI? I think it’s similar to CQRSish example from your last example. And it goes well together with our Redux-driven client.
And yes, this kind of command api is cool for data manipulation. For data querying we use client-side implementation of FlowQuery. It’s easy to grasp, flexible and familiar to Neos developers.
https://github.com/PackageFactory/PackageFactory.Guevara/tree/master/Resources/Private/JavaScript/API
Is there a wrap-up somewhere that shows how this would be used by clients? The link only leads to some command classes as far as I can tell
For querying data REST APIs used to work quite well already. Using some DSL (like FlowQuery) makes sense for complex things IMO and as long as that’s encapsulated in some query parameters I don’t see a problem with this, e.g. /api/nodes?filter=<some-flow-query>
Ah, sorry, I don’t think there are any docs for this yet, but here’s an example of how to create a node: https://github.com/PackageFactory/PackageFactory.Guevara/blob/master/Resources/Private/JavaScript/Host/Containers/AddNodeModal/index.js#L57
Basically, you have a stream of change objects, each change has a type, subject (contextPath of a node it’s acting upon) and a payload.
You POST arrays of change objects to API endpoint and for such request:
{
"changes": [
{
"type": "PackageFactory.Guevara:Create",
"subject": "/sites/neosdemotypo3org@user-dimaip;language=en_US",
"payload": {
"nodeType": "TYPO3.Neos.NodeTypes:Page",
"initialProperties": {
"title": "test"
}
}
}
]
}
you get this response:
{
"timestamp": "2016-03-16T14:51:37+0300",
"feedbacks": [
{
"type": "PackageFactory.Guevara:Info",
"description": "Information available",
"payload": {
"message": "1 change(s) successfully applied.",
"severity": "INFO"
}
}
]
}
Don’t know if what I write to you makes sense. If not, perhaps @wbehncke may provide a better explanation.
Hi all
@dimaip Thanks for the example, it’s much clearer for me now - And sorry for the lack of feedback.
In the meantime I deep-dived into this topic (including GraphQL and graph databases) and started to write a (very long) response. But I realized this would be better discussed in person (or at least in sync).
There are quite a few people currently working on related topics and we think it’s time to stick our heads together.
@andi started a doodle poll for such a meeting and it would be great to have you guys join it!
Everyone is welcome to join the meeting, but please fill out the poll so we can find other channels in case there are too many for a google hangout
Hi all,
for everyone who could not attend the hangout. Basti worte a short summary and we decided to collect our API and format ideas, maybe even small prototypes to eventually come to a general architecture/standard/format decision: RFC: Neos API - next steps
Would be great to add your proposals/findings there.
Greets Andi