Usage of AI coding agents in Neos project

Creating Neos & Flow Code
1

Hi everyone

As some of you know, visol (the agency I work for) and Internezzo are hosting a Neos Sprint in April. We assume that by now we’re not the only ones using AI coding agents such as Claude Code. Overall, we think that these agents are very powerful tools that - if used correctly (whatever that means in an ever-changing world) - enable us to deliver good code.

However, these tools tend to work better and use less resources (financial and environmental impact) if they know “how to behave” and have the necessary context. In our projects, we try to achieve this by having the proper documentation (e.g. CLAUDE.md, context, agent instructions etc.).

In our opinion, it would make sense for the Neos project to develop and provide such documentation so not everyone working on the Neos project needs their AI tools to “find out about Neos” from scratch.

In practice, this means that the project could contain at least an agent-agnostic markdown file. Furthermore, we could think about how the Neos coding convention, patterns and best practises would be best documented so agents deliver better results in less iterations.

What is your opinion about the topic? We’re aware that the AI coding topic is constantly evolving, and therefore what is the correct way today might not be the best way going forward. But we think that this shouldn’t stop us to start with “something” and constantly adapt.

3 Likes
2

Thats a topic I thought about a lot since I started using Claude Code because it has a big overlap with “normal” LLM applications like Agent-Orchestration and RAG/Graph-Pipelines.

I think there is not only one Question to answere but a few.

  • What should when be part of the context?

  • How does the context get built?

  • Who is responsible for the context being built?

In theory the LLM should have everything in its context to know everything about Neos Flow, Neos CMS and the Vendor Site.

Knowledge and thereby context of the Vendor Site has to be provided by the User.

That leaves Neos Flow and Neos CMS.

So what about Neos (Framework/CMS) should when be put in the context.

What ?

My personal opion about that is to see it in 3 parts:

  1. The Neos landscape:

    1. Split between Framework and CMS

    2. Package folder stuff

    3. Composer stuff like neos-package etc.

    4. Neos 8 vs. 9 CR stuff

    5. There is that small thing called Fusion

    6. Hier könnte Ihre Werbung stehen

  2. Technical overview

    1. Technical hierarchy/list of installed Neos packages, what they are and why they are there.

    2. Coding guidelines

  3. Detailed Technical Information about installed Packages (Packages/{Framework,Application}, DistributionPackages/)

    1. Package configuration

    2. Reference

      1. Controller

      2. Models

      3. etc.

    3. Fusion ?

When ?

The Neos Landscape MUST be always in the context. Means it has to come via a Skill or CLAUDE.md/AGENTS.md

Detailed Technical Information SHOULD be loaded on demand as it can be very large. Which means Tools/MCP.

Technical overview Lays in between. The Coding guidelines MUST, the Technical list of packages SHOULD be loaded into the context directly. So again via a Skill or CLAUDE.md/AGENTS.md

Implementation

CLAUDE.md offers the ability to include other files using @ which is great because that means the User can create a CLAUDE.md for its own project and just include the other ones.

AGENTS.md do not support file includes IIRC and Claude Code does not use AGENTS.md files…

Looking into the future and hoping that all might change I propose this setup for that whole .md stuff:

vendor-website/CLAUDE.md
        @Packages/Framework/Neos.Replicant/AGENTS.md

vendor-website/Packages/Framework/Neos.Replicant/AGENTS.md
        @Documentation/FlowAndNeosOverview.md
        @Documentation/FlowComposerShenanigans.md
        @Documentation/NeosCR.md
        @Documentation/FusionOverview.md
        @Documentation/CodingGuidelines.md

        @auto_generated/packages_overview.md

        <!— @reference/neos_neos.md —>
        <!— @reference/neos_flow.md —>

vendor-website/Packages/Framework/Neos.Replicant/Skills/Flow-MVC.md
        ## TBD: define Flow MVC Skill

So there would be a new Package (Neos.Replicant) which provides the documention and some generated information and maybe Skills and maybe even the MCP/Tools.

I think it’s important to have one entry point for the most important stuff and then load the rest on-demand. One thing to look at is https://context7.com in this regard.

3

One thing first. Neos probably still lacks documentation and ever has. There is never enough good documentation. Now the interesting part is that with AI we finally have a motivation to fix that because we like AI to suffer less than we did?:smiley: No im joking, i think its a great motivation. As Neos and Flow is strongly based on conventions we luckily have a lot of answers already that are either written down someplace or just part of our mindset. Now if we were to formulate that all and write that down everyone and everything who reads this would understand how to produce “correct” code? In that case i would see that as a huge plus not only for the machines but for every one learning or using Neos.

But if you have been to the discussions about code-style and the like you also know that we often are unsure about a single solution and make up many decisions as we go. Especially for core development you need to adapt to our current code style which depends on which part one is working on and is still and ever in evolution. So after PSR12 there is no real code style we follow but we continue to evolve and never standardise. This is also mostly as PHP itself is continuously evolving leading always for everyone to learn new things. So a PR review probably always points out a thing or two which can be done better. So im unsure how we can archive such a document and still keep it up to date. We should probably try to use more linting rules though it will also be hard to tell a linter to say: This should be a value object in this design.

Also we discussed in the past often times if we are happy with our documentation in docs.neos or rather prefer them in the source code. So far some generated and manual created references for fusion prototypes and node type editors still live in rst but documentation on what the Neos CR and how to use it is on docs.neos. I prefer documentation in the code and AI seems so to;)

So i mean by that if we create meaning full documentation we would also let humans profit directly and it should probably live directly in the Neos or Flow codebase. For generated documentation and highly AI specific information id also prefer Neos.Replicant as i would take that input with a grain of salt and not for human use. What do you think does it make sense to try to improve general documentation of Neos as well?

1 Like
4

Ah maybe my answere was a bit to fast and unprecise :smiley:

When looking at the context for such things it is possible to cluster them in two parts:

  • soft facts

    • unlikely to change

    • general ideas and concepts

    • remarks on how to get hard context

  • hard facts

    • things like documentation

    • (code) references

    • the code itself

    • exact coding styles

So the soft facts can be written once and are okay beeing updated once in a while. Appling that to this probem, means that everything in Neos.Replicant/Documentation is considered a soft fact.

As the LLM “reads” differently than we do (it does not read at all) the text in these soft facts should be dense and high on “context-forming-words”.

That means it is not meant to be read by humans.

That also means things like coding guidelines are not the same we might be used to, as it shows the LLM how to code for Neos and not where and how to put braces.

CodingGuidelines.md

## Attributes
Always add:
    use Neos\Flow\Annotations as Flow;

Always use PHP native attributes instead of doc-block. Only use doc-block if the user says so.

### Injection
Neos Flow provides dependency injection.

Always write it as:
    #[Flow\Inject()]
	protected MyServiceInterface $myServiceInterface

## Configuration
Create configuration files ONLY in the `Configuration` Folder.

The only allowed configuration FileNamePrefixes are: 
- Settings
- Objects
- Routes
 
When creating a new Configuration File you should add the package as part of the filename:
- Settings.Neos.Flow.yaml
- Settings.Neos.Neos.yaml

FlowAndNeosOverview.md

Neos constists of two parts:
1. Neos.Flow: the framework
2. Neos.Neos: the CMS which just a Neos.Flow application

Neos.Flow first processes the PHP Classes to implement dependency injection, AOP and more. 

etc....

Looking at the hard facts are a lot of possibilities how the LLM might know about things.

Be it using context7 to look at the current Neos reference, Neos could create its own MCP server for that or using the Neos.Replicant package to generate a file for each package that the LLM is instructed to look at before doing stuff.

As the LLM does not really differentiate between “real” code and text it is sufficient to generate (hard fact) some files it so it can get an overview what exists and only look at the code if it has to:

auto_generated/neos-flow.md

## Classes 
Neos\Flow\Log\PsrLoggerFactory:
	description: lorem ipsum
	file: Packages/Framework/Neos.Flow/Classes/Log/PsrLoggerFactory.php
	methods: 
		public function __construct: Line 35 - Line 38
		public function get: Line 48 - Line 67

Again it must not be easily human readable but dense in information with some kind of structure.

The hard facts are a bit into the future as i think we should start with some soft facts and the whole package infrastructure. Except maybe the package_overview.md. That seems quite helpful having it from the start.

The Neos documentation needs a bit of love in every sense :smiley: from better tutorials to implementation examples ending with use-cases-based-documentation.

Correctly structured that will help the LLMs immensely as that would mean we have “perfect fusion code” and reference material.

But the pipeline “Developer documentation” → [Magic] → “LLMs know how to write working fusion code” is tricky to say the least :smiley:

1 Like
5

I created a quick example repo if anyone wants to try it out GitHub - sjsone/SJS.Replicant · GitHub

1 Like