Creating Files Best practice

Hi All

Sorry to ask again but I am wondering what is the best practice for creating & updating files from the back end e.g. if I wanted a back up CSV list from an enquiry form. I am wondering if I should be using flow Resource and ResourceManager of something else?

Many thanks

nikos

Hi Nick

You should definitely use the Resource API for such things. See http://flowframework.readthedocs.org/en/stable/TheDefinitiveGuide/PartIII/ResourceManagement.html

Hi Aske
Thanks for the link I have read through it however I am left wondering what is the “right” way to create a new Resource file. In the documentation it refers to creating new resource files by importing or uploading a file resource. It may be a silly question but how do you create a new file Resource from scratch ? Sorry to be such a noob!

That would be importing. You import the data string for this file.

@christianm thanks have tried that but still left scratching my head.

I have set up a storage (usingWritableFileSystemStorage), collection and target in my settings.yaml

Now I want to create the CSV Resource file if it doesn’t exist in a form finisher and append to it if it does exist.

If I use
$cvsResource = $this->resourceManager->importResourceFromContent($csvContent , ,$cvsPathAndFilename, CSVResourceFinisher::CSV_COLLECTION_NAME);

A new file is created but I am not sure how and if I can append to it.

I have tried getting the content using :smile:

$cvsContent = Files::getFileContents(‘resource://’.$cvsResource);

but $cvsContent is empty. when I try the following as in documentation:
$cvsContent = file_get_contents(‘resource://’.$cvsResource);
I get an exception that no resource exists.

Even if I got the content how would I then update it back in the same file?

Sorry for my noob confusion :confused:

Hi all, I am just wondering on my issue with
$cvsContent = Files::getFileContents(‘resource://’.$cvsResource);

is it related to this?
https://jira.neos.io/browse/FLOW-302

But also are Resources intended for what i am trying to do i.e. create a csv file which is created on a forms first submission and appended to on subsequent submissions ?

Looking behind the code I can’t see how

$csvContent = Files::getFileContents(‘resource://’.$cvsResource);

would work at all. Given that the path to the resource is made up of directories from the first 4 chars from the $sha1Hash of the file name - as created by the FileStorageSystem. So I guess the documentation is wrong, but I am not sure how it would work anyways.

So I am left wondering if using Resource framework is appropriate for my use case where I want an editable file. It seems to me that that Resource framework is for write once only ‘resources’.

I am interested to know if I’ve come to the right conclusion and if the is another way which would be good Neos practice.

I am also unsure about the best practice. In my case it’s for uploading and rendering files in the media collection.

Problem:
When I tried to update a media file, the new version got a new URI. The previous URI stayed valid and everybody following external references will still receive the outdated version.

Possible Solution?
Document nodes?: May implement “attaching files to nodes” but leveraging the media collection. They may render a link (like current reference nodes?) named like the document or an icon/preview, and provide a constant SEO-URI that is mapped to the selected file in the media collection (the currently published version) .

Or have a general mapping of of file names to file versions?

In any case, after uploading a new file to a document node or re-uploading a new version of a file in the media collection the link should point to the newly uploaded media file. Keeping a version history, allowing to delete or re-publishing old versions could be a nice bonus.

The advantage of document nodes is their possibility of configurable rendering, and the explicit relation to parent or other nodes.
Document nodes allow to ask if orphaned files that are not referenced any more should also be deleted, when a document node is deleted.

Related: Is it possible to filter the media collection for files that are not referenced any more?

Actually, the “asset list” content element may already be seen as some sort of multi-document node, however, uploading a new file version created a new file and not a new version, and the rendered link changes when selecting the newly uploaded file version in the list element.

I don’t think so. The resource management defines a resource by it’s content. Different content, different resource. So you cannot append to an existing resource. It would be a new one then.
Need to check about the Files::getFileContents('resource://'.$cvsResource); example as that used to work and should probably still work. Ignore the file paths, we rewrite everything internally. The resource:// is a stream wrapper that will take the resource id given and transform it to a proper file

Hash based naming is very good for the backend side, I guess we all agree this is desired for the storage de-duplication and cache management.

At the same time, on the request side, permanent (file name based SEO) URIs are just as desireable for any non-html file type. It provides consistency and allows to append content and always deliver the most up to date content.

A nice solution I saw elsewere was to base the URI on the file name plus a query that refers to the file content.

Example for a URI to the current version:

  • domain.tld/dir/sites-statistics.ods?Thu_02-12-2015_10:00:01h

If a later request still matches the current version, tell the browser its cache is still valid, else rewrite to current URI.
If the request is done without the ?* part, always return the latest version.

Example for a URI to a specific version:

  • domain.tld/dir/sites-statistics~Thu_02-12-2015_10:00:01h.ods

The problem with such approaches is that they require processing power to get the file. We wanted to have the files deliverable directly via webserver, but it’s true. we are working on replacing a resource by another and maybe there will be a solution for the stable URL problem as well…

Oh, you are also targeting scriptless serving (webserver-only) operation. Very nice!

I think apache can serve files with question marks in the name, if you create the file cache in the webroot.

I experimented with scriptless serving using the Zena CMS. That CMS keeps the cache in the webroot. For sites that do only change occasionally (when edited), I could perfectly limit the loading of the application to non existing files.

For full support of scriptless serving, the CMS would have to warm the cache with pages (files) as well as symlinks or redirects to the content repository files where it makes sense, and schedule cache expiration events (cron, runwhen, or own daemon process).

When I experimented with manual deleting of cached files after the occasional update, I found it is important to expire the cache only selectively, because apaches browser cache management seems to work based on file modification date. When I deleted too much, the identically recreated files were transfered again instead of just confirming the browser cache.

Another aspect of scriptless serving is the format and language selection. Apache supports this by its “mulitviews” completion.

Automaitic completion works nicely for URIs that end in: .format.language-code

The automatic completion may be disabled in the .htaccess for URIs starting with special language code directories ( /en/, /de/ etc.). See http://www.fellerich.lu/articles/multiviews for “multilingual site without PHP”.

Overall pattern:
domain.tld/{optional lang code}/path/filename{~optional timestamp}.{auto completed format}.{autocompleted lang}?{timestamp}

The CMS defaults to render the links without the optional and autocompleted parts, but the user may also choose to link to a specific version in the asset list content element.

Edit: Maybe the final ?{timestamp} isn’t even necessary these days any more, if there is a (http header?) mechanism by with browser and apache can coordinate their cache/refresh operation.