Focal Point for Images

bastianschwarz · January 25, 2022, 9:19am

Hey NEOS Team,

I come to you in a great time of need … with a feature request and maybe a solution

Focal points for images are a really important feature to have especially in responsive websites with multiple aspect ratios for different devices. This feature is currently
missing from NEOS but - in the shortest way possible: We need this. And: We are willing to implement this into the NEOS core.

For this there’s this small introduction and some questions that need to be clarified first.

Why?

If you render images that are not uploaded to NEOS within a NEOS project there is currently no way to influence where the image will be cropped.
This can lead to “sub-optimally” cropped images especially with responsive images with multiple aspect ratios. You can work around this when the images
are sourced from NEOS by adding an additional property to your elements. One per aspect ratio. But this is not only inconvenient for the backend user
it’s also just not feasible if you have more than 2-3 aspect ratios.

As I see it it’s also currently not possible for external images (e.g. images you render using deep links let’s say from an HTTP API) as there is currently no way to get the focal
point information to the layer that crops the images.

Proposed solution

Instead of a “focal area” as it exists in TYPO3 we propose a simple “focal point”. This will cover the OVERWHELMING amount of use cases while cutting down
some complexity that is (in our experience) not needed for the web.

The focal point can be set in the media collection for each asset. The GUI therefore just needs a relatively simple “Click to set” event to generate X/Y coordinates,
a “cross-hair” on the image to visualize it and an “remove” button. If cropping is used in an inspector the focal point for that component is “discarded” since the crop
should take precedence. Of course a focal point in the inspector similar to cropping would be possible but this would increase the complexity tremendously and is probably
not needed.

The backend already generates a box for creating the variants. The assets need to be extended by the X/Y coordinates and the box needs to take those in account along with all the
other “don’t exceed the image dimensions” calculations that are being done. We did something similar for our very own imx.CMS:

github.com

infomaxwebsolutions/imxplatformphp.image/blob/master/src/main/php/de/imxnet/imxplatformphp/image/editor/croppingStrategy/ZoomOnFocalPoint.php

<?php
/*
 * Copyright 2016 Bastian Schwarz <bastian@codename-php.de>.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

/**
 * @namespace
 */

This file has been truncated. show original

As the PHPDoc for the above cropping strategy states it just enforces the aspect ratio while preserving as much of the original image and keeping the focal point visible. Or: The focal point
mostly decides the corner from where the cropping starts. This solves pretty much every use case.

We tried several other strategies but beside this one only one other made it into the final code and we never used it in production.

After all, not that complicated and not too far off of what NEOS is doing right now.

Timeframe

We have several projects with several customers that have asked specifically for this feature (it was available in their previous CMS after all). This means we would need this “as soon
as possible” (what a surprise, eh? ) The github issue on this has seen no further communication since 11/2018 https://github.com/neos/neos-development-collection/issues/2296

Questions and where to go from here

I remember reading about a “rework” of the asset manager. Is this still the case? If so, is something like focal points already planned?
Is my initial assumption correct that there’s no way to get the info to the cropping logic? As I see it the \Neos\Media\Domain\Model\ImageInterface is the breaking point since this is all
that’s available when cropping images.
Is this even a wanted feature? I would assume so since it’s really important for users to have and the issue hasn’t been closed (yet)
What would be the next steps? I assume there needs to be some kind of preparation/spec to be done so it’s probably not a good idea for us to just fork the repo and get going?

Feel free to reach out for questions. If it speeds things up we can also switch to German. You can also @ me on The NEOS Slack (@BastianSchwarz)

As I said: Our customers are already asking for this so it would be awesome if you could give me a first quick response rather soon so we at least know if/how we can proceed with this feature.

mficzel · January 28, 2022, 1:47pm

Hello Bastian,

currently the ui for the media package is rewritten. You can install the current state from here GitHub - Flowpack/media-ui: The development repository for the new Neos media management interface.

As far as I am aware of this does not affect features like focal point or area. So the topic is still available and we would like to see contributions for that. I personally would prefer a focal area to a focal point as i think that it the results of that would be easier to understand for editors.

I assume you would extend the Image and ImageVariant Model in Neos.Media probably with a FocalAreaTrait that both use.
The ThumbnailService would have to respect an existing focal point or area of Images. This would have to work for Imagine. I only found this example (GitHub - maagi/imagine-focalresizer: A tool for Imagine for PHP to resize an image around a focal point) that seems to be not very popular or actively maintained.
To actually set the focal area you would extend the ui of the media module to configure the focal point/area for images and the crop dialog to set one for image variants once selected.

The first two steps are not related to any ui but for the last one you will have to decide wether you want to use the old or the new media module. I assume that the major complexity is not in the ui but in the first two steps.

I would love to see this but it is a quite a bit of work. If you find a way we would be very happy for the contribution.

Regards Martin

creative-resort · February 1, 2022, 6:53am

I agree on the focal / safe area (instead of point) aspect.

lorenzulrich · February 1, 2022, 5:55pm

I was quite sure to have posted this before, but obviously not: A link to a discussion in Slack about this topic.

I think that the integration in TYPO3 is quite nice:

Source

mficzel · February 2, 2022, 8:48pm

How does the crop in typo3 work? Can the editor move the focus area or is the visualization of focus and cover areas a help for editors to define the correct crop.

If it is “just” a visual hint it could be added to the image crop dialog without adjustments to the actual image cropping.

lorenzulrich · February 2, 2022, 9:05pm

Cropping works exactly as in Neos, so you drag/resize a rectangle, optionally with a fixed asset ratio.
The focus area is a draggable/resizable area on top of the cropping rectangle, coordinates are saved relatively to the cropping area as far as I remember.
Covered area is a very useful helper feature to overlay a non-movable area that will be covered by “something”. Example: If you have an image that will always display a logo at a certain position, you define this position as a cover area to make sure editors don’t place the most relevant image information there.
As you can see, it is also possible to define crop variants, see documentation.

If it is useful, I can create a video showing the usage.

jiriki · April 21, 2023, 6:11pm

RFC: Solving the art direction problem of responsive images in Neos CMS

Since Martin Ficzel and I were interested in tackling the art direction problem for the Neos world, we discussed an approach in Slack [3]. We focused on the minimal problem and skipped additions like the covered area mentioned above. It would be great to brainstorm here and cast it into a concept so that it could be implemented by one or more developers at any time. I would be very interested in participating, but I am also a Neos newbie.

Concept

Neos MVC and UI

Add a new focal point property (x, y) to the image model.
Add a new focal point UI button next to the cropping button below the image property thumbnail.
On clicking the new UI button, the secondary inspector displays the full image (similar to the cropping procedure) and allows setting a focal point somewhere on the image, having x and y as form input fields as well.
Changes of the focal point or the input fields are reflected in each other.
Changes of the focal point to other than the default (50%, 50%) shows a focal point on the related image property thumbnail (similar to the cropping procedure).
Saving the secondary inspector saves the new focal point in the image object (similar to the cropping procedure).
The focal point property of the image object is considered in all cropping calculations for responsive images (e.g. by the Sitegeist.Kaleidoscope package).

Cropping calculations

Before:

The cropping area always set the focus to the center of the image.
The cropping area could not overlap the edge of the image. The maximum width and height were equal to the image width and height.

Now:

The cropping area sets the focus to the focal point of the image.
If the cropping area overlaps the image border, move the cropping area in x/y direction until it is completely over the image.

Questions

What is the benefit of defining a focal area instead of a focal point? From an algorithm point of view I do not see any, but maybe from an UX point of view?
Using plain Typescript or a special React package for the secondary inspector?
Where should the focal points be inserted in the Neos MVC?

Out of scope / Add-Ons

Enable global setting of the focal point per image in the Media UI, which serves as default for all image variants.
Automatic detection of the focal point could be implemented and replace the center-focused crop as default - but still be overwritten with the manually set focal point. See the smartcrop package [5] mentioned by Robert Lemke in GitHub [1] as a template (perhaps a similar PHP package already exists).
Provide the covered area feature as in TYPO3 (would be interesting if this is really used).

Sources

[1] Focus for image-based assets · Issue #2296 · neos/neos-development-collection · GitHub (concurrent discussion)
[2] Slack (first discussion in Slack, one year ago)
[3] Slack (second discussion in Slack, this year)
[4] Focal Point for Images - #4 by lorenzulrich (this page)
[5] GitHub - flownative/smartcrop: smartcrop finds good image crops for arbitrary crop sizes

mficzel · May 14, 2023, 4:23pm

After chewing on this topic further especially after discussing this with @jiriki i think this is really achievable with the following steps. I personally plan to work on Phase 1 (Backend) of this after the Neos 9.0 release but am happy to step aside if anyone is able to do this before. I dump this here as documentation in case i get distracted or have to adjust priorities down the road.

Regards Martin.

The main problem we had with the implementation was how to ensure that the focal point information gets handed down through the chain of ImageAdjustments that all may or may not modify the image dimensions. It would be right to have this handled by each adjustment on its own but that requires communication between different adjustments. The kind of trivial solution i just found was storing the focal point in the imagine->metadata which can then be read and adjusted by the following adjustments until the image is written.

The full plan would look like so:

Phase 1 - Backend:

Add “SetFocalPointImageAdjustment” to store the focal point in the imagine->metadata (see metaDataBag->offsetSet() )
Adjust all other ImageAdjustments to handle the focal point from imagine metadata
- The cropAdjustment has to adjust the focalPoint if it was in the copped area, otherwise it should remove it
- The resizeAdjustment has to calculate the target box, place it on the focal point and move it until it is fully above the image and set the focal point again on the resulting image
- All other adjustments that do not affect dimensions should make sure the focalPoint is present in the result meta data as in the input
Add focalPoint to the Image and ImageVariant Domain Model
Adjust Thumbnail service to set the focal point adjustment if a focal point was defined for the asset/variant.

Phase 2 - Neos UI:

Adjust ImageVariantEditor to allow to specify a focalPoint for a Crop
Adjust MediaUi to specify a focal point for an asset

Phase 3 - Tooling on top:

Tools like kaleidoscope should be able to read the focal point from the variant and calculate a proper value for object-fit-center from that to also make use of the focal point in css.