Antivirus: Recommendation

Hello together,

as you all are probably also handling assets within neos or might have editors that want to upload media using the Media Browser, i’m wondering how you treat the AV topic.

I haven’t found any reference/knowledge article or comment about this so far. But i’m (almost) certain that some must have implemented this already.

I had thoughts about adding clam-av as sidecar container to my neos cms container that just scans a given path in a regular schedule. Anyway, better would be something like a upload-hook that scans the file directly after upload or in best case a sandbox(API?) that first uploads the file into a encapsulated container, scans the file and returns it as a positive/negative result to neos.

I’m without any clue what is possible within neos and also what that could mean to the running container/instance when suddenly a file is being deleted.

Any feedback/knowledge or reference would be great :slight_smile:

Greetings!

Hello together,

i want to bring that topic back and i’m looking for some information/recommendation to make sure i’m doing the right thing. I’ve developed a container based clam-av image and a Go application that accepts files, and returns an HTTP header (200 if the file is okay, 409 is something is infected)

Now within my neos image, i’m running a kinda filewatcher in python that sends new uploaded files/and existing files during installation/import of a neos project to that service.

I currently did the following:

  • File is infected: Python script deletes the file, re-creates the file with the original name but instead any binary code, there is just a text within the file that says “infected”.
  • Fils is not infected: Have a nap

Questions:

  • Is it correct and enough to scan only /app/Data/Persistent/Resources?
  • It seems like that files are constantly generated whenever i open a neos projekt in the frontend. I supposed those are images/thumbnails. So i’m not sure how good that approach is e.g. as i just want to scan the files a editor is uploading.

Thanks!

The a really good tool to check only uploaded files would be a psr7-middleware. That way you could check and reject / replace / return status 422 (unprocessable entity) or whatever you like. See HTTP Foundation — Flow Framework 8.3.x documentation

Data/Persistent/Resources holds all resources Neos knows about which often are thumbnails for images or other stuff. Scanning all this would only make sense to also find infected files that were uploaded in the past.

Also one can configure resource storage in other locations aswell so in very rare cases you may even have to include another folder. See Resource Management — Flow Framework 8.3.x documentation

1 Like

Thanks for your input @mficzel!

So it might not be the very best way to monitor that path :sweat:

I’m curious, as the middleware would be a very interesting solution, at least it sounds good. Right now i feel totally clueless about how and where to start with this.

I feel like i kinda understood, that for simply starting is having those two parts mentioned in the documentation:

=> middleware, which implements the MiddlewareInterface & an entry within the Settings.yaml.

However, in the documentation there is no simple “where do i even put that piece of code” to work with. I suppose the Settings.yaml is clear, but right now i just can guess, that the middleware is probably located in /Packages/Application/Xx.Xx

Do you have recommendations on how to dig deeper into this?

Usually you would create a custom package for that and put the configuration and the class in there. If you already have a custom site package that would also be a good place for that

To create a new package you can use the following flow command:

./flow package:create Namespace.PackageName

by default new package will be created in and installed from the DistributionPackages folder wich is part of your distributions git repository.

Regards Martin

1 Like

Good Morning @mficzel,

i’ve had the time to read a bit more within the documentation and at least i’ve managed to create a own package, using the example code with the header response, which is working.

However, as i was rereading your comment i’ve seen that i’ve implemented a psr-15, instead your mentioned psr-7. As i’m brand new with that psr terminology i’ve searched more on that page and found, that this part seems to be where i should focus on, right?

As there is not that much i can find/read or unterstand here, i’m now trying on how to proceed in:

  • How to detect whenever a new file is about to be uploaded (to send it first to the anti-virus service)
  • Is there a difference between new media ui/old media ui?

Are you maybe aware of any examples that used an equal method before?

Everything is helpful in my current state, thanks a lot! :slight_smile:

Sorry for causing confusion. PSR 15 is correct as this defines the Middleware Interface. PSR-15 is built on top of PSR-7 that defines the Request and Response interfaces which is why my mind always mixes those PSRs up.

Any file upload in an incoming http-request is newly uploaded. That is why this is such a nice place for filtering.

Not regarding filtering. The handling of Errors may deviate and from a ui perspective a rejected upload is an error that has to be handled. You will have to test that but i expect nothing drastic here.

Good Morning @mficzel,

i’m currently not sure if i’m doing the steps into the right direction. I found GuzzleHttp but i’m not sure if that client is recommended, as there was nothing to read about in the neos documentation.

So far i’m testing a bit around with:

<?php

namespace Vendor\Intercept\Http;

use Psr\Http\Message\ResponseInterface;
use Psr\Http\Message\ServerRequestInterface;
use Psr\Http\Server\MiddlewareInterface;
use Psr\Http\Server\RequestHandlerInterface;
use GuzzleHttp\Client;
use GuzzleHttp\Exception\RequestException;

/**
 * A sample HTTP middleware that adds a custom header to the response
 */
final class InterceptionMiddleware implements MiddlewareInterface {

        public function process(ServerRequestInterface $request, RequestHandlerInterface $next): ResponseInterface
        {
            // Check the the request type
            if ($request->getMethod() === 'POST' && !empty($request->getUploadedFiles())) {
                
                $uploadedFiles = $request->getUploadedFiles();

                $response = $next->handle($request);
                $requestMethod = $request->getMethod();

                return new \GuzzleHttp\Psr7\Response(500);
            }

            if ($request->getMethod() === 'HEAD') {

                $response = $next->handle($request);
                $requestMethod = $request->getMethod();
                return $response->withAddedHeader('REQUEST', $requestMethod);
            }

            // If no file upload or no interception needed, proceed with the request
            return $next->handle($request);
        }
}

Where there is an actual response when i upload an image to the (old) Media Mangement Module. To test, i’ve set the HTTP response to 500, but the upload bar within the Module goes until 100% and then responds with:

image

When i refresh the page, the file is uploaded :confused:

Also i wonder about $request->getMethod() === 'POST' && !empty($request->getUploadedFiles()

Do i intercept every POST request to the whole neos? Can i limit that somehow to only the Media Module / Backend, or maybe less preferable: An URI?

You are detecting that the request is post and has uploaded files … so far everything is well.

Then you pass the request to the handler in $response = $next->handle($request); and afterwards return a 500 response instead of what $next returned.

Is suggest to actually check the uploadedFiles and only pass the request to next if this was fine. If the file check failed returning the 500 is fine but you should not let the request be processed in this case.

I actually have no clue what you intend with the second if that checks for “HEAD” requests but the last line is fine again and will allow all requests without files be processed normally.

1 Like

Hey @mficzel,

thanks a lot! I’m now a step ahead (i hope) and at least the file upload is interrupted.

I actually have no clue what you intend with the second if that checks for “HEAD” requests but the last line is fine again and will allow all requests without files be processed normally.

That was just test where i wanted to check with curl if “something” works :smiley:

Currently my code looks like this:

<?php

namespace Vendor\Intercept\Http;

use Psr\Http\Message\ResponseInterface;
use Psr\Http\Message\ServerRequestInterface;
use Psr\Http\Server\MiddlewareInterface;
use Psr\Http\Server\RequestHandlerInterface;

/*
use GuzzleHttp\Client;
use GuzzleHttp\Exception\RequestException;
*/
use Psr\Log\LoggerInterface;

/**
 * A sample HTTP middleware that adds a custom header to the response
 */
final class InterceptionMiddleware implements MiddlewareInterface {

    public function injectLogger(LoggerInterface $logger)
    {
        $this->logger = $logger;
    }

    public function process(ServerRequestInterface $request, RequestHandlerInterface $next): ResponseInterface
    {
        // Check the the request type
        if ($request->getMethod() === 'POST' && !empty($request->getUploadedFiles())) {

            $this->logger->debug('av - post request with non empty uploaded files');
            
            parse_str($request->getUri()->getQuery(), $queryArguments);
           
            if (!isset($queryArguments['upload'])) {

                $this->logger->debug('av - query argument is upload');

                $uploadedFiles = $request->getUploadedFiles();

                foreach ($uploadedFiles as $uploadedFile) {
                        $this->logger->debug('av - started the iteration');
                        $fileContents = $uploadedFile->getStream()->getContents();
                        $this->logger->debug('av - the iteration worked');
                }

            }
            $this->logger->debug('av - whatever happens here');
            return new Response(200, ['Content-Type' => 'application/json'], json_encode(['success' => true]));
        }

        $this->logger->debug('av - nothing to do');
        return $next->handle($request);
    }
}

I’m going step by step, as this is the very first time i’m investing time into that new topic. At first i want to have the file, and later think about what i do with it, and how i can send it to an API or Service that scans the file (no clue about that yet actually :smiley:)

But it seems like currently i fail using the getStream() method. Because when i’m using the above code, read the System_Development.log i do see the following:

tail -f System_Development.log | grep av

24-03-21 12:58:50 19922      DEBUG                          av - post request with non empty uploaded files
24-03-21 12:58:50 19922      DEBUG                          av - query argument is upload
24-03-21 12:58:50 19922      DEBUG                          av - started the iteration
24-03-21 12:58:55 19922      DEBUG                          av - nothing to do

And also, not using grep the following exception is thrown:

Exception in line 74 of /app/Data/Temporary/Development/SubContextDocker/Cache/Code/Flow_Object_Classes/Neos_Flow_Http_Middleware_MiddlewaresChain.php: Call to a member function getStream() on array - See also: 20240321125850318b25.txt

So again i’m stuck, and i’m trying to find out how to use the methods of Psr\Http\Message\UploadedFileInterface, if that’s even the right approach.

I found out that this is because the data model of the old media ui:

(
    [asset] => Array
        (
            [resource] => GuzzleHttp\Psr7\UploadedFile Object
                (
                    [clientFilename:GuzzleHttp\Psr7\UploadedFile:private] => stock.jpg
                    [clientMediaType:GuzzleHttp\Psr7\UploadedFile:private] => image/jpeg
                    [error:GuzzleHttp\Psr7\UploadedFile:private] => 0
                    [file:GuzzleHttp\Psr7\UploadedFile:private] => /tmp/phpz7SL3b
                    [moved:GuzzleHttp\Psr7\UploadedFile:private] =>
                    [size:GuzzleHttp\Psr7\UploadedFile:private] => 38986
                    [stream:GuzzleHttp\Psr7\UploadedFile:private] =>
                )
        )
)

Where using the new one, i directly can access $file->getStream();

(
    [clientFilename:GuzzleHttp\Psr7\UploadedFile:private] => stock.jpg
    [clientMediaType:GuzzleHttp\Psr7\UploadedFile:private] => image/jpeg
    [error:GuzzleHttp\Psr7\UploadedFile:private] => 0
    [file:GuzzleHttp\Psr7\UploadedFile:private] => /tmp/phpnH3DF2
    [moved:GuzzleHttp\Psr7\UploadedFile:private] =>
    [size:GuzzleHttp\Psr7\UploadedFile:private] => 38986
    [stream:GuzzleHttp\Psr7\UploadedFile:private] =>
)

So i quite need to make an difference between those two e.g. do something like

$uploadedFile = $file['resource'];
to access the file using $uploadedFile->getStream();

I’m now having “something” that uses the virustotal.com API. Still a bit hacky and far away from clean. However, it seems to work even if the upload takes quite round about 20-30 seconds, as it has to wait until there is a result from virustotal using (status → completed) :thinking: