Convert / import database data to NEOS nodes

Community Newcomer's Corner
1

Hi there,

I’m currently create a new NEOS page which should work without flow models and bigger database structures. I only want to create nodes for it, so far as I can.

Does anyone has a hint or link for me to show, how I can convert existing Database data (NOT NEOS) to a Neos node?

As Example:
I have in a foreign database the table “xyz” with columns “title”, “description” & “crdate” and I want to create a NEOS document node from it, which should get this “title”, “description” and “crdate”.

I see there is the table “neos_contentrepository_domain_model_nodedata” which hold the nodes basically. But it seems not the right way, to push content directly into.

Can someone gives me hint, for what I’m searching for?

2

Hi Maximilian

Is is correct that all nodes are stored in the nodedata table. However, you don’t want to insert data directly using SQL, as we have quite some processed around nodes (e.g. cache invalidation) that need to be triggered.

You can find a little (but German) tutorial here.

And there are at least to packages that help you do that:

I don’t know the first and I’m using the latter, which doesn’t mean that it’s better - I just didn’t get around testing the first.

I did a presentation of the Content Repository Importer at a Neos Online Meetup this year, you can check it out here (from ca. minute 142).

1 Like
3

Hi Maximilian,

did nearly the same few weeks ago.

The task was to create pages in neos under a specified node and add some nodetypes to this elements. First I converted the data i had (excel file) to an php array and than used a foreach to iterate through the data.

$count=0;

foreach($jayParsedAry as $rezeptur){

	// Excel Datei bei Google Sheets importieren
	// als CSV Datei speichern
	// https://csvjson.com/csv2json CSV TO JSON bei Options nur array anhaken alles andere aus
	// https://jsontophp.com/ JSON nach PHP Array

	$context = $this->contextFactory->create();
	$nodeIdentifier='3ea6344f-5233-4951-ac1e-23ab5330886f';
	$parentNode = $context->getNodeByIdentifier($nodeIdentifier);

	// Seite anlegen

	$nodeTemplate = new \Neos\ContentRepository\Domain\Model\NodeTemplate();
	$nodeTemplate->setNodeType($this->nodeTypeManager->getNodeType('Example.Site:Document.Produkt'));
	$nodeTemplate->setProperty('title',$this->replaceAsterisk($rezeptur['Artikel Bezeichnung']));

	$newNode = $parentNode->createNodeFromTemplate($nodeTemplate);
	$this->output->outputLine($newNode->getIdentifier());

	// ContentCollection der neuen Seite

	$contentCollection = $newNode->getPrimaryChildNode();


	// Produkt-Slider anlegen
	$nodeTemplate = new \Neos\ContentRepository\Domain\Model\NodeTemplate();
	$nodeTemplate->setNodeType($this->nodeTypeManager->getNodeType('Example.Site:Content.ProductSlider'));
	$nodeTemplate->setProperty('title',$this->replaceAsterisk($rezeptur['Artikel Bezeichnung']));
	$nodeTemplate->setProperty('subtitle','Art-Nr '.$this->replaceAsterisk($rezeptur['Artikel Nr.']));
	$nodeProduktSlider = $contentCollection->createNodeFromTemplate($nodeTemplate);
	$produktSliderCollection = $nodeProduktSlider->getPrimaryChildNode();

	// Produkt-Slider Item anlegen
	$nodeTemplate = new \Neos\ContentRepository\Domain\Model\NodeTemplate();
	$nodeTemplate->setNodeType($this->nodeTypeManager->getNodeType('Example.Site:Content.ProductSlider.Item'));
	$produktSliderCollection->createNodeFromTemplate($nodeTemplate);

	// Anker-Links anlegen
	$nodeTemplate = new \Neos\ContentRepository\Domain\Model\NodeTemplate();
	$nodeTemplate->setNodeType($this->nodeTypeManager->getNodeType('Example.Site:Content.AnkerLinks'));
	$nodeAnkerLinks = $contentCollection->createNodeFromTemplate($nodeTemplate);
	$ankerLinksContentCollection = $nodeAnkerLinks->getPrimaryChildNode();


	// Anker-Links Items anlegen
	$nodeTemplate = new \Neos\ContentRepository\Domain\Model\NodeTemplate();
	$nodeTemplate->setNodeType($this->nodeTypeManager->getNodeType('Example.Site:Content.AnkerLinks.Item'));
	$nodeTemplate->setProperty('title','Produktdetails');
	$nodeTemplate->setProperty('anker','produktdetails');
	$ankerLinksContentCollection->createNodeFromTemplate($nodeTemplate);

	$nodeTemplate = new \Neos\ContentRepository\Domain\Model\NodeTemplate();
	$nodeTemplate->setNodeType($this->nodeTypeManager->getNodeType('Example.Site:Content.AnkerLinks.Item'));
	$nodeTemplate->setProperty('title','Downloads');
	$nodeTemplate->setProperty('anker','downloads');
	$ankerLinksContentCollection->createNodeFromTemplate($nodeTemplate);

	$nodeTemplate = new \Neos\ContentRepository\Domain\Model\NodeTemplate();
	$nodeTemplate->setNodeType($this->nodeTypeManager->getNodeType('Example.Site:Content.AnkerLinks.Item'));
	$nodeTemplate->setProperty('title','Ansprechpartner');
	$nodeTemplate->setProperty('anker','as');
	$ankerLinksContentCollection->createNodeFromTemplate($nodeTemplate);

	$nodeTemplate = new \Neos\ContentRepository\Domain\Model\NodeTemplate();
	$nodeTemplate->setNodeType($this->nodeTypeManager->getNodeType('Example.Site:Content.AnkerLinks.Item'));
	$nodeTemplate->setProperty('title','Kundenbereich');
	$nodeTemplate->setProperty('title','Kundenbereich');
	$nodeTemplate->setProperty('anker','login');
	$ankerLinksContentCollection->createNodeFromTemplate($nodeTemplate);

	// Text (SUB-HEADLINE) anlegen, class=subtitle-text
	$nodeTemplate = new \Neos\ContentRepository\Domain\Model\NodeTemplate();
	$nodeTemplate->setNodeType($this->nodeTypeManager->getNodeType('Example.Site:Content.Text'));

	$nodeTemplate->setProperty('stil','sub-title-text');
	$nodeTemplate->setProperty('ausdehnung','col-lg-12');
	$nodeTemplate->setProperty('ausrichtung_rahmen','');
	$nodeTemplate->setProperty('ausrichtung','text-left');
	$nodeTemplate->setProperty('text','<p><strong>Art.-Nr: '.$this->replaceAsterisk($rezeptur['Artikel Nr.']).' | '.$this->replaceAsterisk($rezeptur['Rezeptur Name']).'</strong></p>');
	$contentCollection->createNodeFromTemplate($nodeTemplate);

	// Headline anlegen
	$nodeTemplate = new \Neos\ContentRepository\Domain\Model\NodeTemplate();
	$nodeTemplate->setNodeType($this->nodeTypeManager->getNodeType('Example.Site:Content.Headline'));
	$nodeTemplate->setProperty('ausdehnung','col-lg-12');
	$nodeTemplate->setProperty('color','');
	$nodeTemplate->setProperty('ausrichtung_rahmen','');
	$nodeTemplate->setProperty('ausrichtung','text-left');
	$nodeTemplate->setProperty('anchorname','produktdetails');
	$nodeTemplate->setProperty('title','<h4>'.$this->replaceAsterisk($rezeptur['Artikel Bezeichnung']).'</h4>');
	$contentCollection->createNodeFromTemplate($nodeTemplate);

	// Text
	$nodeTemplate = new \Neos\ContentRepository\Domain\Model\NodeTemplate();
	$nodeTemplate->setNodeType($this->nodeTypeManager->getNodeType('Example.Site:Content.Text'));

	$nodeTemplate->setProperty('stil','');
	$nodeTemplate->setProperty('ausdehnung','col-lg-12');
	$nodeTemplate->setProperty('ausrichtung_rahmen','');
	$nodeTemplate->setProperty('ausrichtung','text-left');
	$nodeTemplate->setProperty('text','<p>Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. </p>');
	$contentCollection->createNodeFromTemplate($nodeTemplate);

	$count++;

}

$this->output->outputLine($count.' datensätze importiert');


// der vollständigkeit wegen...
protected function replaceAsterisk($value){
	$value = nl2br($value);
	$value = str_replace('***','"', $value);
	return $value;
}

I truncated the example a bit because there is just a lot of repetition adding the nodetypes.
Created this in an CommandController to do the import over the cli. It is a “import once do not touch again”-thing because of recreating the pages on every execution…

Hope this helps!
Göks

1 Like
4

Small addition for all people who want to make an database import:

  • I followed the approach of @sirin to use an own command controller for it (Thanks for your answer!)
  • You can find a further (and very similar) import example (old code with old Namespaces and old NodeTypes) here: http://dimaip.github.io/2014/08/14/content-migration-to-neos/
  • Hint: I went into some trouble on import text(area)-fields from an older project. Use a utf8_encode solves it for me: utf8_encode($row[“myText”]). Impact of this issue: When a not utf8-compatible content is filled to one field, ALL fields (properties of a node) are created without any content.

$newArticleNode->setProperty('description', utf8_encode($row["myText"]));

$uriPathGenerator = new \Neos\Neos\Utility\NodeUriPathSegmentGenerator();
$uriPathSegment = $uriPathGenerator->generateUriPathSegment($newNode);
$newNode->setProperty('uriPathSegment', $uriPathSegment);
3 Likes