RFC: Reference-label schema for the cross referencing of the documentation

mficzel · July 28, 2015, 8:12am

Cross-references for the neos-documentation

The documenation should make use of cross-referencing whereever possible. Those cross-references should prefer to link into the autogenereated reference-section of the docs. Other parts can and shall be linked whereever it makes sense.

Reference labels are placed generally before each headline but at least at the beginning of every rst-file.
The name of the reference label is determined from the path to the element in the docs and the referenced title and subtitle.
- The different Levels of the Structure are seperated by a “-” Character.
- Each Level of the Structure is represented as an UpperCamelCase word.
- SpecialChars in reference labels are omitted so reference labels do’nt have to be quoted.
- A long title may be shortened in rare cases.
- If part of the Structure is a repetition of the Container-Title it is omitted.

This would results in a structure of the reference-labels like this:

# Getting Started            .. _GettingStarted:
# Technical Principles       .. _TechnicalPrinciples:
# User Guide                 .. _UserGuide:
# Creating a Site with Neos  .. _CreatingASite:
# Extending Neos             .. _ExtendingNeos:
# Inside Neos                .. _InsideNeos:
# References                 .. _References:
      
  # EelHelper Reference      .. _References-EelHelper:
  # TypoScript Reference     .. _References-TypoScript:
  # ViewHelper Reference     .. _References-ViewHelper:
     # Media ViewHelpers     .. _References-ViewHelper-Media:
     # Fluid ViewHelpers     .. _References-ViewHelper-Fluid:
       # For ViewHelper      .. _References-ViewHelper-Fluid-For:
       # Alias ViewHelper    .. _References-ViewHelper-Fluid-Alias:
   
# How Tos                    .. _HowTos:
# Neos Operations            .. _NeosOperations:
# Appendixes                 .. _Appendixes:

christianm · July 28, 2015, 8:52am

Sounds like a good idea! Thanks for preparing this.

kapale · July 29, 2015, 6:19am

Really good! Think this will solve the issue.

mficzel · July 29, 2015, 7:00am

Note: This topic is also related to the question wether we use rst-domains or not. For now i suggest to use the label-schema for cross-references and think about creating custom rst-domains on some point in the future.

mficzel · July 31, 2015, 8:11am

I added a jira ticket for that https://jira.neos.io/browse/NEOS-1472

cognifloyd · August 4, 2015, 1:41am

For the major chapters, I would rather not depend on the reference labels at the beginning of each file. Instead we can use doc links:

:doc:`/GettingStarted/index`
:doc:`The Getting Started Guide </GettingStarted/index>`
:doc:`/ExtendingNeos/CreatingAPlugin`

Doc links, unlike adding a reference at the beginning of a file, don’t create links to an anchor at the top of the file. For example: In the first paragraph of this page, there is a link to an anchor in the content dimensions chapter. But, we’re not linking to something in the chapter, so it makes no sense to reference an anchor like this. We should just link to the page itself.

Of course, that only works when linking to a document, not a section. Sections within a document will still need to have the reference labels, and it makes sense to link to these sections with anchors.

I like the namespaced reference labels for the “References” part of the manuals. That’s a good way to do it until (and if) we create a RST Domain.

mficzel · August 4, 2015, 6:42am

I think it is more likely that cross references are created if there is a single best practice for creating labels and linking to them. I see that an anchor at the beginning of each file is not that elegant but it also does no harm. I still prefer a single reference naming schema across the whole documentation. Imho it makes it a bit easier to contribute to the docs and also to refactor the labels and references of a certain part if needed.

cognifloyd · August 6, 2015, 11:49pm

I was avoiding adding a label at the top of every file, but I’ve changed my mind.

Though we could avoid adding the label at the top of every file by linking between docs with :doc:, those links don’t work with intersphinx. That means that if someone adds the Neos or Flow docs to their intersphinx_mapping, then they won’t be able to link directly to any of our pages unless they use full URIs instead of reference links.

So, adding a label to each file makes sense from a community building perspective. We want to make the docs as easy to refer to as possible, including by linking from 3rd party (or even our own!) projects.

cognifloyd · August 7, 2015, 2:05am

So, now that I’m on board with the labels, I have a few comments about how to create them.

It looks like you’ve started using “_” instead of “-” as the divider for docs that are in a folder. At least with the links at the top of each file, couldn’t we use “/” instead of “-” or “_”? That looks more like a path instead of having to mentally translate the characters when looking for the file. That would mean that we’d have something like the following:

File: HowTos/index.rst
- label at top of file: .. _HowTos
- reference elsewhere: ``:ref:`HowTos```
File: HowTos/TranslatingContent.rst
- label at top of file: .. _HowTos/TranslatingContent
- reference elsewhere: ``:ref:`HowTos/TranslatingContent```
File: References/ViewHelpers/Fluid.rst
- label at top of file: .. _References/ViewHelpers/Fluid
- reference elsewhere: ``:ref:`References/ViewHelpers/Fluid```

For sections within documents (like the For ViewHelper or Preset Constraints), I think we need something to clarify that these are references to just a section not an entire document. So, we could do something very verbose like this, keeping everything in UpperCamelCase:

Section: Preset Constraints is in file: CreatingASite/ContentDimensions
- section label: .. _CreatingASite/ContentDimensions_PresetConstraints
- reference: ``:ref:`References/ViewHelpers/Fluid```

Or, we could use all-lowercase-with-dashes whenever something is a section, maybe using an underscore when the title alone would not make a good link without something to identify which file it’s in. So, to use the same Preset Constraints section, we could use one of these:

.. _preset-constraints ← We might need more than this to keep the labels unique & easy to use.
.. _ContentDimensions_preset-constraints
.. _CreatingASite/ContentDimensions_preset-constraints ← this would be very annoying/repetitive
.. _CAS/ContentDimensions_preset-constraints ← some kind of shorthand to avoid verbosity?

For reference materials, let’s use something more compact (less verbose) that will flow well in the raw restructured text. That will make adjusting to using a domain easier. When discussing Fluid ViewHelpers, for example, it’s pretty obvious that it’s fluid, so the extra “References-ViewHelpers-…” will get really old and tiresome. Here’s a sampling of how we could do references:

Fluid For Viewhelper (very short for things that everyone refers to alot)
- label: .. _f:for
- reference: ``:ref:`f:for```
Form DatePicker Viewhelper (slightly more verbose)
- label: .. _typo3.form:form.datepicker (or “neos.form:…” once we change the namespace)
- reference: ``:ref:`typo3.form:form.datepicker```
TypoScript Array
- label: .. _typo3.typoscript:array
- reference: ``:ref:`typo3.typoscript:array```
FlowQuery add operation
- label: .. _flowquery:add
- reference: ``:ref:`flowquery:add```
Eel Array Helper, concat method
- label: .. _eel:array.concat
- reference: ``:ref:`eel:array.concat```

Changing the labels for references will require some changes to TYPO3.DocTools which we use to generate those references. So, there’s already something that we have to maintain. I wonder if adding some sphinx domains wouldn’t make it easier to create links like these. But, then again, I think we can do all or some of this later, perhaps using links like these as an intermediate step between where we are and using sphinx domains.

mficzel · August 7, 2015, 7:36am

Right that should be fixed.

I’m not shure wether the “/” character works well but if there are no technical reasons against i’m fine with “/”.

Id use a single structure for documents and sections. That allows us to refactor large documents into parts whenever we want to without much hassle. There are already single .rst files that are much to long.

In the references we already have the case that all EelHelpers are rendered into a single .rst but all ViewHelpers in a .rst file per package. That has technical reasons but should not lead to different label styles.

Hmm at least the “:” will cause trouble. Additionally the namespace is only defined for the f: view Helpers. So all other namespaces can only be suggested.

Maybe we just skip the “References” part there because this section should be referenced a lot . The type of referenced-item should be part of the label because otherwise we have no clear distinction between objects of the same name.

That could lead to something like this:

# References                 .. _References:

  # EelHelper Reference      .. _EelHelper
    # Math                   .. _EelHelper/Math
      # random               .. _EelHelper/Math/Random
  # TypoScript Reference     .. _TypoScript:
  # ViewHelper Reference     .. _ViewHelper:
     # Media ViewHelpers     .. _ViewHelper/Media:
     # Fluid ViewHelpers     .. _ViewHelper/Fluid:
       # For ViewHelper      .. _ViewHelper/Fluid/For:
       # Alias ViewHelper    .. _ViewHelper/Fluid/Alias:

I’m aware of that, thats why i want to have this defined and the main docs relabeled
before going over the doctools again to adjust the generation there.

cognifloyd · August 7, 2015, 5:42pm

Based on my testing with sphinx 1.3.1 (which is the version used on readthedocs[1]) and docutils 0.12 (readthedocs uses 0.11, but there are no significant changes in 0.12[2][3]), a slash works just fine. When it creates the anchors in html, everything is lowercased, and the slashes become dashes.

Fascinating thought. That would mean that those section labels would become the filename. I think I’m warming to the idea. That means that sections will end up with really long labels, is that okay? Will that make it harder or more annoying to use the references (``:ref:`FooFooFoo\BarBarBar\BazBazBaz```) in other parts of the docs?

I don’t think using colons will cause any trouble, as long as we don’t have two colons right next to each other. Am I missing something? I’d really like a different label/ref style for the reference section of the manuals that is as concise as possible. Everything else is great with the path-style labels.

Based on my testing, colons don’t cause (much) trouble. I tried using each of these labels: “.. _test:withcolon:” and “.. _`test:withcolon`:”. In both cases, I tried referencing the label with each of these: “:ref:`bla <test:withcolon>” and “:ref:`bla <test:withcolon>”.

The only “issue” I had without the backticks, was highlighting in my editor (vim) choked on the colon. Sphinx (and consequently Docutils) worked without any error or warning message.

The docutils spec says this about allowed characters (Sphinx docs don’t mention allowed characters, so the docutils spec still applies):

Simple reference names are single words consisting of alphanumerics plus isolated (no two adjacent) internal hyphens, underscores, periods, colons and plus signs; no whitespace or other characters are allowed.
…
Reference names using punctuation or whose names are phrases (two or more space-separated words) are called “phrase-references”. Phrase-references are expressed by enclosing the phrase in backquotes and treating the backquoted text as a reference name:
…
Simple reference names may also optionally use backquotes.
Reference names are whitespace-neutral and case-insensitive.

And more specifically about using colons, we should use backticks, even though I didn’t get any errors/warnings without them:

If the reference name contains any colons, either:

the phrase must be enclosed in backquotes:

 .. _`FAQTS: Computers: Programming: Languages: Python`:
    http://python.faqts.com/

or the colon(s) must be backslash-escaped in the link target:

.. _Chapter One\: "Tadpole Days":

   It's not easy being green...

cognifloyd · August 8, 2015, 7:07pm

Simple reference names are single words consisting of alphanumerics plus …

Reference names using punctuation or whose names are phrases (two or more space-separated words) are called “phrase-references”.

Just to clarify. We do not use “simple reference names”, we always use “phrase-references” because we do not use internal links directly. An internal link would be something like label_ or ```label`_which is a restructured text construct that only works within a single file/document. However, with sphinx we use the:ref:role to link between files/documents. The:ref:role is *always* a phrase-reference thanks to the fact that:ref:`` is a role, and is, therefore, interpreted text.

Even with internal links, the character limitations only applied to using the reference not creating them. When creating a reference, (with .. _label or .. _`label` ), there are only three character limitations:

the first character can’t be a space or a backtick;
if the phrase is not surrounded with backticks, then any colons should be escaped.
the last character can’t be a space, a colon, or a backtick unless they are escaped.

Every other character is allowed, including colon, and forward slash. And surrounding the reference with backticks is optional when creating it, even if we use characters like space, colon and slash.

mficzel · August 24, 2015, 8:50am

To sum up the discussion i suggest to use the following reference schema with “/” instead of underscores:

Reference labels are placed generally before each headline but at least at the beginning of every rst-file.
The name of the reference label is determined from the path to the element in the docs and the referenced title or subtitle.
- The different Levels of the Structure are seperated by a “/” Character.
- Each Level of the Structure is represented as an UpperCamelCase word.
- SpecialChars in reference labels are omitted so reference labels do’nt have to be quoted.
- A long title may be shortened in rare cases.
- If part of the Structure is a repetition of the Container-Title it is omitted.
- Inside the “References” Section the first part of the path is omitted

This would results in a structure of the reference-labels like this:

# Getting Started            .. _GettingStarted:
# Technical Principles       .. _TechnicalPrinciples:
# User Guide                 .. _UserGuide:
# Creating a Site with Neos  .. _CreatingASite:
# Extending Neos             .. _ExtendingNeos:
  # Creating A Plugin        .. _ExtendingNeos/CreatingAPlugin:
# Inside Neos                .. _InsideNeos:
# References                 .. _References:
  # EelHelper Reference      .. _EelHelper:
  # TypoScript Reference     .. _TypoScript:
  # ViewHelper Reference     .. _ViewHelper:
     # Media ViewHelpers     .. _ViewHelper/Media:
     # Fluid ViewHelpers     .. _ViewHelper/Fluid:
       # For ViewHelper      .. _ViewHelper/Fluid/For:
       # Alias ViewHelper    .. _ViewHelper/Fluid/Alias:       
# How Tos                    .. _HowTos:
# Neos Operations            .. _NeosOperations:
# Appendixes                 .. _Appendixes: