Changes to Joomla manual examples to support lazy loading of plugins

[robbiejackson]

I'm starting to document more fully the lazy plugin aspect.

This leads us into having different code for different versions of Joomla, and we need to agree in general how to document this within the Joomla manual examples repository.

I believe that any code in the manual examples repo should work for all the versions which are maintained in the Joomla manual, so at the moment that would be 4.4, 5.4, 6.0 and 6.1.

I've been thinking about the best way to handle these version-specific code sections and have proposed this PR, which is now up for review.

While the code in the manual examples repository needs to have the switch on JVERSION, I would propose NOT having this in the manual - so in the manual the particular document version would just have the code appropriate to that Joomla version.

For example at https://manual.joomla.org/docs/building-extensions/plugins/basic-content-plugin/ it would just show the services/provider.php file for that version alone.

I've also put comments for each version change which have links to the Joomla code changes. I'm not sure that everyone will want to look at these, but there are probably some people who will, and having this makes it easy for them.

Instead of having the version if/then statement at the $container->set() level we could have it inside the set() to show finer granularity on the differences between 4.4 and 5.3 - eg

function (Container $container) {
    $config = (array) PluginHelper::getPlugin('content', 'shortcodes');
    $app = Factory::getApplication();

    if (version_compare(JVERSION, '5.3', '>=')) {

        // Dispatcher passed to plugin constructor deprecated 
        // See https://github.com/joomla/joomla-cms/pull/43430
        $plugin = new Shortcode($config);

    } else {

        $dispatcher = $container->get(DispatcherInterface::class);
        $plugin = new Shortcode($dispatcher, $config);

    }

    $plugin->setApplication($app);
    return $plugin;
}

I think that might be clearer - but let me know what you think.

[Fedik]

I would show only code for specific version.
And keep old versions as they are.

Latest would be:

...
$container->lazy(Shortcode::class, function (Container $container) {
  ...
  $plugin = new Shortcode($config);
});
...

I believe that any code in the manual examples repo should work for all the versions which are maintained in the Joomla manual

Not really.
In my opinion it is more like an each manual version should cover "version related" state:
5.0 manual only stuff recommended for 5.0
6.0 manual only stuff recommended for 6.0, with maybe a note for migration if anything special happened.
and so on

I would avoid if (version_compare(JVERSION, '5.3', '>=')) as much as possible.
It is a bad example and make code look more scary than it really is 😉

[robbiejackson]

The problem is that while the Joomla manual does support different versions, the manual examples repo (currently) doesn't.

So, for example, in the manual we have the plugin tutorial at https://manual.joomla.org/docs/building-extensions/plugins/basic-content-plugin/ and this page has a link to the plugin tutorial code in the manual examples repo - ie it has the link https://github.com/joomla/manual-examples/tree/main/plugin-example-shortcode

This link is the same no matter what version you've selected within the manual.

So let's say we have the code in the manual-examples repo set to be the latest code, using the lazy() functionality.

Then an extension developer is reading the manual version 4.4 documentation because he has to write a plugin that works under Joomla 4.4. He looks at the plugin tutorial page, follows the link to the manual-examples repo and installs the plugin-example-shortcode on his test Joomla 4.4 instance. It will obviously crash because the DI container has no lazy() function and the plugin constructor parameters are wrong.

This is what we need to avoid.

The alternative to switching on JVERSION is to have multiple versions of plugin-example-shortcode in the manual-examples repo, and have different links to them. For example on https://manual.joomla.org/docs/building-extensions/plugins/basic-content-plugin/:

For the 4.4 page link to https://github.com/joomla/manual-examples/tree/main/plugin-example-shortcode-4-4

For the 6.0 page link to https://github.com/joomla/manual-examples/tree/main/plugin-example-shortcode-6-0

I'm ok to do this if you think that would be better, but it does feel like a bigger maintenance overhead.

I recognise that the code with the JVERSION logic looks complex. But on the other hand it does make it clear to extension developers what they need to consider - in general they will be looking to develop an extension that runs on multiple versions of Joomla, so this information is useful for them, and actually may be the way they decide to code their extension.

[HLeithner]

I fully understand your concerns about supporting multiple versions, but to be realistic, if you want to learn how to create a new plugin, you don't care about b/c. If you like to create extensions for multiple versions we should have a section explaining how to use the JVERSION constant.

For different plugin tutorial version we should different branches, we have for the cms and the framework already a url schema: cms-api:// and framework-api:// which we can extend for the example repository:

example://
leads to

https://github.com/joomla/manual-examples/tree/<JVERSION_MINOR_OR_ONLY_MAJOR>/plugin-example-shortcode

this would allow people to easily compare j4 version of the plugin with j6 version directly in github.

what this be acceptable for you?

[robbiejackson]

OK, so I believe you're saying that having different versions of the plugin in the repo would be better.

Let me work on that approach and push another commit ...

[Fedik]

The problem is that while the Joomla manual does support different versions, the manual examples repo (currently) doesn't.

I am sorry, I totally missed that it is not Manual repo 😄

So, for example, in the manual we have the plugin tutorial at https://manual.joomla.org/docs/building-extensions/plugins/basic-content-plugin/ and this page has a link to the plugin tutorial code in the manual examples repo - ie it has the link https://github.com/joomla/manual-examples/tree/main/plugin-example-shortcode

For the 4.4 page link to https://github.com/joomla/manual-examples/tree/main/plugin-example-shortcode-4-4
For the 6.0 page link to https://github.com/joomla/manual-examples/tree/main/plugin-example-shortcode-6-0

Well, I think it should not be that complicated.
Do git branches, so can keep old version as they are but update only new. The links would be kind of:

For the 4.4: https://github.com/joomla/manual-examples/tree/4.4/plugin-example-shortcode
For the 6.0: https://github.com/joomla/manual-examples/tree/6.0/plugin-example-shortcode

I think that should be a good path.

upd: I think I wrote the same what @HLeithner wrote :)

[robbiejackson]

OK, is this a workable approach using git branches?

I'll take the example of wanting to change the plugin tutorial example for 6.1, in order to use lazy plugins. We really only need 2 versions for this - make the change to drop the dispatcher parameter in the constructor at the same time as implementing the lazy plugin, ie 6.1.

So to do this, first create a branch on the joomla repo called something like "up-to-6.0" so that this would be the same as the "main" branch at this point.

Then make the changes for lazy plugins (and removal of dispatcher in constructor parameters) and merge into the main branch.

In the manual for the plugin tutorial on version 6.1, point to the main branch

In the manual for the plugin tutorial on version 6.0, 5.4, 4.4 etc point to the up-to-6.0 branch

However in the manual for all the other examples, continue to point to the main branch.

So the up-to-6.0 branch is really just for exceptions, where the new code in 6.1 won't work on previous versions.

If we put a new extension into the manual-examples repo, then it needs to be visible to all versions. This is where the requirements for the manual-examples repo differs from the requirements for the joomla cms repo. If we always used a named version branch (eg in the manual for version 4.4 linking always to a 4.4 branch in the manual-examples repo) then we'd have to merge the new extension code into each of the 4.4, 5.4, 6.0 etc branches. Too complex.

So this is why I'm suggesting we just use the main branch, and point everything at it (as we do now), and have other branches just for exceptional cases.

What do you think?

[Fedik]

Hm, not really like that.

The branch should be for "starting from version X" not "up to version X".
Also no need new branch for each version, this only add extra work.
It should be done only when it is really needed.

Example currently the main branch collects examples that works from 4.x, and from 5.x and from 6.0.
For lazy() service, and with removed dispatcher, we do start from 6.1 branch.

When in future need to fix some comments etc, do fix in the main branch and then up-merge changes to 6.1 (and 7.x etc).

In the manual the old documentation will still point on main branch (that is valid for that versions), and only newest will point to 6.1. Then for example, when will be released 6.5 then the documentation still will point on "6.1 of examples". When will be another big changes, example in 7.3, then create a new "7.3 of examples". Old branches will stay for old manuals, and new manual will use new branch. And so on.
This also can be useful for Developers to see a difference between versions.

Unless I missing something.

[robbiejackson]

I think we may be suggesting the same thing, and the only difference is how we have named the branches.

So after 6.1 the branches would be:

Your scheme	My scheme
main	up-to-6.0
6.1	main

After 7.3 the branches would be:

Your scheme	My scheme
main	up-to-6.0
6.1	up-to-7.2
7.3	main

I just prefer the aspect of always merging new stuff into the main branch. I think that's simpler for contributors - they just have to sync the main branch, and check out new branches from main etc. It also avoids any confusion in the mind of contributors - eg if they're developing documentation for version 7.1 and adding an example, they may be wondering why they should be working with the 6.1 branch.

[Fedik]

Well yes. But in the way how git is working I am not very imagine how it can work easily.
You probably will be need to rename the branches back and forth, that could lead to unexpected surprises 😉

Also this would means that you have to edit Manual page for older versions which is another extra work.
Example when you do new "up-to-7.2" branch you have to rename all existing links and references in old Manual.

I think it is better to avoid an extra work here.

In the github it is possible to set "default branch". Currently it is "main" in future it can be "6.1".
This way when user visit the examples repo they always get latest version first.

[robbiejackson]

I'm afraid I disagree!

You probably will be need to rename the branches back and forth, that could lead to unexpected surprises 😉

As I said, I believe we're talking about the same solution, but just labelling the branches differently. So if you had to rename the branches with one scheme you'd have to do it with the other. (But I don't believe you need to rename the branches anyway).

Also this would means that you have to edit Manual page for older versions which is another extra work.
Example when you do new "up-to-7.2" branch you have to rename all existing links and references in old Manual.

No you don't need to update all the links, you just update the links for the extension which is affected. So in the case of lazy plugins you'd just update the links for those pages which reference the plugin tutorial, for versions prior to 6.1.

For all the other extensions in the examples repo they're pointed at using the main branch, and they just continue to use that.

In fact, you'd need to do a lot more updating of links in the manual if we went with your scheme.

For example, say there was a missing bracket in the module tutorial and that needed to be fixed. If we're using a 6.1 branch then the fix would get merged into 6.1. Then all the links from the manual would need to be updated from pointing to main to pointing to 6.1.

And then if there was another fix in 7.3 you'd need to update all those links again from 6.1 to 7.3.

So pretty much any change to existing examples would need to have links updated, or at the least, you'd need to check if they need to be updated. As the links are scattered through the manual this would involve a lot of work.

If you're going with my scheme where the branch you're merging into is main, then none of the links need to be updated when you make a change like this.

It's much better to go with a scheme where links don't need to be updated, because people are always going to forget to update them properly.

In the github it is possible to set "default branch". Currently it is "main" in future it can be "6.1".
This way when user visit the examples repo they always get latest version first.

Well, I reckon they would usually go from the link in the manual, and that link has the branch within it.

But it's not just that. It's to make it as easy as possible for contributors as well.

For example, at the moment on my local PC I do:

git checkout main
git checkout -b feature-branch

and then I'm always merging into main.

If you're merging into 6.1 then you have to do

git checkout 6.1
git checkout -b feature-branch

and remember which is the current default branch, which someone can change and other contributors can be unaware of the change.

I reckon most documentation contributors are like me and don't do a lot of development, and are far from being git experts. So the whole process needs to be as straightforward as possible.

I think I've talked myself to a position where whatever solution we adopt we have to continue having main as the default branch which we're merging new stuff into. That makes it easiest for contributors, and minimises the amount of updating of links in the manual.

[HLeithner]

About the renaming in the manual, that would not be needed if we match branch name with version in the manual and preserve all minor versions.

Then we do the same as for the framework api using instead of cms-api:// the example-repo:// which automatically includes the version number. As I wrote above.
Example:
the Link example-repo://component-ajaxdemo would map to:
https://github.com/joomla/manual-examples/tree/6.1/component-ajaxdemo for the 6.1 version
and
https://github.com/joomla/manual-examples/tree/7.0/component-ajaxdemo for the 7.0 version

this has a little maintainance overhead which means creating a new branch from the current default branch when we release a new docs version for 6.1 for example. This took 2 minutes, not really relevant when I out branch the new manual version.

Advantages:

• Developers coming to the repo directly see what should work for the target version
• We have a matching version for the CMS automated tests (I would really like to have them on each PR run but I didn't head time yet and no body else had a look on it)

Disadvantages:

• if we have to fix a syntax error we have to cherry pick the change or do an upmerge for all active version

The only thing what never changes or better does it automatically is the link in the manual to the corresponding branch/version.

[Fedik]

If you're going with my scheme where the branch you're merging into is main, then none of the links need to be updated when you make a change like this.

Yes, for latest documentation the renaming will not be needed.
However what I meant, if we keep all "recent' in the "main" then all older documentation will reference to "main" also.
Like:
https://manual.joomla.org/docs/4.4/
https://manual.joomla.org/docs/5.3/

Will show links to "main" where is the code already updated for latest 6.1 version.
That will be incorrect. And that a loot more of renaming 😉

Then we do the same as for the framework api using instead of of cms-api:// the example-repo://

Yes, that will work. I just not sure if we need a branch for each version.
This add a lot of "up-merging" work when need to fix typo in the comment.

[HLeithner]

This add a lot of "up-merging" work when need to fix typo in the comment.

not really, because we only have max 4 branches at the same time we support (active versions)

• 5.4
• 6.0
• 6.1
• upcoming (could be main branch)

[robbiejackson]

Using the example-repo:// would create more problems than it solves.

For example, if you're fixing a typo and that is then merged into branch 7.1 how does example-repo:// know that it should reference the 7.1 branch if it's on a 5.4 page? The alternative is to merge the change into all the other branches as well.

And you'd need to do the same if you wanted to include a new extension in the manual examples repo, as it would be merged into the latest branch, and not visible on older branches if you didn't merge it in there as well.

Merging all the changes into multiple branches is a route that I'm not prepared to go down.

I can understand that your guys' experience with the Joomla CMS and its version numbering tends to lead you down a route like that here. However, it just makes it all too complex. And it's a different problem we're trying to solve.

Up until now we have been using just the main branch on the manual examples repo and that has worked really well. The process for contributing is straightforward and the links don't change for different versions.

This will continue to be the case for > 99% of all of the code, so let's just keep it that way, and use separate branches just for exceptional cases. I really don't mind what these exceptional branches are called.

But overall, please just let's keep it simple.

[Fedik]

Up until now we have been using just the main branch on the manual examples repo and that has worked really well.

The reason you made this PR is that we are reaching the limit of this approach 😉
You see, while all code examples relatively the same across the versions and able to work in all versions it is not a problem and can continue to be like that.
But we reaching the point when newest version have newest feature that would be nice to reflect in the example.

It is possible to use if (version_compare(JVERSION, '5.3', '>=')) workaround, but with each newest version and more new features/changes this will be like a snowball rolling down hill. At some point it will become unmaintainable code with not very friendly examples.

Merging all the changes into multiple branches is a route that I'm not prepared to go down.

I totally understand, everyone hate it 😄

how does example-repo:// know that it should reference the 7.1 branch if it's on a 5.4 page?

The thing that we have to crate branch in manual-examples repository for each version.
Then Docusaurus script just replace example-repo:// to https://github.com/joomla/manual-examples/tree/X.Y/ for the version it is building. It does not need to know anything except current manual X.Y version.

The main work need to be done in manual-examples repository.
I can see 2 way:

1. What I and Harald suggested: Each version have a branch, and default branch is latest version.
   When need to update "typo" we fix it in the older version when it first occur (let say in 5.3) and then up-merge up to latest version (potentiality still could be a merge conflicts).
2. Close to what you have suggested (but not exactly): The default branch is always main, but each version still have own branch. When we do changes in "main" we have to up-merge it in to latest x.y branch.
   When need to update "typo" then we have to "cherry-pick" the fix in to other affected versions. An "up-merge" will not work here and can just break other versions.

In both cases we do not need to do anything in the Manual, except update link like example-repo://module-tutorial.
Second way requires a bit more work I think.

Third way would be to branch out "up to X.Y" in example repo, as you suggested, and still keep "main" for latest version. But here I not sure how to update all that in Manual, because our Docusaurus script is dumb and do not know if external repository exist or not, it just know the Manual version, and even if it know how do we handle it hmhm.

Maybe any other ideas?

[Fedik]

Oh, and the way 2.1 :
Similar what suggested in point (2), but the latest version always "main" without need to up-merging to latest, then Docusaurus script will point version current to main branch in the examples repository.
With this approach we do new branch only when add new version in Docusaurus.