Multi‑Repo Git Script for AEM Cloud Service: cmreactor.sh

TL;DR: Don't be an irritable CM Developer + Deployment Manager—add our portable cmreactor script to your AEM Cloud Service project today!

Adobe Experience Manager (AEM) developers are often caught by surprise the first time they deploy a second codebase to an AEM as a Cloud Service environment when they discover that the first codebase has been completely removed. This is due to a fundamental change to how AEM loads code at runtime in Cloud Service, compared to how the process has traditionally worked in AEM Classic.

Instead of uploading and installing any content packages in the build to a running AEM Package Manager, a Cloud Manager pipeline now builds a complete docker volume from scratch every time it is run by combining the standard AEM platform libraries with the build artifacts resulting only from the current git commit. A major benefit of this new approach is that the server code in every AEM environment is directly traceable back to a particular pipeline execution, AEM SDK version, and git commit, which makes automated testing more predictable and valuable.

One side effect of this behavior, however, is that you can't incrementally install application content packages in an environment by running a CM pipeline to build one feature's source branch and then running a pipeline to build another feature's source branch like you can with AEM Classic. All of the application packages needed to run all desired AEM features need to be sourced from a single branch in the Cloud Manager program's git repository.

The trick is to check out all of the source repositories you want to deploy as subfolders beneath a common parent folder containing a simple maven reactor pom.xml file, then initialize a new git repository that contains all of it, and push that to your Cloud Manager git remote instead.

Unfortunately, some AEM developers who are trying to maintain a stable sandbox environment for training or for testing an important feature may become irritable when they think about all the git commands they will have to carefully execute when another developer requests a deployment of a different project to run in parallel, even if there is no reason to think that there would be code conflicts.

^{Pictured: This Author}

Multiple Source Git Repositories

Of course, this change hasn't just affected partners like Bounteous. More large AEM as a Cloud Service customers who are used to multitenant CI deployments to AEM classic are facing a similar challenge as they try to ramp up parallel dev teams after successfully migrating their website from AMS hosting or on-prem to Cloud Service. 

Adobe has examples for handling multiple source git repositories in your CI application of choice, but Adobe doesn't provide the external CI infrastructure for you, and setting one up may not be worth the added cost and complexity if you aren't hosting your code in a git repository service that provides CI for free.

Zip-Distributed Project Sources

Project Firefly has come a long way, and now it is becoming more attractive as an approach for building external AEM integration points in situations that might have called for internal, OSGi-based solutions in AEM classic. A fundamental step for bridging AEM to Firefly runtimes is to get AEM to send events to Adobe I/O journals so that workers can subscribe and react asynchronously and reliably.

If you follow the official instructions linked in a recent Adobe Tech Blog post by Kelvin Xu (Register/Map Custom AEM I/O Events), you will be advised to download a zip file containing the exported maven project sources for Adobe's aem-event-proxy-skyline feature. The instructions then recommend extracting it into your project's git repository, but since the code doesn't need to be a compile dependency and will very likely be part of the AEM cloud SDK in some form in the future, it would be cleaner to avoid merging it into your upstream project if you aren't going to be customizing it.

There's Got to Be a Better Way!

To solve these two challenges without depending on CI, we developed cmreactor.sh: a standalone, portable bash script that encapsulates the fundamental operations recommended by Adobe for pushing multiple source git repositories to Cloud Manager, while adapting Git's own configuration file format to avoid tight-coupling of the script itself to your particular project's needs.

It endows any developer with the right permissions in Cloud Manager with the power to integrate any number of git repositories into a single multi-module maven project in a safely-named branch. The script also enables developers to integrate a zip-distributed maven project like aem-event-proxy in the same way.

To install the script, just copy the file to your project source root and make it executable. Use the following command to get started with your existing project as the base module, and event-proxy as an additional module.

./cmreactor.sh --config-template > my-reactor.gitconfig

To quickly try it out without pushing to Cloud Manager, run this command, and take a peek at the resulting git repository in the target/cmreactor-project subfolder:

./cmreactor.sh --file my-reactor.gitconfig --no-push

After you read the command usage (./cmreactor.sh --help) and adjust the configuration to satisfy your particular needs, you can run that same command without the --no-push flag, and then update your pipeline of choice in Cloud Manager to use the newly created branch. After the pipeline passes the maven build step, you should see new event-proxy packages detected for deployment at the end of the build log file.

Get Your Head in the Cloud

With cmreactor.sh, you don't have to jealously guard access to the Cloud Manager pipeline that builds to your dev AEM environment. You can now "share the road" with other teams' sprint cycles, and your privileged developers can more easily try out Adobe's prerelease platform features in a live environment without the burden of merging and committing it to an existing internal project repository.

The script itself can easily fit into your organization's current or future AEM CI process, with no external dependencies apart from curl, unzip, find, rsync, bash, and git. If the CM pipeline limitations have been keeping you grounded in localhost, grab a copy of cmreactor.sh and get your head in the cloud, today!