In Wikipedia, Application Lifecycle Management (ALM) is defined as :

“The marriage of business management to software engineering made possible by tools that facilitate and integrate requirements management, architecture, coding, testing, tracking, and release management”

Starting a development effort with the appropriate source control mechanisms and release methodology can save you hundreds of painful hours when managing releases and your source code. Adopting a smart, reliable and robust application lifecycle and release process can exceed your expectations and actually be the foundation for your project success.

PSG – Alfresco Sdk Rock and Roll with Jenkins

Following is a list of “mortal sins” of application lifecycle” what we seek to avoid.

– Manual changes in production environments
– Manual error prone testing procedures and stressful UAT phases
– Multiple development standards and unmanaged versioning policies

The main commandments of Application Lifecycle management.

• Identify and respect your release
• If it’s not tested (automatically) it’s not working
• If it’s not documented it doesn’t exist
• Controlled integration is possible and should not limit business improvement
• Centralize common configuration while leave projects enough flexibility for special cases

The PSG Methodology

PSG stands for “Plain Simple Goals” and its a application lifecycle and release management methodology focused at Alfresco based installations. It uses the Alfresco Maven SDK and Jenkins as the foundations providing a methodology for Alfresco development, release management and application lifecycle management.

The main goal of this methodology if to provide a reproducible and scalable way to manage application build, test, release, maintenance and integration policies.

My invite to you, on this particular post, is not only to read the post but to actually try the project, following easy and exact step by step instructions along the post, in return you will get :

• Alfresco Development and Test Infra-structure
• A Rapid and Smart Alfresco Development methodology
• A Build Infra-structure
• An reliable Release Process
• A robust Alfresco Application lifecycle management approach

Technically speaking you will end up with :

• In memory H2 database and in-memory application server (tomcat)
• A extension module for the alfresco repository that creates an Alfresco Module Package(.amp)
• A extension module for alfresco Share that creates an Alfresco Module Package(.amp)
• A running instance of the alfresco repository with your overrides and your .amp extension deployed and tested
• A running instance of the alfresco share application with your overrides and your .amp extension deployed and tested

Let’s start then, PSG is a project hosted on my personal github  and it contains the technical foundations of this methodology.

Step 0 – Download your working copy of the foundation

Start by clicking here to download your copy of the “Plain Simple Goals” methodology. Unzip the contents to a place on your computer. This will be your development environment home, so take note of this path. You will need to come back here later.

Step 1 – Pre-Requirements

1 – Software requirements

If you don’t have Java JDK, Maven and Jenkins installed on your computer/server now is the time to do so. Visit the urls below to install the pre-requirements.

• JDK 1.7
• Apache Maven 3.0.4+
• Jenkins Continuous integration server

About Maven

Maven’s primary goal is to allow a developer to comprehend the complete state of a development effort in the shortest period of time. In order to attain this goal there are several areas of concern that Maven attempts to deal with:

• Making the build process easy
• Providing a uniform build system
• Providing quality project information
• Providing guidelines for best practices development
• Allowing transparent migration to new features

About Jenkins

Jenkins is an award-winning application that monitors executions of repeated jobs, such as building a software project or jobs run by cron. Among those things, current Jenkins focuses on the following two jobs:

• Building/testing software projects continuously, just like CruiseControl or DamageControl. In a nutshell, Jenkins provides an easy-to-use so-called continuous integration system, making it easier for developers to integrate changes to the project, and making it easier for users to obtain a fresh build. The automated, continuous build increases the productivity.
• Monitoring executions of externally-run jobs, such as cron jobs and procmail jobs, even those that are run on a remote machine. For example, with cron, all you receive is regular e-mails that capture the output, and it is up to you to look at them diligently and notice when it broke. Jenkins keeps those outputs and makes it easy for you to notice when something is wrong.

About Java 

Just kidding

2 – Credentials for Enterprise

If you wish to work with alfresco enterprise you need to have login credentials on the Alfresco Nexus repository (artifacts.alfresco.com). You can request login credentials on the Alfresco support portal. Alternatively you can just build and run the open source version Alfresco Community.

3 – Configuration requirements to build Alfresco Enterprise/Community

During the installation of maven, a new file name settings.xml was created. This file is our entry point to the your local maven settings configuration, including the remote maven repositories. Edit your settings.xml file and update the server’s section including the alfresco server id and your credentials.

Note that the root pom.xml references 2 different repositories : alfresco-private, alfresco-private-snapshots. If you are building Community you should call those alfresco-public and alfresco-public-snapshots. Note that the id of each repository must match with a server id on your settings.xml (where you specify your credentials for that server).

Section from the settings.xml maven configuration file

...
   <repository>
     <id>alfresco-private</id>
     <url>https://artifacts.alfresco.com/nexus/content/groups/private</url>
   </repository>
   <repository>
     <id>alfresco-public-snapshots</id>
     <url>https://artifacts.alfresco.com/nexus/content/groups/private-snapshots</url>
   </repository>
 ...

...
   <repository>
     <id>alfresco-public</id>
     <url>https://artifacts.alfresco.com/nexus/content/groups/public</url>
   </repository>
   <repository>
     <id>alfresco-public-snapshots</id>
     <url>https://artifacts.alfresco.com/nexus/content/groups/public-snapshots</url>
   </repository>
 ...

Section from the root pom.xml (Enterprise)

...
        <server>
            <id>alfresco-private</id>
            <username>YOUR_USERNAME</username>
            <password>YOUR_PASSWORD</password>
        </server>
        <server>
            <id>alfresco-private-snapshots</id>
            <username>YOUR_USERNAME</username>
            <password>YOUR_PASSWORD</password>
        </server>
 ...

Section from the root pom.xml (Community)

...
        <server>
            <id>alfresco-public</id>
            <username>YOUR_USERNAME</username>
            <password>YOUR_PASSWORD</password>
        </server>
        <server>
            <id>alfresco-public-snapshots</id>
            <username>YOUR_USERNAME</username>
            <password>YOUR_PASSWORD</password>
        </server>
 ...

Source Control

The project uses the maven SCM plugin. The SCM Plugin offers vendor independent access to common scm commands by offering a set of command mappings for the configured scm. Each command is implemented as a goal.

Configure the main pom.xml with your own source control mechanism, the example in the download is configured with my github account. If you don’t have one yet, you can create your own free github account and use it.  If you use subversion you can have something like :

...
<scm>
  <connection>scm:svn:http://<YourRepo>/svn_repo/trunk</connection>
  <developerConnection>scm:svn:https://<YourRepo>/svn_repo/trunk</developerConnection>
  <url>http://<YourRepo>/view.cvs</url>
</scm>

 ...

Distribution Management
We need to have configured the repository that will hold the artifacts of the releases. That is configured using the deploy plugin of maven. In the psg project i have my own public-cloudbees repository configured, you can create your own free cloudbees repository and update the pom.xml accordingly so that your releases are stored in your repository. Since during this post we will install a instance of artifactory (see the bottom of this post for installation instructions), we should use it to hold our release artifacts. Configure it as follows:

...
<distributionManagement>
  <repository>
    <id><your-company>-private-release-repository</id>
    <url>dav:https://<YOUR_CI_SERVER_IP>:8080/artifactory/<project_name>/release/</url>
  </repository>
  <snapshotRepository>
   <id><your-company>-private-snapshot-repository</id>
   <url>dav:https://<YOUR_CI_SERVER_IP>:8080/artifactory/<project_name>/snapshot/</url>
  </snapshotRepository>
</distributionManagement>
 ...

Note that the repository id configured on the pom.xml must match a server id on your local maven settings.xml file.

Section from your local settings.xml maven configuration file

...
<server>
        <id><your-company>-private-snapshot-repository</id>
        <username>YOUR_PRIVATE_MAVEN_REPOSITORY_USERNAME</username>
        <password>YOUR_PRIVATE_MAVEN_REPOSITORY_PASSWORD</password>
        <filePermissions>664</filePermissions>
        <directoryPermissions>775</directoryPermissions>
    </server>
    <server>
        <id><your-company>-private-release-repository</id>
        <username>YOUR_PRIVATE_MAVEN_REPOSITORY_USERNAME</username>
        <password>YOUR_PRIVATE_MAVEN_REPOSITORY_PASSWORD</password>
        <filePermissions>664</filePermissions>
        <directoryPermissions>775</directoryPermissions>
    </server>
...

This will enable you to perform releases with :

• Prepare release : mvn release:prepare
• Perform release : mvn release:perform
• Prepare and Perform release : mvn release:prepare release:perform

Step 3 – Finally , let run it ! 

On the projects root folder you have the heart of the project, the parent pom.xml. This is the file that aggregates your full build, including all the modules and overlays to the different applications and generates deployable artifacts ready for your release. Before we run it for the first time let’s review once more What is included in this project build
applications(apps folder)

• alfresco repository overlay module
• alfresco share overlay module
• alfresco solr configuration overlay module

Alfresco module packages(amps folder)

• alfresco respository extension amp module
• alfresco share extension amp module

To run the project for the first time, issue the following maven command to run the project from the root of your development environment.

# mvn clean install -Prun

This will build and run all the modules, when the maven process finishes, open a browser and point it to :

• http://localhost:8080/alfresco (The url to access the alfresco repository via explorer client)
• http://localhost:8080/share (The alfresco share repository client)

Step 4 – Jenkins Integration – Build and Release Processes 

Now that we see what our build can do we will go a step further and we’ll perform the integration with Jenkins. This is where the automation fun begins.

Our goals are automating the build and deployment process. The project artifacts are always built after every check-in of new source code, which means early warnings of broken builds.

Process Goals

• Deploy to any environment by the push of a button
• Revert to a previous deployment by the push of a button.
• Deploy automatically every night on the dev environment.
• Log of each build

Our engine for the automated deployments and continues integration (CI) will be Jenkins. Jenkins connects to your SCM (svn, git, …), downloads the source code and then it builds your project artifacts.  When the build has produced the required artifacts ( <your>-alfresco.war, <your>-share.war) those can be deployed automatically to the different environments (e.g. DEV, TEST, PRODUCTION).

Step 1 – Install Jenkins and Artifactory

Let’s check the installation of required software on the Continuous Integration server. This should ideally be a independent server machine that will run:

• Jenkins Server
• Artifactory Repository

We will now cover the installation of those components on the designated CI server box (LINUX). The Jenkins server will act as an Automation tool that :

• Monitors execution of repeated jobs
• Allows for Continuous Integration
• Test orchestration
• Executes and Tests Releases
• Rolls back and redeploy of previous builds

The releases can be scheduled and run periodically on early development stages

Jenkins will deploy remotely to any Alfresco environment and run remote integration tests. The release reports should be published on the site and recorded as part of the release documentation. Your tests should be self contained and runnable in CI and the must produce intelligible reports. Every task-development must contain the appropriate tests.

Jenkins installation on CI server box (let’s hire a free buttler :))

STEP 1)

sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat/jenkins.repo

STEP 2)

sudo rpm --import http://pkg.jenkins-ci.org/redhat/jenkins-ci.org.key

STEP 3)

sudo yum install jenkins

Jenkins Details

Jenkins will be launched as a daemon up on start. See /etc/init.d/jenkins for more details.
To Start/Stop or stop Jenkins you can use the following command:

sudo service jenkins start/stop/restart

The ‘jenkins’ user is created to run this service. If you change this to a different user via the config file, you must change the owner of

• /var/log/jenkins
• /var/lib/jenkins
• /var/cache/jenkins

The Log file will be placed in /var/log/jenkins/jenkins.log. Check this file if you are troubleshooting Jenkins.

The configuration file /etc/sysconfig/jenkins will capture configuration parameters for the launch. By default, Jenkins listen on port 8080 but we can change this port to 9090 because the default 8080 may be taken by other tomcat instance.

Note that the built-in firewall may have to be opened to access this port from other computers. (See http://www.cyberciti.biz/faq/disable-linux-firewall-under-centos-rhel-fedora/ for instructions how to disable the firewall permanently)

To test your Jenkins server, just type http://<server_ip>:9090 on your browser.

This concludes the installation of Jenkins, official documentation available:

https://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenkins+on+RedHat+distributions

Artifactory installation on CI server box

Start by going to http://www.jfrog.com/open-source/#os-arti and download your zip bundle with artifactory. By the time of this post the latest version is available at http://bit.ly/Hqv9aj

Setup is really easy, we just needed to follow the instructions available on their 1 minute setup video : http://www.jfrog.com/video/artifactory-1-min-setup/

After the installation is completed you can access artifactory by going to :

http://<server_ip>:8081

Source code control strategy

Your project should have

• Standard trunk/branches/tags per project structure
• Maintenance branches
• Tagging per release
• Build number based on SVN revision

This way you can have separate business projects (parties) to run independently on separate SVN roots, whilst allowing the final binary product to be integrated in the main Enterprise Alfresco instance.

Release Artifacts

We will be producing and storing 2 different types of artifacts on our release. The artifacts have 2 categories, deployment artifacts and storage-only.

Deployment artifacts: alfresco.war and share.war and server-configuration.alf

Storage-only artifacts: repo-extension-amp, share-extension-amp, custom-amp

The deployment artifacts can be deployed to any target environment either by the click of a button or by a Jenkins schedule task. Note that storage-only artifacts are self-contained in the deployment artifacts, this is achieved by the dependency management implicit in the alfresco maven Sdk. The next diagram illustrates this.

The server-configuration.alf is a compressed package that follows a specific configuration structure that is part of every deployment. This contains environment specific configuration for the target server, it contains a set of configuration files that are specific to each target server (Dev, Test, Prod) such as, for example, the alfresco-global.properties.

So, before we move on we need to make sure we got that working (Using the alfresco sdk to build the customised war files). Using the psg template you can easily configure this so i will not get on details on this process.

Unit Testing

A set of unit tests is included as part of every code release/deployment to check that code is working and continues to work as intended.

Automated tests must meet very specific objectives:

• Every developer must be able to run the combined collection of all the developer’s tests.
• The continuous integration (CI) server must be able to run the entire suite of tests without any manual intervention.
• The outcome of the tests must be unambiguous and repeatable.

Running this automated unit tests will allows any developer to verify that their current changes do not break existing code-under-test. The team leader or manager showd insist that this happen. It very important as it virtually eliminates the accidental or unintended side-effect problems.

There are 3 key objectives for developers to keep in mind when writing the unit tests:

• Readability: Write test code that is easy to understand and communicates well.
• Maintainability: Write tests that are robust and hold up well over time.
• Automation: Write tests that require little setup and configuration (preferably none).

Integration Testing

Once unit tested components are delivered Jenkins should integrate them together. These “integrated” components are tested to weed out errors and bugs caused due to the integration. This is a very important step in on the Development Life Cycle.

Goal is to avoid programmers developing different components and that some bugs emerge during the integration step. In most projects a dedicated testing team focuses on Integration Testing.

The integration team should be able to

• Step 1: Create a Test Plan
• Step 2: Create Test Cases and Test Data
• Step 3: If applicable create scripts to run test cases
• Step 4: Once the components have been integrated execute the test cases
• Step 5: Fix the bugs if any and re test the code
• Step 6: Repeat the test cycle until the components have been successfully integrated

To write an Integration Test Case you should describe exactly how the test should be carried out. The Integration test cases specifically focus on the flow of data/information/control from one component to the other.

Integration Test cases should focus on scenarios where one component is being called from another. Also the overall application functionality should be tested to make sure the app works when the different components are brought together.

The various Integration Test Cases will be executed as part of the build process.

The release process

Maven maven will be compiling the extension amps and adding them to the correspondent .war files. We will have 3 main artifacts for deployment (alfresco.war and share.war and server-configuration.alf).

The remaining artifacts (.amps) will also be created as part of the release but will not be directly deployed. They are allready part of the war artifacts via the maven dependency management

To execute part of the release process in Jenkins we are using the maven release-plugin.

This plugin is used to release a project with Maven, saving a lot of repetitive, manual work.

Releasing a project is made in two steps: prepare and perform.

Try to prepare a release with maven by running the command : # mvn release:prepare

So what happens when preparing a release (release:prepare) ?

Preparing a release goes through the following release phases:

• Check that there are no uncommitted changes in the local sources
• Check that there are no SNAPSHOT dependencies
• Change the version in the POMs from x-SNAPSHOT to a new version
• Transform the SCM information in the POM to include the final destination of the tag
• Run the project tests against the modified POMs to confirm everything is in working order
• Commit the modified POMs
• Tag the code in the SCM with a new version name
• Bump the version in the POMs to a new value y-SNAPSHOT
• Commit the modified POMs

After a successful build, maven has prepared your release by:

1. Creating a new tag in your SCM with the release source code.
2. Creating a new release version and lock the code for deployment
3. Create automatically a new development version

See more details on the prepare stage of a release on the official maven documentation

Performing a release (mvn release:perform)

After a successful prepare stage, you are now ready to perform your first release by running the command : # mvn release:perform

Performing a release runs the following release phases:

• Checkout from an SCM URL with optional tag
• Run the predefined Maven goals to release the project (by default, deploy site-deploy).
• Upload the artifacts to your configured maven repository

See more details on the perform stage of a release on the official maven documentation

Deploy your artifacts

This is the last step of the release process; it’s where the release manager (will be impersonated by Jenkins) actually deploys the release to the target environment (DEV,QA, PROD), this are the steps that will be performed by Jenkins.

1. Stops your application server (tomcat)
2. Unzips the server-configuration.alf
3. Updates server configuration.
4. Copy the alfresco artifact war file (with your overlays) to your application server replacing the existing copy.
5. Copy the share artifact war file (with your overlays) to your application server replacing the existing copy.
6. Starts your application server (tomcat) with the new release

Note that we have configured (using maven dependency management) both the alfresco and share overlay modules to build their target artifacts (war files) including your amp extensions, meaning that they are already installed on the war artifacts (thanks to the alfresco Sdk).

All this steps are automated in Jenkins as part of the release process, no human intervention is necessary to perform a release.

Automating your build and releases in Jenkins

After configuring Jenkins to automate the main deployment steps (release prepare) and (release perform) we will still need to configure Jenkins to perform the final deployment. The goal is to completely eliminate human intervention in the deployment. We will be using an automated action in Jenkins that will do the following:

• Stops target tomcat (DEV,PRE-PROD or PROD)
• Download the artifacts from artifactory with CURL
• Updates server configuration with alfresco server-configuration.alf
• Copy the alfresco.war and share.war with scp to <tomcatRoot>/webapps replacing the existing versions.
• Starts target tomcat (DEV, PRE-PROD or PROD)

NOTE: On the first run with the new release process you should probably backup the existing alfresco.war and share.war as those artifacts will not be on artifactory yet.

Conclusion

In my opinion, a smart application lifecycle and release process is the foundation for any successful project . I hope you’ve enjoyed this post.

Stay tuned for more posts and advices on Alfresco and ECM

Until then, One Love !

“We’re together and we share, that is what makes us Strong”