<< Lightning is moving onwards to the ASF | Home | Properties Proposal >> | Conferences & TechTalks Calendar | Legal Notice | Google+ Profile | Twitter Page | My Profile Page

Jenkins and the Git Branch Selection

How to make Jenkins aware of Git branches

This time I want to talk about something "totally" different. Something just from work.

At work we recently changed the revisionsystem to Git to meet our requirements in feature driven development using a lot of feature branches.

To fulfill our high standards in quality and stability we moved over to a silently different workflow where QA tests every feature branch before it'll merged in the final release. The day where the deployment meeting takes place it'll decided which branches are stable and will be merged into the actual release branch. To make it more convenient for our QA to build, deploy and test branches we setup a bunch of QA servers (one server for each QA tester) and gave them the chance to build any branch and deploy it to a choosen QA server.

Since Jenkins currently has no real Git branch support (beside giving the branchname by yourself) we had to figure out how to tell Jenkins which branches are available. To do so we made a small shellscript which collects all of the feature branches into some properties file that later will be read by Jenkins.

In this blogpost I'll show the way on how this can be done since currently there's no complete documentation but a lot of hints that needs to be collected and made up in the right order.

So let's start which how do we find all feature branches?

We did a clean naming schema to be searched for - in our case (since we use Atlassian JIRA) it was quite easy to find it - just using the JIRA project key. So if your project would be "My fantastic new project" your JIRA could look like "MFNP" and any JIRA task is a combination of the project key followd by a task id, e.g. "MFNP-1337". So we named the feature branches after the keys including the summary written in uppercase "MFNP1337_SOME_NEW_IMPORTANT_ADDITION". In this case everyone knows the JIRA key and can search for further information on that special branch.

Ok now we know how the branches are named and we see that searching for feature branches just means "looking for branches starting which "MFNP". This is, thanks to the great toolset Linux provides, just a matter of calling the right commands and pipe output from one to the next.

So let's enter the shell of the Jenkins buildserver and start over with cloning the interesting Git repository (a bare repository should work as well but we haven't tested).

 

someone@jenkins:/home/jenkins$ git clone http://git-server/repo/mfnp mfnp
someone@jenkins:/home/jenkins$ chown -R jenkins:jenkins mfnp

 

We know have a now repository clone we will use for further work which our shellscript "/home/jenkins/build-branches.sh". So how does it look like?

 

#!/bin/bash
cd /home/jenkins/mfnp
git fetch --all
git pull
git remote prune origin echo -ne "branches=" > /home/jenkins/branches.properties
git branch -r | awk '/origin\/MFNP/{printf "%s,", $1 }' | head -c -1 >>/home/jenkins/branches.properties
echo "" >> /home/jenkins/branches.properties

 

This script will update the actual repository using the latest available changesets and grabs for all remote branches using the naming pattern "origin/MFNP" since we're searching only for remote branches. The result is piped into a properties file called "branches.properties" using the property key "branches=" which will later be needed by Jenkins.

To update the properties file on regular basis just add it as a cronjob.

 

# m h dom mon dow user    command
* * * * * jenkins /home/jenkins/build-branches.sh

 

Now the build-branches.sh file will be executed every minute which is fast enough to quickly find new feature branches in Jenkins. So let's move on the the last stage - the Jenkins configuration.

We're using Jenkins 1.488 and start by installing some of the required plugins which are at least "Jenkins Parameterized Trigger plugin", "Git Plugin", "Git Parameter Plug-In", "Extended Choice Parameter Plug-In".

After the plugins are installed we go on which creating a new job (or copy one of the existing configurations - this is our case since we have a fairly complex build) and change it the way we need to manage build by using the branch selection. We give it the simple name "GIT-BranchSelection" and toggle "This build is parameterized". In the revealing properties we add our previously created properties file. We name the property "branches", give it a nice description and the type "Single Select", as well as adding the full path to the properties file in "Property File" and the property key which we defined as "branches" in the shellscript.

In the Sourcecode Managenemt we select Git as our repository and enter the URL of the remote repository (Jenkins will use it's own clones while building, it is not using the one created above!) and leave Name and Refspec blank but we fill in the previously configured property into Branch Specifier by setting it to "${branches}".

Save the new configuration and you're done. If you click on "Build Now" it should look quite similar to the next picture - just select the branch to be build and kick it.

 

Related Posts
The Hazelcast Incubator
snowcast - Hazelcast Client and the snowcast logo
snowcast - Migration and Failover


Categories : Projects, Other


Avatar: Noctarius

Re: Jenkins and the Git Branch Selection

 PS: All of this asumes that Jenkins is running as useraccount "jenkins".

Avatar: Rahul

Re: Jenkins and the Git Branch Selection

Hello Chris

The post is simply great.!! I had a few concerns listed by my developer team..here are few unresolved one's..

You post works absolutely fine when i choose "Build now".

What will be the case if i have a poll request?

It takes the default value i have given. What if there is changes on all the branches i have listed? It will still poll only my "default value" and leave rest of the branches.

My second question - Sometimes developers click the "build now" option without confirming. Do you have any idea...how can we have "space/blank" in select menu so we can force the user to go through the options before clicking "build now"

Avatar: Noctarius

Re: Jenkins and the Git Branch Selection

 Hi Rahul,

First sorry for the late response, were on JavaOne in SF for the whole week.

On the first question:
I guess there should be an option to give the branch name to build as an query parameter but I would need to have a look as well. Just look at the documentation of the "<span style="font-family: 'Trebuchet MS', 'Lucida Grande', Tahoma, Verdana, Arial, sans-serif; font-size: 13px; line-height: 19px;">Git Parameter"</span><span style="font-family: 'Trebuchet MS', 'Lucida Grande', Tahoma, Verdana, Arial, sans-serif; font-size: 13px; line-height: 19px;"> </span>plugin.

For the second question:
Never tried something like that, maybe just make the first value in the properties file blank? No idea so far, sorry ;-(

Chris

Avatar: Anonymous

Re: Jenkins and the Git Branch Selection

This is very nifty, thanks for writing it up.

Discovered an alternate way using "git ls-remote" to get a list of branches, without maintaining a local clone:

git ls-remote -h ssh://user@host/repo.git | awk '{print $2}' | sed s%^refs/heads%origin%

Gives a list of branches in the form

origin/master

origin/my-feature

etc.

Avatar: avinash

Re: Jenkins and the Git Branch Selection

I have setup jenkins that will fetch the git repo every time the build is trigerred. Right now our requirement is that developers want to push there developed code into the jenkins server and test it before pushing it to the central repo. I have googled and found that we can jenkins can trigger a build if the changes are pushed to the central repo. But, i am not able to figure out how to set the push from local branch to jenkins server and trigger the build.

Avatar: Noctarius

Re: Jenkins and the Git Branch Selection

For a remote repository you could use post commit hooks ~/repo/.git/hooks to start a build using Jenkins REST API but I'm not sure if this will work for local repositories (at least there seems to be no reason for that).

Re: Jenkins and the Git Branch Selection

Hello,

your article helped me, so posting here what I made. Actually this can be done simplier way by short groovy script. Here's link to mine post.

Avatar: Noctarius

Re: Jenkins and the Git Branch Selection

 Sorry but your link wasn't in the comment but since I'm not a Groovy guy it would be great to see the script.

Avatar: tugelblend

Re: Jenkins and the Git Branch Selection

Created a small script to create the branches properties file. Maybe some of you will find this useful.

https://github.com/tugelbend/gitbranch2property

Avatar: Noctarius

Re: Jenkins and the Git Branch Selection

 Pretty cool thanks for contribution :)

Re: Jenkins and the Git Branch Selection

Hi All,

 

Can you please help me to get the GIT revision number by the automatic build I need the hash value for further tasks I am not being able to get that What I get through .bat file is the parent commit id but I need the current which is generated after the .bat process

 

Thank you,

 

 

 

<span> </span> 

Avatar: Noctarius

Re: Jenkins and the Git Branch Selection

Not really sure what you're searchng for. Which commit id do you actually want? The one that is built?

git rev-parse HEAD

Where HEAD needs to be the name of the branch you build

Avatar: Noctarius

Re: Jenkins and the Git Branch Selection

 Found another amazing alternative with real time updating git branches (on the fly):

http://domando-los-bits.weebly.com/blog/jenkins-and-branch-selection-in-git

 

Re: Jenkins and the Git Branch Selection

come on, run the script every minutes to get a list branch name. not a good solution. 


Add a comment Send a TrackBack