git branches : easy way out (Not for purists)

Bouncing on this old article about git (http://libe.toile-libre.org/?p=1318), I give you some more commands to help you work with git in an efficient way…
Warn : this is useful in the case where you work on a centralized git version control, and when the work is stored in the ‘origin’ remote server.


[alias]
singleCommit = "!f() { git pull origin develop --no-edit && git push && git reset --soft origin/master && git commit -m \"$1\" && git push -f; }; f"
singleCommitAs = "!f() { git pull origin develop --no-edit && git push && git reset --soft origin/master && git commit -m \"$2\" --author \"$1\" && git push -f; }; f"
singleCommitFrom = "!f() { git pull origin $1 --no-edit && git push && git reset --soft $1 && git commit -m \"$2\" && git push -f; }; f"
addToSingleCommit = "!f() { git add \"$1\" && git singleCommit $(git log --format=%B -n 1); }; f"

git singleCommit « message » : group together all the commits made in the actual branch, ahead of master, into one, and updates the current branch
git singleCommitAs « someone » « message » : same as git singleCommit, with the ability to commit for someone else
git singleCommitFrom « branch » « message » : same as git singleCommit, but synchronizes the work with a branch other than master
git addToSingleCommit « path » : adds path to the version control, grab the last commit message, then group all the commits ahead of master together into one commit.

Android : what if « git push » becomes your Play Store publish command

Ever wondered why you need these repetitive tasks each time (in these three pictures below) ?
apk from intellij
intellij2

playstore

Not only you run the risk of losing time, but you can go wrong because of a lack of concentration (by publishing in prod instead of alpha), or you can even forget to publish your app intermediary updates. Your users might lose patience while waiting for some fixes, or some features.

Now think about how devops teams works. The team developers change a piece of code, than ensure nothing is broken, then let the software factory prepare, package, deploy and monitor the program.

How to achieve this, without money nor time ?
Start by using :

  • a source control management : either a private one, or BitBucket, or GitHub.
  • an Application Lifecycle Management tool : maven or gradle
  • a Continuous Integration service : sure there is Jenkins, but you can use a free cloud CI : http://travis-ci.org
  • an account of the Play Store of course.


This example below will use GitHub + maven + Travis :

  • go to your play store params > API Access > Create a service account. Display this item in the Google Developers Console and download the .p12 file
  • make sure you know where your .keystore file is to sign your APK. If you haven’t got one yet, please visit the android developers help https://developer.android.com/studio/publish/app-signing.html
  • add your repository to travis and configure a .travis.yml file in the root folder of your repository
  • install the travis client with gem install travis
  • base64 encode both the keystore and the p12 file, then concat them together (cat key.p12.b64 account.keystore.b64 > credentials.b64). Do not place the file in your repo, instead move it to the parent folder.
  • ask travis to add it securely without comprimising the security (travis encrypt-file ../credentials.b64 --add). You can add the credentials.b64.enc securely to your repository without fear.
  • have a look at this travis build file : https://github.com/libetl/singIn/blob/alpha/.travis.yml. We have the openssl command that was asked to be added, but the ALM must be aware of both keystore and keypass passwords.
  • There is a tip to unpack a single .enc file into both the keystore and the .p12 file (.travis.yml lines 19-20). Split the file on the == token to know where the p12 file ends and where the keystore begins. base64 decode them both, and you are set…
  • on the maven command line, we ask to proceed to the tests (org.jacoco:jacoco-maven-plugin:prepare-agent test org.jacoco:jacoco-maven-plugin:report org.eluder.coveralls:coveralls-maven-plugin:report), then the packaging phase (android:manifest-merger install android:zipalign) then the publishing phase (android:publish-apk), and we pass as parameters the keystore and keypass secrets right from the travis hidden vars settings
  • now we just need to setup the android-maven-plugin so it is aware of how it must be configured.
    • The android manifest versionCode must be incremented for each publish, therefore we rely on the $TRAVIS_BUILD_NUMBER value for the counter.
    • We ask the plugin to override the manifest from the repository (by specifying the output file name of the manifest-merger target).
    • We must fill in the service account e-mail and the path to the .p12 file
    • We must provide a path to the keystore, with the following secrets given as properties : keystorepass, keypass
    • Have a look at this pom.xml file to know more about the challenge : https://github.com/libetl/singIn/blob/alpha/pom.xml
  • publish a first version of your app if it is not already done (by the classic workflow with your IDE or bash commands)
  • change any file on your git repository, commit the changes, push them…
  • go to your travis app job, and see it working : https://travis-ci.org/libetl/singIn/builds/171238769

Wait about 10-15 minutes, and your app is deployed on the play store automatically :
playstore2

From now on, each time you change something in your codebase, will be published on the Play Store… painless.
Now you can say : to deploy on the Play Store, just do git push

My Git hint : a feature per commit

Have you had a look on your project git log ? It is often an enumeration of changes which are sometimes not explained correctly… « bugfixes », « some changes », « work in progress », « pull request review ».

wip git log

The log what you would want to see is a log of all features recently added from the JIRA (or any other issue tracker) board.
Moreover, having your features grouped in one commit will be a way to know all of the necessary changes for each issue (because a git diff will reveal all the changes of a feature at once)

Here is how it can be done :

– As a setup, create this command alias in your .gitconfig file (I assume that your main branch is called « master » here.)

[alias]
singleCommit = "!f() { git pull origin master --no-edit && git push && git reset --soft origin/master && git commit -m \"$1\" && git push -f; }; f"

– create a git branch from « master »
– commit everything you want inside this branch, don’t hesitate to commit or push any correction for a mistake you found.
– create a pull request if you need to, review the comments, commit / push, and so on.
– when everything is ready and the code is ready to be merged, find the JIRA code (or your issue tracker issue id) (we will name it MYPROJECT-XXXX)
– issue this command : git singleCommit "MYPROJECT-XXXX a comment for this feature" (change this text to the aim, or the title of the feature)
– if the command fails, your branch is not synced with master. Resolve the conflicts, git add . and git commit, then go back to the instruction before.
– now you can merge. There will be ONE commit for all the content of the branch.

Yet another GIT PS1 command

Whenever you develop something and make a mistake when committing (or pushing some code) on the wrong branch onto your SCM server, think of the classic tip of adding the branch name into your shell prompt.

This tip can display each time in the shell the checked out branch and can display some additional info (do you have any locally untracked file, are you behind or ahead of the origin).

Here is a version I made at my work, in 2 flavors :
– a colored version, if your terminal supports it easily (which is unfortunately not the case with konsole or xterm) :
coloredgitps1.txt (rename it to coloredgitps1.bashrc)
coloredgitps1
– a b&w version, :
whitegitps1 (rename it to whitegitps1.bashrc)
whitegitps1

How to integrate that ?
1°) move the git bashrc file to etc
2°) at the begining of your bashrc file, add the following line of unix script :
. /etc/coloredgitps1.bashrc
or
. /etc/whitegitps1.bashrc
3°) append $(__git_ps1) to the PS1 var
4°) restart your terminal

JIRA and git : how to get a changelog automagically

Install « jilla » (a nodejs program) onto your CI server. This is a command line jira access.
Then change the parameters below in this script.

This script applies if you work using a pull Request system and version branching (for example if there are branches called release/x.x.x)


#!/bin/bash
#change the category here
JIRA_CATEGORY="CAT"
#change your project here
JIRA_PROJECT="PROJECT"
#write your JIRA_URL here
JIRA_URL="http://www.myjira.com/"
RELEASE_PREFIX="release"
PR_MERGE_COMMENT_PREFIX="Merged in"

BUG="\033[37;41m\u25BA\033[1;37;41mbug\033[0m\033[37;41m\u25C0\033[0m"
DOC="\033[37;44m\u25BA\033[1;37;44mdoc\033[0m\033[37;44m\u25C0\033[0m"
TEST="\033[37;42m\u25BA\033[1;37;42mtest\033[0m\033[37;42m\u25C0\033[0m"
OTHER=\033[37;46m\u25BA\033[1;30;46mother\033[0m\033[37;46m\u25C0\033[0m"
STORY="\033[37;45m\u25BA\033[1;37;45mstory\033[0m\033[37;45m\u25C0\033[0m"
FEATURE="\033[37;43m\u25BA\033[1;30;43mfeature\033[0m\033[37;43m\u25C0\033[0m"
JIRADESCR="~/.jiradescr"
SEP=";"

snapshotVersion (){
MILESTONE=$(echo $1 | cut -d'.' -f1);
MAJOR=$(echo $1 | cut -d'.' -f2);
MINOR=$(echo $1 | cut -d'.' -f3);

MINOR=$(expr $MINOR + 1)
NEXTVERSION=$MILESTONE.$MAJOR.$MINOR
echo $NEXTVERSION-SNAPSHOT
}

lastMerge (){
if [ -z "$(echo $1 | grep SNAPSHOT)" ];then
echo $(git rev-list origin/$(echo "$1") --merges | head -n 1)
fi
}

applyVersion (){
echo "$1 $2"
}

listVersions (){
VERSIONS="$(git for-each-ref --sort=-committerdate --format='%(refname:short)' | grep origin/$RELEASE_PREFIX | cut -d'/' -f3 | tac)"
VERSIONS="$(echo "$VERSIONS")"
echo "$VERSIONS"
snapshotVersion $(echo "$VERSIONS" | tail -n 1)
}

shiftList (){
echo "$1" | tr '\n' '-' | cut -d'-' -f2- | sed 's/-$//' | tr '-' '\n'
}

printType (){
TYPE=$1
if [ "$TYPE" == "Bogue" ] || [ "$TYPE" == "Bug" ];then
echo -n "$BUG"
elif [ "$TYPE" == "Specification" ];then
echo -n "$DOC"
elif [ "$TYPE" == "Other task" ];then
echo -n "$OTHER"
elif [ "$TYPE" == "Acceptance Test" ];then
echo -n "$TEST"
elif [ "$TYPE" == "Récit" ] || [ "$TYPE" == "Technical Story" ];then
echo -n "$STORY"
elif [ "$TYPE" == "Implementation" ] || [ "$TYPE" == "Development" ];then
echo -n "$FEATURE"
else
echo -n "$TYPE"
fi
}

changelog () {
VERSIONS=$(listVersions)
LAST=$(echo "$VERSIONS" | head -n 1)
echo "$VERSIONS" | while read VERSION;do
if [ "$VERSION" != "$LAST" ]; then
VERSIONMERGE=$(lastMerge release/$VERSION)
LASTMERGE=$(lastMerge release/$LAST)
echo -e "\033[1;4m$VERSION :\033[0m"
TICKETS="$(git --no-pager log --oneline --decorate --color $LASTMERGE..$VERSIONMERGE | grep $PR_MERGE_COMMENT_PREFIX | grep -o "\($JIRA_CATEGORY$JIRA_PROJECT\+-[0-9]\+\)" | sort | uniq)"
if [ -z "$(echo $TICKETS)" ];then
echo "Nothing"
fi
echo "$TICKETS" | while read JIRA; do
if [ ! -z "$(echo $JIRA)" ];then
LINE="$(grep $JIRA $JIRADESCR)"
if [ ! -z "$LINE" ];then
TYPE="$(echo "$LINE" | cut -d"$SEP" -f2)"
TITLE="$(echo "$LINE" | cut -d"$SEP" -f3)"
else
DESCRIBE=$(jilla describe $JIRA)
TITLE=$(echo "$DESCRIBE" | grep "^Summary: " | head -n 1 | sed "s/^Summary: //")
TYPE=$(echo $(echo "$DESCRIBE" | grep 'Type:' | head -n 1 | cut -d':' -f2))
echo "$JIRA$SEP$TYPE$SEP$TITLE" >> $JIRADESCR
fi
printType "$TYPE"
echo -n " $TITLE "
echo -n "($JIRA_URL"
echo -n $JIRA
echo ") "
fi
done
LAST=$VERSION
fi
done
}

echo "$(changelog)"

You will get this display in a console :
changelog

You can integrate that into your webApp or e-mail it at each release date.

git & maven – Voir automatiquement les numéros de version des branches sur git

Hello,

Avez vous un projet sous git avec des tags 3.0.1, 3.3.2 mais vos projets affichent un pom.xml avec un triste 0.0.1-SNAPSHOT depuis le début de l’historique git ?

Si vous créez des tags/branches dont le nom commence par ‘release/’, avec ce script, vous pourrez savoir dans quelle version vous êtes sur chacune de vos branches.

#!/bin/bash
VERSIONS="$(git for-each-ref --sort=-committerdate --format='%(refname:short)' | grep origin/release | cut -d'/' -f3 | tac)";

snapshotVersion (){
MAXMILESTONE=0;MAXMAJOR=0;MAXMINOR=0;

for VERSION in $VERSIONS;do
MILESTONE=$(echo $VERSION | cut -d'.' -f1);
MAJOR=$(echo $VERSION | cut -d'.' -f2);
MINOR=$(echo $VERSION | cut -d'.' -f2 | cut -d'-' -f1);

if [ "$(expr "$MILESTONE" : '^[0-9]\+$')" -gt 0 ];then
if [ "$MAXMILESTONE" -lt "$MILESTONE" ] ||
[ "$MAXMILESTONE" -eq "$MILESTONE" -a "$MAXMAJOR" -lt "$MAJOR" ] ||
[ "$MAXMILESTONE" -eq "$MILESTONE" -a "$MAXMAJOR" -eq "$MAJOR" -a "$MAXMINOR" -lt "$MINOR" ];then
MAXMILESTONE=$MILESTONE;
MAXMAJOR=$MAJOR;
MAXMINOR=$MAXMINOR;
fi
fi;
done;

MAXMINOR=$(expr $MAXMINOR + 1)
MAXVERSION=$MAXMILESTONE.$MAXMAJOR.$MAXMINOR
echo $MAXVERSION-SNAPSHOT
}

pomVersion (){

THIS_BRANCH=$1
REVS="$(git rev-list remotes/origin/$THIS_BRANCH)"

POMVERSION=$SNAPSHOTVERSION

CONTINUE=1
for VERSION in $VERSIONS;do
MILESTONE=$(echo $VERSION | cut -d'.' -f1);

if [ "$(expr "$MILESTONE" : '^[0-9]\+$')" -gt 0 ];then
if [ -z "$(echo "$REVS" | grep "$(git rev-list origin/release/$VERSION --merges | head -n 1)")" -a $CONTINUE -eq 1 ];then
CONTINUE=0
POMVERSION=$VERSION-SNAPSHOT
fi
fi
done

if [ ! -z "$(echo "$THIS_BRANCH" | grep "release/")" ];then
POMVERSION=$(echo "$THIS_BRANCH" | cut -d'/' -f2)
fi

echo $POMVERSION
}

applyVersion (){
echo "$1 $2"
}

listBranches (){
git branch -a | grep remotes | grep -v 'master' | sed 's/ remotes\/origin\///'
}

SNAPSHOTVERSION=$(snapshotVersion)

echo "____________________________________________________________________________________"
listBranches | while read BRANCH;do
POMVERSION=$(pomVersion $BRANCH)
applyVersion "$(printf "%50s\n" "$BRANCH") | $(printf "%30s\n" "$POMVERSION")"
done
echo "____________________________________________________________________________________"

Vous pourrez éventuellement ensuite faire un git rebase pour inclure des commits qui vont changer les versions des pom successives en fonction de ce que vous voyez dans le résultat de ce script.
/!\ Attention, un git rebase, c’est dangereux, et ça a des impacts sur les branches en aval.