TwitterFlickrYouTube

Let your code know… your code

formats
Posté le par Posté dans G33k, Programmes .

Has this ever happened to you ?

… you work on a legacy project, and there is this method called `getAccount` which sounds simple.
But this method is a compound one.
1°) It actually calls a database A to fetch a user Id.
2°) Then completes the data with a B Web service based on LDAP
3°) Then completes it with some extra data based on the account preferences from a service C
4°) It updates the database with the found data (yes, « getAccount » writes something… in a legacy project, you will certainly see that)
5°) It returns all the data.

What if you call getAccount from another point of view ? If you are the caller of the method, you could even do this :

if (getAccount (user) != null) {
String userName = getAccount (user).getName ();
}

Ugh… Did you realize that these 4 lines will call different external services 3 * 2 = 6 times… It has awful performance, and can be harmful (= with side effects) if the method getAccount is not idempotent (which is likely to happen in a legacy project)

What you need is not (really) refactoring. The technical debt is too tough to be attacked right now. What you need IS knowledge about the code. How can you know what is happening in the code ? Think painless, think about code to understand code …

I have created what I call a DiveDeepAnalyzer. The aim of this is to understand which sensitive methods are called directly or indirectly in each method.

https://gist.github.com/libetl/00bc6079b3dd91af55bb6cf8229e942a

Write down the list of sensitive calls (webservices, LDAP, databases, cache, drivers, hardware), and the algorithm will annotate each and every method that is likely to use these calls.

How to ?
1°) Copy this class in your project,
2°) change the params in the main method (you will need to exclude the DiveDeepAnalyzer class itself from the scan)
3°) run the class.

This analyzer uses spoon, an OSS made by a research lab in France (inria). You can find more about spoon at this address : http://spoon.gforge.inria.fr/

My Git hint : a feature per commit

formats
Posté le par Posté dans G33k, Programmes .

Have you had a look on your project git log ? It is often an enumeration of changes which are sometimes not explained correctly… « bugfixes », « some changes », « work in progress », « pull request review ».

wip git log

The log what you would want to see is a log of all features recently added from the JIRA (or any other issue tracker) board.
Moreover, having your features grouped in one commit will be a way to know all of the necessary changes for each issue (because a git diff will reveal all the changes of a feature at once)

Here is how it can be done :

– As a setup, create this command alias in your .gitconfig file (I assume that your main branch is called « master » here.)

[alias]
singleCommit = "!f() { git pull origin master --no-edit && git push && git reset --soft origin/master && git commit -m \"$1\" && git push -f; }; f"

– create a git branch from « master »
– commit everything you want inside this branch, don’t hesitate to commit or push any correction for a mistake you found.
– create a pull request if you need to, review the comments, commit / push, and so on.
– when everything is ready and the code is ready to be merged, find the JIRA code (or your issue tracker issue id) (we will name it MYPROJECT-XXXX)
– issue this command : git singleCommit "MYPROJECT-XXXX a comment for this feature" (change this text to the aim, or the title of the feature)
– if the command fails, your branch is not synced with master. Resolve the conflicts, git add . and git commit, then go back to the instruction before.
– now you can merge. There will be ONE commit for all the content of the branch.

Jackson : Make the most of POJO versus Immutable Objects

formats
Posté le par Posté dans G33k, Programmes .

There is a simple dilemma that comes into my thoughts :
-> I have an object, right… class A { String b;int c}
-> It must be filled automatically thanks to unmarshalling from a json string {"b":"val1","c":2}
-> I want it to be immutable because I don’t want to alter my input, or to trust some insecure hashcode/equals methods.

Therefore : I must have a deserializable class, with final fields, a comprehensive constructor, and a builder pattern.

That sounds awkward with jackson. But is it possible ? Yes, it is, and I’ll prove it.
https://gist.github.com/libetl/853029faf7999c98159f36d1c229c961#file-a-java

Neat, but let’s complicate this attempt a bit. Suppose we have a polymorphic object to be deserialized. It is a paradigmatic issue between Java and Json : Java is typed, Json does not care.
let’s declare this in pseudo DSL : class TripSegment { TransportMode mode; SegmentDetails detailsOfTheSegment }
with enum TransportMode { FLIGHT, TRAIN, BOAT, CAR_SHARING, LOCAL_TRANSIT }
and with several details classes like this one : class LocalTransitDetails implements SegmentDetails {String lineCode;DateTime timeOfDeparture;DateTime timeOfArrival;String departureStationName;String arrivalStationName;}

We want neither to create a TripSegmentBean as a POJO, nor to have a non jackson deserializer… So how to ?
Here is how I could successfully answer to that problem :
https://gist.github.com/libetl/853029faf7999c98159f36d1c229c961#file-tripsegment-java

Now, I can handle my unmarshalled value object as if I built it manually. Now forget about dozer or BeanUtils. Let’s concentrate on the workflow.

curl4j

formats
Posté le par Posté dans G33k, Programmes .

Would you enjoy being able to cURL something within one command line in java ?

Seems sometimes annoying to instantiate an HttpPost, or even an HttpClientBuilder, with its fluent interface.

What I suggest you is to use my code sample, to enhance it and to transform it in your customized cURL command. You can even add custom arguments to it, so the commands will be shorter to type.

How to use it ? Like a normal curl !
Response curlResponse = curl(" -X'POST' -H'Content-Type:application/x-www-form-urlencoded' -d 'token=mytoken&channel=mychannel' 'https://slack.com/api/chat.postMessage'")

Here you are, with a curl response that you are able to read easily with the HttpClient API.
Where is this code sample ? There : https://gist.github.com/libetl/edd72cca5464aa403395029430360344

EDIT : 2016-07-22 : https://github.com/libetl/curl and https://mvnrepository.com/artifact/org.toile-libre.libe/curl

BTW, this blog is now 8 years old, and this is the 200th post… way to go.

Convert List(a,b,c,d) into Map(a->b,c->d) in Java 8

formats
Posté le par Posté dans G33k, Programmes .

Let input be a List of String, you want it to become Map<String, String> where even elements are the keys, and odd elements are the values. Works only when key elements are unique.

Map<String, String> mappedValues = input.stream().collect(Collectors.toMap( Function.identity(), input::indexOf)).entrySet().stream().collect(Collectors.toMap(
entry -> {return entry.getKey();}, entry -> {return entry.getValue() == input.size() − 1 ? −1 : entry.getValue() + 1;})).entrySet().stream().filter(entry -> {return entry.getValue() % 2 == 1;}).collect(Collectors.toMap(entry -> {return entry.getKey();}, entry -> {return input.get(entry.getValue());}));

My opinion on the « Best-so-far » architecture

formats
Posté le par Posté dans Blabla, G33k .

This post is not to talk about something I have made, but to talk about something I have lived with.

First, what is it ?

« Best-so-far » (or BSF) means to work with the latest pieces of software each and every day.

It relies on several IT infrastructure components : a software factory, a source code management, a build scheduler and an integration monitoring.

The software factory

It is a software whose role is to collect all the libraries across a company and to release the binaries for the employees, partners and customers.
It can be implemented by a storage access on a platform, or a machine, or a shared drive.
It also contains archived binaries of the previous versions of each component (for redundancy purposes), but is mostly focused on providing the latest framework chunks, with the latest features and the latest bugfixes.
All of these binaries can be used dynamically to build and release other binaries, and so on…

The SCM (source code management)

It is a set of tools to optimize the branches code management in a team (allowing several users to commit on the same project with the least possible conflicts).
Most of the teams appreciate to use standard tools like git, svn, mercurial, cvs, ClearCase, perforce, bazaar. Some others wanted to create their own scm tool (thinking of adl, http://www.maruf.ca/files/caadoc/CAAWsmQuickRefs/).
It is often managed by having one single integration branch where every team member has to merge in after finishing a task.

The build scheduler

It has two modules : a batch whose role is to execute some actions on some files, and an UI to help the users to administrate the jobs to be run each day.
It uses the latest version from « the SCM » to build it and install the packages on « The software factory ». It also contains some details about the errors found while trying to build or release something.

The integration monitoring

It is a UI tool with red and green lights to inform all of the employees that a build failed or suceeded. The most common standard software existing are Jenkins, Bamboo, CircleCI, CruiseControl, or Drone. Some companies prefer to create their own, with better or worse usability (thinking of ReleaseWork). The integration monitoring collects data from « the SCM », « the build scheduler » and « The software factory » to gives an insight of what is working and what is not.

2. Why do we use it ?

The thinking behind this implementation is to wonder how different dev teams can work across a company on the same product without suffering from waiting for each other. The expectation is to « break the silos », and to get the teams in a rhythm.
Letting a dev team beneath another in terms of software dependencies will help the dependent team to update the subsequent code as soon as possible. For example, a team building a polygon extrusion lib will update every day the dependencies to benefit from the work of the maths library. As a consequence, each lib will get the « Best-so-far » state of each lib when it comes to releasing to the employees, partners, customers. And so on…

3. Is it good ? I don’t know. Is it easy ? NO, it is NOT.

This is where I will express opinions of that practice.
I have worked for a company using that practice since birth. The « break the silos » fantasy is far away from reality.
I have never worked in such a partitioned way of thinking company.
The thing that turned wrong is that the work streams who managed the frameworks were separated from those who managed the BSF architecture.
As a result, a lot of improvements in this architecture landed in a total contradiction of the dev teams practices :
– The team handling the integration monitoring is pressured to release the build. The team members try to identify who is late by seeing who has worked on « the SCM ». They identify some culprits, without any consideration about the team responsibility. This team is not appreciated because it is seen like a toilet lady to the developers. « Have you fixed your issue ? The next release must be submitted by noon. I won’t leave the office until it is fixed and working »
– The team working for « the SCM » delivers some features that are useful for some users. Not all. All the other teams are forced to work on ONE SCM that does not fit their needs (if the company uses cvs instead of git, no one is allowed to work with git, and the Internet proxy must forbid any request sent to github or bitbucket).
– « The software factory » uses a common repository which can work with any kind of project lifecycle tool. You must be able to package Java classes, C++ projects or DSL scripts at once and to send them on the same storage access. Do we know a PL tool which is working on anything ? Yes, but it is really primitive. « make » can do the job. That was the most awkard answer I could ever think of. To understand it, just imagine that I work on a Java project but I have to build and to release my software using the « make » C++ tool. And last that not least, just imagine that the choosen storage can be as clever as a Windows share (that is it, samba share) !
– « The build scheduler » is maybe the tool which is the least risky to maintain, because the responsibility lies on the dev teams. It basically consists in launching some tasks and display the logs. Here, no specific tool to an environment. Windows shells are launched, that is all. Neither any JUnit parsing, nor any performance report.

Last but not least,

what is the most disturbing item for developers in the BSF way ?

→ You are not going to believe it, but it is the principle itself. Working on snapshots itself is at once a good, and a bad practice. Imagine that your company makes yogurts. It wants to rely on a single milk provider, which is considered as the most talented milk provider locally. « We don’t need fallback providers, we use the best one, the best-so-far ». Now imagine that a brutal and unexpected disease cause the death of the whole livestock in one night. How the company is going to adapt to that situation ?
So does a company with a « Best-so-far » architecture. You cannot work with any other provider, and you cannot work with older but functional versions of the frameworks.
The activity stops in the hope that the situation gets better in the future. In my opinion, that explains why my previous company decided to release one product version per year at most.

Replayable REST logs on spring-mvc

formats
Posté le par Posté dans G33k, Programmes .

« What happens on this app ? I cannot figure out what these logs mean… »
That is one of the typical sentences we are likely to say when it comes to debugging in blackbox mode (without access to the code itself).

Although, there is one thing we all know : In a situation, the app creates some pieces of data and then send them to another object (whether it is a person or a service).

In some recent web architectures, with spring, we are likely to send REST requests to other services. We can customize how the data is sent and also how we should trace it.

The idea I want to underline is that a kind of logs that help is a « replayable log », which means that someone looking at the code can try a request once more to understand a failing scenario, step by step, accurately

Then, why not curl logs ? A curl log syntax is really short, understandable and concise. It helps to understand how the data is being sent to another actor.

How to start ?
1. You can copy this gist into your app : https://gist.github.com/libetl/2d2060ce892987a9b7a0bdcb9b234a93
2. Add it inside your RestTemplate in your App Spring Configuration :

@Bean
public RestTemplate thirdPartyServiceRestTemplate () {
final RestTemplate restTemplate = new RestTemplate();
//...
restTemplate.setInterceptors(java.util.Collections.singletonList(new org.toilelibre.libe.logs.CurlLogHttpRequestInterceptor()));
//...

return restTemplate ;
}

My android app is responsive, but my icons aren’t !

formats
Posté le par Posté dans Programmes .

That is one of the most recurring aspects I think of while writing an app on android.
Because of the folder hierarchy, which is not likely to accept vector graphics under Android 5.0 (Lollipops), it can be a repetitive task to scale each icon using each resolution (ldpi, mdpi, hdpi, xhdpi, xxhdpi).

Then I found this simply stupid idea to make each icon once : create it using a vector graphics software (inkscape is free, but there is also adobe illustrator), put it in a separate folder than use a maven goal to generate the drawables.

Here is the plugin configuration I made in my current Android app :
https://gist.github.com/libetl/3582de6bcc8d4d025d4f

Then remove the non-vector pics from your SCM. If your README.md needs it, pick up a free API to render it at runtime : https://cdn.rawgit.com/<github_author>/<github_repo>/<github_branch>/pathtosvg

You can see a working example of this idea on my current project : https://github.com/libetl/singin

Query your app data like you would query a database

formats
Posté le par Posté dans Programmes .

The interesting job in creating an app is to send or receive data to another actor :
– whether it is human or not,
– whether it is UI, JSON/Xml/Yaml or console text,
– whether it is a webApp, a program or a script

Now, what happens between the data exchange and the added value of your software ? Boring data manipulation stuff.

Furthermore, let’s imagine that you need to filter every bank account transaction on your account server with a synthetic data (for example, the impact of an economic crisis on the account itself) ?
public float estimatedBalance (int nbDays, float economicHealth)
-> Most of the time you would not compute that with a simple database, because it can depend on some calculation thanks to an algorithm.

Then, build a data cache and try to query your data inside your application instead…
I have 2 solutions to suggest (with a JVM) :
– Switch to Java 8, and to the easy lambda function syntax :
List result = accounts.filter(account -> account.estimatedBalance(90, -1) < max);
– Use my lib : http://github.com/libetl/prime. You are now able to query some java objects by using a simplified query syntax. You can even pass a String object and therefore construct a query on your own java objects. On the homepage of this project, you will get a preview of what can be done using this syntax.

It can be used as a backend if you don’t need to persist the data really. I am actually thinking if it could be useful to actually save the data in a file…

No EAR, no WAR but a jar for your webapp

formats
Posté le par Posté dans Programmes .

Simplify your deployment package.

– An Ear requires a lot of configuration and nested elements (some DB config xml files, some jars and some wars)
– A War requires some xml files, jsps and some jars
– A jar needs some classes, that’s all.

It would be neat to use the third option, for ease of use and installation. After all, the jar needs no more than itself to start.
An example of a web app project using this tip : jenkins
java -jar jenkins.war (the file can be launched as a java app or as a war webapp without any difficulty)

So, how to install this idea into your webapp project ?
1) Integrate the jetty server dependency into your project :

<dependency>
<groupid>org.eclipse.jetty</groupid>
<artifactid>jetty-server</artifactid>
<version>9.2.14.v20151106</version>
</dependency>
<dependency>
<groupid>org.eclipse.jetty
<artifactid>jetty-webapp
<version>9.2.14.v20151106
</dependency>
</code>

2) Write a server start class :
https://github.com/libetl/GenericRestWebApp/blob/master/src/main/java/org/toilelibre/libe/restwebapp/ioc/webapp/ServerMain.java

3) Add a plugin to bundle your sources into a jar :

<plugin>
<artifactid>maven-assembly-plugin</artifactid>
<configuration>
<archive>
<manifest>
<mainclass>org.toilelibre.libe.restwebapp.ioc.webapp.ServerMain</mainclass>
</manifest>
</archive>
<descriptorrefs>
<descriptorref>jar-with-dependencies</descriptorref>
</descriptorrefs>
<finalname>${project.build.finalName}</finalname>
<appendassemblyid>false</appendassemblyid>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package

4) Type `mvn install` and wait. Your jar is now ready to be started with `java -jar yourproject.jar`