Raw js imported as a real module

Modern javascript libraries are all made of modules. They can be plugged inside any project : web, server, batch or program.

That would be really straightforward to include any dependency as a single line of configuration inside a package.json file.

Unfortunately, some teams want to expose web javascript code in a raw format : no ‘export’, no ‘module.exports’, and even nothing injected inside ‘global’ or ‘window’. ES5-like format.

Naive var(s) and function(s) are declared. Private companies do so, and they minify/obfuscate their code.
Some teams even make a javascript code generator when the source code simply cannot be shared with other companies.

//this is a raw javascript file
var foo = "some important data"; 

function bar(){ 
  return "successful."; 
}; 
//This script is unmodifiable
//You are expected to use these symbols, without any access to them.

A simple option would suggest you to copy/paste the library inside your app.
Too bad it affects a lot the quality of your delivery : big chunks, troubles when the library is upgraded but not in your code, ‘package.json’ not explicit (some libs are not visible in the manifest although they are effectively included)

I have found another solution that I wish to share.
1°) Bundle that library in package.json (you can set the URL inside the version).
2°) Then import that lib in your code, but not with the ‘import’ or ‘require’ shortcut… write that boilerplate code :

//it is like an es6 import, excepted the fact that it also collects non exported variables and functions
const _imports = moduleImports => Object.assign({}, ...moduleImports.map(moduleImport => {
    //build a closure outside the non-structured code
    const theModule = new (eval('(function(){' + fs.readFileSync(`${moduleImport}.js`) + ';this.eval = (name) => eval(name)})'))()

    //pick the functions and var names
    const stringified = theModule.constructor.toString()
    const functions = (stringified.match(/function\s+([a-zA-Z_][a-zA-Z_0-9]*)\s*\(/g)||[]).map(oneFunction => oneFunction.replace(/^function/g, '').replace('(', '').trim())
    const variables = (stringified.match(/(?:var|let|const)\s+([a-zA-Z_][.a-zA-Z_0-9]*)\s*=/g)||[]).map(oneFunction => oneFunction.replace(/^(var|let|const)\s/g, '').replace('=', '').trim())

    //resolve the references inside the module
    return Object.assign({}, ...functions.concat(variables).map(symbol => {try{return {[symbol]:theModule.eval(symbol)}}catch(e){}}).filter(symbol => symbol))
}))

3°) Call the _imports function where needed

// top of your js file
const { foo, bar } = _imports(['my-es5-module-1', 'my-es5-module-2'])
// your other imports
import { Component } from 'react'
// and it works...
console.log(foo)
console.log(bar())

There you go, use the invisible members of the lib like if they were really exported.
In this example, it also works when the lib is a set of javascript files that are dependent on each other but without import.

By the way,
Maybe you noticed that the layout of the blog has changed once again.
I am pretty sure that my visitors will prefer some rich content rather then a beautiful decoration, given the fact that the blog no longer talks about artistic concerns.
That is why I chose the most sober design I could find.

How to parse a phone number

There are tons of different writings for a phone number. The simplest one is the spoke one. A list of numbers that can identify several targets in the world.
It is not unique because of the country indicator that is used for disambiguation, what we call the « international format ». Even after that, behind a phone number can be hidden several phones thanks to the extension.

To all of to who think the phone format is a quite conventionnal one, how would your program read this one ?
+1 400-CALL-0ME-NOW
or this one ?
49 345 67890
or this one ?
(0) 33 1 400 500 600,700

Google made an attempt to parse a phone number accurately thanks to the phone number util library. Among others, it is able to recognize any phone number format, to guess which operator it is, and to isolate the country prefix from the rest.

Trouble exists when you want to record the region indicator as well. It does not exist in every country and it is a tough work when the input is all in one block (no space).

I spent several hours to parse the number in a reliable way. And here is the result : https://gist.github.com/libetl/2122945b868e5c73198ab0a4f6f4fb85

Every input will get split into 4 parts, whatever the complexity of the format is :

country prefix, region prefix, trunk, extension.

Notice that the mobile prefix won’t be included in the region prefix, it usually identifies a device rather than a place.

Tableau de visualisation SNCF (sorry, post in french only)

Je dois dire que je suis un habitué de l’app fusion de la sncf sobrement appelé « SNCF » https://play.google.com/store/apps/details?id=com.sncf.fusion.
Et rien à dire, elle est plutôt complète puisqu’elle offre la possibilité de visualiser les horaires par gare ou par ligne, où que l’on se trouve.
Elle permet aussi d’avoir toutes les infos « publiées » (donc pas toutes évidemment) sur les difficultés de circulation.

Mais une fonctionnalité, pourtant visible dans toutes les gares, n’est pas offerte à l’identique sur l’app : le tableau de visualisation.
Sur l’app, on voit les horaires, mais il faut toucher chaque passage pour voir la desserte.
J’aimerais bien mieux avoir un écran identique à celui en gare sur mon smartphone.

Certains ont déjà fait une version plutôt fidèle à la réalité : http://monrer.fr, mais ça ne marche qu’autour de paris.

Donc j’ai essayé de la refaire en sauce react-native. Vous pouvez y jeter un coup d’œil à la version expo (n’étant pas encore définitive je ne l’ai pas publiée sur play store) : https://expo.io/@libe/sncf-le-panneau
Il existe encore un ou deux bugs : des fois la desserte est affichée totalement, y compris les gares de provenance, et des fois le défilement de la desserte n’est pas complètement fait (on voit des bouts de la ligne suivante)

Amusez vous bien avec ce prototype.
screenshot_20170731-094208_480_360

How to split an input text into arguments

Hi dear Apache commons-cli lover,

You know certainly how to parse a list of arguments into options.

But possibly you may want to provide the same as a pure JVM method ; what about the : callWith(String theArgs) ?

Might sound impossible with a commons-cli library.
You probably have thought about splitting around whitespace tokens, but you forgot that arguments can contain whitespaces characters themselves.

Then how to achieve the parsing from a simple text ?
My solution is to build a powerful regex, using an online IDE called regex101.https://regex101.com/r/B5Rga8/16/tests

An argument can be this, « that », ‘that’, ‘or  »even » that’. Use the regex here from the hyperlink into your code to build a list of arguments from a plain text command line.

react & react native at once : 1 app = 1 program ?

One question everyone is likely to ask one day, while using the react framework, is :

« can I use the same app on a browser AND on a mobile device without rewriting the app ? » In other words, is the principle « write once, run everywhere » respected ?

The short answer is no, and the longer one is « yes but ».
There is a quite simple to trick to make a react app work on different devices.

On a react-web app, you will write this kind of render :
<div>

<div>This is some text</div>
<img src= »logo.png » alt= »this is a picture » />

</div>
Whereas on a react-native app, the layout would look like this one :

<View><Text>This is some text</Text>
<Image source= »logo.png » alt= »this is a picture »/></View>

Quite similar, isn’t it ? But problem is that you cannot declare a common declaration for both environments.

…Or you would need to branch with an if instruction. That will work, but it will make the code less concise and you will have some ‘copy/paste’ sessions between your components.

I found a better option, not the best one (I am more of a beginner in react) : Just provide the same language to both environment. Declare a provider in each environment, and make it provide exactly the same markup with some common attributes.

This is how your code looked like before :

import {View, TextInput} from 'react-native'
export default class Console extends Component {
render(){return (
 <View className="console" style={{height:'100%', width:'100%', flex: 1}}>
 <TextInput id="commandLine" value={this.state.command}/>
 </View>
) /* and it only works on a mobile */
}}

And this is how you can change it to make it work on several envs :

export default class Console extends Component {
render(){
const [View, TextInput] = ['View', 'TextInput'].map(componentName => this.context.componentsProvider(componentName))
return (
 <View className="console" style={{height:'100%', width:'100%', flex: 1}}>
 <TextInput id="commandLine" value={this.state.command}/>
 </View>
) /* and it only works on a mobile */
}}

And last but not least, declare a provider which will gives you these elements :

import { Component } from 'react'
import PropTypes from 'prop-types'

export default class ComponentsProvider extends Component {
    getChildContext() {
        return {
            componentsProvider: this.props.impl,
        }
    }

    render() {
        return this.props.children;
    }
}

ComponentsProvider.propTypes = {
    children: PropTypes.element,
    impl: PropTypes.func.isRequired
}
ComponentsProvider.childContextTypes = {
    componentsProvider: PropTypes.func.isRequired,
}


Your app will need to be surrounded by your provider, in each version :

<ComponentsProvider impl={(name) => nativeMapping(name) || ReactNative[name] || MaterialUi[name] || ReactNative.View}>
    <MaterialUi.ThemeProvider uiTheme={uiTheme}>
        <App muiTheme={uiTheme} />
    </MaterialUi.ThemeProvider>
</ComponentsProvider>


Now you are ready to deploy the same app on different devices with very little effort.

Please have a look at my NON-HELLOWORLD example : https://github.com/libetl/amadeus-console
I deployed the same app on google chrome and android. The app is what you see on the screen when you board in a plane or when you complain about your flight booking. The UI is a kind of a console which can help you manage all your bookings.

Webpack2 : an example, a real one ?

« Has this ever happened to you ? »
-> You work in a non open minded company, or a company willing to make some progress, but behind in tech.
-> The front part of your stack is in (Java/PHP/Python/You name it) + (naive javascript or some former trendy framework like jquery / backbone / handlebars), and your build tool is as shifted from reality as maven or ant can be.
-> You think you deserve more than to maintain such a legacy stack and to write IE9 compliant code

My solution was to think with Webpack2.

Rather than explaining you what it is, let me define you what it is not :

– Webpack is not (really) a build tool. It would rather be « a module emulation builder ». In other words, it uses all of your resources to bundle them into very few « big files ».
Which means that you will no longer see this :
[Long resources list loading]
…but actually two resources : a webpage and a big script. Which will save the patience of some mobile users, and avoid repetitive server requests to load the same code over and over again.

– Webpack is not a code minifier. It groups css, medias (sound/picture), javascript resources at once. Don’t call that a minifier.

– Webpack is not a compiler. It uses a grammar to filter which resources must be handled, and some rules to declare how they should be transpiled/compressed/grouped/moved/compiled. Actually, it uses some loaders that are able to do the compiling job. Like babel, to be able to code using the latest javascript version (ES2017) while targeting IE9… Like handlebars, even if it is old, if you like it.

– Webpack is not a file manipulation tool. But it can add/change/delete some text inside files, it can copy/cut/paste/zip/unzip some resources. Some plugins are dedicated to such operations, like the CopyWebpackPlugin, or the ExtractTextPlugin.

Enough talk, let’s take an example stack : a « mavened » java webapp with the following js stack :
handlebars, backbone, underscore, jquery, and an invasive homegrown framework to avoid some non homogeneous display.
1°) Start by removing all of the DIY maven plugins. Write this instead in the plugins section:

<plugin>
<groupid>com.github.eirslett</groupid>
                <artifactid>frontend-maven-plugin</artifactid>
                <version>1.3</version>
                <executions>
                    <execution>
                        <id>install node and yarn</id>
                        <goals>
                            <goal>install-node-and-yarn</goal>
                        </goals>
                        <phase>generate-resources</phase>
                        <configuration>
                            <yarnversion>v0.23.2</yarnversion>
                        </configuration>
                    </execution>
                    <execution>
                        <id>yarn install</id>
                        <goals>
                            <goal>yarn</goal>
                        </goals>
                        <phase>generate-resources</phase>
                        <configuration>
                            <arguments>install</arguments>
                        </configuration>
                    </execution>
                    <execution>
                        <id>webpack</id>
                        <goals>
                            <goal>webpack</goal>
                        </goals>
                        <phase>process-resources</phase>
                        <configuration>
                            <environmentvariables>
                                <node_env>${env}</node_env>
                                <version>${project.version}</version>
                                <m2_repository>${maven.repo.local}</m2_repository>
                                <uitk_version>${uitkCoreVersion}</uitk_version>
                            </environmentvariables>
                            <arguments>--hide-modules</arguments>
                        </configuration>
                    </execution>
                </executions>
                <configuration>
                    <nodeversion>v7.9.0</nodeversion>
                </configuration>
            </plugin>

2°) Start a small package.json, it will be the bootstrap of your webpack pipeline :

{
  "name": "my-webapp",
  "description": "EcmaScript pipeline to build the assets of my-webapp",
  "version": "1.0.0",
  "private": true,
  "devDependencies": {
    "webpack": "latest",
    "copy-webpack-plugin": "latest",
    "extract-text-webpack-plugin": "latest",
    "before-build-webpack": "latest",
    "noop-webpack-plugin": "latest",
    "modernizr-webpack-plugin": "latest",
    "style-loader": "latest",
    "css-loader": "latest",
    "less-loader": "latest",
    "img-loader": "latest",
    "url-loader": "latest",
    "imports-loader": "latest",
    "exports-loader": "latest",
    "babel-cli": "latest",
    "babel-core": "latest",
    "babel-loader": "latest",
    "babel-preset-env": "latest",
    "babel-preset-es2015": "latest",
    "babel-polyfill": "latest",
    "handlebars": "latest",
    "handlebars-loader": "latest",
    "file-loader": "latest",
    "modernizr": "latest",
    "modernizr-loader": "latest",
    "jquery": "latest",
    "unzip2": "latest",
    "underscore": "latest",
    "phoneparser": "latest",
    "intl-tel-input": "latest",
    "backbone": "latest",
    "moment": "latest",
    "i18n-js": "latest",
    "mkdirp": "latest",
    "ncp": "latest",
    "less": "latest",
    "glob": "latest",
    "properties-reader": "latest"
  },
  "engines": {
    "node": ">=7.9.0"
  }
}

3°) Write a small webpack pipeline (wepack.config.js) and test it immediately :

const webpack = require('webpack')
const path = require('path')
const fs = require('fs')
const unzip = require('unzip2')
const glob = require('glob')
const WebpackBeforeBuildPlugin = require('before-build-webpack')
const ExtractTextPlugin = require('extract-text-webpack-plugin')
const CopyWebpackPlugin = require('copy-webpack-plugin')
const ModernizrWebpackPlugin = require('modernizr-webpack-plugin'

const javascriptFiles = glob.sync(`js/**/*.js`)
const target = path.join(__dirname, 'target/')

module.exports = {
    name: 'webapp-bundle',
    target: 'web',
    entry:  [{`js/bundles/bundle.js`: javascriptFiles,}], output: { path: target, filename: '[name].js' }, module: { rules: [ { test: /\.js$/, use: [{ loader: 'babel-loader', options: { presets: [['es2015']] } }] }, ] } }

That works, but there are so much work to do for a real webapp if you are not working on a hello world page or a calculator, and over all if you must cope with an homegrown framework from your company

4°) Add some plugins to your pipeline :
– prepare your assets before the build (unzip, copy, …) :WebpackBeforeBuildPlugin
– provide some variables that nobody wants to declare explicitely : jquery ($), underscore or lodash (_) :webpack.ProvidePlugin
– extract your assets in non js files, for example for css files : ExtractTextPlugin
– manipulate the results after the compilation step : CopyWebpackPlugin
my own plugins list looks like this one :

plugins: [beforeBuild, listAllHandlebarsFiles, provideGlobalVariables, new ModernizrWebpackPlugin(), /* compile, */ extractCss, copyResults, minify]

5°) Add some rules to help webpack understands what is supposed to happen for each case :

{
                test: /\.hbs$/,
                use: [{
                    loader: 'handlebars-loader',
                    options: {
                        knownHelpers: ['i18nMsg', 'block', 'include', 'includeJsBundle', 'eq', 'json', 'i18nJs',
                            'partial'],
                        partialDirs: [
                            path.join(__dirname, `${webpackAssetsFolder}views`)
                        ],
                        rootRelatve: `${webpackAssetsFolder}views`
                    }
                }],
            },
            {
                test: /\.js$/,
                use: [{
                    loader: 'babel-loader',
                    options: {
                        presets: [['es2015']]
                    }
                }]
            },
            {
                test: /\.less$/,
                use: extractCss.extract({
                    use: ['css-loader', 'less-loader']
                })
            },
            {
                test: /\.css$/,
                use: extractCss.extract({
                    fallback: 'style-loader',
                    use: 'css-loader'
                })
            },

5°) Adapt your homegrown framework to your pipeline. Unzip the resources of the framework and mix them with yours.

You can have a look at how I did at my work : https://gist.github.com/libetl/79023dec57e4f4a02b48ee05b8d34d68
The pipeline (I mean the webpack.config.js file) can be composed of all these steps (mainly) :

import some modules
definition of resources and paths
definitions of complex plugins
output folder and output bundles
source maps (yes or no ?)
webpack rules (which file for which operation)
plugins list

Please do this at home, or at school. You will make your app more reliable and efficient. Good luck.

Write Java code that even your CEO can read (https://github.com/libetl/ideaomatic)

Were you trying to write boilerplate code in Java lately ?

List> messageConverters = new ArrayList<>();
MappingJackson2HttpMessageConverter jackson2ConverterOther = new MappingJackson2HttpMessageConverter();
jackson2ConverterOther.setObjectMapper(pluginObjectMapper);
messageConverters.add(jackson2ConverterOther);
return messageConverters;

How much can you be far from the reality of your customers ?

try {
m_jaxbContext = JAXBContext.newInstance(m_contextPackages);
m_state = State.OPERATIONAL;
} catch (Throwable t) {
m_state = State.ERROR;
s_logger.error("Exeption creating JAXBContext for context: " + m_contextPackages, t);
}

Do you feel like understanding your own code ? And your other team members’ code too ?

Then write code that everyone will understand…
https://github.com/libetl/ideaomatic

Make the code as expressive as in English text. Here are some samples that can help you understand what is going on in this library :
https://github.com/libetl/ideaomatic/blob/master/src/test/java/outside/Tests.java

Make your own opinion on the topic and don’t hesitate to tell me if you want to add extra language elements (please post me some issues or some pull requests)

How to colorize an xml by code

The following class will help you to do that.

You can easily transform this :

<?xml version="1.0" encoding="UTF-8"?>
<kcfg xmlns="http://www.kde.org/standards/kcfg/1.0"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
     xsi:schemaLocation="http://www.kde.org/standards/kcfg/1.0
     http://www.kde.org/standards/kcfg/1.0/kcfg.xsd">
    <kcfgfile name=""/>
    <group name="com.librehat.yahooweather">
        <entry name="woeid" type="String">
            <default></default>
        </entry>
        <entry name="interval" type="Int">
            <default>60</default>
        </entry>
        <entry name="celsius" type="Bool">
            <default>true</default>
        </entry>
        <entry name="fahrenheit" type="Bool">
            <default>false</default>
        </entry>
        <entry name="ms" type="Bool">
            <default>false</default>
        </entry>
        <entry name="kmh" type="Bool">
            <default>true</default>
        </entry>
    </group>
</kcfg>

Into this :

coloredxml

import java.io.File;
import java.io.IOException;
import java.util.Scanner;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
import java.util.stream.IntStream;

public class XmlInColorsConverter {

    public static void main (String [] args) throws IOException {
        Scanner scanner = new Scanner(new File(args[0]));
        scanner.useDelimiter("\\A");
        System.out.println(new XmlInColorsConverter().convert(scanner.next()));
    }

    private static final Pattern NAME = Pattern.compile("(?:([a-zA-Z][a-zA-Z0-9]+):)?([a-zA-Z][a-zA-Z0-9._]+)");
    private static final Pattern VAL = Pattern.compile("\"((?:[^\"]|(?< =\\\\)\")*)\"");
    private static final Pattern ATTR = Pattern.compile("(" + NAME + ")\\s*(=)\\s*(" + VAL + ")\\s*");
    private static final Pattern MARKUP = Pattern.compile("(\\s*)(< )(/|\\?)?" + NAME + "\\s*(?:" + ATTR + ")*(/|\\?)?>");

    public String convert(String text) {
        return colorize(text, MARKUP.matcher(text));
    }

    private static String colorize(String input, Matcher matcher) {
        int lastEnd = 0;
        int level = 0;
        StringBuilder result = new StringBuilder();
        while (matcher.find()) {
            String oneMarkup = matcher.group();
            Matcher attrMatcher = ATTR.matcher(oneMarkup);
            String afterLastMatch = input.substring(lastEnd, matcher.start(2) == -1 ? 0 : matcher.start(2));
            level -= "/".equals(matcher.group(3)) ? 1 : 0;
            if (afterLastMatch.matches("\\s*")) {
                result.append("\n").append(IntStream.range(0, level).boxed().map(i -> "    ").collect(Collectors.joining()));
            }else {
                result.append(afterLastMatch);
            }
            level += matcher.group(3) == null ? 1 : 0;
            result.append("\033[1;33m< ").append((matcher.group(3) != null ? matcher.group(3) : "")).append((matcher.group(4) != null ?  "\033[34m" + matcher.group(4) + "\033[1;33m:" : "\033[1;33m")).append(matcher.group(5)).append("\033[0m");
            while (attrMatcher.find()) {
                result.append(" ").append((attrMatcher.group(2) != null ? "\033[34m" + attrMatcher.group(2) + "\033[35m:" : "\033[35m")).append(attrMatcher.group(3)).append("\033[32m=\033[36m").append(attrMatcher.group(5));
            }
            level -= "/".equals(matcher.group(12)) ? 1 : 0;
            result.append("\033[1;33m").append((matcher.group(12) != null ? matcher.group(12) : "")).append(">\033[0m");
            lastEnd = input.charAt(matcher.end()) == '\n' ? matcher.end() + 1 : matcher.end();
        }
        return result.toString();
    }
}

Hubot : how to pipeline your commands ?

If you got used to creating new commands in your Hubot server, you will probably remember of this syntax :

robot.hear(/say hello/i, function(res) {/*...*/}
robot.respond(/how are you/i, function(res) {/*...*/}

When you want to combine then into one, the easy way out is to create a spare command like this one :

robot.respond(/hello, how are you/i, function(res) {/*...*/}

Things become more difficult when you need combine a set of different instructions. Theorically, it means that you should create n^k different commands.
I was interested in finding how I could change the commands into a full pipeline with the least possible amount of changes.
And I found an answer (maybe not the best indeed) :

https://gist.github.com/libetl/56763f55cf1d1ac7a01eb6476559eb69
Insert the pipeline object into your code.

And then this is how your existing code is going to change :

instead of this :

robot.hear(/say hello/i, function(res) {/*...*/}
robot.respond(/how are you/i, function(res) {/*...*/}

you will write this :

pipeline.hear(/say hello/i, function(robot, res, resolve) {/*...*/;resolve();}
pipeline.respond(/how are you/i, function(robot, res, resolve) {/*...*/;resolve();}

It is important to call resolve() at the end of the execution so the pipeline knows when to start the next step.

the module.exports should look like this one :
module.exports = pipeline.buildAnswer('(my-commands)')

Here you are, your commands now are all included in your pipeline. Enjoy.

ES6 (or Javascript) : how to loop on the same promise

In the era of the promises, who likes to wait for an uncertain amount of time ?

A promise is the key to hope.
No need to setTimeout(function(){}, XXXXX) milliseconds. You don’t know if you waited too much, and even worse… : you don’t know if you did not wait enough.

Actually, you can tell your program to wait for a promise, then check a condition, then chain a new promise if the condition is not met. The only subtle delta to achieve this is to trigger the condition testing and the return value inside functions ; here is an example :


aPromise.then((pollResponse) => {
const promiseWhile = (condition, oldResponse, promiseGenerator) => {
if (!condition(oldResponse)) {
return typeof oldResponse === 'function' ? oldResponse : () => oldResponse
}
return promiseGenerator().then((newResponse) => promiseWhile(condition, newResponse, promiseGenerator))
}

const promiseGenerator = (execute, oldResponse) => {
return promiseWhile((newResponse) => newResponse.code === 'PENDING', oldResponse,
() => execute().then((newResponse) => promiseGenerator(execute, newResponse)))
}

return promiseGenerator(() => query(`/myApi/pollStatus`), pollResponse)
})
.then((pollResponse) => {
const result = pollResponse()
console.log(result.code) //'COMPLETE'
})

Given a method called query, which returns either {code: ‘PENDING’} or {code: ‘COMPLETE’}, This code runs the query method regularly and stops when the value of code is no more ‘PENDING’. But this does not wait for an arbitrary amount of time.

If the condition is not true anymore, it returns a function whose result is the value to fetch.
Else it chains the previously resolved promise with a promise calling query again and checking the same condition again.

That is all.

There is also a lighter version without value return, and where ‘query’ is immediate (actually, it occurs within condition()), but unfortunately you have to define a small interval to avoid saturation on your event loop :


const waitWhile = (condition, resolve) => {
if (!condition()) {
resolve()
}
setTimeout(() => waitWhile(condition, resolve), 1)
}