Pages

14/09/2020

Tesla Model 3 1000km test review

I was considering buying an electric car and Tesla is an obvious choice, so I decided to rent a Model 3 for a weekend and drove it about 1000km in order to evaluate it.

I am from Genova, Italy but live in Zurich, Switzerland so my use case is to find a car that allows me to enjoy a weekend/day trip in Switzerland and occasionally can comfortably bring me to Genova without increasing the total trip time too much (over 30 minutes) due to charging needs.

With a common ICE car, the fastest one way trip without any rest/fuel stop can be done in about 5-5.30 hours; on average it takes 6-6.30 due to queues (damn Gotthard) and rarely it is 7+ hours.

So what about an EV? Here is my review after a 960km round trip in two days.

15/07/2020

[bash] Extract all distinct lines from file

Using a combination of cut and uniq, it is possible to extract all distinct lines from a given file:

cut -d\; -f1 file.txt | uniq


It is optionally possible to sort the file as well adding sort to the command:


cut -d\; -f1 file.txt | sort | uniq

[bash] Extract characters before marker from all lines in a file with sed

Using sed, it is possible to quickly extract all characters before a given marker from all lines in a file with:

sed 's/MARKER.*//' file.txt

27/05/2020

[C++] Graph adjacency list shortest path BFS

Here is a simple graph model using adjacency lists. The edges are undirected and we will also track edge weights, but ignore them at this time for the shortest path calculation between two given points (therefore we can use a simple BFS).

29/04/2020

[Docker] Move docker directory to different location on NTFS filesystem

The default location of docker files (images, volumes, networks, etc) on most distributions is /var/lib/docker

This can be easily changed with the daemon.json configuration file.

First stop the docker service and make sure it is completely stopped.

You can now edit or create the configuration file under /etc/docker folder. In there, you can specify a new location for the docker files with the graph attribute:

{
   "graph": "/path/to/new/docker_folder"
}



You can then copy ALL the content to the new location or let docker recreate everything from scratch there.

WARNING: if the new location is on a different filesystem you might need to change the storage driver AND existing data might become inaccessible (it will remain on the filesystem though)

To set a new storage driver, set the storage-driver attribute as well (example for NTFS):

{
   "graph": "/path/to/new/docker_folder",
   "storage-driver": "vfs"
}

28/04/2020

[GNOME] Create application shortcut that can be added to favorites

Some applications you install come as archives that you extract in a desired location. However, not all come with an installer that takes care of creating the proper application entries, leaving the task to the user.

You can add a shortcut for any application by creating a file APPLICATION_NAME.desktop under /usr/share/applications and adding this content:

[Desktop Entry]
Version=1.0
Encoding=UTF-8
Type=Application
Terminal=false
Categories=OPTIONAL_CATEGORY_LIST

GenericName=APPLICATION_NAME
Comment=OPTIONAL_COMMENT
Name=APPLICATION_NAME
Path=FULL_PATH_TO_APPLICATION_FOLDER
Exec=FULL_PATH_TO_APPLICATION_EXECUTABLE

Icon=OPTIONAL_FULL_PATH_TO_ICON

StartupWMClass=APPLICATION_WM_CLASS


[Linux] Find WM_CLASS value for an application

In order to group application windows together, the WMClass and WMInstance attributes have been specified in the Inter-Client Communication Conventions Manual for X server 11.

Long story short, they are two strings that are used to associate application windows to a specific application, they can be set by the application (WMClass) or the user (WMInstance).

They are very useful especially for those applications that are customizations or based on existing tools, which alone would have a different value (eg Java or Eclipse based apps).

Finding the value for a window is as simple as opening a terminal and running:

xprop

Then clicking on the desired window


25/04/2020

[Linux] Custom keyboard layout QWERTZ to QWERTY

We already saw how to customize a keyboard layout on Windows, now it's the turn of Linux, using XKB for Xorg.

Remapping keys is as simple as editing a text file. You will find all available layouts under /usr/share/X11/xkb/symbols, you can use them as basis for a new layout or directly edit them.

Pick the layout you want to change (maybe make a backup first), then invert the VALUES ONLY of the two lines containing the lower and uppercase z and y, then save.

You will need to restart X server and the changes will be in effect.

21/04/2020

[Windows] Share environment variables in Windows Subsystem for Linux

Now that you have your fancy WSL up and running, you want to use it and all your environment variables are NOT available (of course).

Luckily there is a workaround, using the WSLENV variable. Unfortunately, it is not a direct 1:1 translation of your existing variables, rather you will need to craft them manually and convert between the two environments before you run a command or switch environment.

For example, to pass your JAVA_HOME correctly to wsl:

set JAVA_HOME=%JAVA_HOME%
set WSLENV=JAVA_HOME/up
wsl
echo $JAVA_HOME

And the magic is behind that last /up flag which says: set the value when invoking WSL from Windows (u) and translate between Windows and Unix paths (p)

WARNING: spaces are NOT escaped
WARNING2: only ONE variable is shared, so you cannot share both JAVA_HOME and PATH for example at the same time

[Windows] Activate and configure Windows Subsystem for Linux

Windows 10 introduced an update that adds a very cool feature: Windows Subsystem for Linux. If you are a fan of Cygwin, or similar tools, you will greatly like this feature as well.

Setup could be a bit more straightforward, but still not a big issue:

  1. Enable Windows Subsystem for Linux as a Windows feature from Control Panel > Programs and Features > Turn Windows features on or off
  2. Reboot
  3. Make sure LxssManager service is running
  4. Install a Linux distro from Windows store, search for "WSL" and pick the one you prefer, for example Alpine or Ubuntu, if no distro is installed, you get an error instead:
Windows Subsystem for Linux has no installed distributions. Distributions can be installed by visiting the Windows Store: https://aka.ms/wslstore Press any key to continue...

And going to that link brings you nowhere.. so just install it yourself from the store.

After the installation is completed (might ask you to setup a user and password), you can open any command prompt and type wsl to launch it.

Be careful, that command was different in the past: lxrun or bash and there are apparently 2 versions of WSL as well, use whatever makes it run for you

17/04/2020

[Windows] Store command output in variable in batch script

For whatever reason, storing the output of a command into a variable from a batch script is not easy at all:

FOR /F "tokens=*" %%g IN ('your_command ^| with_escaped_pipe_if_you_need_it') do (SET VAR=%%g)

07/04/2020

[Windows] Custom keyboard layout QWERTZ to QWERTY

Some people are unfortunate enough to use a QWERTZ keyboard layout (and then there are the French), not because they need ready access to that specific letter, but maybe just to avoid losing yet another war.

Of course, for us with functioning opposable thumbs, this might introduce some difficulties as years of CTRL+Z trained our muscle memory for ready undo (Z) rather than redo (Y).

Windows provides a tool: Microsoft Keyboard Layout creator, that helps us overcome this by defining a new key map and generate an installable keyboard layout. Cool.

Install it, File - Load existing keyboard, pick your ümläuted keyboard of choice and invert the z and y keys to restore universe order. Done.

Not so fast. What about UPPERCASE Z and Y? You need to change those too. On the left side, check the Shift under Shift states section and perform the binding again, this time for capital Z and Y. Done.

Not so fast, What about the shortcuts that were using Z and Y? They still map to the old layout where the keys were inverted, you need to change those too. Unfortunately, it can't be done from the tool. 

No worries, save your project and open your .klc file with a text editor and search for the line defining your Z and Y keys, they will look like this (example for the y key):

15 Z 1 y Y -1 -1 // LATIN SMALL LETTER Y, LATIN CAPITAL LETTER Y

As you can see, almost all changes were done, but the damn keys are still left in the wrong place in the underlying mapping. Simply change them to reflect the correct mapping (example for the y key):

15 Y 1 y Y -1 -1 // LATIN SMALL LETTER Y, LATIN CAPITAL LETTER Y

Then save and reopen the file with the tool. Now finally: Project - Build DLL and setup package to generate the installer, run it, and we have our new keyboard available to select in the language list for available keyboards.

If the tool fails to build, try to install it in c:\mklc14 folder as it might fail if it is installed in another location or in a directory containing spaces

Of course, you need to restart the system in order for ALL remappings to take effect.

13/03/2020

[Java] Schedule tasks with Quartz and Spring

We saw how to schedule tasks with ScheduledExecutorService, and we now we will see how to achieve the same result using the Quartz framework.

Much like before, we have two key concepts:
  • job: a job is a runnable class that implements the Job interface
  • trigger: a Trigger is a specific schedule for a job. A trigger can only be linked to a job, and a job can be linked to multiple triggers.
You can find Quartz on Maven.

12/03/2020

[Java] Schedule tasks with ScheduledExecutorService

Java offers a ScheduledExecutorService that allows developers to create and schedule tasks.

A task is a class that implements ONE method to be executed that performs the desired duties. It is NOT necessary for the task itself to implement the Runnable interface, although the tasks will be run as a Thread.

An important thing to notice is that by default threads are NOT marked as daemons, therefore it might be worth setting such flag according to your needs.

Another important thing to note, is that the scheduler will hold on to references to cancelled tasks, which might lead to memory leaks, so it might be preferable to setRemoveOnCancelPolicy to true.

22/02/2020

[Kafka] Delay poll return for testing scenarios

One of the challenges you will face when testing applications that rely on Kafka to send and receive messages, is that the consumer behaviour is quite erratic.

This is all due to how the call to the poll method works.

In short, three key factors are considered before the method returns some messages, the set poll duration and two consumer configurations:
  • fetch.min.bytes: as soon as this many bytes can be read from the desired topic, the consumer's fetch request is ready to be fulfilled and all messages it has retrieved by then will be returned. Default is 1 byte.
  • fetch.max.wait.ms: how long should the fetch request be kept waiting if there is not enough data to return (see previous parameter). default is 500ms.
  • poll timeout: how long will the consumer wait for the fetch request to provide some data before returning. 0 means no wait.
As you can see, in a test environment you might want to control these timings very precisely, to ensure all your send and retrieve message requests are correctly and consistently fulfilled.

I suggest increasing immediately your fetch.min.bytes configuration to something reasonable according to your test message size.
Then you might want to slow the application down a little with the fetch.max.wait.ms setting to account for slight delays in message reception by the test broker.
Lastly, set the poll timeout to a value higher than the fetch.max.wait.ms to ensure your consumer will never return too early while some test messages might still be in the queue for the specific test case. Also remember that behind the scenes, poll does MUCH MORE than simply retrieving messages, therefore you want to allow for enough time to perform the various administrative tasks.

[Java] Kafka send and receive JSON messages

If you wish to send and receive JSON messages via Apache Kafka, you need to provide a way to convert between your POJOs to JSON and viceversa.

Fortunately, if you use generics, you can provide a single configuration that works in all but the most exceptional cases. 

[Kafka] Exactly once delivery

Among the various configuration settings that you set for your producers and consumers, there is a specific set that is very important if you care about EOD (Exactly Once Delivery).

Producers:
  • enable.idempotence = true
  • max.in.flight.requests.per.connection <= 5
  • retries > 0
  • acks = "all"
Consumers:
  • isolation.level = "read_committed"

The consumer configuration is NOT strictly necessary, since messages that are not part of a transaction are not affected by it, but in case your producer would send messages as part of a transaction, you might want to avoid reading aborted ones.

[Java] Testcontainers copy file to container

Once you have your container up and running, you might want to upload some files to it. The outcome of this action is largely dependent on the container type and configuration, but in general you want to use the copyFileToContainer API paying attention to a couple things:

1. If the target location in the container does not exist, it will not be automatically created for you, you can work around this by issuing a command to do so:

container.execInContainer("mkdir", "-p", "TARGET_LOCATION");

2. The uploaded files will likely be owned by root, but you won't be running apps in it as root, so you want to set good enough permissions (at least 775) to allow the actual application user to access them:

container.copyFileToContainer(MountableFile.forHostPath(FILE_ON_HOST, 0777, "TARGET_LOCATION");

where FILE_ON_HOST is a File object representing the resource you are trying to copy to the container.

[Java] Testcontainers set container name

When you create a new container with testcontainers, you might want to assign it a name so that it's easier to spot when you list all of them from docker. This is as simple as:

 GenericContainer container = new GenericContainer<>("IMAGE_NAME")  
                   .withCreateContainerCmdModifier(cmd -> cmd.withName("YOUR_NAME"));  


[Java] Testcontainers start container with fixed port binding

If you're working in Java, you will likely be using docker in your testing process and quite likely you'll be using docker-compose or the amazing testcontainers project so configure and start a bunch of necessary dependencies.

While in a CI/CD environment for obvious reasons it is NOT recommended to fix a port binding for your containers, there might be instances (Kafka anyone?) where it might be preferable to grab a random port BEFORE the container is started AND force the binding to that port. This allows for easier configuration setup while your test suite boots up.

Once you have your random port, you need to configure your FixedHostPortGenericContainer to bind to it:

 int port = findFreePort();  
   
 GenericContainer container = new FixedhostPortGenericContainer<>("IMAGE_NAME")  
                   .withNetworkMode("host")  
                   .withFixedExposedPort(port, port);  
   
 container.setPortBindings(Collections.singletonList(port + ":" + port));  


which will configure the container to run in host network mode and will fix the host and container port to expose and bind to. In the Kafka example you could then set the KAFKA_ADVERTISED_LISTENERS environment variable directly in the container specification, since your port is now fixed.

[Java] Locate resource on classpath

Sometimes you might need to access a resource from your classpath and you might be tempted to use Spring's ClassPathResource which obviously requires you to have Spring available.

An alternative is the getResource method from the Java classloader:

 File myResource;  
   
 try{  
  final URL resource = getClass().getClassLoader().getResource("MY_RESOURCE");  
   
  if(resource != null){  
   myResource = new File(resource.toURI());  
  }  
 } catch(URISyntaxException e){  
  //something went wrong  
 }  

31/01/2020

[Maven] Copy files to folder

Similarily to the Maven dependency plugin which copies resources from an artifact to a specific location, we can use Maven resources plugin to copy resources from a folder instead.

 <build>   
   <plugins>   
    <plugin>   
     <groupId>org.apache.maven.plugins</groupId>   
     <artifactId>maven-resources-plugin</artifactId>   
     <version>3.1.0</version>   
      <executions>   
       <execution>   
        <id>YOUR_ID</id>   
        <phase>SOME_PHASE</phase>   
        <goals>   
         <goal>copy-resources</goal>   
        </goals>   
        <configuration>    
         <outputDirectory>some/folder</outputDirectory>  
         <resources>   
          <resource>  
           <directory>some/other/folder</directory>   
          </resource>   
         </resources>   
        </configuration>   
      </execution>   
     </executions>   
    </plugin>   
   </plugins>   
  </build>   


Important parameters are:

  • outputDirectory to set where to copy the resources
  • directory to set from where to copy the resources (you can have multiple resource entries if you need to copy from multiple sources)