15/07/2020

[bash] Extract all distinct lines from file

Using a combination of cut and uniq, it is possible to extract all distinct lines from a given file:

cut -d\; -f1 file.txt | uniq


It is optionally possible to sort the file as well adding sort to the command:


cut -d\; -f1 file.txt | sort | uniq

[bash] Extract characters before marker from all lines in a file with sed

Using sed, it is possible to quickly extract all characters before a given marker from all lines in a file with:

sed 's/MARKER.*//' file.txt