fragmented knowledge

bruchstückhaftes Wissen

integer overflow in awk

On Debian mawk is the default awk implementation, but mawk only works with 32 bit long data types. If you need more, you have to use gawk with the --bignum option. The example sums up the first column of a file.

gawk '{ sum += $1 } END { printf "%d\n", sum }'

Warning: In my example gawk is 40% slower and with --bignum activated it is more than 100% slower.

getting list of file sizes for an extreme amount of files

This is much faster than using find’s -exec, becauce -exec forks an du for every input line.

find . -type f | xargs -d '\n' -n 65535 du -b --apparent-size > file_sizes

listing active users (local)

users=$(egrep '^.*:\$.*$' /etc/shadow | cut -d ":" -f 1) ; for u in $users ; do lastlog -u $u | tail -n +2 | grep -v root ; done

detect log4shell with trivy on active docker imgages

for image in `docker ps --format "{{.Image}}"` ; do echo -n "$(hostname): $image "; trivy image  --skip-update -s CRITICAL $image | grep "CVE-2021-44228"; echo ; done |  grep "CVE-2021-44228" | cut -d "|" -f 1

pdf cut

Cut pages from 363 to 378 from proceedings.pdf to paper.pdf:

qpdf --collate --empty --pages proceeding.pdf 363-378 -- paper.pdf

kvm and qemu

Create qcow2 image which depends on base image

qemu-img create -F qcow2 -b BASE.qcow2 -f qcow2 NEW.qcow2

Commit changes from the NEW image to the BASE image

qemu-img commit -f qcow2 -b BASE.qcow2 NEW.qcow2

bash

sorting a tsv file by its second column (biggest value first)

sort  -t$'\t' -k2 -n -r

read from a newline separated file and rearrange lines into three columns

paste - - -

show matching lines from two files

grep -F -f file1 file2

Take the lines from file1 (-f) and use them as a raw string pattern (-F) and grep file2.

midi -> mp3

timidity file.mid -Ow -o - | ffmpeg -i - -acodec libmp3lame -ab 128k output.mp3

docker

autostart container:

docker update --restart=always 0576df221c0b

e-mail from cli via remote smtp server

using swaks

echo "Das ist eine Nachricht" | swaks --to martin@nachtsieb.de \
--from FROM_EMAIL --server FQDN:PORT --auth PLAIN \
--auth-user "USERNAME" --auth-password "SECRET" \
-tls --header "Subject: foo" --body -

PostgreSQL

show all stored procedures/user defined functions with the pattern anon42:

\df *anon42*

date conversion: epoch time in UTC -> date string

date -ud @1606482000
Fr 27. Nov 13:00:00 UTC 2020

professional downloading with youtube-dl

Downloading the (most AVC/H.264) video track with the given height in pixel and merge (+) it wit the best audio track (aac) provided in an a4m container. The default behaviour is: ‘bestvideo+bestaudio’.

youtube-dl -f 'bestvideo[height=1080][ext=mp4]+bestaudio[ext=a4m]' URL

Maven

build simple maven project

mvn archetype:generate -DgroupId=XXX -DartifactId=YYY -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.4

invoke single test method within test class

mvn -Dtest=TestClass#testMethod test

validate XML against DTD

xmllint -noout --dtdvalid recipes.dtd recipes.xml

disable mouse acceleration

get mouse description

xinput list

list mouse properties

xinput list-props 'USB Optical Mouse'

disable acceleration

xinput --set-prop 'USB Optical Mouse' 'libinput Accel Profile Enabled' 0, 1

svn: create ignore rules

Ignore rules effecting directories and only suport globbing (wildcard *)

svn propedit svn:ignore ./path

rules:

*.out
*.aux
*.toc
*.swp
*.tmp
*.log
*.out
*.exe
*.gz
*.tgz
*.backup
_minted*
*.o
*.bbl
*.blg
*.class
*.pdf

shrink images (prevent data trash)

for i in `ls -1` ; do convert $i  -quality 90 -resize 3200  $i ; done

in parallel with gnu-parallel:

parallel --eta 'convert {} -quality 90 -resize 3200' {} ::: *jpg

epub optimization for mobile e-ink reader

for i in `ls -1` ; do convert $i -colorspace Gray  -quality 80 -resize 50%  $i ; done

remote wireshark per ssh

ssh [TARGET] sudo tcpdump -i [INTERFACW] -U -w -port 22 | wireshark -i – -k

sort and rename files according to atime (bash)

IFS=$'\n' ; i=0 ; for file in `ls -1 -u --sort=time -r` ; do let i++; \
newfile=$(printf "%03d-%s" $i $file); mv $file $newfile ; done

SSH Portforwarding

local

I have no direct connection to binfalse.de:443, i have to tunnel through example.com.

CLIENT <--> SSH-GATEWAY <--> SERVER
^^^^^^      example.com      binfalse.de

ssh -L 9000:binfalse.de:443 user@example.com
ssh -L LOCALPORT:DESTADDR:DESTPORT SSH-GATEWAY

remote

I tell the SSH gateway to open up a port and forward this one to my machine.

CLIENT <--> SSH-GATEWAY <--> BigBadInternet
tcp/3000  example.com:9000

ssh -R 9000:localhost:3000 user@example.com

no tty

ssh -nNT -L 9000:binfalse.de:443 user@example.com

concatenate pdf files

pdftk *.pdf cat output onepdf.pdf

extract partition from image

sfdisk -l -uS image-file
dd if=image-file of=partition-file skip=NUM count=NUM

Linux Ramdisk

mount -o size=1G -t tmpfs none /mnt/tmpfs

awk oneliners:

http://www.pement.org/awk/awk1line.txt

Linux iptables: statisches NAT:

iptables -t nat -A PREROUTING -d 17.17.17.50 -i eth0 -j DNAT --to-destination 10.200.0.1
iptables -t nat -A POSTROUTING -s 10.200.0.1 -o eth1 -j SNAT --to-source 17.17.17.50

Linux iptables: Portforwarding:

iptables -t nat -A PREROUTING -p tcp -i eth2 -d 10.11.20.25 --dport 2222  -j DNAT --to 10.0.0.4:22

IPv6 privacy extensions unter Linux aktivieren (RFC3041):

sysctl net.ipv6.conf.wlan0.use_tempaddr=2

ASCII-armored PGP with Sylpheed

Der MUA Sylpheed ist nicht in der Lage ascii-armored gpg/pgp Nachrichten zu entschlüsseln. Man kann jedoch mit einer selbstdefinierten “Aktion” Abhilfe schaffen:

urxvt -e /bin/bash -c "/usr/bin/gpg -d %p | less"

convert an hex string to ascii, using perl:

perl -e 'chomp($HEX=); while($i < length($HEX)) {$PART=substr($HEX,$i,2);print chr(hex($PART)); $i=$i+2;}'

bash IFS-variable, only split on Newline:

export IFS=$'\n'

snippetsJanuary 1, 2000