Machine Learning Primer




Claudius Gros, WS 2024/25

Institut für theoretische Physik
Goethe-University Frankfurt a.M.

Linux - Basics


what is Linux

history

online utilities


Linux directories

Linux directory structure

user@pc:~$ cd /
user@pc:/$ cd var
user@pc:/var$ 

mkdir - create directory

user@pc:~$ mkdir test
user@pc:~$ ls
test

rmdir – remove directory

user@pc:~$ rmdir test
user@pc:~$ rmdir downloads
rmdir: failed to remove `downloads': Directory not empty
user@pc:~$ 

recursive removal

Linux files

more – show file on screen

Linux file names

mv – move/rename files

user@pc:~$ mv foo.txt bar.txt
user@pc:~$ mv bar.txt downloads
user@pc:~$

rm – remove files or directories

user@pc:~$ rm downloads/bar.txt 
user@pc:~$ rm -r downloads/
user@pc:~$ 

cat – concatenate files and print to standard output

user@pc:~$ cat file1
first line of file1
second line of file1

user@pc:~$ cat file2
first line of file2
second line of file2

user@pc:~$ cat file* > all
user@pc:~$ cat all
first line of file1
second line of file1
first line of file2
second line of file2

hidden files:
files starting with a . are hidden

user@pc:~/TEST$ ls -l
total 32
-rw-rw-r-- 1 user user  1775 2012-03-09 17:58 HashMapMapDemo.class
-rw-rw-r-- 1 user user  2019 2012-03-09 17:59 HashMapMapDemo.java
-rw-rw-r-- 1 user user 20961 2012-03-09 15:45 hashMap.svg
user@pc:~/TEST$ ls -la
total 56
drwxrwxr-x   2 user user  4096 2012-03-12 09:57 .
drwxr-xr-x 153 user user 12288 2012-03-12 09:56 ..
-rw-rw-r--   1 user user  1775 2012-03-09 17:58 HashMapMapDemo.class
-rw-rw-r--   1 user user  2019 2012-03-09 17:59 HashMapMapDemo.java
-rw-rw-r--   1 user user 20961 2012-03-09 15:45 hashMap.svg
-rw-rw-r--   1 user user     3 2012-03-12 09:56 .I-am-a-hidden-file

Linux command options

ls – list directory contents

convert – audio, video and text formats

wget https://upload.wikimedia.org/wikipedia/commons/3/30/Jud-morote-gari.gif
eog Jud-morote-gari.gif
mv Jud* test.gif
convert test.gif frame%04d.gif
convert -delay 50 frame*.gif -loop 0 animated.gif
eog animated.gif

Linux process control

ps – report a snapshot of the current processes

user@pc:~$ ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0   4020   476 ?        Ss   Feb20   1:08 /sbin/init
root         2  0.0  0.0      0     0 ?        S    Feb20   0:00 [kthreadd]
root         3  0.0  0.0      0     0 ?        S    Feb20   0:02 [migration/0]
root         4  0.0  0.0      0     0 ?        S    Feb20   0:23 [ksoftirqd/0]
root         5  0.0  0.0      0     0 ?        S    Feb20   0:00 [watchdog/0]
root         6  0.0  0.0      0     0 ?        S    Feb20   0:04 [migration/1]
root         7  0.0  0.0      0     0 ?        S    Feb20   1:04 [ksoftirqd/1]
1635      4650  0.0  0.0  92248  2752 ?        S    Apr21   0:00 /usr/lib/eclipse/eclipse -vm /usr/lib/jvm/java-gcj/bin/java -install /usr/lib/eclipse
1635      4656  0.7  9.4 1021416 381456 ?      Sl   Apr21  29:27 /usr/lib/jvm/java-gcj/bin/java -Djava.library.path=/usr/lib/jni -Dgnu.gcj.precompiled
daemon    4715  0.0  0.0   8084   524 ?        Ss   Feb20   0:00 /sbin/portmap
root      4776  0.0  0.0    740    36 ?        Ss   Feb20   0:37 /usr/local/bin/netbeep eth0
statd     4813  0.0  0.0  14416   816 ?        Ss   Feb20   0:00 /sbin/rpc.statd -p 2051 -o 2052
1635     17246  0.0  0.1  17052  7332 pts/1    Ss   09:13   0:00 bash
user@pc:~$ 

kill – send a signal to a process

user@pc:~$ ps -l
F S   UID   PID  PPID  C PRI  NI ADDR SZ WCHAN  TTY          TIME CMD
0 S  1467  8100  8095  0  80   0 -  8641 wait   pts/0    00:00:02 bash
0 S  1467 16732  8100  1  80   0 - 133912 poll_s pts/0   00:00:00 kwrite
0 R  1467 16797  8100  0  80   0 -  5796 -      pts/0    00:00:00 ps
0 S  1467 26446     1  0  80   0 - 221014 futex_ pts/0   00:02:11 soffice.bin
user@pc:~$ kill 16732

piping of Linux commands

| - output of one command as input of next

echo 'Ralph Junghans
Anna Rosenkranz
Joachim Neumayer
Anna Furlger' > names.txt
user@pc:~$ more names.txt 
Ralph Junghans
Anna Rosenkranz
Joachim Neumayer
Anna Furlger

user@pc:~$ more names.txt | grep "Anna" | sort > names.anna

user@pc:~$ more names.anna 
Anna Furlger
Anna Rosenkranz

wc - counting lines/words/characters

user@pc:~$ more names.txt | grep Anna
Anna Rosenkranz
Anna Furlger

user@pc:~ more names.txt | grep Anna | wc
      2       4      29

user@pc:~$ wc names.txt 
 4  8 61 names.txt

Linux editors

use the editor you feel at ease with

connection to remote hosts

using university computers from home

user@pc:~$ ssh -l gros karlstad.th.physik.uni-frankfurt.de
gros@karlstad.th.physik.uni-frankfurt.de's password: 

gros@karlstad:~$ cd TEST/
gros@karlstad:~$ javac Test.java 
gros@karlstad:~$ nohup time java Test test.old test.new &
[1] 2176
gros@karlstad:~$ nohup: ignoring input and appending output to `nohup.out'

gros@karlstad:~$ exit
logout
Connection to karlstad.th.physik.uni-frankfurt.de closed.

user@pc:~$ scp gros@karlstad.th.physik.uni-frankfurt.de:TEST/nohup.out .
gros@karlstad.th.physik.uni-frankfurt.de's password: 
nohup.out                                                 100%  326     0.3KB/s   00:00    

user@pc:~$ more nohup.out 
# starting Test.main()
# scanning the file  test.old
# scanning the file  test.new
     1000    lines in    test.old
     1000    lines in    test.new
       84    lines in    test.output
0.15user 0.02system 0:00.21elapsed 77%CPU (0avgtext+0avgdata 77248maxresident)k
888inputs+80outputs (6major+5261minor)pagefaults 0swaps

user@pc:~$ ping 141.2.247.163
PING 141.2.247.163 (141.2.247.163) 56(84) bytes of data.
64 bytes from 141.2.247.163: icmp_req=1 ttl=64 time=4.99 ms
64 bytes from 141.2.247.163: icmp_req=2 ttl=64 time=0.132 ms
64 bytes from 141.2.247.163: icmp_req=3 ttl=64 time=0.263 ms
64 bytes from 141.2.247.163: icmp_req=4 ttl=64 time=0.299 ms

Linux commands - overview


Directory Operations


 

Change into the given directory ("." present directory, ".." one level up, "/" top (root) directory)

 

Show (list) the content of the given directory (-a all, -l long, -h human readable)

 

Create (make) a new directory

 

Show path to the present directory

 

Delete (remove) given directory when empty


Editors


 

Windowed editor with a large palette of functions

 

Standard in-console Unix-editors with full keyboard control


File Operations


 

For changing access rights for -r reading, -w writing and -x executing

 

Copy files and directory (-r recursively)

 

Shows file type and information

 

Prints a text (file) on screen consecutively

 

For moving (name change) of files and directories

 

Deletes (removes) a given file or directory (-r recursively)

 

Prints the last lines of a file to screen


File Manipulation


 

Combines (concatenates) several files into a single file

 

Search in a output for a string or expression (more test.txt | grep "information")

 

Sending file to a printer

 

Universal viewer for documents, images and graphics

 

Sorting an output or a file (-r reverse, -u unique, -g numerical)

 

Splitting a given file into parts

 

Counting lines, words and chars in an output or file (-l only lines)


Info and Documentation


 

Gives a list of man-pages for a given subject

 

Prints man pages (documentation) for a given command or program to screen


Data Compression


 

For compression of files and data

 

For compression and decompression (with -x) of archives and multiple files

 

Compression and decompression WinZip (DOS/Win) files


Miscellaneous


  • /

 

Tabbing through previous commands

 

Prints current data to screen

 

Prints a list of all environment variables

 

Evaluation of symbolic mathatical expresions, mostly for scripts

 

Searching for files, staring from current directory.

 

History of previous shell commands

 

Name of hosting computer

 

Password change

  • tab

 

Autocompletion of commands


Process Management


  • &

 

Process running in the background (used at the end of a command line)

  • ^C

 

Terminates (kills) the current process (running in current console)

 

Terminates (kills) a process with a given ID (PID)

 

Setting the priority of a given process, mostly in order not to disturb other users

 

Starting a program which will continue to run after the user logs out (no hangup)

 

Lists processes started by the user in present console (-a all, -l long)

 

Measuring the time needed to excute a given process or program

 

Shows all running processe with real-time updating of active processe and their respective CPU and memory consumption


System Infos


 

Disc use of all partitions

 

Disc use of current and child directories

 

Time since last reboot


Remote Connections


 

Data transfer to and from a remote host with line commands (file transfer protocol)

 

Sending pings to remote host

 

Copy data to and from remote host (secure copy)

 

Connect to remote host and log in


Linux - shell scripts

automated execution of commands

user@pc:~$ vi test.sh
user@pc:~$ chmod -x test.sh
Copy Copy to clipboad
Downlaod Download
#! /bin/bash 
i=0
N=2
while test $i -le $N
 do
 a=`expr 50 \\* $i`
 b=`expr $a + 1`
 more "inputFile_$i.dat" | grep "Anna" > "outFile_$b.out"
 echo "ich gehe jetzt für $i Sekunden schlafen " $i
 sleep $i;
 i=`expr $i + 1`
done 
cat outFile_*.out > outFile.all
echo 'Ralph Junghans
Anna Rosenkranz
Joachim Neumayer
Anna Furlger' > inputFile_0.dat

echo 'Keine Anna hier,
oder etwa doch?' > inputFile_1.dat

echo 'Hier ist nun wirklich
keine anna zu finden.' > inputFile_2.dat
user@pc:~$ ls
inputFile_0.dat  inputFile_1.dat  inputFile_2.dat  test.sh

user@pc:~$ ./test.sh 
ich gehe jetzt für 0 Sekunden schlafen  0
ich gehe jetzt für 1 Sekunden schlafen  1
ich gehe jetzt für 2 Sekunden schlafen  2

user@pc:~$ ls
inputFile_0.dat  inputFile_2.dat  outFile_1.out   outFile.all
inputFile_1.dat  outFile_101.out  outFile_51.out  test.sh
user@pc:~$ more inputFile_*.dat
::::::::::::::
inputFile_0.dat
::::::::::::::
Ralph Junghans
Anna Rosenkranz
Joachim Neumayer
Anna Furlger
::::::::::::::
inputFile_1.dat
::::::::::::::
Keine Anna hier,
oder etwa doch?
::::::::::::::
inputFile_2.dat
::::::::::::::
Hier ist nun wirklich 
keine anna zu finden.

user@pc:~$ more outFile.all 
Anna Rosenkranz
Anna Furlger
Keine Anna hier,

Linux open-source utilities

a very large selection of free utilities