Advanced Introduction to C++, Scientific Computing and Machine Learning




Claudius Gros, WS 2019/20

Institut für theoretische Physik
Goethe-University Frankfurt a.M.

Linux - Basics


what is Linux

history

online utilities


Linux commands - examples


Directory Operations


 

Change into the given directory ("." present directory, ".." one level up, "/" top (root) directory)

 

Show (list) the content of the given directory (-a all, -l long, -h human readable)

 

Create (make) a new directory

 

Show path to the present diretroy

 

Delete (remove) given directory when empty


Editors


 

Windowed editor with a large palette of functions

 

Standard in-console Unix-editors with full keyboard control


File Operations


 

For changing access rights for -r reading, -w writing and -x executing

 

Copy files and directory (-r recursively)

 

Shows file type and information

 

Prints a text (file) on screen consecutively

 

For moving (name change) of files and directories

 

Deletes (removes) a given file or directory (-r recursively)

 

Prints the last lines of a file to screen


File Manipulation


 

Combines (concatenates) several files into a single file

 

Search in a output for a string or expression (more test.txt | grep "information")

 

Sending file to a printer

 

Universal viewer for documents, images and graphics

 

Sorting an output or a file (-r reverse, -u unique, -g numerical)

 

Splitting a given file into parts

 

Counting lines, words and chars in an output or file (-l only lines)


Info and Documentation


 

Gives a list of man-pages for a given subject

 

Prints man pages (documentation) for a given command or program to screen


Data Compression


 

For compression of files and data

 

For compression and decompression (with -x) of archives and multiple files

 

Compression and decompression WinZip (DOS/Win) files


Miscellaneous


  • /

 

Tabbing through previous commands

 

Prints current data to screen

 

Prints a list of all environment variables

 

Evaluation of symbolic mathatical expresions, mostly for scripts

 

Searching for files, staring from current directory.

 

History of previous shell commands

 

Name of hosting computer

 

Password change

  • tab

 

Autocompletion of commands


Process Management


  • &

 

Process running in the background (used at the end of a command line)

  • ^C

 

Terminates (kills) the current process (running in current console)

 

Terminates (kills) a process with a given ID (PID)

 

Setting the priority of a given process, mostly in order not to disturb other users

 

Starting a program which will continue to run after the user logs out (no hangup)

 

Lists processes started by the user in present console (-a all, -l long)

 

Measuring the time needed to excute a given process or program

 

Shows all running processe with real-time updating of active processe and their respective CPU and memory consumption


System Infos


 

Disc use of all partitions

 

Disc use of current and child directories

 

Time since last reboot


Remote Connections


 

Data transfer to and from a remote host with line commands (file transfer protocol)

 

Sending pings to remote host

 

Copy data to and from remote host (secure copy)

 

Connect to remote host and log in


Linux commands: man

man – show manual

NAME
       man - an interface to the on-line reference manuals

SYNOPSIS
       man  [-c|-w|-tZ]  [-H[browser]]  [-T[device]] [-X[dpi]] [-adhu7V] [-i|-I] [-m system[,...]] [-L locale] [-p string] [-C file] [-M path] [-P
       pager] [-r prompt] [-S list] [-e extension] [--warnings [warnings]] [[section] page ...] ...
       man -l [-7] [-tZ] [-H[browser]] [-T[device]] [-X[dpi]] [-p string] [-P pager] [-r prompt] [--warnings[warnings]] file ...
       man -k [apropos options] regexp ...
       man -f [whatis options] page ...

DESCRIPTION
       man is the system's manual pager. Each page argument given to man is normally the name of a program, utility or function.  The manual  page
       associated  with  each of these arguments is then found and displayed. A section, if provided, will direct man to look only in that section
       of the manual.  The default action is to search in all of the available sections, following a pre-defined order and to show only the  first
       page found, even if page exists in several sections.

Linux commands: ls

ls – list directory contents

user@pc:/$ ls -l
total 91
drwxr-xr-x   2 root root  4096 Mar 19 08:46 bin
drwxr-xr-x   4 root root  3072 Feb 20 14:49 boot
drwxr-xr-x  14 root root 14240 Apr  6 13:45 dev
drwxr-xr-x 192 root root 12288 Apr 24 09:18 etc
drwxr-xr-x   3 root root     0 Apr 24 09:18 home
drwxr-xr-x   2 root root     6 Aug 14  2008 initrd
drwxr-xr-x  17 root root  8192 Jan 29 09:53 lib
drwxr-xr-x   2 root root     6 Apr 15  2008 mnt
drwxr-xr-x   8 root root  4096 Apr 21 13:50 opt
dr-xr-xr-x 181 root root     0 Feb 20 10:25 proc
drwxr-xr-x   9 root root  4096 Jan 29 09:57 root
drwxr-xr-x   2 root root  8192 Feb 20 14:48 sbin
drwxr-xr-x  12 root root     0 Feb 20 10:25 sys
drwxrwxrwt 132 root root 12288 Apr 24 09:20 tmp
drwxr-xr-x  12 root root   133 Aug 14  2008 usr
drwxr-xr-x  17 root root  4096 Aug 14  2008 var
user@pc:/$ 

Linux commands: more

more – show file on screen

Linux file names

Linux commands: cd

cd – change directory

user@pc:~$ cd /
user@pc:/$ cd var
user@pc:/var$ 

Linux commands: mkdir / rmdir / rm -r

mkdir – make directory

user@pc:~$ mkdir test
user@pc:~$ ls
test

rmdir – remove directory

user@pc:~$ rmdir test
user@pc:~$ rmdir downloads
rmdir: failed to remove `downloads': Directory not empty
user@pc:~$ 

recursive removal

Linux commands: mv / rm

mv – move/rename files

user@pc:~$ mv foo.txt bar.txt
user@pc:~$ mv bar.txt downloads
user@pc:~$

rm – remove files or directories

user@pc:~$ rm downloads/bar.txt 
user@pc:~$ rm -r downloads/
user@pc:~$ 

Linux commands: cat

cat – concatenate files and
print to standard output

user@pc:~$ cat file1
first line of file1
second line of file1

user@pc:~$ cat file2
first line of file2
second line of file2

user@pc:~$ cat file* > all
user@pc:~$ cat all
first line of file1
second line of file1
first line of file2
second line of file2

Linux commands: ps / kill

ps – report a snapshot of the current processes

user@pc:~$ ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0   4020   476 ?        Ss   Feb20   1:08 /sbin/init
root         2  0.0  0.0      0     0 ?        S    Feb20   0:00 [kthreadd]
root         3  0.0  0.0      0     0 ?        S    Feb20   0:02 [migration/0]
root         4  0.0  0.0      0     0 ?        S    Feb20   0:23 [ksoftirqd/0]
root         5  0.0  0.0      0     0 ?        S    Feb20   0:00 [watchdog/0]
root         6  0.0  0.0      0     0 ?        S    Feb20   0:04 [migration/1]
root         7  0.0  0.0      0     0 ?        S    Feb20   1:04 [ksoftirqd/1]
1635      4650  0.0  0.0  92248  2752 ?        S    Apr21   0:00 /usr/lib/eclipse/eclipse -vm /usr/lib/jvm/java-gcj/bin/java -install /usr/lib/eclipse
1635      4656  0.7  9.4 1021416 381456 ?      Sl   Apr21  29:27 /usr/lib/jvm/java-gcj/bin/java -Djava.library.path=/usr/lib/jni -Dgnu.gcj.precompiled
daemon    4715  0.0  0.0   8084   524 ?        Ss   Feb20   0:00 /sbin/portmap
root      4776  0.0  0.0    740    36 ?        Ss   Feb20   0:37 /usr/local/bin/netbeep eth0
statd     4813  0.0  0.0  14416   816 ?        Ss   Feb20   0:00 /sbin/rpc.statd -p 2051 -o 2052
1635     17246  0.0  0.1  17052  7332 pts/1    Ss   09:13   0:00 bash
user@pc:~$ 

kill – send a signal to a process

user@pc:~$ ps -l
F S   UID   PID  PPID  C PRI  NI ADDR SZ WCHAN  TTY          TIME CMD
0 S  1467  8100  8095  0  80   0 -  8641 wait   pts/0    00:00:02 bash
0 S  1467 16732  8100  1  80   0 - 133912 poll_s pts/0   00:00:00 kwrite
0 R  1467 16797  8100  0  80   0 -  5796 -      pts/0    00:00:00 ps
0 S  1467 26446     1  0  80   0 - 221014 futex_ pts/0   00:02:11 soffice.bin
user@pc:~$ kill 16732

piping of Linux commands

| - output of one command as input of next

user@pc:~$ more names.txt 
Ralph Junghans
Anna Rosenkranz
Joachim Neumayer
Anna Furlger

user@pc:~$ more names.txt | grep "Anna" | sort > names.anna

user@pc:~$ more names.anna 
Anna Furlger
Anna Rosenkranz

wc - counting lines/words/characters

user@pc:~$ more names.txt | grep Anna
Anna Rosenkranz
Anna Furlger

user@pc:~ more names.txt | grep Anna | wc
      2       4      29

user@pc:~$ wc names.txt 
 4  8 61 names.txt

Linux editors

use the editor you feel at ease with

hidden files

files starting with a . are hidden

user@pc:~/TEST$ ls -l
total 32
-rw-rw-r-- 1 user user  1775 2012-03-09 17:58 HashMapMapDemo.class
-rw-rw-r-- 1 user user  2019 2012-03-09 17:59 HashMapMapDemo.java
-rw-rw-r-- 1 user user 20961 2012-03-09 15:45 hashMap.svg
user@pc:~/TEST$ ls -la
total 56
drwxrwxr-x   2 user user  4096 2012-03-12 09:57 .
drwxr-xr-x 153 user user 12288 2012-03-12 09:56 ..
-rw-rw-r--   1 user user  1775 2012-03-09 17:58 HashMapMapDemo.class
-rw-rw-r--   1 user user  2019 2012-03-09 17:59 HashMapMapDemo.java
-rw-rw-r--   1 user user 20961 2012-03-09 15:45 hashMap.svg
-rw-rw-r--   1 user user     3 2012-03-12 09:56 .I-am-a-hidden-file

connection to remote hosts

using university computers from home

user@pc:~$ ssh -l gros karlstad.th.physik.uni-frankfurt.de
gros@karlstad.th.physik.uni-frankfurt.de's password: 

gros@karlstad:~$ cd TEST/
gros@karlstad:~$ javac Test.java 
gros@karlstad:~$ nohup time java Test test.old test.new &
[1] 2176
gros@karlstad:~$ nohup: ignoring input and appending output to `nohup.out'

gros@karlstad:~$ exit
logout
Connection to karlstad.th.physik.uni-frankfurt.de closed.

user@pc:~$ scp gros@karlstad.th.physik.uni-frankfurt.de:TEST/nohup.out .
gros@karlstad.th.physik.uni-frankfurt.de's password: 
nohup.out                                                 100%  326     0.3KB/s   00:00    

user@pc:~$ more nohup.out 
# starting Test.main()
# scanning the file  test.old
# scanning the file  test.new
     1000    lines in    test.old
     1000    lines in    test.new
       84    lines in    test.output
0.15user 0.02system 0:00.21elapsed 77%CPU (0avgtext+0avgdata 77248maxresident)k
888inputs+80outputs (6major+5261minor)pagefaults 0swaps

user@pc:~$ ping 141.2.247.163
PING 141.2.247.163 (141.2.247.163) 56(84) bytes of data.
64 bytes from 141.2.247.163: icmp_req=1 ttl=64 time=4.99 ms
64 bytes from 141.2.247.163: icmp_req=2 ttl=64 time=0.132 ms
64 bytes from 141.2.247.163: icmp_req=3 ttl=64 time=0.263 ms
64 bytes from 141.2.247.163: icmp_req=4 ttl=64 time=0.299 ms

Linux - shell scripts

automated execution of commands

user@pc:~$ vi test.sh
user@pc:~$ chmod -x test.sh
Copy Copy to clipboad
Downlaod Download
#! /bin/bash 
i=0
N=2
while test $i -le $N
 do
 a=`expr 50 \\* $i`
 b=`expr $a + 1`
 more "inputFile_$i.dat" | grep "Anna" > "outFile_$b.out"
 echo "ich gehe jetzt für $i Sekunden schlafen " $i
 sleep $i;
 i=`expr $i + 1`
done 
cat outFile_*.out > outFile.all
user@pc:~$ ls
inputFile_0.dat  inputFile_1.dat  inputFile_2.dat  test.sh

user@pc:~$ ./test.sh 
ich gehe jetzt für 0 Sekunden schlafen  0
ich gehe jetzt für 1 Sekunden schlafen  1
ich gehe jetzt für 2 Sekunden schlafen  2

user@pc:~$ ls
inputFile_0.dat  inputFile_2.dat  outFile_1.out   outFile.all
inputFile_1.dat  outFile_101.out  outFile_51.out  test.sh

user@pc:~$ more inputFile_*.dat
::::::::::::::
inputFile_0.dat
::::::::::::::
Ralph Junghans
Anna Rosenkranz
Joachim Neumayer
Anna Furlger
::::::::::::::
inputFile_1.dat
::::::::::::::
Keine Anna hier,
oder etwa doch?
::::::::::::::
inputFile_2.dat
::::::::::::::
Hier ist nun wirklich 
keine anna zu finden.

user@pc:~$ more outFile.all 
Anna Rosenkranz
Anna Furlger
Keine Anna hier,

Linux open-source utilities

a very large selection of free utilities