http://labor-liber.org
Last significant update : 1 July 2004
Copyright © 2004 Cédric Musso, http://labor-liber.org.
Some rights reserved, according to the terms of theCreative Commons - Attribution-ShareAlike 1.0 license. http://creativecommons.org/licenses/by-sa/1.0/
There are many UNIX flavors. Mainly:
GNU/Linux is a POSIX compliant complete rewrite of Unix, and a blend of System V and BSD.
An operating system is a complex system, made up of a large number of components, or software layers. Ideally, these components are independant from each other, so that the large complex system becomes a set of smaller and more simple components.
The kernel of an operating system is its core component ; or its bottom layer, the interface with hardware.
The Linux kernel:
Version numbers:
Text mode / command line (instructions are given line by line) user interface:
The shell is much more flexible and powerful than a graphical user interface.
sh: Bourne shell, the original and perhaps the simplest
Unix shell.csh: C shell, with a syntax modeled after the C
programming language.tcsh: a superset of the C shell.ksh: Korn shell, compatible with the Bourne shell, it
includes many features of the C shell as well.bash: Bourne Again shell, largely backward-compatible with
sh, it includes ideas drawn from ksh and csh.The X Window system, or X, or X11:
X enables text copy&paste with the mouse: select text using the left button, and copy it elsewhere with the middle button.
Client Server system: a X server on a machine can display a client application running on another machine.
An application server can run a client application for many machines with an X server (e.g. OpenOffice.org).
There is no clear distinction between desktop environments and window managers.
The freedesktop.org project works on interoperability and shared technology for desktop environments for the X Window System.
Desktop environments like Gnome and KDE provide a platform for software applications development, and a large and increasing number of applications with a common look and feel.
g"
or "gnome-".k".GNU/Linux is configured by default to have 7 virtual consoles, with X Window running on the seventh (when it is running).
Switching from X Window to one of the 6 consoles: Ctrl + Alt + F1, ..., Ctrl + Alt + F6.
Switching from a text console to another text console: Alt + F1, ..., Alt + F6.
Switching back to X Window: Alt + F7.
To use the system, a user must have an account on it. Basically, an account is a login, a password, and a personal home directory.
A user identifies himself to the system by typing his login and password when they are requested (by a virtual console, or by a graphical program called a display manager):
Logging in is the beginning of a user session. Sessions end when users log out.
logoutexitThe prompt is a character or a character sequence displayed by the shell when it is waiting for the user's instructions.
command [options] [arguments]
Convention: Operands which are enclosed in square brackets ( [ ] ) are optional
Most commands have many options.
-).whoamipasswdechodatecal or ncalclearunamealiasalias ls='ls --color'
alias rm='rm -i'
Many commands have a --help option.
helpmaninfowhatisapropos or man -kAnd you are free to study and use existing Free code.
A manual (reference) page may have the following sections:
NAME, SYNOPSIS, DESCRIPTION, OPTIONS, USAGE, ENVIRONMENT, FILES, SEE ALSO.
Manual pages are also organised into sections (or directories in
/usr/share/man/):
| Section | Type of pages |
|---|---|
| 1 | Executable programs or shell commands. |
| 2 | System calls (functions provided by the kernel) |
| 3 | Library calls (functions within program libraries) |
| 4 | Special files (usually found in /dev) |
| 5 | File formats and conventions eg /etc/passwd |
| 6 | Games |
| 7 | Miscellaneous (including macro packages and conventions). |
| 8 | System administration commands (usually only for root) |
man 1 crontab man 5 crontab
/word to search for word, then n
to go to its following occurence, and N for the previous.info info
info --apropos string
| Key Combination | Effect |
|---|---|
| Ctrl + C | Terminate a running program. |
| Ctrl + D | Equivalent to exit or logout.
EOF (End Of File) character. |
| Ctrl + L | Equivalent to clear. |
| Ctrl + Z | Suspend current foreground process (Job Control). |
| Shift + Page Up and Shift + Page Down | Browse terminal buffer (scroll up or down). |
| Ctrl + Alt + Backspace | Kill X Window. |
| Character | Meaning or Use |
|---|---|
newline space tab |
Word separators |
; |
Command separator |
& |
Background execution |
( ) |
Command grouping (subshell) |
{ } |
Command block |
| > < & |
Redirections |
* ? [ ] ~ ! |
Filename metacharacters |
` |
Command substitution |
$ |
Variable (or command) substitution |
# |
Comment |
\ " ' |
Escape or quote other characters |
| Key | Action |
|---|---|
| Home | Move the cursor at the beginning of the command line. |
| End | Move the cursor at the end of the command line. |
| Arrow Left | Move the cursor one character to the left. |
| Arrow Right | Move the cursor one character to the right. |
fc| Key (combination) | Action |
|---|---|
| Arrow Up | Move up |
| Arrow Down | Move down |
| Ctrl + R | Search command history. |
| Event designator | Reference |
|---|---|
!n |
Refer to command line n. |
!-n |
Refer to the current command line minus n. |
!! |
Refer to the previous command (same as !-1). |
!string |
Refer to the most recent command starting with string. |
!?string? |
Refer to the most recent command containing string. |
^string1^string2^ |
Repeat the last command, replacing string1 with string2. |
man history
to know more and learn about Word Designators and Modifiers.
You don't have to type the full names of commands or files. Bash can complete the names for you.
*?[abcde][a-e][!abcde][!a-e]{debian,linux}The system's operations on a file, a directory, a CD drive, ..., a screen, are fundamentally identical.
Files and directories, physical devices, and even streams of data or network connections are files (of a different type), so that:
| File Type | First column of ls -l output |
|---|---|
| Regular file | - |
| Block device | b |
| Character device | c |
| Directory | d |
| Symbolic link | l |
| FIFO (named pipe) | p |
| Socket | s |
The system stores information about files in a structure known as an inode. This information includes:
ctime, the time the file's inode was last modified,mtime, the time the file's content was last modified,atime, the time the file was last accessed (read, executed),and file permissions:
UID),GID),stat
/ is the root directory.
/bin: Essential binaries for all users./boot: Static files of the boot loader./dev: Device files./etc: System-wide configuration files./home: Contains the users' home directory./lib: Essential shared libraries and kernel modules./mnt: Mount point for a temporarily mounted filesystem./proc: Kernel and process information virtual filesystem./root: Home directory of the root user./sbin: Essential binaries for the root user./tmp: Temporary files./usr: Shareable, read-only data./var: Variable files. This includes spool directories and files,
administrative and logging data, and transient and temporary files.Every directory always contains two special files:
.../home/user/directory/file.txt
Relative to the current directory.
./directory/file.txt is the same as
directory/file.txt or
../user/directory/file.txt when the current directory is
/home/user/.
cddfdulspwdfindlocateupdatedb.whichwhereiscpmkdirmvrmrmdirtouchln-s option, it makes symbolic
(or "soft") links.A "hard link" is another name for an existing file.
The link and the original are indistinguishable (they share the same inode).
Caution: the link can be removed without removing the file
with unlink, but rm will remove the link and
the file.
A symbolic link, on the other hand, refers to a different file. Most operations (opening, reading, writing, and so on) are passed the symbolic link file, the kernel automatically "dereferences" the link and operates on the target of the link. But some operations (e.g. removing) work on the link file itself, rather than on its target.
A filesystem is the methods and data structures that an operating system uses to keep track of files on a disk or partition; that is, the way the files are organized on the disk.
| Filesystem | Description |
|---|---|
ext2 |
Linux native filesystem. |
ext3 |
ext2 plus journaling. |
reiserfs |
Robust and journaling filesystem. |
iso9660 |
The standard CD-ROM filesystem. |
fat and vfat |
Microsoft Windows fat |
ntfs |
Microsoft Windows (NT, ...) |
syncTo gain access to files on a device, you must first tell it where in the directory tree you would like those files to appear (mount point). This process is called mounting a filesystem.
mountumount
The system administrator can control which file systems are mounted
at boot time, pre-determine the mount points for specific file systems,
and allow some file systems to be mounted by normal users.
This is configured in /etc/fstab, which includes the
following fields:
noauto: not done at boot timero: read-only deviceusers: any user can mount it/dev/hda1/dev/hda2/dev/hdb1/dev/sda1/dev/sdb1/dev/fd0/dev/cdrom/dev/null/dev/null
will disappear.Files may contain (human readable) text, binary data, or both. Files may be meant to be executed, processed by an application, or edited in a simple text editor.
A file does not need any extension (like a .exe suffix)
to be executed (it needs execution permission).
Applications may need to identify the type of files. File extensions are often used for this purpose.
filefile.txt and File.txt are
different files./".However, file names should only contain the following characters:
"a", ... "z",
"A", ... "Z",
"0", ... "9",
"-", "_",
"." because:
\".For example, in a graphical user interface, one can click on a file name
which contains a space character; but in a shell, such a file has to be written
file\ name\ with\ spaces.txt or
"file name with spaces.txt" or
'file name with spaces.txt'.
~" is the current user's home directory.." are hidden
files. Most of these files are configuration files, located in the
user's home directory.~" are backup files
which are automatically created by some editing tools. They contain the
file's former version.
In general, a command (a program):
stdin (default: keyboard).stdout (default: screen).stderr
(default: screen).These three are files, and are always open. As all open files, they are assigned to a file descriptor (an integer).
| File | File descriptor |
|---|---|
/dev/stdin or /dev/fd/0 |
0 |
/dev/stdout or /dev/fd/1 |
1 |
/dev/stderr or /dev/fd/2 |
2 |
|><>>< Input_data_file program1 | program2 > Output_data_file
A "filter" is a program which reads data from standard input, processes it in some way, and sends the processed data to standard output.
2> file2>> file
2>&1
1>&2
xargsRead remaining arguments from standard input instead of specifying them and execute the command.
cd /bin ls | xargs whatis
morelessmore./word to search for word, then n
to go to its following occurence, and N for the previous.vi is a powerful text editor, available on any UNIX like
system. vim stands for Vi IMproved.
With gvim or kvim, you can use the mouse.
emacs is another powerful programmers text editor,
and much more.
There are others:
neditkedit (KDE)gedit (Gnome)hexedit: hexadecimal or ASCII editorvi(m) basically has two modes of operation:
Command mode is the initial mode.
The commands you type are displayed in the status (bottom) line
(only some of them in vi).
They are executed with Enter.
Press a, A, i, I, o or 0 to enter Insert mode. Press Esc to return to command mode.
vimtutor is an interactive vim tutorial
(25 to 30 minutes).
filestatwctargzipgunzipbzip2bunzip2zipunzipunrarrsyncdiffpatchdiff file to an original.zdiffbzdiffdiff3pdiffdiffppwdiffsdiffcommcksummd5sumsha1sumcattacrevsplitcsplitjoinsortuniqcat *.txt | sort | uniq > result-file
headtailodhexdumpFilename expansion belongs to the shell. There is a more general and widely used tool for matching strings, called regular expressions. There are mainly three flavors:
| Expression | Match |
|---|---|
. |
Matches any single character. |
* |
Matches the last character or expression 0 or more times. |
+ |
Matches the last character or expression 1 or more times. |
? |
Matches the last character or expression 0 or 1 time. |
^ |
Matches the beginning of a line. |
$ |
Match the end of a line. |
[abcde] |
Matches exactly one character listed. |
[a-e] |
Matches exactly one character in the given range. |
[^abcde] |
any character that is not listed |
[^a-e] |
any character that is not in the given range |
\ |
Escapes special characters. |
greppcregrepagrepexpandunexpandtrflipcutpastefoldfmtnlcat -n)pra2pssedExamples:
sed "1,32 d" file.txtsed 's/[ ^t]*$//' file.txtDelete both leading and trailing whitespace from each line.sed 's/foo/bar/g' file.txtsed '/^Delete/d' file.txtsed --in-place 's/^\(.\{20\}\).*/\1/g' file| UNIX command | sed equivalent |
|---|---|
cat |
sed ':' |
tac |
sed '1!G;h;$!d' |
grep |
sed '/patt/!d' |
grep -v |
sed '/patt/d' |
head |
sed '10q' |
tail |
sed -e ':a' -e '$q;N;11,$D;ba' |
tail -f |
sed -u '/./!d' |
cut -c 10 |
sed 's/\(.\)\{10\}.*/\1/' |
cut -d: -f4 |
sed 's/\(\([^:]*\):\)\{4\}.*/\2/' |
tr A-Z a-z |
sed 'y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/' |
wc -l |
sed -n '$=' |
uniq |
sed 'N;/^\(.*\)\n\1$/!P;D' |
rev |
sed '/\n/!G;s/\(.\)\(.*\n\)/&\2\1/;//D;s/.//' |
basename |
sed 's,.*/,,' |
dirname |
sed 's,[^/]*$,,' |
cp orig new |
sed 'w new' orig |
The super user, root can do anything he wants,
including erasing the system.
An average user does not have the necessary permissions to modify any system file. Any software he uses can't either, as it inherits the user's permissions (alhough there are exceptions).
The use of the system as the root user must be
strictly limited to system administration tasks.
A running program (a process) has the same permissions as the user who executes it.
There are exceptions to this rule, but it explains why there are more
users of the system than root ant its real users
(see /etc/passwd).
Users usually belong to one or more groups
(see /etc/group).
Users and groups names are human-friendly identifiers. The systems uses integers: UIDs and GIDs.
idsuEvery file has:
There are three sorts of permissions:
r)w)x)And each file has permissions defined for three types of users:
They appear as follows in colums 2 to 10 of the output of ls -l.
The letter is replaced by a hyphen (-) where the permission
is not set.
| user | group | others | ||||||
r |
w |
x |
r |
w |
x |
r |
w |
x |
| Permission | Read | Write | eXecute |
|---|---|---|---|
| Letter | r | w | x |
| Octal valuee | 4 | 2 | 1 |
| Octal digit | Text equivalent | Binary value | Meaning |
|---|---|---|---|
| 0 | --- |
000 | No permission |
| 1 | --x |
001 | Execute |
| 2 | -w- |
010 | Write |
| 3 | -wx |
011 | Write + Execute |
| 4 | r-- |
100 | Read |
| 5 | r-x |
101 | Read + Execute |
| 6 | rw- |
110 | Read + Write |
| 7 | rwx |
111 | Read + Write + Execute |
| Permissions | on a File | on a Directory |
|---|---|---|
| Read | Read the file. | List the contents of the directory. |
| Write | Modify the file (can be emptied, but not deleted) | Create, delete or rename files in the directory. |
| Execute | Execute the file (the program) | Access to the contents of the directory. Read, write to or execute a file in the directory. |
| Special Permissions | on a File | on a Directory |
|---|---|---|
| SUID or Set User ID | A program is executed with the file owner's permissions (rather than with the permissions of the user who executes it). | Files created in the directory inherit its UID. |
| SGID or Set Group ID | The effective group of an executing program is the file owner group. | Files created in the directory inherit its GID. |
| Sticky (bit) | A program sticks in memory after execution. | Any user can create files, but only the owner of a file can delete it. |
| Permissions | Meaning |
|---|---|
--S------ |
SUID is set, but user (owner) execute is not set. |
--s------ |
SUID and user execute are both set. |
-----S--- |
SGID is set, but group execute is not set. |
-----s--- |
SGID and group execute are both set. |
--------T |
Sticky bit is set, bot other execute is not set. |
--------t |
Sticky bit and other execute are both set. |
System calls to create new files and directories have default or base permissions:
0777 or rwxrwxrwx.0666 or rw-rw-rw-.The first octal digit is for special permissions. The next three are for user, group and others.
Before creating the file or directory, the base permissions are compared to a mask that will "mask out" permission bits to determine the final permissions for the object being created.
The calculation to determine the final permissions is to take the binary of the base permissions and perform a logical AND operation on the ones complement representation of the binary umask.
umaskThe usual umask is 0022, which results in 0755
for directories, and 0644 for regular files.
0 0 2 2 0 7 7 7 0 6 6 6
000000010010 000111111111 000110110110
111111111111 111111101101 111111101101
------------ ------------ ------------
111111101101 000111101101 000110100100
0 7 5 5 0 6 4 4
chmodchownchgrpchmod u+x file
chmod 744 file
Variables are prefixed by a dollar sign ($).
A value is assigned to a variable through an equals sign (=)
(no whitespace on either side of it).
MY_VARIABLE='my value' echo $MY_VARIABLE
By default, variables are seen only within the shell itself. To pass variables to other programs invoked within the shell, they have to be exported to the environment.
setprintenvenvexportThe behaviour of many programs can be altered by environment variables.
| Variable | Meaning |
|---|---|
PATH |
Colon-separated list of directories to search for each command. |
EDITOR |
Default editor. |
DISPLAY |
In the X window system, a display consists (simplified) of a keyboard, a mouse and a screen. |
PS1 |
Prompt displayed before each new command. |
You have a X Window server running on your machine, at hostname.
You can run a client application on a remote machine, at rhostname,
and view it on the local machine.
On the remote client:
export DISPLAY=hostname:0
(or in general hostname:D.S where S is the screen
and D is the display.)
On the server:
xhost + rhostname xhost -
Xhost is a very insecure mechanism. Use ssh -X
to do this more easily and securely.
/etc/profile/etc/bash.bashrc~/.bash_profile, ~/.bash_login or
~/.profile~/.bashrcinit is the parent of all processes.init is a daemon.PID or process ID, an integer.PPID or parent process ID, an integer.RUID, or real user ID.
The user issuing the command.EUID, or effective user ID.
The one determining access permissions to system resources.RGID, or real group owner.
The group of the user who started the process.EGID, or effective group owner. Different from RGID
when SGID has been applied to a file.toppspstreetimeA program can be killed by sending it a termination signal.
killkillallxkillkill -15 is the default, and orders the process to terminate
in an clean way.
kill -9 sends a termination signal which can't be ignored.
Process priority is calculated from the nice number, and recent CPU usage of the process.
nicereniceJob control means switching between several jobs or processes.
A foreground job occupies the terminal which initialized it. It can be put in the background, so that the terminal can accept new commands.
command &command in the background.bgfgjobsn
and can be referred to by %n.atat hh:mm mm/dd/yy at> echo "File created using the at utility" > atfile at> ^D
at now + 1 hour
atqat -l.atrmbatchThe cron daemon examines crontab files every minute, and
checks if the command in each line should be run in the current minute.
crontabcrontab -e
minute hour day_of_month month day_of_week command
| Character | Meaning or Use |
|---|---|
newline space tab |
Word separators |
; |
Command separator |
& |
Background execution |
( ) |
Command grouping (subshell) |
{ } |
Command block |
| > < & |
Redirections |
* ? [ ] ~ ! |
Filename metacharacters |
` |
Command substitution |
$ |
Variable (or command) substitution |
# |
Comment |
\ " ' |
Escape or quote other characters |
File hello.sh:
#!/bin/bash echo Hello World
The first line can be ommitted to run the script using a non interactive
shell: bash hello.sh.
Execute permission must be set to simply run it: ./hello.sh.
If the directory which contains the file is in the PATH
environment variable, you don't need the path: hello.sh.
Character codes map characters to numbers encoded as one or more bytes.
Unicode imposes additional constraints on implementations to ensure that they treat characters uniformly across platforms and applications.
| Operating System | Hexadecimal Newline Character(s) |
|---|---|
| Microsoft Windows | 0D 0A |
| Apple Macintosh OS 9 and earlier | 0D |
| UNIXes, GNU/Linux, Mac OS X and higher | 0A |
flipMany commands have a --help option.
helpmaninfowhatisapropos or man -kAnd you are free to study and use existing Free code.
