Lets start with Sticky bit first. Since this is the most simplest to explain. Setting the sticky bit tells Unix that once the concerned application is executed, it should remain in memory. Remember that Unix is a multi-user OS and was mainly designed so that multiple users can work simultaneously. Thus the logic used is that a program that exists in memory requires lesser time to start when a new user requests for the same program. Thus when one user has just used a program and then a new user wants to use the same program, the second user doesn’t have to face a time delay for the program to initialize itself. It would be readily available to him. The concept of the sticky bit was a very useful one, long back when fast disk access and other memory access technologies weren’t around. But in today’s age the concept of sticky bit is obsolete, since modern day technology is advanced enough to reduce the time delay while loading applications into the memory. Thus currently the sticky bit is of very little significance. Sticky bit is only associated with executables.

SUID (Set User ID) Bit

Sometime you may faced an error while trying to run any application stating that the application must be ‘SUID root’ . You might have been confused that time, but now once you read this article you would no longer find it confusing.

SUID stands for Set User ID. This means that if the SUID bit is set for any application then your user ID would be set as that of the owner of application/file rather than the current user, while running that application. That means in case I have an application whose owner is ‘ root ‘ and it has its SUID bit set, then when I run this application as a normal user, that application would still run as root. Since the SUID bit tells Linux that the the User ID root is set for this application and whenever this application executes it must execute as if root was executing it (since root owns this file).

In case you have really understood the above you may be wondering – isnt this a major security risk? If users are able to run applications as root, then it must be definitely posing as a threat to the security of the system. Actually the SUID is used to increase the security in a way. Let me explain this with my own example I use on my machine.

One way I use SUID on my machine

I have a few files that I modify through Linux and then before I shutdown Linux I have to transfer them to my Windows partition for further use there. As a normal user I do not have write access to the Windows partitions that I have mounted. So I have to be the superuser to be able to write to that partition. I have created a simple shell script that copies my files to the Windows partitions. This script was created by root user and the SUID bit was set. Access rights to this script have been given to all users. Now whenever I want to copy my files I simply run this script. Even though I have logged in as a normal user, the SUID bit which is set causes this script to execute as if the root was executing it and it allows me to write to the Windows partitions.

Had the SUID bit not been set, I would have to type ‘ su ‘ at the prompt and get temporary superuser access to get write access to the Windows partitions. Hope you got the point..

Note : In case you do not know how to access your Windows partitions through Linux, refer to Article No. 3

You may be thinking that since these applications would run as root they can do harmful things and destroy the system. The concept behind SUID bit is that you as the superuser would be able to allow certain applications / scripts to be run by the users as if they were the superuser for the time being. What these application / scripts do when they execute should be completely known to you. Even though the users would be allowed to execute these programs as root they would be able to do ONLY THOSE things that these programs were designed to do. So in case a script was designed to copy 5 files from one place to another. Then the user who would run that script would be able to ONLY copy those 5 files from one place to another. He would not be able to modify that script in any way since he would not have write access to the script. He would only be having execute rights for that script. Hence its an excellent idea to allow users to do some important backup using a script that does only that and by setting the SUID bit for that script. This way the users don’t have to know the superuser password but can still use some facilities that are only available to the superuser

Important : Think twice before setting the SUID bit for scripts (owned by root) that take arguments at the command line. Since you never know what parameters a malicious user may pass to your script. Since the script would run as root it could do great damage if misused.

SGID (Set Group ID) bit

Just like SUID, setting the SGID bit for a file sets your group ID to the file’s group while the file is executing. IT is really useful in case you have a real multi-user setup where users access each others files. As a single homeuser I haven’t really found a lot of use for SGID. But the basic concept is the same as the SUID, the files whose SGID bit are set would be used as if they belong to that group rather than to that user alone.

Note : Making SUID and SGID programs completely safe is very difficult (or maybe impossible) thus in case you are a system administrator it is best to consult some professionals before giving access rights to root owned applications by setting the SUID bit. As a home user (where you are both the normal user and the superuser) the SUID bit helps you do a lot of things easily without having to log in as the superuser every now and then.

source : http://www.codecoffee.com

This article explains 2 simple commands that most people want to know when they start using Linux. They are finding the size of a directory and finding the amount of free disk space that exists on your machine. The command you would use to find the directory size is ‘ du ‘. And to find the free disk space you could use ‘ df ‘.

All the information present in this article is available in the man pages for du and df. In case you get bored reading the man pages and you want to get your work done quickly, then this article is for you.

-

‘du’ – Finding the size of a directory

$ du
Typing the above at the prompt gives you a list of directories that exist in the current directory along with their sizes. The last line of the output gives you the total size of the current directory including its subdirectories. The size given includes the sizes of the files and the directories that exist in the current directory as well as all of its subdirectories. Note that by default the sizes given are in kilobytes.

$ du /home/david
The above command would give you the directory size of the directory /home/david

$ du -h
This command gives you a better output than the default one. The option ‘-h’ stands for human readable format. So the sizes of the files / directories are this time suffixed with a ‘k’ if its kilobytes and ‘M’ if its Megabytes and ‘G’ if its Gigabytes.

$ du -ah
This command would display in its output, not only the directories but also all the files that are present in the current directory. Note that ‘du’ always counts all files and directories while giving the final size in the last line. But the ‘-a’ displays the filenames along with the directory names in the output. ‘-h’ is once again human readable format.

$ du -c
This gives you a grand total as the last line of the output. So if your directory occupies 30MB the last 2 lines of the output would be

30M .
30M total

The first line would be the default last line of the ‘du’ output indicating the total size of the directory and another line displaying the same size, followed by the string ‘total‘. This is helpful in case you this command along with the grep command to only display the final total size of a directory as shown below.

$ du -ch | grep total
This would have only one line in its output that displays the total size of the current directory including all the subdirectories.

Note : In case you are not familiar with pipes (which makes the above command possible) refer to Article No. 24 . Also grep is one of the most important commands in Unix. Refer to Article No. 25 to know more about grep.

$ du -s
This displays a summary of the directory size. It is the simplest way to know the total size of the current directory.

$ du -S
This would display the size of the current directory excluding the size of the subdirectories that exist within that directory. So it basically shows you the total size of all the files that exist in the current directory.

$ du –exculde=mp3
The above command would display the size of the current directory along with all its subdirectories, but it would exclude all the files having the given pattern present in their filenames. Thus in the above case if there happens to be any mp3 files within the current directory or any of its subdirectories, their size would not be included while calculating the total directory size.

-

‘df’ – finding the disk free space / disk usage

$ df
Typing the above, outputs a table consisting of 6 columns. All the columns are very easy to understand. Remember that the ‘Size’, ‘Used’ and ‘Avail’ columns use kilobytes as the unit. The ‘Use%’ column shows the usage as a percentage which is also very useful.

$ df -h
Displays the same output as the previous command but the ‘-h’ indicates human readable format. Hence instead of kilobytes as the unit the output would have ‘M’ for Megabytes and ‘G’ for Gigabytes.

Most of the users don’t use the other parameters that can be passed to ‘df’. So I shall not be discussing them.

I shall in turn show you an example that I use on my machine. I have actually stored this as a script named ‘usage‘ since I use it often.

Example :

I have my Linux installed on /dev/hda1 and I have mounted my Windows partitions as well (by default every time Linux boots). So ‘df’ by default shows me the disk usage of my Linux as well as Windows partitions. And I am only interested in the disk usage of the Linux partitions. This is what I use :

$ df -h | grep /dev/hda1 | cut -c 41-43

This command displays the following on my machine

45%

Basically this command makes ‘df’ display the disk usages of all the partitions and then extracts the lines with /dev/hda1 since I am only interested in that. Then it cuts the characters from the 41st to the 43rd column since they are the columns that display the usage in % , which is what I want.

Note : In case you are not familiar with pipes (which is used in the above command) then refer to Article No. 24 . ‘cut’ is another tool available in Unix. The above usage of cut gets the the characters that are present in the specified columns. If you are interested in knowing how to mount you Windows partitions under Linux, please refer to Article No. 3 .

There are a few more options that can be used with ‘du’ and ‘df’ . You could find them in the man pages.

source : http://www.codecoffee.com

In Linux every file present on the disk has associated permissions with it. These permissions decide on who and in what manner these files should be used. The rest of this article explains these file / directory permissions in details.

In order to view the permissions associated with a file, you could use the ‘ ls ‘ command. On executing ‘ ls ‘ you would be presented with a directory listing with one filename per line. I shall explain file permissions with the help of a sample output as shown below

Yours would obviously be different from this one. But this output should be enough to explain file permissions. The above output shows that within the current directory there are 3 entries. Lets start with the 2nd line.

The first character ‘f‘ indicates that ‘ viewresume ‘ is a file. In case it was the name of a directory there would have been a ‘d‘ instead of a ‘f‘

The next part rwxr-xr-x (a total of 9 characters) should be spilt into 3 parts each consisting of 3 consecutive letters

The meaning of these 3 characters which form this 9 character sequence is shown in the table below.

In Part 1 r,w and x, all the 3 permissions exist. This means that the the concerned file, ‘ viewresume ‘ can be read, written to as well as executed. Thus in case you want to just read the contents of that file you could do so. In case you want to modify the file that too would be allowed. Assuming that ‘ viewresume ‘ is some kind of a script it also has execute permissions assigned to it. So you could execute this program from the shell prompt as well.

Some of you’ll who are really smart must have already started thinking about how you could protect your data from others (in a multi-user system) when you have provided r,w and x permissions to the file. That is exactly why there is a 9 character sequence present instead of just 3 characters.

Part 1 decides the permissions for the User (the owner of the file)
Part 2 decides the permissions for other users who belong to the same Group as the file
Part 3 decides the permissions for Others (rest of the world) who might access your folder

As the owner any file you create would be having the r and w permissions present. In case its a script you should also add the execute permission. This is explained in a later section of this article.

In case you are a part of a project involving other users, you should ask the administrator to create a separate group and include all the project members in that group. Then you could create all your programs as a part of that group and use the group permissions so that only those members belonging to your project group can read, modify your files that concern to that project.

For others (rest of the world) it is always best to leave the default permissions which would be generally r and x. Never ever give w permissions to all, else anyone would be able to modify your files.

The rest of the fields don’t have anything to with file permissions as such. So they shall not be dealt with in this article. Now lets consider the 3rd line in the ‘ ls ‘ output.

This shows the permissions for a file aptly named ‘ privatedata.txt ‘. The name itself suggests that this is some important file that only the owner of the file should be allowed to read, write or execute. Thus no one else (group or others) should be allowed to even view the contents of this file. Thus you can see that the permissions for the file are rwx——

Dividing it into 3 parts you would get ‘ rwx ‘ and ‘ — ‘ and ‘ — ‘

The – (hyphen) indicates that the particular property is not existing for that file or directory.

Thus in this case the 2nd and 3rd Part only consist of hyphens thus indicating that neither the Group members nor Others would be allowed to either read, write or execute this file. You on the other hand have all these 3 properties set so that you are free to do anything with the file.

Now consider the 1st line in the ‘ ls ‘ output

Note that the first character on the line is a ‘ d ‘ which indicates that ‘ tutorials ‘ is the name of a directory and not a file.

Important : The permissions for directories take on a slightly different meaning than those for files. This is explained in some detail.

In our example the ‘ tutorials ‘ directory has r and x permissions set for group and world. So basically all the users could view the files that are present within that directory. Since the w permission is missing for group and world, they cannot modify add or delete any of the files within the tutorials directory (unless there is a situation as described in the Note below). You on the other hand as usual are allowed to do as you wish. Doesn’t Linux make you feel powerful !!

Now for some technical language. Though I have been calling these r,w and x as permissions, you would generally call them as bits. So don’t look surprised when a Group members asks you to set the read bit for a directory. It basically means, set the read permissions for that directory.

Important : As beginners until you are completely familiar with file permissions, remember one important rule. Never give a directory lesser privileges and the files within that directory more privileges. I mean in case you do not set the x bit for a directory and set the w bit for the files within that directory. Though you may expect that since the directory doesn’t have the x bit set, users cannot enter the directory and so they wont be able to modify your files. Actually the meaning of the x bit for directories is not so simple to understand. Setting permissions as above would allow anyone to delete all your files in that directory. So always give the equal or lesser privileges to the files within a directory as that to the directory itself. If you don’t want the users to have write permissions to your files, remember not to set the w permissions for the files rather than trying to restrict write access using the directory permissions.

Now that you have learnt what file permissions are, the next obvious part is to learn how to modify them. You have to use the chmod command to change the permissions of a file or directory. To run chmod on a file you should either own the file or you should be the superuser .

The way to use chmod is

$ chmod [newpermissions] [filenames]

Now comes a bit tricky part for beginners (more so for those who don’t have a mathematical background), but I shall try to explain the problem. For chmod the newpermissions have to set using an octal number rather than a decimal number. In case you understood the previous sentence, then you have no problems. If you didn’t then read the next paragraph.

Note : In case you don’t want to understand the octal system method there is a simpler method stated at the end of this article. But the octal number method is almost used by all (atleast by those who consider themselves to be powerusers)

I will not explain the concepts behind octal numbers. I shall only talk about the octal numbers that could be used with chmod. Below are the octal numbers representing different permissions

You have learnt that there are 9 bits associated with every file / directory (split into 3 parts) to decide the permissions. So in case you have the r,w,x permissions set for a file translate that to a 111 which you should further translate to the number 7 using the above table.

Suppose there is a file with the following permissions as shown in this sample ‘ ls ‘ output

The existing permissions for the above file in octal numbers could be represented as follows

rwxr-xr-x ==> 111101101 ==> 755

That’s it!! I guess it wasn’t so tough after all. Use the above table and figure out the permissions for other files as well. Once you get used to these conversions, you would be able to do it in no time.

Now in case you want to change the permissions so that group members and others can neither read nor execute this file, you would require the new permissions to look something like the following

rwx—— ==> 111000000 ==> 700

So the exact command that you would be typing at the prompt would be

$ chmod 700 viewresume

Now check the permissions of the file once again with an ‘ ls ‘ command and you would see the changes that you just made.

For your quick reference here are a few standard numeric codes (that’s what it is called) that are often used..

Frequently used numeric parameters for chmod
755	The general preferred permissions for almost all the files on your disk
700	Extremely private data
500	Extremely private data that you would not like to accidentally modify. So write protect it
775	General files used when working as a Group (Others can only view/execute your files)
770	Important files used when working as a Group (Others cannot do anything with your files)
750	Allowing group to view your files but no write access (Others cannot do anything with your files)
777	Something you should never want to do

There’s another method to change the permissions of files rather than using these octal numbers (in case you just didn’t get the hang of it). I prefer the octal number method. Others may prefer the following method

$ chmod g-r,g-x,o-r,o-x viewresume

The above command does exactly the same thing that ‘ chmod 700 ‘ command did. Yeah this one is lengthier but its simpler to understand. Its explained below in case you couldn’t figure it..

I guess you got the point.. the other 2 parameters (g-x,o-r) can also be expanded in the same way. Thus the above command asks Linux to remove the r and x permission for both the group members and others (rest of world).

Here is a quick reference if you prefer to use this method (its called the symbolic method)

Here is another example to make things more clear.

$ chmod g=rwx myprogram.c

The above command would give the group that the file belongs to, read-write-execute permissions irrespective of what the previous permissions were (for the file named myprogram.c)

I have discussed how to use chmod with parameters in numeric mode(755,700, etc.) in more detail than using it with parameters in the symbolic mode (u,g,o, etc.). This is because I have never used the symbolic mode of chmod. I had to refer to my books to get the technical details for this article. I have been using the octal numeric mode since the first time I used chmod.

source : http://www.codecoffee.com/

Linux comes with quite a few shells such as Bourne Shell, Bourne Again Shell, C Shell, Korn Shell, etc. The default shell for Redhat Linux is ‘ bash ‘ which is very popular since being the default, most users start by learning bash. I shall talk about the bash shell only in this article.

Windows users would be familiar with a program called command.com which had to be present for the OS to boot. Command.com is the Windows equivalent of the Linux shell.

Typing the following at the shell

$ echo $SHELL

would give you the name of the current shell you are using. It would most probably be the bash shell in case you are a new user and have been assigned the default shell.

The bash shell is actually a program that is located at /bin/bash and is executed by Linux the moment a user successfully logs in after entering his user-pass. Once this shell starts, it takes over control and accepts all further user commands. The bash shell presents a $ prompt by default (for normal user accounts). You can change this prompt to whatever you like but leaving it at the default is best. Other shells present different prompts. Changing the prompt is explained below.

-

Shell Environment

All the programs that run under Linux are called as processes. Processes run continuously in Linux and you can kill or suspend different processes using various commands. When you start a program a new process is created. This process runs within what is called an environment. This particular environment would be having some characteristics which the program/process may interact with. Every program runs in its own environment. You can set parameters in this environment so that the running program can find desired values when it runs.

Setting a particular parameter is as simple as typing VARIABLE=value . This would set a parameter by the name VARIABLE with the value that you provide.

To see a list of the environment variables that are already set on your machine, type the following

$ env

This would produce a long list. Just go through the list before reading the next part of the article. Linux by default sets many environment variables for you. You can modify the values of most of these variables. A few of the variables that are set are

HOME=/home/david

would set the home directory to /home/david. This is perfect in case your login name is david and you have been given a directory named /home/david . In case you don’t want this to be your home directory but some other one you could indicate so by typing the new directory name. The HOME directory is always the directory that you are put in when you login.

There are many advantages of using the HOME variable. You can always reach your home directory by only typing ‘ cd ‘ at the prompt, irrespective of which directory you are presently within. This would immediately transfer you to your HOME directory. Besides in case you write scripts that have $HOME present in them to refer to the current HOME directory, these scripts can be used by other users as well since $HOME in their case would refer to their home directories.

-

PATH=/usr:/bin/:usr/local/bin:.

This is a very important environment variable. This sets the path that the shell would be looking at when it has to execute any program. It would search in all the directories that are present in the above line. Remember that entries are separated by a ‘ : ‘ . You can add any number of directories to this list. The above 3 directories entered is just an example.

Note : The last entry in the PATH command is a ‘ . ‘ (period). This is an important addition that you could make in case it is not present on your system. The period indicates the current directory in Linux. That means whenever you type a command, Linux would search for that program in all the directories that are in its PATH. Since there is a period in the PATH, Linux would also look in the current directory for program by the name (the directory from where you execute a command). Thus whenever you execute a program which is present in the current directory (maybe some scripts you have written on your own) you don’t have to type a ‘ ./programname ‘ . You can only type ‘ programname ‘ since the current directory is already in your PATH.

Remember that the PATH variable is a very important variable. In case you want to add some particular directory to your PATH variable and in case you try typing the following

PATH =/newdirectory

This would replace the current PATH value with the new value only. What you would want is to append the new directory to the existing PATH value. For that to happen you should type

PATH=$PATH:/newdirectory

This would add the new directory to the existing PATH value. Always a $VARIABLE is substituted with the current value of the variable.

-

PS1=boss

PS1 is the shell prompt. It defines what you want your shell prompt to look like. By default it looks like a ‘ $ ‘ in bash shell. The above case would replace the default ‘ $ ‘ with a new ‘boss’ . Hence an ls command would look something like

boss> ls

All your commands would now be typed at a ‘ boss ‘ prompt instead of a ‘ $ ‘ prompt.

-

SHELL=/bin/bash

This tells where the program that represents your shell is to be found. In case you typed /bin/ksh in the above, then your bash shell would be replaced with the ksh shell (korn shell). So in case you are not happy with the bash shell, you could replace the bash with some other shell.

-

LOGNAME=david

The LOGNAME is automatically set for you as the same as your login name. This variable is used in case you want to use your own login name in any script. This is the simplest way of getting your login name from within a script. Thus in case you use $LOGNAME in any script the script would work for all users since the LOGNAME always holds the name of the current user.

A good use is in case you have been given a temporary directory to work with and to make temporary files then you would want to delete the files that you created. You could use a command with the $LOGNAME in it to locate files that were created by you and then you could pass the result of this command to a rm command. This would be a neat way to delete all the files at one go, rather than find them one at a time.

Note : Finding files based on a particular criteria and then passing those files to another program is explained in Article No. 21

-

There are more environment variables then the ones that are mentioned here. But most users would find the ones given here to be useful.

Important : To make the above changes permanent (that is it should work every time you login) make the changes to the .profile file that exists in your HOME directory. Simply type the required commands one line for each. And there you go. It will be available every time you login. You could check the variables using ‘env’ command.

source : http://www.codecoffee.com

I call this a Special Guide since the information in this article helps you with many of your commands and is not restricted to any particular application as such. Thus reading this article is a must for all Linux users.

-

In Linux whenever you are not sure about the name of a file and you want to do something with files such as either search for them or copy them or delete some files based on some knowledge you have about the filenames then you can use Wildcards. Wildcards are basically an indicator to the shell that some particular part of the filename is not known to you and the shell can insert a combination of characters in those places and then work on all the newly formed filenames. This concept would be clear by the end of this article

There are 3 types of wildcards that can be used Linux. They are the * ? and [] . All the 3 shall be explained in detail.

* (Asterisk) Wildcard

This represents any sequence of characters. This means that if you include a * in your filename then that part of the filename can be formed using any sequence of characters. The example below explains this concept

$ cat article* > combinedarticle
Would find all the files that begin with the letter sequence ‘ article ‘ and can have anything following those letters. I mean article01 or article10 or articlenew.. and any such file would be considered. All these files would be merged and would be written to a file named combinedarticle

$ ls *gif
Would list all the files having the letters ‘ gif ‘ in them. So a file named somegifs.txt as well as a file named 123.gif would be listed in the output.

$ ls *.gif
Would list all the files having ‘ gif ‘ as the extension of their filename. Thus in this case a file named somegifs.txt file would NOT be listed since its extension is not ‘ gif ‘. Whereas a file named 123.gif would be listed.

$ ls *day*
Would list all the files that have the letters ‘ day ‘ anywhere in their filenames. Thus files such as today.txt , dayone.txt and lastday.gif would all be listed in the output.

$ ls .*gif*
Would list all the hidden files in the current directory that have the letters ‘ gif ‘ in their filenames. Hidden files in Linux begin with a . (period) in their filenames.

$ pl * a.txt
Notice that there is a space between the * and ‘ a.txt ‘ . It is this space that causes the command to act as if 2 parameters have been passed to ‘ pl ‘ rather than one. The above command would print all the files that are present in the current directory. Once that is done it would proceed to the next file named a.txt and would print that also if it exists.

Note : The * would not work with the ‘ . ‘ (period) that exists in filenames. Thus in case you use ‘ *a ‘ and there is a file whose name begins with ‘ .a ‘ , it would not be listed. When a . (period) is the first character in a filename then the file becomes a hidden file in Linux.

-

? (Question Mark) Wildcard

This represents only one character which can be any character. Thus in case you know a part of a filename but not one letter, then you could use this wildcard. If there are many files that are named such as article10, article11 and so on till article19 you could get all these files by using article1? . In this case the ? would be interpreted as any one character and it would find all the files like article10, article11..and so on till article19, since all these files differ in their names by only the last letter.

$ ls article1?
Would list all the files that begin with ‘ article1 ‘ and have one more character in their names which can be any one valid character.

$ ls ??.gif
Would list all the .gif files in the current directory that have only 2 characters before the extension. Thus files such as ab.gif or xy.gif would be listed but 123.gif would not be listed

Remember that the ? means any ONE character to be substituted in the place of the ?

$ ls *.???
Would list all the files in the current directory that have a file extension of 3 characters in length. Thus files having extensions such as .gif , .jpg , .txt would be listed.

-

[] (Square Brackets) Wildcard

This represents a range of characters. So in case you have files that are named article10, article11, article12..all the way till article19 then you could select all the first 5 of them using the above wildcard as shown below

$ ls article1[0-4]
Remember that [] represents a range from which any character can be present. This range can be something like [0-4] or [1-9] or anything like that in case of numbers. Letters could also be selected such as [a-g] or [F-Z] or [A-Z].

Note : Remember that in Linux the filenames are case sensitive, Thus a range of [a-z] is different from [A-Z]

$ ls beckham[123].jpg
Would list all the files that begin with the letter sequence ‘ beckham ‘ and end with either a 1, 2 or 3. Thus the possible filenames that could be listed (if they exists) are beckham1.jpg , beckham2.jpg and beckham3.jpg

$ ls [a-d,A-D]*.jpg
This would list all the files that have an extension as .jpg and have as their first letter either a,b,c,d,A,B,C or D . The [ , ] imply that this entire range indicates ONLY ONE letter which can be from any of the two given sub-ranges. A comma is used to merge different range of letters or numbers.

Note : I would once again like to mention that [a-d,p-z] does NOT mean that there can be two letters, the first one from a to d and the second one from p to z. It means that there is ONLY ONE letter, and that letter can be from either a to d or from p to z.

In case you want to specify the range for 2 characters in the filenames then use the following as follows
$ ls beckham[0-9][0-9]

Read the second part of this article to know about the use of special characters such as quotes and apostrophe marks.

Now I shall discuss the use of special characters such as ” ‘ and ` in various commands that you type at the shell prompt. The information given here is general and has to be followed when typing any command.

” ” (Double Quotes) : Suppress Expansion

Whenever you use double quotes (” “) the shell suppress the filename expansion. Thus even if you use a wildcard such as * but enclose it within double quotes you would not get the standard feature of matching for all characters. I mean a command such as

$ ls c*
would list all the files with the names beginning with the letter ‘ c ‘ . But a command such as

$ ls “c*”
would search for a file named ‘ c* ‘. There would be no expansion of the * to match other letter sequences. The shell would expect the filename to have the actual character * in its name. Thus you would mostly get a ‘No File Found error’ message.

-

` (Back Quotes) : Command Substitution

The ` character (found on the key with the ~) is very important when used in shell commands. This ` indicates that command substitution is required wherever it is used. Hence whenever ` is used, whatever part of the command is enclosed by these Backquotes marks would be executed (as if it was the only command) and then the result of that command would be substituted in the original shell command that you typed. The following explains this clearly

$ echo “The contents of this directory are ” `ls -l` > dir.txt

Note : Remember to use the ` (found on the key with the ~ and NOT the one found next to the Enter button)

The above command would basically execute the ‘ ls -l ‘part first and then substitute the result after the string “The contents of this directory are ” and both of these together (directory listing + the string) would be written to a file named dir.txt

Basically after command substitution the new substituted value would act as additional parameter to the main command that was present in your initial statement.

-

‘ (Apostrophe Marks) : No Change

The ‘ character (found on the button next to the Enter button) is a very powerful character whenever used in any shell command. Basically the ‘ (apostrophe marks) disables all kinds of transformations or modifications. It would consider whatever is enclosed with the ‘ marks as a single entity i.e. a single parameter. Absolutely no sort of substitution or expansion would take place.

$ echo ‘$HOME’
would produce at the output the string $HOME itself and would not print the path to your home directory. Since the single quotes prevents any sort of expansion, substitution and simple considers whatever to be present as a simple parameter in itself.

Just so that you remember in case you had typed the following

$ echo `$HOME`
(with the backquotes) you would get an error stating that the command not found. Since in this case the $HOME would be substituted with the path to your home directory (suppose /home/david) and the shell would try to execute the path as such. It would search for a program named (such as) /home/david. Remember that backquotes cause it to consider the part within the quotes to be considered as separate command and the output of that command would be substituted here. Hence in this case there is no command / program named as /home/david. Thus you would get an error when bash tries to execute that command

On the other hand when you type

$ echo “$HOME” or $ echo $HOME
You would get the expected output. i.e the path to your home directory would be printed at the output.

Thus you are now familiar with forming various filenames using wildcards. You would generally end up using the special characters such as quotes when trying to make complex shell commands. With this knowledge I hope you can get the shell to do some real good stuff for you.

This is definitely much more than you must have expected… Once you read this you would no longer be a beginner.. you would already be on your way to master Bash programming!!

Arithmetic with Bash

bash allows you to perform arithmetic expressions. As you have already seen, arithmetic is performed using the expr command. However, this, like the true command, is considered to be slow. The reason is that in order to run true and expr, the shell has to start them up. A better way is to use a built in shell feature which is quicker. So an alternative to true, as we have also seen, is the “:” command. An alternative to using expr, is to enclose the arithmetic operation inside $((…)). This is different from $(…). The number of brackets will tell you that. Let us try it

As always, whichever one you choose, is purely up to you. If you feel more comfortable using expr to $((…)), by all means, use it.

bash is able to perform, addition, subtraction, multiplication, division, and modulus. Each action has an operator that corresponds to it

Everyone should be familiar with the first four operations. If you do not know what modulus is, it is the value of the remainder when two values are divided. Here is an example of arithmetic in bash

Again, the above code could have been done with expr. For instance, instead of add=$(($x + $y)), you could have used add=$(expr $x + $y), or, add=`expr $x + $y`.

Reading User Input

Now we come to the fun part. You can make your program so that it will interact with the user, and the user can interact with it. The command to get input from the user, is read. read is a built in bash command that needs to make use of variables, as you will see

The variable here is user_name. Of course you could have called it anything you like. read will wait for the user to enter something and then press ENTER. If the user does not enter anything, read will continue to wait until the ENTER key is pressed. If ENTER is pressed without entering anything, read will execute the next line of code. Try it. Here is the same example, only this time we check to see if the user enters something

Here, if the user presses the ENTER key without typing anything, our program will complain and exit. Otherwise, it will print the greeting. Getting user input is useful for interactive programs that require the user to enter certain things.

Functions

Functions make scripts easier to maintain. Basically it breaks up the program into smaller pieces. A function performs an action defined by you, and it can return a value if you wish. Before I continue, here is an example of a shell program using a function

Try running the above. The function hello() has only one purpose here, and that is, to print a message. Functions can of course be made to do more complicated tasks. In the above, we called the hello() function by name by using the line

When this line is executed, bash searches the script for the line hello(). It finds it right at the top, and executes its contents.

Functions are always called by their function name, as we have seen in the above. When writing a function, you can either start with function_name(), as we did in the above, or if you want to make it more explicit, you can use the function function_name(). Here is an alternative way to write function hello()

Functions always have an empty start and closing brackets: “()”, followed by a starting brace and an ending brace: “{…}”. These braces mark the start and end of the function. Any code enclosed within the braces will be executed and will belong only to the function. Functions should always be defined before they are called. Let us look at the above program again, only this time we call the function before it is defined

Here is what we get when we try to run it

$ ./hello.sh
Calling function hello()…
./hello.sh: hello: command not found
You are now out of function hello()

As you can see, we get an error. Therefore, always have your functions at the start of your code, or at least, before you call the function. Here is another example of using functions

In order for this to work properly, you will need to be the root user, since adduser is a program only root can run. Hopefully this example (short as it is) shows the usefulness of functions.

Trapping

You can use the built in command trap to trap signals in your programs. It is a good way to gracefully exit a program. For instance, if you have a program running, hitting CTRL-C will send the program an interrupt signal, which will kill the program. trap will allow you to capture this signal, and will give you a chance to either continue with the program, or to tell the user that the program is quitting. trap uses the following syntax

action is what you want to do when the signal is activated, and signal is the signal to look for. A list of signals can be found by using the command trap -l. When using signals in your shell programs, omit the first three letters of the signal, usually SIG. For instance, the interrupt signal is SIGINT. In your shell programs, just use INT. You can also use the signal number that comes beside the signal name. For instance, the numerical signal value of SIGINTis 2. Try out the following program

Now, while the program is running and counting down, hit CTRL-C. This will send an interrupt signal to the program. However, the signal will be caught by the trap command, which will in turn execute the sorry() function. You can have trap ignore the signal by having “”” in place of the action. You can reset the trap by using a dash: “-”. For instance

When you reset a trap, it defaults to its original action, which is, to interrupt the program and kill it. When you set it to do nothing, it does just that. Nothing. The program will continue to run, ignoring the signal.

Boolean AND & OR

We have seen the use of control structures, and how useful they are. There are two extra things that can be added. The AND: “&&” and the OR “||” statements. The AND statement looks like this

The AND statement first checks the leftmost condition. If it is true, then it checks the second condition. If it is true, then the rest of the code is executed. If condition_1 returns false, then condition_2 will not be executed. In other words

Here is an example making use of the AND statement

Here, we find that x and y both hold the values we are checking for, and so the conditions are true. If you were to change the value of x=5 to x=12, and then re-run the program, you would find that the condition is now false.

The OR statement is used in a similar way. The only difference is that it checks if the leftmost statement is false. If it is, then it goes on to the next statement, and the next

In pseudo code, this would translate to the following

Therefore, any subsequent code will be executed, provided at least one of the tested conditions is true

Here, you will see that one of the conditions is true. However, change the value of y and re-run the program. You will see that none of the conditions are true.

If you think about it, the if structure can be used in place of AND and OR, however, it would require nesting the if statements. Nesting means having an if structure within another if structure. Nesting is also possible with other control structures of course. Here is an example of a nested if structure, which is an equivalent of our previous AND code

This achieves the same purpose as using the AND statement. It is much harder to read, and takes much longer to write. Save yourself the trouble and use the AND and OR statements.

Using Arguments

You may have noticed that most programs in Linux are not interactive. You are required to type arguments, otherwise, you get a “usage” message. Take the more command for instance. If you do not type a filename after it, it will respond with a “usage” message. It is possible to have your shell program work on arguments. For this, you need to know the “$#” variable. This variable stands for the total number of arguments passed to the program. For instance, if you run a program as follows:

$ foo argument

$# would have a value of one, because there is only one argument passed to the program. If you have two arguments, then $# would have a value of two. In addition to this, each word on the command line, that is, the program’s name (in this case foo), and the argument, can be referred to as variables within the shell program. foo would be $0. argument would be $1. You can have up to 9 variables, from $0 (which is the program name), followed by $1 to $9 for each argument. Let us see this in action

This program expects one, and only one, argument in order to run the program. If you type less than one argument, or more than one, the program will print the usage message. Otherwise, if there is an argument passed to the program, the shell program will print out the argument you passed. Recall that $0 is the program’s name. This is why it is used in the “usage” message. The last line makes use of $1. Recall that $1 holds the value of the argument that is passed to the program.

Temporary Files

Often, there will be times when you need to create a temporary file. This file may be to temporarily hold some data, or just to work with a program. Once the program’s purpose is completed, the file is often deleted. When you create a file, you have to give it a name. The problem is, the file you create, must not already existing in the directory you are creating it in. Otherwise, you could overwrite important data. In order to create a unique named temporary file, you need to use the “$$” symbol, by either prefixing, or suffixing it to the end of the file name. Take for example, you want to create a temporary file with the name hello. Now there is a chance that the user who runs your program, may have a file called hello, so that would clash with your program’s temporary file. By creating a file called hello.$$, or $$hello instead, you will create a unique file. Try it

$ touch hello
$ ls
hello
$ touch hello.$$
$ ls
hello     hello.689

There it is, your temporary file.

Return Values

Most programs return a value depending upon how they exit. For instance, if you look at the manual page for grep, it tells us that grep will return a 0 if a match was found, and a 1 if no match was found. Why do we care about the return value of a program? For various reasons. Let us say that you want to check if a particular user exists on the system. One way to do this would be to grep the user’s name in the /etc/passwd file. Let us say the user’s name is foobar

$ grep “foobar” /etc/passwd
$

No output. That means that grep could not find a match. But it would be so much more helpful if a message saying that it could not find a match was printed. This is when you will need to capture the return value of the program. A special variable holds the return value of a program. This variable is $?. Take a look at the following piece of code

Now when you run the program, it will capture the return value of grep. If it equals to 0, then a match was found and the appropriate message is printed. Otherwise, it will print that there was no match found. This is a very basic use of getting a return value from a program. As you continue practicing, you will find that there will be times when you need the return value of a program to do what you want.

If you happen to be wondering what 2>&1 means, it is quite simple. Under Linux, these numbers are file descriptors. 0 is standard input (eg: keyboard), 1 is standard output (eg: monitor) and 2 is standard error (eg: monitor). All normal information is sent to file descriptor 1, and any errors are sent to 2. If you do not want to have the error messages pop up, then you simply redirect it to /dev/null. Note that this will not stop information from being sent to standard output. For example, if you do not have permissions to read another user’s directory, you will not be able to list its contents

$ ls /root
ls: /root: Permission denied
$ ls /root 2> /dev/null
$

As you can see, the error was not printed out this time. The same applies for other programs and for file descriptor 1. If you do not want to see the normal output of a program, that is, you want it to run silently, you can redirect it to /dev/null. Now if you do not want to see either standard input or error, then you do it this way

$ ls /root > /dev/null 2>&1

This means that the program will send any output or errors that occur to /dev/null so you never ever see them.

Now what if you want your shell script to return a value upon exiting? The exit command takes one argument. A number to return. Normally the number 0 is used to denote a successful exit, no errors occurred. Anything higher or lower than 0 normally means an error has occurred. This is for you, the programmer to decide. Let us look at this program

By specifying return values upon exit, other shell scripts you write making use of this script will be able to capture its return value.

Porting your Bash Scripts

It is important to try and write your scripts so that they are portable. This means that if your script works under Linux, then it should work in another Unix system with as little modification as possible, if any. In order to do this, you should be careful when calling external programs. First consider the question, “Is this program going to be available in this other Unix variant?”. Recall that if you have a program foo that works in the same way as echo, you can use it in echo’s place. However, if it so happens that your script is used on a Unix system without the foo program, then your script will start reporting errors. Also, keep in mind that different versions of bash may have new ways to do different things. For instance, the VAR=$(ps) does the same thing as VAR=`ps`, but realize that older shells, for instance the Bourne shell, only recognizes the latter syntax. Be sure that if you are going to distribute your scripts, you include a README text file which warns the user of any surprises, including, the version of bash the script was tested on, as well as any required programs or libraries needed by the script to run.

Conclusion

That completes the introduction to bash scripting. Your scripting studies are not complete however. There is more to cover. As I said, this is an introduction. However, this is enough to get you started on modifying shell programs and writing your own. If you really want to master shell scripting, I recommend buying Learning the bash shell, 2nd Edition by O’Reilly & Associates, Inc. bash scripting is great for your everyday administrative use. But if you are planning on a much bigger project, you will want to use a much more powerful language like C or Perl.

Good luck…

source : http://www.codecoffee.com

If you have never programmed in Bash before, this is the best place to begin. In case you have a little knowledge of bash, then too you could have a look.. a lot of interesting scripts have been explained by Harold.

Introduction

Like all the shells available in Linux, the Bourne Again SHell is not only an excellent command line shell, but a scripting language in itself. Shell scripting, allows you to fully utilize the shell’s abilities and to automate a lot of tasks that would otherwise require a lot of commands to perform. A lot of programs lying around your Linux box are shell scripts. If you are interested in learning how they work, or in modifying them, it is essential that you understand the bash syntax and semantics. In addition, by understanding the bash language, you will be able to write your own programs to do things exactly the way you want them done.

Programming or Scripting?

People who are new to programming are generally confused as to what the difference is between a programming and scripting language. Programming languages are generally a lot more powerful and a lot faster than scripting languages. Examples of programming languages are C, C++, and Java. Programming languages generally start from source code (a text file containing instructions on how the final program is to be run) and are compiled (built) into an executable. This executable is not easily ported into different operating systems. For instance, if you were to write a C program in Linux, you would not be able to run that C program in a Windows 98 system. In order to do so, you would have to recompile the source code under the Windows 98 system. A scripting language also starts from source code, but is not compiled into an executable. Rather, an interpreter reads the instructions in the source file and executes each instruction. Unfortunately, because the interpreter has to read each instruction, interpreted programs are generally slower than compiled programs. The main advantage is that you can easily port the source file to any operating system and have it interpreted there right on the spot. bash is a scripting language. It is great for small programs, but if you are planning on doing major applications, a programming language might be more beneficial to you. Other examples of scripting languages are Perl, Lisp, and Tcl.

What do you need to know?

Writing your own shell scripts requires you to know the very basic everyday Linux commands. For example, you should know how to copy, move, create new files, etc. The one thing you must know how to do is to use a text editor. There are three major text editors in Linux, vi, emacs and pico. If you are not familiar with vi or emacs, go for pico or some other easy to use text editor.

Warning!!!!

Do not practice scripting as the root user! Anything could happen! I will not be held responsible if you accidentally make a mistake in the coding and screw up your system. You have been warned! Use a normal user account with no root privileges. You may even want to create a new user just for scripting practice. This way the worst thing that can happen is the user’s directory gets blown away.

Your first Bash program

Our first program will be the classical “Hello World” program. Yes, if you have programmed before, you must be sick of this by now. However, this is traditional, and who am I to change tradition? The “Hello World” program simply prints the words “Hello World” to the screen. So fire up your text editor, and type the following inside it:

The first line tells Linux to use the bash interpreter to run this script. In this case, bash is in the /bin directory. If bash is in a different directory on your system, make the appropriate changes to the line. Explicitly specifying the interpreter is very important, so be sure you do it as it tells Linux which interpreter to use to run the instructions in the script. The next thing to do is to save the script. We will call it hello.sh. With that done, you need to make the script executable

$ chmod 700 ./hello.sh

Refer to the manual page for chmod if you do not know how to change permissions for a file. Once this is done, you will be able to run your program just by typing its name:

$ ./hello.sh
Hello World

There it is! Your first program! Boring and useless as it is, this is how everyone starts out. Just remember the process here. Write the code, save the file, and make it executable with chmod.

Commands, Commands and Commands

What exactly did your first program do? It printed the words “Hello World” to the screen. But how did it do that? It used commands. The only line of code you wrote in the program was echo “Hello World”. Well, which one is the command? echo. The echo program takes one argument and prints that argument to the screen.

An argument is anything that follows after you type the program name. In this case, “Hello World” was the argument that you passed to echo When you type the command ls /home/root, the argument to ls is /home/root. So what does this all mean? It means that if you have a program that takes an argument and prints it to the screen, you can use that instead of using echo. Let us assume that we have a program called foo This program will take a single argument, a string of words, and print them to the screen. We can rewrite our program as such

Save it and chmod it and then run it

$ ./hello
Hello World

The exact same result. Was there any unique code at all? No. Did you really write anything? Not unless you are the author of the echo program. All you did, was to embed the echo program into your shell program, with a argument already given. A real world example of an alternative to the echo command is the printf command. printf offers more control, especially if you are familiar with C programming. In fact, the exact same result could have been done without making a shell program

$ echo “Hello World”
Hello World

bash shell scripting offers a wide variety of control and is easy to learn. As you have just seen, you use Linux commands to write your shell programs with. Your shell program is a collection of other programs, specially put together to perform a task.

A More Useful Program

We will write a program that will move all files into a directory, and then delete the directory along with its contents, and then recreate the directory. This can be done with the following commands

$ mkdir trash
$ mv * trash
$ rm -rf trash
$ mkdir trash

Instead of having to type all that interactively on the shell, write a shell program instead

Save it as clean.sh and now all you have to do is to run clean.sh and it moves all files to a directory, deletes them, recreates the directory, and even prints a message telling you that it successfully deleted all files. So remember, if you find that you are doing something that takes a while to type over and over again, consider automating it with a shell program.

Comments

Comments help to make your code more readable. They do not affect the output of your program. They are specially made for you to read. All comments in bash begin with the hash symbol: “#”, except for the first line (#!/bin/bash). The first line is not a comment. Any lines after the first line that begin with a “#” is a comment. Take the following piece of code

Even if you do not know bash scripting, you immediately know what the above program does, because of the comment. It is good practice to make use of comments. You will find that if you need to maintain your programs in the future, having comments will make things easier.

Variables

Variables are basically “boxes” that hold values. You will want to create variables for many reasons. You will need it to hold user input, arguments, or numerical values. Take for instance the following piece of code

What you have done here, is to give x the value of 12. The line echo “The value of variable x is $x” prints the current value of x. When you define a variable, it must not have any whitespace in between the assignment operator: “=”. Here is the syntax

variable_name=this_value

The values of variables can be accessed by prefixing the variable name with a dollar symbol: “$”. As in the above, we access the value of x by using echo $x.

There are two types of variables. Local variables, and environmental variables. Environmental variables are set by the system and can usually be found by using the env command. Environmental variables hold special values. For instance, if you type

$ echo $SHELL
/bin/bash

You get the name of the shell you are currently running. Environmental variables are defined in /etc/profile and ~/.bash_profile. The echo command is good for checking the current value of a variable, environmental, or local.

Note : Setting of Environmental variables are explained in detail in Article  How to set Shell Environment Variables . The article also explains some features of the bash shell.

If you are still having problems understanding why we need variables, here is a good example

Okay, now suppose you decide that you want the value of x to be 8 instead of 12. What do you do? You have to change all the lines of code where it says that x is 12. But wait… there are other lines of code with the number 12. Should you change those too? No, because they are not associated with x. Confusing right? Now, here is the same example, only it is using variables

Here, we see that $x will print the current value of variable x, which is 12. So now, if you wanted to change the value of x to 8, all you have to do, is to change the line x=12 to x=8, and the program will automatically change all the lines with $x to show 8, instead of 12. The other lines will be unaffected. Variables have other important uses as well, as you will see later on.

Control Structures

Control structures allow your program to make decisions and to make them more compact. More importantly as well, it allows us to check for errors. So far, all we have done is write programs that start from the top, and go all the way to the bottom until there are no more commands left in the program to run. For instance

This little shell program, call it bar.sh, copies a file called /etc/foo into the current directory and prints “Done” to the screen. This program will work, under one condition. You must have a file called /etc/foo. Otherwise here is what happens

$ ./bar.sh
cp: /etc/foo: No such file or directory
Done.

So you can see, there is a problem. Not everyone who runs your program will have /etc/foo in their system. It would perhaps be better if your program checked if /etc/foo existed, and then if it did, it would proceed with the copying, otherwise, it would quit. In pseudo code, this is what it would look like

Can this be done in bash? Of course! The collection of bash control structures are, if, while, until, for and case. Each structure is paired, meaning it starts with a starting “tag” and ends with an ending “tag”. For instance, the if structure starts with if, and ends with fi. Control structures are not programs found in your system. They are a built in feature of bash. Meaning that from here on, you will be writing your own code, and not just embedding programs into your shell program.

if … else … elif … fi

One of the most common structures is the if structure. This allows your program to make decisions, like, “do this if this conditions exists, else, do something else”. To use the if structure effectively, we must make use of the test command. test checks for conditions, that is, existing files, permissions, or similarities and differences. Here is a rewrite on bar.sh

Notice how we indent lines after then and else. Indenting is optional, but it makes reading the code much easier in a sense that we know which lines are executed under which condition. Now run the program. If you have /etc/foo, then it will copy the file, otherwise, it will print an error message. test checks to see if the file /etc/foo exists. The -f checks to see if the argument is a regular file. Here is a list of test’s options

else is used when you want your program to do something else if the first condition is not met. There is also the elif which can be used in place of another if within the if. Basically elif stands for “else if”. You use it when the first condition is not met, and you want to test another condition.

If you find that you are uncomfortable with the format of the if and test structure, that is

if test -f /etc/foo
then

then, you can do it like this

if [ -f /etc/foo ]; then

The square brackets form test. If you have experience in C programming, this syntax might be more comfortable. Notice that there has to be white space surrounding both square brackets. The semicolon: “;” tells the shell that this is the end of the command. Anything after the semicolon will be run as though it is on a separate line. This makes it easier to read basically, and is of course, optional. If you prefer, just put then on the next line.

When using variables with test, it is a good idea to have them surrounded with quotation marks. Example:

if [ "$name" -eq 5 ]; then

the -eq operator will be explained later on in this article.

while … do … done

The while structure is a looping structure. Basically what it does is, “while this condition is true, do this until the condition is no longer true”. Let us look at an example

true is actually a program. What this program does is continuously loop over and over without stopping. Using true is considered to be slow because your shell program has to call it up first and then run it. You can use an alternative, the “:” command

This achieves the exact same thing, but is faster because it is a built in feature in bash. The only difference is you sacrifice readability for speed. Use whichever one you feel more comfortable with. Here is perhaps, a much more useful example, using variables

As you can see, we are making use of the test (in its square bracket form) here to check the condition of the variable x. The option -le checks to see if x is less than, or equal to the value 10. In English, the code above says, “While x is less than 10 or equal to 10, print the current value of x, and then add 1 to the current value of x.”. sleep 1 is just to get the program to pause for one second. You can remove it if you want. As you can see, what we were doing here was testing for equality. Check if a variable equals a certain value, and if it does, act accordingly. Here is a list of equality tests

The above looping script we wrote should not be hard to understand, except maybe for this line

x=$(expr $x + 1)

The comment above it tells us that it increments x by 1. But what does $(…) mean? Is it a variable? No. In fact, it is a way of telling the shell that you want to run the command expr $x + 1, and assign its result to x. Any command enclosed in $(…) will be run

Try it and you will understand what I mean. The above code could have been written as follows with equivalent results

You decide which one is easier for you to read. There is another way to run commands or to give variables the result of a command. This will be explained later on. For now, use $(…).

until … do … done

The until structure is very similar to the while structure. The only difference is that the condition is reversed. The while structure loops while the condition is true. The until structure loops until the condition is true. So basically it is “until this condition is true, do this”. Here is an example

This piece of code may look familiar. Try it out and see what it does. Basically, until will continue looping until x is either greater than, or equal to 10. When it reaches the value 10, the loop will stop. Therefore, the last value printed for x will be 9.

for … in … do … done

The for structure is used when you are looping through a range of variables. For instance, you can write up a small program that prints 10 dots each second

In case you do not know, the -n option to echo prevents a new line from automatically being added. Try it once with the -n option, and then once without to see what I mean. The variable dots loops through values 1 to 10, and prints a dot at each value. Try this example to see what I mean by the variable looping through the values

When you run the program, you see that x will first hold the value paper, and then it will go to the next value, pencil, and then to the next value, pen. When it finds no more values, the loop ends.

Here is a much more useful example. The following program adds a .html extension to all files in the current directory

If you do not know, * is a wild card character. It means, “everything in the current directory”, which is in this case, all the files in the current directory. All files in the current directory are then given a .html extension. Recall that variable file will loop through all the values, in this case, the files in the current directory. mv is then used to rename the value of variable file with a .html extension.

case … in … esac

The case structure is very similar to the if structure. Basically it is great for times where there are a lot of conditions to be checked, and you do not want to have to use if over and over again. Take the following piece of code

The case structure will check the value of x against 3 possibilities. In this case, it will first check if x has the value of 0, and then check if the value is 5, and then check if the value is 9. Finally, if all the checks fail, it will produce a message, “Unrecognized value.”. Remember that “*” means “everything”, and in this case, “any other value other than what was specified”. If x holds any other value other than 0, 5, or 9, then this value falls into the *’s category. When using case, each condition must be ended with two semicolons. Why bother using case when you can use if? Here is the equivalent program, written with if. See which one is faster to write, and easier to read

Quotation Marks

Quotation marks play a big part in shell scripting. There are three types of quotation marks. They are the double quote: “, the forward quote: ‘, and the back quote: `. Does each of them mean something? Yes.

Note :Wildcards, Quotes, Back Quotes, Apostrophes in shell commands ( * ? [] ” ` ‘) deals with special characters in great detail. Please refer to that article in case you are not familiar with the use of these special characters in shell commands. Below is a quick explanation of some of those features

The double quote is used mainly to hold a string of words and preserve whitespace. For instance, “This string contains whitespace.”. A string enclosed in double quotes is treated as one argument. For instance, take the following examples

$ mkdir hello world
$ ls -F
hello/     world/

Here we created two directories. mkdir took the strings hello and world as two arguments, and thus created two directories. Now, what happens when you do this:

$ mkdir “hello world”
$ ls -F
hello/     hello world/     world/

It created a directory with two words. The quotation marks made two words, into one argument. Without the quotation marks, mkdir would think that hello was the first argument, and world, the second.

Forward quotes are used primarily to deal with variables. If a variable is enclosed in double quotes, its value will be evaluated. If it is enclosed in forward quotes, its value will not be evaluated. To make this clearer, try the following example

See the difference? You can use double quotes if you do not plan on using variables for the string you wish to enclose. In case you are wondering, yes, forward quotes can be used to preserve whitespace just like double quotes

$ mkdir ‘hello world’
$ ls -F
hello world/

Back quotes are completely different from double and forward quotes. They are not used to preserve whitespace. If you recall, earlier on, we used this line

x=$(expr $x + 1)

As you already know, the result of the command expr $x + 1 is assigned to variable x. The exact same result can be achieved with back quotes:

x=`expr $x + 1`

Which one should you use? Whichever one you prefer. You will find the back quote used more often than the $(…). However, I find $(…) easier to read, especially if you have something like this

This was just the beginning. You will learn lots of more concepts in the concluding part of this article. Till then happy shell scripting..

source : http://www.codecoffee.com

Each file in Linux inherits a set of properties.  One vital set of properties is the file’s permissions.  Permissions determine what any particular user (or group of users) is able to do that file.  File permissions help prevent unwanted deletion and safeguard your data.  In order to use Linux’s file permissions, you need to understand Linux’s categories of users and groups.

Categories of Users

You are asked to enter a login name and password when you first log into Linux.  When we talk of a user, we refer to the account issuing commands to the operating system at the time and not to the actual person operating the computer.  As soon as Linux authenticates your login name and password you “become” that user and operate using that user account.

Users belong to one or more groups. (The SuperUser allocates Users to particular groups.)  Each user has a default group.

Linux organises users into three broad categories (the values in brackets are Linux’s accepted abbreviations):

1. user (u) The owner of the file.  A user who creates a file automatically owns it.  Only the owner and the SuperUser (alias root) can change the permissions of a file.
2. group (g) The group of a file.  One group of users is given special access to a file.  This is determined by the file owner.
3. others (o) All other users on the system.  In other words, every account except the file’s owner, or users in the file’s group.

File access attributes

Each file has a set of attributes specifying what the user in each category (user, group, others) can do with the file.  Here are the three types of access available in Linux:

1. read (r) This category of users can display, but not necessarily alter, the file.
2. write (w) This category of users can alter the file (but not necessarily read it.)
3. execute (x) This category of user can execute (i.e. run) the file.

Displaying file permissions

You can display a file’s permissions by executing the ls -l command.  Here is a sample output:

lloy0076@localhost bin2dec]$ ls -l
total 23
-rw-r-----  1 lloy0076 root    286 Aug 28 02:17 b2d.lex
-rwxr-xr-x  1 lloy0076 root  20390 Aug 28 02:17 b2d
-rw-r--r--  1 lloy0076 root     49 Aug 27 22:08 Makefile

You can see the file permissions in the left-most column.  The first character is usually a `-’ or `d’.  This actually refers to the type of the file, and does not refer to the file permissions; a `-’ indicates the file is a “normal” file, and a `d’ indicates the it is a directory.  Other letters indicate files with special meanings to Linux.  The next nine characters refer to file permissions.

The first three (of the nine permission characters) shows what access to the file is permitted for the owner; the next three shows the permissions for anyone in the file’s group; and the last three are for those classified as other.  A letter (r, w or x) indicates that the permission for that particular user, group or other is set, and a `-’ indicates that the permission is not available.

Schematically you could represent it like this:

[-rwxrwxrwx]

The b2d.lex file is owned by the user lloy0076, who can read and write it; anyone in the root group can read the file; and nobody else is permitted any access at all.  The b2d file is also owned by lloy0076, who has read, write and execute permission on it.  Anyone in the “root” group has read and execute permissions for the file; and so does everyone else.

Changing file permissions

chmod

To change permissions use the command chmod from the command-line.  You must be the owner of the file (or you must be the SuperUser.)  Take care when changing a file’s permissions and be especially careful when you are working with any system files.

The basic format for chmod is:

• chmod [OPTION]… MODE… FILE…

FILE is a file or directory, which will have its permissions set.  MODE is the permissions being set on the [FILE].

You can use a number of OPTIONs with chmod. Two useful ones are:

• -v chmod produces verbose output; useful to see exactly what chmod is doing
• -R chmod will descend (recursively) into all subdirectories, changing all file permissions contained within.  This means that it will iterate through all the files in all the specified FILE’s subdirectories (if it has any) changing them at it goes.  This option should be used with care.

You use the `+’, ‘-’ or ‘=’ action symbols to add, subtract or set file permissions.  Here is how you do it:

1. Specify the category of users with the abbreviations for the categories (u, g or o).  A special category ,’a', also exists which means all users.  You can add these together like ug, which means the user and the group
2. Specify an appropriate action symbol (‘+’, ‘-’ or ‘=’)
3. Specify a file access attribute (r, w, or x).  As with the specification for users, you can add these together like rw, which means read and write permissions

Here are two examples of how to use chmod on a file called `test’; for our purposes we will assume that `test’ has absolutely no access permited at the start:

1. chmod ug+rx test This gives read and execute permissions to the user and group, the permissions are now -r-xr-x—
2. chmod a-x test This removes execute permissions from all users, after these two steps, the permissions are -r–r—–
3. chmod u=x test This sets execute permission, and removes all others, for the owner.  After these three steps the permissions are —xr—–

An easy way to determine what the mode string, ug+rx for example, means is by actually saying it fully out loud.  This example would be user; group; add; read permissions; execute permission.  Although it is terrible English, it should be plain what this particular mode is trying to achieve.

Conclusion

info chmod and man chmod are both good reference points for chmod.  Whilst chmod also understands another way of specifying modes – the octal method – I find it easier to explain this method to new Linux Users.  The octal method is adequately explained in the man pages for chmod.

Please submit any suggestions to: lloy0076@senet.com.au

source : http://www.linuxsa.org.au

How do I know what size to make my disk partitions?

This is one of the more often asked questions I hear. Usually the answer is “It depends”, so here is my experience with partitioning Linux boxes for various applications over the last few years.

First of all it helps to know exactly what the file systems are all used for and where stuff goes. A good reference for this sort of thing is in the Linux Documentation Project’s “System Administrator’s Guide” or SAG. You can find a good bit of info on the file system here

Alternatively, if you have a copy of “A Practical Guide to Linux”, then check out page 74.

Here is a brief rundown…

/       Root file system. Should just contain /bin, /sbin, /dev,
        /root,
        /lib, and /etc.
/usr    Programmes and source code.
/var    Variable data, such as spools, man pages, news and mail
        queues, database data.
/boot   Boot kernels.
/home   User data and "stuff".
/tmp    Temporary file locations

The / file system will never need to be more than 100Meg. Make it that.

The /usr file system will vary depending on how big your initial installation is and how much extra software you download. For a RedHat 6.2 minimal install you’ll be needing about 250 to 300 Meg (typical server), and for a full install you need around 1.5 Gig (typical workstation). Other distributions will need more or less, but this is a good guide. Any extra software you download may also go in this file system, so if you are planning installing an office suite or a cad package, be aware it that it may go in here.

If you are installing software to build from a tar ball or installing software that isn’t part of a vendor’s distribution, like an RPM or a DEB is, you will probably want to install it in the /usr/local file system. This file system is usually left untouched by the installation or upgrade process of a linux distribution and is ideal for installing third party software. If you plan on doing a lot of this, a separate partition is a great idea, because if you want to do a re-install rather than an upgrade, you can simply tell your distribution not to format the /usr/local file system when installing and you will leave your third party software in tact. The format of the /usr/local file system is almost identical to the / file system. Handy huh?

/usr/local/bin and /usr/local/sbin are also the correct place to put any scripts you may write after you have your system up and running. This is preferable to placing them in /usr/bin and /usr/sbin or even /bin and /sbin, as these should really be static and left the way the distribution intended them. It also makes backing up a system much easier if all your locally created scripts are in one convenient place.

The /var file system is the most varying file system, hence its name. The function of the machine will determine how much you need. For a vanilla system, I recommend 400 Meg. This is usually sufficient for a workstation. If you are building a proxy server, you will need a separate partition, but preferably a separate disk, for /var/spool/squid. The same goes for a mail server, except the file systems of interest are /var/spool/mqueue and /var/spool/mail. The size of /var/spool/mail will depend on how much storage you want for user’s mailboxes, and the size /var/spool/mqueue will depend on how much mail ‘in transit’ you wish to spool. Mail server’s acting as a secondary MX might need a lot here.

There are other smaller directories in /var/spool that are of interest, so I would recommend a /var/spool of 300 to 500 Meg for any server application in conjunction with the /var of 400 Meg. For a workstation you may be able to use the 400Meg /var partition to house your /var/spool as well, but it may pay to enlarge it a bit.

/var/log, as the name suggests, is the final resting place for logs. Once again the size of this will depend on the function of the system, but as a general rule it is highly recommended that you have a separate /var/log to your /var partition, regardless of the machine’s function. This way any stray system logs that fill up will have no effect on your system other than stopping logging. This goes for both servers and workstations alike. If you are running a heavily loaded proxy, mail or web server, you will need heaps and heaps of disk space here. Fully loaded proxy servers in peering arrangements can easily generate hundreds of thousands of bytes of log files an hour. The same goes for mail servers. The mail can come in and go out very quickly on a fast link, but the log files stay around. You also don’t want a slash-dotted web site to fill up your logging directory, so careful thought here will pay off in the future.

The /var file system is also often used for the storage of database data. /var/db or /var/lib is the file system that is used, and you will need to keep this big enough to hold your data. Often a separate fast SCSI disk or RAID will make your database much faster. IO is often the biggest bottleneck in database systems, and an IDE drive in /var/db or /var/lib wont help.

The /boot directory is probably the most useful file system, and often the most forgotten. Having your kernels on a separate partition will make rescuing a system that has crashed a whole lot easier. This means that booting the system and recovering the partitions can be attacked as two separate tasks. Having a small /boot in a primary partition is also the best cure for the famous “I just installed linux and now all I get is ‘LI’” LILO installation problems. LILO still has issues with hard drive space above 1024 cylinders. A small 20 Meg /boot partition as the first primary partition on the system will alleviate this. Some distributions, such as RedHat, are smart enough to assign automatically the first primary partition to /boot for just this reason.

/home is where you hang your hat. It is also where you “keep your stuff”. Files you download, projects, mail, documents, mp3′s, everything. This is the equivalent of Windows’ “My Documents”, “C:/download”, the desktop, etc. Even if the system is only used by you at your desk, and no-one else, you should still have your own home directory in the /home file system. Don’t be tempted to add partitions to the root file system such as /scripts, /downloads, etc. You are breaking stuff when you do that. Linux is still a true multiuser operating system, even if you are the only person using it. Try to keep this in mind when building a partition table. This all starts to make sense when you stop logging in as root, and start logging in as a regular user. It never ceases to amaze me how many people run X as root. *sigh*.

Many distributions nowadays are geared towards easy and quick upgrades and everything has it’s place. If you keep you stuff in /home/yourname and no-where else, you can be sure that when your next upgrade of linux comes, you can just chuck in the CD and hit “upgrade” and your Metallica mp3′s will still be there when your system comes back on-line.

/home is also where the storage file system for a file server should go. The same is true for web server pages, and ftp server data. Obviously if you are building a web server, have a separate /home/httpd file system on a nice fast SCSI disk. Same with /home/ftp.

Sometimes it’s a great idea to have a separate /tmp directory, because temporary files can get out of control. Having /tmp on the same partition as the root file system can cause problems if you scan a 60 Meg picture into a graphics manipulation programme and it decides to store it in /tmp.

The only other partition of major interest is the swap partition. It is often a good idea to place this in the physical middle of the drive. Then the heads have less far to travel to swap out data when the system gets loaded. Alternatively you can just throw more memory at the problem.

Now I’ll give you a few ‘real life’ examples of servers that I maintain. The names have been changed to protect the innocent.

Here is my bog standard workstation. It runs X. It may get used for some server functions in the future, so there is lot’s of space ready. I even have a big block of space hanging of /mnt/tmp, and one day I’m sure I’ll think of a use for it.

[alex@workstation alex]$ df
Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/hda13               85530     34264     46850  42% /
/dev/hda1               101089      6802     89068   7% /boot
/dev/hda6              1517920    154616   1286196  11% /home
/dev/hda12             2150420        20   2041160   0% /mnt/tmp
/dev/hda10              248895        27    236018   0% /tmp
/dev/hda5              2016016   1292380    621224  68% /usr
/dev/hda7               758936     37592    682792   5% /var
/dev/hda9               497829       657    471470   0% /var/log
/dev/hda8               758936       292    720092   0% /var/spool

This next beast is a mail server. Note the use of separate drives for critical server file systems.

[alex@mail alex]$ df
Filesystem        1k-blocks      Used Available Use% Mounted on
/dev/sda12            79941     39339     36474  52% /
/dev/sda1             21011      5463     14463  27% /boot
/dev/sda11           701636     43332    622664   7% /home
/dev/sda9            202031        13    191587   0% /tmp
/dev/sda5           1210800    456856    692436  40% /usr
/dev/sda7            496695      7069    463981   2% /var
/dev/sda6           1009724    197880    760552  21% /var/log
/dev/sda8            496695       982    470068   0% /var/spool
/dev/sdb1           4382932    766640   3393648  18% /var/spool/mail

Here is a proxy server. Mix of SCSI and IDE.

[root@proxy /root]# df
Filesystem        1k-blocks      Used Available Use% Mounted on
/dev/hda11           101485     28799     67446  30% /
/dev/hda1             23393      2647     19538  12% /boot
/dev/hda7            199085      2101    186704   1% /home
/dev/hda8             81954       985     76737   1% /tmp
/dev/hda5            809556    170444    597988  22% /usr
/dev/hda6            199085      4243    184562   2% /var
/dev/hda10          1611224     10408   1518968   1% /var/log
/dev/sda1          17654736    354220  16403692   2% /var/spool/squid

When partitioning a machine for use, it is often a bad idea to install everything into a single / partition. Even if you don’t need separate partitions, the practice you get from partitioning disks and learning how much space each partition needs in a given situation will be invaluable when someone asks you to build a server for them. Spend a few minutes before installation considering the functions of the machine you are building and this will yield a useful and efficient partition table. The more often you do it the more of a feel you will get for how much space your distribution needs for different tasks. Now that you have the above information there is no excuse for poor partitioning, and you can help make the world a safer place for data!

Next time you see…

[lame@nothought /]$ df
Fileystem      1k-blocks      Used Available Use% Mounted on
/dev/hda1       17654736   1354220  15403692   8% /

…you can do something about it!

source : http://www.linuxsa.org.au/

This document explains how to set your computer’s clock from Linux, how to set your timezone, and other stuff related to Linux and how it does its time-keeping.

Your computer has two timepieces; a battery-backed one that is always running (the “hardware”, “BIOS”, or “CMOS” clock), and another that is maintained by the operating system currently running on your computer (the “system” clock). The hardware clock is generally only used to set the system clock when your operating system boots, and then from that point until you reboot or turn off your system, the system clock is the one used to keep track of time.

On Linux systems, you have a choice of keeping the hardware clock in UTC/GMT time or local time. The preferred option is to keep it in UTC because then daylight savings can be automatically accounted for. The only disadvantage with keeping the hardware clock in UTC is that if you dual boot with an operating system (such as DOS) that expects the hardware clock to be set to local time, the time will always be wrong in that operating system.

Setting your timezone

The timezone under Linux is set by a symbolic link from /etc/localtime^[1] to a file in the /usr/share/zoneinfo^[2] directory that corresponds with what timezone you are in. For example, since I’m in South Australia, /etc/localtime is a symlink to /usr/share/zoneinfo/Australia/South. To set this link, type:

ln -sf ../usr/share/zoneinfo/your/zone /etc/localtime

Replace your/zone with something like Australia/NSW or Australia/Perth. Have a look in the directories under /usr/share/zoneinfo to see what timezones are available.

[1] This assumes that /usr/share/zoneinfo is linked to /etc/localtime as it is under Red Hat Linux.

[2] On older systems, you’ll find that /usr/lib/zoneinfo is used instead of /usr/share/zoneinfo. See also the later section “The time in some applications is wrong”.

Setting UTC or local time

When Linux boots, one of the initialisation scripts will run the /sbin/hwclock program to copy the current hardware clock time to the system clock. hwclock will assume the hardware clock is set to local time unless it is run with the --utc switch. Rather than editing the startup script, under Red Hat Linux you should edit the /etc/sysconfig/clock file and change the “UTC” line to either “UTC=true” or “UTC=false” as appropriate.

Setting the system clock

To set the system clock under Linux, use the date command. As an example, to set the current time and date to July 31, 11:16pm, type “date 07312316” (note that the time is given in 24 hour notation). If you wanted to change the year as well, you could type “date 073123161998”. To set the seconds as well, type “date 07312316.30” or “date 073123161998.30”. To see what Linux thinks the current local time is, run date with no arguments.

Setting the hardware clock

To set the hardware clock, my favourite way is to set the system clock first, and then set the hardware clock to the current system clock by typing “/sbin/hwclock --systohc” (or “/sbin/hwclock --systohc --utc” if you are keeping the hardware clock in UTC). To see what the hardware clock is currently set to, run hwclock with no arguments. If the hardware clock is in UTC and you want to see the local equivalent, type “/sbin/hwclock --utc”

The time in some applications is wrong

If some applications (such as date) display the correct time, but others don’t, and you are running Red Hat Linux 5.0 or 5.1, you most likely have run into a bug caused by a move of the timezone information from /usr/lib/zoneinfo to /usr/share/zoneinfo. The fix is to create a symbolic link from /usr/lib/zoneinfo to /usr/share/zoneinfo: “ln -s ../share/zoneinfo /usr/lib/zoneinfo”.

Summary

• /etc/sysconfig/clock sets whether the hardware clock is stored as UTC or local time.
• Symlink /etc/localtime to /usr/share/zoneinfo/... to set your timezone.
• Run “date MMDDhhmm” to set the current system date/time.
• Type “/sbin/hwclock --systohc [--utc]” to set the hardware clock.

Other interesting notes

The Linux kernel always stores and calculates time as the number of seconds since midnight of the 1st of January 1970 UTC regardless of whether your hardware clock is stored as UTC or not. Conversions to your local time are done at run-time. One neat thing about this is that if someone is using your computer from a different timezone, they can set the TZ environment variable and all dates and times will appear correct for their timezone.

If the number of seconds since the 1st of January 1970 UTC is stored as an signed 32-bit integer (as it is on your Linux/Intel system), your clock will stop working sometime on the year 2038. Linux has no inherent Y2K problem, but it does have a year 2038 problem. Hopefully we’ll all be running Linux on 64-bit systems by then. 64-bit integers will keep our clocks running quite well until aproximately the year 292271-million.

Other programs worth looking at

• rdate – get the current time from a remote machine; can be used to set the system time.
• xntpd – like rdate, but it’s extremely accurate and you need a permanent ‘net connection. xntpd runs continuously and accounts for things like network delay and clock drift, but there’s also a program (ntpdate) included that just sets the current time like rdate does.

Further information

• date(1)
• hwclock(8)
• /usr/doc/HOWTO/mini/Clock

Source  : http://www.linuxsa.org.au