Troubleshooting

How to find - Size of a directory & Free disk space

‘du’ - Finding the size of a directory

$ du
Typing the above at the prompt gives you a list of directories that exist in the current directory along with their sizes. The last line of the output gives you the total size of the current directory including its subdirectories. The size given includes the sizes of the files and the directories that exist in the current directory as well as all of its subdirectories. Note that by default the sizes given are in kilobytes.

$ du /home/shailesh
The above command would give you the directory size of the directory /home/nikesh

$ du -h
This command gives you a better output than the default one. The option ‘-h’ stands for human readable format. So the sizes of the files / directories are this time suffixed with a ‘k’ if its kilobytes and ‘M’ if its Megabytes and ‘G’ if its Gigabytes.

$ du -ah
This command would display in its output, not only the directories but also all the files that are present in the current directory. Note that ‘du’ always counts all files and directories while giving the final size in the last line. But the ‘-a’ displays the filenames along with the directory names in the output. ‘-h’ is once again human readable format.

$ du -c
This gives you a grand total as the last line of the output. So if your directory occupies 30MB the last 2 lines of the output would be

30M .
30M total

The first line would be the default last line of the ‘du’ output indicating the total size of the directory and another line displaying the same size, followed by the string ‘total‘. This is helpful in case you this command along with the grep command to only display the final total size of a directory as shown below.

$ du -ch | grep total
This would have only one line in its output that displays the total size of the current directory including all the subdirectories.

$ du -s
This displays a summary of the directory size. It is the simplest way to know the total size of the current directory.

$ du -S
This would display the size of the current directory excluding the size of the subdirectories that exist within that directory. So it basically shows you the total size of all the files that exist in the current directory.

$ du –exculde=mp3
The above command would display the size of the current directory along with all its subdirectories, but it would exclude all the files having the given pattern present in their filenames. Thus in the above case if there happens to be any mp3 files within the current directory or any of its subdirectories, their size would not be included while calculating the total directory size.

-
‘df’ - finding the disk free space / disk usage

$ df
Typing the above, outputs a table consisting of 6 columns. All the columns are very easy to understand. Remember that the ‘Size’, ‘Used’ and ‘Avail’ columns use kilobytes as the unit. The ‘Use%’ column shows the usage as a percentage which is also very useful.

$ df -h
Displays the same output as the previous command but the ‘-h’ indicates human readable format. Hence instead of kilobytes as the unit the output would have ‘M’ for Megabytes and ‘G’ for Gigabytes.

Execute more than one commands 

Executing the second command only if the first is successful

To do this you would type: command1 && command2

command2 will be executed if command1 successfully completes (if command1 fails command2 won't be run). This is called a logical AND.

Executing the second command only if the first fails

To do this you would type: command1 || command2

command2 will be executed if command1 does not successfully completes (if command1 is successful command2 won't be run). This is called a logical OR.

Executing commands sequentially

To execute command sequentially regardless of the success/failure of the previous you simply type: command1; command2

command2 will execute once command1 has completed.

Head and Tail example

1. Suppose I need to read last 10 lines of a file
# tail -n 20 /var/log/boot.log

2. Suppose I need to read forst 10 lines of a file
# head -n 30 /etc/sensors.conf

Now imagine you want to follow what is been written “live” to the /var/log/messages file

# tail -f /var/log/messages

How to clear the cache from memory

Linux Command
  echo 3 > /proc/sys/vm/drop_caches

Sending email attachment​s from linux command line

Many times we required to send mail attachments from command line, when administering a server remotely. The procedure is simple using mutt that is available in all Unix flavors and Windows Microsoft.

$ mutt -s "Logs" -a http_access.log shailesh@domain.com

Or Without subject:

$ echo | mutt -a [file] [mail@address.com]

How To increase the limit for file handles

There are cases when you get a lot of error messages about "running out of  file handles", increasing this limit of this handler can solve this issue. To change the value, just write the new number into the file as below:

# cat /proc/sys/fs/file-max
8192

# echo 943718 > /proc/sys/fs/file-max

# cat /proc/sys/fs/file-max
934718

This value also can be changed using "sysctl" command. To make the change permanent, add the entries to /etc/sysctl.conf

# vi /etc/sysctl.conf
fs.file-max = 934718

 Execute command at regular intervals

If you anytime need to execute a command once and again and again, you can use watch and tell it to execute the Linux command in a give interval.

The syntax of the command is:

watch [option(s)] command

The default interval of execution of the given command is two (2) seconds, but you define a different interval using the option -n.

Here are two useful uses of watch.

watch -n 5 free -m

Which will show you the use of memory each five seconds.

watch -n 30 vnstat -h

Which will show you the bandwidth used hourly, read more about vnstat.

You can also make watch to highlight any change it detect between two screens of output.

watch -n 30 -d vnstat -h

And if you want to maintain the highlight on, add =cummulative to the -d option, like this:

watch -n 30 -d=cummulative vnstat -h

Creating backup/res​tore images using dd

Create a hard disk image: dd if=/dev/hda1 of=/home/hda1.bin

Create a compressed disk image: dd if=/dev/hda1 | gzip > /home/hda1.bin.gz

Back up the MBR: dd if=/dev/hda of=/home/hda.boot.mbr bs=512 count=1

Restore MBR (from a Live CD): dd if=/mnt/hda1/home/hda.boot.mbr of=/dev/hda bs=512 count=1

Backup a drive to another drive: dd if=/dev/hda of=/dev/hdb conv=noerror,sync bs=4k

The command:

dd -if /dev/hda1 > partitionimage.dd

will backup "/dev/hda1" partition. A whole drive (including the MBR) could be backed up using just /dev/hda as the input "file". Restoring is done by: dd -if partitionimage.dd -of /dev/hda1

If you have a complete new harddrive and want to restore the backup (or copy your old system to the new drive). First, the new drive has to be bigger or exactly the same size as the old one. First go superuser and switch to runlevel 1 so that you can fumble around with the harddisk without other services interfering

restore either the whole disk to the new drive or one partition (depending on how you made the backup): dd -if partitionimage.dd -of /dev/hda1

If you restored the whole drive (/dev/hda), the system will not automatically create the devices (/dev/hda1, /dev/hda2) if you just restored the whole drive. If you know how to make the devices show up without reboot, write it here, otherwise this is a good moment to reboot.

If you restored the system to a new drive, and your device names changed (for example from /dev/hda to /dev/sda) then you must adapt the bootloader and the mount points. While still on runlevel 1, edit these files:

/boot/grub/menu.list
/etc/fstab

    HowTo Find out if Installed Linux is 32 bits or 64 bits

To figure out  whether we have installed Linux is of 32 bits or 64 bits, you can run the following command in a Terminal:

uname -m
The possible outcomes and their meanings are:

    * i686       : Installed 32-bit version.
    * x86_64 : Installed 64-bit version. 

How To Set $PATH Variable in Linux

When you type a command at the shell prompt, it has to be read off disk out of one or other directory. On Linux all such executable commands are located in one of the directories. A file is located in the directory tree according to its type, and not according to what software package it belongs to. Hence, for example, a word processor may have its actual executable stored in a directory with all other executables, while its font files are stored in a director with other fonts from all other packages.

The shell has a procedure for searching for executable when you type them in. If you type in a command with slashes, like /bin/cp then it tries to run the named program: cp out of the /bin directory. If you just type cp on its own, then it tries to find the cp command in each of the sub-directories of your PATH. To see what your PATH is, just type

echo $PATH

You will see a colon separated list of directories. Note that the current directory . is not listed. It is very important that the current directory not be listed for security reasons. To execute a command in the current directory, we hence always type ./.

To append for example a new directory /opt/gnome/bin to your PATH, do

PATH="$PATH:/opt/bin"
export PATH

you can also do this in one line: export PATH="$PATH:/opt/bin"

Linux performance tuning – vm.swappiness

Linux kernel has improved memory subsystem, with which administrators now have a simple interface to fine-tune the swapping behavior of the kernel.  The linux kernel tunable parameter vm.swappiness (/proc/sys/vm/swappiness) can be used to define how aggressively memory pages are swapped to disk.

Linux moves memory pages that have not been accessed for some time to the swap space even if there is enough free memory available. By changing the percentage in /proc/sys/vm/swappiness you can control the swapping behavior, depending on the system configuration.

 

A high swappiness value means that the kernel will be more apt to unmap mapped pages. A low swappiness value means the opposite, the kernel will be less apt to unmap mapped pages. In other words, the higher the vm.swappiness value, the more the system will swap.

vm.swappiness takes a value between 0 and 100 to change the balance between swapping applications and freeing cache. At 100, the kernel will always prefer to find inactive pages and swap them out; in other cases, whether a swapout occurs depends on how much application memory is in use and how poorly the cache is doing at finding and releasing inactive items.

Systems with memory constraints that run batch jobs (processes that sleeps for long time) might benefit from an aggressive swapping behavior.

To change swapping behavior, use either echo or sysctl

# sysctl -w vm.swappiness=90

Tuning the Linux memory subsystem is a tough task that requires constant monitoring to ensure that changes do not negatively affect other components in the server. If you do choose to modify the virtual memory parameters (in /proc/sys/vm), change only one parameter at a time and monitor how the server performs.

Increasing swap performance

 

If your Linux machine is configured with several swap partitions, the default trend is to use of one partition at a time. Once one swap partition has been full the next will be used. This is not always the best method for performance, since when a new process needs to be swapped to disk it may be forced to wait until another process is swapped out.

For instance, if there are two swap partitions specified in /etc/fstab it will look something like this:

/dev/sda2    swap     swap    defaults           0 0
/dev/sda3    swap     swap    defaults           0 0

Change the mount options section from ''defaults'' to ''pri=0'' :

/dev/sda2    swap      swap    pri=0           0 0
/dev/sda3    swap      swap    pri=0           0 0

If you want to do this in a live system, then swapoff and swapon with "-p 0" option for each swap device - one by one. Once this has been done the system will be able to access any of the designated swap partitions independently of the others. This can increase the swap performance of a machine which is regularly swapping memory to disk. However it is important to bear in mind that in most situations a machine should not make heavy use of swap partitions.

How to prevent Linux from remembering your sudo password

You may know that if you type

sudo [command]

Your password will be asked, but if you type it again in a few seconds, it will not be asked, because Linux “remember” your password for some time, well if you are really concerned about this, you may force Linux to “forget” your password inmediately.

How to do it?

sudo visudo

And add this line:

Defaults  timestamp_timeout = 0

You may change 0 to any number representing the minutes you may want your password to be “remembered”, or let in 0 so you will need to type your password each time you type sudo

How to force users to change their passwords on 1st login

1.) Firstly, lock the account to prevent the user from using the login until the change has been made:

# usermod -L <username>

2.) Change the password expiration date to 0 to ensure the user changes the password during the next login:

# chage -d 0 <username>

3.) To unlock the account after the change do the following:

# usermod -U <username>

Some Useful command

kill nobody process
ps aux | grep nobody | awk ‘{print $2}’ | xargs kill -9
To see only the memory resources occupied by each category of processes, such as Apache httpd, MySQL mysqld or Java, use the following command:
ps aux | awk ‘{print $4″\t”$11}’ | sort | uniq -c | awk ‘{print $2″ “$1″ “$3}’ | sort -nr
====================================================
ps -eo pid,%cpu,vsz,args,wchan
Shows every process, their pid, % of cpu, memory size, name, and what syscall they are currently executing. Nifty.
whereis locates source/binary and manuals sections for specified files. The supplied names are first stripped of leading pathname components and any (single) trailing
extension of the form .ext, for example, .c. Prefixes of s. resulting from use of source code control are also dealt with. whereis then attempts to locate the desired program in
a list of standard Linux places. For example find out where is ls command?
$ whereis ls

For netstat
netstat -nap | grep :110 | awk ‘{print $5}’ | cut -d”:” -f1 | sort | uniq -c | sort -nr | head
netstat -nap | grep :25 | awk ‘{print $5}’ | cut -d”:” -f1 | sort | uniq -c | sort -nr | head
netstat -n | grep :80 | wc -l;uptime ; netstat -n | wc –l
netstat -ntu | awk ‘{print $5}’ | cut -d: -f1 | sort | uniq -c | sort -n
netstat -nap | grep :80 | awk ‘{print $5}’ | cut -d”:” -f1 | sort | uniq -c | sort -nr | head

# How to find Top 10 memomry & cpu consuming processes
ps -eo pcpu,%mem,pid,user,args | sort -k 1 -r | head -10

pidof php
shows the PIDs for php
history | netstat
shows no of connection details
lsof  -p pid
shows the details of pid
netstat -na |grep :80 |sort
Use below mentioned command to get top memory consuming processes”
ps aux | head -1;ps aux –no-headers| sort -rn +3 | head
Use below command to get top cpu consuming processes:
ps aux | head -1;ps aux –no-headers | sort -rn +2 |more
You can check if any backup is going on, run the following commands:
# ps aux | grep “pkg”
# ps aux | grep “gzip”
# ps aux | grep “backup”
If any backup process is going on, kill that process.