INTELLIGENT WORK FORUMS
FOR COMPUTER PROFESSIONALS

Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!

*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Jobs

Useful scripts for investigating space problems by mrn
Posted: 21 Mar 02

Space problems are a common occurrence because applications are forever incresing in size. Often space problems can be caused by rouge processes or just careless users. Often the cause is obvious, but occasionally it can be a bit of a mystery.

After investigating numerous space problems over the years I have written the following scripts to ease the problem.

The First script is dfsort. All this script does is sort your filesystems into order of percentage space used. This immediately shows you where the problem is.

dfsort Script

#!/usr/bin/ksh
#
# Script: dfsort
# Aim:    Show filesystems sorted by percentage full
#

awk="/usr/bin/awk"
basename="/usr/bin/basename"
df="/usr/bin/df"
egrep="/usr/bin/egrep"
grep="/usr/bin/grep"
sort="/usr/bin/sort"
tail="/usr/bin/tail"
uname="/usr/bin/uname"

#
# Functions
#

initialisation()
{
  for parm in $PARMS
        do
         case "$parm" in
           -k) DFOPT="-k" ;;
            *) : ;;
         esac
        done
  return
}

showAIXspace()
{
$df $DFOPT |
  $tail +2 |
  $grep -v ":" |
  $grep "^/dev" |
  $awk '{
         space = substr($4,1,length($4)-1)
         inode = substr($6,1,length($6)-1)
         if ( space > inode )
         {
           print space, $0
         }
         else
         {
           print inode, $0
         }
        }' |
  $sort -t" " +0 -1 -n |
  $awk -v bs=$DFOPT 'BEGIN {
         if ( bs == "" )
         {
           print "Filesystem     512-blocks      Free   %Used   Iused   %Iused  
    Mounted on"
         }
        else
        {
           print "Filesystems   1024-blocks      Free   %Used   Iused   %Iused  
Mounted on"
        }
        }
        {
        printf "%-15s %9s %9s %7s %7s %7s %-s\n", $2, $3, $4, $5, $6, $7, $8    
        }'
     return
  }

showSpace()
  {
        OS=`$uname `
        case "$OS" in
         AIX)   showAIXspace ;;
         SunOS) showSUNspace ;;
         *) : ;;
        esac
        return
  }

showSUNspace()
  {
        $df -k |
         $tail +2 |
         $grep -v ":" |
         $egrep -e "^/dev/|^/dev/|^swap" |
         $awk '{
                space = substr($5,1,length($5)-1)
                print space, $0
        }' |
        sort -t " " +0 -1 -n |
        $awk 'BEGIN {
                print "Filesystem     1024-blocks       Used    Avail   %Used   
Mounted on"
        }
        {
         printf "%-25s %9s %9s %9s %7s  %-s\n", $2, $3, $4, $5, $6, $7 }'
        return
}

#
# Start of main processing
#
SCRIPT=`$basename $0 `
PARMS="$*"

initialisation
showSpace

exit 0

Once the problem Filesystem has been identified, the fisrt method of attack is to determine the largest files within that filesystem. Use the script below finds. This script lists all the files within the filesystem in size order. I usually pipe the output through tail just to list the largest files, eg:

finds /usr | tail -10

Here is the finds script:

#!/usr/bin/ksh
#
# Script: finds
# Aim:    List files in a filesystem by size
#
awk="/usr/bin/awk"
basename="/usr/bin/basename"
cut="/usr/bin/cut"
find="/usr/bin/find"
sort="/usr/bin/sort"

#
# Functions
#
checkParms()
{
        FILELIST=""
        for parm in $PARMS
        do
         if test "$parm" = "-r"
         then
          REVERSE="r"
         else
          check=`echo "$parm" | $cut -c1 `
          if test "$check" != "-"
          then
           FILELIST="$FILELIST $parm"
          fi
         fi
        done
        return
}
listFiles()
{
        $find $FILELIST -xdev -type f -ls |
         $awk '{ print $7 ": " $0 }' |
         $sort -t: +0 -1 -n$REVERSE |
         $awk '{ printf "%7s %4s %10s %2s %-8s %-8s %10s %3s %2s %5s %s\n", $2,
$3, $4, $5, $6, $7, $8, $9, $10, $11, $12 }'
        return
}

#
# Start of main processing
#
SCRIPT=`$basename $0 `
PARMS="$*"

checkParms
listFiles

exit 0

If this does not immediately highlight the cause of the problem, my next line of attack is to list the files in the filesystem sorted by modification time, in the hope that the files most recently changed are the cause of the problem. I use findt listed below.

Findt script

#!/usr/bin/ksh
#
# Script: finds
# Aim:    List files in a filesystem by modification time
#
awk="/usr/bin/awk"
basename="/usr/bin/basename"
cut="/usr/bin/cut"
date="/usr/bin/date"
find="/usr/bin/find"
sed="/usr/bin/sed"
sort="/usr/bin/sort"

#
# Functions
#
checkParms()
{
        FILELIST=""
        for parm in $PARMS
        do
         if test "$parm" = "-r"
         then
          REVERSE="r"
         else
          check=`echo "$parm" | $cut -c1 `
          if test "$check" != "-"
          then
           FILELIST="$FILELIST $parm"
          fi
         fi
        done
        return
}

showFiles()
{
        (
        $date +"%Y %m %d"
        $find $FILELIST -xdev -type f -ls
        ) |
         $awk '
                NR == 1 {
                         thisyear = $1
                         lastyear = $1
                         thismonth = substr($2+100,2,2)
                         thisday   = substr($3+100,2,2)
                        }
                NR != 1 {
                         day = substr($9+100,2,2)

                         if ( $8 == "Jan" ) { month = "01" }
                         if ( $8 == "Feb" ) { month = "02" }
                         if ( $8 == "Mar" ) { month = "03" }
                         if ( $8 == "Apr" ) { month = "04" }
                         if ( $8 == "May" ) { month = "05" }
                         if ( $8 == "Jun" ) { month = "06" }
                         if ( $8 == "Jul" ) { month = "07" }
                         if ( $8 == "Aug" ) { month = "08" }
                         if ( $8 == "Sep" ) { month = "09" }
                         if ( $8 == "Oct" ) { month = "10" }
                         if ( $8 == "Nov" ) { month = "11" }
                         if ( $8 == "Dec" ) { month = "12" }

                         if ( length($10) == 4 )
                         {
                           year = $10
                           hour = "00:00"
                        }
                        else
                        {
                         if ( month <= thismonth ) { year = thisyear }
                         if ( month > thismonth ) { year = lastyear }
                          hour = $10
                        }
                        print year "/" month "/" day, hour, $0
                }
        ' |
        $sort $REVERSE |
        $cut -c17-
        
        return
}

#
# Start of main processing
#
SCRIPT=`$basename $0 `
PARMS="$*"

showFiles

exit 0

Once you have discovered the cause of your space problem, it may be best to copy the offending file elsewhere and then clear down the offending file, ie:

cat /dev/null > /filename

This has the advantage of freeing the disk space immediately, which removing a file will not do if the file is open to a running process.

Back to UNIX Scripting FAQ Index
Back to UNIX Scripting Forum

My Archive

Resources

Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close