INTELLIGENT WORK FORUMS
FOR COMPUTER PROFESSIONALS

Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!

*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Jobs

awk script to average values from multiple files

awk script to average values from multiple files

awk script to average values from multiple files

(OP)
Hello awk experts, I need help with writing an awk script for the following task:

I have a 10 files full of values (8 fixed size columns), and I want to average out the values from each field from all of the input files.

Here are the values from 1 of 10 files (other 9 files have same fields with different values:

Input file1:
1 99.7% 0.0% 90.9% 8.0GB 7.7GB 9.000GB 5.812GB
2 94.8% 23.0% 94.6% 8.0GB 7.6GB 2.000GB 10.781GB
3 93.9% 36.0% 110.3% 5.0GB 6.1GB 4.000GB 3.812GB
4 99.2% 8.0% 116.3% 2.0GB 2.3GB 5.000GB 5.312GB
5 97.5% 12.0% 86.9% 3.0GB 2.6GB 3.000GB 9.562GB
6 95.4% 4.0% 49.4% 12.0GB 5.9GB 8.000GB 10.625GB
7 99.6% 44.0% 142.6% 8.0GB 11.4GB 7.875GB 5.562GB
8 95.4% 35.0% 131.1% 5.0GB 7.2GB 6.188GB 9.375GB
9 98.2% 14.0% 104.6% 6.0GB 6.3GB 6.000GB 12.375GB
10 93.3% 3.0% 79.8% 8.0GB 6.4GB 1.000GB 220.500GB

Input file2:
1 98.5% 14.0% 104.6% 6.0GB 6.3GB 6.000GB 12.500GB
2 93.5% 56.0% 121.0% 10.0GB 12.1GB 5.500GB 9.500GB
3 98.8% 52.0% 126.8% 12.0GB 15.2GB 7.312GB 11.375GB
4 98.7% 42.0% 129.4% 8.0GB 10.4GB 6.250GB 10.500GB
5 95.3% 12.0% 89.7% 10.0GB 9.0GB 5.500GB 11.250GB
6 97.5% 19.0% 107.8% 2.0GB 2.7GB 3.000GB 4.438GB
7 99.4% 17.0% 109.4% 13.0GB 14.2GB 10.000GB 5.500GB
8 94.5% 27.0% 96.7% 5.0GB 4.8GB 3.000GB 11.375GB
9 99.7% 26.0% 123.0% 12.0GB 14.8GB 12.000GB 5.875GB
10 99.0% 58.0% 162.2% 8.0GB 13.0GB 8.375GB 10.938GB

Input file3:
...
...

Input file10:
1 98.4% 14.0% 104.5% 6.0GB 6.3GB 6.000GB 12.500GB
2 93.2% 46.0% 109.8% 10.0GB 11.0GB 5.500GB 9.500GB
3 99.3% 49.0% 126.1% 12.0GB 15.1GB 7.312GB 11.375GB
4 98.8% 43.0% 128.9% 8.0GB 10.3GB 6.250GB 10.500GB
5 95.3% 12.0% 89.7% 10.0GB 9.0GB 5.500GB 11.250GB
6 97.4% 18.0% 106.2% 2.0GB 2.7GB 3.000GB 4.438GB
7 98.6% 16.0% 108.5% 13.0GB 14.1GB 10.000GB 5.500GB
8 94.7% 27.0% 93.0% 5.0GB 4.6GB 3.000GB 11.375GB
9 99.7% 24.0% 120.1% 12.0GB 14.4GB 12.000GB 5.875GB
10 99.6% 60.0% 166.3% 8.0GB 13.3GB 8.375GB 10.938GB


---------------------------------------------------
output:
1 average(99.7% + 98.5% +... + ... +98.4%) average(0.0% + 14% +... + ... 14.0%) average(90.9% + 104.6% +... + ...+ 104.5%) average(8.0GB + 6.0GB +... + ...+ 6.0GB) average 7.7GB + 6.3GB +... + ...+ 6.3GB) average(9.000GB + 6.000GB +... + ... +6.000GB) average(5.812GB + 12.5000GB +... + ... 12.5000GB)
...
...
10 average(99.7% + 98.5% +... + ... + 99.6%) average(0.0% + 14% +... + ... + 60.0%) average(90.9% + 104.6% +... + ...+ 166.3%) average(8.0GB + 6.0GB +... + ...8.0GB) average 7.7GB + 6.3GB +... + ...+ 13.3GB) average(9.000GB + 6.000GB +... + ...+ 8.375GB) average(5.812GB + 12.5000GB +... + ...10.938GB)

-------
I first stripped the non-numeric characters from the files: $ sed s/%//g file1 | sed s/GB//g > file1 (did same for file2 file3 file4 file5 file6 file7 file8 file9 file10)

I then need to average out the same field from each file (ie. [file1, field1], [file2, field1], [file3, field1]... [file10, field1])

Any help is greatly appreciated. Thanks!

RE: awk script to average values from multiple files

Compute the column averages using something like this:

CODE

# run:
# awk -f pdtak1.awk pdtak1_01.txt pdtak1_02.txt .. pdtak1_10.txt
BEGIN {
   # initialize sums and averages
   for (i=1; i<=8; i++) { 
     sum[i] = 0
     avg[i] = 0
   }
}
{
   # remove unwanted characters
   gsub(/%/, "", $0)
   gsub(/GB/, "", $0)
   # sum colum values
   for (i=1; i<=8; i++) { 
     sum[i] = sum[i] + $i
   }   
}
END {
   total_records = NR
   print "Record processed: " total_records
   # compute and print averages
   print "Column Averages :"
   for (i=1; i<=8; i++) { 
     avg[i] = sum[i] / total_records
     printf "%8.4f ", avg[i]
   }
   printf "\n"
} 

Applying it on your 3 files we get

CODE

$ awk -f pdtak1.awk pdtak1_01.txt pdtak1_02.txt pdtak1_10.txt
Record processed: 30
Column Averages :
  5.5000  97.2300  27.0333 111.0067   7.9000   8.8933   6.1979  16.0073 

RE: awk script to average values from multiple files

Quote:


I then need to average out the same field from each file (ie. [file1, field1], [file2, field1], [file3, field1]... [file10, field1])
What do you mean with field ? The column ?

Then you can run the script extra on every file any you get the averages form this file - e.g.:

CODE

$ awk -f pdtak1.awk pdtak1_01.txt
Record processed: 10
Column Averages :
  5.5000  96.7000  17.9000 100.6500   6.5000   6.3500   5.2063  29.3716 

$ awk -f pdtak1.awk pdtak1_02.txt
Record processed: 10
Column Averages :
  5.5000  97.4900  32.3000 117.0600   8.6000  10.2500   6.6937   9.3251 

$ awk -f pdtak1.awk pdtak1_10.txt
Record processed: 10
Column Averages :
  5.5000  97.5000  30.9000 115.3100   8.6000  10.0800   6.6937   9.3251 

RE: awk script to average values from multiple files

(OP)
Thank you Mikrom!
You are awsome!
Yes, I meant columns.

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Tek-Tips Forums free from inappropriate posts.
The Tek-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Tek-Tips forums is a member-only feature.

Click Here to join Tek-Tips and talk with other members!

Resources

Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close