×
INTELLIGENT WORK FORUMS
FOR COMPUTER PROFESSIONALS

Contact US

*
*
*
*

Thanks. We have received your request and will respond promptly.

Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!
*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Students Click Here

awk script to average values from multiple files

thread271-1766011

awk script to average values from multiple files

awk script to average values from multiple files

(OP)
Hello awk experts, I need help with writing an awk script for the following task:

I have a 10 files full of values (8 fixed size columns), and I want to average out the values from each field from all of the input files.

Here are the values from 1 of 10 files (other 9 files have same fields with different values:

Input file1:
1 99.7% 0.0% 90.9% 8.0GB 7.7GB 9.000GB 5.812GB
2 94.8% 23.0% 94.6% 8.0GB 7.6GB 2.000GB 10.781GB
3 93.9% 36.0% 110.3% 5.0GB 6.1GB 4.000GB 3.812GB
4 99.2% 8.0% 116.3% 2.0GB 2.3GB 5.000GB 5.312GB
5 97.5% 12.0% 86.9% 3.0GB 2.6GB 3.000GB 9.562GB
6 95.4% 4.0% 49.4% 12.0GB 5.9GB 8.000GB 10.625GB
7 99.6% 44.0% 142.6% 8.0GB 11.4GB 7.875GB 5.562GB
8 95.4% 35.0% 131.1% 5.0GB 7.2GB 6.188GB 9.375GB
9 98.2% 14.0% 104.6% 6.0GB 6.3GB 6.000GB 12.375GB
10 93.3% 3.0% 79.8% 8.0GB 6.4GB 1.000GB 220.500GB

Input file2:
1 98.5% 14.0% 104.6% 6.0GB 6.3GB 6.000GB 12.500GB
2 93.5% 56.0% 121.0% 10.0GB 12.1GB 5.500GB 9.500GB
3 98.8% 52.0% 126.8% 12.0GB 15.2GB 7.312GB 11.375GB
4 98.7% 42.0% 129.4% 8.0GB 10.4GB 6.250GB 10.500GB
5 95.3% 12.0% 89.7% 10.0GB 9.0GB 5.500GB 11.250GB
6 97.5% 19.0% 107.8% 2.0GB 2.7GB 3.000GB 4.438GB
7 99.4% 17.0% 109.4% 13.0GB 14.2GB 10.000GB 5.500GB
8 94.5% 27.0% 96.7% 5.0GB 4.8GB 3.000GB 11.375GB
9 99.7% 26.0% 123.0% 12.0GB 14.8GB 12.000GB 5.875GB
10 99.0% 58.0% 162.2% 8.0GB 13.0GB 8.375GB 10.938GB

Input file3:
...
...

Input file10:
1 98.4% 14.0% 104.5% 6.0GB 6.3GB 6.000GB 12.500GB
2 93.2% 46.0% 109.8% 10.0GB 11.0GB 5.500GB 9.500GB
3 99.3% 49.0% 126.1% 12.0GB 15.1GB 7.312GB 11.375GB
4 98.8% 43.0% 128.9% 8.0GB 10.3GB 6.250GB 10.500GB
5 95.3% 12.0% 89.7% 10.0GB 9.0GB 5.500GB 11.250GB
6 97.4% 18.0% 106.2% 2.0GB 2.7GB 3.000GB 4.438GB
7 98.6% 16.0% 108.5% 13.0GB 14.1GB 10.000GB 5.500GB
8 94.7% 27.0% 93.0% 5.0GB 4.6GB 3.000GB 11.375GB
9 99.7% 24.0% 120.1% 12.0GB 14.4GB 12.000GB 5.875GB
10 99.6% 60.0% 166.3% 8.0GB 13.3GB 8.375GB 10.938GB


---------------------------------------------------
output:
1 average(99.7% + 98.5% +... + ... +98.4%) average(0.0% + 14% +... + ... 14.0%) average(90.9% + 104.6% +... + ...+ 104.5%) average(8.0GB + 6.0GB +... + ...+ 6.0GB) average 7.7GB + 6.3GB +... + ...+ 6.3GB) average(9.000GB + 6.000GB +... + ... +6.000GB) average(5.812GB + 12.5000GB +... + ... 12.5000GB)
...
...
10 average(99.7% + 98.5% +... + ... + 99.6%) average(0.0% + 14% +... + ... + 60.0%) average(90.9% + 104.6% +... + ...+ 166.3%) average(8.0GB + 6.0GB +... + ...8.0GB) average 7.7GB + 6.3GB +... + ...+ 13.3GB) average(9.000GB + 6.000GB +... + ...+ 8.375GB) average(5.812GB + 12.5000GB +... + ...10.938GB)

-------
I first stripped the non-numeric characters from the files: $ sed s/%//g file1 | sed s/GB//g > file1 (did same for file2 file3 file4 file5 file6 file7 file8 file9 file10)

I then need to average out the same field from each file (ie. [file1, field1], [file2, field1], [file3, field1]... [file10, field1])

Any help is greatly appreciated. Thanks!

RE: awk script to average values from multiple files

Compute the column averages using something like this:

CODE

# run:
# awk -f pdtak1.awk pdtak1_01.txt pdtak1_02.txt .. pdtak1_10.txt
BEGIN {
   # initialize sums and averages
   for (i=1; i<=8; i++) { 
     sum[i] = 0
     avg[i] = 0
   }
}
{
   # remove unwanted characters
   gsub(/%/, "", $0)
   gsub(/GB/, "", $0)
   # sum colum values
   for (i=1; i<=8; i++) { 
     sum[i] = sum[i] + $i
   }   
}
END {
   total_records = NR
   print "Record processed: " total_records
   # compute and print averages
   print "Column Averages :"
   for (i=1; i<=8; i++) { 
     avg[i] = sum[i] / total_records
     printf "%8.4f ", avg[i]
   }
   printf "\n"
}

Applying it on your 3 files we get

CODE

$ awk -f pdtak1.awk pdtak1_01.txt pdtak1_02.txt pdtak1_10.txt
Record processed: 30
Column Averages :
  5.5000  97.2300  27.0333 111.0067   7.9000   8.8933   6.1979  16.0073

RE: awk script to average values from multiple files

Quote:


I then need to average out the same field from each file (ie. [file1, field1], [file2, field1], [file3, field1]... [file10, field1])
What do you mean with field ? The column ?

Then you can run the script extra on every file any you get the averages form this file - e.g.:

CODE

$ awk -f pdtak1.awk pdtak1_01.txt
Record processed: 10
Column Averages :
  5.5000  96.7000  17.9000 100.6500   6.5000   6.3500   5.2063  29.3716 

$ awk -f pdtak1.awk pdtak1_02.txt
Record processed: 10
Column Averages :
  5.5000  97.4900  32.3000 117.0600   8.6000  10.2500   6.6937   9.3251 

$ awk -f pdtak1.awk pdtak1_10.txt
Record processed: 10
Column Averages :
  5.5000  97.5000  30.9000 115.3100   8.6000  10.0800   6.6937   9.3251

RE: awk script to average values from multiple files

(OP)
Thank you Mikrom!
You are awsome!
Yes, I meant columns.

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Tek-Tips Forums free from inappropriate posts.
The Tek-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Tek-Tips forums is a member-only feature.

Click Here to join Tek-Tips and talk with other members! Already a Member? Login
Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close