INTELLIGENT WORK FORUMS
FOR COMPUTER PROFESSIONALS

Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!

*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Jobs

Apply awk program "from file 00001 to file 06831" (getting awk to repeat task on 6000+ fil

Apply awk program "from file 00001 to file 06831" (getting awk to repeat task on 6000+ fil

Apply awk program "from file 00001 to file 06831" (getting awk to repeat task on 6000+ fil

(OP)

Hello all!

Here's my question -> I have about 6000 files each with 9 columns, each columns containing about 2000+ rows. So, lots of data!
I have been able to successfully grab the 3 values that I need to extract from 1 file and to write those 3 values into an output.txt file.
I now need to be able to somehow get awk to accept multiple input files to do the same for the remaining 5999 files and write the 3 values it grabs to the SAME output.txt file. This would mean that by the end of it, I end up with 1 output.txt file with 3x6000 numbers in 3 neat columns.

I know that awk can accept multiple files in the following way: awk ’program’ input-file1 input-file2 etc. But clearly I can't write the 6000 names out manually.

Does anyone know of a smart way to tell it "Apply awk program from file Data00001.dat to Data06831.dat"

Thank you! :)

#!/usr/bin/gawk -f

BEGIN {
inSpill=0;
outSpill=0;
}

// {
if (($8>3000000)&&(inSpill==0)) {
inSpill=1;
start=$5
}
if (($8>10000000)&&(outSpill==0)) {
outSpill=1;
end=$5
}
lastRead=$5
}

END {
NumCoincParticles=(end-start);
percentage = NumCoincParticles/end;
print NumCoincParticles" "percentage" "lastRead" "start" "end;
}

RE: Apply awk program "from file 00001 to file 06831" (getting awk to repeat task on 6000+ fil

Hi

Which version of gawk ? With 4.0 and newer I would try to change BEGIN and END with BEGINFILE and ENDFILE, then running the script as script.awk Data*.dat > output.txt

Feherke.
http://feherke.github.com/

RE: Apply awk program "from file 00001 to file 06831" (getting awk to repeat task on 6000+ fil

(OP)
Hi Feherke,

I'm using GNU Awk 3.1.8 :S

RE: Apply awk program "from file 00001 to file 06831" (getting awk to repeat task on 6000+ fil

Hi

Then try it like this:

CODE --> Awk

#!/usr/bin/gawk -f

FNR==1 {
  whatever()
}

// {
  if (($8>3000000)&&(inSpill==0)) {
    inSpill=1;
    start=$5
  }

  if (($8>10000000)&&(outSpill==0)) {
    outSpill=1;
    end=$5
  }
  lastRead=$5
}

END {
  whatever()
}

function whatever()
{
  if (inSpill) {
    NumCoincParticles=(end-start);
    percentage = NumCoincParticles/end;
    print NumCoincParticles" "percentage" "lastRead" "start" "end;
  }

  inSpill=0;
  outSpill=0;
} 
Warning The above code was not tested

Feherke.
http://feherke.github.com/

RE: Apply awk program "from file 00001 to file 06831" (getting awk to repeat task on 6000+ fil

(OP)
Hi!

I'm not exactly sure how to implement this? I mean, I altered my .awk file as you suggested. But how do I now call it from within the Terminal so that it uses all 6000 files as its input files?
I used to call it as follows: awk -f 'Grab_values.awk' trigger_spill00001.dat > output.txt
(this would be for 1 input file only)

But now that we're trying to apply it to 6000+ files, what command exactly do i put in Terminal? Since the only format I know off for awk in Terminal is: awk -f 'awk program' inputfile1 > output.txt

Thanks!

RE: Apply awk program "from file 00001 to file 06831" (getting awk to repeat task on 6000+ fil

Hi

Already made a suggestion on that in my earlier post :

Quote (Feherke)

running the script as script.awk Data*.dat > output.txt

Feherke.
http://feherke.github.com/

RE: Apply awk program "from file 00001 to file 06831" (getting awk to repeat task on 6000+ fil

(OP)
Awesome worked perfectly!
Thanks a lot!

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Tek-Tips Forums free from inappropriate posts.
The Tek-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Tek-Tips forums is a member-only feature.

Click Here to join Tek-Tips and talk with other members!

Resources

Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close