×
INTELLIGENT WORK FORUMS
FOR COMPUTER PROFESSIONALS

Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!
  • Students Click Here

*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Students Click Here

Jobs

Apply awk program "from file 00001 to file 06831" (getting awk to repeat task on 6000+ fil

Apply awk program "from file 00001 to file 06831" (getting awk to repeat task on 6000+ fil

Apply awk program "from file 00001 to file 06831" (getting awk to repeat task on 6000+ fil

(OP)

Hello all!

Here's my question -> I have about 6000 files each with 9 columns, each columns containing about 2000+ rows. So, lots of data!
I have been able to successfully grab the 3 values that I need to extract from 1 file and to write those 3 values into an output.txt file.
I now need to be able to somehow get awk to accept multiple input files to do the same for the remaining 5999 files and write the 3 values it grabs to the SAME output.txt file. This would mean that by the end of it, I end up with 1 output.txt file with 3x6000 numbers in 3 neat columns.

I know that awk can accept multiple files in the following way: awk ’program’ input-file1 input-file2 etc. But clearly I can't write the 6000 names out manually.

Does anyone know of a smart way to tell it "Apply awk program from file Data00001.dat to Data06831.dat"

Thank you! :)

#!/usr/bin/gawk -f

BEGIN {
inSpill=0;
outSpill=0;
}

// {
if (($8>3000000)&&(inSpill==0)) {
inSpill=1;
start=$5
}
if (($8>10000000)&&(outSpill==0)) {
outSpill=1;
end=$5
}
lastRead=$5
}

END {
NumCoincParticles=(end-start);
percentage = NumCoincParticles/end;
print NumCoincParticles" "percentage" "lastRead" "start" "end;
}

RE: Apply awk program "from file 00001 to file 06831" (getting awk to repeat task on 6000+ fil

Hi

Which version of gawk ? With 4.0 and newer I would try to change BEGIN and END with BEGINFILE and ENDFILE, then running the script as script.awk Data*.dat > output.txt

Feherke.
http://feherke.github.com/

RE: Apply awk program "from file 00001 to file 06831" (getting awk to repeat task on 6000+ fil

(OP)
Hi Feherke,

I'm using GNU Awk 3.1.8 :S

RE: Apply awk program "from file 00001 to file 06831" (getting awk to repeat task on 6000+ fil

Hi

Then try it like this:

CODE --> Awk

#!/usr/bin/gawk -f

FNR==1 {
  whatever()
}

// {
  if (($8>3000000)&&(inSpill==0)) {
    inSpill=1;
    start=$5
  }

  if (($8>10000000)&&(outSpill==0)) {
    outSpill=1;
    end=$5
  }
  lastRead=$5
}

END {
  whatever()
}

function whatever()
{
  if (inSpill) {
    NumCoincParticles=(end-start);
    percentage = NumCoincParticles/end;
    print NumCoincParticles" "percentage" "lastRead" "start" "end;
  }

  inSpill=0;
  outSpill=0;
} 
Warning The above code was not tested

Feherke.
http://feherke.github.com/

RE: Apply awk program "from file 00001 to file 06831" (getting awk to repeat task on 6000+ fil

(OP)
Hi!

I'm not exactly sure how to implement this? I mean, I altered my .awk file as you suggested. But how do I now call it from within the Terminal so that it uses all 6000 files as its input files?
I used to call it as follows: awk -f 'Grab_values.awk' trigger_spill00001.dat > output.txt
(this would be for 1 input file only)

But now that we're trying to apply it to 6000+ files, what command exactly do i put in Terminal? Since the only format I know off for awk in Terminal is: awk -f 'awk program' inputfile1 > output.txt

Thanks!

RE: Apply awk program "from file 00001 to file 06831" (getting awk to repeat task on 6000+ fil

Hi

Already made a suggestion on that in my earlier post :

Quote (Feherke)

running the script as script.awk Data*.dat > output.txt

Feherke.
http://feherke.github.com/

RE: Apply awk program "from file 00001 to file 06831" (getting awk to repeat task on 6000+ fil

(OP)
Awesome worked perfectly!
Thanks a lot!

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Tek-Tips Forums free from inappropriate posts.
The Tek-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Tek-Tips forums is a member-only feature.

Click Here to join Tek-Tips and talk with other members!

Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close