cannot see the error, need help 1

[Baraka69]

I am writing a script that calls an awk script and takes 2 arguments:
argument #1 is a directory
argument #2 is a filter

The awk script performs some statistical operations. The input file(s) are gzip-ed files called "access_log.gz". In order to get a "per file" output I use a "while-loop" in which I call my awk-script.
To implement an element of sorting, my idea was to "grep" for the searchword.

In order I do a "gzcat" piped into the "grep" piped into the awk-script.

For some strange reason when I manually execute my command line (HP-UX-11.00) I get the desired result, but when I call the exact same line within my shell script I get no filtering (grep seems to fail completely).

To get an idea what I am talking about, some lines of code from my script:

# check for arguments (1st argument must be a valid directory)
if [ $# -eq 0 ] || [ ! -d $1 ]; then
echo "ERROR."; exit
fi
myTargetDir=$1

# check for 2nd argument (will be used as search pattern)
if [ $2 ]; then
mySearchPattern="| grep $2 "
myCustomVars="-v mySuffix=$2 "
else
mySearchPattern=""
myCustomVars=""
fi

# start looking in myTargetDir and find all instances of mySearchFile
cd $myTargetDir
find . -name $mySearchFile | while read myFileName
do
gzcat $myFileName $mySearchPattern| awk $myCustomVars-v myFileName=$myFileName -f $myScriptDir$myScriptName
done 2>> $myLogfileDir$myLogfileName


I have added an extra line with echo "..." duplicating the red line of code and copy & pasted this command into my command line - that works just finde ... I don't see my error. Please help!

 

[vgersh99]

Shouldn't this be like this?
I don't know if you need a '/' in your path to an awk script, but you're missing a space BEFORE the '-v'

Only doesn't 'gzcat' have only one parameter - the name of the file to 'uncompress'?

gzcat $myFileName $mySearchPattern| awk $myCustomVars -v myFileName=$myFileName -f ${myScriptDir}/${myScriptName}


vlad
+----------------------------+
| #include<disclaimer.h> |
+----------------------------+

 

[sampsonr]

In the variable mySearchPattern, you include a pipe. That won't work because of the way the shell interprets variables. If the pipe is in a variable, it gets treated as a literal value, not as a shell meta-character like you're expecting.

Try this instead:

if [ $2 ]; then
mySearchPattern=&quot;grep $2 &quot;
myCustomVars=&quot;-v mySuffix=$2 &quot;
else
# forward input with no modifications
mySearchPattern=&quot; cat &quot;
myCustomVars=&quot;&quot;
fi

# start looking in myTargetDir and find all instances of mySearchFile
cd $myTargetDir
find . -name $mySearchFile | while read myFileName
do
gzcat $myFileName | $mySearchPattern | awk $myCustomVars-v myFileName=$myFileName -f $myScriptDir$myScriptName

 

[Baraka69]

Hi vlad,
thanks for the quick reply. You are right on both accounts, the one parameter for gzcat and the space in front of the -v. But (excuse me for pointing this out) the variables are in my example and they do contain all the information to clarify that.

[tt]if [ $2 ]; then
mySearchPattern=&quot;| grep $2 &quot;
myCustomVars=&quot;-v mySuffix=$2 &quot;
else
mySearchPattern=&quot;&quot;
myCustomVars=&quot;&quot;
fi
[/tt]

case #1 with variables:
[tt]gzcat $myFileName $mySearchPattern| awk $myCustomVars-v myFileName=$myFileName -f $myScriptDir$myScriptName
[/tt]
case #2 argument $2 is set:
[tt]gzcat $myFileName | grep $2 | awk -v mySuffix=$2 -v myFileName=$myFileName -f $myScriptDir$myScriptName
[/tt]
case #3 argument $2 is not set:
[tt]gzcat $myFileName | awk -v myFileName=$myFileName -f $myScriptDir$myScriptName
[/tt]
Case #2 does not work as expected, #3 is just fine ... I am going mad - still need help!

 

[Baraka69]

Hi sampsonr,
your idea sounded great and did work for me - thanks a lot!