INTELLIGENT WORK FORUMS
FOR COMPUTER PROFESSIONALS

Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!

*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Jobs

awk csv file, pass output to bash variables, use variables to make sed replacements & new file n

awk csv file, pass output to bash variables, use variables to make sed replacements & new file n

awk csv file, pass output to bash variables, use variables to make sed replacements & new file n

(OP)
Hi,

I need to
1. read various csv files in a hard-coded folder ( 2 columns)
2. Save the output of columns 1 and column 2 to var1 and var2
3. Copy a template file I have and generate a new file name based on var2.
4. Make substitutions inside the new file for instances of Field1 = (var1) and Field2 = (var2)

For the CSV file:

Column1 has a 4 to 6 digit value
Column2 has names with "\", spaces and "-", but no commas

So far, I have most of the code pseudo-code

#!/bin/bash
FILES=/mydir/*.txt
for f in $FILES
do
echo "Processing $f file..."
var1=$(awk -F ',' ' {print $1}' $f )
var2=$(awk -F ',' ' {print $2}' $f )
echo "Doing $var1 and $var2"
cp /myotherdir/Skeleton-File /myotherdir/"$var2"-"$var1"-newfile
sed -e "s/Field1/"$var1"/g" -i /myotherdir/"$var2"-"$var1"-newfile
sed -e "s/Field2/"$var2"/g" -i /myotherdir/"$var2"-"$var1"-newfile
done

I have had mixed results attempting nested do loops, but my main problem seems to be with var2, attempting $var2, bash treats spaces as a new line (even though awk prints it ok) and "$var2" treats the variable as a single long line and generates a combined file name of all the contents.

Any ideas on how to accomplish this?

Thanks



RE: awk csv file, pass output to bash variables, use variables to make sed replacements & new file n

I'd try to do all the stuff in awk, eg (typed, untested)

CODE

awk -F, 'NR==FNR{t[NR]=$0;next} {out="/myotherdir/"$2"-"$1"-newfile" for(i=1;(i in t);++i){ x=t[i];gsub(/Field1/,$1,x);gsub(/Field2/,$2,x) print x >out;close(out) } }' /myotherdir/Skeleton-File /mydir/*.txt

Hope This Helps, PH.
FAQ219-2884: How Do I Get Great Answers To my Tek-Tips Questions?
FAQ181-2886: How can I maximize my chances of getting an answer?

RE: awk csv file, pass output to bash variables, use variables to make sed replacements & new file n

(OP)
Hi,

I have begun dissecting your code, but I receive an error on the print.

Does it need to be in { print x} ?

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Tek-Tips Forums free from inappropriate posts.
The Tek-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Tek-Tips forums is a member-only feature.

Click Here to join Tek-Tips and talk with other members!

Resources

Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close