INTELLIGENT WORK FORUMS
FOR COMPUTER PROFESSIONALS

Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!

*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Jobs

removing duplicate lines depending on value of $1
2

removing duplicate lines depending on value of $1

removing duplicate lines depending on value of $1

(OP)
I have a file that looks like this:

CODE

apple4 yellow
apple1 red
apple6 blue
apple3 green
grape red
grape purple
apple68 yellow
apple32 red
apple98 orange
grape yellow 

I would like to remove duplicate lines using the following:

CODE

awk '!x[$2]++' 

However I only want to remove duplicates if $1 equals grape.

Desired output:

CODE

apple4 yellow
apple1 red
apple6 blue
apple3 green
grape red
grape purple
apple68 yellow
apple32 red
apple98 orange
grape yellow 

Not sure where to go from there..

The log is if $1 equals "grape" and I have seen $2 before then don't print.



RE: removing duplicate lines depending on value of $1

2
Hi

Based on your sample data, this seems to be what you are looking for :

CODE --> command-line

awk '!x[$2]++ || $1!="grape"' /input/file

Feherke.
http://feherke.github.com/

RE: removing duplicate lines depending on value of $1

(OP)
Thanks for your suggestions..

I ended up just filtering out the "grapes" into a 2nd file. Then doing something like

awk 'NR==FNR && !x[$4,$5]++' wanted_info.txt print_this_if_not_in_first_file.txt

Thanks!

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Tek-Tips Forums free from inappropriate posts.
The Tek-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Tek-Tips forums is a member-only feature.

Click Here to join Tek-Tips and talk with other members!

Resources

Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close