Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!
  • Students Click Here

*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Students Click Here


using curl with awk in command line/shell script

using curl with awk in command line/shell script

using curl with awk in command line/shell script

I have a text file url.txt with urls one per line. Something like:

If I put the following command in command line or in a shell script it works fine:


cat urls.txt | awk '{ print $0}'
This simply prints out the content of urls.txt line by line

So now I want to go 'get' the images from the urls so I thought I could do something similarly with curl such as


cat urls.txt | awk '{ curl --remote-name $0 }'
but this isn't working.
What I thought this would do is substitue each url from url.txt in $0 so that I would be doing a


curl --remote-name http://www.something.com/image1.jpg
   and so on.
Any ideas how to use curl with awk and use a variable instead of hardcoded http value(since my urls are coming from a text file)? Thanks.

RE: using curl with awk in command line/shell script

You can use the "System" function to execute shell commands within AWK:


cat urls.txt | awk '{ system("curl --remote-name " $0 }'

RE: using curl with awk in command line/shell script

I'd use xargs:


xargs -n1 curl --remote-name <url.txt

tgmlify - code syntax highlighting for your tek-tips posts

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Tek-Tips Forums free from inappropriate posts.
The Tek-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Tek-Tips forums is a member-only feature.

Click Here to join Tek-Tips and talk with other members!

Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close