measuring download time for a *.pdf file

[buxtonicer1]

I'm pretty new to unix having previously worked with java.

Anyway here is my problem. I wrote an app in java that takes as input a text file with a list of url's. These urls all point to reports that are in a *.pdf format. My app calculated the time it took for each report to download. Basically I used a while loop that read each byte of the file until I got to the end of the file. A timestamp was used either side of this process. By getting the difference between the timestamps I could calculate how long it took for the file to download.
So my question is how do I contruct a .ksh script to do something similar. Ideally I would run the script from a few different locations to get an idea of how fast/slow my network connection is.

Time to google me thinks...

 

[feherke]

Hi

The [tt]time[/tt] built-in command will tell you the duration of the command it get as parameter on [tt]bash[/tt] and [tt]ksh[/tt] :

time wget -i /input/file_with_pdf_report_urls

Not sure if this is what you want.

Feherke.

http://rootshell.be/~feherke/

 

[buxtonicer1]

I'll try it

 

[buxtonicer1]

Can you use unix to work with urls then

 

[feherke]

Hi

No. I think there is no operating system which works with URLs directly. So neither Unix. But of course, there are good programs to do that.

Feherke.

http://rootshell.be/~feherke/

 

[buxtonicer1]

So there is not way to call a website even from unix, thats a pity. Not sure how I'm going to convert my java app to what I thought would be a simple shell script .... .....

:-(

 

[feherke]

Hi

way to call a website

Sorry to say it, but this is kind of kitchen language. You get better help if you discribe you problem with professional terms.

If you want to download a file/site, you can use a lot of downloaders and programs suitable for download : [tt]wget[/tt], [tt]curl[/tt], [tt]httrack[/tt], [tt]lynx[/tt], [tt]GET[/tt], [tt]netcat[/tt], etc. All those can de used in your shell script.

But if you already have the Java application, why not use it ?

Feherke.

http://rootshell.be/~feherke/

 

[buxtonicer1]

Its just I'm new to UNIX and appreciate that it has very powerful tools / functions and commands. Its just that I thought that I could copy my script to a box in a different location, run the script and see how long the files take to dl.

Those tools you mentioned - are they standard with the korn shell or are they added extras

 

[feherke]

Hi

Those tools you mentioned - are they standard with the korn shell or are they added extras

Sorry, I use Linux, no idea about Unix standards.

One thing is sure : [tt]GET[/tt] is part of the [tt]perl[/tt] so if you have [tt]perl[/tt] installed, you have [tt]GET[/tt] too. And also [tt]POST[/tt] and [tt]HEAD[/tt].

I would bet on [tt]wget[/tt]. That is old and popular. Most probably you have it.

run the script and see how long the files take to dl

A simple download, can be executed from command line ( should be one line ) :

while read one; do echo "$one"; time GET "$one" > "${one##*/}"; done < /input/url_list.txt

This reads the input file line by line and tries to download each one, then outputs the time spent. Basic, no error checking.

Feherke.

http://rootshell.be/~feherke/

 

[buxtonicer1]

ya - perl, I like the sound of that, surely perl can be called from the .ksh and that can help me do what I want

 

[buxtonicer1]

1. while read one;
2. do echo "$one";
3. time GET "$one" > "${one##*/}";
4. done < /input/url_list.txt

What does this code mean - I'm new to this remember

1./2. do while loop , is read one significant , is one the value of each url ?

3. one##* not sure what that means

4. Is this where you read from the list of urls

 

[buxtonicer1]

#!/usr/local/bin/perl5 -w
### Language: PERL 5
###############################################

#require "G:\\Shared\\Microbank\\Bin\\Library.perl";


time GET "

http://www.google.ie"

this file is called download_time.pl with permissions 700
, when I do

prompt=> download_time.pl and hit return at the command

I get the following errors

Bareword found where operator expected at download_time.pl line 14, near "time GET"
(Do you need to predeclare time?)
syntax error at download_time.pl line 14, near "time GET "
String found where operator expected at download_time.pl line 14, near "GET "

http://www.google.ie""

(Do you need to predeclare GET?)
Execution of download_time.pl aborted due to compilation errors.

 

[feherke]

Hi

[gray]# do a loop while the condition id true[/gray]
[gray]# read on line of text from input and place it in variable one[/gray]
while read one; do
[gray]# print to the standard output the line currently processed[/gray]
echo "$one"
[gray]# GET from the given ( hopefully ) URL and redirect the output to the file[/gray]
[gray]# ( the file name is the part afteer last slash ( / ) of the URL[/gray]
time GET "$one" > "${one##*/}"
[gray]# end of while loop[/gray]
[gray]# take the input from the the given file[/gray]
done < /input/url_list.txt

#!/usr/local/bin/perl5 -w
### Language: PERL 5
###############################################

#require "G:\\Shared\\Microbank\\Bin\\Library.perl";


time GET "www.google.ie"

Wrong. The [tt]GET[/tt] is an executable, written in [tt]perl[/tt]. So you can use it from command line or a shell script :

#!/bin/ksh

time GET "example.com"

If you want to use it in a [tt]perl[/tt] script, then use it like external executable :

exec 'time GET "example.com"';

If you want all in [tt]perl[/tt] you can use the same [tt]LWP[/tt] module as [tt]GET[/tt] use.

http://search.cpan.org/dist/libwww-perl/

Feherke.

http://rootshell.be/~feherke/

 

[buxtonicer1]

$ time GET "

http://www.google.com"

ksh: GET: not found

real 0m0.01s
user 0m0.00s
sys 0m0.01s

GET is not working !

will try it in the perl script instead

 

[feherke]

Hi

That is weird. Even the ActivePerl for Window has the [tt]GET[/tt] ported.

Feherke.

http://rootshell.be/~feherke/

 

[buxtonicer1]

No worries

 

[Annihilannic]

On a couple of Red Hat systems I checked GET seems to be a part of the perl-lib

http://www-perl

package, not perl itself. On Solaris GET doesn't usually seem to be installed.

Annihilannic.