FTP through a pipe 1

[screwloose]

I have done an ftp through a pipe before but cannot remember exactly how it is done. It eliminates the need to have extra disk space on the client machine (i.e., you don't have to have space for a 10GB file because the individual files can be ftp'd through the pipe)

Thanks.

 

[Ogzilal]

Hi,
Mybe you have used a named pipe. Here is some idea :

#make a named pipe calle ftp.fifo
mknod ftp.fifo p
#give values to some VARS
IP_ADDRESS=1.2.3.4
LOGIN_FTP=your_choice
PASSWD=your_passwd
# execute ftp with a here file in background and send the named pipe
ftp -i -n -v $IP_ADDRESS <<!END &
user $LOGIN_FTP $PASSWD
cd /some/direction
send ftp.fifo
qui
!END
# at this stage, ftp will wait for the data in ftp.fifo and will stop at the receivein of END_OF_FILE (CTRL_D)


# from another filter like oracle export, write to ftp.fifo

exp $ORACLEUSER/$ORACLE PASSWORD BUFFER=4096 COMPRESS=Y GRANTS=Y FILE=ftp.fifo LOG=/tmp/export.log OWNER=$ORACLEUSER
# when oracle export is done, ftp will stop

Ali

 

[sbix]

I have some problem with this solution.
When I fire the "filling pipe" program (a backup of a filesystem onto the pipe), ftp aborts the send command with the message FTP (name of the pipe): NOT A PLAIN FILE

 

[KenCunningham]

A couple of suggestions from Google - is the file it falls over on a directory, or is it > 2Gb?

 

[sbix]

Ken,
the "filling pipe" program is:
find /Starting_Dir |backup -qvf MYPIPE

ftp is already waiting (reading) from MYPIPE in order to send to remote computer

 

[Ogzilal]

Hi,

try sending in binary mode if it's not the default.

add the command bin before send

Ali

 

[p5wizard]

What type of box are you FTPing to? Is FTP your only possible means of getting the backup data over to the server?

The ftp command interpreter does not support pipes (I checked man page on aix53) Neither does ftpd apparently though it is not mentioned in the man page. So getting the pipe from your client to the server (other way around) is not an option...

rcp? rsh? NFS?



HTH,

p5wizard

 

[sbix]

p5wizard,
the problem is not into the ftp INTERPRETER ... because I don't pipe any command TO THE INTERPRETER ...
It seems like ftp can't "send" a PIPE.
In effect the error messaqge is enough clear "pipe_name: NOT A PLAIN FILE

 

[p5wizard]

I misinterpreted the man page (pipe/fifo) my mistake.

And you are right, ftp and ftpd do not support pipes/fifos whatever you call 'em, only plain files: ascii text or binary data.


HTH,

p5wizard

 

[p5wizard]

Strangely enough the ftpd can handle named pipes (just tried it) but unfortunately that doesn't help you...

you can PUT/SEND to a pipe, but you can't GET from a pipe...

There is probably a way (with .netrc and auto logon and maybe expect also)

(from the man page)
if you type at the ftp> prompt
put - remotefile
all data from stdin is sent to the remotefile specified

so

 (echo "put - remotefile"; cat /named/pipe)|\
 ftp RemoteServer

might work if your .netrc performs the logon on RemoteServer. I have tried it with an lptest (16000 bytes) through a pipe and that seems to work.

It could be worth a try with your ORACLE exp stream down a pipe.



HTH,

p5wizard

 

[p5wizard]

Any luck on this?

(echo "put - remotefile"; cat /named/pipe)|\
ftp RemoteServer

or

(echo "binary"; echo "put - remotefile"; cat /named/pipe)|\
ftp RemoteServer

both with automated logon via .netrc file



HTH,

p5wizard

 

[sbix]

Uhm .... looks like an interesting solution.
I cannot use .netrc for security reasons so I have to setup an user who can only execute ftps on a specific home dir .... (this just to pass over the "password in clear" problem)