Filehandle from data?

[Kirsle]

A while back I wrote a module called Archive::Tyd, which is for *simplistic* archiving of multiple files. Currently how it works is:

The Tyd archive has a format like this for storing all of its files (when in unencrypted form):

filename::data
filename::data
filename::data
...

With one file on its own line. End-of-line characters are substituted and can be rechanged later when the file needs to be accessed.

When all the file data is combined in this format, the resulting scalar is encrypted using Crypt::CipherSaber using a password provided by the program running the module.

How it currently works is that the entire archive is loaded into memory, so that $tydobj->{files}->{'filename'} has the data to that file. This works when the archive is full of small files that can easily fit into memory.

I was thinking of ways to allow Tyd to hold larger files but without draining memory to do so. I was thinking that the first line in the unencrypted file could be an index listing what lines each file exists on.

But the problem still would arrise when one of those lines is to be read from, or when the module is going to write all of the files back into the archive again.

The question is: is it possible to make a filehandle from a set of internal data, so that large files can be read by their filehandle instead of slurping all of their data at once?

So in this case, if the data for "movie.mpeg" existed on line 6 of the archive, and my program wanted to access this file (maybe to write it directly to an external file)... how could I turn its data into a filehandle, so that a while(<FH>) could be used on it?

 

[Kirsle]

I should also note that I don't want to use Archive::Zip. The purpose for this module is mainly for my own programs to keep their own files, with the program's own password. With Archive::Zip, users could just rename the file to a format that WinZip can readily open and read or change the files. I want to make it difficult on them.

 

[Xaqte]

I can't take credit for the code, but I think this is what your asking for (I haven't tested it):
Assuming you already have all the filenames in an array

my @handles = map {FileHandle->new("> $_")} @filenames;

If you want to write to one in a while statement:

while ( defined( my $line = $handles[ 3 ]->getline ) ) {
  # do something with $line
}

or:

my $tmp = $handles[ 3 ];
while ( <$tmp> ) {
  # do something
}

Let us know your results!

X