Handling a 450,000x450,000 array with Perl

[cadbilbao]

Hi.

I'd like to create a program to handle data (natural numbers)
within a 450,000 x 450,000 array. I'm considering Perl, but
I don't know is it's powerful enough to manage it.

Thank you very much.

 

[stevexff]

How big are the numbers? Assuming a ball park of 8 bytes, that's 1.62 TB. I hope you've got a really big box.

Perhaps you should tell us what you want to do, as I suspect some kind of file processing is in order here...

Steve

[small]"Every program can be reduced by one instruction, and every program has at least one bug. Therefore, any program can be reduced to one instruction which doesn't work." (Object:erlDesignPatterns)[/small]

 

[cadbilbao]

Hi.

Thank you very much for your quick answers. I try to handle data from a
450x450 meter surface.

I've got 100,000 elements distributed on this surface, and each
square millimeter can hold one or more elements. So the array to create
is a sparse one: most of square millimeters holds zero elements, and
the rest, 20 or less.

I'm trying to find the square meter with more elements inside.

Thank you very much.

 

[mikrom]

Your data will be so called "sparce matrix" which will be often used in numerical mathematics.
There are alorithms for handling and storing sparse matrices.
Python has such a module and I would wonder, when Perl hasn't.

I looked and I found)
Here you have "Math::MatrixSparse - Perl extension for sparse matrices" at

http://search.cpan.org/~kesinger/Math-MatrixSparse-0.01/MatrixSparse.pm