well... 177*177*171*171*8bytes = 6.83Gb... Each 32 bits program has access to 4 (or is it 2?) Gb of memory, since the address space is 32 bits. I suggest you buy a 64 bit system, with a 64 bit compiler... assuming their adressing space is 64 bits. Or, better yet, I suggest you rewrite your program, you must have some wasted space in there!
Good luck,
Vincent