INTELLIGENT WORK FORUMS
FOR COMPUTER PROFESSIONALS

Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!

*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Jobs

Create 250^4 or 250^5 elements-arrays

Create 250^4 or 250^5 elements-arrays

(OP)
Hi,

How can I declare the arrays of such size

REAL, DIMENSION(250,250,250,250) :: A ??ponder


I use intel visual Fortran and it generates the error message: The size of the array dimension is too large, and overflow occurred when computing the array size.

I cannot get away with sparse matrix as well.

Please give some suggestion.

Thanks.

RE: Create 250^4 or 250^5 elements-arrays


Well, let's think for a moment...you are requesting an array that will have 250*250*250*250 = 3906250000 items in it; if each item was 1 byte long, you would need 3.9 GB of memory...but you want this array to be REAL. If the length of the default REAL is 8 bytes, you are requesting 31.25 GB of RAM...does your computer have this kind of memory?

You really need to figure out what the problem at hand is and how else you can solve it.

If sparse techniques cannot be used, maybe you need to stage your handling by working only on a portion of your matrix at a time and keep the rest on disk....read part of it, process, write it down, load another portion, process, write it down and so on.

RE: Create 250^4 or 250^5 elements-arrays

How do you need to access these values? Are you just reading in the values once and then using them all over the place or will you be writing to them all the time?

RE: Create 250^4 or 250^5 elements-arrays

(OP)
Thanks for the answer.
For xwb, the values are calculated from some other part of the code and need to be stored in this huge arrays. Then, I need to pick the largest element from these 250^4 values simultaneously. Do you think I can reduce the dimension, find the MAXLOC from each dimension, and then MAXLOC again from the 1st MAXLOC ?

Thanks.


RE: Create 250^4 or 250^5 elements-arrays

If all you are going to do is figure out which is the largest value out of all those, there is no need to store them at all...all you need is one variable.

RE: Create 250^4 or 250^5 elements-arrays

(OP)
To salgerman,
Nope. It is the algorithm that must be done. It works for 2*(250^3) and I tried to extend that to 250^4.

Thanks for your answer.

RE: Create 250^4 or 250^5 elements-arrays

(OP)
The algorithm is called the gridsearch method.

RE: Create 250^4 or 250^5 elements-arrays

(OP)
Uh Oh

I have no idea on how to work only a portion. It seems that I cannot avoid declaring things like

REAL, DIMENSION(250,250,250,30) :: A1
REAL, DIMENSION(250,250,250,30) :: A2
...
REAL, DIMENSION(250,250,250,30) :: A8
REAL, DIMENSION(250,250,250,10) :: A9

What is worse is that the arrays I really need to work on is of dimension (260,260,2,260,260,260), which can be declared for (260,260,2,260,3,3) (3 is the highest number allowed)



RE: Create 250^4 or 250^5 elements-arrays

If speed isn't really a problem, there is no difference in syntax between accessing an array and a function when you are reading values

CODE

x = A1(i,j,k,l)
    y = FUNC(i,j,k,l) 
You could get your algorithm working first with an array, then switch to a function, where the function reads in blocks depending on whether the area you are looking at is in scope or not.

RE: Create 250^4 or 250^5 elements-arrays

(OP)
To xwb,

Suppose the working code look like this.

CODE --> FORTRAN

REAL, DIMENSION :: A(250,250,250,250)
A=0.0
        DO i=1,250           
            DO j=1,250       
                DO jj=1,250  
                   DO ii = 1,250
                    x=f(i,ii,j,jj)  !f is some function
                    WHERE (y>0.0)                         
                        y=g(x)      !g is some function
                    ELSEWHERE 
                        y=1e-12
                    END WHERE
                    A(i,ii,j,jj)=y                      
                END DO
            END DO
        END DO
     END DO     
z = MAXOC(A) 

How can I do as you suggested ?

Thank you very much.

RE: Create 250^4 or 250^5 elements-arrays


My previous comment about not needing a huge matrix and simply needing a single variable is this:

CODE

real Vmax
Vmax = -999999.99
        DO i=1,250           
            DO j=1,250       
                DO jj=1,250  
                   DO ii = 1,250
                    x=f(i,ii,j,jj)  !f is some function
                    WHERE (y>0.0)                         
                        y=g(x)      !g is some function
                    ELSEWHERE 
                        y=1e-12
                    END WHERE
                    if (y > Vmax) Vmax = y
                END DO
            END DO
        END DO
     END DO     
write(*,*) 'Maximum value  = ', Vmax 
and that is it.

RE: Create 250^4 or 250^5 elements-arrays

(OP)
To salgerman, xwb

Actually, the actual code is not this simple just to choose the largest element from A. I need part of the A matrix to do some matrix operation, i.e.

CODE --> fortran

REAL, DIMENSION (250,250) :: B

A(i,:,j,jj) + B(j,:) 

(Sorry I did not show you my actual code since it would be too long here.)

So far I still don't have any idea other than declare thing bit by bit.

Someone is telling me that I need to access a super computer in order to do this.

RE: Create 250^4 or 250^5 elements-arrays

Quote (armeros)


I need part of the A matrix to do some matrix operation

If you do matrix operation you are only using 2 indices.
So IMO you really need only 2D-portion of matrix A and therefore should be able to reduce your 4D-task to smaller 2D tasks.
Then you can compute the result X of every 2D-tier in dependence of the previous tier for example something like this:

CODE

! matrix of ij=th tier
real, dimension (:,:), allocatable :: A
! result of the ij-th tier to be used in the next tier
real, DIMENSION (250,250) :: X
...
...
X = 0
do i=1, 256
  do j=1, 256
    ! for every i,j-th tier allocate the matrix A 
    allocate(A(256,256))
    call initialize_matrix(A)
    call compute_result(A, X)    
    deallocate(A)
  end do
end do
...
subroutine initialize_matrix(A)
  ! set initial_values of the matrix A
  ...
end subroutine
 
subroutine compute_result(A, X)
  ! compute result X in dependence of previous X and A
  ...
  X = some_function (X, A)
end subroutine
... 

The result from one 2D-tier to the next tier will be transfered using the matrix X, but maybe you only need a vector or a skalar - it depends of your problem.

RE: Create 250^4 or 250^5 elements-arrays

(OP)

Quote (mikrom)

do i=1, 256
do j=1, 256
! for every i,j-th tier allocate the matrix A
allocate(A(256,256))
call initialize_matrix(A)
call compute_result(A, X)
deallocate(A)
end do
end do

This seems to be a good way to try. I will try to do this. Thank you very much.

RE: Create 250^4 or 250^5 elements-arrays

Looking back on the code I posted, I think that you can omit the usage allocatable.
Maybe, even if you initialize the matrix on the current tier, it would be good to know the matrix of the previous tier.
So then the code simplify to:

CODE

! A= matrix of ij-th tier
! X = result of the ij-th tier to be used in the next tier
real, DIMENSION (250,250) :: A, X
...
...
X = 0
do i=1, 256
  do j=1, 256
    ! for every i,j-th tier use the matrix A 
    call initialize_matrix(A)
    call compute_result(A, X)    
  end do
end do
...
subroutine initialize_matrix(A)
  ! set initial_values of the matrix A
  ...
end subroutine
 
subroutine compute_result(A, X)
  ! compute result X in dependence of previous X and A
  ...
  X = some_function (X, A)
end subroutine
... 

RE: Create 250^4 or 250^5 elements-arrays

(OP)
mikrom, I really appreciate your help. All I need to do is just to fix the indices.

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Tek-Tips Forums free from inappropriate posts.
The Tek-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Tek-Tips forums is a member-only feature.

Click Here to join Tek-Tips and talk with other members!

Resources

Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close