×
INTELLIGENT WORK FORUMS
FOR COMPUTER PROFESSIONALS

Contact US

Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!

*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Students Click Here

Help to reduce the processing time of a program

Help to reduce the processing time of a program

Help to reduce the processing time of a program

(OP)

I have got a program to extract data from 100 files and write it to 1 master file and 3
slave files. Each file in 100 input file contains 12000 rows and 4 columns. Data is to be
extracted for every entry in the first column of master file. The program takes ages to complete as
everytime it has to open a file and read the contents from start.
Can anybody please help to get around this by using better tool..
I am pasting the program...

++++++++++++++++++++++++++++++++++++++++++++++++++
       real tim
       real xmov(101)
       real ymov(101)
       real zmov(101)
       integer majcnt
       integer sidr
       real sfa
       real sfo
       real offa
       real offo
       integer dattyp     
       character carry*1
       integer dof
       integer vad
       integer lcid
       real sf
       integer vid
       integer death
       integer birth
       integer nid
       integer lineadd
       integer nend
       integer nserch
       integer nofil
       integer fstnod(1000)
       integer cnt
       integer tally
       integer node(100000)
       real x(101)
       real y(101)
       real z(101)
       character nodeserch*5
       integer serch
       integer serchmas
       integer serchslv
       character filetoconv*25
       character ansa*1
       character ansb*1   
       character filenam(101)*32
       character dummy*4
       call system("clear")
       call system("rm X_Curves.key Y_Curves.key Z_Curves.key")
       call system("rm Boundary_Cards.key")
       call system("clear")
       print*,'Please ensure *NODE is on the same line number'
       print*,'in all your files !'
       print*,' '
       print*,'Please enter line number of *NODE keyword'
       read*,nserch
       print*,' '
       print*,'Please enter line number of last node'
       read*,nend
       print*,' '
       print*,'Would you like to list .key files prior to file'
       print*,'selection ?'
       read(*,'(a)')ansa
         if(ansa.eq.'y') then
            call system("ls *.key")
         end if
       print*,' '
       print*,'Enter first 25 character filename to be read'
       print*,'(this should read filename00.key)'
       read(*,'(a)')filetoconv
       print*,' '
       print*,'How many key files do you have (1-100)'
       read*,nofil
       print*,' '
*       print*,'Your DYNA files will be Boundary_Cards.key'
*       print*,'and X/Y/Z_Curves.key.  If you have files of'
*       print*,'these names please delete them first!'
*       print*,' '
*       print*,'Enter a key to carry on ..'
*       read*,carry        
*
*
*      establsih file names
*
*
       tally = 1
       do cnt = 0,9
       filenam(tally)=filetoconv//char(cnt+48)//char(48)//'.key'
       tally = tally + 1
       filenam(tally)=filetoconv//char(cnt+48)//char(49)//'.key'
       tally = tally + 1
       filenam(tally)=filetoconv//char(cnt+48)//char(50)//'.key'
       tally = tally + 1
       filenam(tally)=filetoconv//char(cnt+48)//char(51)//'.key'
       tally = tally + 1
       filenam(tally)=filetoconv//char(cnt+48)//char(52)//'.key'
       tally = tally + 1
       filenam(tally)=filetoconv//char(cnt+48)//char(53)//'.key'
       tally = tally + 1
       filenam(tally)=filetoconv//char(cnt+48)//char(54)//'.key'
       tally = tally + 1
       filenam(tally)=filetoconv//char(cnt+48)//char(55)//'.key'
       tally = tally + 1
       filenam(tally)=filetoconv//char(cnt+48)//char(56)//'.key'
       tally = tally + 1
       filenam(tally)=filetoconv//char(cnt+48)//char(57)//'.key'
       tally = tally + 1
       end do
*
*
*     write boundary cards ..
*
*
      vad=2
      sf=1.0
      vid=0
      death=0
      birth=0
      lcid=1
      open(1,status='old',file=filenam(1))
      open(2,status='new',file="Boundary_Cards.key")
      write(2,*)'*KEYWORD'              
        do serchmas = 1,nserch
          read(1,'(a5)')nodesearch
        end do
        do serchslv = 1,nend-nserch
        read(1,880)node(serchslv)
        dof=1        
        write(2,*)'*BOUNDARY_PRESCRIBED_MOTION_NODE'
        write(2,890)node(serchslv),dof,vad,lcid,sf,vid,death,birth
        dof=2
        lcid=lcid+1
        write(2,*)'*BOUNDARY_PRESCRIBED_MOTION_NODE'
        write(2,890)node(serchslv),dof,vad,lcid,sf,vid,death,birth
        dof=3
        lcid=lcid+1
        write(2,*)'*BOUNDARY_PRESCRIBED_MOTION_NODE'
        write(2,890)node(serchslv),dof,vad,lcid,sf,vid,death,birth
        lcid=lcid+1
      end do
      write(2,*)'*END'
      close(1)
      close(2)
*
*     write curve cards ..
*
*
      tim=0.001
      lcid=1
      sidr=0
      sfa=1.0
      sfo=1.0
      offa=0.0
      offo=0.0
      dattyp=0     
      linadd=0
      open(2,status='new',file="X_Curves.key")
      open(3,status='new',file="Y_Curves.key")
      open(4,status='new',file="Z_Curves.key")
      write(2,*)'*KEYWORD'
      write(3,*)'*KEYWORD'
      write(4,*)'*KEYWORD'
      do majcnt = 1,nend-nserch
      do serch = 1,nofil
        open(1,status='old',file=filenam(serch))
        do serchmas = 1,nserch+lineadd
          read(1,'(a5)')nodesearch
        end do
        read(1,900)node(serch),x(serch),y(serch),z(serch)
*        print*,node(serch),x(serch),y(serch),z(serch)
        close(1)
      end do
      do serch = 1,nofil
         if (serch.eq.1) then
           goto 10
         end if
        xmov(serch)=x(serch)-x(serch-1)
        ymov(serch)=y(serch)-y(serch-1)
        zmov(serch)=z(serch)-z(serch-1)
*        print*,xmov(serch)
*        print*,ymov(serch)
*        print*,zmov(serch)
        write(2,910)tim,xmov(serch)
        write(3,910)tim,ymov(serch)
        write(4,910)tim,zmov(serch)
        tim=tim+0.001
        goto 20
*       
10      write(2,*)'*DEFINE_CURVE'
        write(2,920)lcid,sidr,sfa,sfo,offa,offo,dattyp
        write(2,910)0,0
        lcid=lcid+1
        write(3,*)'*DEFINE_CURVE'
        write(3,920)lcid,sidr,sfa,sfo,offa,offo,dattyp
        write(3,910)0,0
        lcid=lcid+1
        write(4,*)'*DEFINE_CURVE'
        write(4,920)lcid,sidr,sfa,sfo,offa,offo,dattyp
        write(4,910)0,0
        lcid=lcid+1        
*
20    end do
      lineadd=lineadd+1
      tim=0.001
      end do
      write(2,*)'*END'
      write(3,*)'*END'
      write(4,*)'*END'
      close(1)
      close(2)
      close(3)
*
*
880    format(i8)
890    format(i10,i10,i10,i10,f10.1,i10,i10,i10)
900    format(i8,f16.6,f16.6,f16.6)
910    format(f20.4,f20.4)
920    format(i10,i10,f10.1,f10.1,f10.1,f10.1,i10)
*
*
       end

RE: Help to reduce the processing time of a program

The only speed improvement I can see is in removing the if statement from the loop.

CODE

10      write(2,*)'*DEFINE_CURVE'
        write(2,920)lcid,sidr,sfa,sfo,offa,offo,dattyp
        write(2,910)0,0
        lcid=lcid+1
        write(3,*)'*DEFINE_CURVE'
        write(3,920)lcid,sidr,sfa,sfo,offa,offo,dattyp
        write(3,910)0,0
        lcid=lcid+1
        write(4,*)'*DEFINE_CURVE'
        write(4,920)lcid,sidr,sfa,sfo,offa,offo,dattyp
        write(4,910)0,0
        lcid=lcid+1        
*
      do serch = 2,nofil
        xmov(serch)=x(serch)-x(serch-1)
        ymov(serch)=y(serch)-y(serch-1)
        zmov(serch)=z(serch)-z(serch-1)
*        print*,xmov(serch)
*        print*,ymov(serch)
*        print*,zmov(serch)
        write(2,910)tim,xmov(serch)
        write(3,910)tim,ymov(serch)
        write(4,910)tim,zmov(serch)
        tim=tim+0.001
20    end do
You will get a minor improvement if nofil is big.  No improvement if nofil is small.  I don't know if rewriting it in another language will improve the speed.  You could also try the various optimization switches on the compiler but for an I/O bound program it may not make very much difference.

RE: Help to reduce the processing time of a program

Since the program is apparently IO bound, One idea.. not for the faint of heart .. is to read the file in much larger 'chunks'.  Suppose something like 1000 bytes at a time, c/r and l/f included (read it as a binary).

Then parse the data strings.. and read more data as needed.  In some, cases the improvement is astounding..in others .. not worth the effort.  

You could try a small test:  part a  read the file record by record; part b read the chunks.  See if ou see any major change.

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Tek-Tips Forums free from inappropriate posts.
The Tek-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Tek-Tips forums is a member-only feature.

Click Here to join Tek-Tips and talk with other members! Already a Member? Login


Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close