Reading 10 MB Flat file into Working storage Variables 1

[Nits1212]

Hi,

I have a input file with 10 MB data in size. I want to read the corresponding 10 MB file into Temporary working storage variables and then i am going to convert that data into MQ message data. Can somebody will tell me, how i can read this 10 MB data in working storage variables? is it possible to do it by using COBOL ARRAY? Please suggest...

Thank you,

Regards,
Nits

 

[Glenn9999]

The first question would probably be why you are thinking of reading a 10MB file completely into working-storage...you probably wouldn't be able to do it (I think the WS limit is 32K), and if you could find a way, you'd get a large number of complaints from the ones that maintain the mainframe system.

You might consider doing it a different way.

 

[Nits1212]

Hi Glenn,

Thank you very much for u r immediate reply. Actually, my requirment is, I have that big input file with me and i have to convert the input file into MQ message data. Because MQ support to carry 100 MB message in size. So to convert file into MQ message data, i am using COBOL program to getting it done...

 

[Nits1212]

Hey Glenn, i forgot to mention one thing that COBOL supports upto 16 MB data size in limit. plz suggest..

 

[Truusvlugindewind]

Consider "USAGE EXTERNAL" in your 01 level declaration. This save a lot of space on your LOADLIB.

10 Mb should not be a problem, just trail-and-error and keep us here informed

 

[Nits1212]

Hi There,

Can you please describe me in detail please... So that i can put the same logic in my COBOL code.

Please suggest,

Regards,
Nits

 

[Truusvlugindewind]

You main something like this?

01  ws-lage-area      usage external.
    03 WS-large-part  pic x(1000) occurs 1000 indexed
    by WS-large-idx.

 

[Nits1212]

Ohh yes, thanks for your input, My input file LRECL length is 2000 variable. and each record length is around 450 characters, So by considering the 2000 variable file length, i can have maximum 4.5 records in one line. SO it will take around 4500 lines to accomodate 10 MB data. Plz correct me if i am wrong

 

[Nits1212]

So as i will be reading the input file line by line the my code will looks like this right?

01 ws-lage-area usage external.
03 WS-large-part pic x(2000) occurs 4500 indexed
by WS-large-idx.

Please suggest.

Where ws-large-idx is going to increment one bye one till 4500 lines maximum.

 

[Truusvlugindewind]

Plz correct me if i am wrong

I'm afraid your mixing things up. You want te read a whole file and store this into memory. Right?
Can you specify the exact properties of the file (=3.4 and place an "i" in front). Can you post the result?
When you have everything in storage, how are you going to process? It is *almost* never required to store the whole dataset first, and then process the data.

 

[Nits1212]

Organization . . . : PS
Record format . . . : VB
Record length . . . : 2000

 

[MarcLodge]

Nits1212,
Each record that you read in will be one record only. Even though you say that they are usually 450 bytes in length, from the variable nature of the file, they can in fact be anything from 1 byte to 2000 bytes each.

You've mentioned that after you have read all the records into storage, you then plan to do something with them regards MQ. Are you able to give more details on exactly what you are planning to do? Is each record going to be treated as a separate record, or do you plan to join one or more of them together in order to make a message for MQ?

Marc

 

[Nits1212]

Dear Marc,

Here is the detail information.

I have a (LRECL = 2000)variable length flat file as a input file. I want to transfer this VB flat file on Unix server by using MQSeries. So i am reading this file as a input.

I am reading each line one by one till the file ends. Maximum possible no. of lines will be 4600.

So i have declared one TEMP Table logic

01 WS-TEMP-TABLE.
03 WS-TABLE-DATA OCCURS 4600 TIMES INDEXED BY WS-INDEX.
05 WS-TABLE-RECORD X(1996).
I am reading each line of the input file one by one and putting the respective record into above temporary table.

after reading all the input file records, i am coverting the corresponding TEMP table records into single MQ message data. Maximum possible input file data can be up to
10 MB.

So i have following questions
1. Is the above defined TEMP Table ARRAY is correct with respect to the INPUT file read?
2. TEMP table will accomodate maximum 10 MB data during the processing? if yes then how?
3. if i have only 2000 records in my input file, then how can i release the remaining empty space from the TEMP table? which commond i will have to use in ARRAY?

Please suggest,
Thanks in advance,
Regards,
Nits

 

[Truusvlugindewind]

So, you want the content of the VB file stored in memory as 1 consequetive area and after that pass that area (max.10Mb of size) to a MQ which will forward that data as a message to unix.

1st: read you file, make sure that each reocrd is stored in a string defined as something like:

01  w-field.
    03 w-length   pic s9(9) binary.
    03 w-string.
       05 filler pic x occurs 1 to 2000 times depending on w-length.

After earch record you read put the length in w-length and store the data in w-string.

2nd: your array. it must be capable of storing 10 Mb of data. The same principle as before:

01  a-field   external.
    03 a-length   pic s9(9) binary.
    03 a-string.
       05 filler pic x occurs 1 to 10000000 times depending on a-length.

3rd: moving the data from the file (using w-string) to the array without trailing spaces/low-values of other trash.

to initialize:

move 1 to a-length.

for each record

string w-string (1:w-length) delimited by size
  into a-string pointer a-length
end-string

I've seen compilers allowing you to omit the (1:w-length) part. Please try and let me know if IBM-COBOL allows that as well.

Well, if you do that for each record you'll end up with the content of you file in "a-string" and the length in "a-length". You can use that to pass to MQ.

Have fun!

 

[Nits1212]

Thank you very much, i really appreciate your attention on my doubt. Thanks a lot once again.

I will be doing the same in reverse way also, i mean converting 10 MB message into flat flat.. so do i need to take some extra care while doing it?
Please suggest...

 

[MarcLodge]

Truusvlugindewind,

I can't see an increment of a-length. Does this field increment automatically? If not, will the 2nd record not overlay the first?

Marc

 

[Truusvlugindewind]

Hai Marc

that's the keyword "POINTER" in the STRING verb. Works like a charm. I saw it in ACUCOBOL programs first. Later I found out that it works on mainframe IBM's cobol as well.

Note the initalization: when you initialize to ZERO (like we're used to) instead of 1 it does nothing.....

 

[Nits1212]

But Still i am facing problem,

STRING WS-INPFILE-REC(1:WS-INPREC-LENGTH)
DELIMITED BY SIZE
INTO WS-DATA-STRING
WITH POINTER WS-DATA-LENGTH
END-STRING
I have move 1 to WS-DATA-LENGTH in my initialisation para.

When i am trying to display WS-DATA-STRING i am getting spaces in SYSOUT.

Please suggest.

 

[Nits1212]

I have defined my INP file like this,

FD INP-FILE
RECORD IS VARYING DEPENDING ON WS-INPREC-LENGTH
BLOCK 0.

01 WS-INPFILE-REC.
03 OCCURS 1 TO 1996 DEPENDING ON WS-INPREC-LENGTH
PIC X(01).

and the second ARRAY like this,

01 WS-TEMP-DATA-BUFFER EXTERNAL.
03 WS-DATA-LENGTH PIC 9(09) BINARY.
03 WS-DATA-STRING.
05 OCCURS 1 TO 1048576 DEPENDING ON WS-DATA-LENGTH
PIC X(01).


and Moving 1 to WS-DATA-LENGTH in initialization PARA.

Still its giving problem.

 

[Nits1212]

I am getting the error as,

A STRING OVERFLOW OCCURED.