INTELLIGENT WORK FORUMS
FOR COMPUTER PROFESSIONALS

Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!

*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Jobs

Puzzling first char in streamed file

Puzzling first char in streamed file

(OP)
I am implementing streaming data straight (after ASCII conversion) to file.
But I find when checking the file with notepad that there is an inexplicable first character in every line.
It seems to be related to the number of characters in each CR/LF terminated line.

there are 844 line in the file:-

If the line is 111 chars in length (including the CR & LF Chars) then there is a lower case 'o' in the first position
If the Line is 112 chars its a 'p'
113 a 'q' and the final line is 114 chars and starts with 'r'

CODE --> Delphi

Temp := ShortString(TempString)+#13#10;
Fs.writebuffer(Temp, length(Tempstring) + 2); 


Where Temp is a short string of max 120 chars, I have found that normal strings cannot be streamed successfully
Tempstring is a normal string.
Fs is a TFilestream object.
Temp looks to be as expected at the point of writing.

Steve: N.M.N.F.
If something is popular, it must be wrong: Mark Twain
That's just perfectly normal Paranoia everyone in the universe has that: Slartibartfast

RE: Puzzling first char in streamed file

In a short string, the leading byte (i.e. Temp[0]) is a length byte, denoting the number of characters in the string. This is why it cannot be more than 254 chars long. In other words, length(Temp) = Temp[0]; see the following from Delphi's online help: Delphi Short String

RE: Puzzling first char in streamed file

I meant to type 255 characters, not 254.

RE: Puzzling first char in streamed file

(OP)
Cheers for that a simple answer, thought there might be smile

Steve: N.M.N.F.
If something is popular, it must be wrong: Mark Twain
That's just perfectly normal Paranoia everyone in the universe has that: Slartibartfast

RE: Puzzling first char in streamed file

Quote:

I have found that normal strings cannot be streamed successfully

That is not true, you can stream a string, and read it back.

To write a string to a stream, you first write the length of the string to the stream, then you load the string into the stream, reading is just the opposite, first read how long the string is, use setlength to set the length of the string you will read it into, and then read that many characters.

First assuming you have a stream created

CODE

var
   MS: TMemoryStream;
begin
   MS := TMemoryStream.Create
   ... 

then you can write a string to it like so

CODE

var
   MyString1, MyString2: String;
   LengthOfString: Integer;
begin
   MyString1 := 'xxxxx';
   MyString2 := 'yyyyy';
   LengthOfString := LengthOf(MyString1);//get the length of the string
   MS.WriteBuffer(LengthOfString, SizeOf(LengthOfString);//write the length of the string to the stream
   if LengthOfString > 0 then
      MS.WriteBuffer(Pointer(MyString1)^, ByteLength(MyString1));//now write the actual string
   //repeat for MyString2
   LengthOfString := LengthOf(MyString2);
   MS.WriteBuffer(LengthOfString, SizeOf(LengthOfString);
   if LengthOfString > 0 then
      MS.WriteBuffer(Pointer(MyString2)^, ByteLength(MyString2));//use bytelength rather than length works for both pre/post unicode Delphi
end; 
now to read it back, you need to know the structure of the items in your stream (MyString1 comes before MyString2)...

CODE

var
   LengthOfString: Integer;
   MyString1, MyString2: String;
begin
   MS.ReadBuffer(LengthOfString, SizeOf(LengthOfString));//get the length of the following string
   SetLength(MyString1, LengthOfString); //set the length of the string you are reading into
   if LengthOfString > 0 then
      MS.ReadBuffer(Pointer(MyString1)^, ByteLength(MyString1);//read correct No. of bytes into the string

   //repeat for mystring2

   MS.ReadBuffer(LengthOfString, SizeOf(LengthOfString));//get the length of the following string
   SetLength(MyString2, LengthOfString);
   if LengthOfString > 0 then
      MS.ReadBuffer(Pointer(MyString2)^, ByteLength(MyString2);
end; 

RE: Puzzling first char in streamed file

2 things:

@sggaunt: why on earth would one use Shortstring?
@majlumbo: When working with strings, TStringStream seems more appropriate :)

/Daddy

-----------------------------------------------------
Helping people is my job...

RE: Puzzling first char in streamed file

@whosrdaddy While I agree in general, using a StringStream is not appropriate if you need to save multiple strings, or multiple data types to the same stream. The method I showed allows any type of data to be saved to the same stream, (although non-string types you would not need to precede with the byte length to read/write the data). The only downside is that you need to know the sequence of data items added, so you can read them in the same order as they were written in.

RE: Puzzling first char in streamed file

The shortstring is the implementation of the original Pascal string type; the OP may be doing something that he wants to be compatible with other Pascal compilers. I generally don't ask why somebody is doing something a certain way unless it is obviously a wrong or dangerous way of doing it, I just assume they have their reasons to be doing something that way.

RE: Puzzling first char in streamed file

(OP)
In this case, I need something quick and dirty. (quick in terms of development and speed of operation).
I Have library functions similar to the ones suggested by majlumbo, but they didn't work and I didn't have the time to mess about with them.
Pressure is off to some extent now, so I will be looking at above stuff.



Steve: N.M.N.F.
If something is popular, it must be wrong: Mark Twain
That's just perfectly normal Paranoia everyone in the universe has that: Slartibartfast

RE: Puzzling first char in streamed file

In that case, TStringStream and TSTringList are both good candidates, and they have the advantage of using the default String type which is Unicode in current versions going back to 2009. TStringList has streaming and file functions built in; you just call TStringlist.LoadFromStream and SaveToStream for streaming, or LoadFromFile and SaveToFile for files. Same goes for TStringStream. The advantage of TStringList is that you can retrieve individual lines using it's Strings property.

RE: Puzzling first char in streamed file

(OP)
Interesting stuff this.
Now the way I understand the concept of streaming, is that when outputting say video the data is pulled out of a buffer as the user watches it, and as soon as the data has been 'consumed' it can be discarded. If the user 'winds back' before the relatively small saved buffer, then the data must be re-streamed (For simple file download and save we don't need to be able to do that).
And the reason why we might want to stream is that we don't have to wait for the entire file to be downloaded, and we don't have to hold the entire file in RAM.

In fact I use the StringList.SaveToFile method in the existing program, incoming data is stored in a stringlist.
When download is complete (up to 16Mb) and the user needs to save this data, Save to file is called.

I don't understood the reason for the Stringlist.SaveToStream Method, as (unless I misunderstand this) it does not allow streaming out at the same time as the list is being populated, so its not true streaming?
To quote the help SaveToStream does the same thing as SaveToFile except the application must create and destroy the File Stream.


Steve: N.M.N.F.
If something is popular, it must be wrong: Mark Twain
That's just perfectly normal Paranoia everyone in the universe has that: Slartibartfast

RE: Puzzling first char in streamed file

@sggaunt If you have a 25 Gig text file (yes I've actually seen them that big), you won't be able to open such a file in a TStringList. As far as I know, there's no option to read just a portion of a file using a StringList. For this type of case, you would use a TFileStream.

Buffering is a completely different matter. Natively, Delphi Streams do not buffer (other that what the OS may provide). If you want a buffered version of a FileStream, the you can look at using the code here.

RE: Puzzling first char in streamed file

A stream in Delphi and C++ is just a more generic mechanism for transferring information. In Object oriented languages like Delphi and C++, streams allow you to send and receive information from a device, whether it be a keyboard, Display device, File system or IP Socket without having to worry about how to go about sending/receiving it. You just call the Write procedure for the Stream, and no matter what kind of stream it is, the data is sent. It is NOT the same concept as streaming a video from the internet, which is a mechanism where you are using the data received earlier while later data is still being downloaded, as opposed to downloading the entire video before viewing it.

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Tek-Tips Forums free from inappropriate posts.
The Tek-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Tek-Tips forums is a member-only feature.

Click Here to join Tek-Tips and talk with other members!

Resources

Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close