I have already described working with files in KOL, at a low level. The set of functions for working with files does not require the use of objects. Working with data stream objects provides a higher level both for working with files and with any data sets, for example, in memory. Without the use of objects, it would be quite difficult to provide an acceptable level of encapsulation for this functionality, so the TStream object type is introduced in KOL, in much the same way as in the VCL. Just like in the VCL, it has methods for reading (Read) and writing (Write) data, to change the current position in the stream (Seek).
But that's where the similarities end right there. Instead of inheriting the required data stream classes from the base TStream class, KOL uses a mechanism of function pointers. In the "constructors" of instances of data streams (that is, in the NewXXXXXStream functions) these pointers are assigned certain sets of functions, as a result objects of the same object type TStream are obtained (constructors, of course, return pointers of created streams, of type PStream) , but these objects provide different functionality based on which constructor is called.
So, the following "constructors" of data streams are defined in the KOL module itself:
NewReadFileStream(s) - creates a stream for reading a file (an existing file is opened in "read-only" mode);
NewWriteFileStream(s) - creates a stream for writing a file (a new file is created, or, if it already exists, the file is opened for writing);
NewReadWriteFileStream(s) - a stream is created for writing and reading a file;
NewFileStream(s, options) - allows you to create a file stream with a more detailed listing of the opening and creation modes (these are the same options that are used in the FileCreate function);
NewMemoryStream - creates a stream in memory (for writing and reading);
NewExMemoryStream(P, n)- also creates a thread in memory, but this time in existing memory. If in the previous "constructor" a stream was created that initially does not contain data and grows as it is written to it by methods like Write, then this function creates a stream on an existing contiguous piece of memory (with address P and length n bytes), and the size of this stream does not may change while working with a stream. This memory is not considered to be "owned" by such a thread, and when the data stream object is destroyed, it is not freed in any way (to free it, if, for example, it was dynamically allocated, the code or object that allocated it should).
The benefits of creating this kind of flow are obvious. Let's say you already have some structured data in memory, and there is a method that can read this data from the stream. Instead of creating a regular stream in memory (NewMemoryStream), writing this data to it, and then reading it, we simply create a stream in existing memory (NewExMemoryStream), and immediately read the data using the available method. At the same time, at least the allocation of memory for a new stream and copying of this data are saved, which in the case of a large data size also has a very positive effect on the performance of the application.
NewMemBlkStream(blksize) and NewMemBlkStream_WriteOnly (blksize) - These two constructors allow you to create a stream of data in memory, but continuity is guaranteed only for the chunk of data written by a single call to the Write method. What is important is that it is guaranteed that the recorded data is not transferable in the future. This type of data flow is very convenient to use to improve the efficiency of the memory manager, providing a single large block of data allocation at once. That is, memory is allocated less frequently, but in large portions (and subsequently freed up faster). Usually, it makes sense to use this kind of stream in write-only mode, getting the address of the next written memory block through the fJustWrittenBlockAddress field. In KOL itself, such a stream is used by the TDirList object type to improve performance.
NewExFileStream(hFile) - similar to the previous one, creates a stream for reading or writing to a file, but based on the existing descriptor of an already open file. Note that the descriptor can also refer to an object of type pipe (pipe), and there is no other way to create a stream for working with a pipe.
In addition to this set of "constructors" of streams, it is possible to create your own types of data streams based on TStream. (For example, the DIUCL package defines the stream constructors NewUCLCStream and NewUCLDStream, which compress and decompress data in a way that works with streams.)
The set of methods on the TStream object in KOL provides everything you need to read and write data. When working with KOL data streams, unlike VCL, you need to remember that all methods and properties, including those that are not typical for this type of data flow, remain open for use. But, for example, it makes no sense to try to write to a read-only file stream, or the Handle property has no meaning for the in-memory data stream (Handle provides access to the file descriptor, but only matters for file streams). In VCL, additional control is provided by the compiler at the stage of writing the code; in KOL, you need a little more care, but it achieves a more compact size of the application, with the same functionality.
Here is a list of the main methods and properties of TStream:
Read(buf, n) - reads a maximum of n bytes from the current position in the stream into the buffer, returns the number of bytes read (it may be less if the end of the data has been reached);
Write(buf, n) - writes n bytes from the buffer in memory to the stream;
Seek(n, method) - moves a position in the stream, returns a new position;
Position - current position in the stream;
Size - stream size (for some stream types the stream size may not be known);
Memory - a pointer to the memory in which the stream data is located in memory (for other types of streams it is always nil);
Capacity - memory reserve for streams in memory (as well as for TList, it can be changed externally in order to optimize the speed of memory allocation);
Handle - file stream descriptor (you can analyze it for the inequality of the constant INVALID_HANDLE_VALUE immediately after opening to make sure that the connection with the file is established normally, for example, or use other low-level functions for working with files that allow an open file descriptor as a parameter, but with a certain caution);
SaveToFile(s) - saves all the contents of the stream to a file named s.
This set is extended with additional methods for working with strings in a stream:
WriteStr(s) - writes the specified string to the stream (neither the terminating byte with the code # 0, nor the length of the string is written, it is assumed that the "reader" of the stream will subsequently know this length: either it is written in a different way to the same stream, or it is constant, or is calculated somehow);
WriteStrZ(s) - writes a string and a terminating null byte to the stream;
ReadStrZ - reads a null-terminated string from the stream;
ReadStr - reads a string from the stream, ending with one of the combination of characters: # 0, # 13 # 10, # 13, # 10;
ReadStrLen(n) - reads a string of length n bytes from the stream;
WriteStrEx(s) - writes to the stream first the length of the string (4 bytes), and then the string itself - without the terminating null byte;
ReadStrEx - reads from the stream first the length of the string, then the string itself (the inverse of the previous write function);
ReadStrExVar(s) - the same as the previous method, but reads a string into the s parameter, and returns the number of bytes read;
WriteStrPas(s) - writes a short string (such strings up to 255 bytes in length were used in the first versions of the Pascal language, if you remember, the size of such a string is stored in the 0th byte of the string), while the length of the string is written first (1 byte);
ReadStrPas - reads a Pascal string (first read the byte storing the length of the Pascal string, from 0 to 255, then the string itself).
And one more set of methods is used to work with threads in asynchronous mode, when the program, having issued a request for a read or write operation, can continue without stopping to wait for the operation to complete, and then, when the result of the operation is already definitely needed by the program, the Wait method is called to completion of the current operation:
SeekAsync(n, method) - the same as Seek, but asynchronously;
ReadAsync(buf, n) - the same as Read (the essential difference is that, since the operation has just begun, but not yet completed, this procedure cannot return the number of bytes read, therefore it is framed as a procedure);
WriteAsync(buf, n) - the same as Write, but asynchronously;
Busy - returns true if the thread has not yet completed the operation;
Wait - permanently waiting for the completion of the last asynchronous operation.
Quite often it is required to transfer a portion of data from one data stream to another, for this there are global functions:
Stream2Stream(dst, src, n)- reads n bytes from the src stream (source - source) and writes them to the dst stream. In the case when one of the streams (or both) is a stream in memory, it performs optimization and does not create an intermediate buffer up to n bytes in size, but uses the memory in the stream in memory as a buffer;
Stream2StreamEx(dst, src, n) - the same as above, but does not optimize for streams in memory, but easily copes with very large data streams (since it sends data in portions through a 64 Kbyte buffer);
Stream2StreamExBufSz(dst, src, n, bufsz)- the same as the previous function, but allows you to set your own size of the intermediate buffer for data transfer. It is likely that allocating a 1 MB buffer will significantly speed up the transfer of large amounts of data, but at the same time allocating an even larger portion of memory for the buffer can only reduce performance if there is not enough memory in the system.
In the case when the resources in the application contain some kind of data that is easy to read through a stream, the following global function comes in handy:
Resource2Stream(dst, inst, s, restype) - allows you to read a resource of any restype type into the stream (not only from the application module, but also from any executable file for which the inst descriptor is obtained).
Among other things, the TStream type has Methods and Data properties for developers of new flavors of data streams. To create a new kind of data stream, you need to define your own "constructor", and in this constructor you specify your set of methods (using the Methods property) for reading, writing and changing the position in the stream. These methods can use the Data structure to place their service data (the usual set should be enough, but, as a last resort, it is always possible to allocate an additional block of memory and use one of the fields of this structure to refer to its structure).
