www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - std.zlib bug

reply "Artem Rebrov" <ar_other mail.ru> writes:
I think Compress and UnCompress classes should be used as in following  
example
---------------------------------
import std.file;
import std.zlib;

void main()
{
	Compress c = new Compress;
	UnCompress u = new UnCompress;
	void[] q,w,e,wa,et; int i;

	q = read("in.dat");

	for(i=0;i+512<q.length;i+=512)
		{
		w=c.compress(q[i..i+512]);
		wa~=w; et=""; et~=w;

		w=c.flush(Z_SYNC_FLUSH);
		wa~=w; et~=w;

		e~=u.uncompress(et);
		}
	
	w=c.compress(q[i..q.length]);
	wa~=w; et=""; et~=w;
     	
	w=c.flush();
	wa~=w; et~=w;
	e~=u.uncompress(et);
	e~=u.flush();

	write("out_c.dat",wa);		
	write("out_u.dat",e);
	e = uncompress(wa);
	write("out_ux.dat",e);	

}
---------------------------------
But it does not work. Look at the .diff. With these changes it works  
properly. (dmd v0.133 win32)
---------------------------------
--- zlib.d	Sat Sep 24 17:25:30 2005
+++ zlib_new.d	Tue Oct  4 01:28:34 2005
   -287,8 +287,9   
  	{   delete destbuf;
  	    error(err);
  	}
-	destbuf.length = zs.total_out;
-	return destbuf;
+	//destbuf.length = zs.total_out;
+	destbuf.length = destbuf.length - zs.avail_out;
+    return destbuf;
      }

      void[] flush()
   -303,16 +304,22   
      }
      body
      {
-	void[] destbuf;
+	void[] destbuf, tmpbuf;
  	int err;

  	if (!inited)
  	    return null;

+	/*  //old code
  	destbuf = new void[zs.avail_in];
  	zs.next_out = cast(ubyte*) destbuf;
  	zs.avail_out = destbuf.length;
+	*/
+    tmpbuf = new void[zs.avail_out]; // may be  zs.avail_out+<some  
constnt>
+	zs.next_out = cast(ubyte*) tmpbuf;
+	zs.avail_out = tmpbuf.length;

+	/* //old code
  	err = deflate(&zs, mode);
  	if (err != Z_STREAM_END)
  	{
   -322,6 +329,27   
  	    error(err);
  	}
  	destbuf = cast(void[])((cast(ubyte *)destbuf)[0 .. zs.next_out -  
cast(ubyte*)destbuf]);
+	*/
+	while( (err = deflate(&zs, mode)) != Z_STREAM_END)
+		{
+		if(err == Z_OK)
+			{
+			if(zs.avail_out != 0 && mode != Z_FINISH)
+				break;
+			else if(zs.avail_out == 0)
+				{
+				destbuf ~= tmpbuf;
+				zs.next_out = cast(ubyte*) tmpbuf;
+				zs.avail_out = tmpbuf.length;
+				continue;
+				}
+			err = Z_BUF_ERROR;
+			}
+		delete destbuf;
+		error(err);
+		}
+	destbuf ~= tmpbuf[0..(tmpbuf.length-zs.avail_out)];
+	
  	if (mode == Z_FINISH)
  	{
  	    err = deflateEnd(&zs);
   -410,7 +438,8   
  	{   delete destbuf;
  	    error(err);
  	}
-	destbuf.length = zs.total_out;
+	//destbuf.length = zs.total_out;
+	destbuf.length = destbuf.length - zs.avail_out;
  	return destbuf;
      }


-- 
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
Oct 03 2005
parent reply Derek Parnell <derek psych.ward> writes:
On Tue, 04 Oct 2005 02:34:51 +0400, Artem Rebrov wrote:


 I think Compress and UnCompress classes should be used ...


Why are they even classes? Are not class meant to be representation of
things rather than activities? What is a Compress? What is an UnCompress?
These are processes, actions, work, and not objects. The closest thing
would be that 'Compress' is a Storage mechanism for compressed data and
that it can do (at least) two activities: compress data into itself and
decompress data that it already has.

-- 
Derek
(skype: derek.j.parnell)
Melbourne, Australia
4/10/2005 9:10:14 AM
Oct 03 2005
parent reply "Artem Rebrov" <ar_other mail.ru> writes:
On Tue, 04 Oct 2005 03:14:46 +0400, Derek Parnell <derek psych.ward> wrote:


 On Tue, 04 Oct 2005 02:34:51 +0400, Artem Rebrov wrote:


 I think Compress and UnCompress classes should be used ...


 Why are they even classes? Are not class meant to be representation of
 things rather than activities? What is a Compress? What is an UnCompress?
 These are processes, actions, work, and not objects. The closest thing
 would be that 'Compress' is a Storage mechanism for compressed data and
 that it can do (at least) two activities: compress data into itself and
 decompress data that it already has.


Just look at dmd\src\phobos\std\zlib.d in the DMD distribution


-- 
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
Oct 04 2005
parent reply "Artem Rebrov" <ar_other mail.ru> writes:
On Tue, 04 Oct 2005 22:01:36 +0400, Artem Rebrov <ar_other mail.ru> wrote:


 On Tue, 04 Oct 2005 03:14:46 +0400, Derek Parnell <derek psych.ward>  
 wrote:


 On Tue, 04 Oct 2005 02:34:51 +0400, Artem Rebrov wrote:


 I think Compress and UnCompress classes should be used ...


 Why are they even classes? Are not class meant to be representation of
 things rather than activities? What is a Compress? What is an  
 UnCompress?
 These are processes, actions, work, and not objects. The closest thing
 would be that 'Compress' is a Storage mechanism for compressed data and
 that it can do (at least) two activities: compress data into itself and
 decompress data that it already has.


 Just look at dmd\src\phobos\std\zlib.d in the DMD distribution


(Un)Compress classes holds state of (un)compressor.
When I can't compress whole data I split it to chunks. I transmit these
chunks to other side of the connection (for example) and uncompress
them one by one. The goal is to be able to decompress the first chunk
before I can receive the last one.



-- 
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
Oct 04 2005
parent Sean Kelly <sean f4.ca> writes:
In article <op.sx4xxhtpncj208 comp>, Artem Rebrov says...

(Un)Compress classes holds state of (un)compressor.
When I can't compress whole data I split it to chunks. I transmit these
chunks to other side of the connection (for example) and uncompress
them one by one. The goal is to be able to decompress the first chunk
before I can receive the last one.


For what it's worth, I typically implement this functionality as a stream filter
in C++.  It would probably be worth doing something similar in D.


Sean
Oct 04 2005