www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - std.zlib uncompress issue

reply "Robik" <foobar123 gmail.com> writes:
Hello.

Recently I was working with std.zlib and experienced error. After 
few hours I realized it was uncompress issue. I've made simple 
example that shows the issue:

import std.stdio,
        std.zlib,
        std.array,
        std.file,
        std.conv;

void main()
{
     createFile();
     loadFile();
}


void createFile()
{
     auto file = new File("test.txt", "wb");
     scope(exit) file.close;
     auto cpr = new Compress(9);
     string str = "abcdef";

     for(int i = 0; i < 500; i++)
     {
         file.write(cast(string)cpr.compress(str~to!string(i)));
     }
     file.write(cast(string)cpr.flush);
}

void loadFile()
{
     auto file = new File("test.txt", "rb");
     scope(exit) file.close;

     auto target = new File("test2.txt", "wb");
     scope(exit) target.close;

     auto ucpr = new UnCompress();

     // Making chunk size bigger than file size will (dirty)fix it.
     foreach(chunk; file.byChunk(64))
     {
         target.write(cast(string)ucpr.uncompress(chunk));
     }
     target.write(cast(string)ucpr.flush);
}

The code creates file with compressed data, then tries to 
uncompress it. That's where the issue is. 
UnCompress.uncompress(src) works only if the src is complete. If 
I pass half of it, I get exception (data error). I don't know how 
to fix it except increasing buffer size to incredible high 
values(some files may be pretty big).
Mar 04 2012
parent reply "Eko Wahyudin" <eko.wahyudin yahoo.co.id> writes:
I have a same problem too..
I think the problem came from Compress class.

I'm creating web server with gzip and deflate support. browser 
(Chrome, Firefox and opera) able to deCompress it. The problem 
begin if file size more than 64KB. The output size is equal to 
the original file, but the content is different at some block.

for instance compressing "jquery-2.1.0.min.js". the result is 
different begining at file offset 73.728 or 0x12000. with file 
buffer 4096 bytes.

well this thread start at March 04, 2012 and now Apr 22, 2014,
it's already two year,,, this problem still doesn't solved :(
Apr 21 2014
parent "Rikki Cattermole" <alphaglosined gmail.com> writes:
On Tuesday, 22 April 2014 at 05:22:03 UTC, Eko Wahyudin wrote:

 I have a same problem too..
 I think the problem came from Compress class.

 I'm creating web server with gzip and deflate support. browser 
 (Chrome, Firefox and opera) able to deCompress it. The problem 
 begin if file size more than 64KB. The output size is equal to 
 the original file, but the content is different at some block.

 for instance compressing "jquery-2.1.0.min.js". the result is 
 different begining at file offset 73.728 or 0x12000. with file 
 buffer 4096 bytes.

 well this thread start at March 04, 2012 and now Apr 22, 2014,
 it's already two year,,, this problem still doesn't solved :(


On a related note I've only recently ported a javascript minifier 
to D as part of Cmsed [0].
It appears to work correctly, but you never know considering it 
was php originally what issues relating to types I mucked up upon.

[0] 
https://github.com/rikkimax/Cmsed/blob/master/source/minifier/cmsed/minifier/jsmin.d
Apr 21 2014