www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 9505] New: std.zlib seem to be bugged

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9505

           Summary: std.zlib seem to be bugged
           Product: D
           Version: D2
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P2
         Component: Phobos
        AssignedTo: nobody puremagic.com
        ReportedBy: bioinfornatics gmail.com



01:33:51 PST ---
Dear,
when I try to read  gzip compressed data with a byChunk modified to uncompress
data at second loop uncompress raise an error:
zlib.d(59): data error


code: http://dpaste.1azy.net/0d8f6eac

EXetoC show to me http://d.puremagic.com/issues/show_bug.cgi?id=3191

but this bug is really old is not possible to be this no ?

$ ./zreader file.bam
--> popFront()
--> front()
BAMP�
--> popFront()
--> front()

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 13 2013
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9505




07:15:41 PST ---
i create a little code using etc.c.zlib to be able to use gzipped as a phobos 
range: http://dpaste.dzfl.pl/5b8db0a2

I would like your opinion on this code and if it could replace the old
std.zlib.Uncompress

if yes I could continue to work on to provides uncompress and compress way in D
modern way

you have the power

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 13 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9505




07:55:27 PST ---
to fit more with std.stdio http://dpaste.dzfl.pl/683c053b
I add a:
- ZFile
- rawRead method into ZFile
- byZChunk use ZFile / ZFile.rawRead

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 13 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9505


monarchdodra gmail.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |monarchdodra gmail.com



Both implementations have the fatal flaw of closing the file on first
destruction.

This makes passing a byZChunk (first case) or a (ZFile) second case a dangerous
operation.

Look into the "File" implementation, it should be reference counted, and only
close the file on the last actual destruction.

nitpick:
Once you've named your type ZFile, calling "byChunk" byZChunk is redudant. Just
leave it at byChunk:

auto r1 = File ("SRR077487_1.filt.fastq"   ).byChunk();
auto r2 = ZFile("SRR077487_1.filt.fastq.gz").byChunk();

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 13 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9505




Oh yeah, also, gzread will return -1 in case of an io error. you don't check
for that.

You'd probably want to use:

_numberRead = gzread( _file, _buffer.ptr, cast(uint)_buffer.length );
errnoEnforce(numberRead >= 0);

I can only guess a failed decompress sets an errno? Not sure.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 13 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9505




10:34:22 PST ---
thanks for your much appropriated comment :-)
I will try to enhance this. That was a snippet code to be able to read gz file
in D as std.zlib and  std.zip is unusable, unmaintened and in not in D
philosophy ( use class instead struct, no phobos range and many bug )

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 13 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9505




03:36:31 PST ---
I added some enhancement
http://dpaste.dzfl.pl/683c053b

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 14 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9505


Artem Tarasov <lomereiter gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |lomereiter gmail.com



PST ---
I guess that's because you are using winbits=15 instead of -15. BAM files
contain GZIP blocks with custom headers, so processing them is not so
straightforward.

(I've developed a library for BAM files during last summer -
github.com/lomereiter/biod)



 Dear,
 when I try to read  gzip compressed data with a byChunk modified to uncompress
 data at second loop uncompress raise an error:
 zlib.d(59): data error
 
 
 code: http://dpaste.1azy.net/0d8f6eac
 
 EXetoC show to me http://d.puremagic.com/issues/show_bug.cgi?id=3191
 
 but this bug is really old is not possible to be this no ?
 
 $ ./zreader file.bam
 --> popFront()
 --> front()
 BAMP�
 --> popFront()
 --> front()


-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 14 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9505




09:49:19 PST ---


 I guess that's because you are using winbits=15 instead of -15. BAM files
 contain GZIP blocks with custom headers, so processing them is not so
 straightforward.
 
 (I've developed a library for BAM files during last summer -
 github.com/lomereiter/biod)
 


 Dear,
 when I try to read  gzip compressed data with a byChunk modified to uncompress
 data at second loop uncompress raise an error:
 zlib.d(59): data error
 
 
 code: http://dpaste.1azy.net/0d8f6eac
 
 EXetoC show to me http://d.puremagic.com/issues/show_bug.cgi?id=3191
 
 but this bug is really old is not possible to be this no ?
 
 $ ./zreader file.bam
 --> popFront()
 --> front()
 BAMP�
 --> popFront()
 --> front()




Thanks a lot for your lib i will take a look. why not to create bioinformatic
group for works together ?

the problem show here is not about BGZF block i written a new module for zlib
to fit with actual way to process in D as phobos range

In anycase i think this code should replace the old one and give a review as
zlib is D1 code

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 14 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9505




06:37:25 PST ---
I updated the code http://dpaste.dzfl.pl/683c053b

just add compress uncompress from the old code but fit with d2 way

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 15 2013
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9505




07:25:50 PST ---
update uncompress function
add zlibException

any comments?

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 19 2013