www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - JPG and PNG decoder

reply "cal" <callumenator gmail.com> writes:
I've been working on decoders for simple (baseline) JPEG and 
PNG's, mostly for my own amusement, and they seem to work ok now, 
so if anyone has need of some simple D modules to load these 
formats you can grab them here:
https://github.com/callumenator/imaged

Notes:
- Will load 8-bit baseline sequential JPEG's only. All PNG image 
types are supported (only critical chunks are decoded however).
- Can load from a stream or a file. In the case of the PNG's, 
this allows progressive display with an interlaced image.
- I haven't put much effort into speeding up these routines, and 
have not tested the JPEG decoder extensively, so there are bound 
to be bugs and there is plenty room for improvement.
- The example shows stream usage, and uses Adam Ruppe's 
simpledisplay to make a little viewer, that can flick through all 
images in a directory.

Cheers,
cal
Jun 16 2012
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/16/2012 12:10 PM, cal wrote:

 I've been working on decoders for simple (baseline) JPEG and PNG's, mostly for
 my own amusement, and they seem to work ok now, so if anyone has need of some
 simple D modules to load these formats you can grab them here:
 https://github.com/callumenator/imaged


Thank you!
Jun 16 2012
next sibling parent Philippe Sigaud <philippe.sigaud gmail.com> writes:
 On 6/16/2012 12:10 PM, cal wrote:

 I've been working on decoders for simple (baseline) JPEG and PNG's, mostly
 for
 my own amusement, and they seem to work ok now, so if anyone has need of
 some
 simple D modules to load these formats you can grab them here:
 https://github.com/callumenator/imaged




Nice one, thanks.

Is there a way to write an Image to disk? I didn't see one while
skimming the source.
Jun 16 2012
prev sibling parent reply Philippe Sigaud <philippe.sigaud gmail.com> writes:
On Sun, Jun 17, 2012 at 8:15 AM, Philippe Sigaud
<philippe.sigaud gmail.com> wrote:


 I've been working on decoders for simple (baseline) JPEG and PNG's, mostly
so if anyone has need of some simple D modules to load these formats you can
grab them here:





 Is there a way to write an Image to disk? I didn't see one while
 skimming the source.


Oops, you*did* say these where for loading. Sorry for the noise :)

Still, I'm interested in writing a JPEG/PNG to disk from a
ubyte[3][][], or whatever.
Jun 17 2012
parent reply "cal" <callumenator gmail.com> writes:
On Sunday, 17 June 2012 at 07:07:35 UTC, Philippe Sigaud wrote:

 Still, I'm interested in writing a JPEG/PNG to disk from a
 ubyte[3][][], or whatever.


Do you mean that you want to encode a ubyte array to disk as 
JPEG/PNG? Encoding a JPEG would be a bit of work I think, the 
format's kind of a monster. PNG should be easier, depending on 
how good you want the compression to be. If you don't care too 
much about compression level, you simply zlib compress the data, 
write it out by image row/scanline, include appropriate header 
and chunk info, and you're done. I'll give a simple encoder a go 
if you think you could use it.

Cheers,
cal
Jun 17 2012
next sibling parent "Chris NS" <ibisbasenji gmail.com> writes:
I second the request for a PNG encoder.  You just saved me a lot 
of trouble, as I was about to implement my own PNG loader... and 
now I don't have to.  ^_^  I'll glance over your code in full 
sometime and see if I notice any readily apparent improvements.
Jun 17 2012
prev sibling next sibling parent Philippe Sigaud <philippe.sigaud gmail.com> writes:
On Sun, Jun 17, 2012 at 9:55 AM, cal <callumenator gmail.com> wrote:

 On Sunday, 17 June 2012 at 07:07:35 UTC, Philippe Sigaud wrote:

 Still, I'm interested in writing a JPEG/PNG to disk from a
 ubyte[3][][], or whatever.



 Do you mean that you want to encode a ubyte array to disk as JPEG/PNG?
 Encoding a JPEG would be a bit of work I think, the format's kind of a
 monster. PNG should be easier, depending on how good you want the
 compression to be. If you don't care too much about compression level, you
 simply zlib compress the data, write it out by image row/scanline, include
 appropriate header and chunk info, and you're done. I'll give a simple
 encoder a go if you think you could use it.


Years ago, one of the first programs I did in D was a raytracer. It
was a good test for OOP, structs, 3D vectors, foreach loops (which
were all the rage 4-5 years ago, even before ranges). It was nice and
fun, but I then discovered that, at the time, I had no easy way to
write my pixel 2D arrays to disk. I had to code a small BMP writer.
Nowdays, I'm playing with the idea to write a new tutorial for D,
using a raytracer and new D technics like CTFE / templates /
std.algorithms. This time, I would like to save the images as JPEG /
PNG.

I guess it's possible to wrap some C lib to do the work, but it's be
nice to have a D module to do that.

Anyway, nice work with the loader!
Jun 17 2012
prev sibling next sibling parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
On 17/06/2012 08:55, cal wrote:

<snip>

 If you don't care too much about compression level, you simply zlib
 compress the data, write it out by image row/scanline, include
 appropriate header and chunk info, and you're done.


<snip>

Not quite.  You need to encode it by scanline, add the filter byte at the
beginning of 
each line, _then_ zlib compress it.  The filter byte can just be 0 throughout
if you don't 
want to worry about filtering.

If you just want a basic truecolour or greyscale PNG encoder, then it's a
matter of:
- write the PNG signature
- write the IHDR, being careful about byte order
- write the IDAT - zlib compressed image data
- write the IEND

You also need to compute the CRC of each chunk, but std.zlib has a function to
do that as 
well.

FWIW a while ago I wrote a simple experimental program that generates an image
and encodes 
it as a PNG.  And I've just tweaked it and updated it to D2 (attached).  It
supports only 
truecolour with 8 bits per sample, but it supports filtering, though it isn't
adaptive 
(you just specify the filter to use when you run the program).

Stewart.
Jun 17 2012
parent reply Philippe Sigaud <philippe.sigaud gmail.com> writes:
On Sun, Jun 17, 2012 at 3:25 PM, Stewart Gordon <smjg_1998 yahoo.com> wrote=
:

 FWIW a while ago I wrote a simple experimental program that generates an
 image and encodes it as a PNG. =C2=A0And I've just tweaked it and updated=


 it to

 D2 (attached). =C2=A0It supports only truecolour with 8 bits per sample, =


but it

 supports filtering, though it isn't adaptive (you just specify the filter=


 to

 use when you run the program).


Nice one, works OK here! (DMD 2.059, Linux 32bits)

(DMD -property asks for ' property' after IHDR bigEndian())

You two should fuse it into one module, to give us loading/writing.
That would be quite cool.
Jun 17 2012
parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
On 17/06/2012 14:38, Philippe Sigaud wrote:

 On Sun, Jun 17, 2012 at 3:25 PM, Stewart Gordon <smjg_1998 yahoo.com> wrote:

 FWIW a while ago I wrote a simple experimental program that generates an
 image and encodes it as a PNG.  And I've just tweaked it and updated it to
 D2 (attached).  It supports only truecolour with 8 bits per sample, but it
 supports filtering, though it isn't adaptive (you just specify the filter to
 use when you run the program).


 Nice one, works OK here! (DMD 2.059, Linux 32bits)

 (DMD -property asks for ' property' after IHDR bigEndian())


Good catch.  It's probably time we deprecated using property syntax on
non-properties.


 You two should fuse it into one module, to give us loading/writing.
 That would be quite cool.


I don't know how much of the code will need rewriting in order to support
interlacing or 
all PNG colour types/depths.

But I also began writing a PNG library a while back - with the intention that
it will be a 
full PNG editing library, not just encoding/decoding of the image data.  But I
haven't had 
much time to work on it recently.  And it would help a lot if we could have the 
replacement for std.stream some time soon.

Stewart.
Jun 17 2012
parent reply "cal" <callumenator gmail.com> writes:
On Sunday, 17 June 2012 at 23:17:54 UTC, Stewart Gordon wrote:

 You two should fuse it into one module, to give us 
 loading/writing.
 That would be quite cool.


 I don't know how much of the code will need rewriting in order 
 to support interlacing or all PNG colour types/depths.

 But I also began writing a PNG library a while back - with the 
 intention that it will be a full PNG editing library, not just 
 encoding/decoding of the image data.  But I haven't had much 
 time to work on it recently.  And it would help a lot if we 
 could have the replacement for std.stream some time soon.

 Stewart.


I just pushed an update which implements a PNG encoder. It is 
implemented in the Image class as a write method. If you have a 
raw buffer, and want to encode it, you can do something like this:

ubyte[] data = some data;
Image img = new Img!(Px.R8G8B8)(width, height, data);
img.write("mypng.png");

It uses adaptive filtering, and should work with the pixel 
formats supported by the image class (except for the 16 bit ones 
I've just realised). I've only tested it on images that I have 
previously loaded in however.

Stewart, I used your makepng.d as a template for this, can I add 
you to the author list?

Also, if preferred, I can keep the master branch as a single 
merged module, just let me know.

Cheers,
cal
Jun 17 2012
parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
On 18/06/2012 00:49, cal wrote:
<snip>

 ubyte[] data = some data;
 Image img = new Img!(Px.R8G8B8)(width, height, data);


Image?  Img?


 img.write("mypng.png");

 It uses adaptive filtering, and should work with the pixel formats supported
by the image
 class (except for the 16 bit ones I've just realised). I've only tested it on
images that
 I have previously loaded in however.


Does it always take in a ubyte[], or does that depend on the bit depth?


 Stewart, I used your makepng.d as a template for this, can I add you to the
author list?


<snip>

By all means.

I look forward to seeing what you've come up with.

Stewart.
Jun 21 2012
parent "cal" <callumenator gmail.com> writes:
On Thursday, 21 June 2012 at 17:58:47 UTC, Stewart Gordon wrote:

 On 18/06/2012 00:49, cal wrote:
 <snip>

 ubyte[] data = some data;
 Image img = new Img!(Px.R8G8B8)(width, height, data);


 Image?  Img?


 img.write("mypng.png");




Image is the interface, Img the templated class that does all the 
work. It is parameterized by the pixel format. It may not be the 
best strategy (or naming scheme!) but enumerating the formats and 
templating based on that limited the number of cases I had to 
think about.


 Does it always take in a ubyte[], or does that depend on the 
 bit depth?


It currently just takes a ubyte, and doesn't do any rearranging 
of the array, so it will only work for 8-bit depths. If you had 
4-bit greyscale tightly packed into your ubyte (two pixels per 
byte) then it would not be correctly interpreted, nor would 16 
bit. The reason was I didn't want to have the Image class 'own' 
the passed-in array. If it did own it, then it would be free to 
unpack the smaller bit depths the way it does when decoding. I 
will fix this when I have time to think about it a bit more.
Jun 21 2012
prev sibling parent "Felix Hufnagel" <suicide xited.de> writes:
it's not that hard to write a baseline jpeg encoder. it's not harder tha=
n  =

decoding.
there are many tutorials and examples online.




Am 17.06.2012, 09:55 Uhr, schrieb cal <callumenator gmail.com>:


 On Sunday, 17 June 2012 at 07:07:35 UTC, Philippe Sigaud wrote:

 Still, I'm interested in writing a JPEG/PNG to disk from a
 ubyte[3][][], or whatever.


 Do you mean that you want to encode a ubyte array to disk as JPEG/PNG?=


  =


 Encoding a JPEG would be a bit of work I think, the format's kind of a=


  =


 monster. PNG should be easier, depending on how good you want the  =



 compression to be. If you don't care too much about compression level,=


  =


 you simply zlib compress the data, write it out by image row/scanline,=


  =


 include appropriate header and chunk info, and you're done. I'll give =


a  =


 simple encoder a go if you think you could use it.

 Cheers,
 cal



-- =

Erstellt mit Operas revolution=E4rem E-Mail-Modul: http://www.opera.com/=
mail/
Jun 21 2012
prev sibling next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
cal:


 I've been working on decoders for simple (baseline) JPEG and 
 PNG's, mostly for my own amusement, and they seem to work ok 
 now, so if anyone has need of some simple D modules to load 
 these formats you can grab them here:
 https://github.com/callumenator/imaged


Suggestions on the code:
- Try to use final switches.
- switch cases don't need (), so instead of case(foo):  write 
case foo:
- Try to add const/immutable/pure/nothrow where possible.

Bye,
bearophile
Jun 17 2012
parent reply "cal" <callumenator gmail.com> writes:
On Sunday, 17 June 2012 at 12:15:36 UTC, bearophile wrote:

 Suggestions on the code:
 - Try to use final switches.
 - switch cases don't need (), so instead of case(foo):  write 
 case foo:
 - Try to add const/immutable/pure/nothrow where possible.

 Bye,
 bearophile


Problem with final switches in this context is that images may be 
corrupt or badly encoded or something, which the decoder should 
just handle gracefully through a default I think, without 
crashing. The case statements though I will change, and though I 
have worlds of trouble with const-ness, I will endeavour to add 
those guarantees :)

Thanks for looking through
Jun 17 2012
parent "Chris NS" <ibisbasenji gmail.com> writes:
On Sunday, 17 June 2012 at 20:10:28 UTC, cal wrote:

 though I have worlds of trouble with const-ness, I will 
 endeavour to add those guarantees :)


What I usually do, unless it's just an obvious case, is write 
code without concern for const-ness, then write a thorough test 
and step through one function/whatever at a time and apply 
const/inout/pure/nothrow/etc, do any apparent refactoring, and 
check the test.  Writing the test helps me grok as many 
imaginable cases as possible, and doing it incrementally avoids 
massive "error cascades" from the compiler.
Jun 17 2012
prev sibling next sibling parent reply David <d dav1d.de> writes:
Am 16.06.2012 21:10, schrieb cal:

 I've been working on decoders for simple (baseline) JPEG and PNG's,
 mostly for my own amusement, and they seem to work ok now, so if anyone
 has need of some simple D modules to load these formats you can grab
 them here:
 https://github.com/callumenator/imaged

 Notes:
 - Will load 8-bit baseline sequential JPEG's only. All PNG image types
 are supported (only critical chunks are decoded however).
 - Can load from a stream or a file. In the case of the PNG's, this
 allows progressive display with an interlaced image.
 - I haven't put much effort into speeding up these routines, and have
 not tested the JPEG decoder extensively, so there are bound to be bugs
 and there is plenty room for improvement.
 - The example shows stream usage, and uses Adam Ruppe's simpledisplay to
 make a little viewer, that can flick through all images in a directory.

 Cheers,
 cal


Cool so I don't need to use my stb_image binding ( 
https://bitbucket.org/dav1d/gl-utils/src/0b97f77c14d7/stb_image ) anylonger!
Jun 17 2012
parent "cal" <callumenator gmail.com> writes:
On Sunday, 17 June 2012 at 12:35:41 UTC, David wrote:

 Cool so I don't need to use my stb_image binding ( 
 https://bitbucket.org/dav1d/gl-utils/src/0b97f77c14d7/stb_image 
 ) anylonger!


I used stb_image with C++ opengl projects, and it was great. 
stb_truetype is another gem, I would love to write a truetype 
loader, but I think the payoff is not worth the effort given the 
freetype bindings.
Jun 17 2012
prev sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
I have a simple png writer too if that's of use to you

https://github.com/adamdruppe/misc-stuff-including-D-programming-language-web-stuff

image.d and png.d. Somewhat suboptimal to use, since I
ported some of my old C code almost directly to it, and it
still works like C.
Jun 17 2012