www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Binary with appended zip-file

reply Henning Hasemann <hhasemann web.de> writes:
Okay this might be of topic, if so sorry for posting this.

I'm about to build a little SDL-application which should run completely
from a single binary.
So my idea was to simple append a .zip-file with all other needed
files (images etc..) to it and extract it in RAM with std.zip.

The big problem is: How does the program know where in the binary the zip-file
begins?
Naturally this information is only available *after* compilation
(its the size of the plain binary then) but how can I insert
it to the program after it has been compiled?

I could reserve the space for a size-variable and compile with the dropped-in
value a second time but maybe someone has a better idea?

TIA
Henning
May 20 2007
next sibling parent reply Frank Benoit <keinfarbton googlemail.com> writes:
Henning Hasemann schrieb:

 Okay this might be of topic, if so sorry for posting this.
 
 I'm about to build a little SDL-application which should run completely
 from a single binary.
 So my idea was to simple append a .zip-file with all other needed
 files (images etc..) to it and extract it in RAM with std.zip.
 
 The big problem is: How does the program know where in the binary the zip-file
 begins?
 Naturally this information is only available *after* compilation
 (its the size of the plain binary then) but how can I insert
 it to the program after it has been compiled?
 
 I could reserve the space for a size-variable and compile with the dropped-in
 value a second time but maybe someone has a better idea?
 
 TIA
 Henning



There are two binary to D converter i know of:
* www.dprogramming.com
* in the tioport project, the bintod folder

So you can compile the zip into a ubyte[] and link it to your sources,
then your app can simply read it.
May 20 2007
parent reply Henning Hasemann <hhasemann web.de> writes:
 There are two binary to D converter i know of:
 * www.dprogramming.com
 * in the tioport project, the bintod folder
 
 So you can compile the zip into a ubyte[] and link it to your sources,
 then your app can simply read it.


Thanks a lot I tried the bintod from dprogramming, works great!
Exactly what I looked for!

Henning
May 20 2007
parent reply BCS <ao pathlink.com> writes:
Reply to Henning,


 There are two binary to D converter i know of:
 * www.dprogramming.com
 * in the tioport project, the bintod folder
 So you can compile the zip into a ubyte[] and link it to your
 sources, then your app can simply read it.
 


 Thanks a lot I tried the bintod from dprogramming, works great!
 Exactly what I looked for!
 
 Henning
 


another solution would be:

auto zipstream = new MemoryStream(import("zipfile.zip));

the import("...") loads the file at compile time and stuffs it into the binary 
as a constant string, that with the memory stream lets you read it like a file
May 20 2007
parent BCS <ao pathlink.com> writes:
Reply to myself,

 
 auto zipstream = new MemoryStream(import("zipfile.zip));
 



note to self: read the whole thread befor posting :b
May 20 2007
prev sibling next sibling parent Deewiant <deewiant.doesnotlike.spam gmail.com> writes:
Henning Hasemann wrote:

 I'm about to build a little SDL-application which should run completely from 
 a single binary. So my idea was to simple append a .zip-file with all other 
 needed files (images etc..) to it and extract it in RAM with std.zip.
 
 The big problem is: How does the program know where in the binary the 
 zip-file begins? Naturally this information is only available *after* 
 compilation (its the size of the plain binary then) but how can I insert it 
 to the program after it has been compiled?
 
 I could reserve the space for a size-variable and compile with the dropped-in
  value a second time but maybe someone has a better idea?
 


http://www.digitalmars.com/d/expression.html#ImportExpression

auto z = new ZipArchive(import("foo.zip"));
May 20 2007
prev sibling next sibling parent reply Frits van Bommel <fvbommel REMwOVExCAPSs.nl> writes:
(Note: This was typed before I read the post by Deewiant reminding me 
that the relatively new import-expression is pretty much what I'm 
hacking together here, and much cleaner (even though it returns char[] 
instead of void[]/ubyte[]/byte[]). I'll post anyway, just so I don't 
feel I wasted too much time :( )

Henning Hasemann wrote:

 I'm about to build a little SDL-application which should run completely
 from a single binary.
 So my idea was to simple append a .zip-file with all other needed
 files (images etc..) to it and extract it in RAM with std.zip.
 
 The big problem is: How does the program know where in the binary the zip-file
 begins?
 Naturally this information is only available *after* compilation
 (its the size of the plain binary then) but how can I insert
 it to the program after it has been compiled?
 
 I could reserve the space for a size-variable and compile with the dropped-in
 value a second time but maybe someone has a better idea?


A hack that should work: use an assembler to include the .zip into an 
object file and link it in. Also, Manually construct an array data 
structure pointing to it (with correct length).
Technically, this isn't what you asked for -- this won't be appending 
but embedding. IMHO it's a "cleaner" solution though.

Step-by-step: (Note: I didn't test this, so there may be some mistakes 
in these steps)

1) Copy-paste the following:
=====
section .rodata
zip_start:
     incbin "resources.zip"
zip_end:

global zip_data
zip_data:
     dd zip_end - zip_start  ; Length
     dd zip_start            ; Pointer
=====
to a file (let's say zip-resources.asm).
(For a 64-bit compiler (such as GDC for AMD64) replace both instances of 
'dd' by 'dq')
For object files other than ELF (the Linux default), you may need to add 
an underscore prefix to zip_data (or pass "--prefix _" to nasm when 
assembling) if you get link errors without it.

2) Install nasm (from nasm.sourceforge.net or your Linux package manager).
(It should also work with yasm. In fact, yasm may be required to 
generate 64-bit object files unless you want to use the CVS version of 
nasm. I will be assuming nasm for the rest though, since that's what I know)

3) To assemble the file:
nasm -f elf -o zip-resources.o zip-resources.asm
3a) For windows, replace 'nasm' with 'nasmw'.
3b) For DMD/win, replace 'elf' with 'obj'. You may also want to change 
the extension to ".obj" (also in the next step) to better fit the naming 
scheme used by DMD/win itself.
3c) For GDC/win(mingw), replace 'elf' with 'coff'.
(Note: I'm not sure about 3b & 3c, but I /think/ those are the options 
to use)
You'll probably want to add this command to your build system (to be 
executed after the resources.zip is updated, obviously). I'm not sure 
how to do this with build/rebuild, but it's trivial with a makefile.

4) Add zip-resources.o to your linker command line as one of the object 
files to link.

5) In your source, add the following line:
	extern extern(C) void[] zip_data;
It's 'extern' because it's defined in another file, 'extern(C)' because 
I don't feel like figuring out what the name mangles to ;), and void[] 
because that's all std.zip requires and manipulating it manually should 
be discouraged. Once the new constness system is implemented & working, 
"final invariant" may be added as well if you want to be nice and 
const-correct.

6) Elsewhere, add
	auto resources = new std.zip.ZipArchive(zip_data);


Of course you're free to rename variables and assembly labels if you 
prefer, just be sure to change them in all places where they're mentioned.
May 20 2007
parent Frank Benoit <keinfarbton googlemail.com> writes:
Frits van Bommel schrieb:

 (Note: This was typed before I read the post by Deewiant reminding me
 that the relatively new import-expression is pretty much what I'm
 hacking together here, and much cleaner (even though it returns char[]
 instead of void[]/ubyte[]/byte[]). I'll post anyway, just so I don't
 feel I wasted too much time :( )


Hehe, i think i can now drop my bintod in tioport :)
Thats cool.
May 20 2007
prev sibling next sibling parent reply Henning Hasemann <hhasemann web.de> writes:
Thank you all for your fast answers,
thanks Frits for your detailed work though it shouldnt
be necessary to do it this "hard way".

Ill also try the import() thingy from deewiant,
should save one memory and time consuming step compared to bintod.

Henning
May 20 2007
parent Henning Hasemann <hhasemann web.de> writes:
On Sun, 20 May 2007 19:17:00 +0200
Henning Hasemann <hhasemann web.de> wrote:


 
 Thank you all for your fast answers,
 thanks Frits for your detailed work though it shouldnt
 be necessary to do it this "hard way".
 
 Ill also try the import() thingy from deewiant,
 should save one memory and time consuming step compared to bintod.


whoot!
This is dramatically faster regarding compile time *and* easier
than bintod.
(Clear if you imagine that bintod converts the binary to hex and
the compiler converts it back to bin wheras import simply inserts
the binary data).

Henning
May 20 2007
prev sibling parent Jan Claeys <usenet janc.be> writes:
Op Sun, 20 May 2007 17:41:28 +0200
schreef Henning Hasemann <hhasemann web.de>:


 I'm about to build a little SDL-application which should run
 completely from a single binary.
 So my idea was to simple append a .zip-file with all other needed
 files (images etc..) to it and extract it in RAM with std.zip.
 
 The big problem is: How does the program know where in the binary the
 zip-file begins?


Check the size of the binary at runtime and subtract the size of
the .zip file from it?   :-)

(That's how many virusses did it, back in the days of DOS...)

-- 
Jan Claeys
May 26 2007