• solidstate1991 (11/11) Dec 05 2018 While working on a lot of enhancements for emesx'es tga library
• angel (4/15) Dec 06 2018 Didn't you think in the opposite direction - a memory mapped file
• H. S. Teoh (33/38) Dec 06 2018 [...]
• solidstate1991 (6/37) Dec 06 2018 Thanks! It's mostly done except for some testing, which I'll do

solidstate1991 <laszloszeremi outlook.com> writes:

While working on a lot of enhancements for emesx'es tga library 
(almost a rewrite), I was thinking on implementing a way for 
handling memory locations as files, which would make handling 
archives mush easier, since I wouldn't have to write a separate 
overload (or more) for loading files from memory every time I 
need access to a new type.

I'm thinking on making a library for it. Let me know if you want 
to add any additional functions. I'll making it under the Boost 
license, and also planning to add better binary writing and 
reading capabilities (e.g. direct reading and writing of simple 
types and certain structs).

angel <andrey.gelman gmail.com> writes:

On Thursday, 6 December 2018 at 02:48:59 UTC, solidstate1991 
wrote:
 While working on a lot of enhancements for emesx'es tga library 
 (almost a rewrite), I was thinking on implementing a way for 
 handling memory locations as files, which would make handling 
 archives mush easier, since I wouldn't have to write a separate 
 overload (or more) for loading files from memory every time I 
 need access to a new type.

 I'm thinking on making a library for it. Let me know if you 
 want to add any additional functions. I'll making it under the 
 Boost license, and also planning to add better binary writing 
 and reading capabilities (e.g. direct reading and writing of 
 simple types and certain structs).

Didn't you think in the opposite direction - a memory mapped file 
(std.mmfile), accessing a file like it were a memory location ?

"H. S. Teoh" <hsteoh quickfur.ath.cx> writes:

On Thu, Dec 06, 2018 at 02:48:59AM +0000, solidstate1991 via Digitalmars-d
wrote:
 While working on a lot of enhancements for emesx'es tga library
 (almost a rewrite), I was thinking on implementing a way for handling
 memory locations as files, which would make handling archives mush
 easier, since I wouldn't have to write a separate overload (or more)
 for loading files from memory every time I need access to a new type.

[...]

There's no need for a separate overload.  All you need to do is:

	import std.stdio;
	auto myfunc(File = std.stdio.File, Args...)(File fp, Args args)
	{
		...
		fp.rawRead(...);
		...
		fp.rawWrite(...);
		... // etc
	}

	// uses std.stdio.File
	myfunc(File("realfile.txt", "r"), ...);

	struct InMemoryFile {
		...
		void[] rawRead(void[]) { ... }
		void rawWrite(void[]) { ... }
		...
	}

	// uses InMemoryFile
	myfunc(InMemoryFile(...), ...);

As long as the type you pass in has the same API as std.stdio.File (or a
subset thereof, as long as that subset is the only thing you use in the
called function), the code will work as-is with no additional work.

Writing it this way also makes it easy to test your code without having
your unittests pollute the filesystem with temporaries (and needing to
clean up the mess afterwards).

If you're concerned about template bloat, you could factor out the
common API into an interface, then write a wrapper for std.stdio.File.


T

-- 
"A man's wife has more power over him than the state has." -- Ralph Emerson

solidstate1991 <laszloszeremi outlook.com> writes:

On Thursday, 6 December 2018 at 16:11:55 UTC, H. S. Teoh wrote:
 There's no need for a separate overload.  All you need to do is:

 	import std.stdio;
 	auto myfunc(File = std.stdio.File, Args...)(File fp, Args args)
 	{
 		...
 		fp.rawRead(...);
 		...
 		fp.rawWrite(...);
 		... // etc
 	}

 	// uses std.stdio.File
 	myfunc(File("realfile.txt", "r"), ...);

 	struct InMemoryFile {
 		...
 		void[] rawRead(void[]) { ... }
 		void rawWrite(void[]) { ... }
 		...
 	}

 	// uses InMemoryFile
 	myfunc(InMemoryFile(...), ...);

 As long as the type you pass in has the same API as 
 std.stdio.File (or a subset thereof, as long as that subset is 
 the only thing you use in the called function), the code will 
 work as-is with no additional work.

 Writing it this way also makes it easy to test your code 
 without having your unittests pollute the filesystem with 
 temporaries (and needing to clean up the mess afterwards).

 If you're concerned about template bloat, you could factor out 
 the common API into an interface, then write a wrapper for 
 std.stdio.File.


 T

Thanks! It's mostly done except for some testing, which I'll do 
tomorrow.

It has a few extra capabilities (such as reading and writing 
single elements with heap allocation), which I might try to put 
into std.stdio.File too through a pull request.