www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Tempfiles in unittests

reply Magnus Lie Hetland <magnus hetland.org> writes:
I'm testing a class that writes various things to a file, and I'd like 
to test that behavior, and I'm wondering what the standard solution 
here is.

In Python, I'd just use a StringIO class, for example. It seems sort of 
impractical to make std.outbuffer to work as a File (even if the 
file-like parameter is a template parameter, the outbuffer methods 
don't work like the File methods, as far as I can see), so I've 
concluded that using std.stdio.tmpfile is the way to go.

However, when I'm done with the tempfile, I'd like to have its contents 
as a string. It seems that File has no method for reading the entire 
file contents into a string, so I'd have to read and concatenate lines 
or chunks or something? Now, if the File returned by tmpfile had a 
name, I could use std.file.readText(). But no such luck (as far as I 
can see, tmpfile().name is null).

So I'm just wondering -- what's "the way" to do this sort of testing? 
And if it is along the lines of what I'm doing, is there some 
function/method for reading the contents of a File as a string?

(I guess this goes for much file-related functionality; much of it 
seems to be very related to file-names and actual files, rather than 
"file-like" objects, i.e., those with the appropriate methods...)

-- 
Magnus Lie Hetland
http://hetland.org
Mar 09 2012
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Magnus Lie Hetland:


 It seems that File has no method for reading the entire file 
 contents into a string, so I'd have to read and concatenate 
 lines or chunks or something?


There are std.file.read() and std.file.readText().

But I agree that std.stdio.File should have a readAll() method
:-) If you think this is a good idea, add it to Bugzilla.

Bye,
bearophile
Mar 09 2012
parent reply Magnus Lie Hetland <magnus hetland.org> writes:
On 2012-03-09 15:08:42 +0000, bearophile said:


 Magnus Lie Hetland:
 

 It seems that File has no method for reading the entire file contents 
 into a string, so I'd have to read and concatenate lines or chunks or 
 something?


 
 There are std.file.read() and std.file.readText().


Yeah, I found those. I guess my post was a bit verbose, but the main 
point was that I have a File object, but no file name (because I'm 
using std.stdio.tmpfile() in a unit test), so std.file.read() and 
std.file.readText() are useless to me... :)


 But I agree that std.stdio.File should have a readAll() method
 :-) If you think this is a good idea, add it to Bugzilla.


Sure.

For now, though: What's The Way[TM] to write unit tests for code that 
writes to files? Use tmpfile and read/concatenate the file contents 
myself? Something else that's more practical? (I would have thought 
this kind of test would be common?-)

-- 
Magnus Lie Hetland
http://hetland.org
Mar 09 2012
next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, March 09, 2012 16:48:33 Magnus Lie Hetland wrote:

 On 2012-03-09 15:08:42 +0000, bearophile said:

 Magnus Lie Hetland:

 It seems that File has no method for reading the entire file contents
 into a string, so I'd have to read and concatenate lines or chunks or
 something?


 
 There are std.file.read() and std.file.readText().


 
 Yeah, I found those. I guess my post was a bit verbose, but the main
 point was that I have a File object, but no file name (because I'm
 using std.stdio.tmpfile() in a unit test), so std.file.read() and
 std.file.readText() are useless to me... :)


File has a name property. You can do something like

auto file = File.tmpfile();
auto filename = file.name;
auto str = readText(filename);

I do grant you though that it's a bit odd to need to get the file name and 
operate on that rather than operating on the File object. It works though, and 
is a lot cleaner than reading in the file line-by-line, if you don't actually 
need to do that.

- Jonathan M Davis
Mar 09 2012
parent reply Magnus Lie Hetland <magnus hetland.org> writes:
On 2012-03-09 18:42:01 +0000, Jonathan M Davis said:


 File has a name property. You can do something like
 
 auto file = File.tmpfile();
 auto filename = file.name;
 auto str = readText(filename);


Yeah -- again, sorry my original post was so verbose, but part of the 
"problem spec" was that my tempfiles don't seem to have names (i.e., 
they are unnamed tempfiles, and file.name is null; platform-dependent, 
I think).

Otherwise, this wouldn't have been a problem :)

-- 
Magnus Lie Hetland
http://hetland.org
Mar 10 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, March 10, 2012 15:44:26 Magnus Lie Hetland wrote:

 On 2012-03-09 18:42:01 +0000, Jonathan M Davis said:

 File has a name property. You can do something like
 
 auto file = File.tmpfile();
 auto filename = file.name;
 auto str = readText(filename);


 
 Yeah -- again, sorry my original post was so verbose, but part of the
 "problem spec" was that my tempfiles don't seem to have names (i.e.,
 they are unnamed tempfiles, and file.name is null; platform-dependent,
 I think).
 
 Otherwise, this wouldn't have been a problem :)


Then you'll probably have to write a wrapper function which just uses byLine 
and join to grab the string line by line and then put it back together again. 
As annoying as it may be, it wouldn't be hard.

- Jonathan M Davis
Mar 10 2012
parent reply Magnus Lie Hetland <magnus hetland.org> writes:
On 2012-03-10 17:10:52 +0000, Jonathan M Davis said:


 Otherwise, this wouldn't have been a problem :)


 
 Then you'll probably have to write a wrapper function which just uses byLine
 and join to grab the string line by line and then put it back together again.


Indeed. Not a problem.

Just surprised me that there wasn't a standard way of writing 
file-related unit tests :)

-- 
Magnus Lie Hetland
http://hetland.org
Mar 11 2012
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, March 11, 2012 13:33:34 Magnus Lie Hetland wrote:

 On 2012-03-10 17:10:52 +0000, Jonathan M Davis said:

 Otherwise, this wouldn't have been a problem :)


 
 Then you'll probably have to write a wrapper function which just uses
 byLine and join to grab the string line by line and then put it back
 together again.


 Indeed. Not a problem.
 
 Just surprised me that there wasn't a standard way of writing
 file-related unit tests :)


Well, most of the file-related unit tests in Phobos don't care much about the 
contents of the file, and if they do, it's becaus they're testing one of the 
functions for reading one (such as byLine or read), in which case, they're not 
looking to read in the whole file at once. So, as far as Phobos' unit tests are 
concerned, there's not really any need for what you're trying to do. That 
doesn't mean that it shouldn't be supported, but it just doesn't appear to 
have ever come up.

- Jonathan M Davis
Mar 11 2012
next sibling parent Ary Manzana <ary esperanto.org.ar> writes:
On 03/11/2012 06:14 PM, Jonathan M Davis wrote:

 On Sunday, March 11, 2012 13:33:34 Magnus Lie Hetland wrote:

 On 2012-03-10 17:10:52 +0000, Jonathan M Davis said:

 Otherwise, this wouldn't have been a problem :)


 Then you'll probably have to write a wrapper function which just uses
 byLine and join to grab the string line by line and then put it back
 together again.


 Indeed. Not a problem.

 Just surprised me that there wasn't a standard way of writing
 file-related unit tests :)


 Well, most of the file-related unit tests in Phobos don't care much about the
 contents of the file, and if they do, it's becaus they're testing one of the
 functions for reading one (such as byLine or read), in which case, they're not
 looking to read in the whole file at once. So, as far as Phobos' unit tests are
 concerned, there's not really any need for what you're trying to do. That
 doesn't mean that it shouldn't be supported, but it just doesn't appear to
 have ever come up.

 - Jonathan M Davis


What real world product did the designers of the D language do?
Mar 11 2012
prev sibling parent reply Magnus Lie Hetland <magnus hetland.org> writes:
On 2012-03-11 21:14:59 +0000, Jonathan M Davis said:


 So, as far as Phobos' unit tests are  concerned, there's not really any 
 need for what you're trying to do.


Sure. Just would have thought that maybe some unit-testing D user out 
there had written a function that wrote something to a file ;-)

But no problem -- not many lines to add to  my own code.

-- 
Magnus Lie Hetland
http://hetland.org
Mar 12 2012
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Mar 12, 2012 at 01:54:08PM +0100, Magnus Lie Hetland wrote:

 On 2012-03-11 21:14:59 +0000, Jonathan M Davis said:
 

So, as far as Phobos' unit tests are  concerned, there's not
really any need for what you're trying to do.


 
 Sure. Just would have thought that maybe some unit-testing D user out
 there had written a function that wrote something to a file ;-)


[...]

I've written unittests that create files to test code with.

However, I never thought far enough to make it something generic that
can be used elsewhere. :-) So I use actual filenames (not tmpfiles) so
that later unittests can reference actual filenames to load different
datasets into the program for testing. Not the nicest solution, since it
will overwrite files in the working directory, but presumably only
developers will bother running unittests anyway so this shouldn't be a
problem.


T

-- 
Truth, Sir, is a cow which will give [skeptics] no more milk, and so
they are gone to milk the bull. -- Sam. Johnson
Mar 12 2012
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Mar 11, 2012 at 02:14:59PM -0700, Jonathan M Davis wrote:

 On Sunday, March 11, 2012 13:33:34 Magnus Lie Hetland wrote:

 On 2012-03-10 17:10:52 +0000, Jonathan M Davis said:

 Otherwise, this wouldn't have been a problem :)


 
 Then you'll probably have to write a wrapper function which just
 uses byLine and join to grab the string line by line and then put
 it back together again.


 Indeed. Not a problem.
 
 Just surprised me that there wasn't a standard way of writing
 file-related unit tests :)


 
 Well, most of the file-related unit tests in Phobos don't care much
 about the contents of the file, and if they do, it's becaus they're
 testing one of the functions for reading one (such as byLine or read),
 in which case, they're not looking to read in the whole file at once.
 So, as far as Phobos' unit tests are concerned, there's not really any
 need for what you're trying to do. That doesn't mean that it shouldn't
 be supported, but it just doesn't appear to have ever come up.


[...]

I think he's talking about unit tests in general D code, not unittests
in Phobos.


T

-- 
Heuristics are bug-ridden by definition. If they didn't have bugs, they'd be
algorithms.
Mar 11 2012
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Mar 09, 2012 at 01:42:01PM -0500, Jonathan M Davis wrote:

 On Friday, March 09, 2012 16:48:33 Magnus Lie Hetland wrote:

 On 2012-03-09 15:08:42 +0000, bearophile said:

 Magnus Lie Hetland:

 It seems that File has no method for reading the entire file
 contents into a string, so I'd have to read and concatenate lines
 or chunks or something?


 
 There are std.file.read() and std.file.readText().


 
 Yeah, I found those. I guess my post was a bit verbose, but the main
 point was that I have a File object, but no file name (because I'm
 using std.stdio.tmpfile() in a unit test), so std.file.read() and
 std.file.readText() are useless to me... :)


 
 File has a name property. You can do something like
 
 auto file = File.tmpfile();
 auto filename = file.name;
 auto str = readText(filename);
 
 I do grant you though that it's a bit odd to need to get the file name
 and operate on that rather than operating on the File object. It works
 though, and is a lot cleaner than reading in the file line-by-line, if
 you don't actually need to do that.


[...]

This seems like a strange limitation. Shouldn't it be possible to slurp
the entire contents of an already-open file into a string? Perhaps a new
method in File?

Also, using file.name is prone to race conditions in certain situations.
(Not that it'd be a problem usually, but just sayin'.)


T

-- 
Designer clothes: how to cover less by paying more.
Mar 09 2012
prev sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, March 09, 2012 11:34:07 H. S. Teoh wrote:

 On Fri, Mar 09, 2012 at 01:42:01PM -0500, Jonathan M Davis wrote:

 On Friday, March 09, 2012 16:48:33 Magnus Lie Hetland wrote:

 On 2012-03-09 15:08:42 +0000, bearophile said:

 Magnus Lie Hetland:

 It seems that File has no method for reading the entire file
 contents into a string, so I'd have to read and concatenate lines
 or chunks or something?


 
 There are std.file.read() and std.file.readText().


 
 Yeah, I found those. I guess my post was a bit verbose, but the main
 point was that I have a File object, but no file name (because I'm
 using std.stdio.tmpfile() in a unit test), so std.file.read() and
 std.file.readText() are useless to me... :)


 
 File has a name property. You can do something like
 
 auto file = File.tmpfile();
 auto filename = file.name;
 auto str = readText(filename);
 
 I do grant you though that it's a bit odd to need to get the file name
 and operate on that rather than operating on the File object. It works
 though, and is a lot cleaner than reading in the file line-by-line, if
 you don't actually need to do that.


 
 [...]
 
 This seems like a strange limitation. Shouldn't it be possible to slurp
 the entire contents of an already-open file into a string? Perhaps a new
 method in File?
 
 Also, using file.name is prone to race conditions in certain situations.
 (Not that it'd be a problem usually, but just sayin'.)


Of course it's possible to read in the open file into a string. It's just that 
the only function in Phobos for doing that is in std.file and does not operate 
on a File. All of the File functions read in only a portion of the file. So, 
either a new function would have to be added to File, or you'd have to write 
one yourself (which wouldn't be all that hard using what's already there).

- Jonathan M Davis
Mar 09 2012