www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Of the use of unpredictableSeed

reply cym13 <cpicard openmailbox.org> writes:
Hi,

I found many times that people use unpredictableSeed in 
combination with normal PRNG for cryptographic purpose. Some even 
go as far as reseeding at each call to try making it more secure.

It is a dangerous practice, most PRNG are not designed with 
security (and unpredictability) in mind, and unpredictableSeed 
was definitely not designed with security in mind (or it failed 
heavily at it). It's a good tool when one needs randomness, not 
security.

I wrote a blog post to present exactly why this is a bad idea and 
how it could be exploited [1].

The best would be to add a standard CSPRNG interface to Phobos 
but we aren't there yet.

[1]: https://cym13.github.io/article/unpredictableSeed.html
Feb 26 2017
next sibling parent reply ketmar <ketmar ketmar.no-ip.org> writes:
cym13 wrote:

 Hi,

 I found many times that people use unpredictableSeed in combination 
 with normal PRNG for cryptographic purpose. Some even go as far as 
 reseeding at each call to try making it more secure.

 It is a dangerous practice, most PRNG are not designed with security 
 (and unpredictability) in mind, and unpredictableSeed was definitely 
 not designed with security in mind (or it failed heavily at it). It's 
 a good tool when one needs randomness, not security.

 I wrote a blog post to present exactly why this is a bad idea and how 
 it could be exploited [1].

 The best would be to add a standard CSPRNG interface to Phobos but we 
 aren't there yet.

 [1]: https://cym13.github.io/article/unpredictableSeed.html
"like /dev/random on Linux" (sighs) it was so good until this...
Feb 26 2017
parent reply cym13 <cpicard openmailbox.org> writes:
On Sunday, 26 February 2017 at 18:33:08 UTC, ketmar wrote:
 cym13 wrote:

 Hi,

 I found many times that people use unpredictableSeed in 
 combination with normal PRNG for cryptographic purpose. Some 
 even go as far as reseeding at each call to try making it more 
 secure.

 It is a dangerous practice, most PRNG are not designed with 
 security (and unpredictability) in mind, and unpredictableSeed 
 was definitely not designed with security in mind (or it 
 failed heavily at it). It's a good tool when one needs 
 randomness, not security.

 I wrote a blog post to present exactly why this is a bad idea 
 and how it could be exploited [1].

 The best would be to add a standard CSPRNG interface to Phobos 
 but we aren't there yet.

 [1]: https://cym13.github.io/article/unpredictableSeed.html
"like /dev/random on Linux" (sighs) it was so good until this...
That's a typo actually, I meant urandom, I'll correct it. Actually it would be better not to use urandom directly but use it as source to regularly reseed another PRNG in order to avoid some warts but meh. As a first step it's good enough as it is.
Feb 26 2017
parent ketmar <ketmar ketmar.no-ip.org> writes:
cym13 wrote:

 "like /dev/random on Linux"
 (sighs) it was so good until this...
That's a typo actually, I meant urandom, I'll correct it.
thank you. sorry for me being rough: i was trying to make a joke, and i was pretty sure that it was a typo. but now i reread my post and found that the joke mysteriously turned into something i didn't meant to say.
 Actually it would be better not to use urandom directly but use it as 
 source to regularly reseed another PRNG in order to avoid some warts 
 but meh. As a first step it's good enough as it is.
yeah. yet, urandom is using cryptoprng (salsa now, afair), so it may be used as-is too. ah, even good old arc4 is not that bad -- as urandom is almost never have a "scratch start" case. and one can throw away 2k-3k of values just to be sure anyway. ;-) ah, i *meant* to say: "good article". so: good article! ;-)
Feb 26 2017
prev sibling next sibling parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 02/26/2017 01:23 PM, cym13 wrote:
 Hi,

 I found many times that people use unpredictableSeed in combination with
 normal PRNG for cryptographic purpose. Some even go as far as reseeding
 at each call to try making it more secure.

 It is a dangerous practice, most PRNG are not designed with security
 (and unpredictability) in mind, and unpredictableSeed was definitely not
 designed with security in mind (or it failed heavily at it). It's a good
 tool when one needs randomness, not security.

 I wrote a blog post to present exactly why this is a bad idea and how it
 could be exploited [1].

 The best would be to add a standard CSPRNG interface to Phobos but we
 aren't there yet.

 [1]: https://cym13.github.io/article/unpredictableSeed.html
FWIW, DAuth[1] uses, and offers, an implementation of Hash_DRBG, a well-known and established CSPRNG algorithm. It's entropy source (not exactly the same as a seed, but basically the CSPRNG equivalent) is customizable, but by default, it uses _RtlGenRandom on Windows (the same source used by the CryptGenRandom algorithms) and '/dev/urandom' on Posix: https://github.com/Abscissa/DAuth/blob/master/src/dauth/hashdrbg.d [1] Ugh, still haven't gotten around to finishing DAuth's new version, renamed "InstaUser".
Feb 26 2017
next sibling parent "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
Speaking of, another good source of more "how to not mess up security" 
info is here:

https://crackstation.net/hashing-security.htm

DAuth/InstaUser is based on the points in there.
Feb 26 2017
prev sibling parent reply cym13 <cpicard openmailbox.org> writes:
On Sunday, 26 February 2017 at 22:56:14 UTC, Nick Sabalausky 
(Abscissa) wrote:
 On 02/26/2017 01:23 PM, cym13 wrote:
 Hi,

 I found many times that people use unpredictableSeed in 
 combination with
 normal PRNG for cryptographic purpose. Some even go as far as 
 reseeding
 at each call to try making it more secure.

 It is a dangerous practice, most PRNG are not designed with 
 security
 (and unpredictability) in mind, and unpredictableSeed was 
 definitely not
 designed with security in mind (or it failed heavily at it). 
 It's a good
 tool when one needs randomness, not security.

 I wrote a blog post to present exactly why this is a bad idea 
 and how it
 could be exploited [1].

 The best would be to add a standard CSPRNG interface to Phobos 
 but we
 aren't there yet.

 [1]: https://cym13.github.io/article/unpredictableSeed.html
FWIW, DAuth[1] uses, and offers, an implementation of Hash_DRBG, a well-known and established CSPRNG algorithm. It's entropy source (not exactly the same as a seed, but basically the CSPRNG equivalent) is customizable, but by default, it uses _RtlGenRandom on Windows (the same source used by the CryptGenRandom algorithms) and '/dev/urandom' on Posix: https://github.com/Abscissa/DAuth/blob/master/src/dauth/hashdrbg.d [1] Ugh, still haven't gotten around to finishing DAuth's new version, renamed "InstaUser".
And it is very interesting to note that you fall into the very exact problem I am describing in my post if it weren't that your default PRNG isn't seedable. Use any other PRNG and you are vulnerable (if I read it correctly, I haven't spent much time on that). From random.d, the function randomToken calls randomBytes which inits a new PRNG which is seeded with unpredictableSeed which becomes the actual PRNG used: string randomToken(Rand = DefaultCryptoRand)(size_t strength = defaultTokenStrength) if(isDAuthRandom!Rand) { return TokenBase64.encode( randomBytes!Rand(strength) ); } ubyte[] randomBytes(Rand = DefaultCryptoRand)(size_t numBytes) if(isDAuthRandom!Rand) { Rand rand; rand.initRand(); return randomBytes(numBytes, rand); } private void initRand(Rand)(ref Rand rand) if(isDAuthRandom!Rand) { static if(isSeedable!Rand) rand.seed(unpredictableSeed); } The only thing that saves you here is that your DefaultCryptoRand isn't seedable. Note that I'm not saying that to bash you or anything, it just happens to show well why I think my article was necessary.
Feb 26 2017
parent Nick Sabalausky <a a.a> writes:
On Monday, 27 February 2017 at 00:10:31 UTC, cym13 wrote:
 The only thing that saves you here is that your 
 DefaultCryptoRand isn't seedable. Note that I'm not saying that 
 to bash you or anything, it just happens to show well why I 
 think my article was necessary.
Oh, don't get me wrong, I'm very much in favor of your article, ESPECIALLY if people are using the phobos rng in security contexts. Keep in mind though, if my Hash_DRBG implementation were seedable, it wouldn't be a valid implementation of Hash_DRBG anyway. (A good example, I think, of why we coders shouldn't be inventing crypto algorithms, only implementing establised ones created and well-studied by the math/crypto experts.)
Feb 26 2017
prev sibling next sibling parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 02/26/2017 01:23 PM, cym13 wrote:
 [1]: https://cym13.github.io/article/unpredictableSeed.html
Good enlightening article. And Veles looks cool, never heard of it before. One detail: auto copyState(uint origin) { for (uint i=1; i>0; i++) { if (origin == Random(i).front) return origin; } return 0; } Shouldn't "return origin;" be "return i;"?
Feb 26 2017
parent cym13 <cpicard openmailbox.org> writes:
On Monday, 27 February 2017 at 04:51:37 UTC, Nick Sabalausky 
(Abscissa) wrote:
 On 02/26/2017 01:23 PM, cym13 wrote:
 [1]: https://cym13.github.io/article/unpredictableSeed.html
Good enlightening article. And Veles looks cool, never heard of it before. One detail: auto copyState(uint origin) { for (uint i=1; i>0; i++) { if (origin == Random(i).front) return origin; } return 0; } Shouldn't "return origin;" be "return i;"?
Yes, it should completely be "return i;", thanks for noticing I'll correct it.
Feb 26 2017
prev sibling next sibling parent reply Yuxuan Shui <yshuiv7 gmail.com> writes:
On Sunday, 26 February 2017 at 18:23:27 UTC, cym13 wrote:
 Hi,

 I found many times that people use unpredictableSeed in 
 combination with normal PRNG for cryptographic purpose. Some 
 even go as far as reseeding at each call to try making it more 
 secure.

 It is a dangerous practice, most PRNG are not designed with 
 security (and unpredictability) in mind, and unpredictableSeed 
 was definitely not designed with security in mind (or it failed 
 heavily at it). It's a good tool when one needs randomness, not 
 security.

 I wrote a blog post to present exactly why this is a bad idea 
 and how it could be exploited [1].

 The best would be to add a standard CSPRNG interface to Phobos 
 but we aren't there yet.

 [1]: https://cym13.github.io/article/unpredictableSeed.html
When I see the code for unpredictableSeed I went face palm really hard. I did some digging, and it was way way worse: https://github.com/dlang/phobos/commit/ff54d867e41abc8261075f0dce1261d68ee09180#diff-713ce153554afc99a07767cc8ba940aeL529 https://github.com/dlang/phobos/commit/c433c36658df45677bf90b00e93cba051883294e
Mar 02 2017
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 3/2/17 4:50 PM, Yuxuan Shui wrote:
 On Sunday, 26 February 2017 at 18:23:27 UTC, cym13 wrote:
 Hi,

 I found many times that people use unpredictableSeed in combination
 with normal PRNG for cryptographic purpose. Some even go as far as
 reseeding at each call to try making it more secure.

 It is a dangerous practice, most PRNG are not designed with security
 (and unpredictability) in mind, and unpredictableSeed was definitely
 not designed with security in mind (or it failed heavily at it). It's
 a good tool when one needs randomness, not security.

 I wrote a blog post to present exactly why this is a bad idea and how
 it could be exploited [1].

 The best would be to add a standard CSPRNG interface to Phobos but we
 aren't there yet.

 [1]: https://cym13.github.io/article/unpredictableSeed.html
When I see the code for unpredictableSeed I went face palm really hard. I did some digging, and it was way way worse: https://github.com/dlang/phobos/commit/ff54d867e41abc8261075f0dce1261d68ee09180#diff-713ce153554afc99a07767cc8ba940aeL529 https://github.com/dlang/phobos/commit/c433c36658df45677bf90b00e93cba051883294e
Could you please submit a PR that makes is better? Thanks! -- Andrei
Mar 02 2017
prev sibling parent reply cym13 <cpicard openmailbox.org> writes:
On Thursday, 2 March 2017 at 21:50:36 UTC, Yuxuan Shui wrote:
 On Sunday, 26 February 2017 at 18:23:27 UTC, cym13 wrote:
 Hi,

 I found many times that people use unpredictableSeed in 
 combination with normal PRNG for cryptographic purpose. Some 
 even go as far as reseeding at each call to try making it more 
 secure.

 It is a dangerous practice, most PRNG are not designed with 
 security (and unpredictability) in mind, and unpredictableSeed 
 was definitely not designed with security in mind (or it 
 failed heavily at it). It's a good tool when one needs 
 randomness, not security.

 I wrote a blog post to present exactly why this is a bad idea 
 and how it could be exploited [1].

 The best would be to add a standard CSPRNG interface to Phobos 
 but we aren't there yet.

 [1]: https://cym13.github.io/article/unpredictableSeed.html
When I see the code for unpredictableSeed I went face palm really hard. I did some digging, and it was way way worse: https://github.com/dlang/phobos/commit/ff54d867e41abc8261075f0dce1261d68ee09180#diff-713ce153554afc99a07767cc8ba940aeL529 https://github.com/dlang/phobos/commit/c433c36658df45677bf90b00e93cba051883294e
This is a misunderstanding: unpredictableSeed is perfectly fine as it is. What's wrong is 1) using it for cryptographic purpose and 2) systematic reseeding. 1) There is no way to make a cryptographically secure pseudo-random number generator that is seedable. If a PRNG is seedable then his number of states is finite which makes it cycle one way or an other once you've expended all possible states. So no cryptographic application should use such PRNG, and therefore any seed. For non-cryptographic purpose unpredictableSeed is, honnestly, random enough. It isn't you're actual PRNG (or shouldn't be, see point 2 but is only used to reseed it from time to time. 2) The big mistake is systematic reseeding which is far more common than it should be. Using unpredictableSeed as a seed is fine, the actual PRNG that is seeded will add a lot of entropy to the output. However by systematically reseeding it makes unpredictableSeed the PRNG that is actually used (ie, it doesn't leave it any time to add entropy). And that is something that should never happen because the PRNG in unpredictableSeed is the weakest possible. It is not meant to be the actual PRNG. So this article wasn't meant to be "Haha, Phobos is broken, *facepalm*". It was about using tools for what they're meant and nothing else (especially when dealing with cryptographic problems). The problem, if anything, is in the documentation that doesn't enforce that point.
Mar 05 2017
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 03/05/2017 05:48 AM, cym13 wrote:
 The problem, if anything, is in the documentation that doesn't enforce
 that point.
Thanks. Could we have you, Yishan, or other security expert submit a PR for the documentation? I'm not an expert and my coding of unpredictableSeed has been at the level of "I don't like those games that always do the same `random` things when you start playing." Another good thing pointed by the article would be to use the Mersenne twister for unpredictableSeed, which would make it difficult to infer the sequence from a few samples. Please share if that would be a good thing to do. Thanks! -- Andrei
Mar 05 2017
next sibling parent David Nadlinger <code klickverbot.at> writes:
On Sunday, 5 March 2017 at 15:30:29 UTC, Andrei Alexandrescu 
wrote:
 Another good thing pointed by the article would be to use the 
 Mersenne twister for unpredictableSeed, which would make it 
 difficult to infer the sequence from a few samples. Please 
 share if that would be a good thing to do.
"From just a few samples" yes, but in general the internal state of Mersenne twister is easy to reverse – e.g. check out https://github.com/fx5/not_random for a nice little demonstration. IMHO there is no problem with unpredictableSeed not being cryptographically secure, as long as the docs point out it is not to be used anywhere near where a CSPRNG should be. But I'm hardly an expert either. — David
Mar 05 2017
prev sibling next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sun, Mar 05, 2017 at 10:30:29AM -0500, Andrei Alexandrescu via Digitalmars-d
wrote:
[...]
 Another good thing pointed by the article would be to use the Mersenne
 twister for unpredictableSeed, which would make it difficult to infer
 the sequence from a few samples. Please share if that would be a good
 thing to do.
[...] Wait, isn't that missing the point? I thought the whole point of the article was that you shouldn't be using unpredictableSeed as your PRNG. It's only supposed to give a random-enough value to get your chosen PRNG into a (hopefully) unpredictable initial state. But you should be using values from the PRNG, not from unpredictableSeed! Otherwise that's totally missing the point. It's possible to use unpredictableSeed for occasionally reseeding your PRNG, but that should be quite infrequent. If you find yourself reseeding your PRNG every other minute, or worse, every time you call your PRNG, then you're doing something very, very wrong. Using the Mersenne twister to generate unpredictableSeed seems to me to be completely backwards. It should be the other way round: the value of unpredictableSeed should be random enough to be suitable for seeding a Mersenne twister algorithm, so that it will start off the algorithm in a random initial state (and you should be getting values from the algorithm thereafter, not from unpredictableSeed). T -- Unix is my IDE. -- Justin Whear
Mar 05 2017
parent "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 03/05/2017 07:25 PM, H. S. Teoh via Digitalmars-d wrote:
 On Sun, Mar 05, 2017 at 10:30:29AM -0500, Andrei Alexandrescu via
Digitalmars-d wrote:
 [...]
 Another good thing pointed by the article would be to use the Mersenne
 twister for unpredictableSeed, which would make it difficult to infer
 the sequence from a few samples. Please share if that would be a good
 thing to do.
[...] Wait, isn't that missing the point? I thought the whole point of the article was that you shouldn't be using unpredictableSeed as your PRNG. It's only supposed to give a random-enough value to get your chosen PRNG into a (hopefully) unpredictable initial state. But you should be using values from the PRNG, not from unpredictableSeed! Otherwise that's totally missing the point. It's possible to use unpredictableSeed for occasionally reseeding your PRNG, but that should be quite infrequent. If you find yourself reseeding your PRNG every other minute, or worse, every time you call your PRNG, then you're doing something very, very wrong. Using the Mersenne twister to generate unpredictableSeed seems to me to be completely backwards. It should be the other way round: the value of unpredictableSeed should be random enough to be suitable for seeding a Mersenne twister algorithm, so that it will start off the algorithm in a random initial state (and you should be getting values from the algorithm thereafter, not from unpredictableSeed).
Well, the big point is that nothing involving seeds should come near anything security-releated. Then secondly, (like you say) don't use any initial-seed-getter *as* an RNG. But that aside, *if*[1] we do want to increase the entropy in unpredictableSeed, we should use /dev/(u)random and _RtlGenRandom. Like you say, anything less is kinda missing the point (unless someone can argue the current one is insufficient for non-security randomization). [1] And I'm not sure we necessarily do want to. Current unpredictableSeed seems good enough as-is for non-security purposes, and we don't want to give people more reason to erroneously think it's ok to use unpredictableSeed for salts, tokens and the like.
Mar 05 2017
prev sibling parent Kagamin <spam here.lot> writes:
On Sunday, 5 March 2017 at 15:30:29 UTC, Andrei Alexandrescu 
wrote:
 Another good thing pointed by the article would be to use the 
 Mersenne twister for unpredictableSeed, which would make it 
 difficult to infer the sequence from a few samples. Please 
 share if that would be a good thing to do.
Seeding a PRNG from the same PRNG often gives bad results. Also maybe rename it to arbitrarySeed, unpredictability is not the right claim here.
Mar 06 2017
prev sibling parent reply Shachar Shemesh <shachar weka.io> writes:
On 26/02/17 20:23, cym13 wrote:
 Hi,

 I found many times that people use unpredictableSeed in combination with
 normal PRNG for cryptographic purpose. Some even go as far as reseeding
 at each call to try making it more secure.

 It is a dangerous practice, most PRNG are not designed with security
 (and unpredictability) in mind, and unpredictableSeed was definitely not
 designed with security in mind (or it failed heavily at it). It's a good
 tool when one needs randomness, not security.

 I wrote a blog post to present exactly why this is a bad idea and how it
 could be exploited [1].

 The best would be to add a standard CSPRNG interface to Phobos but we
 aren't there yet.

 [1]: https://cym13.github.io/article/unpredictableSeed.html
Excuse me if I'm asking a trivial question. Why not just seed it from /dev/urandom? (or equivalent on non-Linux platforms. I know at least Windows has an equivalent). Shachar
Mar 06 2017
parent reply sarn <sarn theartofmachinery.com> writes:
On Monday, 6 March 2017 at 10:12:09 UTC, Shachar Shemesh wrote:
 Excuse me if I'm asking a trivial question. Why not just seed 
 it from /dev/urandom? (or equivalent on non-Linux platforms. I 
 know at least Windows has an equivalent).

 Shachar
One reason is that /dev/urandom isn't always available, e.g., in a chroot. Sure, these are corner cases, but it's annoying when stuff like this doesn't "just work".
Mar 06 2017
next sibling parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 03/06/2017 05:19 PM, sarn wrote:
 On Monday, 6 March 2017 at 10:12:09 UTC, Shachar Shemesh wrote:
 Excuse me if I'm asking a trivial question. Why not just seed it from
 /dev/urandom? (or equivalent on non-Linux platforms. I know at least
 Windows has an equivalent).

 Shachar
One reason is that /dev/urandom isn't always available, e.g., in a chroot. Sure, these are corner cases, but it's annoying when stuff like this doesn't "just work".
I don't claim to be any sort of linux expert or anything, but doesn't chroot have a reputation for being a bit of a finicky, leaky abstraction anyway? I haven't really used them, but that's been my understanding...?
Mar 06 2017
next sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Monday, March 06, 2017 22:04:44 Nick Sabalausky  via Digitalmars-d wrote:
 On 03/06/2017 05:19 PM, sarn wrote:
 On Monday, 6 March 2017 at 10:12:09 UTC, Shachar Shemesh wrote:
 Excuse me if I'm asking a trivial question. Why not just seed it from
 /dev/urandom? (or equivalent on non-Linux platforms. I know at least
 Windows has an equivalent).

 Shachar
One reason is that /dev/urandom isn't always available, e.g., in a chroot. Sure, these are corner cases, but it's annoying when stuff like this doesn't "just work".
I don't claim to be any sort of linux expert or anything, but doesn't chroot have a reputation for being a bit of a finicky, leaky abstraction anyway? I haven't really used them, but that's been my understanding...?
If you want a fully secure chroot, then what you really want is BSD jails or Solaris zones. chroots are indeed too leaky to be secure. But secure container-ization doesn't really matter here, since a D program using D's standard number generator should work regardless of where it's running. So, it's a question of whether we're guaranteed to get at /dev/urandom or not, and if not, how reasonable it is to require that it be accessible for the program to run. There _are_ programs that require access to /dev, and /dev is _usually_ available. Regardless, if there is no guarantee that /dev/urandom (or whatever system resource for getting randomness is) is going to be accessible, and we want to use it, then we either have to require that it be accessible and error out if it isn't, or we have to have a backup if accessing it fails. Ideally, you'd be able to just use /dev/urandom and not worry about it, but I don't know how common it is for /dev/urandom to be unavailable or how reasonable it is to require that it be available. In general though, using /dev/urandom to seed the pseudo-random number generator seems like a good plan. - Jonathan M Davis
Mar 06 2017
parent reply Seb <seb wilzba.ch> writes:
On Tuesday, 7 March 2017 at 03:43:42 UTC, Jonathan M Davis wrote:
 On Monday, March 06, 2017 22:04:44 Nick Sabalausky  via 
 Digitalmars-d wrote:
 On 03/06/2017 05:19 PM, sarn wrote:
 On Monday, 6 March 2017 at 10:12:09 UTC, Shachar Shemesh 
 wrote:
 Excuse me if I'm asking a trivial question. Why not just 
 seed it from /dev/urandom? (or equivalent on non-Linux 
 platforms. I know at least Windows has an equivalent).

 Shachar
One reason is that /dev/urandom isn't always available, e.g., in a chroot. Sure, these are corner cases, but it's annoying when stuff like this doesn't "just work".
I don't claim to be any sort of linux expert or anything, but doesn't chroot have a reputation for being a bit of a finicky, leaky abstraction anyway? I haven't really used them, but that's been my understanding...?
If you want a fully secure chroot, then what you really want is BSD jails or Solaris zones. chroots are indeed too leaky to be secure. But secure container-ization doesn't really matter here, since a D program using D's standard number generator should work regardless of where it's running. So, it's a question of whether we're guaranteed to get at /dev/urandom or not, and if not, how reasonable it is to require that it be accessible for the program to run. There _are_ programs that require access to /dev, and /dev is _usually_ available. Regardless, if there is no guarantee that /dev/urandom (or whatever system resource for getting randomness is) is going to be accessible, and we want to use it, then we either have to require that it be accessible and error out if it isn't, or we have to have a backup if accessing it fails. Ideally, you'd be able to just use /dev/urandom and not worry about it, but I don't know how common it is for /dev/urandom to be unavailable or how reasonable it is to require that it be available. In general though, using /dev/urandom to seed the pseudo-random number generator seems like a good plan. - Jonathan M Davis
As apparently no one here hasn't mentioned this, Linux >= 3.17 has a dedicated syscall API. Please see: http://man7.org/linux/man-pages/man2/getrandom.2.html And this excellent introductory article: https://lwn.net/Articles/605828 I did work on getting a nice getEntropy function into mir-random: https://github.com/libmir/mir-random/pull/13 (For which it was planned to backport it to Phobos after some testing and real-world feedback on the API.)
Mar 07 2017
next sibling parent reply Kagamin <spam here.lot> writes:
On Tuesday, 7 March 2017 at 10:18:52 UTC, Seb wrote:
 http://man7.org/linux/man-pages/man2/getrandom.2.html
Unnecessarily reading large quantities of data will have a 
negative impact on other users of the /dev/random and 
/dev/urandom devices.  Therefore, getrandom() should not be used 
for Monte Carlo simulations or other programs/algorithms which 
are doing probabilistic sampling.
In other words it shouldn't be used when not strictly necessary.
Mar 07 2017
parent "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 03/07/2017 09:46 AM, Kagamin wrote:
 On Tuesday, 7 March 2017 at 10:18:52 UTC, Seb wrote:
 http://man7.org/linux/man-pages/man2/getrandom.2.html
 Unnecessarily reading large quantities of data will have a negative
 impact on other users of the /dev/random and /dev/urandom devices.
 Therefore, getrandom() should not be used for Monte Carlo simulations
 or other programs/algorithms which are doing probabilistic sampling.
In other words it shouldn't be used when not strictly necessary.
Yes, although that's true of reading /dev/(u)random too. (Just to be clear.)
Mar 07 2017
prev sibling parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 03/07/2017 05:18 AM, Seb wrote:
 As apparently no one here hasn't mentioned this, Linux >= 3.17 has a
 dedicated syscall API. Please see:

 http://man7.org/linux/man-pages/man2/getrandom.2.html

 And this excellent introductory article:

 https://lwn.net/Articles/605828
Ooh, that's great to know! (Kinda sad that it seems necessary, given the "unix filesystem and unix design" ideals, but oh well, realities are realities.) Is there a "proper" way to check for this function's existence on a local machine, other than test-compiling, or is parsing 'uname -a' as "right way" as it gets? And anyone know about OSX? Would OSX use the getentropy the article you linked to mentions for OpenBSD? Or something else? Or just fallback to /dev/(u)random? This really deserves a Phobos PR, IMO, FWIW. Maybe I'll try my hand at it if any OSX folk have tips for that OS.
Mar 07 2017
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2017-03-07 21:15, Nick Sabalausky (Abscissa) wrote:

 And anyone know about OSX? Would OSX use the getentropy the article you
 linked to mentions for OpenBSD?
As far as I can see, there's no "getentropy" on macOS. I see references to it online, but I cannot find it in any header files.
 Or something else?
I found something called "arc4random" and a couple of related functions [1].
 Or just fallback to /dev/(u)random?
/dev/(u)random does exist on macOS. This is what Apple seems to recommend [2]. [1] https://developer.apple.com/legacy/library/documentation/Darwin/Reference/ManPages/man3/arc4random.3.html [2] https://developer.apple.com/library/content/documentation/Security/Conceptual/cryptoservices/RandomNumberGenerationAPIs/RandomNumberGenerationAPIs.html -- /Jacob Carlborg
Mar 07 2017
prev sibling parent reply Yuxuan Shui <yshuiv7 gmail.com> writes:
On Tuesday, 7 March 2017 at 20:15:56 UTC, Nick Sabalausky 
(Abscissa) wrote:
 On 03/07/2017 05:18 AM, Seb wrote:
[...]
Ooh, that's great to know! (Kinda sad that it seems necessary, given the "unix filesystem and unix design" ideals, but oh well, realities are realities.) Is there a "proper" way to check for this function's existence on a local machine, other than test-compiling, or is parsing 'uname -a' as "right way" as it gets? And anyone know about OSX? Would OSX use the getentropy the article you linked to mentions for OpenBSD? Or something else? Or just fallback to /dev/(u)random? This really deserves a Phobos PR, IMO, FWIW.
https://github.com/dlang/phobos/pull/5230
Mar 08 2017
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 3/8/17 11:24 PM, Yuxuan Shui wrote:
 On Tuesday, 7 March 2017 at 20:15:56 UTC, Nick Sabalausky (Abscissa) wrote:
 On 03/07/2017 05:18 AM, Seb wrote:
 [...]
Ooh, that's great to know! (Kinda sad that it seems necessary, given the "unix filesystem and unix design" ideals, but oh well, realities are realities.) Is there a "proper" way to check for this function's existence on a local machine, other than test-compiling, or is parsing 'uname -a' as "right way" as it gets? And anyone know about OSX? Would OSX use the getentropy the article you linked to mentions for OpenBSD? Or something else? Or just fallback to /dev/(u)random? This really deserves a Phobos PR, IMO, FWIW.
https://github.com/dlang/phobos/pull/5230
Thanks Yuxuan, sorry for missing this. Can we have this peer reviewed by 1-2 crypto experts? Thanks! -- Andrei
Mar 21 2017
next sibling parent reply sarn <sarn theartofmachinery.com> writes:
On Tuesday, 21 March 2017 at 10:27:27 UTC, Andrei Alexandrescu 
wrote:
 Thanks Yuxuan, sorry for missing this. Can we have this peer 
 reviewed by 1-2 crypto experts? Thanks! -- Andrei
By API, unpredictableSeed() only returns a 32b uint and will never meet crypto standards. Beware of anyone who offers to review it based on their "crypto expertise". unpredictableSeed() is just for things like making single-player games more interesting. It simply isn't for security, and that's pretty much what cym13's post was about.
Mar 21 2017
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Tue, Mar 21, 2017 at 10:11:44PM +0000, sarn via Digitalmars-d wrote:
 On Tuesday, 21 March 2017 at 10:27:27 UTC, Andrei Alexandrescu wrote:
 Thanks Yuxuan, sorry for missing this. Can we have this peer
 reviewed by 1-2 crypto experts? Thanks! -- Andrei
By API, unpredictableSeed() only returns a 32b uint and will never meet crypto standards. Beware of anyone who offers to review it based on their "crypto expertise". unpredictableSeed() is just for things like making single-player games more interesting. It simply isn't for security, and that's pretty much what cym13's post was about.
Yeah, why is it that people still keep thinking unpredictableSeed(), or indeed, the whole of the current std.random, is useful for *anything* related to crypto?? If you want to do crypto, you should be using a crypto library that is *designed* to be cryptographically secure and *verified* by cryptoanalysts to be secure. std.random is a far cry from that, and crypto isn't even its charter anyway. What std.random is useful for is to make games more interesting, or for certain kinds of Monte Carlo simulations. (Note that some Monte Carlo simulations may be sensitive to hidden patterns in std.random PRNGs, so you should choose your PRNG carefully, and/or take the simulation results with a grain of salt.) Or for things like probabilistic algorithms (e.g., probabilistic approximate solution finders for NP-complete problems and the like). Using it for anything crypto- or security-related is just begging to be hacked, esp. in this day and age. T -- Why waste time learning, when ignorance is instantaneous? -- Hobbes, from Calvin & Hobbes
Mar 24 2017
parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 03/24/2017 02:56 PM, H. S. Teoh via Digitalmars-d wrote:
 Yeah, why is it that people still keep thinking unpredictableSeed(), or
 indeed, the whole of the current std.random, is useful for *anything*
 related to crypto??
Seems there's two issues there: 1. The name "unpredictableSeed" is highly misleading, if not outright false. 2. Maybe the std.random docs need to be more clear, right up top, that it's not for anything security-related.
Mar 26 2017
parent reply Yuxuan Shui <yshuiv7 gmail.com> writes:
On Sunday, 26 March 2017 at 17:55:20 UTC, Nick Sabalausky 
(Abscissa) wrote:
 2. Maybe the std.random docs need to be more clear, right up 
 top, that it's not for anything security-related.
Agreed. Like what Python did here: https://docs.python.org/3/library/random.html
Mar 26 2017
parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sun, Mar 26, 2017 at 11:46:44PM +0000, Yuxuan Shui via Digitalmars-d wrote:
 On Sunday, 26 March 2017 at 17:55:20 UTC, Nick Sabalausky (Abscissa) wrote:
 
 2. Maybe the std.random docs need to be more clear, right up top,
 that it's not for anything security-related.
Agreed. Like what Python did here: https://docs.python.org/3/library/random.html
https://github.com/dlang/phobos/pull/5306 T -- I see that you JS got Bach.
Mar 26 2017
prev sibling parent reply Kagamin <spam here.lot> writes:
On Tuesday, 21 March 2017 at 10:27:27 UTC, Andrei Alexandrescu 
wrote:
 Thanks Yuxuan, sorry for missing this. Can we have this peer 
 reviewed by 1-2 crypto experts? Thanks! -- Andrei
It's not recommended to use system CSPRNG for non-cryptographic purposes: https://forum.dlang.org/post/xwlzzeyvatwsohqcynka forum.dlang.org
Mar 22 2017
parent reply Shachar Shemesh <shachar weka.io> writes:
On 22/03/17 11:08, Kagamin wrote:
 On Tuesday, 21 March 2017 at 10:27:27 UTC, Andrei Alexandrescu wrote:
 Thanks Yuxuan, sorry for missing this. Can we have this peer reviewed
 by 1-2 crypto experts? Thanks! -- Andrei
It's not recommended to use system CSPRNG for non-cryptographic purposes: https://forum.dlang.org/post/xwlzzeyvatwsohqcynka forum.dlang.org
That does not apply here. We're not talking about using the random pool for generating random numbers. Only for generating the random algorithm's seed. Shachar
Mar 22 2017
parent Kagamin <spam here.lot> writes:
On Wednesday, 22 March 2017 at 09:12:14 UTC, Shachar Shemesh 
wrote:
 That does not apply here. We're not talking about using the 
 random pool for generating random numbers. Only for generating 
 the random algorithm's seed.
See OP: On Sunday, 26 February 2017 at 18:23:27 UTC, cym13 wrote:
 Some even go as far as reseeding at each call to try making it 
 more secure.
Also MonoTime resets on every reboot.
Mar 22 2017
prev sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Mon, 06 Mar 2017 22:04:44 -0500
schrieb "Nick Sabalausky (Abscissa)"
<SeeWebsiteToContactMe semitwist.com>:

 On 03/06/2017 05:19 PM, sarn wrote:
 On Monday, 6 March 2017 at 10:12:09 UTC, Shachar Shemesh wrote:  
 Excuse me if I'm asking a trivial question. Why not just seed it
 from /dev/urandom? (or equivalent on non-Linux platforms. I know
 at least Windows has an equivalent).

 Shachar  
One reason is that /dev/urandom isn't always available, e.g., in a chroot. Sure, these are corner cases, but it's annoying when stuff like this doesn't "just work".
I don't claim to be any sort of linux expert or anything, but doesn't chroot have a reputation for being a bit of a finicky, leaky abstraction anyway? I haven't really used them, but that's been my understanding...?
chroots were used for security stuff in the past (chrooting a ftp server and similar stuff) and they're indeed a leaky abstraction in that case. However, chroots can also be used to 'chroot into another OS'. E.g. people sometimes chroot into the OS on a harddisk from a livecd. This is sometimes useful to repair a system, install packages, ... -- Johannes
Mar 06 2017
parent Shachar Shemesh <shachar weka.io> writes:
On 07/03/17 09:13, Johannes Pfau wrote:
 chroots were used for security stuff in the past (chrooting a ftp
 server and similar stuff) and they're indeed a leaky abstraction in
 that case.
I'm not aware of an attack against a process chrooting and dropping privileges (in other words, I'm not aware of a way to escape a chroot jail unless you have root access). Care to provide counter examples? Thanks, Shachar
Mar 06 2017
prev sibling parent Shachar Shemesh <shachar weka.io> writes:
On 07/03/17 00:19, sarn wrote:
 On Monday, 6 March 2017 at 10:12:09 UTC, Shachar Shemesh wrote:
 Excuse me if I'm asking a trivial question. Why not just seed it from
 /dev/urandom? (or equivalent on non-Linux platforms. I know at least
 Windows has an equivalent).

 Shachar
One reason is that /dev/urandom isn't always available, e.g., in a chroot. Sure, these are corner cases, but it's annoying when stuff like this doesn't "just work".
If I write a program that relies on there being /dev/null, /dev/zero or /dev/tty, I don't go ahead and publish this on the program requirements. It is okay to assume those are available to me, and if your chroot/LFS install doesn't have them and my program breaks, well, tough to be you. I don't think /dev/urandom is any different in that regard. Worst case, you can try to use it, and fall back to whatever it is we're doing today if it doesn't work. I don't think the chroot argument is a good one, though. Shachar
Mar 06 2017