www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Cryptography and D

reply "Charles" <charles.hoskinson gmail.com> writes:
Is there a native D crypto library like Crypto++?
Jun 28 2014
parent reply "Adam Wilson" <flyboynw gmail.com> writes:
On Sat, 28 Jun 2014 23:08:51 -0700, Charles <charles.hoskinson gmail.com>  
wrote:


 Is there a native D crypto library like Crypto++?


No. And for good reason. Building a cryptography library is an extremely  
dificult proposition. Even after you've completed the build, you still  
face a trust problem. You need to convince people that your library is not  
subject to a myriad of side-channel attacks. The only way to do that is to  
battle-test is, which requires that people use it in the first place. The  
philosophy of the D community is to binding to more trusted and tested  
libraries.

I currently am working on one such binding to the Botan library called  
Titanium.

https://github.com/ellipticbit/titanium
https://github.com/randombit/botan/

Botan isn't as battle-tested as OpenSSL or Crypto++ but it was designed  
 from the ground up to mitigate or prevent the kind of problems that  
OpenSSL is currently experiencing, and was implemented by someone who has  
done multiple Cryptographic Module Verifications for NIST. I personally  
went with Botan because it's relatively easy to create bindings for the  
languages I use and API is relatively sane for a crypto library (I'm  
looking at you OpenSSL).

-- 
Adam Wilson
GitHub/IRC: LightBender
Aurora Project Coordinator
Jun 29 2014
next sibling parent reply Brad Roberts via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 6/29/14, 12:19 AM, Adam Wilson via Digitalmars-d wrote:

 On Sat, 28 Jun 2014 23:08:51 -0700, Charles <charles.hoskinson gmail.com>
wrote:


 Is there a native D crypto library like Crypto++?


 No. And for good reason. Building a cryptography library is an extremely
dificult proposition. Even
 after you've completed the build, you still face a trust problem. You need to
convince people that
 your library is not subject to a myriad of side-channel attacks. The only way
to do that is to
 battle-test is, which requires that people use it in the first place. The
philosophy of the D
 community is to binding to more trusted and tested libraries.

 I currently am working on one such binding to the Botan library called
Titanium.

 https://github.com/ellipticbit/titanium
 https://github.com/randombit/botan/

 Botan isn't as battle-tested as OpenSSL or Crypto++ but it was designed from
the ground up to
 mitigate or prevent the kind of problems that OpenSSL is currently
experiencing, and was implemented
 by someone who has done multiple Cryptographic Module Verifications for NIST.
I personally went with
 Botan because it's relatively easy to create bindings for the languages I use
and API is relatively
 sane for a crypto library (I'm looking at you OpenSSL).


A safe rule of thumb with crypto code / libraries:  If the thought of writing
that type of code 
doesn't scare you for at least a dozen or so reasons, you don't know enough to
tread in that 
playground.  Or you're one of the damned few people on the planet qualified and
are already working 
on one.
Jun 29 2014
next sibling parent reply "Tobias Pankrath" <tobias pankrath.net> writes:
 A safe rule of thumb with crypto code / libraries:  If the 
 thought of writing that type of code doesn't scare you for at 
 least a dozen or so reasons, you don't know enough to tread in 
 that playground.  Or you're one of the damned few people on the 
 planet qualified and are already working on one.


The best way to become one of these damned few people is getting 
started though.

No need to scare the newcomers off. Everyone was a newcomer once.
Jun 29 2014
parent reply "Chris Cain" <zshazz gmail.com> writes:
On Sunday, 29 June 2014 at 09:24:39 UTC, Tobias Pankrath wrote:

 The best way to become one of these damned few people is 
 getting started though.


If "getting started" means go to college and get a doctorate for 
Crypto, I agree.

If "getting started" means write some crypto libraries until you 
get it right, I'm running away from this topic in horror.
Jun 29 2014
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 6/29/2014 12:53 PM, Chris Cain wrote:

 On Sunday, 29 June 2014 at 09:24:39 UTC, Tobias Pankrath wrote:

 The best way to become one of these damned few people is getting
 started though.


 If "getting started" means go to college and get a doctorate for Crypto,
 I agree.

 If "getting started" means write some crypto libraries until you get it
 right, I'm running away from this topic in horror.


The crypto algorithms are very well defined and documented. You don't 
need to understand the theory behind them in order to implement them. 
You just need to be able to:

- Read/follow the spec accurately
- NOT invent your own variants/algorithms
- Be pedantic about avoiding the normal sets of potential software 
exploits (as you would with any software that handles sensitive data).
- Write/use sufficiently pedantic tests
- Be up-to-date on what's algos are considered outdated and questionably 
secure.

This is a standard "scientist vs engineer" issue. The crypto experts are 
the scientists who figured it all out. We're the engineers who take 
their information and use it.

Obviously being well-versed in crypto theory *in addition* to everything 
above is even better still, but it isn't essential. The five critica 
above are essential.
Jun 29 2014
next sibling parent reply "Xinok" <xinok live.com> writes:
On Sunday, 29 June 2014 at 17:45:41 UTC, Nick Sabalausky wrote:

 The crypto algorithms are very well defined and documented. You 
 don't need to understand the theory behind them in order to 
 implement them. You just need to be able to:

 - Read/follow the spec accurately
 - NOT invent your own variants/algorithms
 - Be pedantic about avoiding the normal sets of potential 
 software exploits (as you would with any software that handles 
 sensitive data).
 - Write/use sufficiently pedantic tests
 - Be up-to-date on what's algos are considered outdated and 
 questionably secure.

 This is a standard "scientist vs engineer" issue. The crypto 
 experts are the scientists who figured it all out. We're the 
 engineers who take their information and use it.

 Obviously being well-versed in crypto theory *in addition* to 
 everything above is even better still, but it isn't essential. 
 The five critica above are essential.


There's so much more to securely implementing cryptography than 
what you listed. I highly recommend reading about side-channel 
attacks:

https://en.wikipedia.org/wiki/Side-channel_attack

https://www.schneier.com/crypto-gram-9806.html#side

Proper cryptographic libraries are written in such a way to 
mitigate these types of attacks. It's a complex field of study 
and something best left to the experts.
Jun 29 2014
parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 29 June 2014 at 18:47:56 UTC, Xinok wrote:

 Proper cryptographic libraries are written in such a way to 
 mitigate these types of attacks. It's a complex field of study 
 and something best left to the experts.


Current crypto libs, aren't capable of doing bound checking 
properly, that should raise in you some doubt about how they 
handle side channels.
Jul 05 2014
prev sibling parent reply "Chris Cain" <zshazz gmail.com> writes:
On Sunday, 29 June 2014 at 17:45:41 UTC, Nick Sabalausky wrote:

 The crypto algorithms are very well defined and documented. You 
 don't need to understand the theory behind them in order to 
 implement them. You just need to be able to:

 - Read/follow the spec accurately
 - NOT invent your own variants/algorithms
 - Be pedantic about avoiding the normal sets of potential 
 software exploits (as you would with any software that handles 
 sensitive data).
 - Write/use sufficiently pedantic tests
 - Be up-to-date on what's algos are considered outdated and 
 questionably secure.

 This is a standard "scientist vs engineer" issue. The crypto 
 experts are the scientists who figured it all out. We're the 
 engineers who take their information and use it.

 Obviously being well-versed in crypto theory *in addition* to 
 everything above is even better still, but it isn't essential. 
 The five critica above are essential.


Of course, following all of those suggestions isn't trivial to 
begin with. Technically, you're right, but because what you said 
isn't easy to follow to begin with, it doesn't support the 
argument of "you can implement a crypto algorithm."

I've seen not-so-subtle violations of "follow the spec 
accurately" and it's especially easy to do in C/C++ where 
"undefined behavior" will cause the compiler to rewrite your 
program in sometimes very unpredictable ways. Sure, that 
situation is better in D, but the precedence is that to suggest 
that any implementation of any crypto algorithm must, at minimum, 
be studied and criticized by several experts in both crypto (to 
verify you're logically following the spec) and experts in the 
language itself (to verify that what you have typed is guaranteed 
to ultimately be accurately represented in machine code).

Basically, if you have data you must have secured (the reason why 
you'd use a crypto algorithm to begin with), you must go beyond a 
sane level of pedantry. The only acceptable insane level of 
pedantry I know of is only possible with people that have 
doctorates in cryptography. :)

Plus, add what Xinok said. That's showing the level of pedantry 
we're talking about with crypto where you have to cover things 
like timing attacks and power analysis (or, admit that your 
crypto library isn't suitable for covering those attacks).

That's not to say you shouldn't ever do it, but you really need 
to truly understand what it is you're doing when you implement 
any crypto. Even using crypto requires a certain (often ignored) 
level of knowledge or you introduce issues.
Jun 29 2014
next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 6/29/2014 3:25 PM, Chris Cain wrote:

 On Sunday, 29 June 2014 at 17:45:41 UTC, Nick Sabalausky wrote:

 The crypto algorithms are very well defined and documented. You don't
 need to understand the theory behind them in order to implement them.
 You just need to be able to:

 - Read/follow the spec accurately
 - NOT invent your own variants/algorithms
 - Be pedantic about avoiding the normal sets of potential software
 exploits (as you would with any software that handles sensitive data).
 - Write/use sufficiently pedantic tests
 - Be up-to-date on what's algos are considered outdated and
 questionably secure.

 This is a standard "scientist vs engineer" issue. The crypto experts
 are the scientists who figured it all out. We're the engineers who
 take their information and use it.

 Obviously being well-versed in crypto theory *in addition* to
 everything above is even better still, but it isn't essential. The
 five critica above are essential.


 Of course, following all of those suggestions isn't trivial to begin
 with. Technically, you're right, but because what you said isn't easy to
 follow to begin with, it doesn't support the argument of "you can
 implement a crypto algorithm."
[...]


Most of what you and Xinok said is certainly right. I was mainly 
objecting to the notion that having formal background in 
cryptographic-therory (or even an informal/autodidactic background in 
crypto theory, for that matter) is a particularly important part of 
implementing a crypto algorithm. (Although again, I'm not saying it 
couldn't be helpful).

Addressing things such as the various side-channel attacks are certainly 
important for a crypto lib, and non-trivial. But they are not directly 
part of cryptographic theory, nor is their importance limited to 
cryptographic algorithms (For example, thwarting timing attacks is a 
prudent measure even when comparing password hashes which have *already* 
been computed via the crypto hash algorithm).


 any implementation of any crypto
 algorithm must, at minimum, be studied and criticized by several experts
 in both crypto (to verify you're logically following the spec) and
 experts in the language itself (to verify that what you have typed is
 guaranteed to ultimately be accurately represented in machine code).


Sure, I can buy that.

Although, naturally, the only way to get such critical analysis 
performed on an implementation is to start by creating an implementation 
in the first place :) Gotta start somewhere.

Besides, if intelligent people scare themselves away from trying, then 
the only people implementing them would be 1% super-experts and 99% 
people too unqualified to even *realize* they don't know what they're 
doing ;)

Additionally, given how widespread heartbleed was, I think it's clear 
that having more crypto implementations in the wild is a good thing - it 
would limit the potential reach of damage from flaws in any one 
particular implementation. Diversity in the digital gene pool, so to 
speak. (Assuming they're of suitable quality, of course, but again: 
gotta start somewhere, can't analyze an implementation that doesn't exist.)

Speaking of which, it certainly wouldn't hurt to get more expert-level 
eyes on std.digest.*, including the recently added SHA-2 support.


 only possible with people that have doctorates in cryptography. :)


Not to get too pedantic (too late? ;) ), but doctorates (like other 
degrees) are merely certification. The important thing is actual 
expertise. Degrees, at their core, are nothing more than an [expensive] 
attempt to *indicate* such expertise, and are highly prone to both false 
positives and false negatives.
Jun 29 2014
prev sibling parent reply "Kagamin" <spam here.lot> writes:
On Sunday, 29 June 2014 at 19:25:30 UTC, Chris Cain wrote:

 Of course, following all of those suggestions isn't trivial to 
 begin with. Technically, you're right, but because what you 
 said isn't easy to follow to begin with, it doesn't support the 
 argument of "you can implement a crypto algorithm."


Following those guidelines don't require knowledge in 
cryptography. And D automtically provides good level of pedantism 
of bound checking, so this part should be easier.


 Basically, if you have data you must have secured (the reason 
 why you'd use a crypto algorithm to begin with), you must go 
 beyond a sane level of pedantry. The only acceptable insane 
 level of pedantry I know of is only possible with people that 
 have doctorates in cryptography. :)

 Plus, add what Xinok said. That's showing the level of pedantry 
 we're talking about with crypto where you have to cover things 
 like timing attacks and power analysis (or, admit that your 
 crypto library isn't suitable for covering those attacks).

 That's not to say you shouldn't ever do it, but you really need 
 to truly understand what it is you're doing when you implement 
 any crypto. Even using crypto requires a certain (often 
 ignored) level of knowledge or you introduce issues.


There was a study, showing that most security vulnerabilities are 
caused by client code rather than cryptographic library code. For 
example, how would you prevent client code from generating weak 
encryption keys or from using weak algorithm for hash comparison, 
or how would you force it to do what's not required to get the 
code compiled? How would you do that with the quality of library 
code? Even if you can do that, it's still not a cryptographic 
task, but a general programming task, the standard only hints you 
that those things are necessary.
Jul 05 2014
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 7/5/2014 8:23 AM, Kagamin wrote:

 There was a study, showing that most security vulnerabilities are caused
 by client code rather than cryptographic library code.


Interesting. Link?


 For example, how
 would you prevent client code from generating weak encryption keys or
 from using weak algorithm for hash comparison, or how would you force it
 to do what's not required to get the code compiled? How would you do
 that with the quality of library code? Even if you can do that, it's
 still not a cryptographic task, but a general programming task, the
 standard only hints you that those things are necessary.


FWIW, DAuth (pending a possible name change, to prevent confusion with 
the completely unrelated OAuth) maintains a list of non-recommended algos:

   https://github.com/Abscissa/DAuth/blob/master/src/dauth/core.d#L109

And, by default, it will error out (at compile-time whenever possible) 
if the application code attempts to use any such algorithm:

   https://github.com/Abscissa/DAuth/blob/master/src/dauth/core.d#L169

Getting around the error requires acknowledging your intent via 
-version=DAuth_AllowWeakSecurity

Also, DAuth encourages passwords to be stored in a special structure:

   https://github.com/Abscissa/DAuth/blob/master/src/dauth/core.d#L311

which attempts to zero-out the password from memory as early as it can 
(and encourages the user to populate it via char[] not string to avoid 
having an un-wipable immutable plain-text copy in memory. See 
'toPassword' vs 'dupPassword'). I'm certain the implementation can be 
improved. And I'd kinda like to make it scoped if I can, instead of 
refcounted. But it's something.

Obviously there are natural limits to these measures, so it can't 
*guarantee* anything, only help guide the application developer. And it 
doesn't/can't address all possible issues. But it's at least something.
Jul 05 2014
next sibling parent reply Rikki Cattermole <alphaglosined gmail.com> writes:
On 6/07/2014 6:06 a.m., Nick Sabalausky wrote:

 On 7/5/2014 8:23 AM, Kagamin wrote:

 There was a study, showing that most security vulnerabilities are caused
 by client code rather than cryptographic library code.


 Interesting. Link?


 For example, how
 would you prevent client code from generating weak encryption keys or
 from using weak algorithm for hash comparison, or how would you force it
 to do what's not required to get the code compiled? How would you do
 that with the quality of library code? Even if you can do that, it's
 still not a cryptographic task, but a general programming task, the
 standard only hints you that those things are necessary.


 FWIW, DAuth (pending a possible name change, to prevent confusion with
 the completely unrelated OAuth) maintains a list of non-recommended algos:

    https://github.com/Abscissa/DAuth/blob/master/src/dauth/core.d#L109

 And, by default, it will error out (at compile-time whenever possible)
 if the application code attempts to use any such algorithm:

    https://github.com/Abscissa/DAuth/blob/master/src/dauth/core.d#L169

 Getting around the error requires acknowledging your intent via
 -version=DAuth_AllowWeakSecurity

 Also, DAuth encourages passwords to be stored in a special structure:

    https://github.com/Abscissa/DAuth/blob/master/src/dauth/core.d#L311

 which attempts to zero-out the password from memory as early as it can
 (and encourages the user to populate it via char[] not string to avoid
 having an un-wipable immutable plain-text copy in memory. See
 'toPassword' vs 'dupPassword'). I'm certain the implementation can be
 improved. And I'd kinda like to make it scoped if I can, instead of
 refcounted. But it's something.

 Obviously there are natural limits to these measures, so it can't
 *guarantee* anything, only help guide the application developer. And it
 doesn't/can't address all possible issues. But it's at least something.


For reference, Cmsed's password authentication that is built in will 
automatically hash passwords and will not store in any way the plain 
text version [0].
Also supports upgrading hashes as needed. From older algorithms to newer 
preferred ones.

[0] 
https://github.com/rikkimax/Cmsed/blob/master/source/user/cmsed/user/models/userauth.d#L57
Jul 05 2014
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 7/6/2014 1:00 AM, Rikki Cattermole wrote:

 On 6/07/2014 6:06 a.m., Nick Sabalausky wrote:

 Also, DAuth encourages passwords to be stored in a special structure:

    https://github.com/Abscissa/DAuth/blob/master/src/dauth/core.d#L311

 which attempts to zero-out the password from memory as early as it can
 (and encourages the user to populate it via char[] not string to avoid
 having an un-wipable immutable plain-text copy in memory. See
 'toPassword' vs 'dupPassword'). I'm certain the implementation can be
 improved. And I'd kinda like to make it scoped if I can, instead of
 refcounted. But it's something.

 Obviously there are natural limits to these measures, so it can't
 *guarantee* anything, only help guide the application developer. And it
 doesn't/can't address all possible issues. But it's at least something.


 For reference, Cmsed's password authentication that is built in will
 automatically hash passwords and will not store in any way the plain
 text version [0].
 Also supports upgrading hashes as needed. From older algorithms to newer
 preferred ones.

 [0]
 https://github.com/rikkimax/Cmsed/blob/master/source/user/cmsed/user/models/userauth.d#L57


Unfortunately, that doesn't really address the issue. The password 
always has to come from somewhere in plaintext. By the time it gets to 
*any* hashing function it's already existed in memory in plaintext.

No offense intended, but here's the thing:

That "class UserPassword" cannot even be *used* until the plaintext 
password already exists in memory. It must get passed into UserPassword 
(via opAssign) as a string. Problem is, AFAICS, nothing appears to be 
removing that original plaintext copy from memory. Once it's in a 
UserPassword object, the original copy *still* sits around waiting to 
[maybe/eventually] get garbage collected or read by some bug/exploit.

Worse, that plaintext data is marked as immutable (because it's a 
string), so it *cannot* be safely wiped from memory without breaking the 
type system.

The idea behind the auto-zeroing Password type in DAuth: you do your 
best to give it [hopefully] the one and only copy of the plaintext 
password data, in mutable form, and DAuth nukes it from memory as soon 
as it's able to. Not if/when GC kicks in, sweeps it, and reuses it. And 
not just a copy of the data, but the actual [hopefully] original buffer. 
Any further attempts (deliberate or accidental) to read that data 
through any other reference will only receive zeros.

On a different note though, it does seem I may have been hasty in 
grouping RIPEMD in the "weak" category. I was unfamiliar with it and 
assumed it was on par with MD5. A brief search seems to suggest that may 
not necessarily be the case.
Jul 05 2014
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/5/2014 11:45 PM, Nick Sabalausky wrote:

 The idea behind the auto-zeroing Password type in DAuth: you do your best to
 give it [hopefully] the one and only copy of the plaintext password data, in
 mutable form, and DAuth nukes it from memory as soon as it's able to. Not
 if/when GC kicks in, sweeps it, and reuses it. And not just a copy of the data,
 but the actual [hopefully] original buffer. Any further attempts (deliberate or
 accidental) to read that data through any other reference will only receive
zeros.


It's an interesting possible application for a 'unique' reference type.
Jul 06 2014
prev sibling parent reply Rikki Cattermole <alphaglosined gmail.com> writes:
On 6/07/2014 6:45 p.m., Nick Sabalausky wrote:

 On 7/6/2014 1:00 AM, Rikki Cattermole wrote:

 On 6/07/2014 6:06 a.m., Nick Sabalausky wrote:

 Also, DAuth encourages passwords to be stored in a special structure:

    https://github.com/Abscissa/DAuth/blob/master/src/dauth/core.d#L311

 which attempts to zero-out the password from memory as early as it can
 (and encourages the user to populate it via char[] not string to avoid
 having an un-wipable immutable plain-text copy in memory. See
 'toPassword' vs 'dupPassword'). I'm certain the implementation can be
 improved. And I'd kinda like to make it scoped if I can, instead of
 refcounted. But it's something.

 Obviously there are natural limits to these measures, so it can't
 *guarantee* anything, only help guide the application developer. And it
 doesn't/can't address all possible issues. But it's at least something.


 For reference, Cmsed's password authentication that is built in will
 automatically hash passwords and will not store in any way the plain
 text version [0].
 Also supports upgrading hashes as needed. From older algorithms to newer
 preferred ones.

 [0]
 https://github.com/rikkimax/Cmsed/blob/master/source/user/cmsed/user/models/userauth.d#L57


 Unfortunately, that doesn't really address the issue. The password
 always has to come from somewhere in plaintext. By the time it gets to
 *any* hashing function it's already existed in memory in plaintext.

 No offense intended, but here's the thing:

 That "class UserPassword" cannot even be *used* until the plaintext
 password already exists in memory. It must get passed into UserPassword
 (via opAssign) as a string. Problem is, AFAICS, nothing appears to be
 removing that original plaintext copy from memory. Once it's in a
 UserPassword object, the original copy *still* sits around waiting to
 [maybe/eventually] get garbage collected or read by some bug/exploit.

 Worse, that plaintext data is marked as immutable (because it's a
 string), so it *cannot* be safely wiped from memory without breaking the
 type system.

 The idea behind the auto-zeroing Password type in DAuth: you do your
 best to give it [hopefully] the one and only copy of the plaintext
 password data, in mutable form, and DAuth nukes it from memory as soon
 as it's able to. Not if/when GC kicks in, sweeps it, and reuses it. And
 not just a copy of the data, but the actual [hopefully] original buffer.
 Any further attempts (deliberate or accidental) to read that data
 through any other reference will only receive zeros.


My reasoning is simple. Only upon setting the password for a user will 
the plaintext version ever be in memory or when logging in for 
authentication.
Now this should be coming as a parameter via routes in some format as a 
string. If this cannot be caught by the GC and cleaned up, how exactly 
does your method help?

So as far as I'm concerned, if it cannot be cleaned up, there are bigger 
problems to worry about then just securing this one tiny piece of code.

Unless of course, we want some method to guard against this e.g. all 
data given to routes in Vibe is actually protected and will be 
forcefully freed upon finishing of the route.


 On a different note though, it does seem I may have been hasty in
 grouping RIPEMD in the "weak" category. I was unfamiliar with it and
 assumed it was on par with MD5. A brief search seems to suggest that may
 not necessarily be the case.


To me it was one of the *better* options available when I wrote it, 
hence it was my initial one.
Jul 06 2014
parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 7/6/2014 4:22 AM, Rikki Cattermole wrote:

 On 6/07/2014 6:45 p.m., Nick Sabalausky wrote:

 The idea behind the auto-zeroing Password type in DAuth: you do your
 best to give it [hopefully] the one and only copy of the plaintext
 password data, in mutable form, and DAuth nukes it from memory as soon
 as it's able to. Not if/when GC kicks in, sweeps it, and reuses it. And
 not just a copy of the data, but the actual [hopefully] original buffer.
 Any further attempts (deliberate or accidental) to read that data
 through any other reference will only receive zeros.


 My reasoning is simple. Only upon setting the password for a user will
 the plaintext version ever be in memory or when logging in for
 authentication.
 Now this should be coming as a parameter via routes in some format as a
 string. If this cannot be caught by the GC and cleaned up, how exactly
 does your method help?

 So as far as I'm concerned, if it cannot be cleaned up, there are bigger
 problems to worry about then just securing this one tiny piece of code.


I wanted to keep the door open for, and help encourage, applications 
that *do* avoid having a password in an immutable buffer.

I realize that your library relies on vibe anyway, so in your case, yes 
this measure wouldn't help without corresponding changes within vibe.d. 
That is a fair point. But DAuth (again, I may want to change the name), 
is designed to be a general-purpose password-based user authentication 
tool, reusable from any sort of program, so that situation/limitation 
doesn't necessarily apply to DAuth.


 Unless of course, we want some method to guard against this e.g. all
 data given to routes in Vibe is actually protected and will be
 forcefully freed upon finishing of the route.


That's actually an interesting idea. I had been imagining a possible 
enhancement to vibe that would handle passwords separately, but 
depending on the application, a password isn't necessarily the only 
sensitive data sitting around.
Jul 06 2014
prev sibling next sibling parent reply "Kagamin" <spam here.lot> writes:
On Saturday, 5 July 2014 at 18:06:34 UTC, Nick Sabalausky wrote:

 On 7/5/2014 8:23 AM, Kagamin wrote:

 There was a study, showing that most security vulnerabilities 
 are caused
 by client code rather than cryptographic library code.


 Interesting. Link?


Dunno, can't find it right now. I thought, I found it following 
links from hap thread, but there's nothing there.


 For example, how
 would you prevent client code from generating weak encryption 
 keys or
 from using weak algorithm for hash comparison, or how would 
 you force it
 to do what's not required to get the code compiled? How would 
 you do
 that with the quality of library code? Even if you can do 
 that, it's
 still not a cryptographic task, but a general programming 
 task, the
 standard only hints you that those things are necessary.


 FWIW, DAuth (pending a possible name change, to prevent 
 confusion with the completely unrelated OAuth) maintains a list 
 of non-recommended algos:

   
 https://github.com/Abscissa/DAuth/blob/master/src/dauth/core.d#L109


Cryptographic algorithms don't cause cryptographic weaknesses as 
bad as ones from user code.
Example: http://samate.nist.gov/SRD/view_testcase.php?tID=58


 Also, DAuth encourages passwords to be stored in a special 
 structure:

   
 https://github.com/Abscissa/DAuth/blob/master/src/dauth/core.d#L311

 which attempts to zero-out the password from memory as early as 
 it can (and encourages the user to populate it via char[] not 
 string to avoid having an un-wipable immutable plain-text copy 
 in memory. See 'toPassword' vs 'dupPassword'). I'm certain the 
 implementation can be improved. And I'd kinda like to make it 
 scoped if I can, instead of refcounted. But it's something.


Yeah, better than nothing, but as it integrates poorly with the 
rest of user code, people will hack it around by writing byte[] 
hashPassword(string) function. Nobody estimates security by 
defending the system, one do it by breaking it.
Jul 06 2014
parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 7/6/2014 9:38 AM, Kagamin wrote:

 On Saturday, 5 July 2014 at 18:06:34 UTC, Nick Sabalausky wrote:

 On 7/5/2014 8:23 AM, Kagamin wrote:

 There was a study, showing that most security vulnerabilities are caused
 by client code rather than cryptographic library code.


 Interesting. Link?


 Dunno, can't find it right now. I thought, I found it following links
 from hap thread, but there's nothing there.


 For example, how
 would you prevent client code from generating weak encryption keys or
 from using weak algorithm for hash comparison, or how would you force it
 to do what's not required to get the code compiled? How would you do
 that with the quality of library code? Even if you can do that, it's
 still not a cryptographic task, but a general programming task, the
 standard only hints you that those things are necessary.


 FWIW, DAuth (pending a possible name change, to prevent confusion with
 the completely unrelated OAuth) maintains a list of non-recommended
 algos:

 https://github.com/Abscissa/DAuth/blob/master/src/dauth/core.d#L109


 Cryptographic algorithms don't cause cryptographic weaknesses as bad as
 ones from user code.
 Example: http://samate.nist.gov/SRD/view_testcase.php?tID=58


Well, of course there are algorithms that *are* inherently insecure at 
the algorithmic level, regardless of implementation. Like MD4/MD5, 
CRC32, non-crypto DRNGs, etc. But yea, user code is a big giant attack 
vector.


 Also, DAuth encourages passwords to be stored in a special structure:

 https://github.com/Abscissa/DAuth/blob/master/src/dauth/core.d#L311

 which attempts to zero-out the password from memory as early as it can
 (and encourages the user to populate it via char[] not string to avoid
 having an un-wipable immutable plain-text copy in memory. See
 'toPassword' vs 'dupPassword'). I'm certain the implementation can be
 improved. And I'd kinda like to make it scoped if I can, instead of
 refcounted. But it's something.


 Yeah, better than nothing, but as it integrates poorly with the rest of
 user code, people will hack it around by writing byte[]
 hashPassword(string) function. Nobody estimates security by defending
 the system, one do it by breaking it.


Yea, it *is* a calculated compromise. Definitely. It doesn't prevent 
anyone from doing the wrong thing since it really *can't* prevent it.

But what it does do is attempt to nudge the user in the right direction 
the best it can. Namely, a function is *already* provided to directly 
construct a Password struct from a string: dupPassword (as opposed to 
the recommended toPassword which requires a mutable char array).

Granted, this dupPassword is a bad thing to use, but just like unsafe 
casts, people will occasionally need it or otherwise think they need it. 
(Heck, *I've* needed to use it, just because vibe provides its POST vars 
in string form.) So by providing this function directly (instead of 
omitting dupPassword or just accepting "string password" everywhere), 
I'm hoping to:

1. Increase the likelihood people are at least *aware* of what they're 
doing: Using a discouraged function (the docs warn about its use) and 
"dup"ing a plaintext password that will still remain in memory.

2. Decrease the likelihood of homemade workarounds that may not be as 
easily identifiable or greppable.

It's no guarantee of anything, since that's wouldn't be realistic 
anyway. But it's a nudge in the right direction, which is the next best 
thing.
Jul 06 2014
prev sibling parent "Kagamin" <spam here.lot> writes:
On Saturday, 5 July 2014 at 18:06:34 UTC, Nick Sabalausky wrote:

 On 7/5/2014 8:23 AM, Kagamin wrote:

 There was a study, showing that most security vulnerabilities 
 are caused
 by client code rather than cryptographic library code.


 Interesting. Link?


http://people.csail.mit.edu/nickolai/papers/lazar-cryptobugs.pdf
Jul 26 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/29/2014 1:12 AM, Brad Roberts via Digitalmars-d wrote:

 A safe rule of thumb with crypto code / libraries:  If the thought of writing
 that type of code doesn't scare you for at least a dozen or so reasons, you
 don't know enough to tread in that playground.  Or you're one of the damned few
 people on the planet qualified and are already working on one.


In a way, it's a bit like writing FP math functions. If using a college
calculus 
book as an implementation guide, the results will be algorithmically correct
but 
quite wrong, as FP doesn't behave like math. Getting them right is something 
that PhD's in comp sci do.

Crypto takes that and squares the difficulty.

If someone wants to learn about crypto by writing crypto libraries in D, I'm
all 
for it.

But I'm not willing to make those part of the official D repository, with its 
implicit endorsement that people could rely on them for critical applications 
such as, say, banking transactions.

What we can do is simply provide a D interface to existing, vetted C crypto 
libraries. The Deimos project is a fine place for those.
Jul 05 2014
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 7/5/2014 5:10 PM, Walter Bright wrote:

 On 6/29/2014 1:12 AM, Brad Roberts via Digitalmars-d wrote:

 A safe rule of thumb with crypto code / libraries:  If the thought of
 writing
 that type of code doesn't scare you for at least a dozen or so
 reasons, you
 don't know enough to tread in that playground.  Or you're one of the
 damned few
 people on the planet qualified and are already working on one.


 In a way, it's a bit like writing FP math functions. If using a college
 calculus book as an implementation guide, the results will be
 algorithmically correct but quite wrong, as FP doesn't behave like math.
 Getting them right is something that PhD's in comp sci do.

 Crypto takes that and squares the difficulty.


Inaccurate comparison.

First of all, FP math is an approximation of "true" real-number 
mathematics. But crypto algorithms (at least any of the ones I've seen, 
and definitely the ones I've implemented) are based on the same discrete 
integer math used by computers and low-level code. NIST's docs even go 
out of their way to formally describe all the nuances of the operations 
their algos use which, surprise, are identical to CPU integer arithmetic 
(at least x86 anyway, I'm not well-versed on ARM but I'd be very 
surprised if it's really *that* different).

Secondly, if the difficulty you're referring to is all the various 
side-channel attacks (like buffer overflows, timing attack, etc), then 
you're conflating crypto and security. The two are certainly related, 
but they are NOT the same. These "side-channel attack" issues are NOT 
something specific to crypto code, they are equally applicable to *all* 
code that has security-related implications, which includes FAR more 
than just crypto. It goes all the way up through application-level code, 
and even into system administration. Implementing a crypto algorithm is 
(comparatively speaking, of course) the "easy" part.


 If someone wants to learn about crypto by writing crypto libraries in D,
 I'm all for it.

 But I'm not willing to make those part of the official D repository,
 with its implicit endorsement that people could rely on them for
 critical applications such as, say, banking transactions.

 What we can do is simply provide a D interface to existing, vetted C
 crypto libraries. The Deimos project is a fine place for those.


With respect: Oh hell no. No, no, no no no.

1. As others have already said, C is terrible for security libs. Even if 
D isn't ideal, it's still vastly better.

2. Just because a C-based crypto lib is popular and well-regarded 
doesn't mean it's as secure and reliable as people think. See what 
others here have said about OpenSSL (hint: Heartbleed is just *one* of 
the issues).

3. Too late anyway: See std.digest. Besides, if anything, std.digest is 
arguably *worse* because (until 2.066) it only provides the worst 
choices. std.random isn't much better. Granted, it doesn't claim to be 
crypto-grade, but it doesn't clearly state that it *isn't* and that's 
just as bad: People are going to to decide (incorrectly) they can use it 
to generate salts or tokens or whatever, and they will do so. Heck, 
*I've* even done it, and *I'm* someone who actually knows better.
Jul 05 2014
parent reply "Kagamin" <spam here.lot> writes:
On Saturday, 5 July 2014 at 21:50:59 UTC, Nick Sabalausky wrote:

 Secondly, if the difficulty you're referring to is all the 
 various side-channel attacks (like buffer overflows, timing 
 attack, etc), then you're conflating crypto and security. The 
 two are certainly related, but they are NOT the same.


It's called Fleming cryptanalysis :)


 3. Too late anyway: See std.digest. Besides, if anything, 
 std.digest is arguably *worse* because (until 2.066) it only 
 provides the worst choices. std.random isn't much better. 
 Granted, it doesn't claim to be crypto-grade, but it doesn't 
 clearly state that it *isn't* and that's just as bad: People 
 are going to to decide (incorrectly) they can use it to 
 generate salts or tokens or whatever, and they will do so. 
 Heck, *I've* even done it, and *I'm* someone who actually knows 
 better.


The default PRNG is routinely used for salt generation :)
Granted, your library makes it easier to use good salts. Though, 
it needs examples or tutorials, how to actually use the library 
correctly.
Jul 06 2014
parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 7/6/2014 9:49 AM, Kagamin wrote:

 On Saturday, 5 July 2014 at 21:50:59 UTC, Nick Sabalausky wrote:

 3. Too late anyway: See std.digest. Besides, if anything, std.digest
 is arguably *worse* because (until 2.066) it only provides the worst
 choices.




Slight correction: Apparently RIPEMD 160 and up are a lot better than I 
thought. My mind automatically associated it with ~MD5, which I guess is 
an inaccurate comparison.


 std.random isn't much better. Granted, it doesn't claim to be
 crypto-grade, but it doesn't clearly state that it *isn't* and that's
 just as bad: People are going to to decide (incorrectly) they can use
 it to generate salts or tokens or whatever, and they will do so. Heck,
 *I've* even done it, and *I'm* someone who actually knows better.


 The default PRNG is routinely used for salt generation :)
 Granted, your library makes it easier to use good salts. Though, it
 needs examples or tutorials, how to actually use the library correctly.


If this isn't good enough then I'm open to pull requests or more 
specific suggestions:

https://github.com/abscissa/DAuth#typical-usage

Granted, the less typical (ie more heavily-customized) use-cases could 
use some tutorials.

In the expected typical use-case, proper salt generation is completely 
transparent to the lib's user.
Jul 06 2014
prev sibling next sibling parent reply Etienne <etcimon gmail.com> writes:
On 2014-06-29 3:19 AM, Adam Wilson wrote:

 Botan isn't as battle-tested as OpenSSL or Crypto++ but it was designed
 from the ground up to mitigate or prevent the kind of problems that
 OpenSSL is currently experiencing, and was implemented by someone who
 has done multiple Cryptographic Module Verifications for NIST. I
 personally went with Botan because it's relatively easy to create
 bindings for the languages I use and API is relatively sane for a crypto
 library (I'm looking at you OpenSSL).


It also seems to link statically with D projects, although it requires 
MSVC. I'd certainly build a dub package for it, but I'm too busy paving 
the way towards a native TLS library although I intend on using your 
crypto interface with it. ;)
Jun 29 2014
parent "Adam Wilson" <flyboynw gmail.com> writes:
On Sun, 29 Jun 2014 05:33:06 -0700, Etienne <etcimon gmail.com> wrote:


 On 2014-06-29 3:19 AM, Adam Wilson wrote:

 Botan isn't as battle-tested as OpenSSL or Crypto++ but it was designed
 from the ground up to mitigate or prevent the kind of problems that
 OpenSSL is currently experiencing, and was implemented by someone who
 has done multiple Cryptographic Module Verifications for NIST. I
 personally went with Botan because it's relatively easy to create
 bindings for the languages I use and API is relatively sane for a crypto
 library (I'm looking at you OpenSSL).


 It also seems to link statically with D projects, although it requires  
 MSVC. I'd certainly build a dub package for it, but I'm too busy paving  
 the way towards a native TLS library although I intend on using your  
 crypto interface with it. ;)


The static linking is due to the fact that D's shared library support is  
very uneven across platforms. The .NET binding is fully modularized and  
will load the correct Arch at runtime to support the AnyCPU arch.

-- 
Adam Wilson
GitHub/IRC: LightBender
Aurora Project Coordinator
Jun 29 2014
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 29 June 2014 at 07:19:49 UTC, Adam Wilson wrote:

 On Sat, 28 Jun 2014 23:08:51 -0700, Charles 
 <charles.hoskinson gmail.com> wrote:


 Is there a native D crypto library like Crypto++?


 No. And for good reason. Building a cryptography library is an 
 extremely dificult proposition. Even after you've completed the 
 build, you still face a trust problem. You need to convince 
 people that your library is not subject to a myriad of 
 side-channel attacks. The only way to do that is to battle-test 
 is, which requires that people use it in the first place. The 
 philosophy of the D community is to binding to more trusted and 
 tested libraries.


I used to think th
Jul 05 2014
prev sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 29 June 2014 at 07:19:49 UTC, Adam Wilson wrote:

 On Sat, 28 Jun 2014 23:08:51 -0700, Charles 
 <charles.hoskinson gmail.com> wrote:


 Is there a native D crypto library like Crypto++?


 No. And for good reason. Building a cryptography library is an 
 extremely dificult proposition. Even after you've completed the 
 build, you still face a trust problem. You need to convince 
 people that your library is not subject to a myriad of 
 side-channel attacks. The only way to do that is to battle-test 
 is, which requires that people use it in the first place. The 
 philosophy of the D community is to binding to more trusted and 
 tested libraries.


I used to think that. A few years ago, I looked into OpenSSL, 
noticed several horrors. Several of them mentioned here:

https://www.youtube.com/watch?v=GnBbhXBDmwU

I had the same reasoning: crytpo is hard and these guys know much 
more than I do.

They don't. The simple fact they are are using C to build 
security related basic block show that they have no idea what 
they are doing. No bound check, no memory safety, integer 
overflow is undefined behavior (which mean that even if you 
remember to check for it, you are not checking for it).
Jul 05 2014
next sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
05-Jul-2014 23:33, deadalnix пишет:

 On Sunday, 29 June 2014 at 07:19:49 UTC, Adam Wilson wrote:

 On Sat, 28 Jun 2014 23:08:51 -0700, Charles
 <charles.hoskinson gmail.com> wrote:


 Is there a native D crypto library like Crypto++?


 No. And for good reason. Building a cryptography library is an
 extremely dificult proposition. Even after you've completed the build,
 you still face a trust problem. You need to convince people that your
 library is not subject to a myriad of side-channel attacks. The only
 way to do that is to battle-test is, which requires that people use it
 in the first place. The philosophy of the D community is to binding to
 more trusted and tested libraries.


 I used to think that. A few years ago, I looked into OpenSSL, noticed
 several horrors. Several of them mentioned here:

 https://www.youtube.com/watch?v=GnBbhXBDmwU

 I had the same reasoning: crytpo is hard and these guys know much more
 than I do.


Indeed a common misconception and I would recommend for anybody thinking 
otherwise to actually go ahead and read e.g. that damn OpenSSL source 
code. Huge and old C libraries are a security problem in their own 
right, without even looking further for potential theoretical faults.


 They don't. The simple fact they are are using C to build security
 related basic block show that they have no idea what they are doing. No
 bound check, no memory safety, integer overflow is undefined behavior
 (which mean that even if you remember to check for it, you are not
 checking for it).



-- 
Dmitry Olshansky
Jul 05 2014
prev sibling next sibling parent reply "Xinok" <xinok live.com> writes:
On Saturday, 5 July 2014 at 19:33:31 UTC, deadalnix wrote:

 I used to think that. A few years ago, I looked into OpenSSL, 
 noticed several horrors. Several of them mentioned here:

 https://www.youtube.com/watch?v=GnBbhXBDmwU

 I had the same reasoning: crytpo is hard and these guys know 
 much more than I do.

 They don't. The simple fact they are are using C to build 
 security related basic block show that they have no idea what 
 they are doing. No bound check, no memory safety, integer 
 overflow is undefined behavior (which mean that even if you 
 remember to check for it, you are not checking for it).


If you don't trust OpenSSL, nobody said you have to use it. There 
are plenty of alternatives available. The fact still remains, 
implementing your own crypto is a very bad idea.

Why implement a crypto lib in C?

(1) Maximum exposure - If a programming language has more than 
100 users, chances are, there's an OpenSSL binding available for 
that language. C is an ideal language to make something available 
for as many platforms and environments as possible.

(2) Determinism - If your intention is to implement crypto that 
is impervious to side-channel attacks, you need a language that's 
"close to the metal" and will behave how you expect it to. For 
example, Java would be a poor choice because things like garbage 
collection and JITing makes code highly non-deterministic.
Jul 05 2014
parent "deadalnix" <deadalnix gmail.com> writes:
On Saturday, 5 July 2014 at 23:45:47 UTC, Xinok wrote:

 If you don't trust OpenSSL, nobody said you have to use it. 
 There are plenty of alternatives available. The fact still 
 remains, implementing your own crypto is a very bad idea.


It seems to be the consensus. In the meantime, people like Mark 
Karpeles build their own implementation of SSH in PHP, and 
proceed to run a multimillion dollar exchange ( MtGox ).

Building your own crypto is a bad idea. And you know who ignore 
bad idea ? Bad programmers. As a results, they are the one 
building crypto libs. And you know what is a worse idea than 
making your own crypto lib ? Letting Dunning-Kruger lemmings do 
it for you.


 Why implement a crypto lib in C?

 (1) Maximum exposure - If a programming language has more than 
 100 users, chances are, there's an OpenSSL binding available 
 for that language. C is an ideal language to make something 
 available for as many platforms and environments as possible.


This is very true. However, as each plateform has its own 
characteristics, you ends up not being able to port that simply, 
and worse, you can break security without knowing it doing so.

I understand the social aspect of it, but from a security POV, 
this is a neat loss.

I'm not sure if another approach is possible. It is clear that 
nobody care about security until catastrophes happens. At least, 
the recent event waked up many people on how bad the state of 
affair is, and how clueless the people handling it right now are.


 (2) Determinism - If your intention is to implement crypto that 
 is impervious to side-channel attacks, you need a language 
 that's "close to the metal" and will behave how you expect it 
 to. For example, Java would be a poor choice because things 
 like garbage collection and JITing makes code highly 
 non-deterministic.


D is an option here. Anything that isn't system related obviously 
isn't, as you must ensure that you clean the memory behind you.
Jul 05 2014
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/5/2014 12:33 PM, deadalnix wrote:

 I used to think that. A few years ago, I looked into OpenSSL, noticed several
 horrors. Several of them mentioned here:

 https://www.youtube.com/watch?v=GnBbhXBDmwU

 I had the same reasoning: crytpo is hard and these guys know much more than I
do.

 They don't. The simple fact they are are using C to build security related
basic
 block show that they have no idea what they are doing. No bound check, no
memory
 safety, integer overflow is undefined behavior (which mean that even if you
 remember to check for it, you are not checking for it).


Sure, but nobody is going to blame us for it :-) whereas they will for an 
official D implementation.
Jul 05 2014
next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 6 July 2014 at 00:18:19 UTC, Walter Bright wrote:

 On 7/5/2014 12:33 PM, deadalnix wrote:

 I used to think that. A few years ago, I looked into OpenSSL, 
 noticed several
 horrors. Several of them mentioned here:

 https://www.youtube.com/watch?v=GnBbhXBDmwU

 I had the same reasoning: crytpo is hard and these guys know 
 much more than I do.

 They don't. The simple fact they are are using C to build 
 security related basic
 block show that they have no idea what they are doing. No 
 bound check, no memory
 safety, integer overflow is undefined behavior (which mean 
 that even if you
 remember to check for it, you are not checking for it).


 Sure, but nobody is going to blame us for it :-) whereas they 
 will for an official D implementation.


I understand. That is reasonable position. The CS guy in me is 
crying, but we got to pick our battle.
Jul 05 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/5/2014 6:54 PM, deadalnix wrote:

 On Sunday, 6 July 2014 at 00:18:19 UTC, Walter Bright wrote:

 On 7/5/2014 12:33 PM, deadalnix wrote:

 I used to think that. A few years ago, I looked into OpenSSL, noticed several
 horrors. Several of them mentioned here:

 https://www.youtube.com/watch?v=GnBbhXBDmwU

 I had the same reasoning: crytpo is hard and these guys know much more than I
 do.

 They don't. The simple fact they are are using C to build security related
basic
 block show that they have no idea what they are doing. No bound check, no
memory
 safety, integer overflow is undefined behavior (which mean that even if you
 remember to check for it, you are not checking for it).


 Sure, but nobody is going to blame us for it :-) whereas they will for an
 official D implementation.


 I understand. That is reasonable position. The CS guy in me is crying, but we
 got to pick our battle.


Yeah I know, I'd like to roll our own, too!
Jul 05 2014
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 06/07/14 02:18, Walter Bright wrote:


 Sure, but nobody is going to blame us for it :-) whereas they will for
 an official D implementation.


If we get D bindings and wrappers of a C library in Phobos, people using 
it might not no it's actually just bindings/wrappers and start to blame us.

-- 
/Jacob Carlborg
Jul 06 2014
prev sibling parent reply "Xinok" <xinok live.com> writes:
On Saturday, 5 July 2014 at 19:33:31 UTC, deadalnix wrote:

 I used to think that. A few years ago, I looked into OpenSSL, 
 noticed several horrors. Several of them mentioned here:

 https://www.youtube.com/watch?v=GnBbhXBDmwU


I watched the video just now and I agree, OpenSSL sounds 
horrible. However, the poor quality of it has nothing to do with 
being written in C.

However, a very important point comes up at 1:00:15 when a person 
asks about the implementation of the "mainstream" ciphers. It 
turns out the implementation of the cryptographic algorithms is 
well done and actually written by CRYPTOGRAPHERS. One of the 
developers admits its beyond their competency and they generally 
leave it alone.

So maybe its okay for us to implement APIs or protocols in D, but 
we should NOT be implementing our own crypto.
Jul 06 2014
parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 7/6/2014 12:06 PM, Xinok wrote:

 However, a very important point comes up at 1:00:15 when a person asks
 about the implementation of the "mainstream" ciphers. It turns out the
 implementation of the cryptographic algorithms is well done and actually
 written by CRYPTOGRAPHERS. One of the developers admits its beyond their
 competency and they generally leave it alone.


Programmers are *intimidated* by crypto algorithms. They mistakenly 
believe they need to understand everything about how/why an algorithm 
work in order to implement it, and that's probably because any good 
programmer is already accustomed to working that way.

I guarantee that programmer would be perfectly capable of correctly 
implementing any of the algos if he grabbed a copy of the spec and 
actually tried. It's NOT that hard. That hard part was coming up with 
(and analyzing/peer-reviewing) the algorithm in the first place, but the 
mathematicians have already taken care of that.

Now, I don't doubt that OpenSSL's crypto implementations are heavily 
optimized and that undoubtedly makes it difficult to understand and not 
mess up (although, due to their deliberate "avalanche effect" nature, a 
broken crypto hashing or encryption algorithm is very likely be 
immediately caught by even a halfway decent unittest suite).

But still, good programmers are all deathly afraid of crypto algorithms, 
but it's completely unjustified:

1. The theory is completely separate and NOT REQUIRED for implementers. 
You don't need to know WHY the W3C defined CSS the way they did in order 
to implement CSS. You just need the spec and a test suite.

2. Implementation correctness is easily tested. More easily than most 
other algorithms.

3. ALL the other difficult, yet critically-important, security issues 
ARE NOT PART OF CRYPTOGRAPHIC THEORY OR THE ACTUAL CRYPTO ALGORITHMS. 
They are things we ALREADY need to be understanding and dealing with 
ANYWAY, REGARDLESS of whether we're implementing SHA or writing a web 
app's frontend.

This is a PROBLEM. It means the ONLY people implementing cryptography 
are A. the bad programmers and B. the crypto mathemeticians who's job 
DOES NOT NECESSARILY REQUIRE any understanding whatsoever of the 
non-crypto security issues that WE programmers ALREADY need to be 
dealing with ANYWAY.
Jul 06 2014