www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - SHA-3 is KECCAK

reply "Xinok" <xinok live.com> writes:
With the recent discussions regarding std.crypt/std.hash, I 
thought it would be worth announcing that the SHA-3 competition 
has ended and the winner is KECCAK.

http://csrc.nist.gov/groups/ST/hash/sha-3/winner_sha-3.html
Oct 03 2012
next sibling parent reply "Chris Cain" <clcain uncg.edu> writes:
On Wednesday, 3 October 2012 at 11:35:42 UTC, Xinok wrote:

 With the recent discussions regarding std.crypt/std.hash, I 
 thought it would be worth announcing that the SHA-3 competition 
 has ended and the winner is KECCAK.

 http://csrc.nist.gov/groups/ST/hash/sha-3/winner_sha-3.html


I would love to see D support SHA3 ASAP. I cringe at the fact 
that many still use MD5.
Oct 03 2012
parent reply "Kagamin" <spam here.lot> writes:
On Wednesday, 3 October 2012 at 12:39:15 UTC, Chris Cain wrote:

 On Wednesday, 3 October 2012 at 11:35:42 UTC, Xinok wrote:

 With the recent discussions regarding std.crypt/std.hash, I 
 thought it would be worth announcing that the SHA-3 
 competition has ended and the winner is KECCAK.

 http://csrc.nist.gov/groups/ST/hash/sha-3/winner_sha-3.html


 I would love to see D support SHA3 ASAP. I cringe at the fact 
 that many still use MD5.


MD5 is good enough for most cases. AFAIK, keccak uses weird bit 
fiddling. Wasn't it considered a bad practice since DES because a 
specialized hardware would give a considerable speedup, which 
will help in brute force attacks?
Jan 17 2014
parent reply "Chris Cain" <clcain uncg.edu> writes:
On Friday, 17 January 2014 at 11:37:26 UTC, Kagamin wrote:

 MD5 is good enough for most cases.


For any use where security isn't an actual concern, sure. If it's 
just to casually verify that a file transferred successfully 
(like an alternative to a checksum), then it's fine to use. But 
don't use it to secure anything against an attacker at this point.


 AFAIK, keccak uses weird bit fiddling. Wasn't it considered a 
 bad practice since DES because a specialized hardware would 
 give a considerable speedup, which will help in brute force 
 attacks?


Actually, the idea is that it _should_ be implemented in 
specialized hardware to make it faster. And improving brute force 
attacks in this manner will only provide a multiplicative 
increase in speed, and that's not a concern. The overall strategy 
of using brute force isn't going to be turned from infeasible to 
feasible because of that. It's still completely infeasible to 
find two different messages s.t. their SHA-3 hash is equal.
Jan 17 2014
next sibling parent reply "Kagamin" <spam here.lot> writes:
On Friday, 17 January 2014 at 13:11:57 UTC, Chris Cain wrote:

 On Friday, 17 January 2014 at 11:37:26 UTC, Kagamin wrote:

 MD5 is good enough for most cases.


 For any use where security isn't an actual concern, sure. If 
 it's just to casually verify that a file transferred 
 successfully (like an alternative to a checksum), then it's 
 fine to use. But don't use it to secure anything against an 
 attacker at this point.


There's no successful preimage attack on MD5, which is the only 
deadly attack on a hash function. SHA3 is just more convenient 
than MD5 because when you want to change the hash function, you 
don't have to ditch the whole system, only change its parameters.


 AFAIK, keccak uses weird bit fiddling. Wasn't it considered a 
 bad practice since DES because a specialized hardware would 
 give a considerable speedup, which will help in brute force 
 attacks?


 Actually, the idea is that it _should_ be implemented in 
 specialized hardware to make it faster.


That's rather inconvenient, that you can't have an efficient 
implementation of the algorithm on common hardware. MD5 family 
has no such flaw.


 And improving brute force attacks in this manner will only 
 provide a multiplicative increase in speed, and that's not a 
 concern. The overall strategy of using brute force isn't going 
 to be turned from infeasible to feasible because of that. It's 
 still completely infeasible to find two different messages s.t. 
 their SHA-3 hash is equal.


You assume that Moore's law doesn't and won't work. Write asserts 
for assumptions.
Jan 17 2014
next sibling parent reply "Chris Cain" <clcain uncg.edu> writes:
On Friday, 17 January 2014 at 14:04:29 UTC, Kagamin wrote:

 On Friday, 17 January 2014 at 13:11:57 UTC, Chris Cain wrote:

 On Friday, 17 January 2014 at 11:37:26 UTC, Kagamin wrote:

 MD5 is good enough for most cases.


 For any use where security isn't an actual concern, sure. If 
 it's just to casually verify that a file transferred 
 successfully (like an alternative to a checksum), then it's 
 fine to use. But don't use it to secure anything against an 
 attacker at this point.


 There's no successful preimage attack on MD5, which is the only 
 deadly attack on a hash function.


Your first part is true, but the second part is not. A generic 
collision attack can cause issues depending on what you're trying 
to do. For general use, MD5 is not safe enough based on that 
alone.

The wikipedia article on MD5 covers this reasonably well enough:
http://en.wikipedia.org/wiki/MD5

Do note that SHA1 is similar enough to MD5 that many are 
recommending against SHA1 as well (and I recommend following that 
advice). It's not as broken as MD5 but since we have SHA2 there's 
no good reason not to use the better version.

For general messages requiring any degree of security against an 
attacker, use SHA2 (this does not necessarily apply to hashing 
passwords which have their own concerns, obviously). Stay away 
from MD5 and no new projects should use SHA1 (although my 
understanding is that no one is saying "jump from the ship" for 
SHA1 right now). These are generally accepted practices in the 
cryptography community. If you want to roll the dice with MD5, by 
all means, but don't spread acceptance of MD5 because someone 
might follow your advice against the clear recommendations of the 
cryptography community. As far as I'm concerned, it's unethical 
to recommend MD5 for security purposes at this point.


 SHA3 is just more convenient than MD5 because when you want to 
 change the hash function, you don't have to ditch the whole 
 system, only change its parameters.


I'm sorry but no. The reason we wanted a SHA3 is because we 
needed a message digest algorithm that is different enough from 
MD5, SHA1, and SHA2 so that an attack discovered on those 
algorithms should not be applicable to SHA3 as well (as was the 
case with MD5 and SHA1). The things you describe are just bonuses.


 That's rather inconvenient, that you can't have an efficient 
 implementation of the algorithm on common hardware. MD5 family 
 has no such flaw.


You seem to not know enough about this topic to discuss it. SHA3 
is fast on common hardware, as it is designed. Specialized 
hardware can be made and that's fine, intentional, and even 
beneficial. Most cryptography is designed that way. DES was, AES 
is (AES instructions exist on many modern hardware, in fact), and 
SHA3 is as well. This is not a flaw and if you think it is, go 
talk with cryptographers instead of me because you're clearly 
well beyond my skill level (that said, since you're discussing 
the viability of MD5, you're not anywhere close to as competent 
on this matter as you think you are, no offense).


 You assume that Moore's law doesn't and won't work. Write 
 asserts for assumptions.


I'm not sure what you mean here. No such assumptions need to be 
made. You are aware of how large the space of 256-bit message 
digests are, right? Multiplicative (specialized hardware) 
increases in performance makes no difference because of the 
ridiculous size of such a space. Exponential increases in 
performance (like Moore's law) are flawed because of how long it 
takes for those improvements to accumulate.

The way you defeat such a thing is not via multiplicative 
increases nor exponential increases over a long time, but by 
instantaneous, SIGNIFICANT breaks. Even effectively cutting a 
256-bit message digest down to the complexity of 128-bit (which 
is a *massive* break) is still not enough, for comparison.
Jan 17 2014
parent reply "Kagamin" <spam here.lot> writes:
On Friday, 17 January 2014 at 14:53:04 UTC, Chris Cain wrote:

 Your first part is true, but the second part is not. A generic 
 collision attack can cause issues depending on what you're 
 trying to do. For general use, MD5 is not safe enough based on 
 that alone.


No technology is safe, and SHA3 doesn't solve this problem. 
Cryptography is particularly not fool-proof. Remember issues in 
RSA? The factorization math behind it is bulletproof, yet, it's 
still considered hackable.


 The wikipedia article on MD5 covers this reasonably well enough:
 http://en.wikipedia.org/wiki/MD5

 Do note that SHA1 is similar enough to MD5 that many are 
 recommending against SHA1 as well (and I recommend following 
 that advice). It's not as broken as MD5 but since we have SHA2 
 there's no good reason not to use the better version.

 For general messages requiring any degree of security against 
 an attacker, use SHA2 (this does not necessarily apply to 
 hashing passwords which have their own concerns, obviously). 
 Stay away from MD5 and no new projects should use SHA1 
 (although my understanding is that no one is saying "jump from 
 the ship" for SHA1 right now). These are generally accepted 
 practices in the cryptography community. If you want to roll 
 the dice with MD5, by all means, but don't spread acceptance of 
 MD5 because someone might follow your advice against the clear 
 recommendations of the cryptography community. As far as I'm 
 concerned, it's unethical to recommend MD5 for security 
 purposes at this point.


Ethics don't help in cryptography, only real factors play role.
Your insistence on blind following to recommendations and 
expecting them to magically protect security - by encouraging 
thoughtless behavior such voodooism is way more troublesome.


 SHA3 is just more convenient than MD5 because when you want to 
 change the hash function, you don't have to ditch the whole 
 system, only change its parameters.


 I'm sorry but no. The reason we wanted a SHA3 is because we 
 needed a message digest algorithm that is different enough from 
 MD5, SHA1, and SHA2 so that an attack discovered on those 
 algorithms should not be applicable to SHA3 as well (as was the 
 case with MD5 and SHA1). The things you describe are just 
 bonuses.


This works both ways: an attack on SHA3 doesn't affect other hash 
functions.
The older algorithms are more trustworthy exactly because they 
are well studied, have no dark corners and evolve gradually by 
fixing discovered weaknesses.


 That's rather inconvenient, that you can't have an efficient 
 implementation of the algorithm on common hardware. MD5 family 
 has no such flaw.


 You seem to not know enough about this topic to discuss it. 
 SHA3 is fast on common hardware, as it is designed. Specialized 
 hardware can be made and that's fine, intentional, and even 
 beneficial. Most cryptography is designed that way. DES was, 
 AES is (AES instructions exist on many modern hardware, in 
 fact), and SHA3 is as well. This is not a flaw and if you think 
 it is, go talk with cryptographers instead of me because you're 
 clearly well beyond my skill level (that said, since you're 
 discussing the viability of MD5, you're not anywhere close to 
 as competent on this matter as you think you are, no offense).


It's even worse that new algorithms force hardware vendors to add 
specialized instructions which are of no use for other software.

On Friday, 17 January 2014 at 15:13:27 UTC, Chris Cain wrote:

 On Friday, 17 January 2014 at 14:04:29 UTC, Kagamin wrote:

 There's no successful preimage attack on MD5, which is the 
 only deadly attack on a hash function.


 For one amusing example of why MD5 is broken (from years ago 
 using a video game console, of all things), consider:
 http://www.win.tue.nl/hashclash/Nostradamus/


They didn't predict the result of elections, not sure what you 
want to prove by a lie.
Jan 18 2014
next sibling parent reply "Chris Cain" <clcain uncg.edu> writes:
On Saturday, 18 January 2014 at 12:48:28 UTC, Kagamin wrote:

 ..snip almost everything..


I'm not going to be doing this. Everything you say suggests you 
have no clue about anything about cryptography and you're going 
to be bit sooner rather than later. Don't spread your 
information, just keep it to yourself so that the damage is 
limited. Thanks.


 They didn't predict the result of elections, not sure what you 
 want to prove by a lie.


You're correct. It was an example of a collision attack which 
means they can put up a single hash and it will match many 
possible documents that include all the names of the candidates. 
If you had _read_ the source, you may have understood that. In 
fact, if you have read anything on cryptography at all before, 
you'd be staying away from MD5, but ignorance is bliss I suppose. 
Just don't spread your ignorance to others because you could 
easily cost people their livelihood. If you lose your job because 
of this, then that's on you, so I won't feel badly about it. But 
I am concerned about an innocent third party walking in thinking 
you might have a clue of what you're talking about, when you 
clearly don't.

Take care.
Jan 18 2014
parent reply "Kagamin" <spam here.lot> writes:
On Saturday, 18 January 2014 at 15:17:52 UTC, Chris Cain wrote:

 You're correct. It was an example of a collision attack which 
 means they can put up a single hash and it will match many 
 possible documents that include all the names of the candidates.


Consequence of a 2-document collision is big enough, that a 
12-document collision doesn't pose any notably bigger threat. 
Nostradamus is all they could come up with, which is laughable. 
If they upgrade this to an actual partial 3.6-bit preimage 
attack, and one collision takes 2 days as they say, for a 
comparison, with such partial preimage attack, full preimage 
attack on a 32-bit hash like crc32 will take 2^^(32-3.6)*2 days = 
1959870 years in an ideal case.


 If you had _read_ the source, you may have understood that. In 
 fact, if you have read anything on cryptography at all before, 
 you'd be staying away from MD5, but ignorance is bliss I 
 suppose.


Isn't it you, who insist on ignorance to how a collision attack 
works and how it doesn't work? You insist on a magical approach 
to cryptography, that MD5 magic doesn't work and SHA3 magic 
works, but you should know that magic is a delusion, and delusion 
leads to failures and damage, so by spreading delusions, it's you 
who cause damage, not me.
Jan 19 2014
next sibling parent "Chris Cain" <clcain uncg.edu> writes:
On Sunday, 19 January 2014 at 15:09:46 UTC, Kagamin wrote:

 Isn't it you, who insist on ignorance to how a collision attack 
 works and how it doesn't work? You insist on a magical approach 
 to cryptography, that MD5 magic doesn't work and SHA3 magic 
 works, but you should know that magic is a delusion, and 
 delusion leads to failures and damage, so by spreading 
 delusions, it's you who cause damage, not me.


Excuse me? Straw men arguments don't work here. Take that crap 
elsewhere. You keep asserting I say things that I don't say like 
crap like "You assume that Moores law doesn't work". Get out of 
here with that.

The fact that MD5 is weak against a collision attack means that 
it strictly provides weaker guarantees than the stronger SHA1, 
SHA2, and SHA3. No magic is necessary to know that with 
cryptography, attackers look for the weakest point in a chain to 
attack. Intentionally using something that provides weaker 
guarantees is foolish. And indeed a collision attack can cause 
problems, depending on what you want to do. I can't predict (nor 
can you) where some one will use a hash function expecting it to 
be secure, so the right thing to do is just suggest people use 
the stronger hash functions.

Furthermore, when talking about anything related to cryptography, 
we discuss things in terms of how much of a safety buffer we get. 
You're right that nothing is "perfectly safe" but when our 
estimations of safety suggest it should remain safe for at least 
20 years, then we're fine with it. That's why we don't use MD5. 
Because it has known flaws AND because it no longer has the 
"shield" of collision resistance. For a hash function, if it is 
"collision resistant" that strictly means that no pre image 
attack exists either. You'd have to break through two walls to 
break the hash function. With MD5, there is no such safety 
buffer. It's one discovery away from being destroyed. Ergo, don't 
use it because it can turn from "no pre image attacks exist" to 
"here's the pre image attack" overnight.

Preempting your counterargument: Indeed such a thing can happen 
with SHA2, but the the likelihood of such a thing is essentially 
non existent. If we haven't even discovered a single collision 
ever, we don't find the likelihood of generating collisions at 
will likely nor do we find the likelihood of generating a 
specific collision likely either.

Finally, I have never suggested anyone work off of "this stuff is 
magic so be ignorant of it while working with it". Quite the 
opposite. You should be well educated with this stuff prior to 
working with it or else you risk exposing confidential 
information which can be quite damaging. Part of this is _don't 
deliberately ignore the recommendations of cryptography experts_. 
What you suggest people do, "despite being a non-expert, I 
suggest you do use MD5 because those experts clearly don't know 
what they're talking about" is dangerous. And stop with the straw 
men attacks because I'm sick of it. Actually, I'm sick of all of 
your crap. You can have the last word and if it's "ignore the 
cryptography experts because I'm smarter than them", then so be 
it. You might want to also let your employer know as well so they 
can give you a raise *wink*.
Jan 19 2014
prev sibling next sibling parent "Chris Cain" <clcain uncg.edu> writes:
On Sunday, 19 January 2014 at 15:09:46 UTC, Kagamin wrote:

 Consequence of a 2-document collision is big enough, that a 
 12-document collision doesn't pose any notably bigger threat. 
 Nostradamus is all they could come up with, which is laughable. 
 If they upgrade this to an actual partial 3.6-bit preimage 
 attack, and one collision takes 2 days as they say, for a 
 comparison, with such partial preimage attack, full preimage 
 attack on a 32-bit hash like crc32 will take 2^^(32-3.6)*2 days 
 = 1959870 years in an ideal case.


By the way, I didn't respond to this, but I suppose I should.

https://code.google.com/p/tweakcrc/
http://blog.fortinet.com/trash-crc32/

Hence why you should keep your butt out of cryptography 
discussions until you take any class on it at the very least. 
Your calculation is garbage and shows how you haven't a single 
clue about what you are talking about.

Again, you can have the last word, but I'm certain I'm done here 
regardless. If you can't see the fact that you aren't smarter 
than the experts by now (with the above) then I don't think I can 
convince you with my words. Experience is the best teacher 
sometimes. I just hope that I never accidentally use a program 
you write expecting my data to be secure.
Jan 19 2014
prev sibling parent "Chris Cain" <clcain uncg.edu> writes:
One more thing, if you're curious about taking your first step 
into actually learning the basics of cryptography, you can use 
this:
https://www.coursera.org/courses?search=cryptography

Dan Boneh is a well respected expert in cryptography and his 
class is very well structured (it wasn't the first one I took, 
but it's definitely comparable to an actual college course on 
cryptography). Do not take it expecting that you'll even be 
remotely capable of going against the recommendations by experts 
by the end (in fact, you'll learn that such things is idiocy if 
you pay any attention to it).

If you do take it, do the assignments and homework. Despite such 
things having the reputation of being just a bother, the 
assignments are necessary to fully understand the material for 
advanced topics like this. They are tough but they'll teach you 
pretty quickly that looking at, for instance, the number of bits 
in crc32 is meaningless for creating attacks.
Jan 19 2014
prev sibling parent reply "Uranuz" <neuranuz gmail.com> writes:
I don't feel myself confident about crypto and security 
questions, but I need to make password hashing and generating of 
session Id. And make it difficult to pick up password with bruto 
force or dictional with single "usual" computer. I'm slightly 
disappointed that then more I read different articles on IT 
forums then less I understand something. And there are several 
opposite ideas that stunning me.
  1. All security systems, cipher, etc can be hacked If someone 
wants it
  2. Do not reinvent the wheel. All have been invented already.
  3. If you use standart implementation it's high risk than it was 
cracked already.
  4. Is it really essential to someone tho crack you security.

About md5 I have read that it's already cracked. It's vulnerable 
to length extension attack. As I feel SHA 2 is better (but it's 
not my opinion - it's just subjective feeling). And may be more 
modern algorithm isn't hacked until now. Higher variety of 
standart implemented hash algorithms can enable to combine them 
in different manner to get not standart implementation of hash. 
As I think it can increse security against attacks with rainbow 
tables.

I don't know if I rigth or not. The reason why I asked is that 
I'm implenenting authentication on site written in D. So I want 
to make password hash generation function enough secure to forget 
about it for ~5 years or more. Because there only a litle of hash 
functions implemented in std.digest and they are not so strong by 
security reasons. It makes it not very useful.

P.S. Sorry for my English.
Jan 21 2014
next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Tuesday, 21 January 2014 at 09:58:34 UTC, Uranuz wrote:

 I don't feel myself confident about crypto and security 
 questions, but I need to make password hashing and generating 
 of session Id. And make it difficult to pick up password with 
 bruto force or dictional with single "usual" computer. I'm 
 slightly disappointed that then more I read different articles 
 on IT forums then less I understand something. And there are 
 several opposite ideas that stunning me.
  1. All security systems, cipher, etc can be hacked If someone 
 wants it
  2. Do not reinvent the wheel. All have been invented already.
  3. If you use standart implementation it's high risk than it 
 was cracked already.
  4. Is it really essential to someone tho crack you security.

 About md5 I have read that it's already cracked. It's 
 vulnerable to length extension attack. As I feel SHA 2 is 
 better (but it's not my opinion - it's just subjective 
 feeling). And may be more modern algorithm isn't hacked until 
 now. Higher variety of standart implemented hash algorithms can 
 enable to combine them in different manner to get not standart 
 implementation of hash. As I think it can increse security 
 against attacks with rainbow tables.

 I don't know if I rigth or not. The reason why I asked is that 
 I'm implenenting authentication on site written in D. So I want 
 to make password hash generation function enough secure to 
 forget about it for ~5 years or more. Because there only a 
 litle of hash functions implemented in std.digest and they are 
 not so strong by security reasons. It makes it not very useful.

 P.S. Sorry for my English.


I don't have any significant expertise on this subject, but I did 
find this highly rated article useful and interesting: 
http://www.codeproject.com/Articles/704865/Salted-Password-Hashing-Doing-it-Right

Note that it recommends against md5
Jan 21 2014
prev sibling next sibling parent "Kapps" <opantm2+spam gmail.com> writes:
On Tuesday, 21 January 2014 at 09:58:34 UTC, Uranuz wrote:

 I don't feel myself confident about crypto and security 
 questions, but I need to make password hashing and generating 
 of session Id. And make it difficult to pick up password with 
 bruto force or dictional with single "usual" computer. I'm 
 slightly disappointed that then more I read different articles


MD5, SHA2, SHA3, etc, none of these are valid for password
hashing. Not because of being able to generate a collision,
because that doesn't matter, but because it can be brute forced
easily. Use bcrypt or scrypt. If you really can't do that, then
hash with a salt at least thousand times (but if done improperly
this can make it actually less secure). You should always use a
built-in thing though, ideally bcrypt or scrypt.
Jan 21 2014
prev sibling next sibling parent reply "Chris Cain" <clcain uncg.edu> writes:
On Tuesday, 21 January 2014 at 09:58:34 UTC, Uranuz wrote:

 I'm slightly disappointed that then more I read different 
 articles on IT forums then less I understand something. And 
 there are several opposite ideas that stunning me.
  1. All security systems, cipher, etc can be hacked If someone 
 wants it


A bit true and untrue at the same time. Let's look at it from the 
perspective of "feasible attacks" first: It's impossible to prove 
one way or the other. On the other hand, the only way we can 
verify something as secure is we have many "Really Smart People" 
look at it and see if they can come up with anything to attack it 
(and often there are often huge incentives provided to discover 
these weaknesses). It stands to reason if many motivated RSP 
can't discover a weakness, the likelihood of some joe-shmoe 
discovering a weakness is extremely low. It's possible but just 
not likely.

If we accept "infeasible attacks" also, then this statement is 
absolutely true. If you have some motivated individual who is 
willing to sit around until after the sun burns out (this is NOT 
an exaggeration but actual calculation), then they can eventually 
brute force a single AES256 instance. Since I'll be long dead and 
so will my children and my children's children (etc), I'm not 
concerned about such attacks.

With all that said, if you use a system that has NOT been 
reviewed by many motivated RSP, then you've (likely) got a 
ticking time bomb. That's the reason for the idea below ("Do not 
reinvent the wheel"). It's sometimes really difficult to discern 
whether you're using an accepted standard way of doing things or 
not, however. Even using the "accepted standard primitives" can 
be done in non-standard/weak ways.

Consider using AES256 poorly:
1. Take any password
2. hash it with MD5
3. Use that hash as the key (pad the extra bits with 0s) to 
encrypt something using AES256 in ECB mode

That's not strong for many reasons. But in particular step 1 is 
very vulnerable to attacks. If you have anything less than a 8 
character password, such a scheme is very easy to break. As a 
general rule, passwords should be no less than 12 characters long 
(my passwords are stored in a password manager with 20+ 
characters in length and my computer is encrypted with a password 
of length 20+ as well) or that'll be the weak point in the system.

MD5 is trash for a variety of reasons but the fact that 
collisions have been discovered means that it's more likely that 
multiple strings will hash to the same hash value. So despite 
being 128-bits long, it has the apparent value of something much 
lower. It's much more likely, for instance, that your 
50-character password will hash the same as a 7-character 
password. Unless you specifically chose your password in such a 
way to avoid such a thing, it's impossible to know whether you've 
chosen an unlucky password where such a thing is true.

Since step 3 involves padding the extra bits with 0s, that means 
AES256 is no more powerful than AES128. (this would be true even 
if you alternated between 0 and 1 or any number of other 
patterns, by the way) Furthermore since MD5 has been proven to 
not actually have the value of 128-bit, the actual encryption 
power is even lower than that.

Furthermore, using ECB mode makes it insecure to certain types of 
attacks too.
http://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Electronic_codebook_.28ECB.29
(look at the image ... you can discover what a message is without 
actually decrypting it! Yes, this works with text too, but it 
requires a bit more analysis that just looking at it.)

Despite AES256 itself being very secure against attacks, the 
scheme used makes it very vulnerable to a variety of attacks. The 
only way to know whether it's good or not is to research 
yourself, or, better, ask a group of experts to break it.

(The "ask a group to break it" really reminds me of Andre's 
"Destroy" philosophy)


One last thing to consider for this: there are usually 
side-channel attacks that can circumvent most security systems. A 
famous attack is known as the "rubber hose attack":
Take some guy who has information you want (such as a password to 
access a system or decrypt something important) and beat him with 
a rubber hose until he tells you what you need to know.
http://en.wikipedia.org/wiki/Rubber-hose_cryptanalysis

So if you also start expanding your scope to things like 
side-channel attacks, the difficulty of implementing a truly 
secure system goes WAAAY up. The probability of all of those 
side-channel attacks being used is fairly low, however. Some of 
them are actually pretty important, though. Like timing attacks.

In general, though, it is possible to get a system secure enough 
that such things become too unlikely to worry about. It's 
absolutely worth putting forth the effort to secure things 
despite it being very hard to nearly impossible to make things 
"perfectly" secure.


  2. Do not reinvent the wheel. All have been invented already.


Do not reinvent the wheel: absolutely true.
All have been invented already: Not necessarily.

If there exists a solution to the problem you're trying to solve 
(and it's likely that such a solution does exist), use the 
existing solution as long as you have good reason to believe it 
to be strong against attackers (a solution described by some 
random guy on a forum that isn't providing citations is an 
example of something you shouldn't necessarily trust ... the 
strength of someone's solution has no relation to how smart they 
think they are). At the very least you should be using 
cryptographic primitives to create your solution (such as using 
AES or one of the SHAs, particularly SHA2). You should also "peer 
review" your solution with others when possible (even using 
accepted primitives can result in weaknesses). Do _not_ hide your 
solution and hope no one discovers how it's done as your form of 
security ("security through obscurity"). That is extremely 
unreliable and will almost certainly result in a successful 
attack if you are guarding anything of value.


  3. If you use standart implementation it's high risk than it 
 was cracked already.



implementations have been reviewed by many very intelligent 
experts in the field and they'll have some pretty good tricks up 
their sleeve to discover potential weaknesses. A more recent 
example, the Dual_EC_DRBG algorithm of RSA infamy, was theorized 
to be weak in November 2007 because of the constants used. 
Although no attack was actually discovered at the time, the fact 
that a RSP saw the likelihood of a flaw in it shows that these 
guys are _very_ good at spotting weaknesses before they become an 
actual issue. Indeed recently it was shown to actually be weak 
against the NSA (who paid the RSA to include the algorithm in 
their standards).

This type of idea usually come from amateurs who have relatively 
low experience in cryptography. It's an idea that, on the surface 
or in theory, looks pretty reasonable. But experience teaches you 
that this is false in practice. This whole concept is derived 
from "use something that no one has seen yet so they haven't 
discovered the weakness yet" ... or, alternatively, "Security 
through Obscurity". An example is "hide a key to your house under 
a rock in front of your front door instead of carrying it 
around." Or "use a lock that has a single pin but the pin is 
located on the horizontal instead of the vertical". Using 
something non-standard hoping to avoid issues will NOT actually 
help you avoid issues. At best you can hope to delay an attack 
whereas things being open means that, despite many motivated RSP 
having looked at it, the system remains secure from attacks.

Look at this wikipedia article for a full explanation of why 
"avoid the standard implementations"/"security through obscurity" 
is bad:
http://en.wikipedia.org/wiki/Security_through_obscurity

FWIW, if you're "smart enough" you might be able to come up with 
a system using Security through Obscurity on parts of it that 
doesn't significantly negatively impact actual security of your 
system. Unfortunately, no one is able to accurately gauge if you 
are "smart enough". However, here's a reasonable heuristic:

Do you have 4 years of specialized training/experience in 
cryptography alone? If no, then you're probably not "smart 
enough". If yes, then you likely know the risks inherent to 
rolling your own solution without others reviewing it and you'll 
know whether its worth the risks to implement it so it's up to 
you.

If you have no experience in cryptography but think that it's 
bogus that you'd actually need experience to do cryptography and 
think you can do it anyway: You're almost absolutely not "smart 
enough" but you're probably "dumb enough" to ignore my advice (or 
anyone's for that matter) anyway, so it's a waste of time to 
discuss it. Do what you will, but hopefully after you fail a few 
times (and you will) you'll get the hint.


  4. Is it really essential to someone tho crack you security.


I don't understand. Could you rephrase this?



Okay, all of that was a bit too much, but hopefully you'll find 
it helpful.
Jan 21 2014
next sibling parent "Uranuz" <neuranuz gmail.com> writes:
 4. Is it really essential to someone tho crack you security.




It's beacuse I wrote it at night.

I mean that developer should estimate security level that needed 
and do not reinvent complicated security system where it's not 
essential. But question is how to evaluate security level itself 
and what type of protection is needed for concrete data to be 
protected.

Thanks for responses. Now I'm trying to use OpenSSL library that 
has complete set of crypto primitives. But I'll try to take 
bcrypt, scrypt into consideration
Jan 22 2014
prev sibling parent reply "Uranuz" <neuranuz gmail.com> writes:
Thanks for responses. I considered to use scrypt for now, because 
as described is's harder to hack it with brute force using GPU, 
ASIC, FPGA than bcrypt. But it has some limitations too. I've 
tried to tweak it to use more memory and less computation time 
but failed. Another question for me is how would I support this 
type of password authentication. After some years computation 
powers will grow and I will need to modify my hased keys stored 
in DB to adequately resist this. So I need some protocol where 
some parameters could be tweaked I'll be able to "rehash" or make 
some additional iterations with existing password hashes with no 
need to disturb users that "we have some security updates and ask 
users to change their passwords". So this part of reply is 
interesting for with from described point of view.


 Consider using AES256 poorly:
 1. Take any password
 2. hash it with MD5
 3. Use that hash as the key (pad the extra bits with 0s) to 
 encrypt something using AES256 in ECB mode


I need kind of function with the these properties.
   KDF(passwd, salt, jobFactor1)
   KDF(passwd, salt, jobFactor2)
   where jobFactor1 - initial number representing amont of job 
needed to calculate hash, and jobFactor2 ( > jobFactor1 ) is 
index that will be actual in the future.
   For old hashes I would do something like this
   KDF_diff(KDF_old, jobFactor2 - jobFactor1)
   But for new users (or for users that wants to change password) 
I will call original KDF(passwd, salt, jobFactor2) with new job 
factor. Also would be suitable implementation where KDF_diff is 
same as KDF and simply used as KDF(KDF_old, salt, jobFactor2 - 
jobFactor1). It can be simply achieved by using some SHA2, SHA3, 
MD5 or other hash function used in cycle. But I want to believe 
that some other elegant solution exists in combination with 
memory-hard function like scrypt.

Also I found some info about new Key Derivation Functions. But 
seems that they are not enough checked for security and have 
enough tested implementation.
http://lyra-kdf.net/
http://eprint.iacr.org/2013/525.pdf

Also I found site about password hashing functions competition. 
It will be interesting to read what D community thinks about it.
https://password-hashing.net/faq.html
Jan 27 2014
parent "Rikki Cattermole" <alphaglosined gmail.com> writes:
On Monday, 27 January 2014 at 09:08:04 UTC, Uranuz wrote:

 Thanks for responses. I considered to use scrypt for now, 
 because as described is's harder to hack it with brute force 
 using GPU, ASIC, FPGA than bcrypt. But it has some limitations 
 too. I've tried to tweak it to use more memory and less 
 computation time but failed. Another question for me is how 
 would I support this type of password authentication. After 
 some years computation powers will grow and I will need to 
 modify my hased keys stored in DB to adequately resist this. So 
 I need some protocol where some parameters could be tweaked 
 I'll be able to "rehash" or make some additional iterations 
 with existing password hashes with no need to disturb users 
 that "we have some security updates and ask users to change 
 their passwords". So this part of reply is interesting for with 
 from described point of view.


Just a quick thought of the hash migration might be to store the 
algorithm version (lets say 0 for scrypt). Then when you do an 
opCmp on the hash password or how ever you're comparing. And it 
checks out it'll update and save it on login.

If it gets too out of date then maybe ask them to login to change 
their password.
Jan 27 2014
prev sibling parent "Kagamin" <spam here.lot> writes:
On Tuesday, 21 January 2014 at 09:58:34 UTC, Uranuz wrote:

 I don't feel myself confident about crypto and security 
 questions, but I need to make password hashing and generating 
 of session Id. And make it difficult to pick up password with 
 bruto force or dictional with single "usual" computer.


The article linked by John Colvin is comprehensive enough to give 
you understanding of the problem, if outlines all mistakes in 
password hashing schemes and how to solve them properly. The 
dictionary attack is the most dangerous, as indicated by the 
article, slow hash like bcrypt can be of some help, it may win 
you a little time, but that time can be enough for the 
administrator to lock the system. Slow hash makes brute force 
slower, because it has to compute hash a lot. So for password 
hashing there's no reason to choose md5, because it's the fastest 
:)
There's no reason to implement bcrypt on your own (like 
repetitive hashing), because it should be well optimized in order 
to not let the attacker to compute the hash faster by a more 
optimal implementation.
Keyed hash is a very strong protection if you keep the key 
secret. The task of protecting the key is more administrative 
than cryptographic. If the attacker can't retrieve the key, he 
can only brute force the password remotely on your system, but 
since he will use your code, you have the chance to detect the 
attack. What to do once you detected it is up to you.
Another possibility to consider is two-factor authentication for 
the case when the password is indeed stolen.


 About md5 I have read that it's already cracked. It's 
 vulnerable to length extension attack.


The length extension attack is used against digital signatures, 
it's useless against password hashing (that's why it's not 
considered in the article). The attack is prevented by prepending 
salt instead of appending.
Jan 22 2014
prev sibling next sibling parent "Chris Cain" <clcain uncg.edu> writes:
On Friday, 17 January 2014 at 14:04:29 UTC, Kagamin wrote:

 There's no successful preimage attack on MD5, which is the only 
 deadly attack on a hash function.


For one amusing example of why MD5 is broken (from years ago 
using a video game console, of all things), consider:
http://www.win.tue.nl/hashclash/Nostradamus/
Jan 17 2014
prev sibling parent "Chris Cain" <clcain uncg.edu> writes:
Here, let me use an analogy. Note, this analogy comes from me so 
it's not necessarily endorsed by any professional 
cryptographers... but it explains where my concerns are coming 
from.

Let's say that everyone has something very precious ... like a 
diamond. Let's imagine a multiverse. Each universe can 
"represent" an algorithm. One is for MD5, another is for SHA1 and 
so on. In each universe, there exists a planet that holds a vault 
that has your diamond. Some universes are similar to others (for 
instance, the SHA1 universe has some passing resemblance to the 
MD5 universe).

When you say "specialized hardware can be made to speed up the 
process", I think "I can visit ten planets in this universe 
looking for the vaults instead of just one at a time if I invest 
in a special ship". Can you see why this doesn't concern me in 
the slightest?

When you say "Moore's law", I think "every 18 months I can know 
which half of the remaining universe contains the diamonds". 
Again, can you see why this doesn't concern me?

The universe is _just too big_ for this stuff to matter. If you 
can't find the vaults containing the diamonds, then these things 
don't bother me nor should it bother you.

For the MD5 universe, people have found a way of discovering the 
planet the diamonds are on very quickly (something like being 
able to find which millionth of the remaining universe contains 
the diamond each attempt ... a very fast solution). They've 
overcome the challenge of finding the diamonds (collision 
resistance) and now we wonder who is going to invent the 
blowtorch capable of opening the vaults up (preimage attack). All 
any cryptographer is saying is move the diamonds you care about 
into another universe before the MD5 blowtorch is invented. It 
may never get invented (we don't have any idea whether it's 
possible or if it is possible, when it might materialize) but 
moving it into another universe where the diamonds haven't even 
been found yet is simply safer. Ideally you don't want a universe 
that is too similar (such as MD5 and SHA1) because it might make 
the planet easier to be found. Also note that despite their 
similarities, no planets with diamonds have been found in the 
SHA1 universe. Yet still cryptographers are recommending moving 
on to SHA2 for new projects. Just food for thought.

Plus knowing which planet the diamond is on is dangerous enough 
since it can be used in certain circumstances to do some 
nefareous things (already showed the example where someone can 
lie about a prediction for the future). It's only in special 
circumstances, but ignoring subtleties isn't recommended in 
cryptography since people almost always figure out a way to use 
things incorrectly.

To summarize, do not use MD5 intending it to be secure against 
attackers. Just don't. I'd also like to say that you really 
shouldn't suggest to anyone that it's a reasonable approach 
either. Obviously, I can't force you to, but do know that 
generally people (not just me) are going to argue against you on 
this, so you can save yourself a lot of trouble by just never 
mentioning MD5 as being acceptable again. At this point I think 
there's been enough discussion on the matter that exceptionally 
few will wander into this topic and be mislead into accepting MD5 
for security purposes, so I'm going to retire from this 
discussion. If you use MD5 for its "security", just remember this 
for later: You were warned™. (Also, remember that people have 
been saying "Don't use MD5" since 1996, so when you do get bit by 
it, not only "You were warned" but "you are 20 years out of date, 
find a new job because you're fired" is also likely)
Jan 17 2014
prev sibling parent reply "Kagamin" <spam here.lot> writes:
On Friday, 17 January 2014 at 13:11:57 UTC, Chris Cain wrote:

 For any use where security isn't an actual concern, sure. If 
 it's just to casually verify that a file transferred 
 successfully (like an alternative to a checksum)


Doesn't TCP take care of that?
Jan 17 2014
next sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
17-Jan-2014 18:06, Kagamin пишет:

 On Friday, 17 January 2014 at 13:11:57 UTC, Chris Cain wrote:

 For any use where security isn't an actual concern, sure. If it's just
 to casually verify that a file transferred successfully (like an
 alternative to a checksum)


 Doesn't TCP take care of that?


As long as 16 bit CHECKSUM feels sufficient for you.

-- 
Dmitry Olshansky
Jan 17 2014
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2014-01-17 15:06, Kagamin wrote:


 Doesn't TCP take care of that?


Doesn't TCP only verify that a package is received intact, not a 
complete file?

-- 
/Jacob Carlborg
Jan 17 2014
prev sibling parent reply "Chris Cain" <clcain uncg.edu> writes:
On Friday, 17 January 2014 at 14:06:57 UTC, Kagamin wrote:

 Doesn't TCP take care of that?


For a packet, yes. In general you can assume that if a transfer 
completes under TCP then it is very likely correct. That's the 
way TCP is designed. If you want to check the entire file at the 
end, MD5 could theoretically be done as a sanity check. I'd still 
use SHA-2 (in the form of a digital signature, obviously) 
minimally if your intention is to ensure it hasn't been tampered 
with, but the speed, simplicity, and availability of MD5 can be 
arguments for its use as long as you're not actually concerned 
about security.
Jan 17 2014
parent "Kagamin" <spam here.lot> writes:
On Friday, 17 January 2014 at 15:00:38 UTC, Chris Cain wrote:

 On Friday, 17 January 2014 at 14:06:57 UTC, Kagamin wrote:

 Doesn't TCP take care of that?


 For a packet, yes. In general you can assume that if a transfer 
 completes under TCP then it is very likely correct. That's the 
 way TCP is designed. If you want to check the entire file at 
 the end, MD5 could theoretically be done as a sanity check. I'd 
 still use SHA-2 (in the form of a digital signature, obviously) 
 minimally if your intention is to ensure it hasn't been 
 tampered with


It's hard and expensive to use digital signatures in public 
projects and doesn't protect from tampering. In fact, direct 
tampering in such setup is cheaper than a collision attack, not 
even speaking, that a collision attack doesn't work here, only a 
preimage attack.
Jan 18 2014
prev sibling parent "Uranuz" <neuranuz gmail.com> writes:
I support the idea that it would be useful to have an 
implementation or binding for Keccak and may be other crypto hash 
functions. Now all what I found are pycrypto implementation an 
official implementation written with C at 
http://keccak.noekeon.org/. But I was unable to deal with the 
interface of this library. There are a little of comments in the 
code and I couldn't find concrete examples how to use it. Phobos 
simple interface like in std.digest.md would be more suitable for 
casual use. Very often in not needed to tweek all of these 
parameters for algorithm, but needed that it will just work.

I will appreciate for some guidelines if someone tried to bind 
this C implementation for using in D.
Jan 16 2014