www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - string encryption

reply Hiemlick Hiemlicker <HH reign.com> writes:
I know this is probably a lot to ask for an many won't see the 
point, but a secure program should not expose readable strings, 
it makes it far too easy for the attacker to see what is going on.

Is it possible to encrypt every static string in D and decrypt 
before it is output in an automatic fashion?

Even something like

e{This is an encrypted string}

which encrypts the string using a user defined function(which is 
ctfe'able) is better than nothing.

Even a simple xor type of cypher is better than nothing. 
Obviously optional.

The problem is that there is no way to encrypt strings in the 
binary through code because one must pass express the string 
non-encrypted to the encryption function.

encrypt("This string will still end up in the binary");

even though the string is only used once, at the encryption call 
site. It seems D won't replace

encrypt("This string will still end up in the binary");

with "skadf2903jskdlfaos;e;fo;aisjdfja;soejfjjfjfjfjfjfeij" or 
whatever the ctfe value of encrypt actually is.

This also seems like a bug in D because manifest constants used 
as sole arguments to ctfe'able functions should be replaced by 
the function result.


e.g.,

factorial(3) should be replaced by 6 and the function should 
never be called at run time. This is the whole point of ctfe, is 
it not?

I'm not actually sure if the functions are ctfe'ed and the 
un-encrypted string is just stored in the binary or what but it's 
there


import std.stdio;

string e(string s)
{
     string q;
     foreach(c; s)
         q ~= c + 1;
     return q;
}

void main()
{
    writeln(e("What is this string doing in the binary?"));
}
Jul 01 2016
next sibling parent reply qznc <qznc web.de> writes:
On Friday, 1 July 2016 at 22:23:23 UTC, Hiemlick Hiemlicker wrote:

 It seems D won't replace

 encrypt("This string will still end up in the binary");

 with "skadf2903jskdlfaos;e;fo;aisjdfja;soejfjjfjfjfjfjfeij" or 
 whatever the ctfe value of encrypt actually is.

 This also seems like a bug in D because manifest constants used 
 as sole arguments to ctfe'able functions should be replaced by 
 the function result.



CTFE is explicit. You could make it `encrypt!"encrypted at 
compile-time"`. Then let encrypt return some struct, which 
decrypts at runtime.
Jul 01 2016
parent reply Hiemlick Hiemlicker <HH reign.com> writes:
On Friday, 1 July 2016 at 22:55:21 UTC, qznc wrote:

 On Friday, 1 July 2016 at 22:23:23 UTC, Hiemlick Hiemlicker 
 wrote:

 It seems D won't replace

 encrypt("This string will still end up in the binary");

 with "skadf2903jskdlfaos;e;fo;aisjdfja;soejfjjfjfjfjfjfeij" or 
 whatever the ctfe value of encrypt actually is.

 This also seems like a bug in D because manifest constants 
 used as sole arguments to ctfe'able functions should be 
 replaced by the function result.



 CTFE is explicit. You could make it `encrypt!"encrypted at 
 compile-time"`. Then let encrypt return some struct, which 
 decrypts at runtime.


? I thought CTFE was for normal functions?

I guess I was mistaken ;/
Jul 01 2016
parent John Colvin <john.loughran.colvin gmail.com> writes:
On Friday, 1 July 2016 at 23:11:34 UTC, Hiemlick Hiemlicker wrote:

 On Friday, 1 July 2016 at 22:55:21 UTC, qznc wrote:

 On Friday, 1 July 2016 at 22:23:23 UTC, Hiemlick Hiemlicker 
 wrote:

 It seems D won't replace

 encrypt("This string will still end up in the binary");

 with "skadf2903jskdlfaos;e;fo;aisjdfja;soejfjjfjfjfjfjfeij" 
 or whatever the ctfe value of encrypt actually is.

 This also seems like a bug in D because manifest constants 
 used as sole arguments to ctfe'able functions should be 
 replaced by the function result.



 CTFE is explicit. You could make it `encrypt!"encrypted at 
 compile-time"`. Then let encrypt return some struct, which 
 decrypts at runtime.


 ? I thought CTFE was for normal functions?

 I guess I was mistaken ;/


CTFE is triggered when the result is immediately required to be a 
compile-time constant.

auto x = anyFunc("fdsa"); // anyFunc executed at runtime
static x = anyFunc("fdsa"); // anyFunc executed at compile time
enum x = anyFunc("fdsa"); // ditto

template value argument are also of course required to be 
compile-time constants, so passing the immediate result of a 
function call to a template will cause the function to be 
evaluated at compile time.

The way I think about it is that functions are (attempted to be) 
executed at ctfe on an as-needed basis.
Jul 02 2016
prev sibling next sibling parent reply Basile B. <b2.temp gmx.com> writes:
On Friday, 1 July 2016 at 22:23:23 UTC, Hiemlick Hiemlicker wrote:

 I know this is probably a lot to ask for an many won't see the 
 point, but a secure program should not expose readable strings, 
 it makes it far too easy for the attacker to see what is going 
 on.

 Is it possible to encrypt every static string in D and decrypt 
 before it is output in an automatic fashion?

 Even something like

 e{This is an encrypted string}

 which encrypts the string using a user defined function(which 
 is ctfe'able) is better than nothing.

 Even a simple xor type of cypher is better than nothing. 
 Obviously optional.

 The problem is that there is no way to encrypt strings in the 
 binary through code because one must pass express the string 
 non-encrypted to the encryption function.

 encrypt("This string will still end up in the binary");

 even though the string is only used once, at the encryption 
 call site. It seems D won't replace

 encrypt("This string will still end up in the binary");

 with "skadf2903jskdlfaos;e;fo;aisjdfja;soejfjjfjfjfjfjfeij" or 
 whatever the ctfe value of encrypt actually is.

 This also seems like a bug in D because manifest constants used 
 as sole arguments to ctfe'able functions should be replaced by 
 the function result.


 e.g.,

 factorial(3) should be replaced by 6 and the function should 
 never be called at run time. This is the whole point of ctfe, 
 is it not?

 I'm not actually sure if the functions are ctfe'ed and the 
 un-encrypted string is just stored in the binary or what but 
 it's there


 import std.stdio;

 string e(string s)
 {
     string q;
     foreach(c; s)
         q ~= c + 1;
     return q;
 }

 void main()
 {
    writeln(e("What is this string doing in the binary?"));
 }


You must make a template that follows this pattern:

     template KrypticString(string s)
     {
         string processor()
         {
             return /*do some stuff here on 's'*/ "";
         }
         enum KrypticString = processor();
     }

It's already used in phobos for example for octal() and 
hexString(). Also seen i dont remember where, for a float format.

But another important thing is that you must absolutely not 
release with -debug.
Strings are a thing but every one who has made the thug with IDA 
knows that the most usefull informations are the "Names" and not 
the "strings". The calls to the OS API can't be easily hidden, 
they are always in the "Names" but D functions names can.

For example it's even not worth crypting the strings if the 
attacker can see

     _D4main7decryptFAyaZAya

in the "Names", because in this case he "just" has to put a 
breakpoint on the C3 of the matching function, look for the 
decrypted string in memory, and bookmark the static addresses of 
the parameters passed to this function during the execution.

Also, if you take a minute to think a bit you'll find that 
cryptic strings will hit the eyes of the attacker quite quickly: 
"mmmh why is the content crypted ?!, let's see that...".
Jul 01 2016
parent reply Hiemlick Hiemlicker <HH reign.com> writes:
On Friday, 1 July 2016 at 22:56:48 UTC, Basile B. wrote:

 On Friday, 1 July 2016 at 22:23:23 UTC, Hiemlick Hiemlicker 
 wrote:

 I know this is probably a lot to ask for an many won't see the 
 point, but a secure program should not expose readable 
 strings, it makes it far too easy for the attacker to see what 
 is going on.

 Is it possible to encrypt every static string in D and decrypt 
 before it is output in an automatic fashion?

 Even something like

 e{This is an encrypted string}

 which encrypts the string using a user defined function(which 
 is ctfe'able) is better than nothing.

 Even a simple xor type of cypher is better than nothing. 
 Obviously optional.

 The problem is that there is no way to encrypt strings in the 
 binary through code because one must pass express the string 
 non-encrypted to the encryption function.

 encrypt("This string will still end up in the binary");

 even though the string is only used once, at the encryption 
 call site. It seems D won't replace

 encrypt("This string will still end up in the binary");

 with "skadf2903jskdlfaos;e;fo;aisjdfja;soejfjjfjfjfjfjfeij" or 
 whatever the ctfe value of encrypt actually is.

 This also seems like a bug in D because manifest constants 
 used as sole arguments to ctfe'able functions should be 
 replaced by the function result.


 e.g.,

 factorial(3) should be replaced by 6 and the function should 
 never be called at run time. This is the whole point of ctfe, 
 is it not?

 I'm not actually sure if the functions are ctfe'ed and the 
 un-encrypted string is just stored in the binary or what but 
 it's there


 import std.stdio;

 string e(string s)
 {
     string q;
     foreach(c; s)
         q ~= c + 1;
     return q;
 }

 void main()
 {
    writeln(e("What is this string doing in the binary?"));
 }


 You must make a template that follows this pattern:

     template KrypticString(string s)
     {
         string processor()
         {
             return /*do some stuff here on 's'*/ "";
         }
         enum KrypticString = processor();
     }

 It's already used in phobos for example for octal() and 
 hexString(). Also seen i dont remember where, for a float 
 format.


ok. For some reason I thought CTFE's applied to normal functions 
but I realize that doesn't make a lot of sense. (would be nice 
but most functions are not CTFE'able and the compiler would have 
trouble figuring that out)


 But another important thing is that you must absolutely not 
 release with -debug.
 Strings are a thing but every one who has made the thug with 
 IDA knows that the most usefull informations are the "Names" 
 and not the "strings". The calls to the OS API can't be easily 
 hidden, they are always in the "Names" but D functions names 
 can.


Yes, of course. Do D names change depending on -debug vs -release?


 For example it's even not worth crypting the strings if the 
 attacker can see

     _D4main7decryptFAyaZAya

 in the "Names", because in this case he "just" has to put a 
 breakpoint on the C3 of the matching function, look for the 
 decrypted string in memory, and bookmark the static addresses 
 of the parameters passed to this function during the execution.

 Also, if you take a minute to think a bit you'll find that 
 cryptic strings will hit the eyes of the attacker quite 
 quickly: "mmmh why is the content crypted ?!, let's see 
 that...".



I'm not too concerned about the attacker seeing them because they 
will have to watch every decryption to get the string(or possibly 
write a utility to automate the decryption).

If everything is encrypted it will give the attacker grief far 
more than being decrypted.

If pretty much every string is encrypted then it will make his 
life a little more difficult I would think. After all, it is 
impossible to completely stop an attacker given enough time and 
energy. The goal is to make it not worth it.

Also, Is there a simple way to make this work for "direct" output:


writeln(de_encrypt!("This is my string"));

it encrypts it at compile time but decrypts it at run time.(hence 
no binary)

Alternatively, treat every string as a call to an anonymous 
function that decrypts it at runtime(possibly using different 
cypher).

e.g.,

enstring s = "This is really an encrypted string function";


enstring could wrap opCall which decrypts it when used. This 
might be a drop in replacement for string?

s() calls the decryption function, which is uniquely generated 
for each string. We can drop the ()...

This would definitely add an order of magnitude to the complexity 
of decoding the strings.
Jul 01 2016
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Friday, 1 July 2016 at 23:23:19 UTC, Hiemlick Hiemlicker wrote:

 ok. For some reason I thought CTFE's applied to normal 
 functions but I realize that doesn't make a lot of sense.


It is applied to normal functions, just when they are used in the 
right context.

int a = factorial(3); // normal runtime

static a = factorial(3); // CTFE


Same function, but different contexts. In a static or enum 
variable, it is CTFE'd. In a normal runtime variable, it is 
runtime interpreted.


 Yes, of course. Do D names change depending on -debug vs 
 -release?


No, he meant -g, not -debug. That puts the function names in the 
executable, whereas they might not be there without it (though 
they might be too... I don't think this makes much of a 
difference)

See my post here:
http://stackoverflow.com/a/38149801/1457000

tbh, I don't think encrypting strings is really worth it either, 
they aren't hard to extract anyway.



 I'm not too concerned about the attacker seeing them because 
 they will have to watch every decryption to get the string(or 
 possibly write a utility to automate the decryption).


And that's not really hard.... where's the string going anyway? 
You might want it in a separate file that you can optionally 
encrypt or ask for it from the user.
Jul 01 2016
parent reply Hiemlick Hiemlicker <HH reign.com> writes:
On Friday, 1 July 2016 at 23:55:08 UTC, Adam D. Ruppe wrote:

 On Friday, 1 July 2016 at 23:23:19 UTC, Hiemlick Hiemlicker 
 wrote:

 ok. For some reason I thought CTFE's applied to normal 
 functions but I realize that doesn't make a lot of sense.


 It is applied to normal functions, just when they are used in 
 the right context.

 int a = factorial(3); // normal runtime

 static a = factorial(3); // CTFE


 Same function, but different contexts. In a static or enum 
 variable, it is CTFE'd. In a normal runtime variable, it is 
 runtime interpreted.


 Yes, of course. Do D names change depending on -debug vs 
 -release?


 No, he meant -g, not -debug. That puts the function names in 
 the executable, whereas they might not be there without it 
 (though they might be too... I don't think this makes much of a 
 difference)

 See my post here:
 http://stackoverflow.com/a/38149801/1457000

 tbh, I don't think encrypting strings is really worth it 
 either, they aren't hard to extract anyway.



 I'm not too concerned about the attacker seeing them because 
 they will have to watch every decryption to get the string(or 
 possibly write a utility to automate the decryption).


 And that's not really hard.... where's the string going anyway? 
 You might want it in a separate file that you can optionally 
 encrypt or ask for it from the user.


Ok, Well, I'm curious. I've been string to write a string 
replacement that does the encryption and decryption. It could be 
used for other things like automatic language translation, etc..


Is there a way to write a wrapper around such that

mystring s = "a string that will be converted and not appear in 
binary";
writeln(s);

where s acts as a function that is called to transform the string?

There are two transformations that take place. One is at compile 
time on the string assignment. The other is at the "call site".

I can't seem to get anything to actually work correctly. writeln 
always ends up writing `mystring("whatever the first 
transformation is")`.



a test case


import std.stdio;




template enString(string s)
{
	string processor()
     {
		string q;
		foreach(c; s)
			q ~= c + 1;
		return q;
     }

     enum enString = processor();
}



template e(string s)
{
	de_enString e()
	{
		de_enString x;
		x.val = enString!(s);
		return x;
	}
}

struct de_enString
{
	string val;

	string decrypt()
	{
		string q;
		foreach(c; val)
			q ~= c - 1;
		return q;
	}

	alias this decrypt;
}

void main()
{
	auto s = e!("This is an encrypted string");
	writeln(s);
	getchar();
}

I've tried playing with opCall, opAssign, alias this,  property 
but writeln(s) never calls what I thought it should.

I thought s was short for s() if s was a property. having alias 
this decrypt and decrypt being a  property should allow this to 
work?
Jul 01 2016
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Saturday, 2 July 2016 at 00:05:14 UTC, Hiemlick Hiemlicker 
wrote:

 Is there a way to write a wrapper around such that

 mystring s = "a string that will be converted and not appear in 
 binary";
 writeln(s);


You just need to put a `string toString() { return whatever; }` 
function on `mystring`
Jul 01 2016
parent Hiemlick Hiemlicker <HH reign.com> writes:
On Saturday, 2 July 2016 at 00:29:45 UTC, Adam D. Ruppe wrote:

 On Saturday, 2 July 2016 at 00:05:14 UTC, Hiemlick Hiemlicker 
 wrote:

 Is there a way to write a wrapper around such that

 mystring s = "a string that will be converted and not appear 
 in binary";
 writeln(s);


 You just need to put a `string toString() { return whatever; }` 
 function on `mystring`


Thanks. That did the trick.

No unencrypted string in the binary and hopefully a drop in 
replacement for string.

Although, iteration of a string that was encrypted could be 
problematic.

I'll have to be careful where I use this.
Jul 01 2016
prev sibling parent reply Basile B. <b2.temp gmx.com> writes:
On Saturday, 2 July 2016 at 00:05:14 UTC, Hiemlick Hiemlicker 
wrote:

 On Friday, 1 July 2016 at 23:55:08 UTC, Adam D. Ruppe wrote:

 On Friday, 1 July 2016 at 23:23:19 UTC, Hiemlick Hiemlicker 
 wrote:


 I've tried playing with opCall, opAssign, alias this,  property 
 but writeln(s) never calls what I thought it should.

 I thought s was short for s() if s was a property. having alias 
 this decrypt and decrypt being a  property should allow this to 
 work?


I think the best you can do is this:

========================
import std.stdio;

struct KryptedString(string value)
{
     alias get this;
     string get()  property
     {
         return "decrypt \"" ~ value ~ "\" here";
     }
}

template krypt(string value)
{
     string process()
     {
         return "crypted"; // encrypt the template param
     }
     enum krypt = KryptedString!process();
}

enum string1 = krypt!"blablabla";
enum string2 = krypt!"blablablabla";

void main()
{
     writeln(string1);
     writeln(string2);
}
========================

The syntax is not so bad.

    enum KryptedString string1 = "blablabla";

is impossible or maybe I don't know the trick yet.
Jul 01 2016
next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Saturday, 2 July 2016 at 00:39:57 UTC, Basile B. wrote:

 The syntax is not so bad.

    enum KryptedString string1 = "blablabla";

 is impossible or maybe I don't know the trick yet.


That's a constructor on KryptedString that takes a string.
Jul 01 2016
prev sibling parent reply Hiemlick Hiemlicker <HH reign.com> writes:
On Saturday, 2 July 2016 at 00:39:57 UTC, Basile B. wrote:

 On Saturday, 2 July 2016 at 00:05:14 UTC, Hiemlick Hiemlicker 
 wrote:

 On Friday, 1 July 2016 at 23:55:08 UTC, Adam D. Ruppe wrote:

 On Friday, 1 July 2016 at 23:23:19 UTC, Hiemlick Hiemlicker 
 wrote:


 I've tried playing with opCall, opAssign, alias this, 
  property but writeln(s) never calls what I thought it should.

 I thought s was short for s() if s was a property. having 
 alias this decrypt and decrypt being a  property should allow 
 this to work?


 I think the best you can do is this:

 ========================
 import std.stdio;

 struct KryptedString(string value)
 {
     alias get this;
     string get()  property
     {
         return "decrypt \"" ~ value ~ "\" here";
     }
 }

 template krypt(string value)
 {
     string process()
     {
         return "crypted"; // encrypt the template param
     }
     enum krypt = KryptedString!process();
 }

 enum string1 = krypt!"blablabla";
 enum string2 = krypt!"blablablabla";

 void main()
 {
     writeln(string1);
     writeln(string2);
 }
 ========================

 The syntax is not so bad.

    enum KryptedString string1 = "blablabla";

 is impossible or maybe I don't know the trick yet.


Wait! It almost works! ;)

But you are declaring string 1 and string 2 an enum. If you 
declare them as a string then the original is embedded in the 
binary!

I don't know why that would change anything but it does.

The reason it matter is because if one wanted to do a quick 
change of strings from normal to encrypted, they would also have 
to change all string variables to enums.




import std.stdio;

struct KryptedString(string value)
{
     alias get this;
     string get()  property
     {
		string q;
		foreach(c; value)
			q ~= c + 1;
		return q;
     }
}

template krypt(string value)
{
     string process()
     {
		string q;
		foreach(c; value)
			q ~= c - 1;
		return q;
     }
     enum krypt = KryptedString!process();
}

string string1 = krypt!"Testing 1 2 3 4";
string string2 = krypt!"ttttttaaaabbccd";

void main()
{
     writeln(string1);
     writeln(string2);
     getchar();
}

I guess one could do

enum string1e = krypt!"Testing 1 2 3 4";

then

string string1 = string1e;

but the goal is to make as clean as possible ;)
Jul 01 2016
next sibling parent Basile B. <b2.temp gmx.com> writes:
On Saturday, 2 July 2016 at 01:31:17 UTC, Hiemlick Hiemlicker 
wrote:

 On Saturday, 2 July 2016 at 00:39:57 UTC, Basile B. wrote:

 On Saturday, 2 July 2016 at 00:05:14 UTC, Hiemlick Hiemlicker 
 wrote:

 On Friday, 1 July 2016 at 23:55:08 UTC, Adam D. Ruppe wrote:

 On Friday, 1 July 2016 at 23:23:19 UTC, Hiemlick Hiemlicker 
 wrote:






 Wait! It almost works! ;)

 But you are declaring string 1 and string 2 an enum.


On purpose


 If you declare them as a string then the original is embedded 
 in the binary!


Ditto.

If you don't store it at compile time, the non-encrypted form, so 
the literal, will be in the DATA segment.
Jul 01 2016
prev sibling parent Mike Parker <aldacron gmail.com> writes:
On Saturday, 2 July 2016 at 01:31:17 UTC, Hiemlick Hiemlicker 
wrote:



 But you are declaring string 1 and string 2 an enum. If you 
 declare them as a string then the original is embedded in the 
 binary!

 I don't know why that would change anything but it does.

 The reason it matter is because if one wanted to do a quick 
 change of strings from normal to encrypted, they would also 
 have to change all string variables to enums.


This is known as a manifest constant [1]:

enum myVal = 10;

When the compiler encounters any use of myVal, it will insert its 
value directly at the point of usage. myVal will not appear in 
the data segment. You cannot take its address.

[1] http://dlang.org/spec/enum.html#manifest_constants
Jul 01 2016
prev sibling next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Friday, 1 July 2016 at 22:23:23 UTC, Hiemlick Hiemlicker wrote:

 This also seems like a bug in D because manifest constants used 
 as sole arguments to ctfe'able functions should be replaced by 
 the function result.


I wouldn't call it a bug, but it could be seen as an enhancement 
request, in a similar vein to keeping the unused ctfe functions 
themselves out of the binary.
Jul 01 2016
prev sibling parent ag0aep6g <anonymous example.com> writes:
On 07/02/2016 12:23 AM, Hiemlick Hiemlicker wrote:

 This also seems like a bug in D because manifest constants used as sole
 arguments to ctfe'able functions should be replaced by the function result.


No. There are functions that don't have any dynamic input, but still 
take a long time to finish or don't finish at all. Eagerly attempting 
CTFE on these would be bad.

The compiler is supposed to produce a binary, which can then be run to 
make the computation. The compiler is not supposed to eagerly act as an 
interpreter and take forever to make the computation itself.
Jul 01 2016