www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - [idea] self-checking option for release versions

reply "Manfred Nowak" <svv1999 hotmail.com> writes:
A couple of month ago I had a discussion in this forum on
usage-prevention-chemes.

www.strongbit.com has a software product called execryptor that promises
even more than that I was talking about, i.e. making the task of analyzing
an executable NP-Hard.

Maybe that this is a technic that should be incorporated into compilers in
the long run.

As a first step it may be sufficient to protect executables against the most
known automatic plague: viruses.

A mounted cheme that reads an md5 checksum incorporated into the executable
and checks the not-md5 checksum part of the executable for consistency with
the checksum part would detect the result of all modifications to the
executable of viruses that are unaware of the integrated self-checking
cheme.

On the other hand every virus that would be aware of the selfchecking option
must be able to find it and switch it off or must find and change the md5
checksum according to the changes that the virus wants to do to the
executable.

Hopefully this searching for the critical points of the mounted cheme of
self checking creates a signature that all self-checking aware viruses must
show, thereby easying the task of antivirus software.

Is this an idea without any impact or is D a candidate to be the first
language that incorporates some antivirus strategy?

-manfred
Jan 14 2005
next sibling parent reply parabolis <parabolis softhome.net> writes:
Manfred Nowak wrote:

 A couple of month ago I had a discussion in this forum on
 usage-prevention-chemes.


I found your repeated use of 'cheme' to be both mauve and 
gustipating. I am assuming you meant 'scheme'?


 www.strongbit.com has a software product called execryptor that promises
 even more than that I was talking about, i.e. making the task of analyzing
 an executable NP-Hard.


Security app promises a new paradigm but does not publish 
algorithm and uses phrases like 'nondeterminate 
transformations'. I have seen this one already. It does not turn 
out well.

Further reading just made me wince more. EXECryptor sounds like 
it should be named the AntiOptimizer. It bloats the compiled 
code to 'a couple of dozens times' the original and sounds like 
it kills almost any attempts at optimization in the process.


 As a first step it may be sufficient to protect executables against the most
 known automatic plague: viruses.
 
 A mounted cheme that reads an md5 checksum incorporated into the executable
 and checks the not-md5 checksum part of the executable for consistency with
 the checksum part would detect the result of all modifications to the
 executable of viruses that are unaware of the integrated self-checking
 cheme.


It would if a virus that prepends itself to the compiled code 
ever decides to actually pass control back to your code and does 
not remove itself from the compiled code before restarting the 
executable.
Jan 14 2005
parent "Manfred Nowak" <svv1999 hotmail.com> writes:
parabolis wrote:

 I found your repeated use of 'cheme' to be both mauve and
 gustipating. I am assuming you meant 'scheme'?


Thanks for pointing this out correctly. It was 'worng', 'Bad speling' and
may be choosen as an indicator of my severe 'dain bramage' :-)


 Further reading just made me wince more. EXECryptor sounds like
 it should be named the AntiOptimizer. It bloats the compiled
 code to 'a couple of dozens times' the original and sounds like
 it kills almost any attempts at optimization in the process.


At first glance I agree with you..


 It would if a virus that prepends itself to the compiled code
 ever decides to actually pass control back to your code and does
 not remove itself from the compiled code before restarting the
 executable.


At present you are right again. But with the growing number of
processors that incorporate hardware protection against writing into
unallowed areas the viruses  types that stay in or want to remove
themselves from main memory will go into their happy hunting grounds.
And the classification of code as a virus depends on the definition. I
prefer the definition that a virus is executable code that propagates itself
by turning other executables into trojan horses ... and an executable
that does not do anything is clearly not a trojan horse.

-manfred
Jan 15 2005
prev sibling next sibling parent reply "Asaf Karagila" <kas1 netvision.net.il> writes:
 www.strongbit.com has a software product called execryptor that promises
 even more than that I was talking about, i.e. making the task of analyzing
 an executable NP-Hard.


execryptors are not just a mere md5 checksum,
they are usually being used against crackers and reversers to protect the 
code.
most of them have been unpacked successfully.
due to this fact, the code is encrypted in several layers,
sometimes there's some "evaluation notify" screen added,
it's mostly used by lazy programmers that have no clue how to protect their
code properly,
viruses on the other hand are far trickier since they are parasiting on your 
code.
and due to that, they load themselves before your code is actually executed,
thus, they have the ability to disappear (as parabolic mentioned), or 
analyze
your code (in search for D compiler signature) and alter any built-in check.

although it's a nice feature to have a self checksum integrated into the 
code
by default, it's pretty useless in protection serious pests or smart 
crackers.

- Asaf.
Jan 14 2005
parent reply "Manfred Nowak" <svv1999 hotmail.com> writes:
Asaf Karagila wrote:

 although it's a nice feature to have a self checksum integrated into the
 code by default, it's pretty useless in protection serious pests or
 smart crackers.


Smart people are unstoppable by defintion :-)
And you are right so far: because a complete shield against all types of
viruses needs a supervising instance, that instance will become the target
of virus attacks until that again is shielded by another super-supervising
instance ...and so on. The only hope to break this coevolving cycle is, that
the code a virus needs to have is so blurred that the pure mass of code
makes it easy to detect it. If someone can proof that this will not happen,
then there is no need to start with. If someone can proof the counterpart,
then there is an urgent need to start. Shouldn't one start in the hope that
someone can proof the counterpart?

-manfred
Jan 15 2005
next sibling parent "Asaf Karagila" <kas1 netvision.net.il> writes:
"Manfred Nowak" <svv1999 hotmail.com> wrote in message 
news:csb073$2158$1 digitaldaemon.com...

 And you are right so far: because a complete shield against all types of
 viruses needs a supervising instance


well, there has been a contest of protecting an applicated in some reverse 
engineering site,
one of the submitted solutions injected codes to random processes and 
created remote threads
that would checksum the original code, he also included a lot of protection 
codes.
it was a brilliant solution, won first place i believe. but the contests 
site is down.
if i'll manage to dig up the files, or if the site is up again, i'll post it 
here.

- Asaf.
Jan 15 2005
prev sibling parent reply parabolis <parabolis softhome.net> writes:
Manfred Nowak wrote:


 Smart people are unstoppable by defintion :-)
 And you are right so far: because a complete shield against all types of
 viruses needs a supervising instance, that instance will become the target
 of virus attacks until that again is shielded by another super-supervising
 instance ...and so on. The only hope to break this coevolving cycle is, that
 the code a virus needs to have is so blurred that the pure mass of code
 makes it easy to detect it.


I think Microsoft found an answer to the 'supervising instance' 
problem. They are working with hardware people to ensure that 
signed startup code is not executed unless the startup code has 
not been changed.

This startup code can then be used to ensure the rest of the OS 
code has not been mucked with and will also allow the OS to 
guarantee your signed code has not been changed either.
Jan 15 2005
parent reply "Manfred Nowak" <svv1999 hotmail.com> writes:
parabolis wrote:

 They are working with hardware people to ensure that
 signed startup code is not executed unless the startup code has
 not been changed.


Would you please point out a reference? Thanks in advance.


 This startup code can then be used to ensure the rest of the OS
 code has not been mucked with and will also allow the OS to
 guarantee your signed code has not been changed either.


But only if the complete OS is selfchecking, right? Otherwise
that would imply the end of all viruses.

-manfred
Jan 15 2005
next sibling parent reply "Asaf Karagila" <kas1 netvision.net.il> writes:
"Manfred Nowak" <svv1999 hotmail.com> wrote in message 
news:csbb31$2c30$1 digitaldaemon.com...

 that would imply the end of all viruses.


and non-MS operation systems..
i think that despite the good idea about have an OS incorporated into the 
CPU,
this movement would give much power to MS.
too much power if you ask me..

the ideas of cracking, viruses, and all that..
it's much less scares me than having no alternative.
at least now, we have a certain amount of freedom. who can tell what will 
happened
when the Hexium will be spreaded across the land ?

- Asaf.
Jan 15 2005
parent "Manfred Nowak" <svv1999 hotmail.com> writes:
Asaf Karagila wrote:

[...]

 when the Hexium will be spreaded across the land ?


http://satirist.org/whale/2003/10/30.html :D

-manfred
Jan 15 2005
prev sibling parent reply parabolis <parabolis softhome.net> writes:
Manfred Nowak wrote:

 parabolis wrote:
 

They are working with hardware people to ensure that
signed startup code is not executed unless the startup code has
not been changed.


 
 Would you please point out a reference? Thanks in advance.


Search google for Palladium (MS's next big OS) and TCPA (Trusted 
Platform Alliance). Here is a TCPA faq to get you started:

http://www.cl.cam.ac.uk/~rja14/tcpa-faq.html

A short exerpt from that FAQ:
----------------------------------------------------------------
The current version has Fritz as a passive monitoring component 
that stores the hash of the machine state on start-up. This hash 
is computed using details of the hardware (audio card, video 
card etc) and the software (O/S, drivers, etc). If the machine 
ends up in the approved state, Fritz will make available to the 
operating system the cryptographic keys needed to decrypt TC 
applications and data. If it ends up in the wrong state, the 
hash will be wrong and Fritz won't release the right key. The 
machine may still be able to run non-TC apps and access non-TC 
data, but protected material will be unavailable.

The operating system security kernel (the `Nexus') bridges the 
gap between the Fritz chip and the application security 
components (the `NCAs'). It checks that the hardware components 
are on the TCG approved list, that the software components have 
been signed, and that none of them has a serial number that has 
been revoked.
...
When it first launched TC as Palladium, Microsoft claimed that 
Palladium would stop spam, viruses and just about every other 
bad thing in cyberspace.
----------------------------------------------------------------
Jan 15 2005
parent reply Juanjo =?iso-8859-1?Q?=C1lvarez?= writes:
On 2005-01-15, parabolis <parabolis softhome.net> wrote:

 Manfred Nowak wrote:

 parabolis wrote:


 components (the `NCAs'). It checks that the hardware components 
 are on the TCG approved list, that the software components have 
 been signed, and that none of them has a serial number that has 
 been revoked.
 ...
 When it first launched TC as Palladium, Microsoft claimed that 
 Palladium would stop spam, viruses and just about every other 
 bad thing in cyberspace.
 ----------------------------------------------------------------


Also please note that free software would be one the the very bad things
(for Microsoft) that Palladium could stop.
Jan 15 2005
parent reply "Thomas Kuehne" <thomas-dloop kuehne.cn> writes:
 components (the `NCAs'). It checks that the hardware components
 are on the TCG approved list, that the software components have
 been signed, and that none of them has a serial number that has
 been revoked.
 ...
 When it first launched TC as Palladium, Microsoft claimed that
 Palladium would stop spam, viruses and just about every other
 bad thing in cyberspace.





 Also please note that free software would be one the the very bad things
 (for Microsoft) that Palladium could stop.


The basic concept of assigning access rights based on "trust" is quite usefull.

(On one of my system only executeables with an embedded valid & trusted GPG
signature are
allowed to be loaded.)

The problem with Palladium is how the "trust" is managed.

1) The local administrator isn't in controll. This leads to lots of legal
problems
- at least here in Old Europe.
2) Online access is required.

Many of todays Bad Soft(tm) are working on the script level. I'm sure IE, OE
and MsOffice
will be trusted ...

Thomas
Jan 16 2005
parent reply Juanjo =?iso-8859-1?Q?=C1lvarez?= writes:
On 2005-01-16, Thomas Kuehne <thomas-dloop kuehne.cn> wrote:

 Also please note that free software would be one the the very bad things
 (for Microsoft) that Palladium could stop.


 The basic concept of assigning access rights based on "trust" is quite usefull.



 (On one of my system only executeables with an embedded valid & trusted GPG
signature are
 allowed to be loaded.)

 The problem with Palladium is how the "trust" is managed.

 1) The local administrator isn't in controll. This leads to lots of legal
problems
 - at least here in Old Europe.
 2) Online access is required.


And probably:
  3) To obtain a certificate for your software that works with Palladium
  hardware you must pay Microsoft (or Intel or...), for every new
  version and minor release. 


 Many of todays Bad Soft(tm) are working on the script level. I'm sure IE, OE
and MsOffice
 will be trusted ...


Sure.
Jan 16 2005
parent "Asaf Karagila" <kas1 netvision.net.il> writes:
"Juanjo ?lvarez" <juanjo nospamplease_juanjoalvarez.net> wrote in message 
news:csf8qi$5jm$1 digitaldaemon.com...

 On 2005-01-16, Thomas Kuehne <thomas-dloop kuehne.cn> wrote:

 Also please note that free software would be one the the very bad things
 (for Microsoft) that Palladium could stop.


 The basic concept of assigning access rights based on "trust" is quite 
 usefull.



 (On one of my system only executeables with an embedded valid & trusted 
 GPG signature are
 allowed to be loaded.)

 The problem with Palladium is how the "trust" is managed.

 1) The local administrator isn't in controll. This leads to lots of legal 
 problems
 - at least here in Old Europe.
 2) Online access is required.


 And probably:
  3) To obtain a certificate for your software that works with Palladium
  hardware you must pay Microsoft (or Intel or...), for every new
  version and minor release.


 Many of todays Bad Soft(tm) are working on the script level. I'm sure IE, 
 OE and MsOffice
 will be trusted ...


 Sure.


and this is EXACTLY what i was refering to as misused power.
this is the reason i prefer viruses over someone telling which files i am 
allowed to load or not.
especially when it's MS and Intel in charge.
and i mean hey, if MS has no bugs in their software, i'm that the hardware 
will be perfect !

- Asaf.
Jan 16 2005
prev sibling next sibling parent reply Norbert Nemec <Norbert Nemec-online.de> writes:
Manfred Nowak wrote:

 Maybe that this is a technic that should be incorporated into compilers in
 the long run.


Making something in that direction part of the compiler seems like a bad
idea. Post-processing executables for specific purposes can be done
perfectly well in a separate stage after compilation. There are many
different needs for post-processing (compression, encryption, checksumming,
prelinking, etc.) and technology evolves quickly. Builing anything like
that into the compiler will kill flexibility and unnecessarily increase the
complexity of the compiler. Modularity seems a far better approach, unless
the individual stages are inherently interconnected.
Jan 15 2005
parent reply "Manfred Nowak" <svv1999 hotmail.com> writes:
Norbert Nemec wrote:
[...]

 Post-processing executables for specific purposes can be done
 perfectly well in a separate stage after compilation.


Is this a defintion? That would mean, that those tasks that cannot be done
perfectly well in a separate stage after compilation have to be built into
the compiler. Is it imperfect enough if you have to renanalyze the just
generated program to find the places where you are able to insert checksums
and routines for checking, thereby increasing the needed cpu-time from one
unit to at least units in the order of the length of the program?


 Modularity seems a far better approach, unless
 the individual stages are inherently interconnected.


So please explain what the currently implemented release option is doing
that is inherently interconnected with the compiling phase and cannot
perfectly well be done in a separate stage after compilation.

-manfred
Jan 15 2005
parent reply Norbert Nemec <Norbert Nemec-online.de> writes:
Manfred Nowak wrote:


 
 Norbert Nemec wrote:
 [...]

 Post-processing executables for specific purposes can be done
 perfectly well in a separate stage after compilation.


 Is this a defintion? That would mean, that those tasks that cannot be done
 perfectly well in a separate stage after compilation have to be built into
 the compiler. Is it imperfect enough if you have to renanalyze the just
 generated program to find the places where you are able to insert
 checksums and routines for checking, thereby increasing the needed
 cpu-time from one unit to at least units in the order of the length of the
 program?


OK, if the checksum - or whatever - algorithm goes so deep into the
structure of the program, it would make sense to build it into the
compiler.


 Modularity seems a far better approach, unless
 the individual stages are inherently interconnected.


 So please explain what the currently implemented release option is doing
 that is inherently interconnected with the compiling phase and cannot
 perfectly well be done in a separate stage after compilation.


I'm not sure what you mean with "release option". If you talk of "release"
vs. "debug" mode, then this option is not a separate stage after
compilation but an option that affects the code generate stage.

In any case: I still think, that self-checking binaries should be left as a
separate project. There are many different purposes for checksumming of
binaries (protection against viruses, intrusion, general corruption,
probably many more) Finding one "general purpuse" checksumming method will
result in something that is suboptimal for at least some of the purposes.
If a system administrator has special need for checksumming, it should be
up to him to decide on the details. In any other case, checksumming would
just cost space and performance with little gain. (Actually, for my
personal machine under Linux, I see no point why certain applications
should be self-checking. If I knew that every binary is self-checking,
there might be a certain gain, but if it is depending on the programming
language, the feature is mostly worthless.
Jan 15 2005
next sibling parent reply "Manfred Nowak" <svv1999 hotmail.com> writes:
Norbert Nemec wrote:


 I'm not sure what you mean with "release option". If you talk of "release"
 vs. "debug" mode, then this option is not a separate stage after
 compilation but an option that affects the code generate stage.


Yes correct: I mean the '-release' option of  'dmd'. And your pure mention,
that
the code generating phase is affected is according to your own argument at
least
incomplete: Why can this switching off of at least asserts, array bounds
checks
and switching on of  code optimization not be done perfectly well in a
postprocessing phase?

[...]

 If a system administrator has special need for checksumming, it should be
 up to him to decide on the details.


Correct. But adminstrators mostly prefer to decide, not to code. So her
decision in case of  lack of an existing decision possibility would not be
"Ahh. Here is a D-Source. I want to code the self checking  I need for it."
but rather "Ahh. Here is a D-Source. Nope. That _was_ a D-Source."

[...]

 If I knew that every binary is self-checking, there might be a certain


gain,
Thanks for the respect.


 but if it is depending on the programming language, the feature is mostly
 worthless.


Why? You just pointed out, that in your opinion the self checking
instrumentation can and should be done in a post processing phase. Why
is this true only for D and not for any of the other languages of that
particular adminstrator?

-manfred
Jan 15 2005
parent reply Norbert Nemec <Norbert Nemec-online.de> writes:
I fear, this whole thread is going nowhere. Essentially it falls back to: If
you think it is essential, either code it yourself or find someone else to
code it. I have the feeling that so far nobody has spoken up for the task. 

Personally, I have no need whatsoever for checksumming and I doubt that D as
a language would gain a lot by it. There are many other feature that might
be nice to have which nobody ever implemented. Maybe this is just one more
of them.

If you think about implementing a checksumming algorithm, you might soon
realize that it is simpler to write a language-ignorant program that just
takes an executable and spits out another one. It may have disadvantages
compared to one built into the compiler, but it certainly has some
advantages
* It works for any language and even third-party programs that you did not
compile yourself.
* You don't have to convince any compiler-writer to include it.
* If anybody thinks it should be done differently, they can just write their
own version any you don't have to fight whose gets into the compiler.
Jan 16 2005
parent "Asaf Karagila" <kas1 netvision.net.il> writes:
yes,
but just for the reference, if you want to write an executable compressor,
take a look at UPX, which is great, free and open source.
if you want to write a protector.. well, i suggest you spend 3 years or more
learning reverse code engineering, PE and COFF formats, encryption,
compression, etc etc etc. and maybe just maybe, you'll manage to write
a decent protector.
illusioning yourself to be able to write a good protector with a mere idea,
your chance to succeed is smaller than the chance of a tsunami hitting 
texas.
(sorry for the cynical remark if i offended anyone..)
it's like writing a very simple serial check routine and think it's 
undefeatable
because you're XORing the value, and no one can guess the result.

my entire point is, most of the programmers can't protect their code right.
either find a GOOD (and proved as good, not by the company as well)
protector, or follow my first paragraph suggestions.

- Asaf.
Jan 16 2005
prev sibling parent Georg Wrede <georg.wrede nospam.org> writes:
I'd rather have the checking done by the operating system.
If you can't trust the operating system, then no checking
of any kind is worthwhile. (So this issue is not relevant
to products from Redmond.)

Programs should come with chekcsums in a separate file,
and those checksums should also be on the net, for
manual checking.

Normally the operating system would checksum an executable
each time it is invocated. Any modification to the file
at runtime should be an "exe in use" exception.

On the other hand, if you have an operating system that
is trustworthy, then your main enemy would mostly only
be trojans. This could be taken care of if every single
executable or dll would have to be signed. The trojan
manufacturer would then either have to sign the code, or
forge it in some (hopefully) labourious way.

This leaves trojans hidden in source code distributions.
So these could be forbidden.   :-)

Seems we're more than skin deep in trouble. And are
going to stay that way.


Norbert Nemec wrote:

 Manfred Nowak wrote:
 
 

Norbert Nemec wrote:
[...]


Post-processing executables for specific purposes can be done
perfectly well in a separate stage after compilation.


Is this a defintion? That would mean, that those tasks that cannot be done
perfectly well in a separate stage after compilation have to be built into
the compiler. Is it imperfect enough if you have to renanalyze the just
generated program to find the places where you are able to insert
checksums and routines for checking, thereby increasing the needed
cpu-time from one unit to at least units in the order of the length of the
program?


 
 
 OK, if the checksum - or whatever - algorithm goes so deep into the
 structure of the program, it would make sense to build it into the
 compiler.
 
 

Modularity seems a far better approach, unless
the individual stages are inherently interconnected.


So please explain what the currently implemented release option is doing
that is inherently interconnected with the compiling phase and cannot
perfectly well be done in a separate stage after compilation.


 
 
 I'm not sure what you mean with "release option". If you talk of "release"
 vs. "debug" mode, then this option is not a separate stage after
 compilation but an option that affects the code generate stage.
 
 In any case: I still think, that self-checking binaries should be left as a
 separate project. There are many different purposes for checksumming of
 binaries (protection against viruses, intrusion, general corruption,
 probably many more) Finding one "general purpuse" checksumming method will
 result in something that is suboptimal for at least some of the purposes.
 If a system administrator has special need for checksumming, it should be
 up to him to decide on the details. In any other case, checksumming would
 just cost space and performance with little gain. (Actually, for my
 personal machine under Linux, I see no point why certain applications
 should be self-checking. If I knew that every binary is self-checking,
 there might be a certain gain, but if it is depending on the programming
 language, the feature is mostly worthless.
Jan 19 2005
prev sibling parent reply Gold Dragon <dragonwing dragonu.net> writes:
Problem is that no only does the Language have to support but the OS and as
people have said they 
don't want the OS to control that aspect. If and only if the OS can store the
checksum in a secure 
location that only the OS can access, then I would say it is a good idea.
Actually, I think this 
idea would be better if it was implemented in the OS anyway, since the OS
controls memory access (in 
most cases) and should be able to seek out attacks.

If it is stored in the Registry, then it would be more of a burden than
anything useful. Another 
problem is that some programs are updated and are dynamic so how does a program
update their 
checksum or would the OS update the checksum? If the OS updated the Checksum
then how does it know 
that it wasn't a virus that changed something. Microsoft probably won't be
doing this... ever, even 
if they did rewrite the OS.
Jan 17 2005
parent "Asaf Karagila" <kas1 netvision.net.il> writes:
it's probably doable as a hooking into CreateProcess or something..
you can inject code that will have a certain checksum (if the file is
on a registered files list or whatever..)
the problem is to control it in a way that won't dramatically decrease
the OS preformance.

and it's a HUGE pain in the ass. (code injection into the kernel and all..)
- Asaf.

"Gold Dragon" <dragonwing dragonu.net> wrote in message 
news:csg0ue$v3d$1 digitaldaemon.com...

 Problem is that no only does the Language have to support but the OS and 
 as people have said they don't want the OS to control that aspect. If and 
 only if the OS can store the checksum in a secure location that only the 
 OS can access, then I would say it is a good idea. Actually, I think this 
 idea would be better if it was implemented in the OS anyway, since the OS 
 controls memory access (in most cases) and should be able to seek out 
 attacks.

 If it is stored in the Registry, then it would be more of a burden than 
 anything useful. Another problem is that some programs are updated and are 
 dynamic so how does a program update their checksum or would the OS update 
 the checksum? If the OS updated the Checksum then how does it know that it 
 wasn't a virus that changed something. Microsoft probably won't be doing 
 this... ever, even if they did rewrite the OS.
Jan 17 2005