www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 1360] New: GC emits HLT when GetThreadContext fails after CreateRemoteThread.

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=1360

           Summary: GC emits HLT when GetThreadContext fails after
                    CreateRemoteThread.
           Product: D
           Version: 2.002
          Platform: PC
        OS/Version: Windows
            Status: NEW
          Severity: critical
          Priority: P2
         Component: Phobos
        AssignedTo: bugzilla digitalmars.com
        ReportedBy: wqeqweuqy hotmail.com


If an external process trys to CreateRemoteThread the current process (to
inject a DLL in this case) the gcx stack scanning code fails in
GetThreadContext(). 

GetLastError() returns ERROR_GEN_FAILURE and the code hard exists via a HLT
instruction.

Seems a few anti virus/spyware utilities use this injection method, so its
likely to be a problem.


--
Jul 20 2007
next sibling parent Sean Kelly <sean f4.ca> writes:
d-bugmail puremagic.com wrote:

 
 If an external process trys to CreateRemoteThread the current process (to
 inject a DLL in this case) the gcx stack scanning code fails in
 GetThreadContext().


I'm not sure there's anything that can be done about this.  The stack 
scanning routine only calls GetThreadContext on threads it knows about 
(ie. D threads), so it's unlikely GetThreadContext is interacting 
directly with the injected thread.  More likely, and more troubling, is 
that injecting a thread is screwing up the application somehow and 
causing it to fail.  If I had to guess, I'd say that the injected thread 
was using debug routines (specifically, SuspendThread and ResumeThread) 
at the same time a collection cycle was occurring, and it resumed a 
thread that was supposed to be suspended for scanning or something like 
that.

More remote is that the process of injecting a thread is screwing 
something up.  From the docs on CreateRemoteThread:

     there are several side effects to using this technique:

     * It converts single-threaded applications into multi-threaded
       applications.
     * It changes the timing and memory layout of the process.
     * It results in a call to the entry point of each DLL in the
       process.

Normally, I'd think these should all be fine, but given the error, I'm 
wondering what the second clause implies.


 GetLastError() returns ERROR_GEN_FAILURE and the code hard exists via a HLT
 instruction.


Unfortunately, this isn't terribly useful.  Near as I can tell, 
ERROR_GEN_FAILURE just indicates a general failure condition.  The 
actual text description is "a device attached to the system is not 
functioning."


 Seems a few anti virus/spyware utilities use this injection method, so its
 likely to be a problem.


A further comment on the function:

     Another common use of this function is to inject a thread into a
     process to query heap or other process information. This can cause
     the same side effects mentioned in the previous paragraph. Also, the
     application can deadlock if the thread attempts to obtain ownership
     of locks that another thread is using.

This doesn't sound terribly encouraging.  Is the error reproducible? 
I'm curious if it's a timing issue.


Sean
Jul 21 2007
prev sibling next sibling parent reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=1360








 d-bugmail puremagic.com wrote:

 
 If an external process trys to CreateRemoteThread the current process (to
 inject a DLL in this case) the gcx stack scanning code fails in
 GetThreadContext().


 
 I'm not sure there's anything that can be done about this.  The stack 
 scanning routine only calls GetThreadContext on threads it knows about 
 (ie. D threads), so it's unlikely GetThreadContext is interacting 
 directly with the injected thread.  More likely, and more troubling, is 
 that injecting a thread is screwing up the application somehow and 
 causing it to fail.  If I had to guess, I'd say that the injected thread 
 was using debug routines (specifically, SuspendThread and ResumeThread) 
 at the same time a collection cycle was occurring, and it resumed a 
 thread that was supposed to be suspended for scanning or something like 
 that.
 
 More remote is that the process of injecting a thread is screwing 
 something up.  From the docs on CreateRemoteThread:
 
      there are several side effects to using this technique:
 
      * It converts single-threaded applications into multi-threaded
        applications.
      * It changes the timing and memory layout of the process.
      * It results in a call to the entry point of each DLL in the
        process.
 
 Normally, I'd think these should all be fine, but given the error, I'm 
 wondering what the second clause implies.
 

 GetLastError() returns ERROR_GEN_FAILURE and the code hard exists via a HLT
 instruction.


 
 Unfortunately, this isn't terribly useful.  Near as I can tell, 
 ERROR_GEN_FAILURE just indicates a general failure condition.  The 
 actual text description is "a device attached to the system is not 
 functioning."
 

 Seems a few anti virus/spyware utilities use this injection method, so its
 likely to be a problem.


 
 A further comment on the function:
 
      Another common use of this function is to inject a thread into a
      process to query heap or other process information. This can cause
      the same side effects mentioned in the previous paragraph. Also, the
      application can deadlock if the thread attempts to obtain ownership
      of locks that another thread is using.
 
 This doesn't sound terribly encouraging.  Is the error reproducible? 
 I'm curious if it's a timing issue.
 
 
 Sean
 


I dont think its a race condition with the thread being created during a
collection cycle, it happens everytime a GC collect is run (with the exception
below). 

This bug had been affecting one of my users for a long time and i could never
figure out what on his PC was causing it to fail (ended up being some anti
spyware stuff injecting DLLs like this somewhere along the line). By chance,
someone had sent me some loader and it was producing the same symptoms that the
one user was having.

Long story short: This loader called CreateProcess with CREATE_SUSPENDED flag,
injected its DLL into the process with CreateRemoteThread, and hit resume. 

I ended up copying the thread enumeration code out of the gcx function:

Thread[] threads = Thread.getAll();
for (uint n = 0; n < threads.length; n++)
{
    Thread t = threads[n];
    if (t && t.getState() == Thread.TS.RUNNING)
    {
        display.printf("Thread[%d]: Thread.isSelf() = %d\n", n, t.isSelf());
        CONTEXT context;
        context.ContextFlags = CONTEXT_INTEGER | CONTEXT_CONTROL;
        if (!GetThreadContext(t.hdl, &context))
            display.printf("GetThreadContext: handle=%08x id=%08x error=%d\n",
t.hdl, GetCurrentThreadId(), GetLastError());
    }
}

produces the output:

Thread[0]: Thread.isSelf() = 0
GetThreadContext: handle=0120 id=00000D50 error=31 (ERROR_GEN_FAILURE)

I should be doing something like GetThreadId(0x0120) here but its vista only,
and i'm on XPSP2. Checking with a debugger shows the main thread is 0xd50.
Assuming the isSelf() isnt failing for another reason, its possible the thread
the GC consumed as "Main" thread at gc_init is pointing to the
CreateRemoteThread one or something. 

I also tried to run a collection cycle in the D programs DLLmain and it worked
ok. Im not sure if thats just because the stack scanning code wasnt hit, due to
there being nothing allocated, or from the CreateRemoteThread thread being
active still or something.


--
Jul 21 2007
parent Sean Kelly <sean f4.ca> writes:
d-bugmail puremagic.com wrote:

 
 I dont think its a race condition with the thread being created during a
 collection cycle, it happens everytime a GC collect is run (with the exception
 below). 
 
 This bug had been affecting one of my users for a long time and i could never
 figure out what on his PC was causing it to fail (ended up being some anti
 spyware stuff injecting DLLs like this somewhere along the line). By chance,
 someone had sent me some loader and it was producing the same symptoms that the
 one user was having.
 
 Long story short: This loader called CreateProcess with CREATE_SUSPENDED flag,
 injected its DLL into the process with CreateRemoteThread, and hit resume.


I think I see the issue.  More below.


 I ended up copying the thread enumeration code out of the gcx function:
 
 Thread[] threads = Thread.getAll();
 for (uint n = 0; n < threads.length; n++)
 {
     Thread t = threads[n];
     if (t && t.getState() == Thread.TS.RUNNING)
     {
         display.printf("Thread[%d]: Thread.isSelf() = %d\n", n, t.isSelf());
         CONTEXT context;
         context.ContextFlags = CONTEXT_INTEGER | CONTEXT_CONTROL;
         if (!GetThreadContext(t.hdl, &context))
             display.printf("GetThreadContext: handle=%08x id=%08x error=%d\n",
 t.hdl, GetCurrentThreadId(), GetLastError());
     }
 }
 
 produces the output:
 
 Thread[0]: Thread.isSelf() = 0
 GetThreadContext: handle=0120 id=00000D50 error=31 (ERROR_GEN_FAILURE)
 
 I should be doing something like GetThreadId(0x0120) here but its vista only,
 and i'm on XPSP2. Checking with a debugger shows the main thread is 0xd50.
 Assuming the isSelf() isnt failing for another reason, its possible the thread
 the GC consumed as "Main" thread at gc_init is pointing to the
 CreateRemoteThread one or something. 


That's exactly it.  I don't understand why the injected app would be 
initialized within the context of the injecting thread (does this app do 
its initialization manually in DLLMain or something?), but that appears 
to be what's happening.  If you look in std.thread:Thread.thread_init() 
(a routine that's called by gc_init), you can see how a proxy object 
representing the 'main' thread is created.


 I also tried to run a collection cycle in the D programs DLLmain and it worked
 ok. Im not sure if thats just because the stack scanning code wasnt hit, due to
 there being nothing allocated, or from the CreateRemoteThread thread being
 active still or something.


The latter, I think.  And you're saying this is just a plain old D 
application which this is being done to?  I'll admit I really don't 
understand why its C main() routine would be called by the injected 
thread in such a situation, but perhaps I simply haven't read the MS 
docs thoroughly enough yet.  I'll give them another look and see if a 
solution can be found.


Sean
Jul 21 2007
prev sibling next sibling parent reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=1360






- The "target" application is in C++. 
- The D code is a DLL that wraps some directx COM interfaces and hooks the
target.
- The problematic loader is in C++ using CreateRemoteThread with LoadLibrary.

The loader is injecting the D DLL into the target application. D DLL's Dllmain
calls gc_init, m_init etc for PROCCESS_ATTACH.

The thing is, using the "madChook" library to inject the dll instead, through
its version of "CreateProcessEx" or its system wide "inject on process creation
through a kernel driver" method works perfectly fine. AFAIK both his methods
inject before the target program's entry point is called too. 

It seems like the presence of a CreateRemoteThread call alone is messing up the
GC. Since the GC worked fine, using the madChook method, on that one user's PC
after uninstalling everything that could possibly be injecting unrelated dlls
with CreateRemoteThread. Yahoo messenger, Norton etc. 

It might work better to do a seperate scan of GetCurrentThread() instead of
having a resident non-D Main thread, but that leads to problems if you dont run
a collection cycle before D functions returns to non-D-space. 

A simple solution for DLLs might be to manually initalize what the GC uses as
the main thread somewhere along the line (if needed). And allow it to fall back
on FullCollectNoStack() if theres no Main thread registered. I'm not sure if
this will result in leaks or not though. 

Also, the behavior on that one users PC makes it seem like it could be
something unrelated to initialization that is breaking things too. 

Seems like this is going to be messy to deal with haha.


--
Jul 21 2007
parent Sean Kelly <sean f4.ca> writes:
d-bugmail puremagic.com wrote:


 - The "target" application is in C++. 
 - The D code is a DLL that wraps some directx COM interfaces and hooks the
 target.
 - The problematic loader is in C++ using CreateRemoteThread with LoadLibrary.
 
 The loader is injecting the D DLL into the target application. D DLL's Dllmain
 calls gc_init, m_init etc for PROCCESS_ATTACH.


Okay that explains it.  Since the injected thread is loading the DLL, it 
is the one which calls PROCESS_ATTACH, which in turn creates a proxy 
object for the injected thread instead of the main thread. 
Unfortunately, I don't know of a way to make this work automatically. 
Basically, you need to change the thread handle (Thread.hdl) to be the 
handle of the main thread before the injected thread exits.


 The thing is, using the "madChook" library to inject the dll instead, through
 its version of "CreateProcessEx" or its system wide "inject on process creation
 through a kernel driver" method works perfectly fine. AFAIK both his methods
 inject before the target program's entry point is called too. 


madChook likely uses a method that tricks the main thread into calling 
PROCESS_ATTACH.  It's been a while since I've used that lib however, so 
I don't know offhand how it's done.


 It seems like the presence of a CreateRemoteThread call alone is messing up the
 GC. Since the GC worked fine, using the madChook method, on that one user's PC
 after uninstalling everything that could possibly be injecting unrelated dlls
 with CreateRemoteThread. Yahoo messenger, Norton etc. 
 
 It might work better to do a seperate scan of GetCurrentThread() instead of
 having a resident non-D Main thread, but that leads to problems if you dont run
 a collection cycle before D functions returns to non-D-space.


Well, a proxy object for the main thread must exist for other reasons 
(for Thread.getThis to work, for example), so while that approach would 
likely fix garbage collection, the app could still segfault if an 
unknown thread (the main thread in this case) calls certain thread routines.

What I'm probably going to do for Tango is provide routines to register 
and de-register proxy objects for 'foreign' threads.  The injected 
thread, then, would have to de-register itself and register the main 
thread before exiting.  I imagine it must be possible to get the thread 
handle of the main process thread using debug routines?  Something 
similar could be done for Phobos as well.


 A simple solution for DLLs might be to manually initalize what the GC uses as
 the main thread somewhere along the line (if needed). And allow it to fall back
 on FullCollectNoStack() if theres no Main thread registered. I'm not sure if
 this will result in leaks or not though. 
 
 Also, the behavior on that one users PC makes it seem like it could be
 something unrelated to initialization that is breaking things too. 
 
 Seems like this is going to be messy to deal with haha.


I like the above approach best, assuming it's possible.  With it, 
scanning wouldn't have to change at all, but non-D threads that wanted 
to use GCed memory would be responsible for letting the D runtime know 
about them.


Sean
Jul 22 2007
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=1360






Heres a temporary fix that seems to solve the problem on that users PC:


Modify DllMain:
-------------------------------
case DLL_PROCESS_ATTACH:  
    DisableThreadLibraryCalls(instance);
    gc_init();
    std.gc.disable();
    _minit();
    _moduleCtor();
    return TRUE;
-------------------------------

Extend Thread class:
-------------------------------
static void setAsMainThread()
{
    if (!threadLock)
        threadLock = new Object();

    if (allThreadsDim == -1)
        allThreadsDim = 1;  

    Thread t = new Thread();

    t.state       = TS.RUNNING;
    t.id          = GetCurrentThreadId();
    t.hdl         = Thread.getCurrentThreadHandle();
    t.stackBottom = os_query_stackBottom();
    t.idx         = 0;
    allThreads[0] = t;
}
-------------------------------

Then from the programs main thread:

Thread.setAsMainThread();
std.gc.enable();


This issue probably affects people who LoadLibrary() a D DLL from a thread
(during regular execution, not just before the entry point is called) that
disappears sometime during the execution of the program. I think it might be
tricky to reliably figure out what the main thread of a process is. You could
probably enumerate threads and look if the entry point of the thread matches
the entry point of the exe. But i think usually the main thread is allowed to
exit without killing the whole program, as long as some threads are still
active. So theres still the issue of allThreads[0] possibly being invalid
right?


--
Jul 22 2007