www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Idea: Implement Concurrent GC on the windows Platform

reply Benjamin Thaut <code benjamin-thaut.de> writes:
Hi,

I recently came accross this article:
https://blogs.msdn.microsoft.com/vcblog/2018/09/26/step-back-going-back-in-c-time/
which mentions the following "new" kernel API of Windows.
https://docs.microsoft.com/de-de/windows/desktop/api/processsnapshot/nf-processsnapshot-psscapturesnapshot

It has the following properties (See "under the hood" chapter in 
the first link.)

<quote>
* Create a ‘snapshot’ which looks suspiciously like the child 
process of an existing process that has no threads running.
* Mark the processes memory, it’s page tables (Wikipedia), as 
copy-on-write (Wikipedia). That means that whenever a table is 
written to, the table is copied.
</quote>

Now this looks a lot like a fork. At least it does what you would 
need from a fork for implementing the concurrent GC. If I 
remember correctly the main problem of the concurrent GC on 
windows was that there is no equivalent of fork.

The only problem is that there are no threads running after the 
"fork" but for that CreateRemoteThread could be used.

Unfortunately I currently don't have the time to try this out 
myself. But in case someone has time and is interrested and this 
does work out I could see the concurrent GC become a viable 
replacement option for the current gc.

Main downside is, that this API has been added with Windows 8.1. 
But as windows 7 is going out of support starting in 2020 I think 
this is not that big of a downside.

Kind Regards
Benjamin Thaut
Jan 06
next sibling parent Andre Pany <andre s-e-a-p.de> writes:
On Sunday, 6 January 2019 at 18:32:58 UTC, Benjamin Thaut wrote:
 Hi,

 I recently came accross this article:
 https://blogs.msdn.microsoft.com/vcblog/2018/09/26/step-back-going-back-in-c-time/
 which mentions the following "new" kernel API of Windows.
 https://docs.microsoft.com/de-de/windows/desktop/api/processsnapshot/nf-processsnapshot-psscapturesnapshot

 [...]
That sounds really great, could you create an issue for this, therefore the idea do not get lost. That is definitely an idea for which a donation round make sense... Kind regards Andre
Jan 06
prev sibling next sibling parent Radu <void null.pt> writes:
On Sunday, 6 January 2019 at 18:32:58 UTC, Benjamin Thaut wrote:
 Hi,

 I recently came accross this article:
 https://blogs.msdn.microsoft.com/vcblog/2018/09/26/step-back-going-back-in-c-time/
 which mentions the following "new" kernel API of Windows.
 https://docs.microsoft.com/de-de/windows/desktop/api/processsnapshot/nf-processsnapshot-psscapturesnapshot

 [...]
Rainer did some experimenting [1] in the past, should be nice to see if that function can improve it. 1 - http://rainers.github.io/visuald/druntime/concurrentgc.html
Jan 06
prev sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
It does not look like this is a viable option.

I could not get a remote thread created. Code below for anyone else that 
wants to have a play. On my machine it'll crash with error 5 trying to 
create a thread.

---

import std.stdio;
import core.sys.windows.basetsd;
import core.sys.windows.windows;
import core.thread;

extern(Windows) {
     alias HPSS = HANDLE;

     enum PSS_CAPTURE_FLAGS {
         PSS_CAPTURE_NONE                                = 0x00000000,
         PSS_CAPTURE_VA_CLONE                            = 0x00000001,
         PSS_CAPTURE_RESERVED_00000002                   = 0x00000002,
         PSS_CAPTURE_HANDLES                             = 0x00000004,
         PSS_CAPTURE_HANDLE_NAME_INFORMATION             = 0x00000008,
         PSS_CAPTURE_HANDLE_BASIC_INFORMATION            = 0x00000010,
         PSS_CAPTURE_HANDLE_TYPE_SPECIFIC_INFORMATION    = 0x00000020,
         PSS_CAPTURE_HANDLE_TRACE                        = 0x00000040,
         PSS_CAPTURE_THREADS                             = 0x00000080,
         PSS_CAPTURE_THREAD_CONTEXT                      = 0x00000100,
         PSS_CAPTURE_THREAD_CONTEXT_EXTENDED             = 0x00000200,
         PSS_CAPTURE_RESERVED_00000400                   = 0x00000400,
         PSS_CAPTURE_VA_SPACE                            = 0x00000800,
         PSS_CAPTURE_VA_SPACE_SECTION_INFORMATION        = 0x00001000,
         PSS_CAPTURE_IPT_TRACE                           = 0x00002000,

         PSS_CREATE_BREAKAWAY_OPTIONAL                   = 0x04000000,
         PSS_CREATE_BREAKAWAY                            = 0x08000000,
         PSS_CREATE_FORCE_BREAKAWAY                      = 0x10000000,
         PSS_CREATE_USE_VM_ALLOCATIONS                   = 0x20000000,
         PSS_CREATE_MEASURE_PERFORMANCE                  = 0x40000000,
         PSS_CREATE_RELEASE_SECTION                      = 0x80000000
     }

     enum PSS_QUERY_INFORMATION_CLASS {
         PSS_QUERY_PROCESS_INFORMATION = 0,
         PSS_QUERY_VA_CLONE_INFORMATION = 1,
         PSS_QUERY_AUXILIARY_PAGES_INFORMATION = 2,
         PSS_QUERY_VA_SPACE_INFORMATION = 3,
         PSS_QUERY_HANDLE_INFORMATION = 4,
         PSS_QUERY_THREAD_INFORMATION = 5,
         PSS_QUERY_HANDLE_TRACE_INFORMATION = 6,
         PSS_QUERY_PERFORMANCE_COUNTERS = 7
     }

     struct PSS_VA_CLONE_INFORMATION {
         HANDLE VaCloneHandle;
     }

     DWORD GetProcessId(HANDLE);
     DWORD PssCaptureSnapshot(HANDLE, PSS_CAPTURE_FLAGS, DWORD, HPSS*);
     DWORD PssQuerySnapshot(HPSS, PSS_QUERY_INFORMATION_CLASS, void*, 
DWORD);
     DWORD PssFreeSnapshot(HANDLE, HPSS);
}

__gshared Thread someThread;

void main() {
     writeln("Hello from host");

     HPSS snapshotRef;
     DWORD status = PssCaptureSnapshot(GetCurrentProcess(),
         PSS_CAPTURE_FLAGS.PSS_CAPTURE_VA_CLONE,
         CONTEXT_ALL, &snapshotRef);
     writeln("status 1: ", status);
     assert(status == 0);

     readln;

     PSS_VA_CLONE_INFORMATION cloneInfo;
     status = PssQuerySnapshot(snapshotRef, 
PSS_QUERY_INFORMATION_CLASS.PSS_QUERY_VA_CLONE_INFORMATION, &cloneInfo, 
PSS_VA_CLONE_INFORMATION.sizeof);
     writeln("status 2: ", status);
     assert(status == 0);
     writeln(GetProcessId(cloneInfo.VaCloneHandle));
     readln;

     HANDLE remoteThreadRef = 
CreateRemoteThread(cloneInfo.VaCloneHandle, null, 0, &threadFuncInit, 
null, CREATE_SUSPENDED, null);
     writeln("status 3: ", GetLastError());
     assert(remoteThreadRef !is null);

     readln;
     status = PssFreeSnapshot(GetCurrentProcess(), snapshotRef);
     writeln("status 4: ", status);

}

void threadFunc() {
     writeln("Hi from thread!");
}

extern(Windows) {
     uint threadFuncInit(void*) {
         writeln("WHAT???");
         stdout.flush;
         someThread = new Thread(&threadFunc);
         someThread.start;
         return 0;
     }
}
Jan 13