www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - rdmd and popen

reply Max Samukha <samukha voliacable.com.removethis> writes:
rebuild is pretty unusable with dmd 2.023, so I've taken the liberty
to port a popen implementation to D for Windows and modify rdmd to use
that.

The handle returned by the popen st
Jan 17 2009
parent reply Max Samukha <samukha voliacable.com.removethis> writes:
On Sat, 17 Jan 2009 16:09:43 +0200, Max Samukha
<samukha voliacable.com.removethis> wrote:


rebuild is pretty unusable with dmd 2.023, so I've taken the liberty
to port a popen implementation to D for Windows and modify rdmd to use
that.

The handle returned by the popen st


..ill fails at the end of stream (maybe due to this issue
http://www.digitalmars.com/d/archives/c++/windows/32-bits/274.html),
so the exception is simply ignored for now.

The rdmd.d patch and popen are here
http://d-coding.com/download/rdmd.zip
Jan 17 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Max Samukha wrote:

 On Sat, 17 Jan 2009 16:09:43 +0200, Max Samukha
 <samukha voliacable.com.removethis> wrote:
 

 rebuild is pretty unusable with dmd 2.023, so I've taken the liberty
 to port a popen implementation to D for Windows and modify rdmd to use
 that.

 The handle returned by the popen st


 
 ..ill fails at the end of stream (maybe due to this issue
 http://www.digitalmars.com/d/archives/c++/windows/32-bits/274.html),
 so the exception is simply ignored for now.
 
 The rdmd.d patch and popen are here
 http://d-coding.com/download/rdmd.zip


That's great! Is it ok with you if I take the popen implementation and 
do the rdmd integration by hand? I made a couple of other changes to 
rdmd and I'm not sure how patch would cope with that. Also, please let 
me know if you allow me to change popen a bit to e.g make symbols more 
local and add documentation. Finally, please let me know if I can 
publish your implementation in Phobos and use it in other functions such 
as shell().

Thanks,

Andrei
Jan 17 2009
parent reply Max Samukha <samukha voliacable.com.removethis> writes:
On Sat, 17 Jan 2009 09:19:38 -0800, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:


Max Samukha wrote:

 On Sat, 17 Jan 2009 16:09:43 +0200, Max Samukha
 <samukha voliacable.com.removethis> wrote:
 

 rebuild is pretty unusable with dmd 2.023, so I've taken the liberty
 to port a popen implementation to D for Windows and modify rdmd to use
 that.

 The handle returned by the popen st


 
 ..ill fails at the end of stream (maybe due to this issue
 http://www.digitalmars.com/d/archives/c++/windows/32-bits/274.html),
 so the exception is simply ignored for now.
 
 The rdmd.d patch and popen are here
 http://d-coding.com/download/rdmd.zip


That's great! Is it ok with you if I take the popen implementation and 
do the rdmd integration by hand? I made a couple of other changes to 
rdmd and I'm not sure how patch would cope with that. Also, please let 
me know if you allow me to change popen a bit to e.g make symbols more 
local and add documentation. Finally, please let me know if I can 
publish your implementation in Phobos and use it in other functions such 
as shell().

Thanks,

Andrei


Yep. But there are numerous issues with this implementation

1. The bad descriptor error.
2. pclose is not implemented accoring to the standard. There is a way
to partially fix this by storing the process ID in the descriptor.
3. 0 is passed to the mode parameter of _open_osfhandle. It should
probably be _O_BINARY if the runtime cares at all.
4. This implementaion is based on a number of existing ones so the
owners of original versions may be bad enough to claim their rights.

So it's far from being a perfect popen :)
Jan 17 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Max Samukha wrote:

 On Sat, 17 Jan 2009 09:19:38 -0800, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:
 

 Max Samukha wrote:

 On Sat, 17 Jan 2009 16:09:43 +0200, Max Samukha
 <samukha voliacable.com.removethis> wrote:


 rebuild is pretty unusable with dmd 2.023, so I've taken the liberty
 to port a popen implementation to D for Windows and modify rdmd to use
 that.

 The handle returned by the popen st


 ..ill fails at the end of stream (maybe due to this issue
 http://www.digitalmars.com/d/archives/c++/windows/32-bits/274.html),
 so the exception is simply ignored for now.

 The rdmd.d patch and popen are here
 http://d-coding.com/download/rdmd.zip


 That's great! Is it ok with you if I take the popen implementation and 
 do the rdmd integration by hand? I made a couple of other changes to 
 rdmd and I'm not sure how patch would cope with that. Also, please let 
 me know if you allow me to change popen a bit to e.g make symbols more 
 local and add documentation. Finally, please let me know if I can 
 publish your implementation in Phobos and use it in other functions such 
 as shell().

 Thanks,

 Andrei


 
 Yep. But there are numerous issues with this implementation
 
 1. The bad descriptor error.
 2. pclose is not implemented accoring to the standard. There is a way
 to partially fix this by storing the process ID in the descriptor.
 3. 0 is passed to the mode parameter of _open_osfhandle. It should
 probably be _O_BINARY if the runtime cares at all.
 4. This implementaion is based on a number of existing ones so the
 owners of original versions may be bad enough to claim their rights.
 
 So it's far from being a perfect popen :)


If your implementation relies heavily on others I think there is no 
question that you need to ask them for permission and also credit them 
in your implementation (whether they ask for that or not). Until then I 
can't do much.

Andrei
Jan 17 2009
parent Max Samukha <samukha voliacable.com.removethis> writes:
On Sat, 17 Jan 2009 12:57:24 -0800, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:


Max Samukha wrote:

 On Sat, 17 Jan 2009 09:19:38 -0800, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:
 

 Max Samukha wrote:

 On Sat, 17 Jan 2009 16:09:43 +0200, Max Samukha
 <samukha voliacable.com.removethis> wrote:


 rebuild is pretty unusable with dmd 2.023, so I've taken the liberty
 to port a popen implementation to D for Windows and modify rdmd to use
 that.

 The handle returned by the popen st


 ..ill fails at the end of stream (maybe due to this issue
 http://www.digitalmars.com/d/archives/c++/windows/32-bits/274.html),
 so the exception is simply ignored for now.

 The rdmd.d patch and popen are here
 http://d-coding.com/download/rdmd.zip


 That's great! Is it ok with you if I take the popen implementation and 
 do the rdmd integration by hand? I made a couple of other changes to 
 rdmd and I'm not sure how patch would cope with that. Also, please let 
 me know if you allow me to change popen a bit to e.g make symbols more 
 local and add documentation. Finally, please let me know if I can 
 publish your implementation in Phobos and use it in other functions such 
 as shell().

 Thanks,

 Andrei


 
 Yep. But there are numerous issues with this implementation
 
 1. The bad descriptor error.
 2. pclose is not implemented accoring to the standard. There is a way
 to partially fix this by storing the process ID in the descriptor.
 3. 0 is passed to the mode parameter of _open_osfhandle. It should
 probably be _O_BINARY if the runtime cares at all.
 4. This implementaion is based on a number of existing ones so the
 owners of original versions may be bad enough to claim their rights.
 
 So it's far from being a perfect popen :)


If your implementation relies heavily on others I think there is no 
question that you need to ask them for permission and also credit them 
in your implementation (whether they ask for that or not). Until then I 
can't do much.

Andrei


I wouldn't say it relies on them heavily. It uses the same pretty
obvious concept: create a pipe and pass its handles to the child
process. I don't think it makes much sense to ask permission from
developers of MSVC, cygwin and a dozen more, who came up with similar
implementations (imperfect in their own ways). Either I credit them
all or nobody. I choose the latter for the sake of brevity and put the
code in public domain.

Meanwhile, I've updated the implementation to use array slices and a
different way of getting to the command interpreter. I tried to use
__tmpnum field of _iobuf to store the process handle and it seemed to
work but then, having no idea of the field's exact purpose, I simply
introduced a global vector that maps file descriptors to process
handles, meaning popen is now not thread-safe, which is ok.

_open_osfhandle doesn't accept the write end of the pipe, so the
function will always fail for "w" mode. As I don't have the sources
for digitalmars C runtime I cannot tell what's going on.

And it would be really nice if Walter implemented a popen in the C
runtime where it belongs. Thanks!

The updated version is here http://d-coding.com/download/rdmd.zip.
cpopen is for ANSI and UTF-16 versions with C linkage and dpopen
contains a wrapper taking UTF-8 strings.
Jan 19 2009