www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - New std.process revival

reply =?ISO-8859-1?Q?Alex_R=F8nne_Petersen?= <alex lycus.org> writes:
Hi,

I decided to take a stab at reviving the new std.process written by Lars 
T. Kyllingstad and Steven Schveighoffer.

The result is here: 
https://github.com/alexrp/phobos/tree/new-std-process-update

I decided to extract the work into new commits because rebasing the old 
branch in Lars's repo was way too cumbersome after so many months (and 
that branch also had a lot of merge commits). The code is obviously not 
written by me; all I did was a couple of build and test fixes.

It currently works on 32-bit and 64-bit Linux. It would be great if 
someone could take it for a spin on OS X, FreeBSD, and Windows to see 
how it fares there (I'm particularly worried that I may have broken the 
Windows build).

Lars or Steven, would either of you be willing to go through the review 
process with this module? I sent the druntime changes upstream a while 
back, so the Phobos changes are really all that remain in order to have 
it included.

-- 
Alex Rnne Petersen
alex lycus.org
http://lycus.org
Dec 06 2012
next sibling parent "Dejan Lekic" <dejan.lekic gmail.com> writes:
On Thursday, 6 December 2012 at 18:40:57 UTC, Alex Rønne Petersen 
wrote:

 Hi,

 I decided to take a stab at reviving the new std.process 
 written by Lars T. Kyllingstad and Steven Schveighoffer.

 The result is here: 
 https://github.com/alexrp/phobos/tree/new-std-process-update

 I decided to extract the work into new commits because rebasing 
 the old branch in Lars's repo was way too cumbersome after so 
 many months (and that branch also had a lot of merge commits). 
 The code is obviously not written by me; all I did was a couple 
 of build and test fixes.

 It currently works on 32-bit and 64-bit Linux. It would be 
 great if someone could take it for a spin on OS X, FreeBSD, and 
 Windows to see how it fares there (I'm particularly worried 
 that I may have broken the Windows build).

 Lars or Steven, would either of you be willing to go through 
 the review process with this module? I sent the druntime 
 changes upstream a while back, so the Phobos changes are really 
 all that remain in order to have it included.


Alex, you rock! You have no idea how happy this makes me. 
std.process made by Lars and Steven was already awesome. It would 
be great if you guys work on it together and make it as 
stable/robust as it should be.
Dec 06 2012
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2012-12-06 19:40, Alex Rnne Petersen wrote:

 Hi,

 I decided to take a stab at reviving the new std.process written by Lars
 T. Kyllingstad and Steven Schveighoffer.

 The result is here:
 https://github.com/alexrp/phobos/tree/new-std-process-update

 I decided to extract the work into new commits because rebasing the old
 branch in Lars's repo was way too cumbersome after so many months (and
 that branch also had a lot of merge commits). The code is obviously not
 written by me; all I did was a couple of build and test fixes.

 It currently works on 32-bit and 64-bit Linux. It would be great if
 someone could take it for a spin on OS X, FreeBSD, and Windows to see
 how it fares there (I'm particularly worried that I may have broken the
 Windows build).


On Mac OS X, move the declaration of "environ" out of the Environment 
class, it's used by two global functions.

Also, running the unit tests for Phobos, I get this:

http://pastebin.com/GrDrHMxi

I don't know if it has anything to do with the new std.process module or 
not. This is all for 32bit.

-- 
/Jacob Carlborg
Dec 06 2012
next sibling parent =?ISO-8859-1?Q?Alex_R=F8nne_Petersen?= <alex lycus.org> writes:
On 06-12-2012 23:25, Jacob Carlborg wrote:

 On 2012-12-06 19:40, Alex Rnne Petersen wrote:

 Hi,

 I decided to take a stab at reviving the new std.process written by Lars
 T. Kyllingstad and Steven Schveighoffer.

 The result is here:
 https://github.com/alexrp/phobos/tree/new-std-process-update

 I decided to extract the work into new commits because rebasing the old
 branch in Lars's repo was way too cumbersome after so many months (and
 that branch also had a lot of merge commits). The code is obviously not
 written by me; all I did was a couple of build and test fixes.

 It currently works on 32-bit and 64-bit Linux. It would be great if
 someone could take it for a spin on OS X, FreeBSD, and Windows to see
 how it fares there (I'm particularly worried that I may have broken the
 Windows build).


 On Mac OS X, move the declaration of "environ" out of the Environment
 class, it's used by two global functions.

 Also, running the unit tests for Phobos, I get this:

 http://pastebin.com/GrDrHMxi

 I don't know if it has anything to do with the new std.process module or
 not. This is all for 32bit.


Probably does. I'll look into it, thanks!

-- 
Alex Rnne Petersen
alex lycus.org
http://lycus.org
Dec 06 2012
prev sibling parent reply =?ISO-8859-1?Q?Alex_R=F8nne_Petersen?= <alex lycus.org> writes:
On 06-12-2012 23:25, Jacob Carlborg wrote:

 On 2012-12-06 19:40, Alex Rnne Petersen wrote:

 Hi,

 I decided to take a stab at reviving the new std.process written by Lars
 T. Kyllingstad and Steven Schveighoffer.

 The result is here:
 https://github.com/alexrp/phobos/tree/new-std-process-update

 I decided to extract the work into new commits because rebasing the old
 branch in Lars's repo was way too cumbersome after so many months (and
 that branch also had a lot of merge commits). The code is obviously not
 written by me; all I did was a couple of build and test fixes.

 It currently works on 32-bit and 64-bit Linux. It would be great if
 someone could take it for a spin on OS X, FreeBSD, and Windows to see
 how it fares there (I'm particularly worried that I may have broken the
 Windows build).


 On Mac OS X, move the declaration of "environ" out of the Environment
 class, it's used by two global functions.

 Also, running the unit tests for Phobos, I get this:

 http://pastebin.com/GrDrHMxi

 I don't know if it has anything to do with the new std.process module or
 not. This is all for 32bit.


Can you run it on 64-bit too? Does the output differ?

Can you try to adjust the code in std.stdio a bit so it prints the 
actual exit code pclose() returns?

-- 
Alex Rnne Petersen
alex lycus.org
http://lycus.org
Dec 10 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-12-10 17:59, Alex Rnne Petersen wrote:


 Can you run it on 64-bit too? Does the output differ?


Same output.


 Can you try to adjust the code in std.stdio a bit so it prints the
 actual exit code pclose() returns?


It prints "-1".

-- 
/Jacob Carlborg
Dec 10 2012
prev sibling next sibling parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Thursday, 6 December 2012 at 18:40:57 UTC, Alex Rønne Petersen 
wrote:

 It currently works on 32-bit and 64-bit Linux. It would be 
 great if someone could take it for a spin on OS X, FreeBSD, and 
 Windows to see how it fares there (I'm particularly worried 
 that I may have broken the Windows build).


How about submitting it as a WIP pull request, so the auto tester 
picks it up and does the testing for you?
Dec 06 2012
parent =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:
On 07-12-2012 00:30, Vladimir Panteleev wrote:

 On Thursday, 6 December 2012 at 18:40:57 UTC, Alex Rønne Petersen wrote:

 It currently works on 32-bit and 64-bit Linux. It would be great if
 someone could take it for a spin on OS X, FreeBSD, and Windows to see
 how it fares there (I'm particularly worried that I may have broken
 the Windows build).


 How about submitting it as a WIP pull request, so the auto tester picks
 it up and does the testing for you?


Good idea, I hadn't even thought of that.

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org
Dec 06 2012
prev sibling next sibling parent Johannes Pfau <nospam example.com> writes:
Am Thu, 06 Dec 2012 19:40:56 +0100
schrieb Alex R=C3=B8nne Petersen <alex lycus.org>:



 I decided to extract the work into new commits because rebasing the
 old branch in Lars's repo was way too cumbersome after so many months
 (and that branch also had a lot of merge commits). The code is
 obviously not written by me; all I did was a couple of build and test
 fixes.


BTW in case you didn't know: The author and committer can be different
in git, so you can always do this:

git commit --author=3D"Full Name <email address.com>"

This will set the author to "Full Name" and the committer to your local
account.
Dec 06 2012
prev sibling next sibling parent reply "Jakob Bornecrantz" <wallbraker gmail.com> writes:
On Thursday, 6 December 2012 at 18:40:57 UTC, Alex Rønne Petersen 
wrote:

 Hi,

 I decided to take a stab at reviving the new std.process 
 written by Lars T. Kyllingstad and Steven Schveighoffer.

 The result is here: 
 https://github.com/alexrp/phobos/tree/new-std-process-update


https://github.com/alexrp/phobos/blob/new-std-process-update/std/process.d#L417

You need to call toArgz and toStringz before the fork, since 
either of those can be in TLS. Test with:


string cmd = "echo";
string[] args1 = ["hello"];
string[] args2 = [];

void main()
{
   spawnProcess(cmd, args2);
   spawnProcess(cmd, args1);
   args ~= "World";
   spawnProcess(cmd, args2);
}


I'm not exactly sure what is up, but in one of my projects I had 
this problem. I solved it by just making all the string[]s into 
__gshared.

Cheers, Jakob.
Dec 07 2012
parent =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:
On 07-12-2012 14:20, Jakob Bornecrantz wrote:

 On Thursday, 6 December 2012 at 18:40:57 UTC, Alex Rønne Petersen wrote:

 Hi,

 I decided to take a stab at reviving the new std.process written by
 Lars T. Kyllingstad and Steven Schveighoffer.

 The result is here:
 https://github.com/alexrp/phobos/tree/new-std-process-update


 https://github.com/alexrp/phobos/blob/new-std-process-update/std/process.d#L417


 You need to call toArgz and toStringz before the fork, since either of
 those can be in TLS. Test with:


 string cmd = "echo";
 string[] args1 = ["hello"];
 string[] args2 = [];

 void main()
 {
    spawnProcess(cmd, args2);
    spawnProcess(cmd, args1);
    args ~= "World";
    spawnProcess(cmd, args2);
 }


 I'm not exactly sure what is up, but in one of my projects I had this
 problem. I solved it by just making all the string[]s into __gshared.

 Cheers, Jakob.


Thanks, fixed.

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org
Dec 07 2012
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 12/6/12, Alex R=F8nne Petersen <alex lycus.org> wrote:

 (I'm particularly worried that I may have broken the
 Windows build).


Does anyone know the URL of the original repository? Some symbols are
missing, but I don't think it's Alex'es fault.
Dec 09 2012
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 12/9/12, Andrej Mitrovic <andrej.mitrovich gmail.com> wrote:

 On 12/6/12, Alex R=F8nne Petersen <alex lycus.org> wrote:

 (I'm particularly worried that I may have broken the
 Windows build).


 Does anyone know the URL of the original repository? Some symbols are
 missing, but I don't think it's Alex'es fault.


Ok I think I found it: https://github.com/schveiguy/phobos/tree/new-std-pro=
cess
Dec 09 2012
prev sibling next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 12/9/12, Andrej Mitrovic <andrej.mitrovich gmail.com> wrote:

 On 12/6/12, Alex R=F8nne Petersen <alex lycus.org> wrote:

 (I'm particularly worried that I may have broken the
 Windows build).




std/c/stdio.d is missing _fdToHandle, see steven's repo and just copy
the file over to the new one.

I'm having a hard time figuring out how to "fork" your own fork, IOW I
have no idea how to make pull requests towards your own fork of
phobos. Your name doesn't show up in the base repo dropdown list when
I try to make a pull, otherwise I'd be of more help.
Dec 09 2012
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-12-09 19:34, Andrej Mitrovic wrote:


 I'm having a hard time figuring out how to "fork" your own fork, IOW I
 have no idea how to make pull requests towards your own fork of
 phobos. Your name doesn't show up in the base repo dropdown list when
 I try to make a pull, otherwise I'd be of more help.


Perhaps manually create a new repository at github. Initialize it with 
the old repository.

-- 
/Jacob Carlborg
Dec 09 2012
prev sibling parent Mike Wey <mike-wey example.com> writes:
On 12/09/2012 07:34 PM, Andrej Mitrovic wrote:

 I'm having a hard time figuring out how to "fork" your own fork, IOW I
 have no idea how to make pull requests towards your own fork of
 phobos. Your name doesn't show up in the base repo dropdown list when
 I try to make a pull, otherwise I'd be of more help.


Create a new branch, AFAIK you can create pull requests between 
different branches of the same repo.

-- 
Mike Wey
Dec 10 2012
prev sibling next sibling parent reply Denis Shelomovskij <verylonglogin.reg gmail.com> writes:
06.12.2012 22:40, Alex Rønne Petersen пишет:

 Hi,

 I decided to take a stab at reviving the new std.process written by Lars
 T. Kyllingstad and Steven Schveighoffer.

 The result is here:
 https://github.com/alexrp/phobos/tree/new-std-process-update

 I decided to extract the work into new commits because rebasing the old
 branch in Lars's repo was way too cumbersome after so many months (and
 that branch also had a lot of merge commits). The code is obviously not
 written by me; all I did was a couple of build and test fixes.

 It currently works on 32-bit and 64-bit Linux. It would be great if
 someone could take it for a spin on OS X, FreeBSD, and Windows to see
 how it fares there (I'm particularly worried that I may have broken the
 Windows build).

 Lars or Steven, would either of you be willing to go through the review
 process with this module? I sent the druntime changes upstream a while
 back, so the Phobos changes are really all that remain in order to have
 it included.


Please, confirm that such std.process implementation and its 
functionality (process and threads listing etc., all what you have in 
.Net once finished) is not needed:
http://forum.dlang.org/thread/k8v45g$15o6$1 digitalmars.com

-- 
Денис В. Шеломовский
Denis V. Shelomovskij
Dec 10 2012
parent reply =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:
On 10-12-2012 10:04, Denis Shelomovskij wrote:

 06.12.2012 22:40, Alex Rønne Petersen пишет:

 Hi,

 I decided to take a stab at reviving the new std.process written by Lars
 T. Kyllingstad and Steven Schveighoffer.

 The result is here:
 https://github.com/alexrp/phobos/tree/new-std-process-update

 I decided to extract the work into new commits because rebasing the old
 branch in Lars's repo was way too cumbersome after so many months (and
 that branch also had a lot of merge commits). The code is obviously not
 written by me; all I did was a couple of build and test fixes.

 It currently works on 32-bit and 64-bit Linux. It would be great if
 someone could take it for a spin on OS X, FreeBSD, and Windows to see
 how it fares there (I'm particularly worried that I may have broken the
 Windows build).

 Lars or Steven, would either of you be willing to go through the review
 process with this module? I sent the druntime changes upstream a while
 back, so the Phobos changes are really all that remain in order to have
 it included.


 Please, confirm that such std.process implementation and its
 functionality (process and threads listing etc., all what you have in
 .Net once finished) is not needed:
 http://forum.dlang.org/thread/k8v45g$15o6$1 digitalmars.com


I don't follow?

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org
Dec 10 2012
parent reply Denis Shelomovskij <verylonglogin.reg gmail.com> writes:
10.12.2012 20:58, Alex Rønne Petersen пишет:

 On 10-12-2012 10:04, Denis Shelomovskij wrote:

 06.12.2012 22:40, Alex Rønne Petersen пишет:

 Hi,

 I decided to take a stab at reviving the new std.process written by Lars
 T. Kyllingstad and Steven Schveighoffer.

 The result is here:
 https://github.com/alexrp/phobos/tree/new-std-process-update

 I decided to extract the work into new commits because rebasing the old
 branch in Lars's repo was way too cumbersome after so many months (and
 that branch also had a lot of merge commits). The code is obviously not
 written by me; all I did was a couple of build and test fixes.

 It currently works on 32-bit and 64-bit Linux. It would be great if
 someone could take it for a spin on OS X, FreeBSD, and Windows to see
 how it fares there (I'm particularly worried that I may have broken the
 Windows build).

 Lars or Steven, would either of you be willing to go through the review
 process with this module? I sent the druntime changes upstream a while
 back, so the Phobos changes are really all that remain in order to have
 it included.


 Please, confirm that such std.process implementation and its
 functionality (process and threads listing etc., all what you have in
 .Net once finished) is not needed:
 http://forum.dlang.org/thread/k8v45g$15o6$1 digitalmars.com


 I don't follow?


What is your question?
I already wrote all my reasons in "[RFC] Modules for processes 
manipulation" thread linked above. Nobody was interested in. So I want 
to be sure my variant is unneeded not just accidentally missed. If I'm 
the only person here who would like to see at least every option .Net 
Framework offer for process manipulation it's OK and I will stop asking 
about it.

-- 
Денис В. Шеломовский
Denis V. Shelomovskij
Dec 10 2012
parent =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:
On 10-12-2012 23:18, Denis Shelomovskij wrote:

 10.12.2012 20:58, Alex Rønne Petersen пишет:

 On 10-12-2012 10:04, Denis Shelomovskij wrote:

 06.12.2012 22:40, Alex Rønne Petersen пишет:

 Hi,

 I decided to take a stab at reviving the new std.process written by
 Lars
 T. Kyllingstad and Steven Schveighoffer.

 The result is here:
 https://github.com/alexrp/phobos/tree/new-std-process-update

 I decided to extract the work into new commits because rebasing the old
 branch in Lars's repo was way too cumbersome after so many months (and
 that branch also had a lot of merge commits). The code is obviously not
 written by me; all I did was a couple of build and test fixes.

 It currently works on 32-bit and 64-bit Linux. It would be great if
 someone could take it for a spin on OS X, FreeBSD, and Windows to see
 how it fares there (I'm particularly worried that I may have broken the
 Windows build).

 Lars or Steven, would either of you be willing to go through the review
 process with this module? I sent the druntime changes upstream a while
 back, so the Phobos changes are really all that remain in order to have
 it included.


 Please, confirm that such std.process implementation and its
 functionality (process and threads listing etc., all what you have in
 .Net once finished) is not needed:
 http://forum.dlang.org/thread/k8v45g$15o6$1 digitalmars.com


 I don't follow?


 What is your question?
 I already wrote all my reasons in "[RFC] Modules for processes
 manipulation" thread linked above. Nobody was interested in. So I want
 to be sure my variant is unneeded not just accidentally missed. If I'm
 the only person here who would like to see at least every option .Net
 Framework offer for process manipulation it's OK and I will stop asking
 about it.


No, I think there are a lot of people who want this. D is basically 
useless for scripting work because of its poor process manipulation support.

But keep in mind that your module is very Windows-centric and most 
people who deal with shell scripting work are on POSIX systems (a 
generalization, of course, but mostly true). I think that's why you 
didn't get much input.

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org
Dec 10 2012
prev sibling parent reply "Lars T. Kyllingstad" <public kyllingen.net> writes:
On Thursday, 6 December 2012 at 18:40:57 UTC, Alex Rønne Petersen
wrote:

 Hi,

 I decided to take a stab at reviving the new std.process 
 written by Lars T. Kyllingstad and Steven Schveighoffer.

 The result is here: 
 https://github.com/alexrp/phobos/tree/new-std-process-update

 I decided to extract the work into new commits because rebasing 
 the old branch in Lars's repo was way too cumbersome after so 
 many months (and that branch also had a lot of merge commits). 
 The code is obviously not written by me; all I did was a couple 
 of build and test fixes.

 It currently works on 32-bit and 64-bit Linux. It would be 
 great if someone could take it for a spin on OS X, FreeBSD, and 
 Windows to see how it fares there (I'm particularly worried 
 that I may have broken the Windows build).

 Lars or Steven, would either of you be willing to go through 
 the review process with this module? I sent the druntime 
 changes upstream a while back, so the Phobos changes are really 
 all that remain in order to have it included.


Great!  Steve and I never got around to doing this, and I haven't
had the time to do much Phobos development for the past year.  I
would be very happy to see this code finally make it into Phobos
-- it is long overdue!

Unfortunately, in the immediate future, I don't think I can
guarantee the degree of availability that is expected in a review
process.  After all, the reviewee(?) should be available for
questions and criticism, and for implementing the changes agreed
upon.  But perhaps Steve and I could do the review together, and
thus share the burden?  I haven't visited the forums in a while,
is Steve still around?

While I remember: std.process.environment was accepted into
Phobos a long time ago.  I'm pretty sure it has received some
updates in Phobos master since then, but I can't remember whether
I backported those to my repo.  You should probably compare them
and see.

Another thing:  Proper unittests for all functionality in this
module would be great.  If anyone has a good idea as to which
processes can be run in a unittest, both safely and with a
predictable outcome, on each platform, please speak up.

Lars
Dec 12 2012
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 12 Dec 2012 13:45:31 -0500, Lars T. Kyllingstad  =

<public kyllingen.net> wrote:


 On Thursday, 6 December 2012 at 18:40:57 UTC, Alex R=C3=B8nne Petersen=



 wrote:

 Hi,

 I decided to take a stab at reviving the new std.process written by  =





 Lars T. Kyllingstad and Steven Schveighoffer.

 The result is here:  =





 https://github.com/alexrp/phobos/tree/new-std-process-update

 I decided to extract the work into new commits because rebasing the o=




ld  =


 branch in Lars's repo was way too cumbersome after so many months (an=




d  =


 that branch also had a lot of merge commits). The code is obviously n=




ot  =


 written by me; all I did was a couple of build and test fixes.

 It currently works on 32-bit and 64-bit Linux. It would be great if  =





 someone could take it for a spin on OS X, FreeBSD, and Windows to see=




  =


 how it fares there (I'm particularly worried that I may have broken t=




he  =


 Windows build).

 Lars or Steven, would either of you be willing to go through the revi=




ew  =


 process with this module? I sent the druntime changes upstream a whil=




e  =


 back, so the Phobos changes are really all that remain in order to ha=




ve  =


 it included.


 Great!  Steve and I never got around to doing this, and I haven't
 had the time to do much Phobos development for the past year.  I
 would be very happy to see this code finally make it into Phobos
 -- it is long overdue!

 Unfortunately, in the immediate future, I don't think I can
 guarantee the degree of availability that is expected in a review
 process.  After all, the reviewee(?) should be available for
 questions and criticism, and for implementing the changes agreed
 upon.  But perhaps Steve and I could do the review together, and
 thus share the burden?  I haven't visited the forums in a while,
 is Steve still around?

 While I remember: std.process.environment was accepted into
 Phobos a long time ago.  I'm pretty sure it has received some
 updates in Phobos master since then, but I can't remember whether
 I backported those to my repo.  You should probably compare them
 and see.

 Another thing:  Proper unittests for all functionality in this
 module would be great.  If anyone has a good idea as to which
 processes can be run in a unittest, both safely and with a
 predictable outcome, on each platform, please speak up.


Hi Alex,

I too have been very uninvolved with D for the past few months (not by  =

choice).  I'm glad someone is picking this up again, and I will try to  =

offer as much help as I can.  I unfortunately have been extremely busy  =

with iOS development and objective C.  I'd love to get back into working=
  =

on D, but I just can't right now.

I have not been keeping up with the forums or with the mailing lists, La=
rs  =

emailed me about your efforts.  The only thing I recall about the Window=
s  =

stuff that was not complete was that I was not setting some of the pipe =
 =

handles to close when the new process is created (someone had pointed th=
at  =

out, I can't remember who).

If I get some time, I will test your branch on my Windows system and  =

review the code.

-Steve
Dec 14 2012