www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - D1 -> D2

reply Fawzi Mohamed <fawzi gmx.ch> writes:
Is there any "porting" guide around in a wiki?
If not a page where to share the best tricks would be nice "D1->D2  
conversion tricks"?

In the short term I don't think that going D2 only is really an option  
for me, so how feasible it is to keep the code base compatible to both  
D1 and D2?

I know that one can define some templates (for example Const(T),....),  
and maybe use mixins, but how much uglier does the code become as  
result?
I choose D to have cleaner code, I am not interested in loosing all  
that just to be D1 and D2, then I prefer to wait, and convert  
everything at once.

Well that is about it...

thanks

Fawzi
Nov 18 2010
next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On 11/18/2010 7:51 PM, Fawzi Mohamed wrote:

 Is there any "porting" guide around in a wiki?
 If not a page where to share the best tricks would be nice "D1->D2
 conversion tricks"?

 In the short term I don't think that going D2 only is really an option
 for me, so how feasible it is to keep the code base compatible to both
 D1 and D2?

 I know that one can define some templates (for example Const(T),....),
 and maybe use mixins, but how much uglier does the code become as result?
 I choose D to have cleaner code, I am not interested in loosing all that
 just to be D1 and D2, then I prefer to wait, and convert everything at
 once.

 Well that is about it...

 thanks

 Fawzi


In maintaining Derelict, which is nothing more than a simple collection 
of bindings to C libraries, I have had headaches keeping compatibility 
between D1/D2. It's nothing that has been difficult to solve, just ugly.

If something as simple as a C-binding is uglified, I cringe at the 
thought of maintaining something more complex. It's going to get very 
ugly, very quickly. My attitude is that any future D projects I make 
available will be D2 only. I just don't think it's worth being 
compatible with both versions from a code maintenance perspective.
Nov 18 2010
next sibling parent Fawzi Mohamed <fawzi gmx.ch> writes:
On 18-nov-10, at 12:07, Mike Parker wrote:


 On 11/18/2010 7:51 PM, Fawzi Mohamed wrote:

 Is there any "porting" guide around in a wiki?
 If not a page where to share the best tricks would be nice "D1->D2
 conversion tricks"?

 In the short term I don't think that going D2 only is really an  
 option
 for me, so how feasible it is to keep the code base compatible to  
 both
 D1 and D2?

 I know that one can define some templates (for example  
 Const(T),....),
 and maybe use mixins, but how much uglier does the code become as  
 result?
 I choose D to have cleaner code, I am not interested in loosing all  
 that
 just to be D1 and D2, then I prefer to wait, and convert everything  
 at
 once.

 Well that is about it...

 thanks

 Fawzi


 In maintaining Derelict, which is nothing more than a simple  
 collection of bindings to C libraries, I have had headaches keeping  
 compatibility between D1/D2. It's nothing that has been difficult to  
 solve, just ugly.

 If something as simple as a C-binding is uglified, I cringe at the  
 thought of maintaining something more complex. It's going to get  
 very ugly, very quickly. My attitude is that any future D projects I  
 make available will be D2 only. I just don't think it's worth being  
 compatible with both versions from a code maintenance perspective.


that was my feeling too, but I wanted some confirmation from people  
having actually done it.
This just reinforces my choice of being D1 only for the moment...
Nov 18 2010
prev sibling next sibling parent =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Mike Parker wrote:

 On 11/18/2010 7:51 PM, Fawzi Mohamed wrote:



 In the short term I don't think that going D2 only is really an option
 for me, so how feasible it is to keep the code base compatible to both
 D1 and D2?

 I know that one can define some templates (for example Const(T),....),
 and maybe use mixins, but how much uglier does the code become as result?
 I choose D to have cleaner code, I am not interested in loosing all that
 just to be D1 and D2, then I prefer to wait, and convert everything at
 once.





 In maintaining Derelict, which is nothing more than a simple collection
 of bindings to C libraries, I have had headaches keeping compatibility
 between D1/D2. It's nothing that has been difficult to solve, just ugly.

 If something as simple as a C-binding is uglified, I cringe at the
 thought of maintaining something more complex. It's going to get very
 ugly, very quickly. My attitude is that any future D projects I make
 available will be D2 only. I just don't think it's worth being
 compatible with both versions from a code maintenance perspective.


wxD had the same experience, with supporting Phobos/Tango and D1/D2.
It's "possible", but leads to ugly/duplicated code and best avoided...

For a C/C++-binding it's doable with liberal amounts of version(Tango)
and version(D_Version2) but that quickly gets hideous and in the way.

--anders

PS. Adding three compilers and three build systems, it just exploded.
     It made even the two OS and the three UI look simple in comparison.

     OS: Windows, Unix (linux, darwin)
     UI: __WXMSW__, __WXGTK__, __WXMAC__
Nov 18 2010
prev sibling parent spir <denis.spir gmail.com> writes:
On Thu, 18 Nov 2010 20:07:38 +0900
Mike Parker <aldacron gmail.com> wrote:


 On 11/18/2010 7:51 PM, Fawzi Mohamed wrote:

 Is there any "porting" guide around in a wiki?
 If not a page where to share the best tricks would be nice "D1->D2
 conversion tricks"?

 In the short term I don't think that going D2 only is really an option
 for me, so how feasible it is to keep the code base compatible to both
 D1 and D2?

 I know that one can define some templates (for example Const(T),....),
 and maybe use mixins, but how much uglier does the code become as resul=




t?

 I choose D to have cleaner code, I am not interested in loosing all that
 just to be D1 and D2, then I prefer to wait, and convert everything at
 once.

 Well that is about it...

 thanks

 Fawzi


=20
 In maintaining Derelict, which is nothing more than a simple collection=20
 of bindings to C libraries, I have had headaches keeping compatibility=20
 between D1/D2. It's nothing that has been difficult to solve, just ugly.
=20
 If something as simple as a C-binding is uglified, I cringe at the=20
 thought of maintaining something more complex. It's going to get very=20
 ugly, very quickly. My attitude is that any future D projects I make=20
 available will be D2 only. I just don't think it's worth being=20
 compatible with both versions from a code maintenance perspective.


I have read your blog post on the topic. This let me think that all true ne=
eds for string mixins, like yours, are cases revealing plain language issue=
s.


denis
-- -- -- -- -- -- --
vit esse estrany =E2=98=A3

spir.wikidot.com
Nov 18 2010
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 18 Nov 2010 05:51:32 -0500, Fawzi Mohamed <fawzi gmx.ch> wrote:


 Is there any "porting" guide around in a wiki?
 If not a page where to share the best tricks would be nice "D1->D2  
 conversion tricks"?

 In the short term I don't think that going D2 only is really an option  
 for me, so how feasible it is to keep the code base compatible to both  
 D1 and D2?

 I know that one can define some templates (for example Const(T),....),  
 and maybe use mixins, but how much uglier does the code become as result?
 I choose D to have cleaner code, I am not interested in loosing all that  
 just to be D1 and D2, then I prefer to wait, and convert everything at  
 once.


I have had two experiences moving code from D1 to D2.  First, I tried to  
port Tango to D2, and found that backwards compatibility is not possible.   
So I was simply reorganizing the files.  The hugest problem for going from  
D1 to D2 is string literals.  Every place you use char[] has to be  
switched to const(char)[] or string.  I came up with a set of rules for  
this:

1. if the 99% common case for string arguments will be string literals,  
then use string -- you never have to worry about duping.
2. if you intend to keep a copy of the string, but you aren't sure whether  
the string to be passed in will have mutable aliases, use const(char)[]  
and dup.
3. if you don't intend to keep a copy of the string, use const(char)[] to  
have the most flexibility.
4. if you intend on returning a portion of the string, well, currently  
you're SOL, but eventually inout will come into play here.

For example, Tango's log package fell under rules 1 and 3 (most of the  
time you are creating log objects with string literals, and logging a  
message does not save the message anywhere).
Many many cases fell under rule 4, which is one of the main reasons I gave  
up (inout still isn't ready).

The second experience I had was porting dcollections to D2.  Since  
dcollections has not many places for string literals, it was much easier  
to port.  I don't think I made it backwards compatible, but it's very very  
similar to the original in terms of constancy.  I've added a few features  
that only really make sense in D2 (ranges for one).

Porting dcollections was quite easy, but with my experience trying to port  
Tango, I never tried to make it backwards compatible.

My recommendation -- when you are ready, switch wholly to D2.  Don't  
bother with compatibility, it's just not possible.

-Steve
Nov 18 2010
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Steven Schveighoffer wrote:

 My recommendation -- when you are ready, switch wholly to D2.  Don't 
 bother with compatibility, it's just not possible.


 From what you wrote, it appears that most of the difficulties were in dealing 
with strings. Might I suggest:

1. Replace all occurrences of char[] with string.

2. Compile to find every place that mutable strings are used.

3. Refactor the code to clearly encapsulate where mutable strings are created 
and manipulated, and as the last step, cast them to string.
Nov 18 2010
next sibling parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Thu, 18 Nov 2010 22:19:12 +0300, Walter Bright  
<newshound2 digitalmars.com> wrote:


 Steven Schveighoffer wrote:

 My recommendation -- when you are ready, switch wholly to D2.  Don't  
 bother with compatibility, it's just not possible.


  From what you wrote, it appears that most of the difficulties were in  
 dealing with strings.


No, it's not. From my experience, a bigger issue is that

version (D2) {
    /* some D2 only code involving const etc */
}

simply won't compile in D1, because even if it's D2-only the code needs to  
be correct D1 code as well.
Nov 18 2010
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Denis Koroskin wrote:

  From my experience, a bigger issue is that
 
 version (D2) {
    /* some D2 only code involving const etc */
 }
 
 simply won't compile in D1, because even if it's D2-only the code needs 
 to be correct D1 code as well.


I agree that's an issue. Currently, the only way to deal with this is one of:

1. Minimize the differences, and maintain two copies of the source code. Using 
the (rather fabulous) meld tool (available on Linux), the merging is pretty 
easy. I use meld all the time to, for example, merge differences in the code 
base between dmd1 and dmd2.

2. Isolate the code that is different into different files, which minimizes the 
work involved in (1).

3. Use string mixins and the token string literal form.


I've also found that using the string alias rather than char[] does account for 
an awful lot of the differences.
Nov 18 2010
next sibling parent Rainer Deyke <rainerd eldwood.com> writes:
On 11/18/2010 12:39, Walter Bright wrote:

 I agree that's an issue. Currently, the only way to deal with this is
 one of:
 
 1. Minimize the differences, and maintain two copies of the source code.
 Using the (rather fabulous) meld tool (available on Linux), the merging
 is pretty easy. I use meld all the time to, for example, merge
 differences in the code base between dmd1 and dmd2.
 
 2. Isolate the code that is different into different files, which
 minimizes the work involved in (1).
 
 3. Use string mixins and the token string literal form.


4. Use an external preprocessor to generate the D1 and D2 code from the
same source file.


-- 
Rainer Deyke - rainerd eldwood.com
Nov 18 2010
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
Walter Bright wrote:

 I agree that's an issue. Currently, the only way to deal with this is 
 one of:



There's a 4th method I almost hesitate to mention, but you can use a common 
source code file, and a macro text preprocessor (such as m4) to generate the D1 
and D2 versions from it.
Nov 18 2010
prev sibling parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Thu, 18 Nov 2010 22:39:31 +0300, Walter Bright  
<newshound2 digitalmars.com> wrote:


 Denis Koroskin wrote:

  From my experience, a bigger issue is that
  version (D2) {
    /* some D2 only code involving const etc */
 }
  simply won't compile in D1, because even if it's D2-only the code  
 needs to be correct D1 code as well.


 I agree that's an issue. Currently, the only way to deal with this is  
 one of:

 1. Minimize the differences, and maintain two copies of the source code.  
 Using the (rather fabulous) meld tool (available on Linux), the merging  
 is pretty easy. I use meld all the time to, for example, merge  
 differences in the code base between dmd1 and dmd2.

 2. Isolate the code that is different into different files, which  
 minimizes the work involved in (1).

 3. Use string mixins and the token string literal form.


 I've also found that using the string alias rather than char[] does  
 account for an awful lot of the differences.


All these are ugly.
Nov 18 2010
parent Walter Bright <newshound2 digitalmars.com> writes:
Denis Koroskin wrote:

 All these are ugly.


I agree.
Nov 18 2010
prev sibling parent reply %u <e ee.com> writes:
== Quote from Denis Koroskin (2korden gmail.com)'s article

 On Thu, 18 Nov 2010 22:19:12 +0300, Walter Bright
 <newshound2 digitalmars.com> wrote:

 Steven Schveighoffer wrote:

 My recommendation -- when you are ready, switch wholly to D2.  Don't
 bother with compatibility, it's just not possible.


  From what you wrote, it appears that most of the difficulties were in
 dealing with strings.


 No, it's not. From my experience, a bigger issue is that
 version (D2) {
     /* some D2 only code involving const etc */
 }
 simply won't compile in D1, because even if it's D2-only the code needs to
 be correct D1 code as well.


Wow, that sucks!!
Can't this simply be mended with special D_DMD1 and D_DMD2 (D_DMD3 :) version
identifiers?
If the compiler version isn't the same then it won't look at the code.
Nov 18 2010
parent reply Bernard Helyer <b.helyer gmail.com> writes:
On Thu, 18 Nov 2010 19:58:42 +0000, %u wrote:


 == Quote from Denis Koroskin (2korden gmail.com)'s article

 On Thu, 18 Nov 2010 22:19:12 +0300, Walter Bright
 <newshound2 digitalmars.com> wrote:

 Steven Schveighoffer wrote:

 My recommendation -- when you are ready, switch wholly to D2.  Don't
 bother with compatibility, it's just not possible.


  From what you wrote, it appears that most of the difficulties were
  in
 dealing with strings.


 No, it's not. From my experience, a bigger issue is that version (D2) {
     /* some D2 only code involving const etc */
 }
 simply won't compile in D1, because even if it's D2-only the code needs
 to be correct D1 code as well.


 
 Wow, that sucks!!
 Can't this simply be mended with special D_DMD1 and D_DMD2 (D_DMD3 :)
 version identifiers?
 If the compiler version isn't the same then it won't look at the code.


No. The contents of a version block is still parsed. If the D2 code 
doesn't _parse_ as valid D1, then the D1 compiler will reject it, even if 
the version block will not be processed further than that.
Nov 19 2010
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Bernard Helyer wrote:

 No. The contents of a version block is still parsed. If the D2 code 
 doesn't _parse_ as valid D1, then the D1 compiler will reject it, even if 
 the version block will not be processed further than that.


Yes. Now, if the D1 compiler is supposed to successfully lex & parse (and then 
ignore) false version blocks, it becomes impossible for a D1 compiler to be 
stable. It will have to track every future D language variant, missteps and all.
Nov 19 2010
next sibling parent reply Andrew Wiley <debio264 gmail.com> writes:
On Fri, Nov 19, 2010 at 6:18 PM, Walter Bright
<newshound2 digitalmars.com>wrote:


 Bernard Helyer wrote:


 No. The contents of a version block is still parsed. If the D2 code
 doesn't _parse_ as valid D1, then the D1 compiler will reject it, even if
 the version block will not be processed further than that.


 Yes. Now, if the D1 compiler is supposed to successfully lex & parse (and
 then ignore) false version blocks, it becomes impossible for a D1 compiler
 to be stable. It will have to track every future D language variant,
 missteps and all.


I'm a fan of parsing false version blocks, but it seems like there should be
some sort of exception for version blocks for a different version of the
language. Could they just be skipped with a warning that an unknown version
of the language is in use? It's somewhat hackish, but to me, that seems
better than parsing it and giving an error because of syntax changes.
Nov 19 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
Andrew Wiley wrote:

 I'm a fan of parsing false version blocks, but it seems like there 
 should be some sort of exception for version blocks for a different 
 version of the language. Could they just be skipped with a warning that 
 an unknown version of the language is in use? It's somewhat hackish, but 
 to me, that seems better than parsing it and giving an error because of 
 syntax changes. 



How can it skip it without knowing the grammar?
Nov 19 2010
next sibling parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Sat, 20 Nov 2010 04:14:05 +0300, Walter Bright  
<newshound2 digitalmars.com> wrote:


 Andrew Wiley wrote:

 I'm a fan of parsing false version blocks, but it seems like there  
 should be some sort of exception for version blocks for a different  
 version of the language. Could they just be skipped with a warning that  
 an unknown version of the language is in use? It's somewhat hackish,  
 but to me, that seems better than parsing it and giving an error  
 because of syntax changes.



 How can it skip it without knowing the grammar?


#ifdef D_Version2
...
#else
...
#endif


D has a precedence of having preprocessor macros (e.g #line) even if it's  
not a full-fledged preprocessor, and I don't see why it couldn't adapt the  
same solution.
Nov 19 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
Denis Koroskin wrote:

 On Sat, 20 Nov 2010 04:14:05 +0300, Walter Bright 

 How can it skip it without knowing the grammar?


 
 #ifdef D_Version2
 ...
 #else
 ...
 #endif
 
 
 D has a precedence of having preprocessor macros (e.g #line) even if 
 it's not a full-fledged preprocessor, and I don't see why it couldn't 
 adapt the same solution.
 



Yes, I know it can be done with a text preprocessor.

[[ Queue run away screaming ]]
Nov 19 2010
parent "Denis Koroskin" <2korden gmail.com> writes:
On Sat, 20 Nov 2010 04:33:36 +0300, Walter Bright  
<newshound2 digitalmars.com> wrote:


 Denis Koroskin wrote:

 On Sat, 20 Nov 2010 04:14:05 +0300, Walter Bright

 How can it skip it without knowing the grammar?


  #ifdef D_Version2
 ...
 #else
 ...
 #endif
   D has a precedence of having preprocessor macros (e.g #line) even if  
 it's not a full-fledged preprocessor, and I don't see why it couldn't  
 adapt the same solution.



 Yes, I know it can be done with a text preprocessor.

 [[ Queue run away screaming ]]


Once again, it's NOT a text preprocessor. It's NOT a separate parsing  
phase (unlike C/C++). It's done at parsing level, similarly to comments.
Nov 19 2010
prev sibling parent reply Andrew Wiley <debio264 gmail.com> writes:
On Fri, Nov 19, 2010 at 7:14 PM, Walter Bright
<newshound2 digitalmars.com>wrote:


 Andrew Wiley wrote:


 I'm a fan of parsing false version blocks, but it seems like there should
 be some sort of exception for version blocks for a different version of the
 language. Could they just be skipped with a warning that an unknown version
 of the language is in use? It's somewhat hackish, but to me, that seems
 better than parsing it and giving an error because of syntax changes.



 How can it skip it without knowing the grammar?


It's probably too much work for too little gain, but would it be possible to
basically just scan for matching braces until the version block ends? If
they always come in matching pairs (or at least they *should*), you could
scan for them and ignore everything else until you find the end of the
block.
Nov 19 2010
parent Walter Bright <newshound2 digitalmars.com> writes:
Andrew Wiley wrote:

 On Fri, Nov 19, 2010 at 7:14 PM, Walter Bright 
 <newshound2 digitalmars.com <mailto:newshound2 digitalmars.com>> wrote:
 
     Andrew Wiley wrote:
 
         I'm a fan of parsing false version blocks, but it seems like
         there should be some sort of exception for version blocks for a
         different version of the language. Could they just be skipped
         with a warning that an unknown version of the language is in
         use? It's somewhat hackish, but to me, that seems better than
         parsing it and giving an error because of syntax changes.
 
 
 
     How can it skip it without knowing the grammar?
 
 
 It's probably too much work for too little gain, but would it be 
 possible to basically just scan for matching braces until the version 
 block ends? If they always come in matching pairs (or at least they 
 *should*), you could scan for them and ignore everything else until you 
 find the end of the block.


It would still have to know about all the new tokens.
Nov 19 2010
prev sibling parent reply torhu <no spam.invalid> writes:
On 20.11.2010 01:18, Walter Bright wrote:

 Bernard Helyer wrote:

  No. The contents of a version block is still parsed. If the D2 code
  doesn't _parse_ as valid D1, then the D1 compiler will reject it, even if
  the version block will not be processed further than that.


 Yes. Now, if the D1 compiler is supposed to successfully lex&  parse (and then
 ignore) false version blocks, it becomes impossible for a D1 compiler to be
 stable. It will have to track every future D language variant, missteps and
all.


But current D 1 compilers are useless for most people anyway, until 
Tango catches up.  And in any case, if you upgrade, you often get some 
nice new bugs in addition to the fixes, so upgrading is often not worth 
the hassle.  But making it easier to go from D 1 to D 2 would still be a 
big gain, so why not do that?  No one cares about "stability" of D 1 
compilers.  If you want "stability", it's very simple.  You just don't 
upgrade.

Tango is dying, and D 1 with it.  We don't want "stability", we want a 
viable way to move to D 2.
Nov 19 2010
parent Walter Bright <newshound2 digitalmars.com> writes:
torhu wrote:

 But current D 1 compilers are useless for most people anyway, until 
 Tango catches up.  And in any case, if you upgrade, you often get some 
 nice new bugs in addition to the fixes, so upgrading is often not worth 
 the hassle.  But making it easier to go from D 1 to D 2 would still be a 
 big gain, so why not do that?  No one cares about "stability" of D 1 
 compilers.  If you want "stability", it's very simple.  You just don't 
 upgrade.
 
 Tango is dying, and D 1 with it.  We don't want "stability", we want a 
 viable way to move to D 2.


I understand. You can use a text preprocessor right now to do it, m4. It's 
quick, easy, and works.
Nov 19 2010
prev sibling parent %u <e ee.com> writes:
== Quote from Bernard Helyer (b.helyer gmail.com)'s article

 On Thu, 18 Nov 2010 19:58:42 +0000, %u wrote:

 == Quote from Denis Koroskin (2korden gmail.com)'s article

 On Thu, 18 Nov 2010 22:19:12 +0300, Walter Bright
 <newshound2 digitalmars.com> wrote:

 Steven Schveighoffer wrote:

 My recommendation -- when you are ready, switch wholly to D2.  Don't
 bother with compatibility, it's just not possible.


  From what you wrote, it appears that most of the difficulties were
  in
 dealing with strings.


 No, it's not. From my experience, a bigger issue is that version (D2) {
     /* some D2 only code involving const etc */
 }
 simply won't compile in D1, because even if it's D2-only the code needs
 to be correct D1 code as well.


 Wow, that sucks!!
 Can't this simply be mended with special D_DMD1 and D_DMD2 (D_DMD3 :)
 version identifiers?
 If the compiler version isn't the same then it won't look at the code.


 No. The contents of a version block is still parsed. If the D2 code
 doesn't _parse_ as valid D1, then the D1 compiler will reject it, even if
 the version block will not be processed further than that.


Well, actually I said the compiler wouldn't look at it, so it wouldn't get
parsed
as well.
What you should have said is that is very difficult to not parse a piece of code
and still know its boundaries ;)
So how can the compiler be sure a version block ends without actually parsing
the
text as code.
A single line version would do :)
Nov 19 2010
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 18 Nov 2010 14:19:12 -0500, Walter Bright  
<newshound2 digitalmars.com> wrote:


 Steven Schveighoffer wrote:

 My recommendation -- when you are ready, switch wholly to D2.  Don't  
 bother with compatibility, it's just not possible.


  From what you wrote, it appears that most of the difficulties were in  
 dealing with strings. Might I suggest:

 1. Replace all occurrences of char[] with string.

 2. Compile to find every place that mutable strings are used.

 3. Refactor the code to clearly encapsulate where mutable strings are  
 created and manipulated, and as the last step, cast them to string.


But string is not always what is desired.  It depends on the library.   
Tango uses mutable arrays everywhere because it prefers not to use the  
heap as much as possible.  For example, something might take a char[]  
buffer, and a function may pass in a stack-allocated array for the  
buffer.  Changing this to string is useless.

-Steve
Nov 18 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
Steven Schveighoffer wrote:

 On Thu, 18 Nov 2010 14:19:12 -0500, Walter Bright 
 <newshound2 digitalmars.com> wrote:
 

 Steven Schveighoffer wrote:

 My recommendation -- when you are ready, switch wholly to D2.  Don't 
 bother with compatibility, it's just not possible.


  From what you wrote, it appears that most of the difficulties were in 
 dealing with strings. Might I suggest:

 1. Replace all occurrences of char[] with string.

 2. Compile to find every place that mutable strings are used.

 3. Refactor the code to clearly encapsulate where mutable strings are 
 created and manipulated, and as the last step, cast them to string.


 
 But string is not always what is desired.  It depends on the library.


Right.



 Tango uses mutable arrays everywhere because it prefers not to use the 
 heap as much as possible.  For example, something might take a char[] 
 buffer, and a function may pass in a stack-allocated array for the 
 buffer.  Changing this to string is useless.


That's where refactoring comes in. I haven't looked at the Tango code, but 
there's a big difference between "using mutable arrays everywhere" and actually 
needing to mutate them. Only the latter needs to be char[], and the boundaries 
between the two are bridged with a .idup or a cast(string).

If Tango is relying on string literals to be char[], i.e. be mutable, this is a 
serious bug and will cause seg faults. So, this issue must have already been 
dealt with in Tango, and codifying string literal types as "string" rather than 
"char[]" should not negatively impact it.

And lastly, if there are APIs in tango that accept char[] and leave the caller 
perplexed about whether the API will mutate the string or not, that is
something 
that using string for the non-mutating ones nicely resolves.
Nov 18 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 18 Nov 2010 14:50:06 -0500, Walter Bright  
<newshound2 digitalmars.com> wrote:


 Steven Schveighoffer wrote:

 On Thu, 18 Nov 2010 14:19:12 -0500, Walter Bright  
 <newshound2 digitalmars.com> wrote:


 Steven Schveighoffer wrote:

 My recommendation -- when you are ready, switch wholly to D2.  Don't  
 bother with compatibility, it's just not possible.


  From what you wrote, it appears that most of the difficulties were in  
 dealing with strings. Might I suggest:

 1. Replace all occurrences of char[] with string.

 2. Compile to find every place that mutable strings are used.

 3. Refactor the code to clearly encapsulate where mutable strings are  
 created and manipulated, and as the last step, cast them to string.


  But string is not always what is desired.  It depends on the library.


 Right.



 Tango uses mutable arrays everywhere because it prefers not to use the  
 heap as much as possible.  For example, something might take a char[]  
 buffer, and a function may pass in a stack-allocated array for the  
 buffer.  Changing this to string is useless.


 That's where refactoring comes in. I haven't looked at the Tango code,  
 but there's a big difference between "using mutable arrays everywhere"  
 and actually needing to mutate them. Only the latter needs to be char[],  
 and the boundaries between the two are bridged with a .idup or a  
 cast(string).

 If Tango is relying on string literals to be char[], i.e. be mutable,  
 this is a serious bug and will cause seg faults. So, this issue must  
 have already been dealt with in Tango, and codifying string literal  
 types as "string" rather than "char[]" should not negatively impact it.

 And lastly, if there are APIs in tango that accept char[] and leave the  
 caller perplexed about whether the API will mutate the string or not,  
 that is something that using string for the non-mutating ones nicely  
 resolves.


Yes, Tango obviously works on Linux, so it does not modify string literals  
(as much as it can, you can't really prevent a user from passing in a  
string literal to sort for example).  However, the char[] type accepts  
both string literals and mutable buffers.  In some cases, those strings  
are stored for later use, in some they are modified (obviously one has to  
avoid passing in literals for that), and in some they are just used  
temporarily.  The differences require you to change the argument type to  
string or const(char)[], and possibly to overload the function.  The rules  
I outlined were what I was going by in order to port D1 code.


peppering idup everywhere, you have created a conflict with the main  
goal.  Refactoring in these cases isn't quite as simple as you say, and it  
certainly won't be backwards compatible.

In general, I tried to make Tango for D2 as const aware as possible (it's  
a standard lib, and shouldn't require people to ignore const), and found  
all sorts of issues with it (const).  The main one that made me quit was  
the problem that inout is supposed to solve.

For someone just trying to "get something working" on D2 from a D1 base,  
the biggest issue is to deal with string literals since you are forced to  
deal with const at that point.  Other than that, you can usually safely  
leave everything non-const-ified.  That's all I was saying.

And I don't think it's worth the effort to try and achieve backwards  
compatibility with D1.

-Steve
Nov 18 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
Steven Schveighoffer wrote:


 By peppering idup everywhere, you have created a conflict with the main 
 goal.  Refactoring in these cases isn't quite as simple as you say, and 
 it certainly won't be backwards compatible.


If you know the string is unique, you can use cast(string) rather than .idup, 
and no allocations will be done.



 In general, I tried to make Tango for D2 as const aware as possible 
 (it's a standard lib, and shouldn't require people to ignore const), and 
 found all sorts of issues with it (const).  The main one that made me 
 quit was the problem that inout is supposed to solve.


Right; inout is intended to fix that.
Nov 18 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 18 Nov 2010 15:49:10 -0500, Walter Bright  
<newshound2 digitalmars.com> wrote:


 Steven Schveighoffer wrote:


 By peppering idup everywhere, you have created a conflict with the main  
 goal.  Refactoring in these cases isn't quite as simple as you say, and  
 it certainly won't be backwards compatible.


 If you know the string is unique, you can use cast(string) rather than  
 .idup, and no allocations will be done.


class obj
{
    char[] name;
    this(char[] name) { this.name = name;}
}

What to do there?  If name is typically a string literal, then string is  
appropriate.  But if there are cases where a name is passed in via a  
char[] allocated on the heap, then const(char) makes sense.  However, you  
really don't want the name to change at some other point, so if it's not  
immutable it should idup the name.

It's not easy to make this code both safe and efficient.  In the end, I  
think in cases like this I did something like:

string name;
this(const(char)[] name) { this.name = name.idup;}
this(string name) {this.name = name;}

These are the kinds of cases I was talking about.  They require more than  
simple "search and replace" refactoring.  In reality, the original code is  
unsafe and flawed, but you want to try and support any existing code that  
uses it as well as possible.  There was also some anti-D2 sentiments to  
overcome when trying to get Tango to accept a D2 port, so I wanted to try  
and make the port as painless as possible.

-Steve
Nov 18 2010
next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday 18 November 2010 13:02:40 Steven Schveighoffer wrote:

 On Thu, 18 Nov 2010 15:49:10 -0500, Walter Bright
 
 <newshound2 digitalmars.com> wrote:

 Steven Schveighoffer wrote:


 By peppering idup everywhere, you have created a conflict with the main
 goal.  Refactoring in these cases isn't quite as simple as you say, and
 it certainly won't be backwards compatible.


 
 If you know the string is unique, you can use cast(string) rather than
 .idup, and no allocations will be done.


 
 class obj
 {
     char[] name;
     this(char[] name) { this.name = name;}
 }
 
 What to do there?  If name is typically a string literal, then string is
 appropriate.  But if there are cases where a name is passed in via a
 char[] allocated on the heap, then const(char) makes sense.  However, you
 really don't want the name to change at some other point, so if it's not
 immutable it should idup the name.
 
 It's not easy to make this code both safe and efficient.  In the end, I
 think in cases like this I did something like:
 
 string name;
 this(const(char)[] name) { this.name = name.idup;}
 this(string name) {this.name = name;}
 
 These are the kinds of cases I was talking about.  They require more than
 simple "search and replace" refactoring.  In reality, the original code is
 unsafe and flawed, but you want to try and support any existing code that
 uses it as well as possible.  There was also some anti-D2 sentiments to
 overcome when trying to get Tango to accept a D2 port, so I wanted to try
 and make the port as painless as possible.


And, of course, the other issue is the ideally, most code which uses strings 
would work with all of the various string types, so if you want to go all the 
way, you have to templatize up the wazoo on string functions.

- Jonathan M Davis
Nov 18 2010
prev sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-11-18 16:02:40 -0500, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:


 class obj
 {
     char[] name;
     this(char[] name) { this.name = name;}
 }
 
 What to do there?  If name is typically a string literal, then string 
 is  appropriate.  But if there are cases where a name is passed in via 
 a  char[] allocated on the heap, then const(char) makes sense.  
 However, you  really don't want the name to change at some other point, 
 so if it's not  immutable it should idup the name.
 
 It's not easy to make this code both safe and efficient.  In the end, I 
  think in cases like this I did something like:
 
 string name;
 this(const(char)[] name) { this.name = name.idup;}
 this(string name) {this.name = name;}


If you're only going to store the string as immutable, I think it's 
better to accept it only as immutable in the constructor. This way you 
pass the responsibility of iduping the string to the caller and this 
might reveal optimization opportunities. For instance:

	obj[100] versions;
	foreach (i; 1..100)
		versions[i] = new obj(name.idup);

See how it the inefficiency obvious when the caller is the one calling idup?

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Nov 18 2010
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 11/18/10 1:50 PM, Michel Fortin wrote:

 On 2010-11-18 16:02:40 -0500, "Steven Schveighoffer"
 <schveiguy yahoo.com> said:


 class obj
 {
 char[] name;
 this(char[] name) { this.name = name;}
 }

 What to do there? If name is typically a string literal, then string
 is appropriate. But if there are cases where a name is passed in via a
 char[] allocated on the heap, then const(char) makes sense. However,
 you really don't want the name to change at some other point, so if
 it's not immutable it should idup the name.

 It's not easy to make this code both safe and efficient. In the end, I
 think in cases like this I did something like:

 string name;
 this(const(char)[] name) { this.name = name.idup;}
 this(string name) {this.name = name;}


 If you're only going to store the string as immutable, I think it's
 better to accept it only as immutable in the constructor. This way you
 pass the responsibility of iduping the string to the caller and this
 might reveal optimization opportunities. For instance:

 obj[100] versions;
 foreach (i; 1..100)
 versions[i] = new obj(name.idup);

 See how it the inefficiency obvious when the caller is the one calling
 idup?


Yes, I think that's the best idiom (I use it in Phobos in a number of 
places). It's safe and as efficient as it could.

Andrei
Nov 18 2010
parent Walter Bright <newshound2 digitalmars.com> writes:
Andrei Alexandrescu wrote:

 On 11/18/10 1:50 PM, Michel Fortin wrote:

 On 2010-11-18 16:02:40 -0500, "Steven Schveighoffer"
 <schveiguy yahoo.com> said:


 class obj
 {
 char[] name;
 this(char[] name) { this.name = name;}
 }

 What to do there?







I agree that it just seems like good class design to have obj.name be
immutable, 
and pass the responsibility for making it so up the call chain to the
constructor.
Nov 18 2010
prev sibling parent reply Fawzi Mohamed <fawzi gmx.ch> writes:
 From the discussion it seems that defining something like:

version(D_Version2){
	template Const(T){
		alias const(T) Const;
	}
	template Immutable(T){
		alias immutable(T) Immutable;
	}
	immutable(T) Idup(T)(T val){
		return val.idup;
	}
	alias const(char)[] cstring;
} else {
	template Const(T){
		alias T Const;
	}
	template Immutable(T){
		alias T Immutable;
	}
	T Idup(T)(T val){
		return val.dup;
	}
	alias char[] string;
	alias char[] cstring;
}

could help a lot
later one can simply replace Const! with const and Immutable! with  
immutable, Idup replacement is more complicated, but doable
What is not so clear is how to cope with scope, because I have lot of  
delegates around that will need it.
For it a preprocessing step might really be the best thing, or someone  
knows a smart way to cope with that?
Nov 18 2010
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Fawzi Mohamed wrote:

  From the discussion it seems that defining something like:
 
 version(D_Version2){
     template Const(T){
         alias const(T) Const;
     }
     template Immutable(T){
         alias immutable(T) Immutable;
     }
     immutable(T) Idup(T)(T val){
         return val.idup;
     }
     alias const(char)[] cstring;
 } else {
     template Const(T){
         alias T Const;
     }
     template Immutable(T){
         alias T Immutable;
     }
     T Idup(T)(T val){
         return val.dup;
     }
     alias char[] string;
     alias char[] cstring;
 }
 
 could help a lot
 later one can simply replace Const! with const and Immutable! with 
 immutable, Idup replacement is more complicated, but doable


The problem with this approach is it requires the D1 compiler to be able to 
parse D2 syntax, including recognizing all of D2's keywords.


 What is not so clear is how to cope with scope, because I have lot of 
 delegates around that will need it.
 For it a preprocessing step might really be the best thing, or someone 
 knows a smart way to cope with that?


Not sure what the issue is.
Nov 18 2010
next sibling parent reply Fawzi Mohamed <fawzi gmx.ch> writes:
On 19-nov-10, at 08:08, Walter Bright wrote:


 Fawzi Mohamed wrote:

 From the discussion it seems that defining something like:
 version(D_Version2){




mixin(`

    template Const(T){
        alias const(T) Const;
    }
    template Immutable(T){
        alias immutable(T) Immutable;
    }
    immutable(T) Idup(T)(T val){
        return val.idup;
    }
    alias const(char)[] cstring;




`);

 } else {
    template Const(T){
        alias T Const;
    }
    template Immutable(T){
        alias T Immutable;
    }
    T Idup(T)(T val){
        return val.dup;
    }
    alias char[] string;
    alias char[] cstring;
 }
 could help a lot
 later one can simply replace Const! with const and Immutable! with  
 immutable, Idup replacement is more complicated, but doable


 The problem with this approach is it requires the D1 compiler to be  
 able to parse D2 syntax, including recognizing all of D2's keywords.


yes you are right, I meant to put a mixin ther but then I forgot it.


 What is not so clear is how to cope with scope, because I have lot  
 of delegates around that will need it.
 For it a preprocessing step might really be the best thing, or  
 someone knows a smart way to cope with that?


 Not sure what the issue is.


I don't find a valid D1 expression to put in place of scope, or to  
somehow hide it, i.e. how do you write something like
module t;

void f(scope void delegate() action){
     action();
}

void main(){
     f(scope delegate(){
         printf("bla\n");
     });
}

so that it is valid D1 and D2?
Nov 18 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
Fawzi Mohamed wrote:

 I don't find a valid D1 expression to put in place of scope, or to 
 somehow hide it, i.e. how do you write something like
 module t;
 
 void f(scope void delegate() action){
     action();
 }
 
 void main(){
     f(scope delegate(){
         printf("bla\n");
     });
 }
 
 so that it is valid D1 and D2?


Just remove the 'scope'.
Nov 18 2010
parent Fawzi Mohamed <fawzi gmx.ch> writes:
On 19-nov-10, at 08:44, Walter Bright wrote:


 Fawzi Mohamed wrote:

 I don't find a valid D1 expression to put in place of scope, or to  
 somehow hide it, i.e. how do you write something like
 module t;
 void f(scope void delegate() action){
    action();
 }
 void main(){
    f(scope delegate(){
        printf("bla\n");
    });
 }
 so that it is valid D1 and D2?


 Just remove the 'scope'.


removing scope causes heap allocation in D2 that I want to avoid.
Still maybe you are right, I will use /+scope+/, so that one has  
something working, and easily go to the efficient D2 version
Nov 19 2010
prev sibling parent Sean Kelly <sean invisibleduck.org> writes:
Walter Bright <newshound2 digitalmars.com> wrote:

 Fawzi Mohamed wrote:

  From the discussion it seems that defining something like:

 version(D_Version2){


     template Const(T){
         alias const(T) Const;
     }
     template Immutable(T){
         alias immutable(T) Immutable;
     }
     immutable(T) Idup(T)(T val){
         return val.idup;
     }
     alias const(char)[] cstring;
 } else {
     template Const(T){
         alias T Const;
     }
     template Immutable(T){
         alias T Immutable;
     }
     T Idup(T)(T val){
         return val.dup;
     }
     alias char[] string;
     alias char[] cstring;
 }

 could help a lot


 later one can simply replace Const! with const and Immutable! with >
 immutable, Idup replacement is more complicated, but doable


 
 The problem with this approach is it requires the D1 compiler to be
 able to parse D2 syntax, including recognizing all of D2's keywords.


Make them string mixins. I had druntime working on both D1 and D2 for a
while and did it with a mix of aliases, string mixins, and "in" for
const args. It worked, but ended up being worse than simply maintaining
two codebases.
Nov 21 2010
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 19 Nov 2010 01:47:27 -0500, Fawzi Mohamed <fawzi gmx.ch> wrote:


  From the discussion it seems that defining something like:

 version(D_Version2){
 	template Const(T){
 		alias const(T) Const;
 	}
 	template Immutable(T){
 		alias immutable(T) Immutable;
 	}
 	immutable(T) Idup(T)(T val){
 		return val.idup;
 	}
 	alias const(char)[] cstring;
 } else {
 	template Const(T){
 		alias T Const;
 	}
 	template Immutable(T){
 		alias T Immutable;
 	}
 	T Idup(T)(T val){
 		return val.dup;
 	}
 	alias char[] string;
 	alias char[] cstring;
 }

 could help a lot
 later one can simply replace Const! with const and Immutable! with  
 immutable, Idup replacement is more complicated, but doable
 What is not so clear is how to cope with scope, because I have lot of  
 delegates around that will need it.
 For it a preprocessing step might really be the best thing, or someone  
 knows a smart way to cope with that?


BTW, this doesn't work as well as you would think.  The problem is with  
IFTI.

For example, IFTI can handle this:

void foo(T)(const(T) arg)

but cannot handle this:

void foo(T)(Const!(T) arg)

It was one of the things we were trying to do in Tango for D2, we had  
aliases for const/immutable/mutable strings templated on character type,  
but all of Tango's IFTI usage failed to compile.  It's a really old bug I  
filed:

http://d.puremagic.com/issues/show_bug.cgi?id=1653

which was superseded by:

http://d.puremagic.com/issues/show_bug.cgi?id=1807

-Steve
Nov 19 2010
prev sibling parent reply spir <denis.spir gmail.com> writes:
On Thu, 18 Nov 2010 07:34:01 -0500
"Steven Schveighoffer" <schveiguy yahoo.com> wrote:


 4. if you intend on returning a portion of the string, well, currently =20
 you're SOL, but eventually inout will come into play here.
=20
 For example, Tango's log package fell under rules 1 and 3 (most of the =20
 time you are creating log objects with string literals, and logging a =20
 message does not save the message anywhere).
 Many many cases fell under rule 4, which is one of the main reasons I gav=


e =20

 up (inout still isn't ready).


I don't understand the issue here. Why don't slices do the job?

Denis
-- -- -- -- -- -- --
vit esse estrany =E2=98=A3

spir.wikidot.com
Nov 18 2010
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 18 Nov 2010 14:43:17 -0500, spir <denis.spir gmail.com> wrote:


 On Thu, 18 Nov 2010 07:34:01 -0500
 "Steven Schveighoffer" <schveiguy yahoo.com> wrote:


 4. if you intend on returning a portion of the string, well, currently
 you're SOL, but eventually inout will come into play here.

 For example, Tango's log package fell under rules 1 and 3 (most of the
 time you are creating log objects with string literals, and logging a
 message does not save the message anywhere).
 Many many cases fell under rule 4, which is one of the main reasons I  
 gave
 up (inout still isn't ready).


 I don't understand the issue here. Why don't slices do the job?


It's the source of the problem for which inout was invented to solve.  How  
do you specify a function that says "I won't molest the data" but returns  
a portion of that data?

Simplest example is strstr:

const(char)[] strstr(const(char)[] haystack, const(char)[] needle)

Notice that the return value *must* be const(char)[].  But what if you  
have a char[]?  All of a sudden, you can't modify that data which was  
returned to you.  In essence, strstr modified your const contract with  
your data.  The useful idiom:

str = strstr(str, "hello");

doesn't work, because you can't reassign the data.

A solution is to have 3 overloads, one for each constancy type, but the  
mutable version doesn't guarantee it won't modify the data, and you have  
to repeat the exact same code.  Another solution is to use a template, but  
that also has the same problem that it doesn't prevent the mutable version  
 from modifying the data, plus does not indicate that you can use const  
data.  In addition, templates cannot be virtual functions.

But inout solves this in a different way that does exactly what we want.   
If you have read TDPL, then it should explain how inout is *supposed* to  
work, or you can read DIP2  
(http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP2).  inout was  
chosen instead of vconst to allow reusing essentially a dead keyword.

-Steve
Nov 18 2010
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2010-11-18 11:51, Fawzi Mohamed wrote:

 Is there any "porting" guide around in a wiki?
 If not a page where to share the best tricks would be nice "D1->D2
 conversion tricks"?

 In the short term I don't think that going D2 only is really an option
 for me, so how feasible it is to keep the code base compatible to both
 D1 and D2?

 I know that one can define some templates (for example Const(T),....),
 and maybe use mixins, but how much uglier does the code become as result?
 I choose D to have cleaner code, I am not interested in loosing all that
 just to be D1 and D2, then I prefer to wait, and convert everything at
 once.

 Well that is about it...

 thanks

 Fawzi


The experience I have with this is the Orange library which supports 
both D1 (Tango) and D2. I say this: it can sometimes be surprisingly 
easy and sometimes a PITA.

Depending on what you do you can get a long way with aliases, I have for 
quite a long time now been using a string alias in D1 (I just like it 
better than char[]).

For some things you will have to use string mixins and after some 
testing with Derelict2 we found out that it's better to use few string 
mixins containing a lot of code than many string mixins containing just 
some code. Otherwise the compile time will increase, sometimes quite a lot.

-- 
/Jacob Carlborg
Nov 18 2010