Ary Manzana <ary esperanto.org.ar> writes:

Having a D compiler available as a library will (at least) give these 
benefits:

   1. Can be used by an IDE: D is statically typed and so an IDE can 
benefit a lot from this. The features Descent had, as far as I remember, 
were:
     1.1. Outline
     1.2. Autocompletion
     1.3. Type Hierarchy
     1.4. Syntax and semantic errors, showing not only the line number 
but also column numbers if it makes sense
     1.5. Automatic import inclusion (say, typing writefln and getting a 
list of modules that provide that symbol)
     1.6. Compile-time view: replace auto with the inferred type, insert 
mixins into scope, rewrite operator overloads and other lowerings (but 
I'm not sure this point is really useful)
     1.7. Determine, given a set of versions and flags, which branches 
of static ifs are used/unused
     1.8. Open declaration
     1.9. Show implementations (of an interface, of interface's method 
or, abstract methods, or method overrides).
     1.10. Propose to override a method (you type some letters and then 
hit some key combination and get a list of methods to override)
     1.11. Get the code of a template when instantiated.
  2. Can be used to build better doc generators: one that shows known 
subclasses or interface implementation, shows inherited methods, type 
hierarchy.
  3. Can be used for lints and other such tools.

As you can see, a simple lexer/parser built into an IDE, doc generator 
or lint will just give basic features but will never achieve something 
exceptionally good if it lacks the full semantic knowledge of the code.

I'll write a list of things I'd like this compiler-as-library to have, 
but please help me make it bigger :-)

  * Don't use global variables (DMD is just thought to be run once, so 
when used as a library it can just be used, well, once)
  * Provide a lexer which gives line numbers and column numbers 
(beginning, end)
  * Provide a parser with the same features
  * The semantic phase should not discard any information found while 
parsing. For example when DMD resolves a type it recursively resolves 
aliasing and keeps the last one. An example:

   alias int foo;
   alias foo* bar;

   bar something() { ... }

   It would be nice if "bar", after semantic analysis is done, carries 
the information that bar is "foo*" and that "foo" is "int". Also that 
something's return type is "bar", not "int*".
  * Provide errors and warnings that have line numbers as well as column 
numbers.
  * Allow to parse the top-level definitions of a module. Whit this I 
mean skipping function bodies. At least Descent first built a the 
outline of the whole project by doing this. This mode should also allow 
specifying a location as a target, and if that location falls inside a 
function body then it's contents are returned (useful when editing a 
file, so you can get the outline as well as semantic info of the 
function currently being edited, which will never affect semantic in 
other parts of the module). This will dramatically speed up the editor.
  * Don't stop parsing on errors (I think DMD already does this).
  * Provide a visitor class. If possible, use visitors to implement 
semantic analysis. The visitor will make it super easy to implement 
lints and to generate documentation.

deadalnix <deadalnix gmail.com> writes:

Le 13/04/2012 11:58, Ary Manzana a écrit :
 Having a D compiler available as a library will (at least) give these
 benefits:

 1. Can be used by an IDE: D is statically typed and so an IDE can
 benefit a lot from this. The features Descent had, as far as I remember,
 were:
 1.1. Outline
 1.2. Autocompletion
 1.3. Type Hierarchy
 1.4. Syntax and semantic errors, showing not only the line number but
 also column numbers if it makes sense
 1.5. Automatic import inclusion (say, typing writefln and getting a list
 of modules that provide that symbol)
 1.6. Compile-time view: replace auto with the inferred type, insert
 mixins into scope, rewrite operator overloads and other lowerings (but
 I'm not sure this point is really useful)
 1.7. Determine, given a set of versions and flags, which branches of
 static ifs are used/unused
 1.8. Open declaration
 1.9. Show implementations (of an interface, of interface's method or,
 abstract methods, or method overrides).
 1.10. Propose to override a method (you type some letters and then hit
 some key combination and get a list of methods to override)
 1.11. Get the code of a template when instantiated.
 2. Can be used to build better doc generators: one that shows known
 subclasses or interface implementation, shows inherited methods, type
 hierarchy.
 3. Can be used for lints and other such tools.

 As you can see, a simple lexer/parser built into an IDE, doc generator
 or lint will just give basic features but will never achieve something
 exceptionally good if it lacks the full semantic knowledge of the code.

 I'll write a list of things I'd like this compiler-as-library to have,
 but please help me make it bigger :-)

 * Don't use global variables (DMD is just thought to be run once, so
 when used as a library it can just be used, well, once)
 * Provide a lexer which gives line numbers and column numbers
 (beginning, end)
 * Provide a parser with the same features
 * The semantic phase should not discard any information found while
 parsing. For example when DMD resolves a type it recursively resolves
 aliasing and keeps the last one. An example:

 alias int foo;
 alias foo* bar;

 bar something() { ... }

 It would be nice if "bar", after semantic analysis is done, carries the
 information that bar is "foo*" and that "foo" is "int". Also that
 something's return type is "bar", not "int*".
 * Provide errors and warnings that have line numbers as well as column
 numbers.
 * Allow to parse the top-level definitions of a module. Whit this I mean
 skipping function bodies. At least Descent first built a the outline of
 the whole project by doing this. This mode should also allow specifying
 a location as a target, and if that location falls inside a function
 body then it's contents are returned (useful when editing a file, so you
 can get the outline as well as semantic info of the function currently
 being edited, which will never affect semantic in other parts of the
 module). This will dramatically speed up the editor.
 * Don't stop parsing on errors (I think DMD already does this).
 * Provide a visitor class. If possible, use visitors to implement
 semantic analysis. The visitor will make it super easy to implement
 lints and to generate documentation.

SDC have a lot of theses, and I proposed a similar stuff for its 
evolution. I think it is easier for SDC than it is for dmd considering 
the codebase of both.

"Jakob Ovrum" <jakobovrum gmail.com> writes:

On Friday, 13 April 2012 at 13:08:51 UTC, deadalnix wrote:
 SDC have a lot of theses, and I proposed a similar stuff for 
 its evolution. I think it is easier for SDC than it is for dmd 
 considering the codebase of both.

I think we've got the lexer and parser completely separate from
most of the rest of the codebase (like the codegen), due to
repeated requests from people who wanted to use these parts for
IDEs and other tools.

I've yet to see anyone actually go through with using it though,
possibly because there is no documentation for a lot of it.
Documenting these parts fully into something of a public API and
then putting it online is definitely on the todo list. Perhaps
there would be more motivation to do this rather than work on
something else if someone actually tried using SDC in their
project instead of just talking about it, so it's kind of a
catch-22.

That said, the parser is currently evolving alongside the
codegen. When we want to start implementing new parts of the
language, we iteratively add it to the parser, hence it's not
complete. It's very easy to work with though and it's mostly a
menial task (although it's kind of fun to produce beautiful
parser errors :P).

Anyway, for anyone interested, you can find us on Github and
#d.sdc on FreeNode.

Jacob Carlborg <doob me.com> writes:

On 2012-04-13 17:25, Jakob Ovrum wrote:

 I think we've got the lexer and parser completely separate from
 most of the rest of the codebase (like the codegen), due to
 repeated requests from people who wanted to use these parts for
 IDEs and other tools.

Cool.

 I've yet to see anyone actually go through with using it though,
 possibly because there is no documentation for a lot of it.
 Documenting these parts fully into something of a public API and
 then putting it online is definitely on the todo list. Perhaps
 there would be more motivation to do this rather than work on
 something else if someone actually tried using SDC in their
 project instead of just talking about it, so it's kind of a
 catch-22.

That's always a problem.

 That said, the parser is currently evolving alongside the
 codegen. When we want to start implementing new parts of the
 language, we iteratively add it to the parser, hence it's not
 complete. It's very easy to work with though and it's mostly a
 menial task (although it's kind of fun to produce beautiful
 parser errors :P).

 Anyway, for anyone interested, you can find us on Github and
 #d.sdc on FreeNode.

How does it compare to DMD, does it implement the whole language yet?

-- 
/Jacob Carlborg

"F i L" <witte2008 gmail.com> writes:

Jacob Carlborg wrote:
 How does it compare to DMD, does it implement the whole 
 language yet?

https://github.com/bhelyer/SDC

It lists feature support.

Trass3r <un known.com> writes:

 I think we've got the lexer and parser completely separate from
 most of the rest of the codebase (like the codegen), due to
 repeated requests from people who wanted to use these parts for
 IDEs and other tools.

Still some things to learn from Clang though.
e.g. it still directly builds an AST instead of using some kind of  
interface.

btw, why do you use exceptions for compiler errors instead of a cheaper  
and more sophisticated system?

 I've yet to see anyone actually go through with using it though,
 possibly because there is no documentation for a lot of it.

Yep, no comments anywhere :/

deadalnix <deadalnix gmail.com> writes:

Le 13/04/2012 22:31, Trass3r a écrit :
 I think we've got the lexer and parser completely separate from
 most of the rest of the codebase (like the codegen), due to
 repeated requests from people who wanted to use these parts for
 IDEs and other tools.

 Still some things to learn from Clang though.
 e.g. it still directly builds an AST instead of using some kind of
 interface.

I'm very interested in what you mean here. Do you have a link or can you 
write an explaination ?

 btw, why do you use exceptions for compiler errors instead of a cheaper
 and more sophisticated system?

Currently it is exception only, but this is a known issue.

Trass3r <un known.com> writes:

 Still some things to learn from Clang though.
 e.g. it still directly builds an AST instead of using some kind of
 interface.

 I'm very interested in what you mean here. Do you have a link or can you  
 write an explaination ?

Clang decouples the parser from AST construction by letting the parser  
take an interface class that has a virtual method for everything that is  
parsed along the lines of ActOnStartOfFunctionDef().
So a client that doesn't need an AST doesn't have to build one.

Don't know if it's properly implemented in Clang now though.

deadalnix <deadalnix gmail.com> writes:

Le 14/04/2012 18:26, Trass3r a écrit :
 Still some things to learn from Clang though.
 e.g. it still directly builds an AST instead of using some kind of
 interface.

 I'm very interested in what you mean here. Do you have a link or can
 you write an explaination ?

 Clang decouples the parser from AST construction by letting the parser
 take an interface class that has a virtual method for everything that is
 parsed along the lines of ActOnStartOfFunctionDef().
 So a client that doesn't need an AST doesn't have to build one.

 Don't know if it's properly implemented in Clang now though.

Ho I see ! That is brilliant !

Manu <turkeyman gmail.com> writes:

On 13 April 2012 18:25, Jakob Ovrum <jakobovrum gmail.com> wrote:

 On Friday, 13 April 2012 at 13:08:51 UTC, deadalnix wrote:

 SDC have a lot of theses, and I proposed a similar stuff for its
 evolution. I think it is easier for SDC than it is for dmd considering the
 codebase of both.

 I think we've got the lexer and parser completely separate from
 most of the rest of the codebase (like the codegen), due to
 repeated requests from people who wanted to use these parts for
 IDEs and other tools.

 I've yet to see anyone actually go through with using it though,
 possibly because there is no documentation for a lot of it.
 Documenting these parts fully into something of a public API and
 then putting it online is definitely on the todo list. Perhaps
 there would be more motivation to do this rather than work on
 something else if someone actually tried using SDC in their
 project instead of just talking about it, so it's kind of a
 catch-22.

 That said, the parser is currently evolving alongside the
 codegen. When we want to start implementing new parts of the
 language, we iteratively add it to the parser, hence it's not
 complete. It's very easy to work with though and it's mostly a
 menial task (although it's kind of fun to produce beautiful
 parser errors :P).

 Anyway, for anyone interested, you can find us on Github and
 #d.sdc on FreeNode.

Just out of curiosity, why would anyone write a code gen these days? With
projects like LLVM, which can perform great codegen to basically any
architecture, you'd be crazy not to use that...
Surely 90% of the value of writing your own D compiler would be the unique
front end, which may have different design principles (like use as a lib,
usable on tools and stuff as discussed here) ?

I'm sorry to say, if you write your own codegen, I would never use your
compiler. If you use LLVM in the back end, I'll definitely give it a look,
then merit will depend entirely on the front end (speed, flexibility,
quality of error messages, runtime error detection, etc).
I know it's a fun exercise to write a codegen, so from an educational
perspective, sure, it's valuable to you as a programmer, but I don't think
it helps your project at all.

Jacob Carlborg <doob me.com> writes:

On 2012-04-14 12:35, Manu wrote:
 On 13 April 2012 18:25, Jakob Ovrum <jakobovrum gmail.com
     That said, the parser is currently evolving alongside the
     codegen. When we want to start implementing new parts of the
     language, we iteratively add it to the parser, hence it's not
     complete. It's very easy to work with though and it's mostly a
     menial task (although it's kind of fun to produce beautiful
     parser errors :P).

     Anyway, for anyone interested, you can find us on Github and
     #d.sdc on FreeNode.


 Just out of curiosity, why would anyone write a code gen these days?
 With projects like LLVM, which can perform great codegen to basically
 any architecture, you'd be crazy not to use that...
 Surely 90% of the value of writing your own D compiler would be the
 unique front end, which may have different design principles (like use
 as a lib, usable on tools and stuff as discussed here) ?

 I'm sorry to say, if you write your own codegen, I would never use your
 compiler. If you use LLVM in the back end, I'll definitely give it a
 look, then merit will depend entirely on the front end (speed,
 flexibility, quality of error messages, runtime error detection, etc).
 I know it's a fun exercise to write a codegen, so from an educational
 perspective, sure, it's valuable to you as a programmer, but I don't
 think it helps your project at all.

I don't know what Jakob means with "codegen" in this case but SDC 
depends on LLVM. Have a look at the bottom of:

https://github.com/bhelyer/SDC

-- 
/Jacob Carlborg

deadalnix <deadalnix gmail.com> writes:

Le 14/04/2012 12:35, Manu a écrit :
 On 13 April 2012 18:25, Jakob Ovrum <jakobovrum gmail.com
 <mailto:jakobovrum gmail.com>> wrote:

     On Friday, 13 April 2012 at 13:08:51 UTC, deadalnix wrote:

         SDC have a lot of theses, and I proposed a similar stuff for its
         evolution. I think it is easier for SDC than it is for dmd
         considering the codebase of both.


     I think we've got the lexer and parser completely separate from
     most of the rest of the codebase (like the codegen), due to
     repeated requests from people who wanted to use these parts for
     IDEs and other tools.

     I've yet to see anyone actually go through with using it though,
     possibly because there is no documentation for a lot of it.
     Documenting these parts fully into something of a public API and
     then putting it online is definitely on the todo list. Perhaps
     there would be more motivation to do this rather than work on
     something else if someone actually tried using SDC in their
     project instead of just talking about it, so it's kind of a
     catch-22.

     That said, the parser is currently evolving alongside the
     codegen. When we want to start implementing new parts of the
     language, we iteratively add it to the parser, hence it's not
     complete. It's very easy to work with though and it's mostly a
     menial task (although it's kind of fun to produce beautiful
     parser errors :P).

     Anyway, for anyone interested, you can find us on Github and
     #d.sdc on FreeNode.


 Just out of curiosity, why would anyone write a code gen these days?
 With projects like LLVM, which can perform great codegen to basically
 any architecture, you'd be crazy not to use that...
 Surely 90% of the value of writing your own D compiler would be the
 unique front end, which may have different design principles (like use
 as a lib, usable on tools and stuff as discussed here) ?

 I'm sorry to say, if you write your own codegen, I would never use your
 compiler. If you use LLVM in the back end, I'll definitely give it a
 look, then merit will depend entirely on the front end (speed,
 flexibility, quality of error messages, runtime error detection, etc).
 I know it's a fun exercise to write a codegen, so from an educational
 perspective, sure, it's valuable to you as a programmer, but I don't
 think it helps your project at all.

Codegen is done by LLVM in SDC, but you still need some codegen glue.

Manu <turkeyman gmail.com> writes:

On 14 April 2012 16:51, deadalnix <deadalnix gmail.com> wrote:

 Le 14/04/2012 12:35, Manu a =C3=A9crit :

 On 13 April 2012 18:25, Jakob Ovrum <jakobovrum gmail.com
 <mailto:jakobovrum gmail.com>> wrote:

    On Friday, 13 April 2012 at 13:08:51 UTC, deadalnix wrote:

        SDC have a lot of theses, and I proposed a similar stuff for its
        evolution. I think it is easier for SDC than it is for dmd
        considering the codebase of both.


    I think we've got the lexer and parser completely separate from
    most of the rest of the codebase (like the codegen), due to
    repeated requests from people who wanted to use these parts for
    IDEs and other tools.

    I've yet to see anyone actually go through with using it though,
    possibly because there is no documentation for a lot of it.
    Documenting these parts fully into something of a public API and
    then putting it online is definitely on the todo list. Perhaps
    there would be more motivation to do this rather than work on
    something else if someone actually tried using SDC in their
    project instead of just talking about it, so it's kind of a
    catch-22.

    That said, the parser is currently evolving alongside the
    codegen. When we want to start implementing new parts of the
    language, we iteratively add it to the parser, hence it's not
    complete. It's very easy to work with though and it's mostly a
    menial task (although it's kind of fun to produce beautiful
    parser errors :P).

    Anyway, for anyone interested, you can find us on Github and
    #d.sdc on FreeNode.


 Just out of curiosity, why would anyone write a code gen these days?
 With projects like LLVM, which can perform great codegen to basically
 any architecture, you'd be crazy not to use that...
 Surely 90% of the value of writing your own D compiler would be the
 unique front end, which may have different design principles (like use
 as a lib, usable on tools and stuff as discussed here) ?

 I'm sorry to say, if you write your own codegen, I would never use your
 compiler. If you use LLVM in the back end, I'll definitely give it a
 look, then merit will depend entirely on the front end (speed,
 flexibility, quality of error messages, runtime error detection, etc).
 I know it's a fun exercise to write a codegen, so from an educational
 perspective, sure, it's valuable to you as a programmer, but I don't
 think it helps your project at all.

 Codegen is done by LLVM in SDC, but you still need some codegen glue.

Ah okay, cool. NM me then :P

Jacob Carlborg <doob me.com> writes:

On 2012-04-13 15:10, deadalnix wrote:

 SDC have a lot of theses, and I proposed a similar stuff for its
 evolution. I think it is easier for SDC than it is for dmd considering
 the codebase of both.

I would guess so as well, although I have no experience of SDC.

-- 
/Jacob Carlborg

Ary Manzana <ary esperanto.org.ar> writes:

On 4/13/12 9:10 PM, deadalnix wrote:
 Le 13/04/2012 11:58, Ary Manzana a écrit :
 Having a D compiler available as a library will (at least) give these
 benefits:

 1. Can be used by an IDE: D is statically typed and so an IDE can
 benefit a lot from this. The features Descent had, as far as I remember,
 were:
 1.1. Outline
 1.2. Autocompletion
 1.3. Type Hierarchy
 1.4. Syntax and semantic errors, showing not only the line number but
 also column numbers if it makes sense
 1.5. Automatic import inclusion (say, typing writefln and getting a list
 of modules that provide that symbol)
 1.6. Compile-time view: replace auto with the inferred type, insert
 mixins into scope, rewrite operator overloads and other lowerings (but
 I'm not sure this point is really useful)
 1.7. Determine, given a set of versions and flags, which branches of
 static ifs are used/unused
 1.8. Open declaration
 1.9. Show implementations (of an interface, of interface's method or,
 abstract methods, or method overrides).
 1.10. Propose to override a method (you type some letters and then hit
 some key combination and get a list of methods to override)
 1.11. Get the code of a template when instantiated.
 2. Can be used to build better doc generators: one that shows known
 subclasses or interface implementation, shows inherited methods, type
 hierarchy.
 3. Can be used for lints and other such tools.

 As you can see, a simple lexer/parser built into an IDE, doc generator
 or lint will just give basic features but will never achieve something
 exceptionally good if it lacks the full semantic knowledge of the code.

 I'll write a list of things I'd like this compiler-as-library to have,
 but please help me make it bigger :-)

 * Don't use global variables (DMD is just thought to be run once, so
 when used as a library it can just be used, well, once)
 * Provide a lexer which gives line numbers and column numbers
 (beginning, end)
 * Provide a parser with the same features
 * The semantic phase should not discard any information found while
 parsing. For example when DMD resolves a type it recursively resolves
 aliasing and keeps the last one. An example:

 alias int foo;
 alias foo* bar;

 bar something() { ... }

 It would be nice if "bar", after semantic analysis is done, carries the
 information that bar is "foo*" and that "foo" is "int". Also that
 something's return type is "bar", not "int*".
 * Provide errors and warnings that have line numbers as well as column
 numbers.
 * Allow to parse the top-level definitions of a module. Whit this I mean
 skipping function bodies. At least Descent first built a the outline of
 the whole project by doing this. This mode should also allow specifying
 a location as a target, and if that location falls inside a function
 body then it's contents are returned (useful when editing a file, so you
 can get the outline as well as semantic info of the function currently
 being edited, which will never affect semantic in other parts of the
 module). This will dramatically speed up the editor.
 * Don't stop parsing on errors (I think DMD already does this).
 * Provide a visitor class. If possible, use visitors to implement
 semantic analysis. The visitor will make it super easy to implement
 lints and to generate documentation.

 SDC have a lot of theses, and I proposed a similar stuff for its
 evolution. I think it is easier for SDC than it is for dmd considering
 the codebase of both.

Cool! SDC is the way to go. Let's focus our efforts on that project. :-)

One thing I saw in the code is that global variables are used in it 
(specially in the sdc.aglobal module).

Also, the lexer returns a TokenStream which contains the full array of 
tokens. This is very slow compared to returning them as they are 
scanned, when requested. But I guess this is easy to fix as it's hidden 
behind the TokenStream interface.

I tried to compile a simple file:

void main() {}

 ./bin/sdc main.d

core.exception.AssertError sdc.gen.sdcmodule(560): Assertion failure
----------------
5   sdc                                 0x000000010b87aa2a _d_assertm + 42
6   sdc                                 0x000000010b4e2daa void 
sdc.gen.sdcmodule.__assert(int) + 26
7   sdc                                 0x000000010b55d19c void 
sdc.gen.sdcmodule.Store.addFunction(sdc.gen.sdcfunction.Function) + 92
8   sdc                                 0x000000010b55d75b void 
sdc.gen.sdcmodule.Scope.add(immutable(char)[], 
sdc.gen.sdcfunction.Function) + 123
9   sdc                                 0x000000010b54ee36 void 
sdc.gen.declaration.declareFunctionDeclaration(sdc.ast.declaration.
unctionDeclaration, 
sdc.ast.sdcmodule.DeclarationDefinition, sdc.gen.sdcmodule.Module) + 1278
10  sdc                                 0x000000010b54e4b2 void 
sdc.gen.declaration.declareDeclaration(sdc.ast.declaration.Declaration, 
sdc.ast.sdcmodule.DeclarationDefinition, sdc.gen.sdcmodule.Module) + 146
11  sdc                                 0x000000010b54d12e void 
sdc.gen.base.genDeclarationDefinition(sdc.ast.sdcmodule.DeclarationDefinition, 
sdc.gen.sdcmodule.Module, ulong) + 374
12  sdc                                 0x000000010b54c8d0 void 
sdc.gen.base.resolveDeclarationDefinitionList(sdc.ast.sdcmodule.Decl
rationDefinition[], 
sdc.gen.sdcmodule.Module, sdc.gen.type.Type) + 564
13  sdc                                 0x000000010b54c439 
sdc.gen.sdcmodule.Module 
sdc.gen.base.genModule(sdc.ast.sdcmodule.Module, 
sdc.aglobal.TranslationUnit) + 157
14  sdc                                 0x000000010b53432d int 
sdc.sdc.realmain(immutable(char)[][]) + 3081
15  sdc                                 0x000000010b4e283e _Dmain + 54
16  sdc                                 0x000000010b87b415 extern (C) 
int rt.dmain2.main(int, char**).void runMain() + 29
17  sdc                                 0x000000010b87adc5 extern (C) 
int rt.dmain2.main(int, char**).void tryExec(scope void delegate()) + 45
18  sdc                                 0x000000010b87b467 extern (C) 
int rt.dmain2.main(int, char**).void runAll() + 63
19  sdc                                 0x000000010b87adc5 extern (C) 
int rt.dmain2.main(int, char**).void tryExec(scope void delegate()) + 45
20  sdc                                 0x000000010b87ad48 main + 232
21  sdc                                 0x000000010b4e2694 start + 52
22  ???                                 0x0000000000000002 0x0 + 2
----------------

I couldn't find the line of the assertion. What are all those "+146" and 
"+54" about?

Jacob Carlborg <doob me.com> writes:

On 2012-04-16 04:35, Ary Manzana wrote:
 On 4/13/12 9:10 PM, deadalnix wrote:
 SDC have a lot of theses, and I proposed a similar stuff for its
 evolution. I think it is easier for SDC than it is for dmd considering
 the codebase of both.

 Cool! SDC is the way to go. Let's focus our efforts on that project. :-)

I noticed that SDC is licensed under the GPL license :( That is not 
compatible with, for example, EPL used by Eclipse.

-- 
/Jacob Carlborg

"Jakob Ovrum" <jakobovrum gmail.com> writes:

On Monday, 16 April 2012 at 06:37:00 UTC, Jacob Carlborg wrote:
 On 2012-04-16 04:35, Ary Manzana wrote:
 On 4/13/12 9:10 PM, deadalnix wrote:
 SDC have a lot of theses, and I proposed a similar stuff for 
 its
 evolution. I think it is easier for SDC than it is for dmd 
 considering
 the codebase of both.

 Cool! SDC is the way to go. Let's focus our efforts on that 
 project. :-)

 I noticed that SDC is licensed under the GPL license :( That is 
 not compatible with, for example, EPL used by Eclipse.

We are changing the license soon, with BSD/MIT in mind. I am 
really just waiting for Bernard to make the change, we've gotten 
permission from all contributors as far as I know.

Jacob Carlborg <doob me.com> writes:

On 2012-04-16 09:00, Jakob Ovrum wrote:

 We are changing the license soon, with BSD/MIT in mind. I am really just
 waiting for Bernard to make the change, we've gotten permission from all
 contributors as far as I know.

Ok, that sounds good. I would prefer Boost but I guess BSD/MIT is also ok.

-- 
/Jacob Carlborg

deadalnix <deadalnix gmail.com> writes:

Le 16/04/2012 09:00, Jakob Ovrum a écrit :
 On Monday, 16 April 2012 at 06:37:00 UTC, Jacob Carlborg wrote:
 On 2012-04-16 04:35, Ary Manzana wrote:
 On 4/13/12 9:10 PM, deadalnix wrote:
 SDC have a lot of theses, and I proposed a similar stuff for its
 evolution. I think it is easier for SDC than it is for dmd considering
 the codebase of both.

 Cool! SDC is the way to go. Let's focus our efforts on that project. :-)

 I noticed that SDC is licensed under the GPL license :( That is not
 compatible with, for example, EPL used by Eclipse.

 We are changing the license soon, with BSD/MIT in mind. I am really just
 waiting for Bernard to make the change, we've gotten permission from all
 contributors as far as I know.

You didn't asked me, but now you have mine. (Anyway, I'm not a big 
contributor, and only have submitted some small patches).

"Bernard Helyer" <b.helyer gmail.com> writes:

On Monday, 16 April 2012 at 11:34:14 UTC, deadalnix wrote:
 Le 16/04/2012 09:00, Jakob Ovrum a écrit :
 On Monday, 16 April 2012 at 06:37:00 UTC, Jacob Carlborg wrote:
 On 2012-04-16 04:35, Ary Manzana wrote:
 On 4/13/12 9:10 PM, deadalnix wrote:
 SDC have a lot of theses, and I proposed a similar stuff 
 for its
 evolution. I think it is easier for SDC than it is for dmd 
 considering
 the codebase of both.

 Cool! SDC is the way to go. Let's focus our efforts on that 
 project. :-)

 I noticed that SDC is licensed under the GPL license :( That 
 is not
 compatible with, for example, EPL used by Eclipse.

 We are changing the license soon, with BSD/MIT in mind. I am 
 really just
 waiting for Bernard to make the change, we've gotten 
 permission from all
 contributors as far as I know.

 You didn't asked me, but now you have mine. (Anyway, I'm not a 
 big contributor, and only have submitted some small patches).

I recall mentioning it via IRC. Which is hardly going to stand up
in a court. :V

Somedude <lovelydear mailmetrash.com> writes:

Le 16/04/2012 13:58, Bernard Helyer a écrit :
 
 I recall mentioning it via IRC. Which is hardly going to stand up
 in a court. :V
 

And given the ongoing Oracle-Google suit over the Java API license, this
has to be taken seriously, I'm affraid....

"Roman D. Boiko" <rb d-coding.com> writes:

On Thursday, 19 April 2012 at 11:01:10 UTC, Somedude wrote:
 Le 16/04/2012 13:58, Bernard Helyer a écrit :
 
 I recall mentioning it via IRC. Which is hardly going to stand 
 up
 in a court. :V
 

 And given the ongoing Oracle-Google suit over the Java API 
 license, this
 has to be taken seriously, I'm affraid....

Definitely. There are actually many stories to learn from. E.g.: 
http://en.wikipedia.org/wiki/SCO_v._IBM (and more at 
http://en.wikipedia.org/wiki/SCO_Group), 
http://blogs.computerworlduk.com/open-enterprise/2010/09/could-this-lawsuit-undermine-the-gnu-gpl/index.htm

"Bernard Helyer" <b.helyer gmail.com> writes:

On Monday, 16 April 2012 at 07:00:19 UTC, Jakob Ovrum wrote:
 On Monday, 16 April 2012 at 06:37:00 UTC, Jacob Carlborg wrote:
 On 2012-04-16 04:35, Ary Manzana wrote:
 On 4/13/12 9:10 PM, deadalnix wrote:
 SDC have a lot of theses, and I proposed a similar stuff for 
 its
 evolution. I think it is easier for SDC than it is for dmd 
 considering
 the codebase of both.

 Cool! SDC is the way to go. Let's focus our efforts on that 
 project. :-)

 I noticed that SDC is licensed under the GPL license :( That 
 is not compatible with, for example, EPL used by Eclipse.

 We are changing the license soon, with BSD/MIT in mind. I am 
 really just waiting for Bernard to make the change, we've 
 gotten permission from all contributors as far as I know.

Oh god, what did we decide on? Boost?

"Roman D. Boiko" <rb d-coding.com> writes:

On Monday, 16 April 2012 at 11:57:29 UTC, Bernard Helyer wrote:
 On Monday, 16 April 2012 at 07:00:19 UTC, Jakob Ovrum wrote:
 On Monday, 16 April 2012 at 06:37:00 UTC, Jacob Carlborg wrote:
 On 2012-04-16 04:35, Ary Manzana wrote:
 On 4/13/12 9:10 PM, deadalnix wrote:
 SDC have a lot of theses, and I proposed a similar stuff 
 for its
 evolution. I think it is easier for SDC than it is for dmd 
 considering
 the codebase of both.

 Cool! SDC is the way to go. Let's focus our efforts on that 
 project. :-)

 I noticed that SDC is licensed under the GPL license :( That 
 is not compatible with, for example, EPL used by Eclipse.

 We are changing the license soon, with BSD/MIT in mind. I am 
 really just waiting for Bernard to make the change, we've 
 gotten permission from all contributors as far as I know.

 Oh god, what did we decide on? Boost?

MIT is great, it is a huge step forward with respect to 
commercial reuse of code.

However, Boost would allow inclusion of sources (e.g., 
lexer/parser) into D standard library (I saw it has been 
mentioned somewhere that such functionality is planned, probably 
by Andrei Alexandrescu). Of course, before inclusion it might be 
rewritten significantly, but at least it could be based on SDC or 
whatever other implementation with Boost license.

What was the motivation behind selecting MIT?

"David Nadlinger" <see klickverbot.at> writes:

On Monday, 16 April 2012 at 11:57:29 UTC, Bernard Helyer wrote:
 Oh god, what did we decide on? Boost?

Imho, MIT is just fine for a compiler. It's not like a line of 
attribution is much to ask when you are getting an entire 
compiler fronend for free – in fact, applications should 
probably mention the compiler frontend they use in the docs 
anyway in case there are incompatibilities, …

David

"Nick Sabalausky" <SeeWebsiteToContactMe semitwist.com> writes:

"David Nadlinger" <see klickverbot.at> wrote in message 
news:voymctvtskltfzhslhkp forum.dlang.org...
 On Monday, 16 April 2012 at 11:57:29 UTC, Bernard Helyer wrote:
 Oh god, what did we decide on? Boost?

 Imho, MIT is just fine for a compiler. It's not like a line of attribution 
 is much to ask when you are getting an entire compiler fronend for free - 
 in fact, applications should probably mention the compiler frontend they 
 use in the docs anyway in case there are incompatibilities, .

There's no attribution in MIT. 

"Roman D. Boiko" <rb d-coding.com> writes:

On Thursday, 19 April 2012 at 09:58:52 UTC, Nick Sabalausky wrote:
 "David Nadlinger" <see klickverbot.at> wrote in message
 news:voymctvtskltfzhslhkp forum.dlang.org...
 On Monday, 16 April 2012 at 11:57:29 UTC, Bernard Helyer wrote:
 Oh god, what did we decide on? Boost?

 Imho, MIT is just fine for a compiler. It's not like a line of 
 attribution is much to ask when you are getting an entire 
 compiler fronend for free - in fact, applications should 
 probably mention the compiler frontend they use in the docs 
 anyway in case there are incompatibilities, .

 There's no attribution in MIT.

Yes, there is no attribution: 
https://github.com/roman-d-boiko/SDC/blob/master/LICENCE. It is 
very similar to Boost. Actually, I prefer Boost only because it 
is slightly more popular and because the D standard library uses 
it. I'm perfectly happy with MIT, though, and already cloned SDC 
:)

Just interested about motivation behind choosing MIT.

"Roman D. Boiko" <rb d-coding.com> writes:

On Thursday, 19 April 2012 at 10:15:36 UTC, Roman D. Boiko wrote:
 On Thursday, 19 April 2012 at 09:58:52 UTC, Nick Sabalausky 
 wrote:
 "David Nadlinger" <see klickverbot.at> wrote in message
 news:voymctvtskltfzhslhkp forum.dlang.org...
 On Monday, 16 April 2012 at 11:57:29 UTC, Bernard Helyer 
 wrote:
 Oh god, what did we decide on? Boost?

 Imho, MIT is just fine for a compiler. It's not like a line 
 of attribution is much to ask when you are getting an entire 
 compiler fronend for free - in fact, applications should 
 probably mention the compiler frontend they use in the docs 
 anyway in case there are incompatibilities, .

 There's no attribution in MIT.

 Yes, there is no attribution: 
 https://github.com/roman-d-boiko/SDC/blob/master/LICENCE. It is 
 very similar to Boost. Actually, I prefer Boost only because it 
 is slightly more popular and because the D standard library 
 uses it. I'm perfectly happy with MIT, though, and already 
 cloned SDC :)

 Just interested about motivation behind choosing MIT.

Motivation for Boost would be reducing the number of licenses 
that the code author must know. Also, here are some differences 
between these licenses, which I consider as Boost advantages:

"The Boost Software License is based upon the MIT license, but 
differs from the MIT license in that it:

(i) makes clear that licenses can be granted to organizations as 
well as individuals;

(ii) does not require that the license appear with executables or 
other binary uses of the library;

(iii) expressly disclaims -- on behalf of the author and 
copyright holders of the software only -- the warranty of title 
(a warranty that, under the Uniform Commercial Code, is separate 
from the warranty of non-infringement)

(iv) does not extend the disclaimer of warranties to licensees, 
so that they may, if they choose, undertake such warranties 
(e.g., in exchange for payment)."

http://ideas.opensource.org/ticket/45

"Nick Sabalausky" <SeeWebsiteToContactMe semitwist.com> writes:

"Roman D. Boiko" <rb d-coding.com> wrote in message 
news:gyhkcrhkaedsjzoohrvp forum.dlang.org...
 Motivation for Boost would be reducing the number of licenses that the 
 code author must know. Also, here are some differences between these 
 licenses, which I consider as Boost advantages:

MIT's *much* easier to understand though. Boost has some real goofy, 
obfuscated wordings. Although it's *worlds* better in that regard than the 
completely impenatrable GPL or Creative Commons.

 "The Boost Software License is based upon the MIT license, but differs 
 from the MIT license in that it:

 (i) makes clear that licenses can be granted to organizations as well as 
 individuals;

 (ii) does not require that the license appear with executables or other 
 binary uses of the library;

My favorite license, zlib ( http://www.opensource.org/licenses/Zlib ) 

MIT.

Plus it doesn't say anything like "to any person", so it should take care of 

It's a deterrent against corporations, which gives it a little bit of the 
benefit of the GPL, but without all the bullshit.)

 (iii) expressly disclaims -- on behalf of the author and copyright holders 
 of the software only -- the warranty of title (a warranty that, under the 
 Uniform Commercial Code, is separate from the warranty of 
 non-infringement)

 (iv) does not extend the disclaimer of warranties to licensees, so that 
 they may, if they choose, undertake such warranties (e.g., in exchange for 
 payment)."

 http://ideas.opensource.org/ticket/45 

"Roman D. Boiko" <rb d-coding.com> writes:

On Thursday, 19 April 2012 at 19:54:48 UTC, Nick Sabalausky wrote:
 MIT's *much* easier to understand though. Boost has some real 
 goofy, obfuscated wordings. Although it's *worlds* better in 
 that regard than the completely impenatrable GPL or Creative 
 Commons.

Comparing http://www.opensource.org/licenses/MIT and 
http://opensource.org/licenses/bsl1.0.html, I would not conclude 
that Boost is  more difficult to understand.
 My favorite license, zlib ( 
 http://www.opensource.org/licenses/Zlib )

 understand than
 MIT.

 Plus it doesn't say anything like "to any person", so it should 
 take care of

 that regard:
 It's a deterrent against corporations, which gives it a little 
 bit of the
 benefit of the GPL, but without all the bullshit.)

Here I do not agree either. Assuming that we would *want* such 
"benefit", MIT does not provide it.

As for zlib, it is very different from MIT/Boost, thus it is 
difficult to compare them (for non-lawyers).

License is a tool, and its up to SDC authors to select one. I 
just wanted to provide some information so that another option 
(Boost) is considered.

Jacob Carlborg <doob me.com> writes:

On 2012-04-19 21:54, Nick Sabalausky wrote:

 MIT's *much* easier to understand though. Boost has some real goofy,
 obfuscated wordings. Although it's *worlds* better in that regard than the
 completely impenatrable GPL or Creative Commons.

What I like about the Boost license is that it says explicitly that no 
attribution is needed for binary distribution.

The zlib/libpng license on the other hand says so implicitly.

-- 
/Jacob Carlborg

"David Nadlinger" <see klickverbot.at> writes:

On Thursday, 19 April 2012 at 10:15:36 UTC, Roman D. Boiko wrote:
 Actually, I prefer Boost only because it is slightly more 
 popular […]

Not to argue about the Boost license being popular in the D 
community, and not that the question would really matter, but 
what leads you to this general conclusion? I couldn't find any 
credible statistics on a quick Google search, but a numer of well 
known projects use the/a MIT license (X, Ruby on Rails, Mono, 
Lua, …).

David

"Roman D. Boiko" <rb d-coding.com> writes:

On Thursday, 19 April 2012 at 15:11:50 UTC, David Nadlinger wrote:
 On Thursday, 19 April 2012 at 10:15:36 UTC, Roman D. Boiko 
 wrote:
 Actually, I prefer Boost only because it is slightly more 
 popular […]

 Not to argue about the Boost license being popular in the D 
 community, and not that the question would really matter, but 
 what leads you to this general conclusion? I couldn't find any 
 credible statistics on a quick Google search, but a numer of 
 well known projects use the/a MIT license (X, Ruby on Rails, 
 Mono, Lua, …).

 David

I wish I could delete that post :) My claim is not based on any 
research.
However, I prefer Boost because:

"The Boost Software License is based upon the MIT license, but 
differs from the MIT license in that it:

(i) makes clear that licenses can be granted to organizations as 
well as individuals;

(ii) does not require that the license appear with executables or 
other binary uses of the library;

(iii) expressly disclaims -- on behalf of the author and 
copyright holders of the software only -- the warranty of title 
(a warranty that, under the Uniform Commercial Code, is separate 
from the warranty of non-infringement)

(iv) does not extend the disclaimer of warranties to licensees, 
so that they may, if they choose, undertake such warranties 
(e.g., in exchange for payment)."

http://ideas.opensource.org/ticket/45

deadalnix <deadalnix gmail.com> writes:

Le 19/04/2012 17:23, Roman D. Boiko a écrit :
 On Thursday, 19 April 2012 at 15:11:50 UTC, David Nadlinger wrote:
 On Thursday, 19 April 2012 at 10:15:36 UTC, Roman D. Boiko wrote:
 Actually, I prefer Boost only because it is slightly more popular […]

 Not to argue about the Boost license being popular in the D community,
 and not that the question would really matter, but what leads you to
 this general conclusion? I couldn't find any credible statistics on a
 quick Google search, but a numer of well known projects use the/a MIT
 license (X, Ruby on Rails, Mono, Lua, …).

 David

 I wish I could delete that post :) My claim is not based on any research.
 However, I prefer Boost because:

 "The Boost Software License is based upon the MIT license, but differs
 from the MIT license in that it:

 (i) makes clear that licenses can be granted to organizations as well as
 individuals;

 (ii) does not require that the license appear with executables or other
 binary uses of the library;

 (iii) expressly disclaims -- on behalf of the author and copyright
 holders of the software only -- the warranty of title (a warranty that,
 under the Uniform Commercial Code, is separate from the warranty of
 non-infringement)

 (iv) does not extend the disclaimer of warranties to licensees, so that
 they may, if they choose, undertake such warranties (e.g., in exchange
 for payment)."

 http://ideas.opensource.org/ticket/45

Very good point. Is it too late to change again ?

By the way, what is the status of the attribution clause ?

"Roman D. Boiko" <rb d-coding.com> writes:

On Thursday, 19 April 2012 at 15:30:21 UTC, deadalnix wrote:

 "The Boost Software License is based upon the MIT license, but 
 differs
 from the MIT license in that it:

 (i) makes clear that licenses can be granted to organizations 
 as well as
 individuals;

 (ii) does not require that the license appear with executables 
 or other
 binary uses of the library;

 (iii) expressly disclaims -- on behalf of the author and 
 copyright
 holders of the software only -- the warranty of title (a 
 warranty that,
 under the Uniform Commercial Code, is separate from the 
 warranty of
 non-infringement)

 (iv) does not extend the disclaimer of warranties to 
 licensees, so that
 they may, if they choose, undertake such warranties (e.g., in 
 exchange
 for payment)."

 http://ideas.opensource.org/ticket/45

 Very good point. Is it too late to change again ?

 By the way, what is the status of the attribution clause ?

If it will be decided to change the license, please pay attention 
to how should programmers apply the license to source and header 
files:

"Add a comment based on the following template, substituting 
appropriate text for the italicized portion:

//          Copyright Joe Coder 2004 - 2006.
// Distributed under the Boost Software License, Version 1.0.
//    (See accompanying file LICENSE_1_0.txt or copy at
//          http://www.boost.org/LICENSE_1_0.txt)
Please leave an empty line before and after the above comment 
block. It is fine if the copyright and license messages are not 
on different lines; in no case there should be other intervening 
text. Do not include "All rights reserved" anywhere.

Other ways of licensing source files have been considered, but 
some of them turned out to unintentionally nullify legal elements 
of the license. Having fixed language for referring to the 
license helps corporate legal departments evaluate the boost 
distribution. Creativity in license reference language is 
strongly discouraged, but judicious changes in the use of 
whitespace are fine."

http://www.boost.org/users/license.html

Alternatively, a license copy may be referenced at 
http://opensource.org/licenses/bsl1.0.html, which looks nicer. :)

Mirko Pilger <pilger cymotec.de> writes:

 If it will be decided to change the license, please pay attention to how
 should programmers apply the license to source and header files:

or do it the way it is done in phobos.

"David Nadlinger" <see klickverbot.at> writes:

On Thursday, 19 April 2012 at 09:58:52 UTC, Nick Sabalausky wrote:
 There's no attribution in MIT.

»The above copyright notice and this permission notice shall be 
included in all copies or substantial portions of the Software« 
– I am not a lawyer, but I what makes you so sure that binaries 
don't fall under this requirement as well?

The Boost people also seem to interpret MIT as requiring binary 
attribution; Dave Abrahams mentioned the binary clause in Boost 
as a difference to MIT in his OSI approval request at least ([1]).

David


[1] http://ideas.opensource.org/ticket/45

"Nick Sabalausky" <SeeWebsiteToContactMe semitwist.com> writes:

"David Nadlinger" <see klickverbot.at> wrote in message 
news:ebhaxqxtjuccyqfyapfo forum.dlang.org...
 On Thursday, 19 April 2012 at 09:58:52 UTC, Nick Sabalausky wrote:
 There's no attribution in MIT.

 »The above copyright notice and this permission notice shall be included 
 in all copies or substantial portions of the Software« - I am not a 
 lawyer, but I what makes you so sure that binaries don't fall under this 
 requirement as well?

 The Boost people also seem to interpret MIT as requiring binary 
 attribution; Dave Abrahams mentioned the binary clause in Boost as a 
 difference to MIT in his OSI approval request at least ([1]).

I'm pretty sure that "attribution" typically refers to the requirement that 
software X which *uses* software Y must include notice that software Y is 
being used.

But you're right that MIT doesn't make it clear whether binary-only 
redistributions of software Y must contain the license and copyright notice.

"Bernard Helyer" <b.helyer gmail.com> writes:

On Monday, 16 April 2012 at 06:37:00 UTC, Jacob Carlborg wrote:
 On 2012-04-16 04:35, Ary Manzana wrote:
 On 4/13/12 9:10 PM, deadalnix wrote:
 SDC have a lot of theses, and I proposed a similar stuff for 
 its
 evolution. I think it is easier for SDC than it is for dmd 
 considering
 the codebase of both.

 Cool! SDC is the way to go. Let's focus our efforts on that 
 project. :-)

 I noticed that SDC is licensed under the GPL license :( That is 
 not compatible with, for example, EPL used by Eclipse.

https://github.com/bhelyer/SDC/commit/2002ec27e8df9144f0d1051032abbfe24d2ad4d2

Go nuts.

Jacob Carlborg <doob me.com> writes:

On 2012-04-16 14:14, Bernard Helyer wrote:

 https://github.com/bhelyer/SDC/commit/2002ec27e8df9144f0d1051032abbfe24d2ad4d2


 Go nuts.

Awesome :)

-- 
/Jacob Carlborg

"Bernard Helyer" <b.helyer gmail.com> writes:

On Monday, 16 April 2012 at 02:34:29 UTC, Ary Manzana wrote:
 On 4/13/12 9:10 PM, deadalnix wrote:
 Le 13/04/2012 11:58, Ary Manzana a écrit :
 Having a D compiler available as a library will (at least) 
 give these
 benefits:

 1. Can be used by an IDE: D is statically typed and so an IDE 
 can
 benefit a lot from this. The features Descent had, as far as 
 I remember,
 were:
 1.1. Outline
 1.2. Autocompletion
 1.3. Type Hierarchy
 1.4. Syntax and semantic errors, showing not only the line 
 number but
 also column numbers if it makes sense
 1.5. Automatic import inclusion (say, typing writefln and 
 getting a list
 of modules that provide that symbol)
 1.6. Compile-time view: replace auto with the inferred type, 
 insert
 mixins into scope, rewrite operator overloads and other 
 lowerings (but
 I'm not sure this point is really useful)
 1.7. Determine, given a set of versions and flags, which 
 branches of
 static ifs are used/unused
 1.8. Open declaration
 1.9. Show implementations (of an interface, of interface's 
 method or,
 abstract methods, or method overrides).
 1.10. Propose to override a method (you type some letters and 
 then hit
 some key combination and get a list of methods to override)
 1.11. Get the code of a template when instantiated.
 2. Can be used to build better doc generators: one that shows 
 known
 subclasses or interface implementation, shows inherited 
 methods, type
 hierarchy.
 3. Can be used for lints and other such tools.

 As you can see, a simple lexer/parser built into an IDE, doc 
 generator
 or lint will just give basic features but will never achieve 
 something
 exceptionally good if it lacks the full semantic knowledge of 
 the code.

 I'll write a list of things I'd like this compiler-as-library 
 to have,
 but please help me make it bigger :-)

 * Don't use global variables (DMD is just thought to be run 
 once, so
 when used as a library it can just be used, well, once)
 * Provide a lexer which gives line numbers and column numbers
 (beginning, end)
 * Provide a parser with the same features
 * The semantic phase should not discard any information found 
 while
 parsing. For example when DMD resolves a type it recursively 
 resolves
 aliasing and keeps the last one. An example:

 alias int foo;
 alias foo* bar;

 bar something() { ... }

 It would be nice if "bar", after semantic analysis is done, 
 carries the
 information that bar is "foo*" and that "foo" is "int". Also 
 that
 something's return type is "bar", not "int*".
 * Provide errors and warnings that have line numbers as well 
 as column
 numbers.
 * Allow to parse the top-level definitions of a module. Whit 
 this I mean
 skipping function bodies. At least Descent first built a the 
 outline of
 the whole project by doing this. This mode should also allow 
 specifying
 a location as a target, and if that location falls inside a 
 function
 body then it's contents are returned (useful when editing a 
 file, so you
 can get the outline as well as semantic info of the function 
 currently
 being edited, which will never affect semantic in other parts 
 of the
 module). This will dramatically speed up the editor.
 * Don't stop parsing on errors (I think DMD already does 
 this).
 * Provide a visitor class. If possible, use visitors to 
 implement
 semantic analysis. The visitor will make it super easy to 
 implement
 lints and to generate documentation.

 SDC have a lot of theses, and I proposed a similar stuff for 
 its
 evolution. I think it is easier for SDC than it is for dmd 
 considering
 the codebase of both.

 Cool! SDC is the way to go. Let's focus our efforts on that 
 project. :-)

 One thing I saw in the code is that global variables are used 
 in it (specially in the sdc.aglobal module).

 Also, the lexer returns a TokenStream which contains the full 
 array of tokens. This is very slow compared to returning them 
 as they are scanned, when requested. But I guess this is easy 
 to fix as it's hidden behind the TokenStream interface.

 I tried to compile a simple file:

 void main() {}

 ./bin/sdc main.d

 core.exception.AssertError sdc.gen.sdcmodule(560): Assertion 
 failure
 ----------------
 5   sdc                                 0x000000010b87aa2a 
 _d_assertm + 42
 6   sdc                                 0x000000010b4e2daa void 
 sdc.gen.sdcmodule.__assert(int) + 26
 7   sdc                                 0x000000010b55d19c void 
 sdc.gen.sdcmodule.Store.addFunction(sdc.gen.sdcfunction.Function) 
 + 92
 8   sdc                                 0x000000010b55d75b void 
 sdc.gen.sdcmodule.Scope.add(immutable(char)[], 
 sdc.gen.sdcfunction.Function) + 123
 9   sdc                                 0x000000010b54ee36 void 
 sdc.gen.declaration.declareFunctionDeclaration(sdc.ast.declaration.
unctionDeclaration, 
 sdc.ast.sdcmodule.DeclarationDefinition, 
 sdc.gen.sdcmodule.Module) + 1278
 10  sdc                                 0x000000010b54e4b2 void 
 sdc.gen.declaration.declareDeclaration(sdc.ast.declaration.Declaration, 
 sdc.ast.sdcmodule.DeclarationDefinition, 
 sdc.gen.sdcmodule.Module) + 146
 11  sdc                                 0x000000010b54d12e void 
 sdc.gen.base.genDeclarationDefinition(sdc.ast.sdcmodule.DeclarationDefinition, 
 sdc.gen.sdcmodule.Module, ulong) + 374
 12  sdc                                 0x000000010b54c8d0 void 
 sdc.gen.base.resolveDeclarationDefinitionList(sdc.ast.sdcmodule.Decl
rationDefinition[], 
 sdc.gen.sdcmodule.Module, sdc.gen.type.Type) + 564
 13  sdc                                 0x000000010b54c439 
 sdc.gen.sdcmodule.Module 
 sdc.gen.base.genModule(sdc.ast.sdcmodule.Module, 
 sdc.aglobal.TranslationUnit) + 157
 14  sdc                                 0x000000010b53432d int 
 sdc.sdc.realmain(immutable(char)[][]) + 3081
 15  sdc                                 0x000000010b4e283e 
 _Dmain + 54
 16  sdc                                 0x000000010b87b415 
 extern (C) int rt.dmain2.main(int, char**).void runMain() + 29
 17  sdc                                 0x000000010b87adc5 
 extern (C) int rt.dmain2.main(int, char**).void tryExec(scope 
 void delegate()) + 45
 18  sdc                                 0x000000010b87b467 
 extern (C) int rt.dmain2.main(int, char**).void runAll() + 63
 19  sdc                                 0x000000010b87adc5 
 extern (C) int rt.dmain2.main(int, char**).void tryExec(scope 
 void delegate()) + 45
 20  sdc                                 0x000000010b87ad48 main 
 + 232
 21  sdc                                 0x000000010b4e2694 
 start + 52
 22  ???                                 0x0000000000000002 0x0 
 + 2
 ----------------

https://github.com/bhelyer/SDC/issues/30

Rename main.d to something else and it should work. Gotta fix 
that. >_<

Jacob Carlborg <doob me.com> writes:

On 2012-04-13 11:58, Ary Manzana wrote:
 Having a D compiler available as a library will (at least) give these
 benefits:

 1. Can be used by an IDE: D is statically typed and so an IDE can
 benefit a lot from this. The features Descent had, as far as I remember,
 were:
 1.1. Outline
 1.2. Autocompletion
 1.3. Type Hierarchy
 1.4. Syntax and semantic errors, showing not only the line number but
 also column numbers if it makes sense
 1.5. Automatic import inclusion (say, typing writefln and getting a list
 of modules that provide that symbol)
 1.6. Compile-time view: replace auto with the inferred type, insert
 mixins into scope, rewrite operator overloads and other lowerings (but
 I'm not sure this point is really useful)

Sure it is, it's very usable.

 1.7. Determine, given a set of versions and flags, which branches of
 static ifs are used/unused
 1.8. Open declaration
 1.9. Show implementations (of an interface, of interface's method or,
 abstract methods, or method overrides).
 1.10. Propose to override a method (you type some letters and then hit
 some key combination and get a list of methods to override)
 1.11. Get the code of a template when instantiated.
 2. Can be used to build better doc generators: one that shows known
 subclasses or interface implementation, shows inherited methods, type
 hierarchy.
 3. Can be used for lints and other such tools.

 As you can see, a simple lexer/parser built into an IDE, doc generator
 or lint will just give basic features but will never achieve something
 exceptionally good if it lacks the full semantic knowledge of the code.

 I'll write a list of things I'd like this compiler-as-library to have,
 but please help me make it bigger :-)

* Show generated documentation for symbols both on hover and in the 
autocompletion list
* Show source for symbols
* Import organizer (or what to call it). Remove unused imports and add 
missing ones
* Code formatting
* Fix it/quick fix. Button for automatically fixing simple errors
* Syntax and semantic highlighting :)
* Refactoring

I can also think about a lot of features that is usable if you also have 
a GUI builder in the IDE.

 * Don't use global variables (DMD is just thought to be run once, so
 when used as a library it can just be used, well, once)
 * Provide a lexer which gives line numbers and column numbers
 (beginning, end)
 * Provide a parser with the same features
 * The semantic phase should not discard any information found while
 parsing. For example when DMD resolves a type it recursively resolves
 aliasing and keeps the last one. An example:

 alias int foo;
 alias foo* bar;

 bar something() { ... }

 It would be nice if "bar", after semantic analysis is done, carries the
 information that bar is "foo*" and that "foo" is "int". Also that
 something's return type is "bar", not "int*".
 * Provide errors and warnings that have line numbers as well as column
 numbers.
 * Allow to parse the top-level definitions of a module. Whit this I mean
 skipping function bodies. At least Descent first built a the outline of
 the whole project by doing this. This mode should also allow specifying
 a location as a target, and if that location falls inside a function
 body then it's contents are returned (useful when editing a file, so you
 can get the outline as well as semantic info of the function currently
 being edited, which will never affect semantic in other parts of the
 module). This will dramatically speed up the editor.
 * Don't stop parsing on errors (I think DMD already does this).
 * Provide a visitor class. If possible, use visitors to implement
 semantic analysis. The visitor will make it super easy to implement
 lints and to generate documentation.

This is a great start and as I've said elsewhere, this would be so cool 
and usable to have.

-- 
/Jacob Carlborg

"Nick Sabalausky" <SeeWebsiteToContactMe semitwist.com> writes:

"Jacob Carlborg" <doob me.com> wrote in message 
news:jm9ulg$ui3$1 digitalmars.com...
 * Refactoring

* Make DustMite even more awesomer by attempting to factor out functions, 
etc.

Marco Leise <Marco.Leise gmx.de> writes:

Am Sat, 14 Apr 2012 22:44:07 -0400
schrieb "Nick Sabalausky" <SeeWebsiteToContactMe semitwist.com>:

 "Jacob Carlborg" <doob me.com> wrote in message 
 news:jm9ulg$ui3$1 digitalmars.com...
 * Refactoring

 
 * Make DustMite even more awesomer by attempting to factor out functions, 
 etc.

Something internal: 

* Keep a view of every opened file in memory.

* Allow to manipulate that file with an interface like insert, delete,
  replace of character ranges.

* Reparse only what has changed through that interface (remember datetime.d?)
  This is to avoid CPU hogging and typing delays.

* Manage multiple open projects. That means sharing of the internal
  representation of druntime/Phobos and libraries.

* We probably want two parser interfaces. Thinking of XML: SAX and DOM.
  One parser iterates over the tokens and calls event handlers (no AST).
    Example: lazy (line-by-line) editor syntax highlighting.
  The other parser builds the whole module as an AST to work on.
    Example: editing code multiple times in an IDE/refactorings

* If necessary, some sort of transaction, so a refactoring that fails in 
  the middle of the operation doesn't leave the code in a broken state.

* DDoc awareness. Cross-linked symbols etc. must be included when that
  symbol is renamed.

I want refactoring to be as simple as Foo.renameSymbol("std.path.sep",
"std.path.dirSeperator"); if the connection between module- and filename allows
"std.path" to be traced back to the original file.

-- 
Marco

Jacob Carlborg <doob me.com> writes:

On 2012-04-18 14:49, Marco Leise wrote:

 I want refactoring to be as simple as Foo.renameSymbol("std.path.sep",
"std.path.dirSeperator"); if the connection between module- and filename allows
"std.path" to be traced back to the original file.

I'm not sure but I don't think that is enough. In Clang you do something 
like this:

1. Get cursor of source location
2. Get a unique global ID of the cursor the corresponds to the symbol 
(unified symbol resolution)
3. Walk the AST to find all matches of this ID
4. Get the source location of the cursors which match
5. Rename the symbol at the source location

-- 
/Jacob Carlborg

"Roman D. Boiko" <rb d-coding.com> writes:

On Wednesday, 18 April 2012 at 16:48:46 UTC, Jacob Carlborg wrote:
 On 2012-04-18 14:49, Marco Leise wrote:

 I want refactoring to be as simple as 
 Foo.renameSymbol("std.path.sep", "std.path.dirSeperator"); if 
 the connection between module- and filename allows "std.path" 
 to be traced back to the original file.

 I'm not sure but I don't think that is enough. In Clang you do 
 something like this:

 1. Get cursor of source location
 2. Get a unique global ID of the cursor the corresponds to the 
 symbol (unified symbol resolution)
 3. Walk the AST to find all matches of this ID
 4. Get the source location of the cursors which match
 5. Rename the symbol at the source location

Convenience interface on top of low-level core functionality can 
be provided and would suffice for many typical situations.

Jacob Carlborg <doob me.com> writes:

On 2012-04-19 10:48, Roman D. Boiko wrote:

 Convenience interface on top of low-level core functionality can be
 provided and would suffice for many typical situations.

In that case I think something like this might work:

Foo.renameSymbol("file.d:16:4", "bar");

Where "file.d" is the file in which the symbol is located. ":16:4" would 
be the row and column number. "bar" would be the new name of the symbol.

-- 
/Jacob Carlborg

Marco Leise <Marco.Leise gmx.de> writes:

Am Thu, 19 Apr 2012 11:24:34 +0200
schrieb Jacob Carlborg <doob me.com>:

 On 2012-04-19 10:48, Roman D. Boiko wrote:
 
 Convenience interface on top of low-level core functionality can be
 provided and would suffice for many typical situations.

 
 In that case I think something like this might work:
 
 Foo.renameSymbol("file.d:16:4", "bar");
 
 Where "file.d" is the file in which the symbol is located. ":16:4" would 
 be the row and column number. "bar" would be the new name of the symbol.

Doesn't the D module system make symbols unique? I thought with a set of
open/relevant files (call it a project if you will) a symbol name can be given
without ambiguities. This might go into bike shedding, but it looks like
dropping one of D's strengths here. It is of course still necessary to have
code to look up symbols under the cursor, no question.

-- 
Marco

"Roman D. Boiko" <rb d-coding.com> writes:

On Thursday, 19 April 2012 at 16:38:31 UTC, Marco Leise wrote:
 Am Thu, 19 Apr 2012 11:24:34 +0200
 schrieb Jacob Carlborg <doob me.com>:

 On 2012-04-19 10:48, Roman D. Boiko wrote:
 
 Convenience interface on top of low-level core functionality 
 can be
 provided and would suffice for many typical situations.

 
 In that case I think something like this might work:
 
 Foo.renameSymbol("file.d:16:4", "bar");
 
 Where "file.d" is the file in which the symbol is located. 
 ":16:4" would be the row and column number. "bar" would be the 
 new name of the symbol.

 Doesn't the D module system make symbols unique? I thought with 
 a set of open/relevant files (call it a project if you will) a 
 symbol name can be given without ambiguities. This might go 
 into bike shedding, but it looks like dropping one of D's 
 strengths here. It is of course still necessary to have code to 
 look up symbols under the cursor, no question.

I doubt specifying symbol it would be any better than location. 
For example, you would not be able to rename a local variable or 
any other symbol nested in a function.

Marco Leise <Marco.Leise gmx.de> writes:

Am Thu, 19 Apr 2012 18:58:26 +0200
schrieb "Roman D. Boiko" <rb d-coding.com>:

 I doubt specifying symbol it would be any better than location. 
 For example, you would not be able to rename a local variable or 
 any other symbol nested in a function.

Yes you can! Try to compile this:

	void x() {
		int i = 1;
		void y() {
			int i = 2;
			{
				int i = 3;
			}
		}
	}

Prints: Error: shadowing declaration x.y.i is deprecated
Local variables have unique names and creating duplicate names is deprecated
since a while.

-- 
Marco

Artur Skawina <art.08.09 gmail.com> writes:

On 04/21/12 15:32, Marco Leise wrote:
 Am Thu, 19 Apr 2012 18:58:26 +0200
 schrieb "Roman D. Boiko" <rb d-coding.com>:
 
 I doubt specifying symbol it would be any better than location. 
 For example, you would not be able to rename a local variable or 
 any other symbol nested in a function.

 
 Yes you can! Try to compile this:
 
 	void x() {
 		int i = 1;
 		void y() {
 			int i = 2;
 			{
 				int i = 3;
 			}
 		}
 	}
 
 Prints: Error: shadowing declaration x.y.i is deprecated
 Local variables have unique names and creating duplicate names is deprecated
since a while.

   void x() {
      int i = 1;
      {
         int i = 2;
         writeln(i);
      }
      writeln(i);
   }

artur

"Roman D. Boiko" <rb d-coding.com> writes:

On Saturday, 21 April 2012 at 13:32:28 UTC, Marco Leise wrote:
 Am Thu, 19 Apr 2012 18:58:26 +0200
 schrieb "Roman D. Boiko" <rb d-coding.com>:

 I doubt specifying symbol it would be any better than 
 location. For example, you would not be able to rename a local 
 variable or any other symbol nested in a function.

 Yes you can! Try to compile this:

 	void x() {
 		int i = 1;
 		void y() {
 			int i = 2;
 			{
 				int i = 3;
 			}
 		}
 	}

 Prints: Error: shadowing declaration x.y.i is deprecated
 Local variables have unique names and creating duplicate names 
 is deprecated since a while.

I guess x.y.i is just a convenience naming used by DMD, and it 
does not appear in specification.

Also, Artur Skawina provided a nice counter-example where it 
would not work.

Marco Leise <Marco.Leise gmx.de> writes:

I'll answer you both here...

Am Sat, 21 Apr 2012 16:11:02 +0200
schrieb Artur Skawina <art.08.09 gmail.com>:

 On 04/21/12 15:32, Marco Leise wrote:
 Am Thu, 19 Apr 2012 18:58:26 +0200
 schrieb "Roman D. Boiko" <rb d-coding.com>:
 
 I doubt specifying symbol it would be any better than location. 
 For example, you would not be able to rename a local variable or 
 any other symbol nested in a function.

 
 Yes you can! Try to compile this:
 
 	void x() {
 		int i = 1;
 		void y() {
 			int i = 2;
 			{
 				int i = 3;
 			}
 		}
 	}
 
 Prints: Error: shadowing declaration x.y.i is deprecated
 Local variables have unique names and creating duplicate names is deprecated
since a while.

 
    void x() {
       int i = 1;
       {
          int i = 2;
          writeln(i);
       }
       writeln(i);
    }
 
 artur

That gives the same error message as before for me. I'm using GDC with DMD
2.057 frontend. Maybe the behavoir has changed. :-(

Am Sat, 21 Apr 2012 16:28:15 +0200
schrieb "Roman D. Boiko" <rb d-coding.com>:

 On Saturday, 21 April 2012 at 13:32:28 UTC, Marco Leise wrote:
 Am Thu, 19 Apr 2012 18:58:26 +0200
 schrieb "Roman D. Boiko" <rb d-coding.com>:

 I doubt specifying symbol it would be any better than 
 location. For example, you would not be able to rename a local 
 variable or any other symbol nested in a function.

 Yes you can! Try to compile this:

 	void x() {
 		int i = 1;
 		void y() {
 			int i = 2;
 			{
 				int i = 3;
 			}
 		}
 	}

 Prints: Error: shadowing declaration x.y.i is deprecated
 Local variables have unique names and creating duplicate names 
 is deprecated since a while.

 
 I guess x.y.i is just a convenience naming used by DMD, and it 
 does not appear in specification.
 
 Also, Artur Skawina provided a nice counter-example where it 
 would not work.

I just returned from the toilet where I flushed down this idea, because it
doesn't work with overloads:

void x(int i) { ... }

void x(float i) { ... }

"x.i" is ambiguous to the refactoring tool. It would only work on the mangled
names and before I go into the depths of what to do with extern(C) and so on...
we can just go with the line/column lookup of symbols.

-- 
Marco

Jacob Carlborg <doob me.com> writes:

On 2012-04-21 15:32, Marco Leise wrote:
 Am Thu, 19 Apr 2012 18:58:26 +0200
 schrieb "Roman D. Boiko"<rb d-coding.com>:

 I doubt specifying symbol it would be any better than location.
 For example, you would not be able to rename a local variable or
 any other symbol nested in a function.

 Yes you can! Try to compile this:

 	void x() {
 		int i = 1;
 		void y() {
 			int i = 2;
 			{
 				int i = 3;
 			}
 		}
 	}

 Prints: Error: shadowing declaration x.y.i is deprecated
 Local variables have unique names and creating duplicate names is deprecated
since a while.

It's possible to have an instance variable and a local variable with the 
same name.

-- 
/Jacob Carlborg

"Roman D. Boiko" <rb d-coding.com> writes:

 It's possible to have an instance variable and a local variable 
 with the same name.

According to Marco's idea, those would be qualified differently: 
className.x vs className.funcName.x, but he recently provided an 
example where that is ambiguous.

Also it would be a problem for symbols declared inside anonimous 
members.

In general, I like symbol location ("file.d:50:7"). It should 
always be available to the tool which uses API.

Jacob Carlborg <doob me.com> writes:

On 2012-04-19 18:38, Marco Leise wrote:
 Am Thu, 19 Apr 2012 11:24:34 +0200
 schrieb Jacob Carlborg<doob me.com>:

 On 2012-04-19 10:48, Roman D. Boiko wrote:

 Convenience interface on top of low-level core functionality can be
 provided and would suffice for many typical situations.

 In that case I think something like this might work:

 Foo.renameSymbol("file.d:16:4", "bar");

 Where "file.d" is the file in which the symbol is located. ":16:4" would
 be the row and column number. "bar" would be the new name of the symbol.

 Doesn't the D module system make symbols unique? I thought with a set of
open/relevant files (call it a project if you will) a symbol name can be given
without ambiguities. This might go into bike shedding, but it looks like
dropping one of D's strengths here. It is of course still necessary to have
code to look up symbols under the cursor, no question.

Hmm, I guess so.

-- 
/Jacob Carlborg

"David Nadlinger" <see klickverbot.at> writes:

On Thursday, 19 April 2012 at 16:38:31 UTC, Marco Leise wrote:
 Doesn't the D module system make symbols unique?

---
void someFunction() {
   {
     int someVariable = 3;
     // ...
   }
   {
     int someVariable = 1000;
     // ...
   }
}
---

How do you target the someVariable symbol from the second block?

David

Ary Manzana <ary esperanto.org.ar> writes:

On 4/19/12 12:48 AM, Jacob Carlborg wrote:
 On 2012-04-18 14:49, Marco Leise wrote:

 I want refactoring to be as simple as Foo.renameSymbol("std.path.sep",
 "std.path.dirSeperator"); if the connection between module- and
 filename allows "std.path" to be traced back to the original file.

 I'm not sure but I don't think that is enough. In Clang you do something
 like this:

 1. Get cursor of source location
 2. Get a unique global ID of the cursor the corresponds to the symbol
 (unified symbol resolution)
 3. Walk the AST to find all matches of this ID
 4. Get the source location of the cursors which match
 5. Rename the symbol at the source location

Unfortunately rename can't be perfect in D because you can't apply it 
inside templates.

"Roman D. Boiko" <rb d-coding.com> writes:

On Thursday, 19 April 2012 at 11:04:20 UTC, Ary Manzana wrote:
 On 4/19/12 12:48 AM, Jacob Carlborg wrote:
 On 2012-04-18 14:49, Marco Leise wrote:

 I want refactoring to be as simple as 
 Foo.renameSymbol("std.path.sep",
 "std.path.dirSeperator"); if the connection between module- 
 and
 filename allows "std.path" to be traced back to the original 
 file.

 I'm not sure but I don't think that is enough. In Clang you do 
 something
 like this:

 1. Get cursor of source location
 2. Get a unique global ID of the cursor the corresponds to the 
 symbol
 (unified symbol resolution)
 3. Walk the AST to find all matches of this ID
 4. Get the source location of the cursors which match
 5. Rename the symbol at the source location

 Unfortunately rename can't be perfect in D because you can't 
 apply it inside templates.

In general, there is nothing preventing renaming in templates or 
mixins, if you do renaming after semantic analysis. However, 
there can be some troubles, e.g., if mixin string is generated on 
the fly from some function. Compiler error should suffice in this 
case, or even better, refactoring tool should give a warning.

"Roman D. Boiko" <rb d-coding.com> writes:

On Thursday, 19 April 2012 at 11:04:20 UTC, Ary Manzana wrote:
 On 4/19/12 12:48 AM, Jacob Carlborg wrote:
 On 2012-04-18 14:49, Marco Leise wrote:

 I want refactoring to be as simple as 
 Foo.renameSymbol("std.path.sep",
 "std.path.dirSeperator"); if the connection between module- 
 and
 filename allows "std.path" to be traced back to the original 
 file.

 I'm not sure but I don't think that is enough. In Clang you do 
 something
 like this:

 1. Get cursor of source location
 2. Get a unique global ID of the cursor the corresponds to the 
 symbol
 (unified symbol resolution)
 3. Walk the AST to find all matches of this ID
 4. Get the source location of the cursors which match
 5. Rename the symbol at the source location

 Unfortunately rename can't be perfect in D because you can't 
 apply it inside templates.

In general, there is nothing preventing renaming in templates or
mixins, if you do renaming after semantic analysis. However,
there can be some troubles, e.g., if mixin string is generated on
the fly from some function. Compiler error should suffice in this
case, or even better, refactoring tool should give a warning.

Ary Manzana <ary esperanto.org.ar> writes:

On 4/19/12 7:25 PM, Roman D. Boiko wrote:
 On Thursday, 19 April 2012 at 11:04:20 UTC, Ary Manzana wrote:
 On 4/19/12 12:48 AM, Jacob Carlborg wrote:
 On 2012-04-18 14:49, Marco Leise wrote:

 I want refactoring to be as simple as Foo.renameSymbol("std.path.sep",
 "std.path.dirSeperator"); if the connection between module- and
 filename allows "std.path" to be traced back to the original file.

 I'm not sure but I don't think that is enough. In Clang you do something
 like this:

 1. Get cursor of source location
 2. Get a unique global ID of the cursor the corresponds to the symbol
 (unified symbol resolution)
 3. Walk the AST to find all matches of this ID
 4. Get the source location of the cursors which match
 5. Rename the symbol at the source location

 Unfortunately rename can't be perfect in D because you can't apply it
 inside templates.

 In general, there is nothing preventing renaming in templates or
 mixins, if you do renaming after semantic analysis. However,
 there can be some troubles, e.g., if mixin string is generated on
 the fly from some function. Compiler error should suffice in this
 case, or even better, refactoring tool should give a warning.

T foo(T)(T x) {
   return x.something();
}

int something(int x) {
   return 1;
}

float something(float x) {
   return 1.0;
}

Now... go and rename the first function named "something". What do you 
do with "x.something()" inside the template... rename it or not?

"Tobias Pankrath" <tobias pankrath.net> writes:

On Thursday, 19 April 2012 at 12:16:05 UTC, Ary Manzana wrote:
 On 4/19/12 7:25 PM, Roman D. Boiko wrote:
 On Thursday, 19 April 2012 at 11:04:20 UTC, Ary Manzana wrote:
 On 4/19/12 12:48 AM, Jacob Carlborg wrote:
 On 2012-04-18 14:49, Marco Leise wrote:

 I want refactoring to be as simple as 
 Foo.renameSymbol("std.path.sep",
 "std.path.dirSeperator"); if the connection between module- 
 and
 filename allows "std.path" to be traced back to the 
 original file.

 I'm not sure but I don't think that is enough. In Clang you 
 do something
 like this:

 1. Get cursor of source location
 2. Get a unique global ID of the cursor the corresponds to 
 the symbol
 (unified symbol resolution)
 3. Walk the AST to find all matches of this ID
 4. Get the source location of the cursors which match
 5. Rename the symbol at the source location

 Unfortunately rename can't be perfect in D because you can't 
 apply it
 inside templates.

 In general, there is nothing preventing renaming in templates 
 or
 mixins, if you do renaming after semantic analysis. However,
 there can be some troubles, e.g., if mixin string is generated 
 on
 the fly from some function. Compiler error should suffice in 
 this
 case, or even better, refactoring tool should give a warning.

 T foo(T)(T x) {
   return x.something();
 }

 int something(int x) {
   return 1;
 }

 float something(float x) {
   return 1.0;
 }

 Now... go and rename the first function named "something". What 
 do you do with "x.something()" inside the template... rename it 
 or not?

1. Ask the user or
2. Check if there is an instance of foo!int which refers to 
something. If so
and there is no other instance, that refers to another something: 
rename it.
3. If there in another instance: Ask the user to rename one, all, 
none.

"Roman D. Boiko" <rb d-coding.com> writes:

On Thursday, 19 April 2012 at 12:31:24 UTC, Tobias Pankrath wrote:

 T foo(T)(T x) {
  return x.something();
 }

 int something(int x) {
  return 1;
 }

 float something(float x) {
  return 1.0;
 }

 Now... go and rename the first function named "something". 
 What do you do with "x.something()" inside the template... 
 rename it or not?

 1. Ask the user or
 2. Check if there is an instance of foo!int which refers to 
 something. If so
 and there is no other instance, that refers to another 
 something: rename it.
 3. If there in another instance: Ask the user to rename one, 
 all, none.

I would go ReSharper way: present user a tree with all 
occurrences of related symbols and checkboxes to include or 
exclude each. Or, in this particular case, display a list of 
affected types and checkboxes for each. This isn't VERY user 
friendly, though, but very flexible and reasonable. Give warnings 
if necessary.

Semantic analysis phase would provide information whether symbol 
is actually related, or only has the same name (but different 
semantics and scope).

"Roman D. Boiko" <rb d-coding.com> writes:

On Thursday, 19 April 2012 at 12:55:50 UTC, Roman D. Boiko wrote:
 On Thursday, 19 April 2012 at 12:31:24 UTC, Tobias Pankrath 
 wrote:

 T foo(T)(T x) {
 return x.something();
 }

 int something(int x) {
 return 1;
 }

 float something(float x) {
 return 1.0;
 }

 Now... go and rename the first function named "something". 
 What do you do with "x.something()" inside the template... 
 rename it or not?



 I would go ReSharper way: present user a tree with all 
 occurrences of related symbols and checkboxes to include or 
 exclude each. Or, in this particular case, display a list of 
 affected types and checkboxes for each. This isn't VERY user 
 friendly, though, but very flexible and reasonable. Give 
 warnings if necessary.

 Semantic analysis phase would provide information whether 
 symbol is actually related, or only has the same name (but 
 different semantics and scope).

Respective convenience API would simply provide overloads and 
reasonable defaults for similar cases.

My point is that there is nothing in language itself preventing 
such functionality, if we have semantic analysis. The problem is 
related to user intentions, and decision should be made by the 
user (but the tool should provide good defaults and enough 
information).

Jacob Carlborg <doob me.com> writes:

On 2012-04-19 13:05, Ary Manzana wrote:

 Unfortunately rename can't be perfect in D because you can't apply it
 inside templates.

Could you elaborate a bit on why this is?

-- 
/Jacob Carlborg

"Roman D. Boiko" <rb d-coding.com> writes:

On Thursday, 19 April 2012 at 14:46:26 UTC, Jacob Carlborg wrote:
 On 2012-04-19 13:05, Ary Manzana wrote:

 Unfortunately rename can't be perfect in D because you can't 
 apply it
 inside templates.

 Could you elaborate a bit on why this is?

Please see several last messages for this discussion.

Jacob Carlborg <doob me.com> writes:

On 2012-04-19 16:55, Roman D. Boiko wrote:
 On Thursday, 19 April 2012 at 14:46:26 UTC, Jacob Carlborg wrote:
 On 2012-04-19 13:05, Ary Manzana wrote:

 Unfortunately rename can't be perfect in D because you can't apply it
 inside templates.

 Could you elaborate a bit on why this is?

 Please see several last messages for this discussion.

Yeah, I saw that now, sorry.

-- 
/Jacob Carlborg

"David Nadlinger" <see klickverbot.at> writes:

On Thursday, 19 April 2012 at 14:46:26 UTC, Jacob Carlborg wrote:
 On 2012-04-19 13:05, Ary Manzana wrote:

 Unfortunately rename can't be perfect in D because you can't 
 apply it
 inside templates.

 Could you elaborate a bit on why this is?

A template can be instantiated in user code with parameters that 
can't be known at library compile/refactor time (or mixed into, 
again unknown, scopes), and this can OTOH lead to problems like:

---
int doSomething(int a) {
   return a;
}

auto foo(T)(T t) {
   // User renames doSomething(int) above – should this be
   // changed as well?
   return t.doSomething();
}

unittest {
   assert(foo(3) == 3);
}
---

The question is, though, whether »perfect« renaming support is 
really required, or if a defensive solution warning on potential 
ambiguities isn't already useful enough for most cases.

David

"Roman D. Boiko" <rb d-coding.com> writes:

On Thursday, 19 April 2012 at 14:55:50 UTC, David Nadlinger wrote:
 The question is, though, whether »perfect« renaming support 
 is really required, or if a defensive solution warning on 
 potential ambiguities isn't already useful enough for most 
 cases.

 David

"Perfect" support is impossible, I guess. For example, if a 
library has been released,  a rename would cause API breaking 
changes. But for code under control of user which is doing the 
raname it is possible to provide customization what to rename 
(ReSharper does that, for example).

Jacob Carlborg <doob me.com> writes:

On 2012-04-19 17:00, Roman D. Boiko wrote:
 On Thursday, 19 April 2012 at 14:55:50 UTC, David Nadlinger wrote:
 The question is, though, whether »perfect« renaming support is really
 required, or if a defensive solution warning on potential ambiguities
 isn't already useful enough for most cases.

 David

 "Perfect" support is impossible, I guess. For example, if a library has
 been released, a rename would cause API breaking changes. But for code
 under control of user which is doing the raname it is possible to
 provide customization what to rename (ReSharper does that, for example).

That's not up to the IDE to decide. If the user wants to break the API 
the IDE should let him/her.

-- 
/Jacob Carlborg

"Roman D. Boiko" <rb d-coding.com> writes:

 That's not up to the IDE to decide. If the user wants to break 
 the API the IDE should let him/her.

Yes, that's exactly my point. It is a matter of user goals, not a 
language issue, and decision should not be made by a tool instead 
of a user.

"Roman D. Boiko" <rb d-coding.com> writes:

On Friday, 13 April 2012 at 09:57:49 UTC, Ary Manzana wrote:
 Having a D compiler available as a library will (at least) give 
 these benefits:

   1. Can be used by an IDE: D is statically typed and so an IDE 
 can benefit a lot from this. The features Descent had, as far 
 as I remember, were:
     1.1. Outline
     1.2. Autocompletion
     1.3. Type Hierarchy
     1.4. Syntax and semantic errors, showing not only the line 
 number but also column numbers if it makes sense
     1.5. Automatic import inclusion (say, typing writefln and 
 getting a list of modules that provide that symbol)
     1.6. Compile-time view: replace auto with the inferred 
 type, insert mixins into scope, rewrite operator overloads and 
 other lowerings (but I'm not sure this point is really useful)
     1.7. Determine, given a set of versions and flags, which 
 branches of static ifs are used/unused
     1.8. Open declaration
     1.9. Show implementations (of an interface, of interface's 
 method or, abstract methods, or method overrides).
     1.10. Propose to override a method (you type some letters 
 and then hit some key combination and get a list of methods to 
 override)
     1.11. Get the code of a template when instantiated.
  2. Can be used to build better doc generators: one that shows 
 known subclasses or interface implementation, shows inherited 
 methods, type hierarchy.
  3. Can be used for lints and other such tools.

 As you can see, a simple lexer/parser built into an IDE, doc 
 generator or lint will just give basic features but will never 
 achieve something exceptionally good if it lacks the full 
 semantic knowledge of the code.

 I'll write a list of things I'd like this compiler-as-library 
 to have, but please help me make it bigger :-)

  * Don't use global variables (DMD is just thought to be run 
 once, so when used as a library it can just be used, well, once)
  * Provide a lexer which gives line numbers and column numbers 
 (beginning, end)
  * Provide a parser with the same features
  * The semantic phase should not discard any information found 
 while parsing. For example when DMD resolves a type it 
 recursively resolves aliasing and keeps the last one. An 
 example:

   alias int foo;
   alias foo* bar;

   bar something() { ... }

   It would be nice if "bar", after semantic analysis is done, 
 carries the information that bar is "foo*" and that "foo" is 
 "int". Also that something's return type is "bar", not "int*".
  * Provide errors and warnings that have line numbers as well 
 as column numbers.
  * Allow to parse the top-level definitions of a module. Whit 
 this I mean skipping function bodies. At least Descent first 
 built a the outline of the whole project by doing this. This 
 mode should also allow specifying a location as a target, and 
 if that location falls inside a function body then it's 
 contents are returned (useful when editing a file, so you can 
 get the outline as well as semantic info of the function 
 currently being edited, which will never affect semantic in 
 other parts of the module). This will dramatically speed up the 
 editor.
  * Don't stop parsing on errors (I think DMD already does this).
  * Provide a visitor class. If possible, use visitors to 
 implement semantic analysis. The visitor will make it super 
 easy to implement lints and to generate documentation.

By the way, I also started a project called <a 
href="https://github.com/roman-d-boiko/DCT">The D Compiler Tools 
(DCT)</a> about a month ago. It is provided under the Boost 
license, and has the goal to enable building third-party tools 
with functionality that would include described above. I'm trying 
to build LLVM-based codegen and also reuse frontend in a separate 
project with basic IDE functionality for D.

I have never implemented compilers before, and probably should 
have called my project SDC if only that name had not been taken 
before ;) Goals are very similar to those of SDC (especially now, 
after its re-licensing). But I don't commit to ever finish the 
project, because my free time is very limited :(.

There is SIGNIFICANTLY less functionality implemented at this 
moment than in SDC. Currently, only primitive lexing is in place 
(I follow the KISS principle where possible) and parsing of auto 
declarations (auto i = 3 * (2 + 8), etc.) with stubs for most 
other cases. (Please note that project Readme file is outdated.) 
Parser is top-down recursive descent, and it follows 
specification very closely, except some differences needed to 
simplify implementation (like using loops to implement 
left-recursion in specification).

Anyone interested in discussing DCT or participating in 
development would be welcome!

"Tobias Pankrath" <tobias pankrath.net> writes:

On Thursday, 19 April 2012 at 09:24:23 UTC, Roman D. Boiko wrote:
 On Friday, 13 April 2012 at 09:57:49 UTC, Ary Manzana wrote:
 Having a D compiler available as a library will (at least) 
 give these benefits:


What about joining forces with sdc then?

"Roman D. Boiko" <rb d-coding.com> writes:

On Thursday, 19 April 2012 at 10:28:08 UTC, Tobias Pankrath wrote:
 On Thursday, 19 April 2012 at 09:24:23 UTC, Roman D. Boiko 
 wrote:
 On Friday, 13 April 2012 at 09:57:49 UTC, Ary Manzana wrote:
 Having a D compiler available as a library will (at least) 
 give these benefits:


 What about joining forces with sdc then?

I think I will use some code from SDC and contibute back to it 
now that it is liberally licensed. It wasn't an option when it 
was under GNU, because I generally prefer possibility for 
commercial usage.

However, I believe that the project I'm working on (DCT) has a 
good potential and the D community will better benefit from two 
alternatives. There are several design differences, and will be 
more. This also gives me additional flexibility and more learning 
opportunities.

I will dual-license (Boost+MIT) any part of my code if somebody 
would like to incorporate such part in SDC. My question about 
motivation for MIT licence was caused in particular by desire to 
minimize possible overhead of dealing with two licenses for me.

"so" <so so.so> writes:

On Thursday, 19 April 2012 at 10:28:08 UTC, Tobias Pankrath wrote:
 On Thursday, 19 April 2012 at 09:24:23 UTC, Roman D. Boiko 
 wrote:
 On Friday, 13 April 2012 at 09:57:49 UTC, Ary Manzana wrote:
 Having a D compiler available as a library will (at least) 
 give these benefits:


 What about joining forces with sdc then?

Wow! Didn't know there was a D compiler with Boost like licence.
Awesome job whoever started it.