www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - dmd support for IDEs

reply Walter Bright <newshound1 digitalmars.com> writes:
In my discussions with companies about adopting D, the major barrier 
that comes up over and over isn't Tango vs Phobos, dmd being GPL, 
debugger support, libraries, bugs, etc., although those are important.

It's the IDE.

They say that the productivity gains of D's improvements are 
overbalanced by the loss of productivity by moving away from an IDE. And 
what is it about an IDE that is so productive? Intellisense (Microsoft's 
word for autocompletion).

So, while I'm not going to be writing an IDE, I figure that dmd can 
help. dmd already puts out .doc and .di files. How about putting out an 
xml file giving all the information needed for an IDE to implement 
autocompletion? There'd be one .xml file generated per .d source file.

The nice thing about an xml file is while D is relatively easy to parse, 
xml is trivial. Furthermore, an xml format would be fairly robust in the 
face of changes to D syntax.

What do you think?
Oct 10 2009
next sibling parent reply Jeremie Pelletier <jeremiep gmail.com> writes:
Walter Bright wrote:

 In my discussions with companies about adopting D, the major barrier 
 that comes up over and over isn't Tango vs Phobos, dmd being GPL, 
 debugger support, libraries, bugs, etc., although those are important.
 
 It's the IDE.
 
 They say that the productivity gains of D's improvements are 
 overbalanced by the loss of productivity by moving away from an IDE. And 
 what is it about an IDE that is so productive? Intellisense (Microsoft's 
 word for autocompletion).
 
 So, while I'm not going to be writing an IDE, I figure that dmd can 
 help. dmd already puts out .doc and .di files. How about putting out an 
 xml file giving all the information needed for an IDE to implement 
 autocompletion? There'd be one .xml file generated per .d source file.
 
 The nice thing about an xml file is while D is relatively easy to parse, 
 xml is trivial. Furthermore, an xml format would be fairly robust in the 
 face of changes to D syntax.
 
 What do you think?


I think it would be great, but XML is only one format and a heavy one at 
that, JSON for example is much lighter and easier to parse. It shouldn't 
be hard to support both.

However I would make the file generation optional, as the IDE might just 
want to read from the standard output stream of dmd instead, this would 
also be useful for shell scripts.

Support to get the semantics information of multiple files at once would 
also be neat, just like dmd can generate one object file from multiple 
source files.

Would it even be possible to have the C api behind the xml/json 
frontends exported in a dll, so IDEs could just dynamically link to it 
and call that API directly instead of parsing an intermediary text format.

Jeremie
Oct 10 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Jeremie Pelletier wrote:

 I think it would be great, but XML is only one format and a heavy one at 
 that, JSON for example is much lighter and easier to parse. It shouldn't 
 be hard to support both.


I'd never heard of JSON, but looking at it, it makes sense. I don't see 
much point in supporting both.


 However I would make the file generation optional, as the IDE might just 
 want to read from the standard output stream of dmd instead, this would 
 also be useful for shell scripts.


Yes, writing it to stdout as an option is a good idea.



 Support to get the semantics information of multiple files at once would 
 also be neat, just like dmd can generate one object file from multiple 
 source files.


Yes.


 Would it even be possible to have the C api behind the xml/json 
 frontends exported in a dll, so IDEs could just dynamically link to it 
 and call that API directly instead of parsing an intermediary text format.


I did that with the C++ compiler, and it's a big pain to support. I 
don't think it would be onerous to fork/exec the compiler to do the 
work, capture the output, and parse it.
Oct 10 2009
parent reply Jeremie Pelletier <jeremiep gmail.com> writes:
Walter Bright wrote:

 Jeremie Pelletier wrote:

 I think it would be great, but XML is only one format and a heavy one 
 at that, JSON for example is much lighter and easier to parse. It 
 shouldn't be hard to support both.


 
 I'd never heard of JSON, but looking at it, it makes sense. I don't see 
 much point in supporting both.


XML makes sense when saving as a file and it can be transformed by XSLT 
to generate formatted html documentation and whatnot, while JSON is 
lightweight and better suited for pipes between dmd and the IDE.


 However I would make the file generation optional, as the IDE might 
 just want to read from the standard output stream of dmd instead, this 
 would also be useful for shell scripts.


 
 Yes, writing it to stdout as an option is a good idea.
 
 

 Support to get the semantics information of multiple files at once 
 would also be neat, just like dmd can generate one object file from 
 multiple source files.


 
 Yes.
 

 Would it even be possible to have the C api behind the xml/json 
 frontends exported in a dll, so IDEs could just dynamically link to it 
 and call that API directly instead of parsing an intermediary text 
 format.


 
 I did that with the C++ compiler, and it's a big pain to support. I 
 don't think it would be onerous to fork/exec the compiler to do the 
 work, capture the output, and parse it.


The IDE usually keeps the files in memory and could therefore just call 
something like getSemantics(char** fileBuffers, int* fileSizes, int 
nFiles, ParseNode* parseTree) and have its parse nodes already allocated 
in process memory ready for use.

Considering a lot of IDEs like to re-parse the current file every time 
the keyboard is idle for a few seconds, this could really help 
performance, nothing is more annoying than an IDE that feels unresponsive.
Oct 10 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Jeremie Pelletier wrote:

 The IDE usually keeps the files in memory and could therefore just call 
 something like getSemantics(char** fileBuffers, int* fileSizes, int 
 nFiles, ParseNode* parseTree) and have its parse nodes already allocated 
 in process memory ready for use.
 
 Considering a lot of IDEs like to re-parse the current file every time 
 the keyboard is idle for a few seconds, this could really help 
 performance, nothing is more annoying than an IDE that feels unresponsive.


I understand and agree, but we are operating under severe manpower 
constraints. I don't have a 100 million dollar budget! (I'm sure MS 
spends more than that on VS.)

You're certainly welcome to take the compiler front end and try and make 
a dll out of it or integrate it directly into an IDE. But what I 
suggested would probably get a lot of results for a minimal investment 
in the front end and a minimal investment in existing IDEs.



My experience with making responsive interactive apps on slow machines 
suggests that using a multithreaded approach would make the IDE 
responsive even if the underlying parsing process is slow. What you do 
is, every time the source file changes, fire off a background thread at 
a low priority to reparse. If the source changes before it finishes, 
restart that thread. When the IDE actually needs the results, it uses 
the results of the most recently finished parse.

With this approach, there won't be any hangs where the keyboard is 
unresponsive.


Experience also suggests that using fork/exec rather than a shared dll 
approach is much more robust and easier to develop. The reason is that 
the former uses separate processes, which cannot step on each other. The 
latter puts everything in one process space, where you've got all the 
lovely, time-consuming, hair-pulling concurrency problems. The utter 
failure of the parse process also cannot bring down the IDE.
Oct 10 2009
next sibling parent Walter Bright <newshound1 digitalmars.com> writes:
Walter Bright wrote:

 Experience also suggests that using fork/exec rather than a shared dll 
 approach is much more robust and easier to develop. The reason is that 
 the former uses separate processes, which cannot step on each other. The 
 latter puts everything in one process space, where you've got all the 
 lovely, time-consuming, hair-pulling concurrency problems. The utter 
 failure of the parse process also cannot bring down the IDE.



In particular, if the compiler seg faults (does it ever do that? <g>) it 
won't stop the IDE.
Oct 10 2009
prev sibling next sibling parent =?ISO-8859-1?Q?=22J=E9r=F4me_M=2E_Berger=22?= <jeberger free.fr> writes:
Walter Bright wrote:

 Jeremie Pelletier wrote:

 The IDE usually keeps the files in memory and could therefore just=20
 call something like getSemantics(char** fileBuffers, int* fileSizes,=20
 int nFiles, ParseNode* parseTree) and have its parse nodes already=20
 allocated in process memory ready for use.

 Considering a lot of IDEs like to re-parse the current file every time=




=20

 the keyboard is idle for a few seconds, this could really help=20
 performance, nothing is more annoying than an IDE that feels=20
 unresponsive.


=20
 ...

 Experience also suggests that using fork/exec rather than a shared dll =



 approach is much more robust and easier to develop. The reason is that =



 the former uses separate processes, which cannot step on each other. Th=


e=20

 latter puts everything in one process space, where you've got all the=20
 lovely, time-consuming, hair-pulling concurrency problems. The utter=20
 failure of the parse process also cannot bring down the IDE.


	Plus, with a DLL you're contaminated by the GPL: the IDE *has* to=20
be GPL-compatible to use your DLL. With fork/exec there are no such=20
constraints...

		Jerome
--=20
mailto:jeberger free.fr
http://jeberger.free.fr
Jabber: jeberger jabber.fr
Oct 11 2009
prev sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2009-10-11 01:57:08 -0400, Walter Bright <newshound1 digitalmars.com> said:


 You're certainly welcome to take the compiler front end and try and 
 make a dll out of it or integrate it directly into an IDE. But what I 
 suggested would probably get a lot of results for a minimal investment 
 in the front end and a minimal investment in existing IDEs.


And I've already done so in D for Xcode (with an old version of DMD).

I had to change the error handling to throw exceptions on errors (no 
call to exit in my IDE please!). I also added some data to tokens to 
get their exact range in the file allowing me to use the DMD lexer for 
syntax highlighting. The semantic also did preserve that information 
and could tell you in what class, template, or function your insertion 
point was on a per-character basis.

And then I stopped there. This is a pain to maintain when DMD gets 
updated, so I didn't. It's buggy because if the compiler crashes, the 
IDE crashes too (keep in mind that parsing incomplete code every few 
seconds has a tendency to cause more crashes than regular compilation).

And finally, Xcode 3 came with a much better syntax definition format 
and a complete revamp of syntax highlighting that obsoleted half the 
integration work I did. So the next version of D for Xcode will get rid 
of DMDFE as an internal component and use Xcode's built-in machinery.

It's not clear to me how much getting supplementary data from the 
compiler could help. If I only get what I can see through Ddoc, it's 
only half useful. I can already parse and get character ranges for the 
the high-level constructs (classes, tempaltes, functions, etc.). What 
will be harder is matching each symbol in function code to the correct 
definition because that depends on the context of the function and 
doing autocompletion for what you type depending on what's available in 
a given context.



 Experience also suggests that using fork/exec rather than a shared dll 
 approach is much more robust and easier to develop. The reason is that 
 the former uses separate processes, which cannot step on each other. 
 The latter puts everything in one process space, where you've got all 
 the lovely, time-consuming, hair-pulling concurrency problems. The 
 utter failure of the parse process also cannot bring down the IDE.


Indeed, you don't want the compiler to crash your IDE.

Also, can DMD accept D files from stdin? That way files wouldn't need 
to be saved on disk on each keystroke.


-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Oct 11 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Michel Fortin wrote:

 It's not clear to me how much getting supplementary data from the 
 compiler could help. If I only get what I can see through Ddoc, it's 
 only half useful. I can already parse and get character ranges for the 
 the high-level constructs (classes, tempaltes, functions, etc.). What 
 will be harder is matching each symbol in function code to the correct 
 definition because that depends on the context of the function and doing 
 autocompletion for what you type depending on what's available in a 
 given context.


I agree it's only half useful. But I think it's still a big win over 
zero useful.



 Also, can DMD accept D files from stdin? That way files wouldn't need to 
 be saved on disk on each keystroke.


No, but it's a good idea.
Oct 11 2009
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2009-10-11 14:19:52 -0400, Walter Bright <newshound1 digitalmars.com> said:


 Michel Fortin wrote:

 It's not clear to me how much getting supplementary data from the 
 compiler could help. If I only get what I can see through Ddoc, it's 
 only half useful. I can already parse and get character ranges for the 
 the high-level constructs (classes, tempaltes, functions, etc.). What 
 will be harder is matching each symbol in function code to the correct 
 definition because that depends on the context of the function and 
 doing autocompletion for what you type depending on what's available in 
 a given context.


 
 I agree it's only half useful. But I think it's still a big win over 
 zero useful.


Indeed. And it may be a perfect fit for other tools such as a 
documentation system that can do cross-references across modules, or... 
hum perhaps even runtime reflection? :-)



 Also, can DMD accept D files from stdin? That way files wouldn't need 
 to be saved on disk on each keystroke.


 
 No, but it's a good idea.


Great.


-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Oct 11 2009
parent Walter Bright <newshound1 digitalmars.com> writes:
Michel Fortin wrote:

 Indeed. And it may be a perfect fit for other tools such as a 
 documentation system that can do cross-references across modules, or... 
 hum perhaps even runtime reflection? :-)


It's one of those things where there may be a lot of unanticipated uses 
for it!
Oct 11 2009
prev sibling next sibling parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
Walter Bright wrote:

 In my discussions with companies about adopting D, the major barrier
 that comes up over and over isn't Tango vs Phobos, dmd being GPL,
 debugger support, libraries, bugs, etc., although those are important.
 
 It's the IDE.
 
 They say that the productivity gains of D's improvements are
 overbalanced by the loss of productivity by moving away from an IDE. And
 what is it about an IDE that is so productive? Intellisense (Microsoft's
 word for autocompletion).
 
 So, while I'm not going to be writing an IDE, I figure that dmd can
 help. dmd already puts out .doc and .di files. How about putting out an
 xml file giving all the information needed for an IDE to implement
 autocompletion? There'd be one .xml file generated per .d source file.
 
 The nice thing about an xml file is while D is relatively easy to parse,
 xml is trivial. Furthermore, an xml format would be fairly robust in the
 face of changes to D syntax.
 
 What do you think?


Well, that's a better solution than reimplementing semantic analysis in
the ide. If you make it, I will stop trying to do the latter.

In the xml, will we see ct stuff and other transformations that DMD
performs on the source expanded?

[very very minor] concerns:

standardized? DMD derivatives will have it, what about hypothetical
other D implementations?

If your ide can't see or doesn't have compiler, it won't be able to do
much (erm duh)

All in all, I think it would be the bomb. I'd even volunteer to help
implementing it if I thought my code contributions would do less harm
than good.
Oct 10 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Ellery Newcomer wrote:

 Well, that's a better solution than reimplementing semantic analysis in
 the ide. If you make it, I will stop trying to do the latter.


That's what I wanted to hear.


 In the xml, will we see ct stuff and other transformations that DMD
 performs on the source expanded?


ct stuff? What is that? Won't see dmd operations on the source expanded. 
What you'll see is basically what ddoc generates, but in a machine 
readable format. I.e. you'll see function names, with their types, line 
number, comment, etc. Essentially what intellisense would pop up.


 [very very minor] concerns:
 
 standardized? DMD derivatives will have it, what about hypothetical
 other D implementations?


It would be part of the dmd front end, so all D compilers based on it 
would have it.


 If your ide can't see or doesn't have compiler, it won't be able to do
 much (erm duh)


Yeah, but without a compiler why edit D source? <g>


 All in all, I think it would be the bomb. I'd even volunteer to help
 implementing it if I thought my code contributions would do less harm
 than good.


I don't think it would be hard to implement.

But if you want to contribute, how about a JSON parser for phobos? 
You'll need one anyway for your IDE.

BTW, JSON parsing comes for free with javascript. Why not incorporate 
dmdscript into your IDE as its extension language?
Oct 10 2009
next sibling parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
Walter Bright wrote:

 Ellery Newcomer wrote:

 Well, that's a better solution than reimplementing semantic analysis in
 the ide. If you make it, I will stop trying to do the latter.




I made it to the symbol table ..



 

 In the xml, will we see ct stuff and other transformations that DMD
 performs on the source expanded?


 
 ct stuff? What is that? Won't see dmd operations on the source expanded.
 What you'll see is basically what ddoc generates, but in a machine
 readable format. I.e. you'll see function names, with their types, line
 number, comment, etc. Essentially what intellisense would pop up.
 


ctfe. compile time (weird connection?). what do string mixins evaluate
to? can I look at their result from the ide? what do templates expand
to? what does this here alias/typedef represent? what does this here
typeof expand to? what does this here c-style type normalize to (in
d-style)?

As for other transformations, it seemed like Ary had some neat tricks in
descent that showed things like int i; going to int i = 0; etc. maybe
wistful thinking.

while we're at it,

when I see a symbol, can I find its type? can I find every symbol that
would follow it in a dot list/exp?
when I see a symbol, can I find everywhere it's used?
when I see a scope, can I see every symbol that's in it?
when I see a module, can I find everywhere it's imported?
can I see exactly what symbols are pulled in? Can I perform analysis to
show me where those dang cyclic dependencies are?
when I see source code, can I perform a simple walk over the xml to
format it?


 It would be part of the dmd front end, so all D compilers based on it
 would have it.


How about the Intel D compiler? (It's going to happen. You know it will)


 
 Yeah, but without a compiler why edit D source? <g>
 


Weird users is the best answer I can offer. It happens.

Also, coming from the java IDEs, I'm feeling apprehensive about
integration on disparate platforms and whatnot. There's multiple ways
the compiler could not be there.


 All in all, I think it would be the bomb. I'd even volunteer to help
 implementing it if I thought my code contributions would do less harm
 than good.


 
 I don't think it would be hard to implement.
 
 But if you want to contribute, how about a JSON parser for phobos?
 You'll need one anyway for your IDE.
 
 BTW, JSON parsing comes for free with javascript. Why not incorporate
 dmdscript into your IDE as its extension language?


<g> sorry. my target is netbeans.

Although I could probably whip up something quick in ANTLR if I really
needed JSON in D.
Oct 10 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Ellery Newcomer wrote:

 ctfe. compile time (weird connection?). what do string mixins evaluate
 to?


No


 can I look at their result from the ide?


No


 what do templates expand
 to?


No


 what does this here alias/typedef represent?


Yes


 what does this here
 typeof expand to?


No


 what does this here c-style type normalize to (in
 d-style)?


No


 As for other transformations, it seemed like Ary had some neat tricks in
 descent that showed things like int i; going to int i = 0; etc. maybe
 wistful thinking.
 
 while we're at it,
 
 when I see a symbol, can I find its type?


Yes


 can I find every symbol that
 would follow it in a dot list/exp?


Yes


 when I see a symbol, can I find everywhere it's used?


No, but could be added


 when I see a scope, can I see every symbol that's in it?


Yes


 when I see a module, can I find everywhere it's imported?


Yes


 can I see exactly what symbols are pulled in?


No, but could be added


 Can I perform analysis to
 show me where those dang cyclic dependencies are?


Don't know


 when I see source code, can I perform a simple walk over the xml to
 format it?


No


Think of what it provides as very similar to what ddoc does, except that 
instead of being in a human-readable format it would be a 
machine-readable one.

In other words, for each module you'll be able to get

. all the symbols in that module, and the members of those symbols 
(recursively)
. the file/line of the source location of each symbol
. the ddoc comment for each symbol
. the type of each symbol

Things could be added over time, I was just thinking of this for starters.
Oct 10 2009
next sibling parent Lutger <lutger.blijdestijn gmail.com> writes:
Walter Bright wrote:

...

 
 Think of what it provides as very similar to what ddoc does, except that
 instead of being in a human-readable format it would be a
 machine-readable one.
 
 In other words, for each module you'll be able to get
 
 . all the symbols in that module, and the members of those symbols
 (recursively)
 . the file/line of the source location of each symbol
 . the ddoc comment for each symbol
 . the type of each symbol
 
 Things could be added over time, I was just thinking of this for starters.


What about file/line/column of the symbol? Is this much work / hard work to 
add?
Oct 11 2009
prev sibling next sibling parent reply =?ISO-8859-1?Q?=22J=E9r=F4me_M=2E_Berger=22?= <jeberger free.fr> writes:
Walter Bright wrote:

 Think of what it provides as very similar to what ddoc does, except tha=


t=20

 instead of being in a human-readable format it would be a=20
 machine-readable one.
=20
 In other words, for each module you'll be able to get
=20
 . all the symbols in that module, and the members of those symbols=20
 (recursively)


	Including local variables for functions?

		Jerome
--=20
mailto:jeberger free.fr
http://jeberger.free.fr
Jabber: jeberger jabber.fr
Oct 11 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Jérôme M. Berger wrote:

 . all the symbols in that module, and the members of those symbols 
 (recursively)


 
     Including local variables for functions?


That seems pointless, as they'll be inaccessible outside of the scope of 
the function.
Oct 11 2009
next sibling parent "Nick Sabalausky" <a a.a> writes:
"Walter Bright" <newshound1 digitalmars.com> wrote in message 
news:has947$1vum$2 digitalmars.com...

 Jérôme M. Berger wrote:

 . all the symbols in that module, and the members of those symbols 
 (recursively)


     Including local variables for functions?


 That seems pointless, as they'll be inaccessible outside of the scope of 
 the function.


void foo()
{
   int bar;
   // big complex func here
}
User: Ok, IDE, Refactor->Rename->foo's "bar" to "baz"
Oct 11 2009
prev sibling next sibling parent =?ISO-8859-1?Q?=22J=E9r=F4me_M=2E_Berger=22?= <jeberger free.fr> writes:
Walter Bright wrote:

 J=E9r=F4me M. Berger wrote:

 . all the symbols in that module, and the members of those symbols=20
 (recursively)


     Including local variables for functions?


=20
 That seems pointless, as they'll be inaccessible outside of the scope o=


f=20

 the function.


	The point would be for smart autocompletion: type the beginning of=20
a variable name, hit the autocomplete key and the IDE offers you a=20
choice of the names available at that point, i.e:
  - Global symbols (including imported ones);
  - Class members if the function is a method;
  - Local variables.

	It's also useful for member completion: type "foo." and the IDE=20
offers you a list of members for foo. This requires knowing its=20
class even if it is a local...

		Jerome
--=20
mailto:jeberger free.fr
http://jeberger.free.fr
Jabber: jeberger jabber.fr
Oct 11 2009
prev sibling parent BCS <none anon.com> writes:
Hello Walter,


 Jérôme M. Berger wrote:
 

 . all the symbols in that module, and the members of those symbols
 (recursively)
 


 Including local variables for functions?
 


 That seems pointless, as they'll be inaccessible outside of the scope
 of the function.
 


Good auto compleat needs a list of *Every* allowed identifier at any point 
in the code. Basicly the full symbol table.
Oct 12 2009
prev sibling next sibling parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
There are several core features that I absolutely must have (because I
want them):

1. autocompletion. everywhere. for everything. including local
variables. And that's not just the *pop* this symbol is a <type> that
does <ddoc>. It includes the completion part. When I type foreach
Ctrl+Space, I get a generic foreach statement (which is consistent with
the current compiler's syntax).

2. usage finding. I'm at a symbol. I want to know everywhere it's used.
If it's a class, I want to know everywhere it's subclassed.

3. code formatting.

4. build system

5. syntax highlight and code folding (easy peasy, but necessary)

6. error reporting aka dem shquiggly red lines

7. import resolution.

On top of these are a bunch of peripheral features:
When you have 1, code navigation should come pretty easy.
When you have 2, renaming any variable should come pretty easy.

When I have problems with those dang cyclic dependencies, I might want
to be able to merge two modules together.

etc


What I wanted out of my semantic analysis was a complete syntax tree and
symbol table that could make all of these things possible. If I can get
this direct from the compiler, it would obviously save me several years
of work and "yeah, this doesn't even try to be compatible" problems.


Walter Bright wrote:

 Ellery Newcomer wrote:

 ctfe. compile time (weird connection?). what do string mixins evaluate
 to?


 
 No
 

 can I look at their result from the ide?


 
 No


Well I want this feature.


 

 what do templates expand
 to?


 
 No
 

 what does this here alias/typedef represent?


 
 Yes
 


Good.


 what does this here
 typeof expand to?


 
 No
 

 what does this here c-style type normalize to (in
 d-style)?


 
 No
 


Well I want this feature.



 As for other transformations, it seemed like Ary had some neat tricks in
 descent that showed things like int i; going to int i = 0; etc. maybe
 wistful thinking.

 while we're at it,

 when I see a symbol, can I find its type?


 
 Yes


Good.


 

 can I find every symbol that
 would follow it in a dot list/exp?


 
 Yes


Good.


 

 when I see a symbol, can I find everywhere it's used?


 
 No, but could be added
 


Good.


 when I see a scope, can I see every symbol that's in it?


 
 Yes
 


Good.


 when I see a module, can I find everywhere it's imported?


 
 Yes


Good.


 

 can I see exactly what symbols are pulled in?


 
 No, but could be added


Good.


 

 Can I perform analysis to
 show me where those dang cyclic dependencies are?


 
 Don't know
 


If I have the previous two I think I can do it. I might need a view of
the code after compile time expansions. Oh, and I'd obviously need to
see which modules have static ctors/dtors.


 when I see source code, can I perform a simple walk over the xml to
 format it?


 
 No
 
 
 Think of what it provides as very similar to what ddoc does, except that
 instead of being in a human-readable format it would be a
 machine-readable one.


Yep, we're thinking of different things.


 
 In other words, for each module you'll be able to get
 
 . all the symbols in that module, and the members of those symbols
 (recursively)
 . the file/line of the source location of each symbol
 . the ddoc comment for each symbol
 . the type of each symbol
 
 Things could be added over time, I was just thinking of this for starters.
Oct 11 2009
prev sibling parent reply BCS <none anon.com> writes:
Hello Walter,


 
 Think of what it provides as very similar to what ddoc does, except
 that instead of being in a human-readable format it would be a
 machine-readable one.


If that's what you are planning, I can tell you right now it's not enough. 
Much of the things IDEs need is not present in ddoc. Just off hand, I'd say 

than ddoc has. What would be better is a dump of a fully annotated AST from 
just before it's passed off for optimization and code gen.
Oct 12 2009
parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
BCS wrote:

 Hello Walter,
 
 What would be better is a dump of a
 fully annotated AST from just before it's passed off for optimization
 and code gen.
 
 


To reiterate my offer,

I would be willing to implement this in DMD provided I can then assume
that any modern D compiler will have it.
Oct 12 2009
prev sibling next sibling parent reply Jeremie Pelletier <jeremiep gmail.com> writes:
Walter Bright wrote:

 But if you want to contribute, how about a JSON parser for phobos? 
 You'll need one anyway for your IDE.
 
 BTW, JSON parsing comes for free with javascript. Why not incorporate 
 dmdscript into your IDE as its extension language?


The official JSON website has tons of bindings, here's the C one:

http://fara.cs.uni-potsdam.de/~jsg/json_parser/

I'm gonna try and get it converted to D over the weekend.
Oct 10 2009
next sibling parent Walter Bright <newshound1 digitalmars.com> writes:
Jeremie Pelletier wrote:

 The official JSON website has tons of bindings, here's the C one:
 
 http://fara.cs.uni-potsdam.de/~jsg/json_parser/
 
 I'm gonna try and get it converted to D over the weekend.


It has a test suite with it!
Oct 10 2009
prev sibling next sibling parent reply Christopher Wright <dhasenan gmail.com> writes:
Jeremie Pelletier wrote:

 Walter Bright wrote:

 But if you want to contribute, how about a JSON parser for phobos? 
 You'll need one anyway for your IDE.

 BTW, JSON parsing comes for free with javascript. Why not incorporate 
 dmdscript into your IDE as its extension language?


 
 The official JSON website has tons of bindings, here's the C one:
 
 http://fara.cs.uni-potsdam.de/~jsg/json_parser/
 
 I'm gonna try and get it converted to D over the weekend.


Tango already has a good JSON parser, but I imagine its license (BSD) 
doesn't meet Walter's requirements.
Oct 11 2009
parent reply Moritz Warning <moritzwarning web.de> writes:
On Sun, 11 Oct 2009 09:04:50 -0400, Christopher Wright wrote:


 Jeremie Pelletier wrote:

 Walter Bright wrote:

 But if you want to contribute, how about a JSON parser for phobos?
 You'll need one anyway for your IDE.

 BTW, JSON parsing comes for free with javascript. Why not incorporate
 dmdscript into your IDE as its extension language?


 
 The official JSON website has tons of bindings, here's the C one:
 
 http://fara.cs.uni-potsdam.de/~jsg/json_parser/
 
 I'm gonna try and get it converted to D over the weekend.


 
 Tango already has a good JSON parser, but I imagine its license (BSD)
 doesn't meet Walter's requirements.


I wrote a JSON parser that is Public Domain and also very fast.
It may be helpful. 
http://dsource.org/projects/tango/ticket/1491

There are also a few others around.
Oct 11 2009
next sibling parent Moritz Warning <moritzwarning web.de> writes:
On Sun, 11 Oct 2009 21:25:57 +0000, Moritz Warning wrote:


 On Sun, 11 Oct 2009 09:04:50 -0400, Christopher Wright wrote:
 

 Jeremie Pelletier wrote:

 Walter Bright wrote:

 But if you want to contribute, how about a JSON parser for phobos?
 You'll need one anyway for your IDE.

 BTW, JSON parsing comes for free with javascript. Why not incorporate
 dmdscript into your IDE as its extension language?


 
 The official JSON website has tons of bindings, here's the C one:
 
 http://fara.cs.uni-potsdam.de/~jsg/json_parser/
 
 I'm gonna try and get it converted to D over the weekend.


 
 Tango already has a good JSON parser, but I imagine its license (BSD)
 doesn't meet Walter's requirements.


 
 I wrote a JSON parser that is Public Domain and also very fast. It may
 be helpful.
 http://dsource.org/projects/tango/ticket/1491
 
 There are also a few others around.


Uh, it says BSD. But I'm fine with PB.
Oct 11 2009
prev sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Moritz Warning wrote:

 I wrote a JSON parser that is Public Domain and also very fast.
 It may be helpful. 
 http://dsource.org/projects/tango/ticket/1491



You need to get this added to the list at http://json.org/
Oct 11 2009
parent Moritz Warning <moritzwarning web.de> writes:
On Sun, 11 Oct 2009 14:58:06 -0700, Walter Bright wrote:


 Moritz Warning wrote:

 I wrote a JSON parser that is Public Domain and also very fast. It may
 be helpful.
 http://dsource.org/projects/tango/ticket/1491


 
 
 You need to get this added to the list at http://json.org/


It would need to write a replacement for the un-/escape routines from 
Tango first.
But so far it sounds like a good idea.
Oct 11 2009
prev sibling parent reply Jeremie Pelletier <jeremiep gmail.com> writes:
Jeremie Pelletier wrote:

 Walter Bright wrote:

 But if you want to contribute, how about a JSON parser for phobos? 
 You'll need one anyway for your IDE.

 BTW, JSON parsing comes for free with javascript. Why not incorporate 
 dmdscript into your IDE as its extension language?


 
 The official JSON website has tons of bindings, here's the C one:
 
 http://fara.cs.uni-potsdam.de/~jsg/json_parser/
 
 I'm gonna try and get it converted to D over the weekend.


After some digging into the C sources, I decided I didn't like the way 
they did it. I wanted the simplicity and elegance of JSON I use with 
jQuery ($.toJSON and $.parseJSON). I looked again at the simple graphs 
on json.org and decided it was simple enough to write a D parser from 
scratch, here's what I have after a few hours of work, both methods 
works, although I didn't thoroughly tested them yet.

http://pastebin.com/f25b67726

Let me know what you think and feel free to add it to phobos if you like 
it :)

Jeremie
Oct 11 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Jeremie Pelletier wrote:

 Jeremie Pelletier wrote:

 Walter Bright wrote:

 But if you want to contribute, how about a JSON parser for phobos? 
 You'll need one anyway for your IDE.

 BTW, JSON parsing comes for free with javascript. Why not incorporate 
 dmdscript into your IDE as its extension language?


 The official JSON website has tons of bindings, here's the C one:

 http://fara.cs.uni-potsdam.de/~jsg/json_parser/

 I'm gonna try and get it converted to D over the weekend.


 
 After some digging into the C sources, I decided I didn't like the way 
 they did it. I wanted the simplicity and elegance of JSON I use with 
 jQuery ($.toJSON and $.parseJSON). I looked again at the simple graphs 
 on json.org and decided it was simple enough to write a D parser from 
 scratch, here's what I have after a few hours of work, both methods 
 works, although I didn't thoroughly tested them yet.
 
 http://pastebin.com/f25b67726
 
 Let me know what you think and feel free to add it to phobos if you like 
 it :)
 
 Jeremie


Looks nice! But it needs to be ported to Phobos, and needs unit tests!
Oct 11 2009
parent Jeremie Pelletier <jeremiep gmail.com> writes:
Walter Bright wrote:

 Jeremie Pelletier wrote:

 Jeremie Pelletier wrote:

 Walter Bright wrote:

 But if you want to contribute, how about a JSON parser for phobos? 
 You'll need one anyway for your IDE.

 BTW, JSON parsing comes for free with javascript. Why not 
 incorporate dmdscript into your IDE as its extension language?


 The official JSON website has tons of bindings, here's the C one:

 http://fara.cs.uni-potsdam.de/~jsg/json_parser/

 I'm gonna try and get it converted to D over the weekend.


 After some digging into the C sources, I decided I didn't like the way 
 they did it. I wanted the simplicity and elegance of JSON I use with 
 jQuery ($.toJSON and $.parseJSON). I looked again at the simple graphs 
 on json.org and decided it was simple enough to write a D parser from 
 scratch, here's what I have after a few hours of work, both methods 
 works, although I didn't thoroughly tested them yet.

 http://pastebin.com/f25b67726

 Let me know what you think and feel free to add it to phobos if you 
 like it :)

 Jeremie


 
 Looks nice! But it needs to be ported to Phobos, and needs unit tests!


Here you go: http://pastebin.com/f64b75abb

Compiled on dmd 2.033 against phobos with a unittest and it works great :)

I also needed to port my unicode character handling module to phobos 
because the one currently used doesn't have a isUniControl() method:

http://pastebin.com/fe47f274

Jeremie
Oct 11 2009
prev sibling parent reply BCS <none anon.com> writes:
Hello Walter,


 Ellery Newcomer wrote:
 

 In the xml, will we see ct stuff and other transformations that DMD
 performs on the source expanded?
 


 ct stuff? What is that? Won't see dmd operations on the source
 expanded. What you'll see is basically what ddoc generates, but in a
 machine readable format. I.e. you'll see function names, with their
 types, line number, comment, etc. Essentially what intellisense would
 pop up.
 


template Foo (T) { T map(T[]) { ... } }

class Bar { mixin Foo!(int); }

void main()
{
   Bar bar;
   bar. /// will the IDE have to do anything special to see 'map' or will 
it just be listed as another function?
}


 If your ide can't see or doesn't have compiler, it won't be able to
 do much (erm duh)
 


 Yeah, but without a compiler why edit D source? <g>
 


My normal setup is running eclipse on a windows box to edit files on a linux 
box (via file share) where I compile things (I need some other stuff from 
linux). For this, I have no need for a compiler on the box with the IDE unless 
the IDE needs it solely for the auto-complete.
Oct 12 2009
parent Christopher Wright <dhasenan gmail.com> writes:
BCS wrote:

 template Foo (T) { T map(T[]) { ... } }
 
 class Bar { mixin Foo!(int); }
 
 void main()
 {
   Bar bar;
   bar. /// will the IDE have to do anything special to see 'map' or will 
 it just be listed as another function?
 }


Probably it would just work.

On the other hand, if you had this:

T manipulate(T)(T value) { ... }
MyObject foo;
manipulate(foo). // IDE dies here

The compiler wouldn't have completion on the line of code you're 
currently editing, since it's a syntax error. The IDE won't know 
anything about D and so can't determine what the return type of 
manipulate() is.
Oct 12 2009
prev sibling next sibling parent reply Robert Clipsham <robert octarineparrot.com> writes:
Walter Bright wrote:

 In my discussions with companies about adopting D, the major barrier 
 that comes up over and over isn't Tango vs Phobos, dmd being GPL, 
 debugger support, libraries, bugs, etc., although those are important.
 
 It's the IDE.
 
 They say that the productivity gains of D's improvements are 
 overbalanced by the loss of productivity by moving away from an IDE. And 
 what is it about an IDE that is so productive? Intellisense (Microsoft's 
 word for autocompletion).
 
 So, while I'm not going to be writing an IDE, I figure that dmd can 
 help. dmd already puts out .doc and .di files. How about putting out an 
 xml file giving all the information needed for an IDE to implement 
 autocompletion? There'd be one .xml file generated per .d source file.
 
 The nice thing about an xml file is while D is relatively easy to parse, 
 xml is trivial. Furthermore, an xml format would be fairly robust in the 
 face of changes to D syntax.
 
 What do you think?


How well will this work with partially parsable files? Will it recover 
and continue parsing the rest of the file, so all the gained data is 
still there in the rest of the file, or will it give an error and make 
the rest of the file lack autocompletion? Or is the idea to merge the 
newly parsed output with an old version so everything still has 
autocompletion but the line in question has an error?

Will it be possible to pass individual statements (maybe along with a 
previous symbol table) to dmd to save reparsing the whole file?

Other than that is sounds like a great idea, I can't wait for someone to 
write an omnicompletion plugin for vim using this!
Oct 11 2009
next sibling parent Jeremie Pelletier <jeremiep gmail.com> writes:
Robert Clipsham wrote:

 Walter Bright wrote:

 In my discussions with companies about adopting D, the major barrier 
 that comes up over and over isn't Tango vs Phobos, dmd being GPL, 
 debugger support, libraries, bugs, etc., although those are important.

 It's the IDE.

 They say that the productivity gains of D's improvements are 
 overbalanced by the loss of productivity by moving away from an IDE. 
 And what is it about an IDE that is so productive? Intellisense 
 (Microsoft's word for autocompletion).

 So, while I'm not going to be writing an IDE, I figure that dmd can 
 help. dmd already puts out .doc and .di files. How about putting out 
 an xml file giving all the information needed for an IDE to implement 
 autocompletion? There'd be one .xml file generated per .d source file.

 The nice thing about an xml file is while D is relatively easy to 
 parse, xml is trivial. Furthermore, an xml format would be fairly 
 robust in the face of changes to D syntax.

 What do you think?


 
 How well will this work with partially parsable files? Will it recover 
 and continue parsing the rest of the file, so all the gained data is 
 still there in the rest of the file, or will it give an error and make 
 the rest of the file lack autocompletion? Or is the idea to merge the 
 newly parsed output with an old version so everything still has 
 autocompletion but the line in question has an error?
 
 Will it be possible to pass individual statements (maybe along with a 
 previous symbol table) to dmd to save reparsing the whole file?
 
 Other than that is sounds like a great idea, I can't wait for someone to 
 write an omnicompletion plugin for vim using this!


That would be hard, if its a missing } then the entire parse tree will 
be in the wrong scope. The best the compiler can do is report an error 
at line x offset y and let the IDE highlight it, while it keeps on 
working with the info from the last successful analysis.
Oct 11 2009
prev sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Robert Clipsham wrote:

 How well will this work with partially parsable files?


Probably not very well. This would work best with getting information 
from modules other than the one being edited.
Oct 11 2009
parent reply BCS <none anon.com> writes:
Hello Walter,


 Robert Clipsham wrote:
 

 How well will this work with partially parsable files?
 


 Probably not very well. This would work best with getting information
 from modules other than the one being edited.
 


Ouch. There goes about half of what people will want.
Oct 12 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
BCS wrote:

 Ouch. There goes about half of what people will want.


Right, it's about half. But it's an easy half to get.
Oct 12 2009
parent reply BCS <none anon.com> writes:
Hello Walter,


 BCS wrote:
 

 Hello Walter,

 Probably not very well. This would work best with getting information
 from modules other than the one being edited.


 Ouch. There goes about half of what people will want.
 


 Right, it's about half. But it's an easy half to get.
 


Are you thinking of the partially parsed bit or the only in other modules 
bit? I was looking at the second.
Oct 12 2009
parent Walter Bright <newshound1 digitalmars.com> writes:
BCS wrote:

 Hello Walter,
 

 BCS wrote:


 Hello Walter,

 Probably not very well. This would work best with getting information
 from modules other than the one being edited.


 Ouch. There goes about half of what people will want.


 Right, it's about half. But it's an easy half to get.


 
 Are you thinking of the partially parsed bit or the only in other 
 modules bit? I was looking at the second.
 
 


Right, the second.
Oct 12 2009
prev sibling next sibling parent reply Frank Benoit <keinfarbton googlemail.com> writes:
Walter Bright schrieb:

 They say that the productivity gains of D's improvements are
 overbalanced by the loss of productivity by moving away from an IDE. And
 what is it about an IDE that is so productive?


If you use Eclipse for Java, you have:
- Debugger in place
- Automatic builder, compile on save
* Jump to line from error list
* Jump to declaration
- Show JavaDoc in tooltip, even in autocompletion
- Show hierarchy tree
- Autocompletion
- Quick assist, e.g.
	- assign ctor parameter to new field
	- extract selected text into local variable
	- Mark source portion, extract to method, the IDE evaluates the needed
parameters and return value
	- ...
There is so much more. But the main thing is, you are not only able to
use grep and friends on the pure text level. With an IDE you have
semantic support. This makes refactoring your code so much easier. You
can say "rename this method" and it works, all references to this method
are also altered. "Move this inner class to a top level class in that
package", "Derive from that class, yes add the needed ctors". There is
even an API to automate refactorings.

I think Descent is the right way. But here, a port of DMD is directly
integrated into the plugin. To put more manpower in this project would
be the best way imo.
Oct 11 2009
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 10/11/09 11:56, Frank Benoit wrote:

 Walter Bright schrieb:

 They say that the productivity gains of D's improvements are
 overbalanced by the loss of productivity by moving away from an IDE. And
 what is it about an IDE that is so productive?


 If you use Eclipse for Java, you have:
 - Debugger in place
 - Automatic builder, compile on save
 * Jump to line from error list
 * Jump to declaration
 - Show JavaDoc in tooltip, even in autocompletion
 - Show hierarchy tree
 - Autocompletion
 - Quick assist, e.g.
 	- assign ctor parameter to new field
 	- extract selected text into local variable
 	- Mark source portion, extract to method, the IDE evaluates the needed
 parameters and return value
 	- ...
 There is so much more. But the main thing is, you are not only able to
 use grep and friends on the pure text level. With an IDE you have
 semantic support. This makes refactoring your code so much easier. You
 can say "rename this method" and it works, all references to this method
 are also altered. "Move this inner class to a top level class in that
 package", "Derive from that class, yes add the needed ctors". There is
 even an API to automate refactorings.

 I think Descent is the right way. But here, a port of DMD is directly
 integrated into the plugin. To put more manpower in this project would
 be the best way imo.


I completely agree. I don't know if it would be better to have the parts 
of dmd that descent needs in a separate library to avoid porting half of 
dmd to java.
Oct 11 2009
parent Jeremie Pelletier <jeremiep gmail.com> writes:
Jacob Carlborg wrote:

 On 10/11/09 11:56, Frank Benoit wrote:

 Walter Bright schrieb:

 They say that the productivity gains of D's improvements are
 overbalanced by the loss of productivity by moving away from an IDE. And
 what is it about an IDE that is so productive?


 If you use Eclipse for Java, you have:
 - Debugger in place
 - Automatic builder, compile on save
 * Jump to line from error list
 * Jump to declaration
 - Show JavaDoc in tooltip, even in autocompletion
 - Show hierarchy tree
 - Autocompletion
 - Quick assist, e.g.
     - assign ctor parameter to new field
     - extract selected text into local variable
     - Mark source portion, extract to method, the IDE evaluates the 
 needed
 parameters and return value
     - ...
 There is so much more. But the main thing is, you are not only able to
 use grep and friends on the pure text level. With an IDE you have
 semantic support. This makes refactoring your code so much easier. You
 can say "rename this method" and it works, all references to this method
 are also altered. "Move this inner class to a top level class in that
 package", "Derive from that class, yes add the needed ctors". There is
 even an API to automate refactorings.

 I think Descent is the right way. But here, a port of DMD is directly
 integrated into the plugin. To put more manpower in this project would
 be the best way imo.


 
 I completely agree. I don't know if it would be better to have the parts 
 of dmd that descent needs in a separate library to avoid porting half of 
 dmd to java.
 


Which is why i suggested that next to the XML/JSON output there can be 
an API for plugins like descent to build on top of, instead of forking 
the entire compiler source making it near impossible to keep up with 
changes.

In my opinion, more manpower needs to be put in dmd itself so more third 
party programs and plugins can use it directly, the goal is to make dmd 
more powerful while supporting more usages.
Oct 11 2009
prev sibling parent Walter Bright <newshound1 digitalmars.com> writes:
Frank Benoit wrote:

 I think Descent is the right way. But here, a port of DMD is directly
 integrated into the plugin. To put more manpower in this project would
 be the best way imo.


Eclipse is probably, along with VS, one of the two most powerful IDEs.

But the JSON approach would also make things like Emacs usable with D 
with a basic level of autocomplete support, as well as the panopoly of 
simpler, more lightweight editors.

JSON support also will not take away anything from efforts to integrate 
DMD into IDE front ends. It's more for IDEs that don't have the 
resources to do that.
Oct 11 2009
prev sibling next sibling parent reply language_fan <foo bar.com.invalid> writes:
Sat, 10 Oct 2009 18:19:56 -0700, Walter Bright thusly wrote:


 In my discussions with companies about adopting D, the major barrier
 that comes up over and over isn't Tango vs Phobos, dmd being GPL,
 debugger support, libraries, bugs, etc., although those are important.
 
 It's the IDE.
 
 They say that the productivity gains of D's improvements are
 overbalanced by the loss of productivity by moving away from an IDE. And
 what is it about an IDE that is so productive? Intellisense (Microsoft's
 word for autocompletion).
 
 So, while I'm not going to be writing an IDE, I figure that dmd can
 help. dmd already puts out .doc and .di files. How about putting out an
 xml file giving all the information needed for an IDE to implement
 autocompletion? There'd be one .xml file generated per .d source file.
 
 The nice thing about an xml file is while D is relatively easy to parse,
 xml is trivial. Furthermore, an xml format would be fairly robust in the
 face of changes to D syntax.
 
 What do you think?


Well since there is already a project working on an Eclipse plugin, I see 
little use for other IDEs at the moment. The D community is rather small 
and only a small amount of people are capable of developing and willing 
to donate their free time on free IDE development (commercial IDEs have 
small potential now that Netbeans/Eclipse/IntelliJ/KDevelop/Visual Studio 
dominate the market). So why not concentrate on fixing the spec and 
fixing compiler bugs instead of building a modest IDE support no one will 
use?
Oct 11 2009
next sibling parent Lutger <lutger.blijdestijn gmail.com> writes:
language_fan wrote:


 Sat, 10 Oct 2009 18:19:56 -0700, Walter Bright thusly wrote:
 

 In my discussions with companies about adopting D, the major barrier
 that comes up over and over isn't Tango vs Phobos, dmd being GPL,
 debugger support, libraries, bugs, etc., although those are important.
 
 It's the IDE.
 
 They say that the productivity gains of D's improvements are
 overbalanced by the loss of productivity by moving away from an IDE. And
 what is it about an IDE that is so productive? Intellisense (Microsoft's
 word for autocompletion).
 
 So, while I'm not going to be writing an IDE, I figure that dmd can
 help. dmd already puts out .doc and .di files. How about putting out an
 xml file giving all the information needed for an IDE to implement
 autocompletion? There'd be one .xml file generated per .d source file.
 
 The nice thing about an xml file is while D is relatively easy to parse,
 xml is trivial. Furthermore, an xml format would be fairly robust in the
 face of changes to D syntax.
 
 What do you think?


 
 Well since there is already a project working on an Eclipse plugin, I see
 little use for other IDEs at the moment. The D community is rather small
 and only a small amount of people are capable of developing and willing
 to donate their free time on free IDE development (commercial IDEs have
 small potential now that Netbeans/Eclipse/IntelliJ/KDevelop/Visual Studio
 dominate the market). So why not concentrate on fixing the spec and
 fixing compiler bugs instead of building a modest IDE support no one will
 use?


Because people do see the use of other IDEs (counting vim here too as an 
IDE) and xml / json output is useful for more than just IDEs.
Oct 11 2009
prev sibling next sibling parent reply Jeremie Pelletier <jeremiep gmail.com> writes:
language_fan wrote:

 Sat, 10 Oct 2009 18:19:56 -0700, Walter Bright thusly wrote:
 

 In my discussions with companies about adopting D, the major barrier
 that comes up over and over isn't Tango vs Phobos, dmd being GPL,
 debugger support, libraries, bugs, etc., although those are important.

 It's the IDE.

 They say that the productivity gains of D's improvements are
 overbalanced by the loss of productivity by moving away from an IDE. And
 what is it about an IDE that is so productive? Intellisense (Microsoft's
 word for autocompletion).

 So, while I'm not going to be writing an IDE, I figure that dmd can
 help. dmd already puts out .doc and .di files. How about putting out an
 xml file giving all the information needed for an IDE to implement
 autocompletion? There'd be one .xml file generated per .d source file.

 The nice thing about an xml file is while D is relatively easy to parse,
 xml is trivial. Furthermore, an xml format would be fairly robust in the
 face of changes to D syntax.

 What do you think?


 
 Well since there is already a project working on an Eclipse plugin, I see 
 little use for other IDEs at the moment. The D community is rather small 
 and only a small amount of people are capable of developing and willing 
 to donate their free time on free IDE development (commercial IDEs have 
 small potential now that Netbeans/Eclipse/IntelliJ/KDevelop/Visual Studio 
 dominate the market). So why not concentrate on fixing the spec and 
 fixing compiler bugs instead of building a modest IDE support no one will 
 use?


Eclipse is heavy, slow and often unresponsive. I use poseidon myself 
because it is light and fast and I don't require much more from an IDE 
at the moment.
Oct 11 2009
parent reply language_fan <foo bar.com.invalid> writes:
Sun, 11 Oct 2009 12:44:08 -0400, Jeremie Pelletier thusly wrote:


 language_fan wrote:

 Well since there is already a project working on an Eclipse plugin, I
 see little use for other IDEs at the moment. The D community is rather
 small and only a small amount of people are capable of developing and
 willing to donate their free time on free IDE development (commercial
 IDEs have small potential now that
 Netbeans/Eclipse/IntelliJ/KDevelop/Visual Studio dominate the market).
 So why not concentrate on fixing the spec and fixing compiler bugs
 instead of building a modest IDE support no one will use?


 
 Eclipse is heavy, slow and often unresponsive. I use poseidon myself
 because it is light and fast and I don't require much more from an IDE
 at the moment.


If you turn off all the advanced features of Eclipse (spell checking, 
interactive parsing & type checking etc), it will become a lot more 
responsive. You can even uninstall many of the plugins if you really do 
not need them. I recommend firing up the latest development build with 
the latest 1.6 jvm. It will take couple of seconds for the JIT to spot 
the hot spots after starting up. Of course Poseidon is faster *now* that 
it lacks all the advanced features, but once you start adding more, it 
will eventually grind to a halt.
Oct 12 2009
parent Michal Minich <michal minich.sk> writes:
 Eclipse is heavy, slow and often unresponsive. I use poseidon myself
 because it is light and fast and I don't require much more from an
 IDE at the moment.




You can try to download just bare Eclipse platform (which is just text editor) 
and install descent into it using Eclipse software updates. It starts up 
faster than C or Java version Eclipse and there is only D related stuff in 
the UI.

http://download.eclipse.org/eclipse/downloads/drops/R-3.5.1-200909170800/index.php
Under "Platform runtime binary"
Oct 12 2009
prev sibling next sibling parent reply "Nick Sabalausky" <a a.a> writes:
"language_fan" <foo bar.com.invalid> wrote in message 
news:hasd5u$1fgu$1 digitalmars.com...

 Well since there is already a project working on an Eclipse plugin, I see
 little use for other IDEs at the moment. The D community is rather small
 and only a small amount of people are capable of developing and willing
 to donate their free time on free IDE development (commercial IDEs have
 small potential now that Netbeans/Eclipse/IntelliJ/KDevelop/Visual Studio
 dominate the market). So why not concentrate on fixing the spec and
 fixing compiler bugs instead of building a modest IDE support no one will
 use?


Any editor that is less responsive than notepad is useless to me, and I'm 
far from alone on that. Thus, we don't go anywhere near Eclipse, VS.NET, or 
anything along those lines (bunch of bloated garbage...and probably designed 
to get developers to buy fancier hardware and thus end up develop apps that 
require their users to buy fancier hardware).
Oct 11 2009
parent reply language_fan <foo bar.com.invalid> writes:
Sun, 11 Oct 2009 17:01:03 -0400, Nick Sabalausky thusly wrote:


 "language_fan" <foo bar.com.invalid> wrote in message
 news:hasd5u$1fgu$1 digitalmars.com...

 Well since there is already a project working on an Eclipse plugin, I
 see little use for other IDEs at the moment. The D community is rather
 small and only a small amount of people are capable of developing and
 willing to donate their free time on free IDE development (commercial
 IDEs have small potential now that
 Netbeans/Eclipse/IntelliJ/KDevelop/Visual Studio dominate the market).
 So why not concentrate on fixing the spec and fixing compiler bugs
 instead of building a modest IDE support no one will use?


 
 Any editor that is less responsive than notepad is useless to me, and
 I'm far from alone on that. Thus, we don't go anywhere near Eclipse,
 VS.NET, or anything along those lines (bunch of bloated garbage...and
 probably designed to get developers to buy fancier hardware and thus end
 up develop apps that require their users to buy fancier hardware).


Fine, if you are happy with your productivity. For me it was a natural 
choice. I started with notepad like editors, then moved to "programmer's 
editors" like crimson editor and jedit, soon discovered vim and emacs, 
and finally have settled down on Eclipse, Netbeans, Visual Studio, and 
IntelliJ IDEA. The productivity boost is enormous. "Practical" languages 
have lots of boiler-plate, and I can easily generate hundreds of lines of 
code with a couple of key combinations or mouse clicks. Another killer 
feature is the "intellisense" stuff. Having the ability to see a) 
inferred type b) integrated html rendered javadoc c) type correct class 
members etc. saves me hours of work time per day. YMMV
Oct 12 2009
next sibling parent reply "Nick Sabalausky" <a a.a> writes:
"language_fan" <foo bar.com.invalid> wrote in message 
news:hav02v$2162$4 digitalmars.com...

 Sun, 11 Oct 2009 17:01:03 -0400, Nick Sabalausky thusly wrote:


 "language_fan" <foo bar.com.invalid> wrote in message
 news:hasd5u$1fgu$1 digitalmars.com...

 Well since there is already a project working on an Eclipse plugin, I
 see little use for other IDEs at the moment. The D community is rather
 small and only a small amount of people are capable of developing and
 willing to donate their free time on free IDE development (commercial
 IDEs have small potential now that
 Netbeans/Eclipse/IntelliJ/KDevelop/Visual Studio dominate the market).
 So why not concentrate on fixing the spec and fixing compiler bugs
 instead of building a modest IDE support no one will use?


 Any editor that is less responsive than notepad is useless to me, and
 I'm far from alone on that. Thus, we don't go anywhere near Eclipse,
 VS.NET, or anything along those lines (bunch of bloated garbage...and
 probably designed to get developers to buy fancier hardware and thus end
 up develop apps that require their users to buy fancier hardware).


 Fine, if you are happy with your productivity. For me it was a natural
 choice. I started with notepad like editors, then moved to "programmer's
 editors" like crimson editor and jedit, soon discovered vim and emacs,
 and finally have settled down on Eclipse, Netbeans, Visual Studio, and
 IntelliJ IDEA.


...

 The productivity boost is enormous.


...

 Another killer
 feature is the "intellisense" stuff. Having the ability to see a)
 inferred type b) integrated html rendered javadoc c) type correct class
 members etc. saves me hours of work time per day. YMMV


I've used all that stuff before. I had to use Eclipse back when I was doing 

VS.NET had most of those nice features, but wasn't nearly as bloated and 
sluggigh as it is today). So I'm very familiar with all of that stuff. In 
fact, I love all of it. But with things like Programmer's Notepad 2, I can 
actually get into a "flow" (which sluggishness instantly breaks) and I end 
up far *more* productive (as long as I'm not using some POS language like 
Java, see below).


 "Practical" languages
 have lots of boiler-plate, and I can easily generate hundreds of lines of
 code with a couple of key combinations or mouse clicks.


I disagree very much. For a puritanical anal-retentive language like Java, 
you *have* to have all that fancy stuff just to make the language even 
usable. But with a practical language like D, I have mixins, and I've gotten 
pretty good with regex find and replace, which obviously isn't as good, but 
it's good enough when dealing with any language that's at least somewhat 
sensible. With practical languages I never need to generate hundreds of 
lines of code. If I ever actually needed to do so (and in a way that an IDE 
could actually help with), I'd take that as a very big red flag that I was 
using an incredibly shitty language in the first place (like Java).

Also, Eclipse, even with descent, is still very Java-centric, particularly 
wrt project/target/workspace management, and that gets in the way. And VS 
just doesn't have any D stuff available at all and takes a lot of work, a 
lot of bad documentation, and IIRC, a paid version of VS, to add or use any 
support at all for a new langauge. PN2 has D syntax highlighting built-in, 
which already puts it well ahead of VS for D.
Oct 12 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Nick Sabalausky wrote:

 "language_fan" <foo bar.com.invalid> wrote in message 

 "Practical" languages
 have lots of boiler-plate, and I can easily generate hundreds of lines of
 code with a couple of key combinations or mouse clicks.


 
 I disagree very much. For a puritanical anal-retentive language like Java, 
 you *have* to have all that fancy stuff just to make the language even 
 usable. But with a practical language like D, I have mixins, and I've gotten 
 pretty good with regex find and replace, which obviously isn't as good, but 
 it's good enough when dealing with any language that's at least somewhat 
 sensible. With practical languages I never need to generate hundreds of 
 lines of code. If I ever actually needed to do so (and in a way that an IDE 
 could actually help with), I'd take that as a very big red flag that I was 
 using an incredibly shitty language in the first place (like Java).


I think Nick has a point. Java lacks template mixins, and so inserting 
hundreds of lines of code from an IDE "template" is normal. But with D, 
doing such would be abnormal.
Oct 12 2009
parent "Nick Sabalausky" <a a.a> writes:
"Walter Bright" <newshound1 digitalmars.com> wrote in message 
news:hb08gf$2hvq$1 digitalmars.com...

 Nick Sabalausky wrote:

 "language_fan" <foo bar.com.invalid> wrote in message

 "Practical" languages
 have lots of boiler-plate, and I can easily generate hundreds of lines 
 of
 code with a couple of key combinations or mouse clicks.


 I disagree very much. For a puritanical anal-retentive language like 
 Java, you *have* to have all that fancy stuff just to make the language 
 even usable. But with a practical language like D, I have mixins, and 
 I've gotten pretty good with regex find and replace, which obviously 
 isn't as good, but it's good enough when dealing with any language that's 
 at least somewhat sensible. With practical languages I never need to 
 generate hundreds of lines of code. If I ever actually needed to do so 
 (and in a way that an IDE could actually help with), I'd take that as a 
 very big red flag that I was using an incredibly shitty language in the 
 first place (like Java).


 I think Nick has a point. Java lacks template mixins, and so inserting 
 hundreds of lines of code from an IDE "template" is normal. But with D, 
 doing such would be abnormal.


And not only that, but D supports higher-level and non-OOP constructs far 
better than Java, and far less verbosely, so you don't always even need to 
resort to mixins. A lot of the time you can just simply use better features. 
Like delegate literals instead of functors.
Oct 12 2009
prev sibling parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
language_fan wrote:

 "Practical" languages
 have lots of boiler-plate, and I can easily generate hundreds of lines of 
 code with a couple of key combinations or mouse clicks.


Can you give some examples? I can only think of some that generate some 
lines of code, not hundreds.
Oct 13 2009
next sibling parent Lutger <lutger.blijdestijn gmail.com> writes:
Ary Borenszweig wrote:


 language_fan wrote:
  > "Practical" languages

 have lots of boiler-plate, and I can easily generate hundreds of lines of
 code with a couple of key combinations or mouse clicks.


 
 Can you give some examples? I can only think of some that generate some
 lines of code, not hundreds.


linq-to-sql
Oct 13 2009
prev sibling parent language_fan <foo bar.com.invalid> writes:
Tue, 13 Oct 2009 10:33:54 +0200, Ary Borenszweig thusly wrote:


 language_fan wrote:
  > "Practical" languages

 have lots of boiler-plate, and I can easily generate hundreds of lines
 of code with a couple of key combinations or mouse clicks.


 
 Can you give some examples? I can only think of some that generate some
 lines of code, not hundreds.


For instance simple things like setters and getters for an aggregate 
class easily generate tens of lines of code (well, not only code lines, 
but also other kinds of editable lines). Like lutger said, all kinds of 
transformations also come in mind in this context. There could be even 
ORM mapping utilities which generate all the code for accessing the 
database structures.
Oct 13 2009
prev sibling parent language_fan <foo bar.com.invalid> writes:
Sun, 11 Oct 2009 10:48:30 +0000, language_fan thusly wrote:


 Well since there is already a project working on an Eclipse plugin, I
 see little use for other IDEs at the moment. The D community is rather
 small and only a small amount of people are capable of developing and
 willing to donate their free time on free IDE development (commercial
 IDEs have small potential now that
 Netbeans/Eclipse/IntelliJ/KDevelop/Visual Studio dominate the market).


BTW totally off-topic, but I just found out that IntelliJ IDEA has been 
jumped on the open source wagon. This probably makes it a bit more 
challenging to make money on the commercial IDE market.
Oct 15 2009
prev sibling next sibling parent Christopher Wright <dhasenan gmail.com> writes:
Walter Bright wrote:

 In my discussions with companies about adopting D, the major barrier 
 that comes up over and over isn't Tango vs Phobos, dmd being GPL, 
 debugger support, libraries, bugs, etc., although those are important.
 
 It's the IDE.
 
 They say that the productivity gains of D's improvements are 
 overbalanced by the loss of productivity by moving away from an IDE. And 
 what is it about an IDE that is so productive? Intellisense (Microsoft's 
 word for autocompletion).
 
 So, while I'm not going to be writing an IDE, I figure that dmd can 
 help. dmd already puts out .doc and .di files. How about putting out an 
 xml file giving all the information needed for an IDE to implement 
 autocompletion? There'd be one .xml file generated per .d source file.
 
 The nice thing about an xml file is while D is relatively easy to parse, 
 xml is trivial. Furthermore, an xml format would be fairly robust in the 
 face of changes to D syntax.
 
 What do you think?


The huge things are:
  - code navigation (go to definition / find usages)
  - reformatting
  - refactoring
  - autocompletion

Code navigation alone is a huge help, and with reliable autocompletion 
would be sufficient for me to switch from vim. What you are suggesting 
would make both of those easier, though the IDE might need to duplicate 
D's symbol lookup.

I'm not sure whether what you are talking about will help at all with 
reformatting or refactoring, and I really have no idea what would be 
required for this.
Oct 11 2009
prev sibling next sibling parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
Walter Bright wrote:

 In my discussions with companies about adopting D, the major barrier 
 that comes up over and over isn't Tango vs Phobos, dmd being GPL, 
 debugger support, libraries, bugs, etc., although those are important.
 
 It's the IDE.
 
 So, while I'm not going to be writing an IDE, I figure that dmd can 
 help. dmd already puts out .doc and .di files. How about putting out an 
 xml file giving all the information needed for an IDE to implement 
 autocompletion? There'd be one .xml file generated per .d source file.
 
 What do you think?


What I think is that even with an xml representing the parse tree (maybe 
with some semantic stuff resolved) it'll be still incomplete for a real 
IDE (the kind of thing users expect from an IDE). You can see this 
video, for example:

http://www.youtube.com/watch?v=KQbTT605ags

So you have:

---
module one;

class Foo(T) {
   static if (is(T == class)) {
     T property;
   } else {
     T someMethod() { return T.init; }
   }
   mixin(guessWhat!(T)());
}
---

You want to define an xml for that module that'll help IDEs. Can you 
think what it'll look like?

Now the user writes in another module:

class Bar {
}

void x() {
   auto foo = new Foo!(Bar)();
   foo. <-- what does the IDE do here?
}

Now, the correct thing for the IDE to do is to suggest the field "Bar 
property". How can the IDE do that just with an xml? It can't. It need 
to perform some kind of semantic anlysis to Foo's argument to see if 
it's a class, match the static if in the template, replace template 
parameters, etc. It also needs to evaluate the string mixin.

Of course you could say "Bah, just show all the declarations inside the 
template in the autocomplete", but that's wrong. That'll lead to files 
that don't compile. You could ommit supporting autocompletion or other 
nice features, but that's exactly the big features of D. If you don't 

could use the advanced features but the IDE won't help you. And in your 
discussions with companies adopting D, I'm sure they were talking about 
great IDEs like JDT Eclipse or Visual Studio, not just some tool that 
helps you a little but not anymore when things get interesting.

Oh, and you need to have some kind of semantic analysis to know the type 
of "auto foo". Again, maybe the IDE can be dummy and see "auto foo = new 
Foo!(Bar)" and say "ok, foo's type is Foo!(Bar)", but then you have:

auto b = foo.property;
b. <-- and here?
// remember "property" is templated and depends on static analysis
// or the IDE could need to resolve alias this or other things

So... my opinion (like some others, I see) is to either ask things to 
the compiler directly (but here the compiler lacks some info, like exact 
source range positions), or to have a compiler (not a full-blown one, 
just the front-end) built into the IDE, and that's what Descent is. 
Unfortunately Descent is sometimes slow, sometimes buggy, but that's 
normal: just a few people develop and maintain it (so I can see a 
similarity with dmd here, where each day I see two or three new bugs 
reported). If more people were into it, more unit tests were written 
into it and, most of all, more people would use it, it'll get better.

Another problem that people see in Descent (maybe also JDT Eclipse and 
Visual Studio0 is that it's huge, it consumes a lot of memory and they 
don't want to open a huge tool just to hack some lines. My answer is: 
memory performance can be improved (but not a lot), but since an IDE is 
a huge tool it requires a lof from the computer. And an IDE is not meant 
to be used to hack some lines, it's meant to help you write big project, 
huge projects without getting lost in the amount of code.

So my bet would be to start supporting an existing IDE that integrates a 
compiler into it. Updating it is easy: just port the diffs between DMD 
versions. It's a huge job for one or two people, but if a lot of people 
were involved it's not that much. Of course I'd recommend you to try 
Descent since I'm one of it's creators and I do believe it can get 
pretty good. :-)
Oct 11 2009
next sibling parent reply Yigal Chripun <yigal100 gmail.com> writes:
On 11/10/2009 15:23, Ary Borenszweig wrote:

 Walter Bright wrote:

 In my discussions with companies about adopting D, the major barrier
 that comes up over and over isn't Tango vs Phobos, dmd being GPL,
 debugger support, libraries, bugs, etc., although those are important.

 It's the IDE.

 So, while I'm not going to be writing an IDE, I figure that dmd can
 help. dmd already puts out .doc and .di files. How about putting out
 an xml file giving all the information needed for an IDE to implement
 autocompletion? There'd be one .xml file generated per .d source file.

 What do you think?


 What I think is that even with an xml representing the parse tree (maybe
 with some semantic stuff resolved) it'll be still incomplete for a real
 IDE (the kind of thing users expect from an IDE). You can see this
 video, for example:

 http://www.youtube.com/watch?v=KQbTT605ags

 So you have:

 ---
 module one;

 class Foo(T) {
 static if (is(T == class)) {
 T property;
 } else {
 T someMethod() { return T.init; }
 }
 mixin(guessWhat!(T)());
 }
 ---

 You want to define an xml for that module that'll help IDEs. Can you
 think what it'll look like?

 Now the user writes in another module:

 class Bar {
 }

 void x() {
 auto foo = new Foo!(Bar)();
 foo. <-- what does the IDE do here?
 }

 Now, the correct thing for the IDE to do is to suggest the field "Bar
 property". How can the IDE do that just with an xml? It can't. It need
 to perform some kind of semantic anlysis to Foo's argument to see if
 it's a class, match the static if in the template, replace template
 parameters, etc. It also needs to evaluate the string mixin.

 Of course you could say "Bah, just show all the declarations inside the
 template in the autocomplete", but that's wrong. That'll lead to files
 that don't compile. You could ommit supporting autocompletion or other
 nice features, but that's exactly the big features of D. If you don't

 could use the advanced features but the IDE won't help you. And in your
 discussions with companies adopting D, I'm sure they were talking about
 great IDEs like JDT Eclipse or Visual Studio, not just some tool that
 helps you a little but not anymore when things get interesting.

 Oh, and you need to have some kind of semantic analysis to know the type
 of "auto foo". Again, maybe the IDE can be dummy and see "auto foo = new
 Foo!(Bar)" and say "ok, foo's type is Foo!(Bar)", but then you have:

 auto b = foo.property;
 b. <-- and here?
 // remember "property" is templated and depends on static analysis
 // or the IDE could need to resolve alias this or other things

 So... my opinion (like some others, I see) is to either ask things to
 the compiler directly (but here the compiler lacks some info, like exact
 source range positions), or to have a compiler (not a full-blown one,
 just the front-end) built into the IDE, and that's what Descent is.
 Unfortunately Descent is sometimes slow, sometimes buggy, but that's
 normal: just a few people develop and maintain it (so I can see a
 similarity with dmd here, where each day I see two or three new bugs
 reported). If more people were into it, more unit tests were written
 into it and, most of all, more people would use it, it'll get better.

 Another problem that people see in Descent (maybe also JDT Eclipse and
 Visual Studio0 is that it's huge, it consumes a lot of memory and they
 don't want to open a huge tool just to hack some lines. My answer is:
 memory performance can be improved (but not a lot), but since an IDE is
 a huge tool it requires a lof from the computer. And an IDE is not meant
 to be used to hack some lines, it's meant to help you write big project,
 huge projects without getting lost in the amount of code.

 So my bet would be to start supporting an existing IDE that integrates a
 compiler into it. Updating it is easy: just port the diffs between DMD
 versions. It's a huge job for one or two people, but if a lot of people
 were involved it's not that much. Of course I'd recommend you to try
 Descent since I'm one of it's creators and I do believe it can get
 pretty good. :-)


well put.

btw, given that we have a port of SWT for D, how hard would it be to 
create our own native D version of eclipse?
Oct 11 2009
parent reply Jacob Carlborg <doob me.com> writes:
On 10/11/09 20:58, Yigal Chripun wrote:

 On 11/10/2009 15:23, Ary Borenszweig wrote:

 Walter Bright wrote:

 In my discussions with companies about adopting D, the major barrier
 that comes up over and over isn't Tango vs Phobos, dmd being GPL,
 debugger support, libraries, bugs, etc., although those are important.

 It's the IDE.

 So, while I'm not going to be writing an IDE, I figure that dmd can
 help. dmd already puts out .doc and .di files. How about putting out
 an xml file giving all the information needed for an IDE to implement
 autocompletion? There'd be one .xml file generated per .d source file.

 What do you think?


 What I think is that even with an xml representing the parse tree (maybe
 with some semantic stuff resolved) it'll be still incomplete for a real
 IDE (the kind of thing users expect from an IDE). You can see this
 video, for example:

 http://www.youtube.com/watch?v=KQbTT605ags

 So you have:

 ---
 module one;

 class Foo(T) {
 static if (is(T == class)) {
 T property;
 } else {
 T someMethod() { return T.init; }
 }
 mixin(guessWhat!(T)());
 }
 ---

 You want to define an xml for that module that'll help IDEs. Can you
 think what it'll look like?

 Now the user writes in another module:

 class Bar {
 }

 void x() {
 auto foo = new Foo!(Bar)();
 foo. <-- what does the IDE do here?
 }

 Now, the correct thing for the IDE to do is to suggest the field "Bar
 property". How can the IDE do that just with an xml? It can't. It need
 to perform some kind of semantic anlysis to Foo's argument to see if
 it's a class, match the static if in the template, replace template
 parameters, etc. It also needs to evaluate the string mixin.

 Of course you could say "Bah, just show all the declarations inside the
 template in the autocomplete", but that's wrong. That'll lead to files
 that don't compile. You could ommit supporting autocompletion or other
 nice features, but that's exactly the big features of D. If you don't

 could use the advanced features but the IDE won't help you. And in your
 discussions with companies adopting D, I'm sure they were talking about
 great IDEs like JDT Eclipse or Visual Studio, not just some tool that
 helps you a little but not anymore when things get interesting.

 Oh, and you need to have some kind of semantic analysis to know the type
 of "auto foo". Again, maybe the IDE can be dummy and see "auto foo = new
 Foo!(Bar)" and say "ok, foo's type is Foo!(Bar)", but then you have:

 auto b = foo.property;
 b. <-- and here?
 // remember "property" is templated and depends on static analysis
 // or the IDE could need to resolve alias this or other things

 So... my opinion (like some others, I see) is to either ask things to
 the compiler directly (but here the compiler lacks some info, like exact
 source range positions), or to have a compiler (not a full-blown one,
 just the front-end) built into the IDE, and that's what Descent is.
 Unfortunately Descent is sometimes slow, sometimes buggy, but that's
 normal: just a few people develop and maintain it (so I can see a
 similarity with dmd here, where each day I see two or three new bugs
 reported). If more people were into it, more unit tests were written
 into it and, most of all, more people would use it, it'll get better.

 Another problem that people see in Descent (maybe also JDT Eclipse and
 Visual Studio0 is that it's huge, it consumes a lot of memory and they
 don't want to open a huge tool just to hack some lines. My answer is:
 memory performance can be improved (but not a lot), but since an IDE is
 a huge tool it requires a lof from the computer. And an IDE is not meant
 to be used to hack some lines, it's meant to help you write big project,
 huge projects without getting lost in the amount of code.

 So my bet would be to start supporting an existing IDE that integrates a
 compiler into it. Updating it is easy: just port the diffs between DMD
 versions. It's a huge job for one or two people, but if a lot of people
 were involved it's not that much. Of course I'd recommend you to try
 Descent since I'm one of it's creators and I do believe it can get
 pretty good. :-)


 well put.

 btw, given that we have a port of SWT for D, how hard would it be to
 create our own native D version of eclipse?


Frank Benoit (who started DWT, the tango version) was think about this. 
Quite a big part is already ported, SWT, JFace, Forms, OSGI and so on, 
see: http://hg.dsource.org/projects/dwt2/file/88652073d1c2/ .

I think one of the biggest challenge to port is class loading, dynamic 
libraries could work for this but we all know how well they work on windows.
Oct 11 2009
parent Frank Benoit <keinfarbton googlemail.com> writes:
Jacob Carlborg schrieb:

 On 10/11/09 20:58, Yigal Chripun wrote:

 On 11/10/2009 15:23, Ary Borenszweig wrote:

 Walter Bright wrote:

 In my discussions with companies about adopting D, the major barrier
 that comes up over and over isn't Tango vs Phobos, dmd being GPL,
 debugger support, libraries, bugs, etc., although those are important.

 It's the IDE.

 So, while I'm not going to be writing an IDE, I figure that dmd can
 help. dmd already puts out .doc and .di files. How about putting out
 an xml file giving all the information needed for an IDE to implement
 autocompletion? There'd be one .xml file generated per .d source file.

 What do you think?


 What I think is that even with an xml representing the parse tree (maybe
 with some semantic stuff resolved) it'll be still incomplete for a real
 IDE (the kind of thing users expect from an IDE). You can see this
 video, for example:

 http://www.youtube.com/watch?v=KQbTT605ags

 So you have:

 ---
 module one;

 class Foo(T) {
 static if (is(T == class)) {
 T property;
 } else {
 T someMethod() { return T.init; }
 }
 mixin(guessWhat!(T)());
 }
 ---

 You want to define an xml for that module that'll help IDEs. Can you
 think what it'll look like?

 Now the user writes in another module:

 class Bar {
 }

 void x() {
 auto foo = new Foo!(Bar)();
 foo. <-- what does the IDE do here?
 }

 Now, the correct thing for the IDE to do is to suggest the field "Bar
 property". How can the IDE do that just with an xml? It can't. It need
 to perform some kind of semantic anlysis to Foo's argument to see if
 it's a class, match the static if in the template, replace template
 parameters, etc. It also needs to evaluate the string mixin.

 Of course you could say "Bah, just show all the declarations inside the
 template in the autocomplete", but that's wrong. That'll lead to files
 that don't compile. You could ommit supporting autocompletion or other
 nice features, but that's exactly the big features of D. If you don't

 could use the advanced features but the IDE won't help you. And in your
 discussions with companies adopting D, I'm sure they were talking about
 great IDEs like JDT Eclipse or Visual Studio, not just some tool that
 helps you a little but not anymore when things get interesting.

 Oh, and you need to have some kind of semantic analysis to know the type
 of "auto foo". Again, maybe the IDE can be dummy and see "auto foo = new
 Foo!(Bar)" and say "ok, foo's type is Foo!(Bar)", but then you have:

 auto b = foo.property;
 b. <-- and here?
 // remember "property" is templated and depends on static analysis
 // or the IDE could need to resolve alias this or other things

 So... my opinion (like some others, I see) is to either ask things to
 the compiler directly (but here the compiler lacks some info, like exact
 source range positions), or to have a compiler (not a full-blown one,
 just the front-end) built into the IDE, and that's what Descent is.
 Unfortunately Descent is sometimes slow, sometimes buggy, but that's
 normal: just a few people develop and maintain it (so I can see a
 similarity with dmd here, where each day I see two or three new bugs
 reported). If more people were into it, more unit tests were written
 into it and, most of all, more people would use it, it'll get better.

 Another problem that people see in Descent (maybe also JDT Eclipse and
 Visual Studio0 is that it's huge, it consumes a lot of memory and they
 don't want to open a huge tool just to hack some lines. My answer is:
 memory performance can be improved (but not a lot), but since an IDE is
 a huge tool it requires a lof from the computer. And an IDE is not meant
 to be used to hack some lines, it's meant to help you write big project,
 huge projects without getting lost in the amount of code.

 So my bet would be to start supporting an existing IDE that integrates a
 compiler into it. Updating it is easy: just port the diffs between DMD
 versions. It's a huge job for one or two people, but if a lot of people
 were involved it's not that much. Of course I'd recommend you to try
 Descent since I'm one of it's creators and I do believe it can get
 pretty good. :-)


 well put.

 btw, given that we have a port of SWT for D, how hard would it be to
 create our own native D version of eclipse?


 
 Frank Benoit (who started DWT, the tango version) was think about this.
 Quite a big part is already ported, SWT, JFace, Forms, OSGI and so on,
 see: http://hg.dsource.org/projects/dwt2/file/88652073d1c2/ .


Sorry, OSGi is just dummy, to make other stuff compile.
Oct 12 2009
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 10/11/09 15:23, Ary Borenszweig wrote:

 Walter Bright wrote:

 In my discussions with companies about adopting D, the major barrier
 that comes up over and over isn't Tango vs Phobos, dmd being GPL,
 debugger support, libraries, bugs, etc., although those are important.

 It's the IDE.

 So, while I'm not going to be writing an IDE, I figure that dmd can
 help. dmd already puts out .doc and .di files. How about putting out
 an xml file giving all the information needed for an IDE to implement
 autocompletion? There'd be one .xml file generated per .d source file.

 What do you think?


 What I think is that even with an xml representing the parse tree (maybe
 with some semantic stuff resolved) it'll be still incomplete for a real
 IDE (the kind of thing users expect from an IDE). You can see this
 video, for example:

 http://www.youtube.com/watch?v=KQbTT605ags

 So you have:

 ---
 module one;

 class Foo(T) {
 static if (is(T == class)) {
 T property;
 } else {
 T someMethod() { return T.init; }
 }
 mixin(guessWhat!(T)());
 }
 ---

 You want to define an xml for that module that'll help IDEs. Can you
 think what it'll look like?

 Now the user writes in another module:

 class Bar {
 }

 void x() {
 auto foo = new Foo!(Bar)();
 foo. <-- what does the IDE do here?
 }

 Now, the correct thing for the IDE to do is to suggest the field "Bar
 property". How can the IDE do that just with an xml? It can't. It need
 to perform some kind of semantic anlysis to Foo's argument to see if
 it's a class, match the static if in the template, replace template
 parameters, etc. It also needs to evaluate the string mixin.

 Of course you could say "Bah, just show all the declarations inside the
 template in the autocomplete", but that's wrong. That'll lead to files
 that don't compile. You could ommit supporting autocompletion or other
 nice features, but that's exactly the big features of D. If you don't

 could use the advanced features but the IDE won't help you. And in your
 discussions with companies adopting D, I'm sure they were talking about
 great IDEs like JDT Eclipse or Visual Studio, not just some tool that
 helps you a little but not anymore when things get interesting.

 Oh, and you need to have some kind of semantic analysis to know the type
 of "auto foo". Again, maybe the IDE can be dummy and see "auto foo = new
 Foo!(Bar)" and say "ok, foo's type is Foo!(Bar)", but then you have:

 auto b = foo.property;
 b. <-- and here?
 // remember "property" is templated and depends on static analysis
 // or the IDE could need to resolve alias this or other things

 So... my opinion (like some others, I see) is to either ask things to
 the compiler directly (but here the compiler lacks some info, like exact
 source range positions), or to have a compiler (not a full-blown one,
 just the front-end) built into the IDE, and that's what Descent is.
 Unfortunately Descent is sometimes slow, sometimes buggy, but that's
 normal: just a few people develop and maintain it (so I can see a
 similarity with dmd here, where each day I see two or three new bugs
 reported). If more people were into it, more unit tests were written
 into it and, most of all, more people would use it, it'll get better.


I use it almost at a daily base and it works great. I can't remember the 
last time it cashed.


 Another problem that people see in Descent (maybe also JDT Eclipse and
 Visual Studio0 is that it's huge, it consumes a lot of memory and they
 don't want to open a huge tool just to hack some lines. My answer is:
 memory performance can be improved (but not a lot), but since an IDE is
 a huge tool it requires a lof from the computer. And an IDE is not meant
 to be used to hack some lines, it's meant to help you write big project,
 huge projects without getting lost in the amount of code.

 So my bet would be to start supporting an existing IDE that integrates a
 compiler into it. Updating it is easy: just port the diffs between DMD
 versions. It's a huge job for one or two people, but if a lot of people
 were involved it's not that much. Of course I'd recommend you to try
 Descent since I'm one of it's creators and I do believe it can get
 pretty good. :-)
Oct 11 2009
parent Ary Borenszweig <ary esperanto.org.ar> writes:
Jacob Carlborg wrote:

 On 10/11/09 15:23, Ary Borenszweig wrote:

 Walter Bright wrote:

 In my discussions with companies about adopting D, the major barrier
 that comes up over and over isn't Tango vs Phobos, dmd being GPL,
 debugger support, libraries, bugs, etc., although those are important.

 It's the IDE.

 So, while I'm not going to be writing an IDE, I figure that dmd can
 help. dmd already puts out .doc and .di files. How about putting out
 an xml file giving all the information needed for an IDE to implement
 autocompletion? There'd be one .xml file generated per .d source file.

 What do you think?


 So... my opinion (like some others, I see) is to either ask things to
 the compiler directly (but here the compiler lacks some info, like exact
 source range positions), or to have a compiler (not a full-blown one,
 just the front-end) built into the IDE, and that's what Descent is.
 Unfortunately Descent is sometimes slow, sometimes buggy, but that's
 normal: just a few people develop and maintain it (so I can see a
 similarity with dmd here, where each day I see two or three new bugs
 reported). If more people were into it, more unit tests were written
 into it and, most of all, more people would use it, it'll get better.


 
 I use it almost at a daily base and it works great. I can't remember the 
 last time it cashed.


Wow, that's good news! :)
Oct 11 2009
prev sibling next sibling parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Sun, 11 Oct 2009 05:19:56 +0400, Walter Bright  
<newshound1 digitalmars.com> wrote:


 In my discussions with companies about adopting D, the major barrier  
 that comes up over and over isn't Tango vs Phobos, dmd being GPL,  
 debugger support, libraries, bugs, etc., although those are important.

 It's the IDE.

 They say that the productivity gains of D's improvements are  
 overbalanced by the loss of productivity by moving away from an IDE. And  
 what is it about an IDE that is so productive? Intellisense (Microsoft's  
 word for autocompletion).

 So, while I'm not going to be writing an IDE, I figure that dmd can  
 help. dmd already puts out .doc and .di files. How about putting out an  
 xml file giving all the information needed for an IDE to implement  
 autocompletion? There'd be one .xml file generated per .d source file.

 The nice thing about an xml file is while D is relatively easy to parse,  
 xml is trivial. Furthermore, an xml format would be fairly robust in the  
 face of changes to D syntax.

 What do you think?


I believe it won't work. It will always be slow and incomplete.

Instead, I would make it easier to embed DMD into an IDE: separate DMDFE  
 from DMDBE, fix memory leaks, remove all the static data (so that code  
would be re-intrable and it could work in different threads in parallel),  
move most of DMD code into a DLL so that an IDE could dynamically link  
with it and whatever it pleases with the source code.

In fact, that's what I do right now.

I'm writing my own D IDE in my spare time (in D, of course). I already  
made a lot of progress and now it's time to start implementing source code  
analysis.

First thing I did is I made complete D bindings for C++ code. It worked  
out quite well but it was leaking so badly that I dropped it.

Instead, I started porting DMD entirely to D (except the backend, of  
course), and I already got some great results. A few simple programs  
compile and produce byte-perfect binaries. It's still in its early stages  
and there is a lot of work to do, but it will be finished soon  
(hopefully). It could probably become a part of an official distribution,  
eventually. :)

If anyone is interested and is willing to test and/or help, I will gladly  
share my code.
Oct 11 2009
next sibling parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
Denis Koroskin wrote:

 On Sun, 11 Oct 2009 05:19:56 +0400, Walter Bright
 <newshound1 digitalmars.com> wrote:
 
 If anyone is interested and is willing to test and/or help, I will
 gladly share my code.


Oooo.. You should put that on dsource or somewhere. Hacking D sounds
like a lot more fun than hacking C++. I wouldn't mind helping out on
this one.
Oct 11 2009
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Ellery Newcomer wrote:

 Denis Koroskin wrote:

 On Sun, 11 Oct 2009 05:19:56 +0400, Walter Bright
 <newshound1 digitalmars.com> wrote:

 If anyone is interested and is willing to test and/or help, I will
 gladly share my code.


 
 Oooo.. You should put that on dsource or somewhere. Hacking D sounds
 like a lot more fun than hacking C++. I wouldn't mind helping out on
 this one.


Guys please use D2 :o).

Andrei
Oct 11 2009
parent "Denis Koroskin" <2korden gmail.com> writes:
On Sun, 11 Oct 2009 19:10:42 +0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 Ellery Newcomer wrote:

 Denis Koroskin wrote:

 On Sun, 11 Oct 2009 05:19:56 +0400, Walter Bright
 <newshound1 digitalmars.com> wrote:

 If anyone is interested and is willing to test and/or help, I will
 gladly share my code.


  Oooo.. You should put that on dsource or somewhere. Hacking D sounds
 like a lot more fun than hacking C++. I wouldn't mind helping out on
 this one.


 Guys please use D2 :o).

 Andrei


I do, it's written entirely in D2 :)
Oct 11 2009
prev sibling parent reply language_fan <foo bar.com.invalid> writes:
Sun, 11 Oct 2009 09:39:32 -0500, Ellery Newcomer thusly wrote:


 Denis Koroskin wrote:

 On Sun, 11 Oct 2009 05:19:56 +0400, Walter Bright
 <newshound1 digitalmars.com> wrote:
 
 If anyone is interested and is willing to test and/or help, I will
 gladly share my code.


 
 Oooo.. You should put that on dsource or somewhere. Hacking D sounds
 like a lot more fun than hacking C++. I wouldn't mind helping out on
 this one.


Wow, I am pretty sure I have already seen a D port of dmd on dsource. Was 
there a good reason to start yet another port of it?
Oct 12 2009
next sibling parent Jeremie Pelletier <jeremiep gmail.com> writes:
language_fan wrote:

 Sun, 11 Oct 2009 09:39:32 -0500, Ellery Newcomer thusly wrote:
 

 Denis Koroskin wrote:

 On Sun, 11 Oct 2009 05:19:56 +0400, Walter Bright
 <newshound1 digitalmars.com> wrote:

 If anyone is interested and is willing to test and/or help, I will
 gladly share my code.


 Oooo.. You should put that on dsource or somewhere. Hacking D sounds
 like a lot more fun than hacking C++. I wouldn't mind helping out on
 this one.


 
 Wow, I am pretty sure I have already seen a D port of dmd on dsource. Was 
 there a good reason to start yet another port of it?


Maybe learning, porting a library or application from a language to 
another is a great way to learn about it inside out. I used to do that a 
lot too.
Oct 12 2009
prev sibling parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Mon, 12 Oct 2009 12:58:56 +0400, language_fan <foo bar.com.invalid>  
wrote:


 Sun, 11 Oct 2009 09:39:32 -0500, Ellery Newcomer thusly wrote:


 Denis Koroskin wrote:

 On Sun, 11 Oct 2009 05:19:56 +0400, Walter Bright
 <newshound1 digitalmars.com> wrote:

 If anyone is interested and is willing to test and/or help, I will
 gladly share my code.


 Oooo.. You should put that on dsource or somewhere. Hacking D sounds
 like a lot more fun than hacking C++. I wouldn't mind helping out on
 this one.


 Wow, I am pretty sure I have already seen a D port of dmd on dsource. Was
 there a good reason to start yet another port of it?


I know about DParser, but it's somewhat outdated, incomplete and doesn't  
support code generation. It also diverged from DMD quite a lot so fixing  
it would be quite hard (I, too, plan to make it more D-ish - get rid of  
casts, replace void* "Array"s with type-safe equivalents etc - but not  
until it is 100% ready).

I just believe rewriting it from scratch will be plain faster. I can also  
test the code much easier - if it produces exact same binary then it works  
correct.
Oct 12 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Denis Koroskin wrote:

 I just believe rewriting it from scratch will be plain faster. I can 
 also test the code much easier - if it produces exact same binary then 
 it works correct.


The dmd front end is also written in a "D-ish" style which should make 
translation to D easier.
Oct 12 2009
parent reply grauzone <none example.net> writes:
Walter Bright wrote:

 Denis Koroskin wrote:

 I just believe rewriting it from scratch will be plain faster. I can 
 also test the code much easier - if it produces exact same binary then 
 it works correct.


 
 The dmd front end is also written in a "D-ish" style which should make 
 translation to D easier.


But some stuff doesn't quite match: for example, Expression is declared 
in expression.h. But not all methods are implemented in expression.c. 
There's at least the interpret() method, which is in interpret.c.

How would you translate this into D? Just mash everything into a single 
.d source file?
Oct 12 2009
parent Walter Bright <newshound1 digitalmars.com> writes:
grauzone wrote:

 But some stuff doesn't quite match: for example, Expression is declared 
 in expression.h. But not all methods are implemented in expression.c. 
 There's at least the interpret() method, which is in interpret.c.
 
 How would you translate this into D? Just mash everything into a single 
 .d source file?


Oh, I agree that the files would have to be organized differently. One 
giant file is a little extreme, though <g>. Probably one module per 
class would be better.
Oct 12 2009
prev sibling next sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Denis Koroskin wrote:

 In fact, that's what I do right now.


I think that's great. But it requires a lot of work (as of course you know).
Oct 11 2009
parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
Walter Bright wrote:

 Denis Koroskin wrote:

 In fact, that's what I do right now.


 
 I think that's great. But it requires a lot of work (as of course you 
 know).


Good things require a lot of work. :)
Oct 11 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Ary Borenszweig wrote:

 Walter Bright wrote:

 Denis Koroskin wrote:

 In fact, that's what I do right now.


 I think that's great. But it requires a lot of work (as of course you 
 know).


 
 Good things require a lot of work. :)


Of course. But getting something done and available in a short amount of 
time is also good!

For example, some people say to me "why should I use D, when C++0x 
solves my issues with C++?"

And I point out "how many more years are you willing to wait for C++0x, 
when D is here now?"

[Setting aside for the moment the issue of whether C++0x really is 
better than D or not!]

The point is, good enough now gets us further down the road of getting a 
user base large enough to justify the effort for a more comprehensive 
solution.

For example, if I have many miles to commute to work, the best choice is 
a car. But if I can't afford a car, or am too young to drive, a bike at 
least gets me there. Before I was 16 and could drive, a bike was very 
liberating for me, I rode it everywhere.
Oct 11 2009
next sibling parent Jacob Carlborg <doob me.com> writes:
On 10/11/09 21:38, Walter Bright wrote:

 Ary Borenszweig wrote:

 Walter Bright wrote:

 Denis Koroskin wrote:

 In fact, that's what I do right now.


 I think that's great. But it requires a lot of work (as of course you
 know).


 Good things require a lot of work. :)


 Of course. But getting something done and available in a short amount of
 time is also good!

 For example, some people say to me "why should I use D, when C++0x
 solves my issues with C++?"

 And I point out "how many more years are you willing to wait for C++0x,
 when D is here now?"

 [Setting aside for the moment the issue of whether C++0x really is
 better than D or not!]

 The point is, good enough now gets us further down the road of getting a
 user base large enough to justify the effort for a more comprehensive
 solution.


I think descent is really good now.


 For example, if I have many miles to commute to work, the best choice is
 a car. But if I can't afford a car, or am too young to drive, a bike at
 least gets me there. Before I was 16 and could drive, a bike was very
 liberating for me, I rode it everywhere.
Oct 11 2009
prev sibling parent reply "Nick Sabalausky" <a a.a> writes:
"Walter Bright" <newshound1 digitalmars.com> wrote in message 
news:hatc8c$277u$1 digitalmars.com...

 Ary Borenszweig wrote:

 Walter Bright wrote:

 Denis Koroskin wrote:

 In fact, that's what I do right now.


 I think that's great. But it requires a lot of work (as of course you 
 know).


 Good things require a lot of work. :)


 Of course. But getting something done and available in a short amount of 
 time is also good!

 For example, some people say to me "why should I use D, when C++0x solves 
 my issues with C++?"

 And I point out "how many more years are you willing to wait for C++0x, 
 when D is here now?"

 [Setting aside for the moment the issue of whether C++0x really is better 
 than D or not!]

 The point is, good enough now gets us further down the road of getting a 
 user base large enough to justify the effort for a more comprehensive 
 solution.

 For example, if I have many miles to commute to work, the best choice is a 
 car. But if I can't afford a car, or am too young to drive, a bike at 
 least gets me there. Before I was 16 and could drive, a bike was very 
 liberating for me, I rode it everywhere.


It's a good point, but, if I'm reading this group right, it sounds like it 
might not be quite as useful as you're expecting it would be.
Oct 11 2009
parent Walter Bright <newshound1 digitalmars.com> writes:
Nick Sabalausky wrote:

 It's a good point, but, if I'm reading this group right, it sounds like it 
 might not be quite as useful as you're expecting it would be.


Yes, I can see that.
Oct 11 2009
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 10/11/09 16:32, Denis Koroskin wrote:

 On Sun, 11 Oct 2009 05:19:56 +0400, Walter Bright
 <newshound1 digitalmars.com> wrote:


 In my discussions with companies about adopting D, the major barrier
 that comes up over and over isn't Tango vs Phobos, dmd being GPL,
 debugger support, libraries, bugs, etc., although those are important.

 It's the IDE.

 They say that the productivity gains of D's improvements are
 overbalanced by the loss of productivity by moving away from an IDE.
 And what is it about an IDE that is so productive? Intellisense
 (Microsoft's word for autocompletion).

 So, while I'm not going to be writing an IDE, I figure that dmd can
 help. dmd already puts out .doc and .di files. How about putting out
 an xml file giving all the information needed for an IDE to implement
 autocompletion? There'd be one .xml file generated per .d source file.

 The nice thing about an xml file is while D is relatively easy to
 parse, xml is trivial. Furthermore, an xml format would be fairly
 robust in the face of changes to D syntax.

 What do you think?


 I believe it won't work. It will always be slow and incomplete.

 Instead, I would make it easier to embed DMD into an IDE: separate DMDFE
 from DMDBE, fix memory leaks, remove all the static data (so that code
 would be re-intrable and it could work in different threads in
 parallel), move most of DMD code into a DLL so that an IDE could
 dynamically link with it and whatever it pleases with the source code.

 In fact, that's what I do right now.

 I'm writing my own D IDE in my spare time (in D, of course). I already
 made a lot of progress and now it's time to start implementing source
 code analysis.

 First thing I did is I made complete D bindings for C++ code. It worked
 out quite well but it was leaking so badly that I dropped it.

 Instead, I started porting DMD entirely to D (except the backend, of
 course), and I already got some great results. A few simple programs
 compile and produce byte-perfect binaries. It's still in its early
 stages and there is a lot of work to do, but it will be finished soon
 (hopefully). It could probably become a part of an official
 distribution, eventually. :)

 If anyone is interested and is willing to test and/or help, I will
 gladly share my code.


Sounds interesting, put it online somewhere.
Oct 11 2009
prev sibling next sibling parent "Phil Deets" <pjdeets2 gmail.com> writes:
On Sun, 11 Oct 2009 09:32:32 -0500, Denis Koroskin <2korden gmail.com>  
wrote:


 On Sun, 11 Oct 2009 05:19:56 +0400, Walter Bright  
 <newshound1 digitalmars.com> wrote:


 In my discussions with companies about adopting D, the major barrier  
 that comes up over and over isn't Tango vs Phobos, dmd being GPL,  
 debugger support, libraries, bugs, etc., although those are important.

 It's the IDE.

 They say that the productivity gains of D's improvements are  
 overbalanced by the loss of productivity by moving away from an IDE.  
 And what is it about an IDE that is so productive? Intellisense  
 (Microsoft's word for autocompletion).

 So, while I'm not going to be writing an IDE, I figure that dmd can  
 help. dmd already puts out .doc and .di files. How about putting out an  
 xml file giving all the information needed for an IDE to implement  
 autocompletion? There'd be one .xml file generated per .d source file.

 The nice thing about an xml file is while D is relatively easy to  
 parse, xml is trivial. Furthermore, an xml format would be fairly  
 robust in the face of changes to D syntax.

 What do you think?


 I believe it won't work. It will always be slow and incomplete.

 Instead, I would make it easier to embed DMD into an IDE: separate DMDFE  
  from DMDBE, fix memory leaks, remove all the static data (so that code  
 would be re-intrable and it could work in different threads in  
 parallel), move most of DMD code into a DLL so that an IDE could  
 dynamically link with it and whatever it pleases with the source code.

 In fact, that's what I do right now.

 I'm writing my own D IDE in my spare time (in D, of course). I already  
 made a lot of progress and now it's time to start implementing source  
 code analysis.

 First thing I did is I made complete D bindings for C++ code. It worked  
 out quite well but it was leaking so badly that I dropped it.

 Instead, I started porting DMD entirely to D (except the backend, of  
 course), and I already got some great results. A few simple programs  
 compile and produce byte-perfect binaries. It's still in its early  
 stages and there is a lot of work to do, but it will be finished soon  
 (hopefully). It could probably become a part of an official  
 distribution, eventually. :)

 If anyone is interested and is willing to test and/or help, I will  
 gladly share my code.


I would like it if this went open so I could examine it and possibly  
contribute to it. I have wanted to do something like this, but all I have  
started so far is a GUI toolkit. I am new to D so I have not spent much  
time on it yet. It would be nice to be able to work on something more  
developed.

-- 
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
Oct 12 2009
prev sibling parent "Sableteeth" <sableteeth gmail.com> writes:
	charset="utf-8"
Content-Transfer-Encoding: quoted-printable

"Denis Koroskin" <2korden gmail.com> =
=D1=81=D0=BE=D0=BE=D0=B1=D1=89=D0=B8=D0=BB/=D1=81=D0=BE=D0=BE=D0=B1=D1=89=
=D0=B8=D0=BB=D0=B0 =D0=B2 =
=D0=BD=D0=BE=D0=B2=D0=BE=D1=81=D1=82=D1=8F=D1=85 =
=D1=81=D0=BB=D0=B5=D0=B4=D1=83=D1=8E=D1=89=D0=B5=D0=B5: =
news:op.u1m30ituo7cclz korden-pc...

 On Sun, 11 Oct 2009 05:19:56 +0400, Walter Bright =20
 <newshound1 digitalmars.com> wrote:
=20
 I believe it won't work. It will always be slow and incomplete.
=20
 Instead, I would make it easier to embed DMD into an IDE: separate =


DMDFE =20

 from DMDBE, fix memory leaks, remove all the static data (so that code =


=20

 would be re-intrable and it could work in different threads in =


parallel), =20

 move most of DMD code into a DLL so that an IDE could dynamically link =


=20

 with it and whatever it pleases with the source code.
=20
 In fact, that's what I do right now.
=20
 I'm writing my own D IDE in my spare time (in D, of course). I already =


=20

 made a lot of progress and now it's time to start implementing source =


code =20

 analysis.
=20
 First thing I did is I made complete D bindings for C++ code. It =


worked =20

 out quite well but it was leaking so badly that I dropped it.
=20
 Instead, I started porting DMD entirely to D (except the backend, of =20
 course), and I already got some great results. A few simple programs =20
 compile and produce byte-perfect binaries. It's still in its early =


stages =20

 and there is a lot of work to do, but it will be finished soon =20
 (hopefully). It could probably become a part of an official =


distribution, =20

 eventually. :)
=20
 If anyone is interested and is willing to test and/or help, I will =


gladly =20

 share my code.


Hi, Denis!

I am beginner in D, but this theme is interesting for me - I very much =
would like to take part in creation IDE.=20

I am interesting. I'd like to help.
I hope I'll be able to help you?

Contact me please at sableteeth gmail.com
Sincerelly, Sergey




__________ Information from ESET NOD32 Antivirus, version of virus signatur=
e database 4862 (20100212) __________

The message was checked by ESET NOD32 Antivirus.

http://www.eset.com
Apr 07 2010
prev sibling next sibling parent reply BLS <windevguy hotmail.de> writes:
On 11/10/2009 03:19, Walter Bright wrote:

 In my discussions with companies about adopting D, the major barrier
 that comes up over and over isn't Tango vs Phobos, dmd being GPL,
 debugger support, libraries, bugs, etc., although those are important.

 It's the IDE.

 They say that the productivity gains of D's improvements are
 overbalanced by the loss of productivity by moving away from an IDE. And
 what is it about an IDE that is so productive? Intellisense (Microsoft's
 word for autocompletion).

 So, while I'm not going to be writing an IDE, I figure that dmd can
 help. dmd already puts out .doc and .di files. How about putting out an
 xml file giving all the information needed for an IDE to implement
 autocompletion? There'd be one .xml file generated per .d source file.

 The nice thing about an xml file is while D is relatively easy to parse,
 xml is trivial. Furthermore, an xml format would be fairly robust in the
 face of changes to D syntax.

 What do you think?


FANTASTIC! Give it a go.

Despite the intention to support a D IDE ...
I would make sense to establish this feature in D1 too.
Imagine XML2D which may help to port D1 code to D2.

I would further suggest to consider to output to Oracle Berkeley DB XML ..
Having an XQuery-based access to D-sources.. I guess you can imagine 
what this could mean.

There are a couple of D IDEs around.
I think what people mean with there is no D IDE is simply : "We want a 
MS Visual Studio like IDE as part of the D tool chain" One download, 
that's it.
I agree. But we need a platform independent IDE.

Our Options :
1) Force Decent.. (Java/SWT)

2) Use gtkD, create the IDE in D. (the only usable platform independent 
  GUI atm.)

3) Use QT and C++. Let's re-use and adapt QT Creator.

Having QT as standard GUI toolkit for D means that we can also reuse the 
QT Designer.

Bjoern
Oct 11 2009
next sibling parent Lutger <lutger.blijdestijn gmail.com> writes:
BLS wrote:

...

 Our Options :
 1) Force Decent.. (Java/SWT)
 
 2) Use gtkD, create the IDE in D. (the only usable platform independent
   GUI atm.)
 
 3) Use QT and C++. Let's re-use and adapt QT Creator.
 
 Having QT as standard GUI toolkit for D means that we can also reuse the
 QT Designer.
 
 Bjoern


Why not QtD?
Oct 11 2009
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 10/11/09 16:38, BLS wrote:

 On 11/10/2009 03:19, Walter Bright wrote:

 In my discussions with companies about adopting D, the major barrier
 that comes up over and over isn't Tango vs Phobos, dmd being GPL,
 debugger support, libraries, bugs, etc., although those are important.

 It's the IDE.

 They say that the productivity gains of D's improvements are
 overbalanced by the loss of productivity by moving away from an IDE. And
 what is it about an IDE that is so productive? Intellisense (Microsoft's
 word for autocompletion).

 So, while I'm not going to be writing an IDE, I figure that dmd can
 help. dmd already puts out .doc and .di files. How about putting out an
 xml file giving all the information needed for an IDE to implement
 autocompletion? There'd be one .xml file generated per .d source file.

 The nice thing about an xml file is while D is relatively easy to parse,
 xml is trivial. Furthermore, an xml format would be fairly robust in the
 face of changes to D syntax.

 What do you think?


 FANTASTIC! Give it a go.

 Despite the intention to support a D IDE ...
 I would make sense to establish this feature in D1 too.
 Imagine XML2D which may help to port D1 code to D2.

 I would further suggest to consider to output to Oracle Berkeley DB XML ..
 Having an XQuery-based access to D-sources.. I guess you can imagine
 what this could mean.

 There are a couple of D IDEs around.
 I think what people mean with there is no D IDE is simply : "We want a
 MS Visual Studio like IDE as part of the D tool chain" One download,
 that's it.
 I agree. But we need a platform independent IDE.

 Our Options :
 1) Force Decent.. (Java/SWT)


I think this the our best option now


 2) Use gtkD, create the IDE in D. (the only usable platform independent
 GUI atm.)

 3) Use QT and C++. Let's re-use and adapt QT Creator.

 Having QT as standard GUI toolkit for D means that we can also reuse the
 QT Designer.


As far as I know neither Qt(d) or gtkD uses native controls on platforms 
other than linux, which to me is unacceptable. The look especially on mac.

4)
Port entire eclipse to D, Frank Benoit (who started DWT, the tango 
version) was thinking of this.


 Bjoern
Oct 11 2009
next sibling parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Mon, 12 Oct 2009 00:36:01 +0400, Jacob Carlborg <doob me.com> wrote:


 4)
 Port entire eclipse to D, Frank Benoit (who started DWT, the tango  
 version) was thinking of this.


 Bjoern




What's the benefit? I believe it'll become even slower. Much slower.
Oct 11 2009
parent Jacob Carlborg <doob me.com> writes:
On 10/11/09 22:37, Denis Koroskin wrote:

 On Mon, 12 Oct 2009 00:36:01 +0400, Jacob Carlborg <doob me.com> wrote:


 4)
 Port entire eclipse to D, Frank Benoit (who started DWT, the tango
 version) was thinking of this.


 Bjoern




 What's the benefit? I believe it'll become even slower. Much slower.


Maybe we could try to slim it down, remove things that aren't needed.
Oct 11 2009
prev sibling parent Lutger <lutger.blijdestijn gmail.com> writes:
Jacob Carlborg wrote:
...

 As far as I know neither Qt(d) or gtkD uses native controls on platforms
 other than linux, which to me is unacceptable. The look especially on mac.


Qt used to try and look like native controls, but now it uses them directly.
Oct 11 2009
prev sibling next sibling parent reply language_fan <foo bar.com.invalid> writes:
Sat, 10 Oct 2009 18:19:56 -0700, Walter Bright thusly wrote:


 In my discussions with companies about adopting D, the major barrier
 that comes up over and over isn't Tango vs Phobos, dmd being GPL,
 debugger support, libraries, bugs, etc., although those are important.
 
 It's the IDE.
 
 They say that the productivity gains of D's improvements are
 overbalanced by the loss of productivity by moving away from an IDE. And
 what is it about an IDE that is so productive? Intellisense (Microsoft's
 word for autocompletion).
 
 So, while I'm not going to be writing an IDE, I figure that dmd can
 help. dmd already puts out .doc and .di files. How about putting out an
 xml file giving all the information needed for an IDE to implement
 autocompletion? There'd be one .xml file generated per .d source file.
 
 The nice thing about an xml file is while D is relatively easy to parse,
 xml is trivial. Furthermore, an xml format would be fairly robust in the
 face of changes to D syntax.
 
 What do you think?


Another point not mentioned here is that modern IDEs use incremental and 
interactive compilation model. The compiler should be run as a persistent 
background process and parsing should happen perhaps on the level of the 
current scope. Re-compiling a million line project after each key stroke 
simply makes no sense. Even compiling the current module once per key 
stroke is too slow.

Specifying an intermediate json/xml file format is a huge task 
considering the amount of language constructs, types etc. available in D.

I'm all for good tool support but as many have already mentioned, the 
support would only bring marginal improvements to small scale tools like 
vim and emacs. Usually small scale D projects (< 10000 lines of code) are 
written with those tools (feel free to prove me wrong). These are not the 
kinds of projects large enterprises would use D for, they use scripting 
languages for smaller tasks. Thus the overall improvement is minimal. 
Spending the time on critical compiler bugs on the other hand would help 
everyone in the community.
Oct 12 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
language_fan wrote:

 Another point not mentioned here is that modern IDEs use incremental and 
 interactive compilation model. The compiler should be run as a persistent 
 background process and parsing should happen perhaps on the level of the 
 current scope. Re-compiling a million line project after each key stroke 
 simply makes no sense.


This would not require recompiling a million lines with every key 
stroke, unless you are editing a million line module.


 Even compiling the current module once per key 
 stroke is too slow.


As you say, it should be done as a background process.


 Specifying an intermediate json/xml file format is a huge task 
 considering the amount of language constructs, types etc. available in D.


It isn't. It's far less work than ddoc is, for example.


 I'm all for good tool support but as many have already mentioned, the 
 support would only bring marginal improvements to small scale tools like 
 vim and emacs. Usually small scale D projects (< 10000 lines of code) are 
 written with those tools (feel free to prove me wrong). These are not the 
 kinds of projects large enterprises would use D for, they use scripting 
 languages for smaller tasks. Thus the overall improvement is minimal.


I think the bang for the buck on this is large.



 Spending the time on critical compiler bugs on the other hand would help 
 everyone in the community.


That hasn't been shorted, look at the changelog!
Oct 12 2009
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Walter Bright wrote:

 language_fan wrote:

 Another point not mentioned here is that modern IDEs use incremental 
 and interactive compilation model. The compiler should be run as a 
 persistent background process and parsing should happen perhaps on the 
 level of the current scope. Re-compiling a million line project after 
 each key stroke simply makes no sense.


 
 This would not require recompiling a million lines with every key 
 stroke, unless you are editing a million line module.
 

 Even compiling the current module once per key stroke is too slow.


 
 As you say, it should be done as a background process.
 

 Specifying an intermediate json/xml file format is a huge task 
 considering the amount of language constructs, types etc. available in D.


 
 It isn't. It's far less work than ddoc is, for example.
 

 I'm all for good tool support but as many have already mentioned, the 
 support would only bring marginal improvements to small scale tools 
 like vim and emacs. Usually small scale D projects (< 10000 lines of 
 code) are written with those tools (feel free to prove me wrong). 
 These are not the kinds of projects large enterprises would use D for, 
 they use scripting languages for smaller tasks. Thus the overall 
 improvement is minimal.


 
 I think the bang for the buck on this is large.


For each snippet of code that doesn't currently compile, I generate a 
red warning in the TDPL draft. Currently there are 28 such red warnings, 
and each may be arbitrarily difficult to fix. There are other issues 
that we know need to be done as soon as yesterday.

IMHO it would be frivolous to spend time on anything else but the 28 
bugs. This XML/JSON generation is like combing one's hair before leaving 
the burning house. Just run! (I'm not saying I don't like combed hair or 
XML/JSON parsing, but the latter is absolutely nothing you need to work 
on now.) Please understand that TDPL is on a crash course and we can't 
have a book without a language (I'm also assuming we can't have a 
language without a book).

Walter, please avoid all distractions and make bringing D in sync with 
the book your sole preoccupation. I am working *extremely* hard on the 
book, and I wish I were seeing the same level of commitment in you.


Andrei
Oct 12 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Andrei Alexandrescu wrote:

 Walter Bright wrote:

 language_fan wrote:

 Another point not mentioned here is that modern IDEs use incremental 
 and interactive compilation model. The compiler should be run as a 
 persistent background process and parsing should happen perhaps on 
 the level of the current scope. Re-compiling a million line project 
 after each key stroke simply makes no sense.


 This would not require recompiling a million lines with every key 
 stroke, unless you are editing a million line module.


 Even compiling the current module once per key stroke is too slow.


 As you say, it should be done as a background process.


 Specifying an intermediate json/xml file format is a huge task 
 considering the amount of language constructs, types etc. available 
 in D.


 It isn't. It's far less work than ddoc is, for example.


 I'm all for good tool support but as many have already mentioned, the 
 support would only bring marginal improvements to small scale tools 
 like vim and emacs. Usually small scale D projects (< 10000 lines of 
 code) are written with those tools (feel free to prove me wrong). 
 These are not the kinds of projects large enterprises would use D 
 for, they use scripting languages for smaller tasks. Thus the overall 
 improvement is minimal.


 I think the bang for the buck on this is large.


 
 For each snippet of code that doesn't currently compile, I generate a 
 red warning in the TDPL draft. Currently there are 28 such red warnings, 
 and each may be arbitrarily difficult to fix. There are other issues 
 that we know need to be done as soon as yesterday.
 
 IMHO it would be frivolous to spend time on anything else but the 28 
 bugs. This XML/JSON generation is like combing one's hair before leaving 
 the burning house. Just run! (I'm not saying I don't like combed hair or 
 XML/JSON parsing, but the latter is absolutely nothing you need to work 
 on now.) Please understand that TDPL is on a crash course and we can't 
 have a book without a language (I'm also assuming we can't have a 
 language without a book).
 
 Walter, please avoid all distractions and make bringing D in sync with 
 the book your sole preoccupation. I am working *extremely* hard on the 
 book, and I wish I were seeing the same level of commitment in you.
 
 
 Andrei


I ammend the above: s/commitment/focus/. Of course Walter is motivate 
and committed more than anyone else to make D successful, but it's time 
to force ourselves to absolutely focus on the important _and_ urgent 
matters.

Andrei
Oct 12 2009
prev sibling parent reply Matthias Fauconneau <matthias.fauconneau gmail.com> writes:
I agree D IDE support is very bad, I was coding in D with Kate and a modified
build plugin.
After programming a project using C++/Qt and a real IDE (QtCreator), I couldn't
go back to D without an IDE so I decided to create my own.
Using existing components like KTextEditor and QDockWidget, it is relatively
easy to get a basic environment.
But source code analysis is not as easy.
I used a powerful parser generator (coincidentally named D_Parser) which allow
me to parse D using a simple grammar(~200 lines).
It parses as you type, report syntax errors, provide an outliner and let you
follow symbols.

Screenshot -> http://matt2000.free.fr/kreator.png
you can go to the definition of the underlined expression

Source -> http://matt2000.free.fr/kreator.zip
depends on kdelibs and dparser (included)
Oct 18 2009
parent reply Matthias Pleh <matthias.pleh gmx.at> writes:
Matthias Fauconneau schrieb:

 I agree D IDE support is very bad, I was coding in D with Kate and a modified
build plugin.
 After programming a project using C++/Qt and a real IDE (QtCreator), I
couldn't go back to D without an IDE so I decided to create my own.
 Using existing components like KTextEditor and QDockWidget, it is relatively
easy to get a basic environment.
 But source code analysis is not as easy.
 I used a powerful parser generator (coincidentally named D_Parser) which allow
me to parse D using a simple grammar(~200 lines).
 It parses as you type, report syntax errors, provide an outliner and let you
follow symbols.
 
 Screenshot -> http://matt2000.free.fr/kreator.png
 you can go to the definition of the underlined expression
 
 Source -> http://matt2000.free.fr/kreator.zip
 depends on kdelibs and dparser (included)
 


Thanks for your code! The grammar was exactly what I was looking for. I 
will try to implement it in my IDE to support code-completion! (a plugin 
for SharpDevelop)

Matthias
Oct 21 2009
parent reply Matthias Fauconneau <matthias.fauconneau gmail.com> writes:
Matthias Pleh Wrote:

 Thanks for your code! The grammar was exactly what I was looking for. I 
 will try to implement it in my IDE to support code-completion! (a plugin 
 for SharpDevelop)
 
 Matthias


Are you going to use D_Parser ?
If you are using a weaker parser, you may be interested in
http://seatd.mainia.de/grammar.html
Oct 21 2009
parent reply Matthias Pleh <matthias.pleh gmx.at> writes:
Matthias Fauconneau schrieb:

 Matthias Pleh Wrote:

 Thanks for your code! The grammar was exactly what I was looking for. I 
 will try to implement it in my IDE to support code-completion! (a plugin 
 for SharpDevelop)

 Matthias


 
 Are you going to use D_Parser ?
 If you are using a weaker parser, you may be interested in
http://seatd.mainia.de/grammar.html
 


I'm not so familiar with parser generator, so I was looking for a 
finished grammar for any generator.
I was glad to find this grammar!

haven't found a grammar for it.



I will have a look at the grammar you mentioned!
Oct 22 2009
parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
Matthias Pleh wrote:

 Matthias Fauconneau schrieb:

 Matthias Pleh Wrote:

 Thanks for your code! The grammar was exactly what I was looking for.
 I will try to implement it in my IDE to support code-completion! (a
 plugin for SharpDevelop)

 Matthias


 Are you going to use D_Parser ?
 If you are using a weaker parser, you may be interested in
 http://seatd.mainia.de/grammar.html


 
 I'm not so familiar with parser generator, so I was looking for a
 finished grammar for any generator.
 I was glad to find this grammar!

 haven't found a grammar for it.








 
 I will have a look at the grammar you mentioned!


That grammar looks okay, but I've counted at least 3 nitpicky problems,
and I didn't read the whole thing (nor very closely)
Oct 22 2009
parent reply Matthias Fauconneau <matthias.fauconneau gmail.com> writes:
 I'm not so familiar with parser generator, so I was looking for a finished
grammar for any generator.




This is the only other D grammar I know of.
It was made for ApageD (a discontinued parser project written in D) but it
should be compatible with most parser generators.








If you want I could create a C library with the code analysis part of my IDE.

I'm currently providing a symbol table and links from a symbol use to
definition.
The IDE use it to display an outliner and for a "Go to Definition" action.
Now, I'm currently coding the handling of expression (i.e finding the type of
each subexpression) which will be used for code completion





I didn't try Coco/R or AntLR but I don't think they would let you specify the D
syntax so concisely.


 That grammar looks okay, but I've counted at least 3 nitpicky problems,
 and I didn't read the whole thing (nor very closely)


I find this grammar is quite lengthy and hard to read, probably because it had
to be accepted by a simple parser generator.
It doesn't benefit from the main advantages of using a parser generator which
is simplicity and maintainability.
Oct 22 2009
next sibling parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
Matthias Fauconneau wrote:

 I didn't try Coco/R or AntLR but I don't think they would let you specify the
D syntax so concisely.


Quite true. Your d_parser grammar is incredibly short (as opposed to my
ANTLR grammar which is about 10 times larger). I bet d_parser is GLR?

And I don't mean to be nitpicky again, but it looks like it is missing
even more than the other grammar.

 

 That grammar looks okay, but I've counted at least 3 nitpicky problems,
 and I didn't read the whole thing (nor very closely)


 
 I find this grammar is quite lengthy and hard to read, probably because it had
to be accepted by a simple parser generator.
 It doesn't benefit from the main advantages of using a parser generator which
is simplicity and maintainability.
 


Well, D is a lengthy language...
Oct 22 2009
parent reply Matthias Fauconneau <matthias.fauconneau gmail.com> writes:
 Quite true. Your d_parser grammar is incredibly short (as opposed to my


ANTLR grammar which is about 10 times larger).

It's good I have chosen D_Parser then, 10 times is worth it.
I hope it wont get much larger when I complete it.


I bet d_parser is GLR?


it's a scannerless GLR parser


 And I don't mean to be nitpicky again,


No problem, I posted my project so I can get critics.


but it looks like it is missing even more than the other grammar.


It did parse all my code but I had forgotten to disable automatic
disambiguation (using greediness and height).
Now, I've been disambiguating my grammar.
I managed to get operator priorities and associativities working (the D_Parser
manual is not very verbose on this)
but I'm running into some problems:

is D syntax supposed to be unambiguous ?

According to http://digitalmars.com/d/2.0/lex.html: "Each phase has no
dependence on subsequent phases"
and it lists syntax analysis and semantic analysis as separate phase.

"type* id;" is either a declaration or a multiplication and I don't see how to
disambiguate this without a symbol table.

this case is ambiguous even with the symbol table:
"{ statements }  .ptr" could evaluate to the function literal context pointer
or a scope + a call to ptr() defined at module scope
Oct 22 2009
parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
If you like, you can have an eyeball at my grammar:

http://www.dsource.org/projects/antlrd/browser/toys/v3d

note it's a port from a v2 grammar (which definitely works) and hasn't
been so extensively tested (or finished, for that matter..)

Matthias Fauconneau wrote:

 It did parse all my code but I had forgotten to disable automatic
disambiguation (using greediness and height).
 Now, I've been disambiguating my grammar.


Now try parsing the standard libraries.

Here's a few things I noticed were missing:

* in/out/body blocks
* comma expressions
* c-style types
* anonymous classes
* parameters can be variadic and/or have no variable symbol
* unions should be able to take template arguments



 I managed to get operator priorities and associativities working (the D_Parser
manual is not very verbose on this)
 but I'm running into some problems:
 
 is D syntax supposed to be unambiguous ?


Ha! No.


 
 According to http://digitalmars.com/d/2.0/lex.html: "Each phase has no
dependence on subsequent phases"
 and it lists syntax analysis and semantic analysis as separate phase.
 
 "type* id;" is either a declaration or a multiplication and I don't see how to
disambiguate this without a symbol table.


convention: it's always a declaration (since as an expression it
wouldn't have any effect, and we don't care about overloading. Or so I
assume). There are other cases that do get resolved during semantic.


 
 this case is ambiguous even with the symbol table:


they're aplenty


 "{ statements }  .ptr" could evaluate to the function literal context pointer
or a scope + a call to ptr() defined at module scope
 


Good one. I hadn't noticed that one before. Looks like the compiler will
go with the second option.
Oct 22 2009
prev sibling parent reply Matthias Pleh <matthias.pleh gmx.at> writes:
Hello Matthias

Matthias Fauconneau schrieb:

 I'm not so familiar with parser generator, so I was looking for a finished
grammar for any generator.




 This is the only other D grammar I know of.
 It was made for ApageD (a discontinued parser project written in D) but it
should be compatible with most parser generators.
 







 If you want I could create a C library with the code analysis part of my IDE.

 I'm currently providing a symbol table and links from a symbol use to
definition.
 The IDE use it to display an outliner and for a "Go to Definition" action.
 Now, I'm currently coding the handling of expression (i.e finding the type of
each subexpression) which will be used for code completion



and the grammar is written in simple BNF.
I've tried to translate your grammar and the ANTLR-grammar from Ellery, 
but I was confronted with many reduce-reduce conflicts and I can't fix 
all of them.
So I try to create a grammar from scratch. But when the execution get to 
  slow, your option with the c-library will be a good option!
Maybe I come back to your offer! :)





 I didn't try Coco/R or AntLR but I don't think they would let you specify the
D syntax so concisely.
 

 That grammar looks okay, but I've counted at least 3 nitpicky problems,
 and I didn't read the whole thing (nor very closely)


 
 I find this grammar is quite lengthy and hard to read, probably because it had
to be accepted by a simple parser generator.
 It doesn't benefit from the main advantages of using a parser generator which
is simplicity and maintainability.
Oct 29 2009
parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
Matthias Pleh wrote:

 Hello Matthias
 
 Matthias Fauconneau schrieb:

 I'm not so familiar with parser generator, so I was looking for a
 finished grammar for any generator.




 This is the only other D grammar I know of.
 It was made for ApageD (a discontinued parser project written in D)
 but it should be compatible with most parser generators.








 If you want I could create a C library with the code analysis part of

 I'm currently providing a symbol table and links from a symbol use to
 definition.
 The IDE use it to display an outliner and for a "Go to Definition"
 action.
 Now, I'm currently coding the handling of expression (i.e finding the
 type of each subexpression) which will be used for code completion


 

 and the grammar is written in simple BNF.
 I've tried to translate your grammar and the ANTLR-grammar from Ellery,
 but I was confronted with many reduce-reduce conflicts and I can't fix
 all of them.
 So I try to create a grammar from scratch. But when the execution get to
  slow, your option with the c-library will be a good option!
 Maybe I come back to your offer! :)
 


You are aware that those reduce-reduce conflicts are inherent to D,
right? As in if you can't fix them in my grammar, you probably won't be
able to fix them in yours.

Also, it took me maybe 6 months of reading through DMD's source to get
something accurate together, and munging through syntactic and semantic
ambiguities was more or less trivial (easier than in GOLD, from what I
understand, though I haven't used GOLD), for what that's worth.

But never mind me. I'm just a big pessimist. Good luck with your grammar!
Oct 29 2009
parent reply Matthias Pleh <matthias.pleh gmx.at> writes:
Hello Ellery


 
 You are aware that those reduce-reduce conflicts are inherent to D,
 right? As in if you can't fix them in my grammar, you probably won't be
 able to fix them in yours.


Yes of course, you are right!
But I couldn't fix them because it's not technically feasible, but I'm 
not so familair with your grammar and I can't overlook the whole grammar
at once.


 
 Also, it took me maybe 6 months of reading through DMD's source to get
 something accurate together, and munging through syntactic and semantic
 ambiguities was more or less trivial (easier than in GOLD, from what I
 understand, though I haven't used GOLD), for what that's worth.
 


I have taken the hole grammar from the Digital Mars Hompage (the grammar
for GOLD is nearly the same, as on the homepage) and I also used the 
same identifiers for the nonterminals, so when I get a conflict, I can 
easaly consult the online documentation, to understand the conflict!
And with your admission I take also some parts from your grammar to 
solve the conflicts.


 But never mind me. I'm just a big pessimist. Good luck with your grammar!


I am an optimist, and I want to do everything I can do to push the D 
programming language, because this is the language I looked so long for. :)



 Oct 30 2009



  parent reply Ellery Newcomer <ellery-newcomer utulsa.edu>  writes:


Matthias Pleh wrote:

 Also, it took me maybe 6 months of reading through DMD's source to get
 something accurate together, and munging through syntactic and semantic
 ambiguities was more or less trivial (easier than in GOLD, from what I
 understand, though I haven't used GOLD), for what that's worth.


 I have taken the hole grammar from the Digital Mars Hompage (the grammar
 for GOLD is nearly the same, as on the homepage) and I also used the
 same identifiers for the nonterminals, so when I get a conflict, I can
 easaly consult the online documentation, to understand the conflict!
 And with your admission I take also some parts from your grammar to
 solve the conflicts.


Note the reason I spent 4 months reading through the compiler is because
the grammar in the spec is incorrect in many places. But yeah, feel free
to use mine.



 Oct 30 2009



  parent  Matthias Pleh <matthias.pleh gmx.at>  writes:


Ellery Newcomer schrieb:

 Matthias Pleh wrote:

 Also, it took me maybe 6 months of reading through DMD's source to get
 something accurate together, and munging through syntactic and semantic
 ambiguities was more or less trivial (easier than in GOLD, from what I
 understand, though I haven't used GOLD), for what that's worth.


 I have taken the hole grammar from the Digital Mars Hompage (the grammar
 for GOLD is nearly the same, as on the homepage) and I also used the
 same identifiers for the nonterminals, so when I get a conflict, I can
 easaly consult the online documentation, to understand the conflict!
 And with your admission I take also some parts from your grammar to
 solve the conflicts.


 
 Note the reason I spent 4 months reading through the compiler is because
 the grammar in the spec is incorrect in many places. But yeah, feel free
 to use mine.


I just realised, it was not your grammar, which I tried to convert. It 
was the grammar I found in the lr-lalr project on dsource.org. (it seems
to be an abandoned project)

Now I spent the whole day to extract the grammar from the spec and bring 
it in a valid GOLD grammar. I fixed the first few conflicts, but I 
realized, that it will be an endless story to fix them all. It's to much 
to overlook at once. One should write the grammar in small steps. But 
anyway, I abandoned int and will use the JSON output for now. It's quite 
simple to parse and I think it's enough for the first step!



 Oct 30 2009



prev sibling next sibling parent  Ellery Newcomer <ellery-newcomer utulsa.edu>  writes:


Walter Bright wrote:

 In my discussions with companies about adopting D, the major barrier
 that comes up over and over isn't Tango vs Phobos, dmd being GPL,
 debugger support, libraries, bugs, etc., although those are important.
 
 It's the IDE.
 
 They say that the productivity gains of D's improvements are
 overbalanced by the loss of productivity by moving away from an IDE. And
 what is it about an IDE that is so productive? Intellisense (Microsoft's
 word for autocompletion).
 
 So, while I'm not going to be writing an IDE, I figure that dmd can
 help. dmd already puts out .doc and .di files. How about putting out an
 xml file giving all the information needed for an IDE to implement
 autocompletion? There'd be one .xml file generated per .d source file.
 
 The nice thing about an xml file is while D is relatively easy to parse,
 xml is trivial. Furthermore, an xml format would be fairly robust in the
 face of changes to D syntax.
 
 What do you think?


Next question: what about conditional compilation?



 Oct 12 2009



prev sibling  parent  Jussi Jumppanen <jussij zeusedit.com>  writes:


Walter Bright Wrote:


 So, while I'm not going to be writing an IDE, I figure that dmd can 
 help. dmd already puts out .doc and .di files. 


If dmd could produce inforamtion similar to the information created 
by the ctags utility then I think that would be a great improvement.

Currently there are several editors that rely on ctags for their language 
information (i.e. Emacs, Zeus, Vim, etc.) but unfortunately ctags does a
poor job of supporting the D language.

If the compiler itself could produce tag information better than ctags
then I think that would be a great improvement.

Jussi Jumppanen
Author: Zeus for Windows IDE
http://www.zeusedit.com

PS: For any one that might be intereseted here are the changes I made to 
ctags in an effort to better support the 1.x version of the D language:

  http://www.zeusedit.com/z300/ctags_src.zip



 Oct 20 2009