digitalmars.D - Appender bug, or impossible?
- Jonathan Marler (29/29) Nov 13 2016 It looks like an Appender field of a struct doesn't work when its
- Steven Schveighoffer (10/34) Nov 13 2016 This is probably because Appender's functions are all templates.
- Timon Gehr (16/24) Nov 14 2016 It does. :)
- =?UTF-8?Q?Ali_=c3=87ehreli?= (4/5) Nov 15 2016 What are the steps to use your front end instead of dmd's? Is the
- Timon Gehr (3/9) Nov 16 2016 Hopefully it is possible eventually.
- Rainer Schuetze (12/40) Nov 15 2016 Coincidentally, I've tried compiling your front end with latest dmd just...
- Timon Gehr (50/98) Nov 16 2016 Oops. Thanks for pointing this out, I have committed it. (It's just a
- Jacob Carlborg (5/47) Nov 16 2016 Is it "only" semantic analysis that is missing for these features or
- Timon Gehr (4/50) Nov 16 2016 Mostly semantic analysis, but parsing is very easy anyway. I think UDAs
- Stefan Koch (3/7) Nov 16 2016 Sounds to me like you should rewrite the compiler a little to get
- Timon Gehr (2/9) Nov 17 2016 "A little" is probably not enough.
- John Colvin (3/18) Nov 17 2016 I'm interested to know what the big differences were that make
- Timon Gehr (2/18) Nov 17 2016 The forward reference bugs are different.
- Stefan Koch (3/27) Nov 17 2016 His compiler is filled with templates.
- Rainer Schuetze (9/77) Nov 18 2016 I've read that, but I don't want to go back that much in time...
- Stefan Koch (4/7) Nov 18 2016 Hmm I plan on cleaning DDMD up a bunch.
- Rainer Schuetze (3/9) Nov 18 2016 That's also an option, but I don't think it is in a state (yet) to be
- Stefan Koch (4/18) Nov 18 2016 Completion engine. Indeed it's not usable as that.
- Rainer Schuetze (3/21) Nov 18 2016 Reading its feature list, I suspect it would be a step backward.
- Stefan Koch (4/27) Nov 18 2016 Then I'd say try to keep dparser :)
- Timon Gehr (11/30) Nov 18 2016 I think 'protected' is the only remaining feature whose implementation
- Rainer Schuetze (8/11) Nov 25 2016 It took some time, but here is a version of dmd:
- Timon Gehr (3/16) Nov 28 2016 Merged! Thank you for taking care of this.
- Rainer Schuetze (5/19) Nov 28 2016 I noticed it fails on a couple of tests from the testsuite. Will have to...
It looks like an Appender field of a struct doesn't work when its element type is the containing struct, i.e. struct Foo { Appender!(Foo[]) fooAppender; } My guess is the problem is that the Appender needs to know how big "Foo" is since it would be storing each element by value, but since Foo's size depends on the size of the appender, we have a circular dependency on knowing the storage size. This theory is reinforced because modifying the element type of fooAppender to "Foo*" fixes the problem. struct Foo { Appender!(Foo*[]) fooAppender; // Works fine } One odd thing is that this error doesn't manifest until the appender is used. It compiles just fine so long as you don't actually use the appender, but once you put something in it or access the data, you will get a compiler error at the location where it was used, and not at the place where the appender was defined. I'm not sure how it does this, but from the error messages I've seen, it looks like the appender may be setting the element type to "void" and silently continuing on as if this isn't a problem. Does anything know if this is a bug, or if it is as designed? If it turns out that the functionality can't be supported, I think that not throwing an error at the location of the appender definition is a problem that should be addressed.
Nov 13 2016
On 11/13/16 11:29 AM, Jonathan Marler wrote:It looks like an Appender field of a struct doesn't work when its element type is the containing struct, i.e. struct Foo { Appender!(Foo[]) fooAppender; } My guess is the problem is that the Appender needs to know how big "Foo" is since it would be storing each element by value, but since Foo's size depends on the size of the appender, we have a circular dependency on knowing the storage size. This theory is reinforced because modifying the element type of fooAppender to "Foo*" fixes the problem. struct Foo { Appender!(Foo*[]) fooAppender; // Works fine } One odd thing is that this error doesn't manifest until the appender is used. It compiles just fine so long as you don't actually use the appender, but once you put something in it or access the data, you will get a compiler error at the location where it was used, and not at the place where the appender was defined. I'm not sure how it does this, but from the error messages I've seen, it looks like the appender may be setting the element type to "void" and silently continuing on as if this isn't a problem.This is probably because Appender's functions are all templates. A Foo[] can be stored in a Foo, because it doesn't need the size. But yes, as soon as you start needing Appender functions, then the compiler chokes. It is a forward reference bug, but still a bug IMO. If you can store the appender, then the compiler knows how big it has to be. So it should be fine at that point. Paging Timon, I'm betting your front end handles this just fine ;) -Steve
Nov 13 2016
On 14.11.2016 00:32, Steven Schveighoffer wrote:A Foo[] can be stored in a Foo, because it doesn't need the size. But yes, as soon as you start needing Appender functions, then the compiler chokes. It is a forward reference bug, but still a bug IMO. If you can store the appender, then the compiler knows how big it has to be. So it should be fine at that point. Paging Timon, I'm betting your front end handles this just fine ;) -SteveIt does. :) Minimal example: struct Appender(A){ alias T = typeof({ A a; return a[0]; }()); T[] data; void put(T item){ data~=item; } } struct Foo{ Appender!(Foo[]) fooAppender; } Foo[] test(){ Foo f; f.fooAppender.put(Foo()); return f.fooAppender.data; } static assert(test().length==1); Error with DMD, works with my front end.
Nov 14 2016
On 11/14/2016 12:26 PM, Timon Gehr wrote:Error with DMD, works with my front end.What are the steps to use your front end instead of dmd's? Is the awesome combo of Timon front-end plus ldc back-end possible? Ali
Nov 15 2016
On 15.11.2016 10:10, Ali Çehreli wrote:On 11/14/2016 12:26 PM, Timon Gehr wrote:The first step is I need to implement all remaining language features. :)Error with DMD, works with my front end.What are the steps to use your front end instead of dmd's?Is the awesome combo of Timon front-end plus ldc back-end possible? AliHopefully it is possible eventually.
Nov 16 2016
On 14.11.2016 21:26, Timon Gehr wrote:On 14.11.2016 00:32, Steven Schveighoffer wrote:Coincidentally, I've tried compiling your front end with latest dmd just yesterday. There is one file missing, though: cent_.d. Could you please commit this, too? The cent code commented out, I noticed your code is suffering from the same issue as the OP (also happens for AAs, e.g. T[int]). A workaround is to use void[] and wrap it in property functions. I've fixed/worked around a number of issues in dmd with respect to incomplete semantic analysis that happen with template mixins, but more still pop up afterwards. Are there limitations to what the front end understands? Is it able to digest itself? That would be pretty impressive ;-)A Foo[] can be stored in a Foo, because it doesn't need the size. But yes, as soon as you start needing Appender functions, then the compiler chokes. It is a forward reference bug, but still a bug IMO. If you can store the appender, then the compiler knows how big it has to be. So it should be fine at that point. Paging Timon, I'm betting your front end handles this just fine ;) -SteveIt does. :) Minimal example: struct Appender(A){ alias T = typeof({ A a; return a[0]; }()); T[] data; void put(T item){ data~=item; } } struct Foo{ Appender!(Foo[]) fooAppender; } Foo[] test(){ Foo f; f.fooAppender.put(Foo()); return f.fooAppender.data; } static assert(test().length==1); Error with DMD, works with my front end.
Nov 15 2016
On 15.11.2016 14:11, Rainer Schuetze wrote:On 14.11.2016 21:26, Timon Gehr wrote:Oops. Thanks for pointing this out, I have committed it. (It's just a stub anyway: It's a wrapper around a long.)On 14.11.2016 00:32, Steven Schveighoffer wrote:Coincidentally, I've tried compiling your front end with latest dmd just yesterday. There is one file missing, though: cent_.d. Could you please commit this, too? ...A Foo[] can be stored in a Foo, because it doesn't need the size. But yes, as soon as you start needing Appender functions, then the compiler chokes. It is a forward reference bug, but still a bug IMO. If you can store the appender, then the compiler knows how big it has to be. So it should be fine at that point. Paging Timon, I'm betting your front end handles this just fine ;) -SteveIt does. :) Minimal example: struct Appender(A){ alias T = typeof({ A a; return a[0]; }()); T[] data; void put(T item){ data~=item; } } struct Foo{ Appender!(Foo[]) fooAppender; } Foo[] test(){ Foo f; f.fooAppender.put(Foo()); return f.fooAppender.data; } static assert(test().length==1); Error with DMD, works with my front end.The cent code commented out, I noticed your code is suffering from the same issue as the OP (also happens for AAs, e.g. T[int]). A workaround is to use void[] and wrap it in property functions. I've fixed/worked around a number of issues in dmd with respect to incomplete semantic analysis that happen with template mixins, but more still pop up afterwards. ...DMD 2.060 is the only version of DMD that builds the code.Are there limitations to what the front end understands?Yup. There are many features missing that are quite easy to implement but require work, and a few that are somewhat messy to specify (e.g. 'protected'). An incomplete list: * UDA's * Built-in members (init, stringof, min, max, ...) * various forms of import statements - static, selective, renaming, ... * Initialization of union fields * anonymous structs & unions - Analysis & Lowering * additional import paths & implicit object.d * implicit inheritance from Object * version declarations * Associative arrays/Associative array literals * module declarations * (implicit) super constructor calls - default constructor call insertion - flow analysis * Destructor and postblit calls - use flow analysis to optimize moves * with statements * associative arrays * foreach statements - automatic decoding for string types - foreach over associative arrays - foreach over delegates - foreach over AliasSeq * pattern matching in old-style template constraints * explicit casts from/to class references - eg. to/from void* and to bool * try-catch-finally statements * scope guards * initialization crossing check * multi-argument struct constructors. * struct postblit & destructors * finish operator overloading support * opDispatch * member alias declarations aliasing members - correctly provide a this pointer * visibility - package, protected * alias thisIs it able to digest itself?Not yet. (Mostly because of missing language features.)That would be pretty impressive ;-)I want to get there eventually. :) Unfortunately I haven't had a lot of time to spend on this lately. Also, DMD 2.060 has quite many annoying bugs that slow down development.
Nov 16 2016
On 2016-11-16 01:11, Timon Gehr wrote:Yup. There are many features missing that are quite easy to implement but require work, and a few that are somewhat messy to specify (e.g. 'protected'). An incomplete list: * UDA's * Built-in members (init, stringof, min, max, ...) * various forms of import statements - static, selective, renaming, ... * Initialization of union fields * anonymous structs & unions - Analysis & Lowering * additional import paths & implicit object.d * implicit inheritance from Object * version declarations * Associative arrays/Associative array literals * module declarations * (implicit) super constructor calls - default constructor call insertion - flow analysis * Destructor and postblit calls - use flow analysis to optimize moves * with statements * associative arrays * foreach statements - automatic decoding for string types - foreach over associative arrays - foreach over delegates - foreach over AliasSeq * pattern matching in old-style template constraints * explicit casts from/to class references - eg. to/from void* and to bool * try-catch-finally statements * scope guards * initialization crossing check * multi-argument struct constructors. * struct postblit & destructors * finish operator overloading support * opDispatch * member alias declarations aliasing members - correctly provide a this pointer * visibility - package, protected * alias thisIs it "only" semantic analysis that is missing for these features or lexing and/or parsing as well? -- /Jacob Carlborg
Nov 16 2016
On 16.11.2016 19:17, Jacob Carlborg wrote:On 2016-11-16 01:11, Timon Gehr wrote:Mostly semantic analysis, but parsing is very easy anyway. I think UDAs are the only listed feature still missing in the parser, but that's just because they were introduced after I had already written the parser.Yup. There are many features missing that are quite easy to implement but require work, and a few that are somewhat messy to specify (e.g. 'protected'). An incomplete list: * UDA's * Built-in members (init, stringof, min, max, ...) * various forms of import statements - static, selective, renaming, ... * Initialization of union fields * anonymous structs & unions - Analysis & Lowering * additional import paths & implicit object.d * implicit inheritance from Object * version declarations * Associative arrays/Associative array literals * module declarations * (implicit) super constructor calls - default constructor call insertion - flow analysis * Destructor and postblit calls - use flow analysis to optimize moves * with statements * associative arrays * foreach statements - automatic decoding for string types - foreach over associative arrays - foreach over delegates - foreach over AliasSeq * pattern matching in old-style template constraints * explicit casts from/to class references - eg. to/from void* and to bool * try-catch-finally statements * scope guards * initialization crossing check * multi-argument struct constructors. * struct postblit & destructors * finish operator overloading support * opDispatch * member alias declarations aliasing members - correctly provide a this pointer * visibility - package, protected * alias thisIs it "only" semantic analysis that is missing for these features or lexing and/or parsing as well?
Nov 16 2016
On Wednesday, 16 November 2016 at 09:11:50 UTC, Timon Gehr wrote:I want to get there eventually. :) Unfortunately I haven't had a lot of time to spend on this lately. Also, DMD 2.060 has quite many annoying bugs that slow down development.Sounds to me like you should rewrite the compiler a little to get it working under 2.07x.
Nov 16 2016
On 17.11.2016 03:52, Stefan Koch wrote:On Wednesday, 16 November 2016 at 09:11:50 UTC, Timon Gehr wrote:"A little" is probably not enough.I want to get there eventually. :) Unfortunately I haven't had a lot of time to spend on this lately. Also, DMD 2.060 has quite many annoying bugs that slow down development.Sounds to me like you should rewrite the compiler a little to get it working under 2.07x.
Nov 17 2016
On Thursday, 17 November 2016 at 21:39:15 UTC, Timon Gehr wrote:On 17.11.2016 03:52, Stefan Koch wrote:I'm interested to know what the big differences were that make updating particularly hard?On Wednesday, 16 November 2016 at 09:11:50 UTC, Timon Gehr wrote:"A little" is probably not enough.I want to get there eventually. :) Unfortunately I haven't had a lot of time to spend on this lately. Also, DMD 2.060 has quite many annoying bugs that slow down development.Sounds to me like you should rewrite the compiler a little to get it working under 2.07x.
Nov 17 2016
On 17.11.2016 22:41, John Colvin wrote:On Thursday, 17 November 2016 at 21:39:15 UTC, Timon Gehr wrote:The forward reference bugs are different.On 17.11.2016 03:52, Stefan Koch wrote:I'm interested to know what the big differences were that make updating particularly hard?On Wednesday, 16 November 2016 at 09:11:50 UTC, Timon Gehr wrote:"A little" is probably not enough.I want to get there eventually. :) Unfortunately I haven't had a lot of time to spend on this lately. Also, DMD 2.060 has quite many annoying bugs that slow down development.Sounds to me like you should rewrite the compiler a little to get it working under 2.07x.
Nov 17 2016
On Thursday, 17 November 2016 at 22:46:33 UTC, Timon Gehr wrote:On 17.11.2016 22:41, John Colvin wrote:His compiler is filled with templates. I have never seen anything like it.On Thursday, 17 November 2016 at 21:39:15 UTC, Timon Gehr wrote:The forward reference bugs are different.On 17.11.2016 03:52, Stefan Koch wrote:I'm interested to know what the big differences were that make updating particularly hard?On Wednesday, 16 November 2016 at 09:11:50 UTC, Timon Gehr wrote:"A little" is probably not enough.I want to get there eventually. :) Unfortunately I haven't had a lot of time to spend on this lately. Also, DMD 2.060 has quite many annoying bugs that slow down development.Sounds to me like you should rewrite the compiler a little to get it working under 2.07x.
Nov 17 2016
On 16.11.2016 10:11, Timon Gehr wrote:On 15.11.2016 14:11, Rainer Schuetze wrote:Thanks.Coincidentally, I've tried compiling your front end with latest dmd just yesterday. There is one file missing, though: cent_.d. Could you please commit this, too? ...Oops. Thanks for pointing this out, I have committed it. (It's just a stub anyway: It's a wrapper around a long.)I've read that, but I don't want to go back that much in time...The cent code commented out, I noticed your code is suffering from the same issue as the OP (also happens for AAs, e.g. T[int]). A workaround is to use void[] and wrap it in property functions. I've fixed/worked around a number of issues in dmd with respect to incomplete semantic analysis that happen with template mixins, but more still pop up afterwards. ...DMD 2.060 is the only version of DMD that builds the code.Quite a list, but the issues that are likely to affect symbol lookup might not be too complicated to implement. I'm looking for candidates that might replace the semantic engine in Visual D, as D_Parser is unlikely to be developed further at the moment.Are there limitations to what the front end understands?Yup. There are many features missing that are quite easy to implement but require work, and a few that are somewhat messy to specify (e.g. 'protected'). An incomplete list: * UDA's * Built-in members (init, stringof, min, max, ...) * various forms of import statements - static, selective, renaming, ... * Initialization of union fields * anonymous structs & unions - Analysis & Lowering * additional import paths & implicit object.d * implicit inheritance from Object * version declarations * Associative arrays/Associative array literals * module declarations * (implicit) super constructor calls - default constructor call insertion - flow analysis * Destructor and postblit calls - use flow analysis to optimize moves * with statements * associative arrays * foreach statements - automatic decoding for string types - foreach over associative arrays - foreach over delegates - foreach over AliasSeq * pattern matching in old-style template constraints * explicit casts from/to class references - eg. to/from void* and to bool * try-catch-finally statements * scope guards * initialization crossing check * multi-argument struct constructors. * struct postblit & destructors * finish operator overloading support * opDispatch * member alias declarations aliasing members - correctly provide a this pointer * visibility - package, protected * alias thisI hope to get the forward reference bugs solved in current dmd. That might also help your motivation to continue the front end ;-)Is it able to digest itself?Not yet. (Mostly because of missing language features.)That would be pretty impressive ;-)I want to get there eventually. :) Unfortunately I haven't had a lot of time to spend on this lately. Also, DMD 2.060 has quite many annoying bugs that slow down development.
Nov 18 2016
On Friday, 18 November 2016 at 10:18:21 UTC, Rainer Schuetze wrote:I'm looking for candidates that might replace the semantic engine in Visual D, as D_Parser is unlikely to be developed further at the moment.Hmm I plan on cleaning DDMD up a bunch. Any chance you could use the DDMD frontend ?
Nov 18 2016
On 18.11.2016 11:33, Stefan Koch wrote:On Friday, 18 November 2016 at 10:18:21 UTC, Rainer Schuetze wrote:That's also an option, but I don't think it is in a state (yet) to be used as a completion engine.I'm looking for candidates that might replace the semantic engine in Visual D, as D_Parser is unlikely to be developed further at the moment.Hmm I plan on cleaning DDMD up a bunch. Any chance you could use the DDMD frontend ?
Nov 18 2016
On Friday, 18 November 2016 at 11:37:40 UTC, Rainer Schuetze wrote:On 18.11.2016 11:33, Stefan Koch wrote:Completion engine. Indeed it's not usable as that. What about DCD ?On Friday, 18 November 2016 at 10:18:21 UTC, Rainer Schuetze wrote:That's also an option, but I don't think it is in a state (yet) to be used as a completion engine.I'm looking for candidates that might replace the semantic engine in Visual D, as D_Parser is unlikely to be developed further at the moment.Hmm I plan on cleaning DDMD up a bunch. Any chance you could use the DDMD frontend ?
Nov 18 2016
On 18.11.2016 12:39, Stefan Koch wrote:On Friday, 18 November 2016 at 11:37:40 UTC, Rainer Schuetze wrote:Reading its feature list, I suspect it would be a step backward. D_Parser comes with pretty good template, CTFE, UFCS and mixin support.On 18.11.2016 11:33, Stefan Koch wrote:Completion engine. Indeed it's not usable as that. What about DCD ?On Friday, 18 November 2016 at 10:18:21 UTC, Rainer Schuetze wrote:That's also an option, but I don't think it is in a state (yet) to be used as a completion engine.I'm looking for candidates that might replace the semantic engine in Visual D, as D_Parser is unlikely to be developed further at the moment.Hmm I plan on cleaning DDMD up a bunch. Any chance you could use the DDMD frontend ?
Nov 18 2016
On Friday, 18 November 2016 at 12:50:30 UTC, Rainer Schuetze wrote:On 18.11.2016 12:39, Stefan Koch wrote:Then I'd say try to keep dparser :) I wonder how it does mixins though ...On Friday, 18 November 2016 at 11:37:40 UTC, Rainer Schuetze wrote:Reading its feature list, I suspect it would be a step backward. D_Parser comes with pretty good template, CTFE, UFCS and mixin support.On 18.11.2016 11:33, Stefan Koch wrote:Completion engine. Indeed it's not usable as that. What about DCD ?On Friday, 18 November 2016 at 10:18:21 UTC, Rainer Schuetze wrote:That's also an option, but I don't think it is in a state (yet) to be used as a completion engine.[...]Hmm I plan on cleaning DDMD up a bunch. Any chance you could use the DDMD frontend ?
Nov 18 2016
On 18.11.2016 11:18, Rainer Schuetze wrote:I think 'protected' is the only remaining feature whose implementation is not "obvious". Otherwise, finishing support for all features is mostly a question of spending the time. Some performance tuning will be necessary too....Quite a list, but the issues that are likely to affect symbol lookup might not be too complicated to implement.I'm looking for candidates that might replace the semantic engine in Visual D,In theory, the frontend would be a good option because due to its design, it is quite easy to extend for incremental semantic analysis support. (It keeps track of all dependencies between nodes, all that is needed is to record those and to re-analyze dependent code as required.)as D_Parser is unlikely to be developed further at the moment.Nice initiative!I hope to get the forward reference bugs solved in current dmd.Is it able to digest itself?Not yet. (Mostly because of missing language features.)That would be pretty impressive ;-)I want to get there eventually. :) Unfortunately I haven't had a lot of time to spend on this lately. Also, DMD 2.060 has quite many annoying bugs that slow down development.That might also help your motivation to continue the front end ;-)Certainly. :)
Nov 18 2016
On 18.11.2016 15:37, Timon Gehr wrote:It took some time, but here is a version of dmd: https://github.com/rainers/dmd/tree/tg_frontend that can compile an updated frontend: https://github.com/tgehr/d-compiler/pull/1 [The debug version of dmd hits 2 assertions during code generation, though (some real/ireal type mismatch, unlikely to make a lot of trouble in the release build).]I hope to get the forward reference bugs solved in current dmd.Nice initiative!
Nov 25 2016
On 25.11.2016 09:46, Rainer Schuetze wrote:On 18.11.2016 15:37, Timon Gehr wrote:What needs to happen that this is merged into master? :)It took some time, but here is a version of dmd: https://github.com/rainers/dmd/tree/tg_frontend ...I hope to get the forward reference bugs solved in current dmd.Nice initiative!that can compile an updated frontend: https://github.com/tgehr/d-compiler/pull/1 [The debug version of dmd hits 2 assertions during code generation, though (some real/ireal type mismatch, unlikely to make a lot of trouble in the release build).]Merged! Thank you for taking care of this.
Nov 28 2016
On 28.11.2016 10:30, Timon Gehr wrote:On 25.11.2016 09:46, Rainer Schuetze wrote:I noticed it fails on a couple of tests from the testsuite. Will have to figure out, why... There are a couple of issues involved, these will likely have to be split up to get a chance of getting merged.On 18.11.2016 15:37, Timon Gehr wrote:What needs to happen that this is merged into master? :)It took some time, but here is a version of dmd: https://github.com/rainers/dmd/tree/tg_frontend ...I hope to get the forward reference bugs solved in current dmd.Nice initiative!
Nov 28 2016