www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DConf Hackathon Ideas

reply Mike Parker <aldacron gmail.com> writes:
This year, DConf has an extra day tacked on for problem solving 
in the form of a hackathon. The intent is to work on issues 
people find frustrating in the D ecosystem. While there will be 
time given at the event for proposals, and those involving 
third-party projects are welcome, it will help speed things along 
for the organizers to come in with a list of big issues to get 
the ball rolling.

To help in compiling the list, what are some major issues from 
the ecosystem that you'd like to see fixed?
Apr 27 2017
next sibling parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Thursday, 27 April 2017 at 14:53:02 UTC, Mike Parker wrote:

 To help in compiling the list, what are some major issues from 
 the ecosystem that you'd like to see fixed?


* multi-threaded parsing in DMD
* https://github.com/dlang/dub/issues/838
* finishing std.experimental.xml 
https://github.com/dlang/phobos/pull/4741
* Docs which allow people to go back and see docs for previous 
versions. Huge headache when using GDC or LDC
Apr 27 2017
parent Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 27 April 2017 at 17:15, Jack Stouffer via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 * Docs which allow people to go back and see docs for previous versions.
 Huge headache when using GDC or LDC


Maybe you could submit a patch to add build hook to generate the
documentation for GDC.  It would be a welcome contribution to add to
the current bulk of documentation I'm starting to build up. ;-)
Apr 27 2017
prev sibling next sibling parent reply Moritz Maxeiner <moritz ucworks.org> writes:
On Thursday, 27 April 2017 at 14:53:02 UTC, Mike Parker wrote:

 This year, DConf has an extra day tacked on for problem solving 
 in the form of a hackathon. The intent is to work on issues 
 people find frustrating in the D ecosystem. While there will be 
 time given at the event for proposals, and those involving 
 third-party projects are welcome, it will help speed things 
 along for the organizers to come in with a list of big issues 
 to get the ball rolling.

 To help in compiling the list, what are some major issues from 
 the ecosystem that you'd like to see fixed?


For starters:
- https://github.com/dlang/dub/issues/104
- https://github.com/dlang/dub-registry/issues/117
I don't think the following ones are feasible to deal with in 
that context, but they are ones that irk me in the ecosystem:
- ABI compatibility between D compilers with the same frontend 
version.
- More than one official package description language (json and 
sdlang). I honestly don't care which it ends up being, but please 
pick *one* (I am aware of the previous discussions on the topic, 
but the current state of supporting both is just one more point 
of friction for newcomers)
Apr 27 2017
parent reply singingbush <singingbush hotmail.com> writes:
 - More than one official package description language (json and 
 sdlang). I honestly don't care which it ends up being, but 
 please pick *one* (I am aware of the previous discussions on 
 the topic, but the current state of supporting both is just one 
 more point of friction for newcomers)


SDL should be dropped. for starters it's poorly supported. just 
go to https://sdlang.org/ and scroll to the links at the bottom. 
official homepage and reference are both broken urls (as 
http://sdl.ikayzo.org/ is offline) and needs mirrors. Then 
there's the fact that there's not very good support for SDL in 
other languages, sure the website says there are implementations 
in other languages but has anyone tried using them? I did 
recently, the Java version (actually written by ikayzo - 
https://github.com/ikayzo/SDL) will not even parse all the 
examples on the homepage! not to mention it being a dead project 
with no response to issues 
(https://github.com/ikayzo/SDL/issues). As far as I can tell the 
only reason dub defaults to sdl now is because s-ludwig maintains 
the sdlang.org website. at least json is universally known and 
can already be handled by just about any tool. If we really want 
to have an alternative it would at least make sense to use 
something like YAML that is also heavily used and supported by 
all IDE's. It would save me the hassle of writing a lexer for SDL 
- which of course has no support in my tool of choice because...

NOBODY USES IT!
Apr 27 2017
next sibling parent Moritz Maxeiner <moritz ucworks.org> writes:
On Thursday, 27 April 2017 at 16:33:02 UTC, singingbush wrote:

 As far as I can tell the only reason dub defaults to sdl now


It did in the past, but hasn't done so for several minor versions 
(it defaults to json).
Apr 27 2017
prev sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Thursday, 27 April 2017 at 16:33:02 UTC, singingbush wrote:

 SDL should be dropped.


Deprecated, sure.  But dropping it seems a bad idea given that 
various projects do still use it for their DUB package config.


 NOBODY USES IT!


Probably not true.  Perhaps a hackathon project could be to 
create a little app to find which projects on GitHub (or at least 
code.dlang.org) still use a `dub.sdl`, and auto-submit a PR to 
fix that? :-)
May 01 2017
parent reply Mike Parker <aldacron gmail.com> writes:
On Monday, 1 May 2017 at 14:38:11 UTC, Joseph Rushton Wakeling 
wrote:

 On Thursday, 27 April 2017 at 16:33:02 UTC, singingbush wrote:

 SDL should be dropped.


 Deprecated, sure.  But dropping it seems a bad idea given that 
 various projects do still use it for their DUB package config.


 NOBODY USES IT!


 Probably not true.  Perhaps a hackathon project could be to 
 create a little app to find which projects on GitHub (or at 
 least code.dlang.org) still use a `dub.sdl`, and auto-submit a 
 PR to fix that? :-)


I love SDL and much prefer it over JSON for DUB configs. Use it 
for all of my D projects. It looks cleaner and supports comments. 
I really would hate to see support dropped.
May 01 2017
next sibling parent reply bachmeier <no spam.net> writes:
On Monday, 1 May 2017 at 14:51:19 UTC, Mike Parker wrote:


 I love SDL and much prefer it over JSON for DUB configs. Use it 
 for all of my D projects. It looks cleaner and supports 
 comments. I really would hate to see support dropped.


I'm not even sure what it would mean to "drop" support for SDL. 
As long as someone decides to support it, it will be supported. 
SDL to my knowledge is not an "official" D project.
May 01 2017
parent Mike Parker <aldacron gmail.com> writes:
On Monday, 1 May 2017 at 15:36:16 UTC, bachmeier wrote:

 On Monday, 1 May 2017 at 14:51:19 UTC, Mike Parker wrote:


 I love SDL and much prefer it over JSON for DUB configs. Use 
 it for all of my D projects. It looks cleaner and supports 
 comments. I really would hate to see support dropped.


 I'm not even sure what it would mean to "drop" support for SDL. 
 As long as someone decides to support it, it will be supported. 
 SDL to my knowledge is not an "official" D project.


Well, sure, but we were specifically talking about support in DUB.

http://forum.dlang.org/post/eejeorhqgwxbduunmenh forum.dlang.org
May 01 2017
prev sibling next sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 1 May 2017 at 16:51, Mike Parker via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Monday, 1 May 2017 at 14:38:11 UTC, Joseph Rushton Wakeling wrote:

 On Thursday, 27 April 2017 at 16:33:02 UTC, singingbush wrote:

 SDL should be dropped.



 Deprecated, sure.  But dropping it seems a bad idea given that various
 projects do still use it for their DUB package config.


 NOBODY USES IT!



 Probably not true.  Perhaps a hackathon project could be to create a
 little app to find which projects on GitHub (or at least code.dlang.org)
 still use a `dub.sdl`, and auto-submit a PR to fix that? :-)



 I love SDL and much prefer it over JSON for DUB configs. Use it for all of
 my D projects. It looks cleaner and supports comments. I really would hate
 to see support dropped.


We should make XML the default config format for DUB.


<?xml version='1.0' encoding='utf-8'?>
<dub
  xmlns="http://code.dlang.org/"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <project
    name="gdcproject"
    description="The vibe.d server application running gdcproject.org."
    copyright="Copyright © 2014, Iain Buclaw">
    <authors>
      <author name="Iain Buclaw">
    </authors>
    <dependencies>
      <dependency name="vibe-d" version="0.7.22">
      <dependency name="mustache-d" version="0.1.0">
    </dependencies>
    <versions>
      <version name="VibeDefaultMain">
    </versions>
  </project>
</dub>


/Runaway!
May 01 2017
next sibling parent reply Moritz Maxeiner <moritz ucworks.org> writes:
On Monday, 1 May 2017 at 17:04:42 UTC, Iain Buclaw wrote:

 [...]

 We should make XML the default config format for DUB.

 [...]

 /Runaway!


Well, at least nearly everyone hates XML equally, which may be an 
indicator of a good compromise.
May 01 2017
parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Mon, May 01, 2017 at 06:02:53PM +0000, Moritz Maxeiner via Digitalmars-d
wrote:

 On Monday, 1 May 2017 at 17:04:42 UTC, Iain Buclaw wrote:

 [...]
 
 We should make XML the default config format for DUB.
 
 [...]
 
 /Runaway!


 
 Well, at least nearly everyone hates XML equally, which may be an
 indicator of a good compromise.


Whoa.  XML for DUB?  That may just be the final nail in the coffin for
me ever picking up DUB in this lifetime. ;-)


T

-- 
Dogs have owners ... cats have staff. -- Krista Casada
May 01 2017
prev sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 1 May 2017 at 17:04:42 UTC, Iain Buclaw wrote:

 On 1 May 2017 at 16:51, Mike Parker via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 On Monday, 1 May 2017 at 14:38:11 UTC, Joseph Rushton Wakeling 
 wrote:

 On Thursday, 27 April 2017 at 16:33:02 UTC, singingbush wrote:

 SDL should be dropped.



 Deprecated, sure.  But dropping it seems a bad idea given 
 that various projects do still use it for their DUB package 
 config.


 NOBODY USES IT!



 Probably not true.  Perhaps a hackathon project could be to 
 create a little app to find which projects on GitHub (or at 
 least code.dlang.org) still use a `dub.sdl`, and auto-submit 
 a PR to fix that? :-)



 I love SDL and much prefer it over JSON for DUB configs. Use 
 it for all of my D projects. It looks cleaner and supports 
 comments. I really would hate to see support dropped.


 We should make XML the default config format for DUB.


 <?xml version='1.0' encoding='utf-8'?>
 <dub
   xmlns="http://code.dlang.org/"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
   <project
     name="gdcproject"
     description="The vibe.d server application running 
 gdcproject.org."
     copyright="Copyright © 2014, Iain Buclaw">
     <authors>
       <author name="Iain Buclaw">
     </authors>
     <dependencies>
       <dependency name="vibe-d" version="0.7.22">
       <dependency name="mustache-d" version="0.1.0">
     </dependencies>
     <versions>
       <version name="VibeDefaultMain">
     </versions>
   </project>
 </dub>


 /Runaway!


You forgot a few / there
May 01 2017
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2017-05-01 16:51, Mike Parker wrote:


 I love SDL and much prefer it over JSON for DUB configs. Use it for all
 of my D projects. It looks cleaner and supports comments. I really would
 hate to see support dropped.


+1

I would be fine with YAML as well.

-- 
/Jacob Carlborg
May 01 2017
prev sibling next sibling parent reply Andre Pany <andre s-e-a-p.de> writes:
On Thursday, 27 April 2017 at 14:53:02 UTC, Mike Parker wrote:

 This year, DConf has an extra day tacked on for problem solving 
 in the form of a hackathon. The intent is to work on issues 
 people find frustrating in the D ecosystem. While there will be 
 time given at the event for proposals, and those involving 
 third-party projects are welcome, it will help speed things 
 along for the organizers to come in with a list of big issues 
 to get the ball rolling.

 To help in compiling the list, what are some major issues from 
 the ecosystem that you'd like to see fixed?


I would vote for getting the std.decimal proposal from the review 
queue into a good shape for std.experimental.


Another big issue for me is using dub in a company. Big companies 
do not want to use the official dub repository due to security 
issues. They want to run their own dub repository with dub 
packages they have done code checks. That means also the git 
projects are mirrored in git repositories within the company. 
First step into this direction is to have the possibility to 
specify the dub repository address within the dub.json.

Kind regards
Andre
Apr 27 2017
next sibling parent reply Moritz Maxeiner <moritz ucworks.org> writes:
On Thursday, 27 April 2017 at 17:06:39 UTC, Andre Pany wrote:

 Another big issue for me is using dub in a company. Big 
 companies do not want to use the official dub repository due to 
 security issues. They want to run their own dub repository with 
 dub packages they have done code checks. That means also the 
 git projects are mirrored in git repositories within the 
 company. First step into this direction is to have the 
 possibility to specify the dub repository address within the 
 dub.json.


Is there a problem with just using the dub command line arguments 
for this?


/usr/bin/env dub --skip-registry=standard 
--registry=YOUR_COMPANYS_REGISTRY
Apr 27 2017
parent reply Andre Pany <andre s-e-a-p.de> writes:
On Thursday, 27 April 2017 at 17:47:19 UTC, Moritz Maxeiner wrote:

 On Thursday, 27 April 2017 at 17:06:39 UTC, Andre Pany wrote:

 Another big issue for me is using dub in a company. Big 
 companies do not want to use the official dub repository due 
 to security issues. They want to run their own dub repository 
 with dub packages they have done code checks. That means also 
 the git projects are mirrored in git repositories within the 
 company. First step into this direction is to have the 
 possibility to specify the dub repository address within the 
 dub.json.


 Is there a problem with just using the dub command line 
 arguments for this?


 /usr/bin/env dub --skip-registry=standard 
 --registry=YOUR_COMPANYS_REGISTRY


As workaround this is possible. Every developer and in every 
continious integration build step, dub is used, you have to 
remember to use these arguments. Defining it one time in dub.json 
would be great.

Kind regards
Andre
Apr 27 2017
parent reply Moritz Maxeiner <moritz ucworks.org> writes:
On Thursday, 27 April 2017 at 19:55:44 UTC, Andre Pany wrote:

 On Thursday, 27 April 2017 at 17:47:19 UTC, Moritz Maxeiner 
 wrote:

 On Thursday, 27 April 2017 at 17:06:39 UTC, Andre Pany wrote:

 Another big issue for me is using dub in a company. Big 
 companies do not want to use the official dub repository due 
 to security issues. They want to run their own dub repository 
 with dub packages they have done code checks. That means also 
 the git projects are mirrored in git repositories within the 
 company. First step into this direction is to have the 
 possibility to specify the dub repository address within the 
 dub.json.


 Is there a problem with just using the dub command line 
 arguments for this?


 /usr/bin/env dub --skip-registry=standard 
 --registry=YOUR_COMPANYS_REGISTRY


 As workaround this is possible. Every developer and in every 
 continious integration build step, dub is used, you have to 
 remember to use these arguments. Defining it one time in 
 dub.json would be great.


Alright, I take your point. Since the internal functionality 
itself is already there, this seems like an easy fix (just add a 
new field to the json/sdlang parsing of dub and bind it to what 
the CLI arguments already do). Have you opened a dub issue about 
this (first step to getting an issue fixed and so on), I wasn't 
able to find one on first glance?
Apr 27 2017
parent Andre Pany <andre s-e-a-p.de> writes:
On Thursday, 27 April 2017 at 22:57:48 UTC, Moritz Maxeiner wrote:

 On Thursday, 27 April 2017 at 19:55:44 UTC, Andre Pany wrote:

 On Thursday, 27 April 2017 at 17:47:19 UTC, Moritz Maxeiner 
 wrote:

 [...]


 As workaround this is possible. Every developer and in every 
 continious integration build step, dub is used, you have to 
 remember to use these arguments. Defining it one time in 
 dub.json would be great.


 Alright, I take your point. Since the internal functionality 
 itself is already there, this seems like an easy fix (just add 
 a new field to the json/sdlang parsing of dub and bind it to 
 what the CLI arguments already do). Have you opened a dub issue 
 about this (first step to getting an issue fixed and so on), I 
 wasn't able to find one on first glance?


I created an issue in the dub github repository
https://github.com/dlang/dub/issues/1119

Kind regards
André
May 01 2017
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2017-04-27 19:06, Andre Pany wrote:


 Another big issue for me is using dub in a company. Big companies do not
 want to use the official dub repository due to security issues.


That would be nice.

-- 
/Jacob Carlborg
Apr 28 2017
prev sibling next sibling parent Brad Roberts via Digitalmars-d <digitalmars-d puremagic.com> writes:
The pending pull requests.  In person is a great high-bandwidth way to 
work through the massive backlog.


On 4/27/2017 7:53 AM, Mike Parker via Digitalmars-d wrote:

 This year, DConf has an extra day tacked on for problem solving in the 
 form of a hackathon. The intent is to work on issues people find 
 frustrating in the D ecosystem. While there will be time given at the 
 event for proposals, and those involving third-party projects are 
 welcome, it will help speed things along for the organizers to come in 
 with a list of big issues to get the ball rolling.

 To help in compiling the list, what are some major issues from the 
 ecosystem that you'd like to see fixed?
Apr 27 2017
prev sibling next sibling parent ag0aep6g <anonymous example.com> writes:
On 04/27/2017 04:53 PM, Mike Parker wrote:

 This year, DConf has an extra day tacked on for problem solving in the
 form of a hackathon. The intent is to work on issues people find
 frustrating in the D ecosystem. While there will be time given at the
 event for proposals, and those involving third-party projects are
 welcome, it will help speed things along for the organizers to come in
 with a list of big issues to get the ball rolling.

 To help in compiling the list, what are some major issues from the
 ecosystem that you'd like to see fixed?


I'm most frustrated by regressions and wrong-code bugs in 
dmd/druntime/phobos (with a focus on dmd). Regressions are annoying, 
because that stuff used to work, dammit! And wrong-code bugs are 
annoying, because they can be hard to track down and work around.

I don't really have a list of them sorted by importance or impact, but I 
guess the one's I have reported are, on average, more dear to me than 
others [1][2].

I haven't reported this one, but it has a special place in my heart, 
because the miscompiled code is so simple:
https://issues.dlang.org/show_bug.cgi?id=15538

Generally, just sorting the issue list by severity and going from the 
top would probably have a good impact:

https://issues.dlang.org/buglist.cgi?bug_status=__open__&columnlist=component%2Cassigned_to%2Cbug_status%2Cbug_severity%2Cshort_desc&content=&no_redirect=1&order=bug_severity%2Cbug_status%2Cpriority%2Cassigned_to%2Cbug_id&product=&query_based_on=&query_format=specific

Instead of trying to fix those bugs during DConf, maybe experienced devs 
could help interested folks get into it. Look at an issue together, lay 
out what needs to be done about it. I'm thinking about dmd in 
particular, because it seems to be understaffed. Only works if there are 
people who want to get into fixing compiler bugs, of course.



[1] My regressions: 
https://issues.dlang.org/buglist.cgi?bug_severity=regression&email1=ag0aep6g&emailreporter1=1&emailtype1=substring&list_id=214636&query_format=advanced&resolution=---
[2] My wrong-code bugs: 
https://issues.dlang.org/buglist.cgi?email1=ag0aep6g&emailreporter1=1&emailtype1=substring&keywords=wrong-code&keywords_type=allwords&list_id=214637&query_format=advanced&resolution=---
Apr 27 2017
prev sibling next sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Thu, Apr 27, 2017 at 11:22:06AM -0700, Brad Roberts via Digitalmars-d wrote:

 The pending pull requests.  In person is a great high-bandwidth way to
 work through the massive backlog.


+1. Phobos at one time was down to about 30-40 PRs, but now it has
clogged back up to around 100. We need to get it back under control so
that valuable contributions aren't just sitting there bitrotting.


T

-- 
Не дорог подарок, дорога любовь.
Apr 27 2017
prev sibling next sibling parent reply =?UTF-8?B?THXDrXM=?= Marques <luis luismarques.eu> writes:
On Thursday, 27 April 2017 at 14:53:02 UTC, Mike Parker wrote:

 To help in compiling the list, what are some major issues from 
 the ecosystem that you'd like to see fixed?


Backtraces with line information on macOS?
Apr 27 2017
parent Jacob Carlborg <doob me.com> writes:
On 2017-04-28 04:12, Luís Marques wrote:


 Backtraces with line information on macOS?


That would be nice. Should be fairly trivial to look at the Linux 
implementation and to the same but read the data from different sections.

-- 
/Jacob Carlborg
Apr 28 2017
prev sibling next sibling parent Adrian Matoga <dlang.spam matoga.info> writes:
On Thursday, 27 April 2017 at 14:53:02 UTC, Mike Parker wrote:

 This year, DConf has an extra day tacked on for problem solving 
 in the form of a hackathon. The intent is to work on issues 
 people find frustrating in the D ecosystem. While there will be 
 time given at the event for proposals, and those involving 
 third-party projects are welcome, it will help speed things 
 along for the organizers to come in with a list of big issues 
 to get the ball rolling.

 To help in compiling the list, what are some major issues from 
 the ecosystem that you'd like to see fixed?


It's probably not a "major issue", but I'd be interested in 
joining any efforts towards making dub more 
cross-compilation-friendly. "--compiler=" and "--arch=" aren't 
quite useful in any serious scenario with multiple target 
platforms that use different compiler settings and sysroots.
Apr 28 2017
prev sibling next sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Thursday, 27 April 2017 at 14:53:02 UTC, Mike Parker wrote:

 This year, DConf has an extra day tacked on for problem solving 
 in the form of a hackathon. The intent is to work on issues 
 people find frustrating in the D ecosystem. While there will be 
 time given at the event for proposals, and those involving 
 third-party projects are welcome, it will help speed things 
 along for the organizers to come in with a list of big issues 
 to get the ball rolling.

 To help in compiling the list, what are some major issues from 
 the ecosystem that you'd like to see fixed?


Here's my wishlist:

D's  safe-ty story

DMD as a library (both frontend and backend):
* the backend usable as a JIT library.
* unittests (!= integration tests like those in 
https://github.com/dlang/dmd/tree/master/test)
* fixing the GC interoperability (AFAIK, dmd currently can't use 
the GC)
* less coupling (e.g. https://github.com/dlang/dmd/pull/6625)
* no use of global state, except in the compiler driver
* task-based parallelism
* better docs
etc.

D REPL based on dmd library. There were a couple of attempts 
(e.g. https://github.com/callumenator/dabble, 
https://github.com/MartinNowak/drepl), but as far I know, none of 
them is using dmd as a library and I'm not sure how close they 
are actually being useful.

AST introspection - given a function definition (!= declaration, 
i.e. the body is available) f, the expression __traits(ast, f) 
should return an instance of FuncDeclaration 
(https://github.com/dlang/dmd/blob/197ff0fd84b7b101f47458ab06e5b6f95959941e/src/
dmd/astnull.d#L151) with all of its members filled. The class-es used to
represent the AST should be plain old data types (i.e. should contain no
functionality). The AST as produced by the parser should be returned, without
any semantic analysis (which can modify it). No backwards compatibility
guarantees should be made about __traits(ast, f) at least for a couple of years.

DMD usable as a library can help the various IDE projects improve 
the user experience, which is major complaint of new comers (e.g. 
better auto-completion, refactoring, etc).
AST introspection (in addition with the previous point) is needed 
for D to remain competitive on the metaprogramming front with 
other rising languages such as Jai or Nim.

I would love to meet and work with other people interested in 
those areas at DConf.
Apr 28 2017
parent reply Moritz Maxeiner <moritz ucworks.org> writes:
On Friday, 28 April 2017 at 09:52:29 UTC, Petar Kirov 
[ZombineDev] wrote:

 AST introspection - given a function definition (!= 
 declaration, i.e. the body is available) f, the expression 
 __traits(ast, f) should return an instance of FuncDeclaration 
 (https://github.com/dlang/dmd/blob/197ff0fd84b7b101f47458ab06e5b6f95959941e/src/
dmd/astnull.d#L151) with all of its members filled. The class-es used to
represent the AST should be plain old data types (i.e. should contain no
functionality). The AST as produced by the parser should be returned, without
any semantic analysis (which can modify it). No backwards compatibility
guarantees should be made about __traits(ast, f) at least for a couple of years.


This sounds interesting, but could you expand on what this 
enables/improves compared to CTFE+mixins?
Apr 28 2017
parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Friday, 28 April 2017 at 10:09:32 UTC, Moritz Maxeiner wrote:

 On Friday, 28 April 2017 at 09:52:29 UTC, Petar Kirov 
 [ZombineDev] wrote:

 AST introspection - given a function definition (!= 
 declaration, i.e. the body is available) f, the expression 
 __traits(ast, f) should return an instance of FuncDeclaration 
 (https://github.com/dlang/dmd/blob/197ff0fd84b7b101f47458ab06e5b6f95959941e/src/
dmd/astnull.d#L151) with all of its members filled. The class-es used to
represent the AST should be plain old data types (i.e. should contain no
functionality). The AST as produced by the parser should be returned, without
any semantic analysis (which can modify it). No backwards compatibility
guarantees should be made about __traits(ast, f) at least for a couple of years.


 This sounds interesting, but could you expand on what this 
 enables/improves compared to CTFE+mixins?


CTFE and mixins are building blocks, needed to for my idea to 
actually useful.
Currently if you want to introspect a piece of code (function 
body) at compile time,
you need to duplicate the code in a string and then pass this 
string to a
CTFE-able D parser. As you can imagine, even with Stefan's new 
CTFE engine,
this would be orders of magnitude slower than just reusing the 
work that parser
inside the compiler *has already done* anyway. This makes such 
code introspection
infeasible for large projects. Strings (at least until mixined)
can contain uncompilable source (though lexically valid, in the 
case of q{}),
further complicating the matter. Additionally, the fact that you 
need to
keep the source code and the strings in sync is just stupid, 
violating DRY
at a whole new level.

In addition to AST introspection, AST transformation should be as 
easy as:

mixin template profileFunctionStatements(alias func, string 
newFunctionName)
{
     enum funcAst = __traits(ast, func);
     enum newAst = insertProfiling(funcAst, newFunctionName);
     mixin(newAst.toString());

     // a further optimization would be AST mixins, which
     // could directly be used by the compiler, instead of
     // first converting the AST to string and then
     // parsing it after mixing:
     mixin(newAst);
}

void main()
{
     int local = 42;

     void fun(int[] arr)
     {
         import std.conv : text;
         import std.file : remove, write;
         arr[] += local;
         string s = text(arr);
         "delete-me.txt".write(s);
     }

     mixin profileFunctionStatements!(print, `funInstrumented`);

     import std.array : array;
     import std.range : iota;
     funInstrumented(10000.iota.array);
}

Output:
{  arr[] += local;  } took 0.002 miliseconds to execute.
{  string s = text(arr);  } took 1.8052 miliseconds to execute.
{ "delete-me.txt".write(s); } took 7.746 miliseconds to execute.

Where funInstrumented could be generated to something like this:

void funInstrumented(int[] arr)
{
     import std.datetime : __Sw = StopWatch, __to = to; // 
generated
     import std.stdio : __wfln = writefln; // generated
     import std.conv : text;
     import std.file : remove, write;

     __Sw __sw; // generated

     __sw.start(); // generated
     arr[] += local;
     __sw.stop(); // generated
     __wfln("{ %s } took %s miliseconds to execute.",
         q{ arr[] += local; },
         __sw.peek().__to!("msecs", double)); // generated

     __sw.start(); // generated
     string s = text(arr);
     __sw.stop(); // generated
     __wfln("{ %s } took %s miliseconds to execute.",
         q{ string s = text(arr); },
         __sw.peek().__to!("msecs", double)); // generated

     __sw.start(); // generated
     "delete-me.txt".write(s);
      __sw.stop(); // generated
     __wfln("{%s} took %s miliseconds to execute.",
         q{ "delete-me.txt".write(s); },
         __sw.peek().__to!("msecs", double)); // generated
}

Other applications include:
* compiling/transpiling D functions to targets
like JS, SPIR-V, WebAssembly, etc. using CTFE.
* CTFE-driven code diagnostics (linting)
* Adding semantic to user defined attributes:
   E.g.  asyncSafe attribute for use in libraries like vibe.d that 
allows calling only
   functions marked as  asyncSafe from  asyncSafe code. That way 
libraries can
   introduce *and enforce* correct use of UDAs without any need 
for language changes.
* ...
Apr 28 2017
next sibling parent Moritz Maxeiner <moritz ucworks.org> writes:
On Friday, 28 April 2017 at 13:31:33 UTC, Petar Kirov 
[ZombineDev] wrote:

 [...]

 Other applications include:
 * compiling/transpiling D functions to targets
 like JS, SPIR-V, WebAssembly, etc. using CTFE.
 * CTFE-driven code diagnostics (linting)
 * Adding semantic to user defined attributes:
   E.g.  asyncSafe attribute for use in libraries like vibe.d 
 that allows calling only
   functions marked as  asyncSafe from  asyncSafe code. That way 
 libraries can
   introduce *and enforce* correct use of UDAs without any need 
 for language changes.
 * ...


Thanks for the summary :)
Apr 28 2017
prev sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Friday, 28 April 2017 at 13:31:33 UTC, Petar Kirov 
[ZombineDev] wrote:

 Other applications include:
 * compiling/transpiling D functions to targets
 like JS, SPIR-V,


I got you covered ;)
(LDC not CTFE though. It would be fiendishly complicated to do at 
CTFE as a fair amount of compiler magic is necessary.)
Apr 28 2017
parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Saturday, 29 April 2017 at 00:54:59 UTC, Nicholas Wilson wrote:

 On Friday, 28 April 2017 at 13:31:33 UTC, Petar Kirov 
 [ZombineDev] wrote:

 Other applications include:
 * compiling/transpiling D functions to targets
 like JS, SPIR-V,


 I got you covered ;)


I know, and I'm looking forward to using your GPU support in LDC 
:P


 (LDC not CTFE though. It would be fiendishly complicated to do 
 at CTFE as a fair amount of compiler magic is necessary.)


The way I see it, metaprogramming is a whole spectrum. Yes, you
need a full compiler if you want to compile a whole program to
e.g. JS or WebAsm, but there are many areas where even the 
smallest
improvement would be highly beneficial.


safely ignore constant folding, which works only with string and 
number literals),
yet it has pretty powerful reflection and code-generation 
capabilities
at run-time. Even though the performance difference between CT 
and RT
reflection is orders of magnitude, those reflection capabilities 
are used
pervasively throughout the whole ecosystem. The prime example 
being LINQ -
probably the most widely used feature of .NET. Given a chain of 
operations
(similar to D's ranges) like:

dbContext.Persons
   .Where(p => p.Birthdate < DateTime.Now.Date.AddYears(-18))
   .Where(p => p.Birthdate > DateTime.Now.Date.AddYears(-28))
   .Select(p => new { Name = p.FirstName + " " + p.LastName, 
Birthdate = p.Birthdate })

It gets compiled to the following SQL:
-- Region Parameters
DECLARE  p0 DateTime = '1989-05-01 00:00:00.000'
DECLARE  p1 DateTime = '1999-05-01 00:00:00.000'
DECLARE  p2 NVarChar(1000) = ' '
-- EndRegion
SELECT ([t0].[FirstName] +  p2) + [t0].[LastName] AS [Name], 
[t0].[Birthdate]
FROM [Person] AS [t0]
WHERE ([t0].[Birthdate] >  p0) AND ([t0].[Birthdate] <  p1)

As you can see the mapping and filtering is done entirely on the
database server side. The only magic need is to make the compiler 
to



InputRange!T filter(T)(InputRange!T source, bool delegate(T) 
predicate)

is expressed as:

IEnumerable<T> Where<T>(IEnumerable<T> source, Func<T, bool> 
predicate)

In order to request the AST of the predicate function, it needs to
be wrapped in Expression<T>:

IQueryable<T> Where<T>(
     IQueryable<T> source, Expression<Func<T, bool>> predicate)

(IQueryable<T> [1] is an extension of IEnumerable<T> [2] 
interface (which is
similar to D's Input/ForwardRange-s) which adds the Expression 
property
which represents the AST of the query against the IQueryable data 
source.)

----

In addition to compiling range operations to database queries, 
this would
also be useful specializing on lambda's in range libraries (see 
[3]) and
much more.


[0]: 
https://msdn.microsoft.com/en-us/library/bb335710(v=vs.110).aspx
[1]: 
https://msdn.microsoft.com/en-us/library/bb351562(v=vs.110).aspx
[2]: 
https://msdn.microsoft.com/en-us/library/9eekhta0(v=vs.110).aspx
[3]: https://github.com/dlang/phobos/pull/4265
May 01 2017
parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Monday, 1 May 2017 at 12:35:11 UTC, Petar Kirov [ZombineDev] 
wrote:

 On Saturday, 29 April 2017 at 00:54:59 UTC, Nicholas Wilson 
 wrote:

 On Friday, 28 April 2017 at 13:31:33 UTC, Petar Kirov 
 [ZombineDev] wrote:

 Other applications include:
 * compiling/transpiling D functions to targets
 like JS, SPIR-V,


 I got you covered ;)


 I know, and I'm looking forward to using your GPU support in 
 LDC :P


 (LDC not CTFE though. It would be fiendishly complicated to do 
 at CTFE as a fair amount of compiler magic is necessary.)


 The way I see it, metaprogramming is a whole spectrum. Yes, you
 need a full compiler if you want to compile a whole program to
 e.g. JS or WebAsm, but there are many areas where even the 
 smallest
 improvement would be highly beneficial.


Oh I agree. For text to text (or lambda to text) this is 
comprehensible and not too difficult (perhaps even simple with 
Dmitry Olshansky's Pry) but SPIRV is rather complicated, as 
evidenced by a 203 page spec *(that doesn't even include the 
OpenCL or Vulkan specific stuff). Having said that it shouldn't 
be too difficult to port the tablegen tables I've been working on 
for LLVM to D (or write a D backend for tablegen) and get most of 
the really boring work out of the way.
I won't stop you, but just letting you know the rabbit hole is 
quite deep :)

The good news is the the runtime stuff will be all transferable.
This approach for SPIRV won't automatically translate for NVPTX 
like it will for ldc though, which is one of the main draws for 
dcompute

*Other languages probably have longer specs, but this is just for 
the format.



 safely ignore constant folding, which works only with string 
 and number literals),
 yet it has pretty powerful reflection and code-generation 
 capabilities
 at run-time. Even though the performance difference between CT 
 and RT
 reflection is orders of magnitude, those reflection 
 capabilities are used
 pervasively throughout the whole ecosystem. The prime example 
 being LINQ -
 probably the most widely used feature of .NET. Given a chain of 
 operations
 (similar to D's ranges) like:

 dbContext.Persons
   .Where(p => p.Birthdate < DateTime.Now.Date.AddYears(-18))
   .Where(p => p.Birthdate > DateTime.Now.Date.AddYears(-28))
   .Select(p => new { Name = p.FirstName + " " + p.LastName, 
 Birthdate = p.Birthdate })

 It gets compiled to the following SQL:
 -- Region Parameters
 DECLARE  p0 DateTime = '1989-05-01 00:00:00.000'
 DECLARE  p1 DateTime = '1999-05-01 00:00:00.000'
 DECLARE  p2 NVarChar(1000) = ' '
 -- EndRegion
 SELECT ([t0].[FirstName] +  p2) + [t0].[LastName] AS [Name], 
 [t0].[Birthdate]
 FROM [Person] AS [t0]
 WHERE ([t0].[Birthdate] >  p0) AND ([t0].[Birthdate] <  p1)

 As you can see the mapping and filtering is done entirely on the
 database server side. The only magic need is to make the 
 compiler to



 InputRange!T filter(T)(InputRange!T source, bool delegate(T) 
 predicate)

 is expressed as:

 IEnumerable<T> Where<T>(IEnumerable<T> source, Func<T, bool> 
 predicate)

 In order to request the AST of the predicate function, it needs 
 to
 be wrapped in Expression<T>:

 IQueryable<T> Where<T>(
     IQueryable<T> source, Expression<Func<T, bool>> predicate)

 (IQueryable<T> [1] is an extension of IEnumerable<T> [2] 
 interface (which is
 similar to D's Input/ForwardRange-s) which adds the Expression 
 property
 which represents the AST of the query against the IQueryable 
 data source.)

 ----

 In addition to compiling range operations to database queries, 
 this would
 also be useful specializing on lambda's in range libraries (see 
 [3]) and
 much more.


 [0]: 
 https://msdn.microsoft.com/en-us/library/bb335710(v=vs.110).aspx
 [1]: 
 https://msdn.microsoft.com/en-us/library/bb351562(v=vs.110).aspx
 [2]: 
 https://msdn.microsoft.com/en-us/library/9eekhta0(v=vs.110).aspx
 [3]: https://github.com/dlang/phobos/pull/4265
May 01 2017
prev sibling next sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Thursday, 27 April 2017 at 14:53:02 UTC, Mike Parker wrote:

 This year, DConf has an extra day tacked on for problem solving 
 in the form of a hackathon. The intent is to work on issues 
 people find frustrating in the D ecosystem. While there will be 
 time given at the event for proposals, and those involving 
 third-party projects are welcome, it will help speed things 
 along for the organizers to come in with a list of big issues 
 to get the ball rolling.

 To help in compiling the list, what are some major issues from 
 the ecosystem that you'd like to see fixed?


Not so much major issue but I would like to:
     * figure out a solution for 
https://github.com/ldc-developers/ldc/issues/2011
     * consider the merits of standardising allocSize 
(https://github.com/ldc-developers/druntime/blob/ldc/src/ldc/attributes.d#L16)
or equivalent among dmd/ldc/gdc
     * get half precision floating point into the language /or/ 
ability to create __vector's of user types (need for intrinsics 
for GPU et al targets of dcompute).

I would also like to hold a mini hackathon/gauge interest in 
dcompute as we could benefit significantly from the ML craze.
Apr 28 2017
prev sibling next sibling parent reply Daniel N <no public.email> writes:
On Thursday, 27 April 2017 at 14:53:02 UTC, Mike Parker wrote:

 This year, DConf has an extra day tacked on for problem solving 
 in the form of a hackathon. The intent is to work on issues 
 people find frustrating in the D ecosystem. While there will be 
 time given at the event for proposals, and those involving 
 third-party projects are welcome, it will help speed things 
 along for the organizers to come in with a list of big issues 
 to get the ball rolling.

 To help in compiling the list, what are some major issues from 
 the ecosystem that you'd like to see fixed?


Wishlist
1) Make alias parameters work with buildin types, I bet anyone 
using D has fallen into trip trap.

alias X(alias T) = T;
X!int x;
Apr 29 2017
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/29/17 5:42 AM, Daniel N wrote:

 On Thursday, 27 April 2017 at 14:53:02 UTC, Mike Parker wrote:

 This year, DConf has an extra day tacked on for problem solving in the
 form of a hackathon. The intent is to work on issues people find
 frustrating in the D ecosystem. While there will be time given at the
 event for proposals, and those involving third-party projects are
 welcome, it will help speed things along for the organizers to come in
 with a list of big issues to get the ball rolling.

 To help in compiling the list, what are some major issues from the
 ecosystem that you'd like to see fixed?


 Wishlist
 1) Make alias parameters work with buildin types, I bet anyone using D
 has fallen into trip trap.

 alias X(alias T) = T;
 X!int x;


+1, I would love to see this fixed. I recall Walter said it should be 
fixed 2 years ago in Utah.

-Steve
May 01 2017
prev sibling parent Joakim <dlang joakim.fea.st> writes:
On Thursday, 27 April 2017 at 14:53:02 UTC, Mike Parker wrote:

 This year, DConf has an extra day tacked on for problem solving 
 in the form of a hackathon. The intent is to work on issues 
 people find frustrating in the D ecosystem. While there will be 
 time given at the event for proposals, and those involving 
 third-party projects are welcome, it will help speed things 
 along for the organizers to come in with a list of big issues 
 to get the ball rolling.

 To help in compiling the list, what are some major issues from 
 the ecosystem that you'd like to see fixed?


Probably the plan anyway, but my suggestion would be for the core 
team to not hack on anything, but spend the time discussing 
issues that have been discussed to death online but not resolved, 
such as some devs not agreeing with Walter and the DIP 1000 path 
he's taking.

Use the in-person time to get some heavy bandwidth on those 
issues and try to make sure the differences are hashed out.  
There may not be a final agreement on the solution, but there 
certainly shouldn't be any more misunderstanding of the proposed 
options.

For people not on the core team, they can hack on a lot of the 
stuff mentioned in this thread, perhaps after coordinating with 
the core team about what's really needed and how to go about 
doing it.

Great idea, btw, to set aside a day just for this.
May 01 2017