www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - D Language Foundation April 2023 Monthly Meeting Summary

reply Mike Parker <aldacron gmail.com> writes:
We held the monthly meeting for April 2023, on Friday the 14th at 
14:00 UTC. It lasted just short of an hour and a half. We'd [had 
a quarterly meeting the previous 
week](https://forum.dlang.org/thread/yuewobigwhvrtfxlzxpk forum.dlang.org).
This was the first time wev'e held a monthly meeting in the month of a
quarterly. Previously, we'd combined them, but [decided in our March
meeting](https://forum.dlang.org/thread/aaebiekxqbwgxzkkynef forum.dlang.org)
to split them up.

The following people attended:

* Walter Bright
* Martin Kinkelin
* Dennis Korpel
* Mathias Lang
* Átila Neves
* Razvan Nitu
* Mike Parker
* Robert Schadek




I opened the meeting with an update on DConf preparations. Most 
of it was boring details. Other than that, the venue contract had 
been signed and a down payment sent a couple of weeks before, and 
Symmetry had reached an agreement with our event planner on the 
terms and conditions of their contract. I was waiting on word 
that it was signed (I have since received it).

I noted that Sönke Ludwig had contacted me before I left for my 
trip in late March. He was doing some maintenance work on 
codemirror.dlang.org and asked if I wanted to redirect the URL to 
his new VPS or if he should move it onto a DLF server. Since this 
is one of the services we need to move over eventually anyway, I 
asked him to go ahead and do it. Vladimir Panteleev had already 
set up the documentation tester on a DLF server, so I reached out 
to him to see if we could move codemirror there or if we needed a 
new one. After discussing it with Sönke, he said we had enough 
resources for both services. So I set up Sönke with an account, 
then he and Vladimir got it up and running while I was on my trip.



__CI failures__

Dennis started by saying that the CI was randomly failing again. 
He didn't have a Mac, so he'd been unable to chase down the 
problem. Random CI failures are a recurring problem. There are so 
many checks, and he doesn't know who created them or who knows 
exactly what the checks are doing. He wishes the tests had 
someone responsible for them who he could turn to when they fail.

Walter asked who had previously been in charge of the tests. 
Razvan said he didn't recall if one person was ever in charge of 
them. At some point, someone decided it was a good idea to have a 
particular test and it got added to the pipeline.

Dennis asked if we should only keep tests that have a maintainer. 
Martin and Mathias quickly rejected that. Martin said the tests 
are good. CI failures are usually caused by CI image bumps or a 
PR. CI image changes are a PITA for LDC's tests, and PR-related 
failures may not be easy to resolve, but failures are hardly ever 
the fault of a test. And there's never been any specific person 
responsible for any of DMD's CI systems. They just grew 
organically. Then someone who knew the details moved on and no 
one else knows them... it's a constant maintenance burden, but 
it's worth the effort.

There was a bit more discussion about the maintenance burden, 
after which I noted that this is the story of our ecosystem. 
We're responsible now for things none of us set up, and we need 
to get a handle on it all. Dennis agreed and added that the CI is 
in a special position. When one of them is outdated, it doesn't 
just sit there out of the way, it becomes an annoyance to 
development.

(NOTE: This is one of the many aspects of our ecosystem that 
we'll be working to improve [under our new 
workflow](https://forum.dlang.org/thread/avvmlvjmvdniwwxemcqu forum.dlang.org).)

__The -J compiler switch__

Next, Dennis said that, as he understood it, `-J` was added as a 
measure against [string 
imports](https://dlang.org/spec/expression.html#import_expressions) being used
as a malware vector. As he sees it, string imports are no more risky than
normal imports. Why can't they be handled the same way? Walter said the concern
was that they could be used to read any file from the filesystem and bake it
into the binary, and that could be used to steal data from the user's system.
With normal imports, the compiler only reads D source files. It can't import
arbitrary data. He then talked about some examples. What it comes down to is
that there are so many clever ways to implement malware, he just doesn't want
to open that door wide to the possibility.

Martin talked a bit about how Symmetry uses `-J` in their 
codebase, and Átila told us how he used it a lot with reggae. 
Dennis proposed that any directory given to `-J` be treated as a 
root, and its subdirectories added to the search path. Martin and 
Átila both said that the compiler already allows string imports 
from subdirectories of any `-J` directory (when the full path is 
included in the import expression).

__The DMD backend__

Dennis said that because the DMD backend was shared with the 
Digital Mars C and C++ compiler (DMC), there was a rule in place 
that changes to the backend couldn't break DMC, but there were no 
checks for it. Now that ImportC is a thing, do we still need to 
maintain DMD's backend with an eye toward not breaking DMC, or 
can we say that DMD's backend is only for DMD and just leave DMC 
frozen in time?

Walter said he had given up on maintaining DMC. The DMD backend 
had already diverged, and he saw no further point in maintaining 
DMC. Dennis said that will make maintaining the backend easier, 
and we could remove the notes in the source about being careful 
because of DMC. Walter agreed.

Dennis said that when building DMD, the backend is using some 
translated header files where it's using extern declarations 
instead of D imports. Those should be changed to more modern D 
instead of this archaic header business. Walter agreed. He said 
there's a lot of old-fashioned stuff in there. He hadn't taken 
advantage of the separation to do any big refactoring but had 
stopped backporting changes to the DMC backend a while ago.

Dennis next noted there were a bunch of `version(Mars)` and 
`version(scpp)` blocks, and asked if they could be changed to 
`version(all)` and `version(none)`, and the latter then removed. 
Walter said he's not in a rush to do a big refactoring, but yes, 
that's fine. He hadn't worked on the backend for a while other 
than for bug fixes and had gradually changed some things to a D 
style in the process. He gave a recent example involving some 
changes he made when he fixed an inliner bug.

Walter said ImportC will probably never replace DMC, mainly 
because DMC also compiles C++ code. Dennis added that DMC also 
still has 16-bit support for the niche programmer.

(UPDATE: Dennis has since submitted several PRs removing function 
prototypes from the backend, [starting with 
pt1](https://github.com/dlang/dmd/pull/15111).)


Mathias said he had nothing to report. Walter asked what he had 
been working on. He said he hadn't been working on any D stuff 
lately. He still [had an open 
PR](https://github.com/dlang/dmd/pull/14999) that fixes [a CTFE 
bug](https://issues.dlang.org/show_bug.cgi?id=23785) related to 
`-preiview=in`. In the PR thread, Martin had pointed out an issue 
with it regarding the evaluation order of argument expressions, 
and then the two had discussed it further outside of the PR 
thread, where Martin explained how the order of destruction in 
the presence of exceptions needs to be taken care of. That led 
Mathias down a rabbit hole, as it seems there are cases where 
destructors aren't called in CTFE, so the PR was stalled (and 
it's still open as I write this). He plans to finish it up, as 
his goal is to make it possible for `-preview=in` to be enabled 
by default.


Robert said he was going to ping me the following week about 
finding some time to get the Bugzilla to GitHub migration. Other 
than that, the compiler was still too slow.

(NOTE: He eventually emailed me everything I need to get the 
Bugzilla to GitHub migration done. Now I'm the bottleneck. I'm 
behind on everything right now, but I'm slowly catching up. When 
I work my way to this task, I'm first going to test it out on a 
private repo to make sure I've got everything in hand. Then I'll 
make an announcement here in the forums with a date on which I 
plan to start. When that date comes around, I'll open a forum 
thread with periodic updates, e.g., "starting the migration of 
tools issues now", and so on. We're almost there!)


__Fixing issues__

Razvan reported that he had mostly been fixing segfaults in the 
compiler and had found that there were some ICEs in the C++ 
mangler. He'd submitted some PRs to fix those. There was [one in 
particular](https://github.com/dlang/dmd/pull/15099) for which he 
wasn't sure if the fix was the right one. He asked Mathias to 
take a look at it as the original author.

Razvan had also been fixing some nosharedaccess issues. He had 
seen a regression that Átila had reported, but other than that 
there were no other open nosharedaccess issues. He expected there 
would be more bugs out there, and Átila expected he'd find some.

__BetterC linker errors__

Next, Razvan said he'd been looking into BetterC errors. He then 
described [an issue related to template instantiations in 
CTFE](https://issues.dlang.org/show_bug.cgi?id=23799). When a 
template includes runtime hooks, it's no problem in BetterC if 
it's only used for CTFE and there's no code generation, but 
sometimes the same instantiation gets used at runtime, the front 
end doesn't catch it, and it results in linker errors. The 
problem was uncovered [by a fix from 
Walter](https://github.com/dlang/dmd/pull/14791) for [an issue 
related to ` nogc` functions in 
BetterC](https://issues.dlang.org/show_bug.cgi?id=23606), and 
that PR also [introduced a 
regression](https://issues.dlang.org/show_bug.cgi?id=23803).

Razvan then described a related BetterC problem. There are some 
cases where the compiler tries to semantically analyze the body 
of a generated function and marks the function as disabled if 
something fails. He then described [a chain of consequences that 
can arise in 
BetterC](https://issues.dlang.org/show_bug.cgi?id=23806) in which 
the generated function ends up instantiating a template that uses 
a DRuntime hook, and that instantiation ends up causing linker 
errors even though the generated function that instantiated it 
was disabled.

He said the fundamental problem is that the compiler currently 
has no way to reanalyze template instances once they've been 
cached. He's not sure how best to solve this. He said he doesn't 
like the way the compiler is going with special cases: if you're 
in ImportC do this, if you're in BetterC do that, if you're in 
CTFE do this other thing. Átila and Martin nodded in agreement. 
Martin said that's his main problem with BetterC. It's full of 
hacks in the front end and the glue layer.

Martin said he'd seen many missing symbol errors in BetterC. The 
majority of them were related to template culling, which is a 
compiler optimization. He described how it works, and noted that 
it doesn't happen when using the `-allinst` switch. That's the 
first thing he suggests trying when linker errors arise in 
BetterC. When `-allinst` works, then the problem is probably just 
template culling.

Regarding reanalyzing template instances, Martin said the 
compiler does try to do it. That's the cause for a funny issue 
where you have `static if(!__traits(compiles, something) { // do 
the something that failed the compiles test }` and it only works 
with the `static if(!_traits(compiles, ...)`. When analysis of 
template instantiation fails and the compiler comes across a 
second instantiation, it tries again and again, and sometimes it 
works due to forward referencing issues (which he calls the root 
of all evil and says is the main problem we have with the 
frontend).

Átila agreed that the special case stuff is something we 
shouldn't be doing. He thinks we should get to the point where 
BetterC is a no-op. If your code doesn't require the runtime, 
then it just shouldn't be linked. You shouldn't have to do 
anything special to get that behavior. The only thing the 
`-betterC` switch should do is tell the compiler to analyze your 
code and tell you if something won't work in BetterC. Nothing 
else would change.

Walter said the root of the problem is trying to use CTFE in 
BetterC. CTFE uses the GC to allocate and is crippled without it. 
Átila said that should be possible in BetterC, and Razvan agreed. 
Walter said the problem there is: how can the compiler know that 
the function is only used in CTFE?

This took the discussion down into the weeds, and I'm not going 
to attempt to summarize all of it. Eventually, Martin said the 
root problem is that the BetterC checks (for `TypeInfo`, 
`ModuleInfo`, etc.) are done in the front end rather than in the 
glue layer, and that's a problem for CTFE. If they were done in 
the glue layer, it would be a PITA in that each compiler 
implementation would have to do it then, but at least then you'd 
only get issues if that thing is going to be codegenned.

Dennis asked if we should have a separate BetterC pass that does 
the checks between Semantic3 and codegen. Martin said that's an 
option at least. Átila reiterated BetterC should be a no-op. 
Martin agreed that's a great long-term goal, but before we get 
there we need to get rid of the module and type infos.

Átila took us back to the idea of indicating a function is 
CTFE-only. Rather than add a new UDA, why not just use an 
`in(__ctfe)` contract? Then it's easier for the compiler to 
analyze, as there's no need to look at the body of the function. 
Walter said he'd used `if(__ctfe} { // code } else assert(0)`. 
Átila thinks the contract is a lot prettier than that, and Walter 
agreed, and it does avoid the need for new syntax.

Átila said it also makes sense in that it declares the function 
contract as being "I only work in CTFE, so don't bother calling 
me otherwise". It's not even a hack. Dennis thought it is a bit 
of a hack because you can still take the address of it. You have 
to treat it as a static condition, but `__ctfe` is a runtime 
variable. Regardless, Walter thought it was a very good idea 
worth exploring.

Martin suggested using `if(__ctfe)` as a template constraint 
instead of an `in` contract. Dennis liked that idea, but Walter 
said he didn't want to start applying template constraints to 
ordinary functions. That would be new syntax, and that's what the 
`in(__ctfe)` was avoiding. Átila said it implies a template 
function and probably isn't what we want.

__More on BetterC__

Razvan next brought up Teodor Dutu's work on [converting DRuntime 
hooks to templates](https://youtu.be/dsa8GWL6TUo). An intended 
side effect of this effort is that it should make things more 
usable in BetterC, but sometimes, getting real work done in one 
of these templates means calling into the C library (`memset`, 
`memcpy`, etc). This is an issue when there's no standard C 
library.

Walter said BetterC was never intended to be used without a 
standard C library, so relying on the presence of those functions 
is fine.

This prompted more discussion about linker issues with BetterC 
and how to alleviate them. At the end of which Martin said 
BetterC is a sledgehammer, and that's exactly why it has the 
special case code Razvan mentioned before. We need to be working 
to cut that down.

Walter said we should look into redesigning features so that they 
don't need `TypeInfo`. Átila thinks `TypeInfo` should be opt-in 
anyway. There are ways to use compile-time reflection to generate 
all of that for us. Martin said the new templated hooks help in 
that regard (though he later admitted that they were a bit of a 
PITA causing some extra work for the GC-to-stack optimization in 
LDC, which he went into some detail about). The big thing 
hindering it is associative arrays. They heavily use `TypeInfo`. 
He mentioned [Steven Schveighoffer's 
implementation](https://github.com/schveiguy/newaa) which, as far 
as he knows, is free of `TypeInfo`. Maybe eventually we can get 
to use that.


__LDC__

Martin said he was swamped with LDC-related work, with no time to 
work on the front end. He still had a lot of catching up to do 
with LLVM versions and the front-end version. LDC wasn't yet at D 
2.103. The tests were looking quite good, but the previous 
release had some problems. LLVM 15 brought some nasty surprises 
that had only been caught after the final release of LDC 1.32, so 
he needed to put out a point release (which [he did a few days 
later](https://forum.dlang.org/thread/kcunuyrsogkhlljmyink forum.dlang.org)).

He had noticed that the LDC beta releases didn't seem to be used 
as much as they used to be. They used to see 1000 downloads of 
beta releases in the first week, but with the most recent they'd 
had only 300 or 400 in the first two weeks. It could be because 
we have more frequent LDC releases now, so people just don't 
bother with the beta anymore. He was thinking about doing some 
announcements about how maintainers of dub packages can 
contribute to compiler stability. For example, in GitHub actions, 
there's the "latest beta" tag for LDC (he says it's a misnomer, 
as it always refers to the most recent release, whether it's a 
final or a beta). If people set up scheduled builds using that 
tag every week or so and report issues when they happen, that 
could improve the situation.

__DIP1000 issues__

Martin said they were still using D 2.100 at Symmetry because the 
2.101 release came with lots of scope-related deprecations, which 
are enabled even if you aren't compiling with `-preview=dip1000` 
(they're errors with the preview, and [without it they're 
deprecations](https://dlang.org/changelog/2.101.0.html#dmd.dip1000_dep
ecation_warnings)). In the beginning, there were about 1000 deprecations in
their main project, which consists of around 150 dub packages overall. At the
time of the meeting, they reduced the count to 200 by hunting them down and
changing the code. Mostly, the deprecations were in third-party packages.

They had also encountered an issue related to using 
`std.algorithm.all` with something else (he couldn't remember). 
It prevents them from looping over an array of structs that have 
`opEquals` in ` safe` code, and there's nothing they can do about 
it on the user side. He said it might be a Phobos bug, but he 
suspects it's a compiler issue regarding attribute inference. 
He'd not yet had time to reduce it and file an issue.

Also, he had worked on fixing deprecations in [the concurrency 
dub 
package](https://github.com/symmetryinvestments/concurrency/pull/77). He got
those fixed for a library using that package, but then trying to compile the
concurrency package's unit tests segfaulted the compiler. This happened in both
the D 2.101 and 2.102 versions of DMD and LDC. Again, he'd not yet had time to
reduce it to file an issue. Dennis offered to take a look at it.

Átila said the main takeaway here is that dealing with 
deprecations is time-consuming, especially when they're in 
third-party code. And in this case, trying to find out why the 
deprecated code would be inferred as ` system` instead of ` safe` 
isn't always trivial. Maybe we should rethink DIP1000. Symmetry, 
at least, is devoting resources to it, but most people would just 
give up.

Dennis said he preferred using `return scope` as little as 
possible and just letting the compiler infer it. It's annoying 
opening BuildKite projects to fix related issues when someone got 
`return scope` wrong because the compiler was wrong.

(There was more discussion about the value of ` safe`, and how 
the GC and tools like ASAN can mitigate the need for it.)


__DIP1008 issue__

Next, one of Symmetry's projects uses DIP1008. It makes sense 
there, as it's a project that interacts with Dot Net code, which 
also has a GC, and you're not able to reliably suspend one GC 
when the other one is active, so they're trying to avoid the D GC 
in most cases. They still need to throw exceptions, so that's why 
they're using `-preview=dip1008`. Now they want to statically 
link that project into the main project. The problem then is that 
all of the dub projects pulled in (around 300) will be compiled 
with it. That leads to garbled exception messages, as 
non-DIP1008-aware code doesn't prevent the exceptions from 
escaping their scope.

Martin proposed coming up with an alternative DIP1008 
implementation where the only change is that `nogc` checks are 
disabled for `throw` expressions. He said the team told him 
they'd be okay with leaving garbage lying around resulting from 
exceptions. He thought it would be pretty easy to get that done. 
He suggested implementing it as an experiment, then upstreaming 
it if it works with their codebase. It would mean not relying on 
`malloc` and reference counting, just allocating from the GC and 
never freeing it. The GC is disabled in that project anyway.

Walter said he's kind of sorry that D has exceptions at all. 
They're an unending source of problems. Using option types would 
have been a better design choice in hindsight, but at the time he 
was just following what C++ and Java did. One thing we might do 
is to remove exceptions from Phobos so that you can reliably use 
it without exceptions at all. Another thing is to continue 
minimizing the use of the GC in Phobos.

Dennis said that there are places in Phobos where exceptions are 
just ingrained in the API. He cited ranges specifically, where 
you can just decode text and get a `UTFException`, or parsing 
numbers could throw a `ConvException`. To change that would 
require all users to unwrap the integers, unwrap the codepoints, 
etc. Átila said we could use an `alias this` that behaves as it 
does now and if the user forgets to unwrap, you throw an 
exception then. Dennis said that was an interesting idea.

Walter gave the example of using `std.algorithm.chain` in 
`std.path` to avoid allocating memory when concatenating paths 
and returning the chain thing. He's wondering if there are other 
opportunities like that in Phobos that can help us avoid memory 
allocations and exceptions, but he hasn't gotten that far in 
looking into how to eliminate exceptions. Átila said he'd been 
thinking about copying Herb Sutter's idea for zero-overhead C++ 
exceptions. He said the basic idea is that you get rid of 
exceptions while keeping the syntax. (Here's [a link to the 
PDF](https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p0709r4.pdf)).

Átila had to leave right after the discussion above on exceptions.


Walter had been continuing his pause on implementing new features 
and had been going through the ImportC bug list. The biggest 
problem he'd run into is the pervasiveness of compiler extensions 
in system header files, and it's almost always unnecessary. He 
then went on an entertaining rant about it (a header will compile 
on FreeBSD but not macOS, then a different header succeeds on 
macOS and not FreeBSD; Microsoft is the worst offender; etc.). 
ImportC was intended to be a standard C compiler, but you can't 
compile any system headers with a standard C compiler. He ended 
up implementing a lot of extensions to get things to work.

On the plus side, he'd made a lot of headway with it. For 
example, he'd figured out how to implement statement expressions 
with just a few lines of code. He was shocked it was possible 
with only a few lines and was happy with the results. So he's 
making good progress with getting ImportC to compile all these 
random `.h` files. That and documenting this stuff was mostly 
what he'd been working on. He hoped to get the ImportC bug list 
down soon and turn to other bugs. (He posted about this in the 
forums a couple of weeks later with [a call for help finding 
problematic C 
headers](https://forum.dlang.org/post/u2el6n$1tee$1 digitalmars.com).)

He then reiterated he'd like to reduce GC and exceptions in 
Phobos and eliminate dependencies such as `TypeInfo`. The further 
we can go with that the better.

Next, he said he'd stepped up his use of Twitter. He encouraged 
us, and anyone with a Twitter account, to tweet more about what 
we're doing with D. He'd looked into common hashtags with 
programming posts and started using them more, and he also signed 
up for the blue checkmark. He has since noticed an increase in 
followers. If Twitter's not your thing, then consider posting on 
whatever social media platform you prefer. This is our most 
effective means of marketing. He and Andrei had tried various 
approaches to publishing ads via Google, Facebook, and other 
platforms in the past with no meaningful results whatsoever. What 
he had found effective was writing articles and just posting on 
social media about what you're doing. Just talk about the work 
you do with D.

I noted we had discussed before the idea of Walter either doing 
an occasional live-coding session or just recording himself 
working throughout the week and editing together a video summary 
of the work he'd done. Walter brought up what John Carmack did 
back in the day with his `.plan` files to keep gamers informed of 
the work he was doing. As Walter sees it, that was an early 
version of what Twitter has become. It's a great way to let your 
customers know what you're doing, and because they make a 
conscious decision to follow you, that's not considered spam. He 
decided to keep his tweets strictly related to programming 
topics. He isn't going to tweet about politics or what he had for 
breakfast. He'd followed several programmers because he wanted 
their opinions about programming, only to be put off when they 
tweet about everything else. He cited John Carmack again as an 
example of someone who tweets mostly about programming topics. He 
said that if you wanted to use Twitter for outreach, just keep it 
on topics related to programming.

I mentioned that I'd like to record some more conversations like 
the two I did with Walter last year ([Part 
One](https://youtu.be/-kkMYJN3MnA) and [Part 
Two](https://youtu.be/G6b62HmsO6M)). I have a list of people I 
plan to contact for that.

Razvan doesn't use Twitter, but he's interested in what Walter 
has to say about what he's working on. He suggested that Walter 
post updates on the forums for people like him. Walter said he 
could do more of that and that's a good idea.

Walter finished by saying that he has so much work to do and it 
keeps growing all the time. He said he appreciated Razvan and 
Dennis helping out with his pull requests. They'd been a big help.


The next meeting was a monthly that we held on May 5 at 14:00 UTC 
just before [our final IVY program 
session](https://forum.dlang.org/thread/avvmlvjmvdniwwxemcqu forum.dlang.org)
with UCora. I'll try to have the summary for that ready before the end of this
month. Then on May 12, we held the first of what will be weekly planning
meetings. I'll write more about that in a separate post.
May 14 2023
next sibling parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 15/05/2023 12:47 AM, Mike Parker wrote:

 Átila said he'd been thinking about copying Herb Sutter's idea for 
 zero-overhead C++ exceptions. He said the basic idea is that you get rid 
 of exceptions while keeping the syntax. (Here's a link to the PDF 
 <https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p0709r4.pdf>).


Design work is already done.

https://github.com/rikkimax/DIPs/blob/value_type_exceptions/DIPs/DIP1xxx-RC.md

I kinda think sumtypes should go first so we can do catch all.
May 14 2023
prev sibling next sibling parent reply ryuukk_ <ryuukk.dev gmail.com> writes:
 Razvan next brought up Teodor Dutu's work on converting 
 DRuntime hooks to templates. An intended side effect of this 
 effort is that it should make things more usable in BetterC, 
 but sometimes, getting real work done in one of these templates 
 means calling into the C library (memset, memcpy, etc). This is 
 an issue when there's no standard C library.



 Walter said BetterC was never intended to be used without a 
 standard C library, so relying on the presence of those 
 functions is fine.



Mistake mistake mistake..... betterC != betterLiBC

It means better C

I wanted to target WASM, if your hooks call into libC, i can't 
target WASM..

I'm pretty sure i reported an issue about this

Nobody understand the implication when they do stuff with the 
compiler

Then we end up with GC everywhere and the only way to workaround 
things is to subscribe to -betterC

You keep tell us that the plan is "pay as you go", well, sir, 
that kind of issue goes against your plan, so again, what's the 
plan?



Dependency on libC just for memcpy.............

Then my project no longer compile, thanks a lot!

https://github.com/dlang/dmd/pull/14910


That reminds me of a dependency on std.exception just to do: 
``return result.array``

https://github.com/dlang/phobos/pull/8723

Nobody care anymore.....

Then people wonder things are the way it is
May 14 2023
parent reply Dennis <dkorpel gmail.com> writes:
On Sunday, 14 May 2023 at 16:02:00 UTC, ryuukk_ wrote:

 I wanted to target WASM, if your hooks call into libC, i can't 
 target WASM..


Good news, there's implementations of libc for WASM (example: 
https://archlinux.org/packages/community/any/wasi-libc/), and 
linking them doesn't even increase build time or binary size that 
much in my experience. I did this for 
[ctod](https://github.com/dkorpel/ctod/tree/master) because I had 
to (the tree-sitter library I use depends on it), but I also 
incorporated it in my other WASM projects to get a better 
`malloc/free` implementation.


 You keep tell us that the plan is "pay as you go", well, sir, 
 that kind of issue goes against your plan, so again, what's the 
 plan?


The "pay as you go" plan relates to druntime, not libc. Not that 
we deliberately depend on libc when it's not needed (don't need 
`itoa`, we have `unsignedToTempString`), but certain libc calls 
are so standard, common, and optimized, that there's not much 
benefit in trying to roll our own implementation for it. I'm 
talking about memset, memcpy, memcmp, malloc, free, realloc, off 
the top of my head.

Even when there's not a single mention of `memcpy` in druntime, 
LDC might still even emit it causing your WASM project to fail to 
link. Just check the asm for this code:

```D
struct S { char[1000] x; }

void c()
{
     S s = S.init; // emits a `memcpy` (-O0) or `memset` (-O3)
     f(&s);
}

void f(S* s);
```

There might be an obscure compiler flag to disable this, but it's 
a lot easier to just include an implementation of `memcpy`, 
`memset` and `memcmp` yourself.


 Then my project no longer compile, thanks a lot!


When you use a custom druntime, you can't expect stability when 
you upgrade the compiler but not your druntime. The real issue 
here is that there is no proper support for WebAssembly in 
upstream druntime.


 https://github.com/dlang/dmd/pull/14910


 Nobody care anymore.....


I fixed that for you: https://github.com/dlang/phobos/pull/8726
May 14 2023
next sibling parent reply ryuukk_ <ryuukk.dev gmail.com> writes:
On Sunday, 14 May 2023 at 18:51:38 UTC, Dennis wrote:

 On Sunday, 14 May 2023 at 16:02:00 UTC, ryuukk_ wrote:

 I wanted to target WASM, if your hooks call into libC, i can't 
 target WASM..


 Good news, there's implementations of libc for WASM


WASI is not WASM, please don't suggest that as a solution, it's 
not, this is frustrating to read what you suggest when i bring to 
you problems

I'll end up just sticking to C if nobody understand
May 14 2023
next sibling parent reply Dennis <dkorpel gmail.com> writes:
On Sunday, 14 May 2023 at 18:59:52 UTC, ryuukk_ wrote:

 WASI is not WASM, please don't suggest that as a solution, it's 
 not, this is frustrating to read what you suggest when i bring 
 to you problems


I've literally had the same problem as you (missing references to 
libc symbols when targeting WebAssembly), and solved it by 
compiling and linking wasi-libc. I can't help you when you reject 
solutions simply by calling them 'not a solution' and no further 
details.


 I'll end up just sticking to C if nobody understand


How would you do array copies when using C targeting WebAssembly?
May 14 2023
parent reply ryuukk_ <ryuukk.dev gmail.com> writes:
On Sunday, 14 May 2023 at 19:13:50 UTC, Dennis wrote:

 On Sunday, 14 May 2023 at 18:59:52 UTC, ryuukk_ wrote:

 WASI is not WASM, please don't suggest that as a solution, 
 it's not, this is frustrating to read what you suggest when i 
 bring to you problems


 I've literally had the same problem as you (missing references 
 to libc symbols when targeting WebAssembly), and solved it by 
 compiling and linking wasi-libc. I can't help you when you 
 reject solutions simply by calling them 'not a solution' and no 
 further details.


 I'll end up just sticking to C if nobody understand


 How would you do array copies when using C targeting 
 WebAssembly?


Do you understand what you are saying?

"Can you target WASM? no, you need WASI"

When the solution i suggested was as simple as to make that 
tempalted hook not call libc, and instead just do the memcpy 
yourself

Now you suggest me to depend on WASI by default

Unbelievable

It's getting hard to maintain composure
May 14 2023
parent Dennis <dkorpel gmail.com> writes:
On Sunday, 14 May 2023 at 19:38:38 UTC, ryuukk_ wrote:

 Now you suggest me to depend on WASI by default

 Unbelievable

 It's getting hard to maintain composure


Then I'll leave it here. There's apparently a big problem with 
the solutions I provided that I'm supposed to know, but I don't, 
and that frustrates you. I would try to inquire more, but I'm 
afraid you will only get more upset, so I won't.
May 14 2023
prev sibling parent reply Dany12L <dany12719l gmail.com> writes:
On Sunday, 14 May 2023 at 18:59:52 UTC, ryuukk_ wrote:

 I'll end up just sticking to C if nobody understand


Fine but what do you solve? If you need memcpy you will in any 
case have to implement it yourself or use some already made 
library.
The same thing happens on D.
May 14 2023
parent reply ryuukk_ <ryuukk.dev gmail.com> writes:
On Sunday, 14 May 2023 at 21:47:51 UTC, Dany12L wrote:

 On Sunday, 14 May 2023 at 18:59:52 UTC, ryuukk_ wrote:

 I'll end up just sticking to C if nobody understand


 Fine but what do you solve? If you need memcpy you will in any 
 case have to implement it yourself or use some already made 
 library.
 The same thing happens on D.


Please read the linked issue, all it does is it makes it painful 
for me to debug, when the solution i provide is both pragmatic 
and simple

Not only my solution works for WASM but for everytime one needs 
to port D to an esoteric platform, the WASI solution only work 
for people who use WASM AND WASI

https://github.com/ldc-developers/ldc/issues/4324
May 15 2023
parent reply Dany12L <dany12719l gmail.com> writes:
On Monday, 15 May 2023 at 14:26:52 UTC, ryuukk_ wrote:

 On Sunday, 14 May 2023 at 21:47:51 UTC, Dany12L wrote:

 On Sunday, 14 May 2023 at 18:59:52 UTC, ryuukk_ wrote:

 I'll end up just sticking to C if nobody understand


 Fine but what do you solve? If you need memcpy you will in any 
 case have to implement it yourself or use some already made 
 library.
 The same thing happens on D.


 Please read the linked issue, all it does is it makes it 
 painful for me to debug, when the solution i provide is both 
 pragmatic and simple

 Not only my solution works for WASM but for everytime one needs 
 to port D to an esoteric platform, the WASI solution only work 
 for people who use WASM AND WASI

 https://github.com/ldc-developers/ldc/issues/4324


I've seen but this has been fixed so I don't understand what is 
your problem
May 15 2023
parent reply ryuukk_ <ryuukk.dev gmail.com> writes:
On Monday, 15 May 2023 at 17:44:20 UTC, Dany12L wrote:

 On Monday, 15 May 2023 at 14:26:52 UTC, ryuukk_ wrote:

 On Sunday, 14 May 2023 at 21:47:51 UTC, Dany12L wrote:

 On Sunday, 14 May 2023 at 18:59:52 UTC, ryuukk_ wrote:

 I'll end up just sticking to C if nobody understand


 Fine but what do you solve? If you need memcpy you will in 
 any case have to implement it yourself or use some already 
 made library.
 The same thing happens on D.


 Please read the linked issue, all it does is it makes it 
 painful for me to debug, when the solution i provide is both 
 pragmatic and simple

 Not only my solution works for WASM but for everytime one 
 needs to port D to an esoteric platform, the WASI solution 
 only work for people who use WASM AND WASI

 https://github.com/ldc-developers/ldc/issues/4324


 I've seen but this has been fixed so I don't understand what is 
 your problem


If whoever decided to template that function overnight, without 
notes on the changelog btw, not call into libc for just memcpy:

- i wouldn't get random segfault
- i wouldn't waste days trying to figure out why my project 
wouldn't compile anymore overnight


It's not that "you don't understand", it's you CHOOSE to not 
understand

The issue is fixed, great, at what cost? that you chose to ignore

And the problem i have is, according to this Monthly Meeting, the 
issue is about to be repeated in the future, who cares about 
having a portable runtime? who cares about the users?!

The goal is still pay as you go? or am i the only one who care 
about that?
May 15 2023
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Monday, 15 May 2023 at 18:02:49 UTC, ryuukk_ wrote:

 [snip]


It can be frustrating when you are are neck deep in some 
complicated problem to explain to people who haven't spent the 
same amount of time with it as you have.
May 15 2023
parent bachmeier <no spam.net> writes:
On Monday, 15 May 2023 at 18:15:54 UTC, jmh530 wrote:

 On Monday, 15 May 2023 at 18:02:49 UTC, ryuukk_ wrote:

 [snip]


 It can be frustrating when you are are neck deep in some 
 complicated problem to explain to people who haven't spent the 
 same amount of time with it as you have.


That poster turns conversation after conversation into the same 
toxic cesspool. If human interaction is that frustrating, they 
should refrain from posting.
May 15 2023
prev sibling parent ryuukk_ <ryuukk.dev gmail.com> writes:
On Sunday, 14 May 2023 at 18:51:38 UTC, Dennis wrote:

 Then my project no longer compile, thanks a lot!


 When you use a custom druntime, you can't expect stability when 
 you upgrade the compiler but not your druntime. The real issue 
 here is that there is no proper support for WebAssembly in 
 upstream druntime.


I don't expect "stability", i expect things to not require things 
it shouldn't require

depending on libc for memcpy, sorry for the word, but it's 
stupid, if you say that's stability, then i should perhaps move 
away
May 14 2023
prev sibling parent reply max haughton <maxhaton gmail.com> writes:
On Sunday, 14 May 2023 at 12:47:59 UTC, Mike Parker wrote:


 __CI failures__

 Dennis started by saying that the CI was randomly failing 
 again. He didn't have a Mac, so he'd been unable to chase down 
 the problem. Random CI failures are a recurring problem. There 
 are so many checks, and he doesn't know who created them or who 
 knows exactly what the checks are doing. He wishes the tests 
 had someone responsible for them who he could turn to when they 
 fail.

 Walter asked who had previously been in charge of the tests. 
 Razvan said he didn't recall if one person was ever in charge 
 of them. At some point, someone decided it was a good idea to 
 have a particular test and it got added to the pipeline.

 Dennis asked if we should only keep tests that have a 
 maintainer. Martin and Mathias quickly rejected that. Martin 
 said the tests are good. CI failures are usually caused by CI 
 image bumps or a PR. CI image changes are a PITA for LDC's 
 tests, and PR-related failures may not be easy to resolve, but 
 failures are hardly ever the fault of a test. And there's never 
 been any specific person responsible for any of DMD's CI 
 systems. They just grew organically. Then someone who knew the 
 details moved on and no one else knows them... it's a constant 
 maintenance burden, but it's worth the effort.

 There was a bit more discussion about the maintenance burden, 
 after which I noted that this is the story of our ecosystem. 
 We're responsible now for things none of us set up, and we need 
 to get a handle on it all. Dennis agreed and added that the CI 
 is in a special position. When one of them is outdated, it 
 doesn't just sit there out of the way, it becomes an annoyance 
 to development.

 (NOTE: This is one of the many aspects of our ecosystem that 
 we'll be working to improve [under our new 
 workflow](https://forum.dlang.org/thread/avvmlvjmvdniwwxemcqu forum.dlang.org).)


Some thoughts on testing:

1. This (MacOS) failure has been fixed (by me). It apparently 
also occurred with some other LibCs out there prior to that too. 
In future these kinds of failures must be prioritized a little 
more aggressively, this didn't just mean "Oh well, we'll ignore 
that pipeline for a while", it meant that Phobos effectively 
didn't work on MacOS (Oops).

2. At a bigger scale: We probably have too many CI pipelines. The 
main ones that I have
in mind that really could go are the OMF pipelines --- In OMF we 
have some ancient baggage which we don't need and shouldn't want 
to support anymore: [Microsoft barely mention OMF 
anymore](https://learn.microsoft.com/en-us/search/?terms=OMF&scope=C%2B%2B),
its not the default from dmd on 32bit windows anymore, and having it in the
testsuite ties us to the
Digital Mars ecosystem for likely zero benefit (Would you, 
reader, use Digital Mars if you were building C code on Windows 
today?)

3. The testing process could also use some love in terms of 
exactly how they're setup. Does everything that should/could use 
the host compiler use that compiler? Although I think its partly 
his own doing in not exerting much control over the compiler 
codebase other than when others try to organize it, Walter is 
right that the test suite should ideally be segmented into tests 
ordered by some measure of the number of features they depend 
upon.

4. We should have either digger or something like digger (likely 
a shell script, shooting from the hip I think digger is a very 
good idea but too complicated, myself and others have all had it 
not work in mysterious ways) being checked on every PR to make 
sure its easy to reproduce.

5. Automatic bisect? When github issues are done this could be an 
interesting use of richer integration with the concept of an 
issue to make developers productive. When a bug report is filed 
it, finding the commit that caused the issue can and should be 
done by a bot.
May 14 2023
next sibling parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 15/05/2023 11:53 AM, max haughton wrote:

 2. At a bigger scale: We probably have too many CI pipelines. The main 
 ones that I have
 in mind that really could go are the OMF pipelines --- In OMF we have 
 some ancient baggage which we don't need and shouldn't want to support 
 anymore: [Microsoft barely mention OMF 
 anymore](https://learn.microsoft.com/en-us/search/?terms=OMF&scope=C%2B%2B),
its not the default from dmd on 32bit windows anymore, and having it in the
testsuite ties us to the
 Digital Mars ecosystem for likely zero benefit (Would you, reader, use 
 Digital Mars if you were building C code on Windows today?)


This bit from Walter says it all I think:


 Walter said he had given up on maintaining DMC. The DMD backend had 


already diverged, and he saw no further point in maintaining DMC.

Time for OMF stuff to become optional in CI.
May 14 2023
prev sibling parent Dany12L <dany12719l gmail.com> writes:
On Sunday, 14 May 2023 at 23:53:41 UTC, max haughton wrote:

 On Sunday, 14 May 2023 at 12:47:59 UTC, Mike Parker wrote:


[...]


 Some thoughts on testing:

 1. This (MacOS) failure has been fixed (by me). It apparently 
 also occurred with some other LibCs out there prior to that 
 too. In future these kinds of failures must be prioritized a 
 little more aggressively, this didn't just mean "Oh well, we'll 
 ignore that pipeline for a while", it meant that Phobos 
 effectively didn't work on MacOS (Oops).

 [...]


Please yes OMF and legacy Digital Mars tools should burn in hell.
Every time one uses DMD on Windows it's a confusion mess of these 
old things that should never be used
May 15 2023