www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - gc attribute for bypassign nogc

reply bitwise <bitwise.pvt gmail.com> writes:
Any thoughts on an  gc attribute for bypassing  nogc?


ignore my level of productivity, as it compares to projects I've 
done in C++.

If I had to write a butter-smooth AAA game, it may be a problem, 
but at present, my thoughts on a few small jitters here and there 
are....who cares?

As long as GC usage is kept to a minimum, things will usually 
work ok fine.

For example, in a game, I may create a script with an "update" 
method that gets called each frame. I wouldn't want people(or 
myself) accidentally allocating memory here, possible 30-60 times 
per second, so I would  make that function  nogc. But, there 
*will* be cases where I actually want to allocate something, for 
example, to spawn a piece of cake.

Example:

class Cake{}

class MyScript {
     SysTime when;
     this() { when = Clock.currTime + 5.seconds; }
     void update()  nogc {
         // ...
         if(Clock.currTime >= when) {
              gc {
                 Cake c = new Cake();
             }
         }
     }
}

void main(string[] args) {
     MyScript script = new MyScript();

     foreach(i; 0..10) {
         script.update();
         Thread.sleep(1.seconds);
     }
}

There is the following, which is clever. But if it came down to 
having to do this to bypass  nogc, I simply wouldn't use  nogc.

https://p0nce.github.io/d-idioms/#Bypassing- nogc

When you have to do it thousands of times throughout your 
codebase, then yes, it's that bad.

     Bit
Jul 24 2016
next sibling parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Sunday, 24 July 2016 at 22:13:02 UTC, bitwise wrote:

 There is the following, which is clever. But if it came down to 
 having to do this to bypass  nogc, I simply wouldn't use  nogc.


that's what i ended up with: not using  nogc. it is pure 
parketing thing. for real world using it adds more troubles than 
it ever solved (and did it solve at least something?!).

i'd say: "don't bother with  nogc, it is mostly useless."
Jul 24 2016
next sibling parent ketmar <ketmar ketmar.no-ip.org> writes:
pure *marketing* thing, lol.
Jul 24 2016
prev sibling parent reply bitwise <bitwise.pvt gmail.com> writes:
On Sunday, 24 July 2016 at 22:21:51 UTC, ketmar wrote:

 On Sunday, 24 July 2016 at 22:13:02 UTC, bitwise wrote:

 There is the following, which is clever. But if it came down 
 to having to do this to bypass  nogc, I simply wouldn't use 
  nogc.


 that's what i ended up with: not using  nogc. it is pure 
 parketing thing. for real world using it adds more troubles 
 than it ever solved (and did it solve at least something?!).

 i'd say: "don't bother with  nogc, it is mostly useless."


I don't understand what you mean, saying it's useless.

I intend to do this:

class Script {
      nogc void update(){}
}

class CustomScript : Script {
     override void update(){}
}

If I did this, it would act as a safety net, and prevent a lot of 
accidental allocations. Keep in mind, "update" is meant to run 
30-60 times per second, so allocations would add up extremely 
fast.

     Bit
Jul 24 2016
parent ketmar <ketmar ketmar.no-ip.org> writes:
On Sunday, 24 July 2016 at 23:14:37 UTC, bitwise wrote:

 I don't understand what you mean, saying it's useless.


exactly that. in the end i can't mark any code  nogc except very 
trivial, where it doesn't matter at all (primitive getters and 
setters). in other code i ocasionally need to allocate or throw. 
so you have to ditch  nogc anyway.

and yes, i'm talking about engines that should maintain solid 60 
FPS.

and for "safety nets" i prefer to use "-vgc" after i done some 
amount of work to see if i accidentally missed something.
Jul 24 2016
prev sibling next sibling parent Lodovico Giaretta <lodovico giaretart.net> writes:
On Sunday, 24 July 2016 at 22:13:02 UTC, bitwise wrote:

 Any thoughts on an  gc attribute for bypassing  nogc?


 can't ignore my level of productivity, as it compares to 
 projects I've done in C++.

 If I had to write a butter-smooth AAA game, it may be a 
 problem, but at present, my thoughts on a few small jitters 
 here and there are....who cares?

 As long as GC usage is kept to a minimum, things will usually 
 work ok fine.

 For example, in a game, I may create a script with an "update" 
 method that gets called each frame. I wouldn't want people(or 
 myself) accidentally allocating memory here, possible 30-60 
 times per second, so I would  make that function  nogc. But, 
 there *will* be cases where I actually want to allocate 
 something, for example, to spawn a piece of cake.

 Example:

 class Cake{}

 class MyScript {
     SysTime when;
     this() { when = Clock.currTime + 5.seconds; }
     void update()  nogc {
         // ...
         if(Clock.currTime >= when) {
              gc {
                 Cake c = new Cake();
             }
         }
     }
 }

 void main(string[] args) {
     MyScript script = new MyScript();

     foreach(i; 0..10) {
         script.update();
         Thread.sleep(1.seconds);
     }
 }

 There is the following, which is clever. But if it came down to 
 having to do this to bypass  nogc, I simply wouldn't use  nogc.

 https://p0nce.github.io/d-idioms/#Bypassing- nogc

 When you have to do it thousands of times throughout your 
 codebase, then yes, it's that bad.

     Bit


In this kind of situation, the solution is to not use  nogc and 
use the built-in gc profiler to decide which functions need 
memory usage optimizations.
We should be very careful, that using  nogc is not the aim. It is 
just one of the available tools to avoid our application randomly 
freezing due to collections. That is the aim. And there are other 
tools to achieve it: the allocators library and the built-in gc 
profiler.
Jul 24 2016
prev sibling next sibling parent reply Dicebot <public dicebot.lv> writes:
 nogc functions must never ever trigger GC collection cycle, not 
under any possible circumstances. Otherwise the attribute serves 
no purpose.
Jul 24 2016
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Sunday, 24 July 2016 at 23:10:18 UTC, Dicebot wrote:

  nogc functions must never ever trigger GC collection cycle, 
 not under any possible circumstances. Otherwise the attribute 
 serves no purpose.


The fact that you can use malloc() in a  nogc function 
invalidates this argument. There are many ways for someone 
publishing a piece of code or DUB package to be irresponsible 
about memory management. Adding this would not make it any more 
likely. Adding  gc to a piece of code to bypass  nogc is not 
something that could be done by accident.

Imagine though, if there was a DUB package that provided some 
base class that had its virtual methods annotated with  nogc. If 
I downloaded the package, and decided that using GC was perfectly 
fine in my application, I would be screwed if I wanted to inherit 
one of the classes from that package.

     Bit
Jul 24 2016
next sibling parent Dicebot <public dicebot.lv> writes:
On Sunday, 24 July 2016 at 23:30:37 UTC, bitwise wrote:

 On Sunday, 24 July 2016 at 23:10:18 UTC, Dicebot wrote:

  nogc functions must never ever trigger GC collection cycle, 
 not under any possible circumstances. Otherwise the attribute 
 serves no purpose.


 The fact that you can use malloc() in a  nogc function 
 invalidates this argument.


malloc is not even on the same order of magnitude of impact as 
GC.  nogc is primarily about latency and preventing unwanted stop 
the world cycle, not about performance or allocation count per se.
Jul 24 2016
prev sibling parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Sunday, 24 July 2016 at 23:30:37 UTC, bitwise wrote:

 On Sunday, 24 July 2016 at 23:10:18 UTC, Dicebot wrote:

  nogc functions must never ever trigger GC collection cycle, 
 not under any possible circumstances. Otherwise the attribute 
 serves no purpose.


 The fact that you can use malloc() in a  nogc function 
 invalidates this argument.


nope. `malloc()` has nothing to do with garbage collection.

what ` nogc` means is "i won't use built-in allocator with 
garbage collection in this code", not "i won't do any memory 
allocations in this code".
Jul 24 2016
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Monday, 25 July 2016 at 00:05:16 UTC, ketmar wrote:

 On Sunday, 24 July 2016 at 23:30:37 UTC, bitwise wrote:

 On Sunday, 24 July 2016 at 23:10:18 UTC, Dicebot wrote:

  nogc functions must never ever trigger GC collection cycle, 
 not under any possible circumstances. Otherwise the attribute 
 serves no purpose.


 The fact that you can use malloc() in a  nogc function 
 invalidates this argument.


 nope. `malloc()` has nothing to do with garbage collection.

 what ` nogc` means is "i won't use built-in allocator with 
 garbage collection in this code", not "i won't do any memory 
 allocations in this code".


If someone knows all their code is  nogc, they can disable the 
GC. If they do that, and then someone GC allocates anyways, it's 
a leak. Based on this, an argument could be made that  nogc 
should *never* be bypassed. But that argument is made on the 
false premise that it's fool-proof in it's current condition. 
It's not, because you can already use malloc() in a  nogc 
function and leak that memory instead.

      Bit
Jul 24 2016
parent ketmar <ketmar ketmar.no-ip.org> writes:
On Monday, 25 July 2016 at 00:26:05 UTC, bitwise wrote:

 If someone knows all their code is  nogc, they can disable the 
 GC. If they do that, and then someone GC allocates anyways, 
 it's a leak. Based on this, an argument could be made that 
  nogc should *never* be bypassed. But that argument is made on 
 the false premise that it's fool-proof in it's current 
 condition. It's not, because you can already use malloc() in a 
  nogc function and leak that memory instead.


` nogc` should not be bypassed due to completely different 
reasons. while compiler is not doing any optimizations or 
generating any GC helpers for gc code now, it *can* do that in 
the future, so cheating the compiler is bad idea.

otherwise, ` nogc` doesn't really guarantee absence of memory 
leaks, and it wasn't designed to do so.
Jul 24 2016
prev sibling next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 25/07/2016 10:13 AM, bitwise wrote:

 Any thoughts on an  gc attribute for bypassing  nogc?


 my level of productivity, as it compares to projects I've done in C++.

 If I had to write a butter-smooth AAA game, it may be a problem, but at
 present, my thoughts on a few small jitters here and there are....who
 cares?

 As long as GC usage is kept to a minimum, things will usually work ok fine.

 For example, in a game, I may create a script with an "update" method
 that gets called each frame. I wouldn't want people(or myself)
 accidentally allocating memory here, possible 30-60 times per second, so
 I would  make that function  nogc. But, there *will* be cases where I
 actually want to allocate something, for example, to spawn a piece of cake.

 Example:

 class Cake{}

 class MyScript {
     SysTime when;
     this() { when = Clock.currTime + 5.seconds; }
     void update()  nogc {
         // ...
         if(Clock.currTime >= when) {
              gc {
                 Cake c = new Cake();
             }
         }
     }
 }

 void main(string[] args) {
     MyScript script = new MyScript();

     foreach(i; 0..10) {
         script.update();
         Thread.sleep(1.seconds);
     }
 }

 There is the following, which is clever. But if it came down to having
 to do this to bypass  nogc, I simply wouldn't use  nogc.

 https://p0nce.github.io/d-idioms/#Bypassing- nogc

 When you have to do it thousands of times throughout your codebase, then
 yes, it's that bad.

     Bit


I've been saying for a very long time we need  assumenogc attribute like 
we have  trusted for  safe.
I have not been taken very seriously about this...

Although I see no reason to allow GC in  assumenogc as long as it does 
not trigger scan/collect phase of it. After all, why don't we have a 
hint on allocation if we do or do not want it to trigger a scan?

Or even better, a thread local stack to specify this!
Jul 24 2016
parent reply Jonathan Marler <johnnymarler gmail.com> writes:
On Monday, 25 July 2016 at 03:18:17 UTC, rikki cattermole wrote:

 On 25/07/2016 10:13 AM, bitwise wrote:

[...]


 I've been saying for a very long time we need  assumenogc 
 attribute like we have  trusted for  safe.


I havent seen this assumegc propsal before, could you provide the 
definition you had in mind for it?


 I have not been taken very seriously about this...

 Although I see no reason to allow GC in  assumenogc as long as 
 it does not trigger scan/collect phase of it. After all, why 
 don't we have a hint on allocation if we do or do not want it 
 to trigger a scan?


If the GC supported this mode (NoCollectionMode), what should the 
GC do when it is in this mode and an allocation is requested that 
can't be done until a collection is done?
Jul 24 2016
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 25/07/2016 5:16 PM, Jonathan Marler wrote:

 On Monday, 25 July 2016 at 03:18:17 UTC, rikki cattermole wrote:

 On 25/07/2016 10:13 AM, bitwise wrote:

 [...]


 I've been saying for a very long time we need  assumenogc attribute
 like we have  trusted for  safe.


 I havent seen this assumegc propsal before, could you provide the
 definition you had in mind for it?


I've mostly kept it off hand for other regulars as I'm not really the 
best person to run with it and make it happen.


 I have not been taken very seriously about this...

 Although I see no reason to allow GC in  assumenogc as long as it does
 not trigger scan/collect phase of it. After all, why don't we have a
 hint on allocation if we do or do not want it to trigger a scan?


 If the GC supported this mode (NoCollectionMode), what should the GC do
 when it is in this mode and an allocation is requested that can't be
 done until a collection is done?


Fail.
On Linux this should almost never happen and Windows should in theory be 
the same way.
Jul 25 2016
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Monday, 25 July 2016 at 07:43:34 UTC, rikki cattermole wrote:

 I've been saying for a very long time we need  assumenogc 
 attribute like we have
  trusted for  safe.


So I'm not crazy then ;)

I'm wondering if Andrei and Walter consider  trusted a win 
though. They seem to have such firm stances on certain issues 
that it makes me wonder if they consider things like  trusted a 
liability.

Personally, I lean way to the side of flexibility, and believe a 
good language shouldn't force you to code a certain way(within 
reason). I feel like this type of feature is very reasonable.

     Bit
Jul 25 2016
parent reply Chris Wright <dhasenan gmail.com> writes:
On Mon, 25 Jul 2016 15:46:54 +0000, bitwise wrote:


 On Monday, 25 July 2016 at 07:43:34 UTC, rikki cattermole wrote:

 I've been saying for a very long time we need  assumenogc attribute
 like we have  trusted for  safe.


 
 So I'm not crazy then ;)
 
 I'm wondering if Andrei and Walter consider  trusted a win though. They
 seem to have such firm stances on certain issues that it makes me wonder
 if they consider things like  trusted a liability.
 
 Personally, I lean way to the side of flexibility, and believe a good
 language shouldn't force you to code a certain way(within reason). I
 feel like this type of feature is very reasonable.
 
      Bit


 nogc conflates "doesn't allocate memory from the GC heap because I don't 
want my application to use the GC at all" with "doesn't cause GC 
collection pauses".

The latter can have a  assumenogc annotation that works -- you call 
GC.disable and GC.enable as appropriate. The former can't.
Jul 25 2016
next sibling parent reply bitwise <bitwise.pvt gmail.com> writes:
On Tuesday, 26 July 2016 at 01:07:27 UTC, Chris Wright wrote:

 On Mon, 25 Jul 2016 15:46:54 +0000, bitwise wrote:


 On Monday, 25 July 2016 at 07:43:34 UTC, rikki cattermole 
 wrote:

 I've been saying for a very long time we need  assumenogc 
 attribute like we have  trusted for  safe.


 
 So I'm not crazy then ;)
 
 I'm wondering if Andrei and Walter consider  trusted a win 
 though. They seem to have such firm stances on certain issues 
 that it makes me wonder if they consider things like  trusted 
 a liability.
 
 Personally, I lean way to the side of flexibility, and believe 
 a good language shouldn't force you to code a certain 
 way(within reason). I feel like this type of feature is very 
 reasonable.
 
      Bit


  nogc conflates "doesn't allocate memory from the GC heap 
 because I don't want my application to use the GC at all" with 
 "doesn't cause GC collection pauses".

 The latter can have a  assumenogc annotation that works -- you 
 call GC.disable and GC.enable as appropriate. The former can't.


Fair point.

You got me thinking though. For my intended usage, I don't 
actually need errors for GC allocation, and that may actually be 
inappropriate. I just need to make a best effort to avoid 
allocations.

Something like  warngc could work nicely. It could function 
exactly as  nogc with the exception that it issued warnings 
instead of errors.

I feel like some people reading this would be very quick to scold 
me for "premature optimization", and recommend a profiler, but in 
the case of 30-60 fps games, it's practically guaranteed that you 
will eventually run into performance problems. Especially on 
mobile devices. So proactively writing good code can save you a 
lot of headache.

     Bit
Jul 25 2016
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Tuesday, 26 July 2016 at 02:42:52 UTC, bitwise wrote:

 Something like  warngc could work nicely. It could function 
 exactly as  nogc with the exception that it issued warnings 
 instead of errors.


i don't think that it will be added, though. but you can emulate 
it with external utility: either with dscanner (writing your own 
linter), or with "-vgc" and then using simple regexp to find in 
which function the reported line is in. somewhat messy, but it is 
the way to get the work done without waiting for new attr. ;-)
Jul 25 2016
next sibling parent ketmar <ketmar ketmar.no-ip.org> writes:
p.s. probably even with -vgc to list line numbers and dscanner to 
find function names/signatures. "-vgc" has the advantage of 
reporting closuses, while with "handmade" linter you'll have to 
track that by yourself.
Jul 25 2016
prev sibling parent bitwise <bitwise.pvt gmail.com> writes:
On Tuesday, 26 July 2016 at 02:49:59 UTC, ketmar wrote:

 On Tuesday, 26 July 2016 at 02:42:52 UTC, bitwise wrote:

 Something like  warngc could work nicely. It could function 
 exactly as  nogc with the exception that it issued warnings 
 instead of errors.


 i don't think that it will be added, though. but you can 
 emulate it with external utility: either with dscanner (writing 
 your own linter), or with "-vgc" and then using simple regexp 
 to find in which function the reported line is in. somewhat 
 messy, but it is the way to get the work done without waiting 
 for new attr. ;-)


Actually, thinking about this a bit more, I think  warngc would 
be bad anyways. Code that was basically which worked exactly as 
is was designed to would issue errors, which would be bad.

Writing a custom linter is not an option. Not only because of the 
work/maintainence involved, but because it would complicate 
workflow. What I want is easily achieved during compilation.

My goal is to proactively deal with allocations that are in 
places that they shouldn't be 90% of the time, like per-frame 
updates in a real-time application or game.

     Bit
Jul 26 2016
prev sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 26 July 2016 at 01:07:27 UTC, Chris Wright wrote:

  nogc conflates "doesn't allocate memory from the GC heap 
 because I don't want my application to use the GC at all" with 
 "doesn't cause GC collection pauses".

 The latter can have a  assumenogc annotation that works -- you 
 call GC.disable and GC.enable as appropriate. The former can't.


For whatever reason, I like the idea of annotations behaving like 
enums. So instead of  gc or  assumenogc, you could have something 
like  nogc.false or  nogc.assume. You wouldn't add new 
annotations, just new behavior for existing annotations.
Jul 26 2016
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Tuesday, 26 July 2016 at 14:51:21 UTC, jmh530 wrote:

 On Tuesday, 26 July 2016 at 01:07:27 UTC, Chris Wright wrote:

  nogc conflates "doesn't allocate memory from the GC heap 
 because I don't want my application to use the GC at all" with 
 "doesn't cause GC collection pauses".

 The latter can have a  assumenogc annotation that works -- you 
 call GC.disable and GC.enable as appropriate. The former can't.


 For whatever reason, I like the idea of annotations behaving 
 like enums. So instead of  gc or  assumenogc, you could have 
 something like  nogc.false or  nogc.assume. You wouldn't add 
 new annotations, just new behavior for existing annotations.


Chris is right that  safe/ trusted is not analogous to 
 nogc/ assumenogc, but I still think  assumenogc is a good 
idea..or maybe  nogc(false) to offer the opposite behaviour of 
 nogc.

There are arguments that can be made against this idea, but IMO, 
it's really hard to screw this up unintentionally. It's not like 
it's an invisible implicit conversion.

There are plenty of valid use cases. One, like I mentioned, about 
strongly deterring allocations in certain contexts instead of 
completely disallowing them. Also, deferring garbage collection 
until the end of a performance sensitive algorithm that may still 
allocate, so that GC.collect can be called manually afterward.

    Bit.
Jul 26 2016
parent Danni Coy via Digitalmars-d <digitalmars-d puremagic.com> writes:
I am not convinced this is the best way to go although it would be a
relatively easy solution.

What I would personally prefer was if there was a language mechanism
to mark theAllocator, processAllocator (iAllocator) to be  nogc as
long as the allocator in question is not the GCAllocator.

I am thinking specifically of the situation where vendor A can create
a dynamic library in D that uses theAllocator and vendor B uses that
library without necessarily having the source code.


On Wed, Jul 27, 2016 at 7:05 AM, bitwise via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Tuesday, 26 July 2016 at 14:51:21 UTC, jmh530 wrote:

 On Tuesday, 26 July 2016 at 01:07:27 UTC, Chris Wright wrote:

  nogc conflates "doesn't allocate memory from the GC heap because I don't
 want my application to use the GC at all" with "doesn't cause GC collection
 pauses".

 The latter can have a  assumenogc annotation that works -- you call
 GC.disable and GC.enable as appropriate. The former can't.



 For whatever reason, I like the idea of annotations behaving like enums.
 So instead of  gc or  assumenogc, you could have something like  nogc.false
 or  nogc.assume. You wouldn't add new annotations, just new behavior for
 existing annotations.



 Chris is right that  safe/ trusted is not analogous to  nogc/ assumenogc,
 but I still think  assumenogc is a good idea..or maybe  nogc(false) to offer
 the opposite behaviour of  nogc.

 There are arguments that can be made against this idea, but IMO, it's really
 hard to screw this up unintentionally. It's not like it's an invisible
 implicit conversion.

 There are plenty of valid use cases. One, like I mentioned, about strongly
 deterring allocations in certain contexts instead of completely disallowing
 them. Also, deferring garbage collection until the end of a performance
 sensitive algorithm that may still allocate, so that GC.collect can be
 called manually afterward.

    Bit.
Jul 27 2016
prev sibling next sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Sunday, 24 July 2016 at 22:13:02 UTC, bitwise wrote:

 There is the following, which is clever. But if it came down to 
 having to do this to bypass  nogc, I simply wouldn't use  nogc.

 https://p0nce.github.io/d-idioms/#Bypassing- nogc

 When you have to do it thousands of times throughout your 
 codebase, then yes, it's that bad.


FWIW I've removed every use of that bypassing (was used for 
runtime initialization or semaphores locks).
You can also use emplace/destroy like in: 
https://github.com/d-gamedev-team/gfm/blob/master/core/gfm/core/memory.d#L238
Jul 27 2016
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Wednesday, 27 July 2016 at 20:42:01 UTC, Guillaume Piolat 
wrote:

 On Sunday, 24 July 2016 at 22:13:02 UTC, bitwise wrote:

 There is the following, which is clever. But if it came down 
 to having to do this to bypass  nogc, I simply wouldn't use 
  nogc.

 https://p0nce.github.io/d-idioms/#Bypassing- nogc

 When you have to do it thousands of times throughout your 
 codebase, then yes, it's that bad.


 FWIW I've removed every use of that bypassing (was used for 
 runtime initialization or semaphores locks).
 You can also use emplace/destroy like in: 
 https://github.com/d-gamedev-team/gfm/blob/master/core/gfm/core/memory.d#L238


The point is though, that I WANT to use the GC. I want the memory 
cleaned up for me, and I don't mind little pauses once in a 
while. I just don't want careless allocations to happen in 
certain performance-sensitive contexts, like per-frame updates.

While working on a past project(C++), I found this little gem:

void draw() {
     Font* f = new Font("arial.ttf", 16);
     drawText(f, "hello world");
}

As utterly moronic as this seems, this was a real bug that I had 
to fix. Our game was literally topping out at 2GB of memory usage 
after ~30 seconds and crashing.

Note: it wasn't my code ;)

If that code was written in D with the feature I'm asking for, 
draw() would have been marked with  nogc. The person who wrote 
the above code would have either had to store the font somewhere 
else, or insert a  assumenogc{}  section to actually do that. So 
it either would not have happened, or would have been much easier 
to find.

If you found that your game/app was using too much GC, searching 
for " assumenogc" would likely uncover the cause, as long as your 
root classes were annotated correctly. The  assumenogc annotation 
would plainly show where allocations were happening that maybe 
shouldn't be.

     Bit
     Bit
Jul 27 2016
next sibling parent reply Lodovico Giaretta <lodovico giaretart.net> writes:
On Thursday, 28 July 2016 at 00:23:57 UTC, bitwise wrote:

 The point is though, that I WANT to use the GC. I want the 
 memory cleaned up for me, and I don't mind little pauses once 
 in a while. I just don't want careless allocations to happen in 
 certain performance-sensitive contexts, like per-frame updates.

 While working on a past project(C++), I found this little gem:

 void draw() {
     Font* f = new Font("arial.ttf", 16);
     drawText(f, "hello world");
 }

 As utterly moronic as this seems, this was a real bug that I 
 had to fix. Our game was literally topping out at 2GB of memory 
 usage after ~30 seconds and crashing.

 Note: it wasn't my code ;)

 If that code was written in D with the feature I'm asking for, 
 draw() would have been marked with  nogc. The person who wrote 
 the above code would have either had to store the font 
 somewhere else, or insert a  assumenogc{}  section to actually 
 do that. So it either would not have happened, or would have 
 been much easier to find.

 If you found that your game/app was using too much GC, 
 searching for " assumenogc" would likely uncover the cause, as 
 long as your root classes were annotated correctly. The 
  assumenogc annotation would plainly show where allocations 
 were happening that maybe shouldn't be.


If  assumegc has to be manually checked to see if it is 
over-allocating whenever the application is going out of memory, 
then it is no more useful than running the gc-profiler when the 
application is over-allocating. The profiler is even better, 
because with  assumegc you have to check ALL marked functions to 
find the leaking one, while the gc-profiler would just put it on 
top of the report, making your job easier. So IMHO we already 
have an instrument to solve these problems.
Jul 28 2016
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Thursday, 28 July 2016 at 07:12:06 UTC, Lodovico Giaretta 
wrote:

 On Thursday, 28 July 2016 at 00:23:57 UTC, bitwise wrote:

 The point is though, that I WANT to use the GC. I want the 
 memory cleaned up for me, and I don't mind little pauses once 
 in a while. I just don't want careless allocations to happen 
 in certain performance-sensitive contexts, like per-frame 
 updates.

 While working on a past project(C++), I found this little gem:

 void draw() {
     Font* f = new Font("arial.ttf", 16);
     drawText(f, "hello world");
 }

 As utterly moronic as this seems, this was a real bug that I 
 had to fix. Our game was literally topping out at 2GB of 
 memory usage after ~30 seconds and crashing.

 Note: it wasn't my code ;)

 If that code was written in D with the feature I'm asking for, 
 draw() would have been marked with  nogc. The person who wrote 
 the above code would have either had to store the font 
 somewhere else, or insert a  assumenogc{}  section to actually 
 do that. So it either would not have happened, or would have 
 been much easier to find.

 If you found that your game/app was using too much GC, 
 searching for " assumenogc" would likely uncover the cause, as 
 long as your root classes were annotated correctly. The 
  assumenogc annotation would plainly show where allocations 
 were happening that maybe shouldn't be.


 If  assumegc has to be manually checked to see if it is 
 over-allocating whenever the application is going out of 
 memory, then it is no more useful than running the gc-profiler 
 when the application is over-allocating. The profiler is even 
 better, because with  assumegc you have to check ALL marked 
 functions to find the leaking one, while the gc-profiler would 
 just put it on top of the report, making your job easier. So 
 IMHO we already have an instrument to solve these problems.


It's not about running out of memory. It's a performance issue.

Example: In the following class, it would be perfectly fine, and 
even expected to allocate in start() but not in update(). This is 
because start() gets called once on object initialization, but 
update() would get called every frame.

class NPC {
     // ....
     void start() { this.hat = new Hat(); }
     void update()  nogc {
         this.timer += Clock.currTime;
         check(this.timer);
         updateUI();
     }
}

Finally, the use of a profiler, and  nogc/ assumenogc wouldn't be 
mutually exclusive. If a profiler had a filter-by-attribute 
functionality, you could set it to  nogc functions, and it would 
narrow down the results to the functions where allocations 
actually matter.

     Bit
Jul 28 2016
parent reply Guillaume Piolat <first.last gmail.com> writes:
On Thursday, 28 July 2016 at 15:24:10 UTC, bitwise wrote:

 It's not about running out of memory. It's a performance issue.

 Example: In the following class, it would be perfectly fine, 
 and even expected to allocate in start() but not in update(). 
 This is because start() gets called once on object 
 initialization, but update() would get called every frame.


Vladimir implemented counting in -profile=gc, it gives you the 
number of allocations and the amount allocated for each 
allocation point.
Jul 28 2016
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Thursday, 28 July 2016 at 16:15:04 UTC, Guillaume Piolat wrote:

 On Thursday, 28 July 2016 at 15:24:10 UTC, bitwise wrote:

 It's not about running out of memory. It's a performance issue.

 Example: In the following class, it would be perfectly fine, 
 and even expected to allocate in start() but not in update(). 
 This is because start() gets called once on object 
 initialization, but update() would get called every frame.


 Vladimir implemented counting in -profile=gc, it gives you the 
 number of allocations and the amount allocated for each 
 allocation point.


I'm not sure if this point is meant to be for or against what I'm 
suggesting, but still, I'm thinking in terms of *proactively* 
writing efficient code, not allowing an app get to the point 
where you need to run a profiler.

The situation is similar to using  safe. You *could* just write 
code however you want, wait until it crashes, then try to find 
the problem using Valgrind or something, but that's bad for 
obvious reasons.

     Bit
Jul 28 2016
next sibling parent reply default0 <Kevin.Labschek gmx.de> writes:
On Thursday, 28 July 2016 at 16:45:05 UTC, bitwise wrote:

 On Thursday, 28 July 2016 at 16:15:04 UTC, Guillaume Piolat 
 wrote:

 On Thursday, 28 July 2016 at 15:24:10 UTC, bitwise wrote:

 It's not about running out of memory. It's a performance 
 issue.

 Example: In the following class, it would be perfectly fine, 
 and even expected to allocate in start() but not in update(). 
 This is because start() gets called once on object 
 initialization, but update() would get called every frame.


 Vladimir implemented counting in -profile=gc, it gives you the 
 number of allocations and the amount allocated for each 
 allocation point.


 I'm not sure if this point is meant to be for or against what 
 I'm suggesting, but still, I'm thinking in terms of 
 *proactively* writing efficient code, not allowing an app get 
 to the point where you need to run a profiler.

 The situation is similar to using  safe. You *could* just write 
 code however you want, wait until it crashes, then try to find 
 the problem using Valgrind or something, but that's bad for 
 obvious reasons.

     Bit


Just my 2 cents on naming: what you want sounds a lot more like 
 warngc than  assumenogc or whatever was floating around before. 
What such an attribute should do and if its worth implementing is 
hard to figure out, though. Presumably you'd have your update() 
method marked  warngc but then how is the compiler supposed to 
know that an allocation like this:

class A {
     void update()  warngc {
         if(rareCondition()) {
             // something that allocates
         }
     }
}

is perfectly fine, but something like this

class B {
     void update()  warngc {
         if(almostAlwaysTrue()) {
             // something that allocates
         }
     }
}

is not?
That issue aside, what exactly do you envision the compiler to 
tell you (examples of error messages and code patterns it should 
detect, examples of code patterns that it should NOT detect and 
are fine) incase it automagically finds problematic code?

I'm having trouble putting a thumb on what you want following 
this thread, because what you are describing feels a bit vague to 
me.
Jul 28 2016
parent bitwise <bitwise.pvt gmail.com> writes:
On Thursday, 28 July 2016 at 16:58:14 UTC, default0 wrote:

 On Thursday, 28 July 2016 at 16:45:05 UTC, bitwise wrote:

 On Thursday, 28 July 2016 at 16:15:04 UTC, Guillaume Piolat 
 wrote:

 On Thursday, 28 July 2016 at 15:24:10 UTC, bitwise wrote:

 It's not about running out of memory. It's a performance 
 issue.

 Example: In the following class, it would be perfectly fine, 
 and even expected to allocate in start() but not in 
 update(). This is because start() gets called once on object 
 initialization, but update() would get called every frame.


 Vladimir implemented counting in -profile=gc, it gives you 
 the number of allocations and the amount allocated for each 
 allocation point.


 I'm not sure if this point is meant to be for or against what 
 I'm suggesting, but still, I'm thinking in terms of 
 *proactively* writing efficient code, not allowing an app get 
 to the point where you need to run a profiler.

 The situation is similar to using  safe. You *could* just 
 write code however you want, wait until it crashes, then try 
 to find the problem using Valgrind or something, but that's 
 bad for obvious reasons.

     Bit


 Just my 2 cents on naming: what you want sounds a lot more like 
  warngc than  assumenogc or whatever was floating around 
 before. What such an attribute should do and if its worth 
 implementing is hard to figure out, though. Presumably you'd 
 have your update() method marked  warngc but then how is the 
 compiler supposed to know that an allocation like this:

 class A {
     void update()  warngc {
         if(rareCondition()) {
             // something that allocates
         }
     }
 }

 is perfectly fine, but something like this

 class B {
     void update()  warngc {
         if(almostAlwaysTrue()) {
             // something that allocates
         }
     }
 }

 is not?
 That issue aside, what exactly do you envision the compiler to 
 tell you (examples of error messages and code patterns it 
 should detect, examples of code patterns that it should NOT 
 detect and are fine) incase it automagically finds problematic 
 code?

 I'm having trouble putting a thumb on what you want following 
 this thread, because what you are describing feels a bit vague 
 to me.


I think you're complicating the issue quite a bit here.

It's very simple:

 nogc { /* in this scope, compiler disallows GC allocations. */ }

 nogc {
      assumenogc { /* here, GC allocations are allowed */  }
}

 assumenogc would simply reverse the effects of the  nogc flag on 
a given scope.

So  nogc could be applied to performance critical scopes, as it 
is now. The exception would be, that if I as a programmer knew 
that I was only making a small one-time allocation, and that the 
GC was still enabled, I could do so by surrounding the allocation 
with  assumenogc.

     Bit
Jul 28 2016
prev sibling parent Lodovico Giaretta <lodovico giaretart.net> writes:
On Thursday, 28 July 2016 at 16:45:05 UTC, bitwise wrote:

 On Thursday, 28 July 2016 at 16:15:04 UTC, Guillaume Piolat 
 wrote:

 On Thursday, 28 July 2016 at 15:24:10 UTC, bitwise wrote:

 It's not about running out of memory. It's a performance 
 issue.

 Example: In the following class, it would be perfectly fine, 
 and even expected to allocate in start() but not in update(). 
 This is because start() gets called once on object 
 initialization, but update() would get called every frame.


 Vladimir implemented counting in -profile=gc, it gives you the 
 number of allocations and the amount allocated for each 
 allocation point.


 I'm not sure if this point is meant to be for or against what 
 I'm suggesting, but still, I'm thinking in terms of 
 *proactively* writing efficient code, not allowing an app get 
 to the point where you need to run a profiler.


Well, here in my opinion there's the wrong attitude of profiling 
only when it becomes necessary (i.e. the app is slow or uses too 
much ram). This is wrong. You should profile before this happens; 
you should profile to see if the time/memory is spent in the 
parts of the application that should spend it and you should 
profile whenever you care about making good code and not just 
"code that works". You should profile while you add features, to 
see their impact on the overall performance. Profiling at the 
end, when you have a 100K LOC codebase deployed, because a user 
ran out of memory is too late: too late to allow to easily spot 
the problem and too late to think an alternative 
approach/algorithm/design, instead of just patching the problem.
Jul 28 2016
prev sibling parent reply Meta <jared771 gmail.com> writes:
On Thursday, 28 July 2016 at 00:23:57 UTC, bitwise wrote:

 While working on a past project(C++), I found this little gem:

 void draw() {
     Font* f = new Font("arial.ttf", 16);
     drawText(f, "hello world");
 }


It sounds like -vgc and --profile=gc are exactly what you want. 
 nogc is meant to *guarantee* that the GC will not be called 
within a function or any other functions called by that function. 
Taking that guarantee away makes it effectively useless.
Jul 28 2016
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Thursday, 28 July 2016 at 18:53:35 UTC, Meta wrote:

 On Thursday, 28 July 2016 at 00:23:57 UTC, bitwise wrote:

 While working on a past project(C++), I found this little gem:

 void draw() {
     Font* f = new Font("arial.ttf", 16);
     drawText(f, "hello world");
 }


 It sounds like -vgc and --profile=gc are exactly what you want. 
  nogc is meant to *guarantee* that the GC will not be called 
 within a function or any other functions called by that 
 function. Taking that guarantee away makes it effectively 
 useless.


I understand that  assumenogc may not be exactly the right 
approach, but the underlying idea is sound, which is to have the 
compiler to disallow all GC allocations in some annotated scope, 
just like  nogc, but to allow the programmer to override the 
 nogc restriction in certain cases.

There may be alternative approaches, and the  warngc idea was 
just one random suggestion.

The point is, a mixture of  nogc and  assumenogc would achieve 
exactly what I'm looking for. The only problem is that it may 
cause issues for people that are using  nogc under a different 
set of assumptions.

     Bit
Jul 28 2016
parent reply Lodovico Giaretta <lodovico giaretart.net> writes:
On Thursday, 28 July 2016 at 20:32:18 UTC, bitwise wrote:

 On Thursday, 28 July 2016 at 18:53:35 UTC, Meta wrote:

 On Thursday, 28 July 2016 at 00:23:57 UTC, bitwise wrote:

 While working on a past project(C++), I found this little gem:

 void draw() {
     Font* f = new Font("arial.ttf", 16);
     drawText(f, "hello world");
 }


 It sounds like -vgc and --profile=gc are exactly what you 
 want.  nogc is meant to *guarantee* that the GC will not be 
 called within a function or any other functions called by that 
 function. Taking that guarantee away makes it effectively 
 useless.


 I understand that  assumenogc may not be exactly the right 
 approach, but the underlying idea is sound, which is to have 
 the compiler to disallow all GC allocations in some annotated 
 scope, just like  nogc, but to allow the programmer to override 
 the  nogc restriction in certain cases.

 There may be alternative approaches, and the  warngc idea was 
 just one random suggestion.

 The point is, a mixture of  nogc and  assumenogc would achieve 
 exactly what I'm looking for. The only problem is that it may 
 cause issues for people that are using  nogc under a different 
 set of assumptions.


Also,  assumenogc or whatever does not play well with public 
APIs: if a library function is marked  nogc, its user can no 
longer know if it is really  nogc or if it does allocate via 
 assumenogc. This is not a good thing, as the user may want 
*absolutely* no GC allocation in his program, and may even have 
disabled the GC at program startup. Also, the code of that 
maybe-not- nogc library may not be available to check for this 
thing. Maybe the user doesn't want to trust the library writer, 
and so wants the compiler to guarantee no allocations at all.
Jul 28 2016
parent reply ag0aep6g <anonymous example.com> writes:
On 07/28/2016 10:46 PM, Lodovico Giaretta wrote:

 Also, the code of that maybe-not- nogc library may not be available to
 check for this thing. Maybe the user doesn't want to trust the library
 writer, and so wants the compiler to guarantee no allocations at all.


If you can't check the code, you have to trust the library writer. One 
can hack around  nogc as it is. It's not like dmd checks the object file 
for GC allocations.
Jul 28 2016
parent bitwise <bitwise.pvt gmail.com> writes:
On Thursday, 28 July 2016 at 21:07:22 UTC, ag0aep6g wrote:

 On 07/28/2016 10:46 PM, Lodovico Giaretta wrote:

 Also, the code of that maybe-not- nogc library may not be 
 available to
 check for this thing. Maybe the user doesn't want to trust the 
 library
 writer, and so wants the compiler to guarantee no allocations 
 at all.


 If you can't check the code, you have to trust the library 
 writer. One can hack around  nogc as it is. It's not like dmd 
 checks the object file for GC allocations.


Yeah... So on one hand, currently, you could potentially have 
some random hack misbehaving inside  nogc code with no way to 
detect it, whereas a simple search for  assumenogc would 
immediately tell you if the  nogc convention was being broken for 
any reason. On the other hand, adding  assumenogc may increase 
the amount of instances where this is happening, cause this 
search to be mandatory for every package you decide to download.

Maybe if there were 3 variations, this could work:

// use current convention. 100% guarantee, no allocations.
 nogc

// same as current, but restriction can be broken by  nogc(off)
 nogc(weak)

// allows allocations. Can only override  nogc(weak), but not 
 nogc.
 nogc(off)


Example:

class NPC {
	Cake cake;
	void start() {
		cake = new Cake();  // OK
	}
	void update()  nogc(weak) {
		cake = new Cake();     // error: cannot allocate here
		 nogc(off) {
			cake = new Cake();  // ok: alloc allowed in  nogc(off)
		}
	}
	void draw()  nogc {
		 nogc(off) {    // error:  nogc(off) not legal inside  nogc
			cake = new Cake();
		}
		bar(); // error: nogc(weak) not callable here
	}
	void bar()  nogc(weak) { }
}
Jul 28 2016
prev sibling parent reply jdfgjdf <jdfgjdf jdfgjdf.ck> writes:
On Sunday, 24 July 2016 at 22:13:02 UTC, bitwise wrote:

 ...


I like the  typo in the title. Let's make a game:

"write the `bypassign` function, its ddoc, and its unittests"

:)
Jul 28 2016
parent jdfgjdf <jdfgjdf jdfgjdf.ck> writes:
On Thursday, 28 July 2016 at 17:54:20 UTC, jdfgjdf wrote:

 On Sunday, 24 July 2016 at 22:13:02 UTC, bitwise wrote:

 ...


 I like the  typo in the title. Let's make a game:

 "write the `bypassign` function, its ddoc, and its unittests"

 :)


Entry one:

/**
  * Bypassign is a struct wrapper that bypasses the original 
type's `opAssign()`,
  * thus any assignation to the wrapped type is a noop.
  */
struct Bypassign(T)
if (is(T == struct))
{
     private T _bypassigned;
     alias _bypassigned this;
     void opAssign(T)(auto ref T t)
     {/*bypassignation*/}
}
///
unittest
{
     static struct Foo
     {
         int i;
         alias i this;
     }
     Bypassign!Foo bpaFoo;
     bpaFoo = 1;
     assert(bpaFoo == 0);
}
Jul 28 2016