www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Pay as you go is really going to make a difference

reply aberba <karabutaworld gmail.com> writes:
https://tonsky.me/blog/disenchantment/

Let's kill the bloat!!

Software disenchantment
=============================
I’ve been programming for 15 years now. Recently, our industry’s 
lack of care for efficiency, simplicity, and excellence started 
really getting to me, to the point of me getting depressed by my 
own career and IT in general.
.....
Only in software, it’s fine if a program runs at 1% or even 0.01% 
of the possible performance. Everybody just seems to be ok with 
it. People are often even proud about how inefficient it is, as 
in “why should we worry, computers are fast enough”:
...
Look around: our portable computers are thousands of times more 
powerful than the ones that brought man to the moon. Yet every 
other webpage struggles to maintain a smooth 60fps scroll on the 
latest top-of-the-line MacBook Pro. I can comfortably play games, 
watch 4K videos, but not scroll web pages? How is that ok?
...
Modern text editors have higher latency than 42-year-old Emacs. 
Text editors! What can be simpler? On each keystroke, all you 
have to do is update a tiny rectangular region and modern text 
editors can’t do that in 16ms. It’s a lot of time. A LOT. A 3D 
game can fill the whole screen with hundreds of thousands (!!!) 
of polygons in the same 16ms and also process input, recalculate 
the world and dynamically load/unload resources. How come?
Jan 12 2020
next sibling parent reply Arine <arine123445128843 gmail.com> writes:
On Sunday, 12 January 2020 at 20:29:59 UTC, aberba wrote:

 https://tonsky.me/blog/disenchantment/



Wow, this person is really uninformed. They know just enough 
about something to make a naive comment but not enough to 
understand *why* it is the way it is.


 An Android system with no apps takes up almost 6 GB. Just think 
 for a second about how obscenely HUGE that number is. What’s in 
 there, HD movies? I guess it’s basically code: kernel, drivers. 
 Some string and resources too, sure, but those can’t be big. 
 So, how many drivers do you need for a phone?


The onboard memory on an android device is generally hardwired 
into the system. That means the 
system/vendor/boot/dtbo/vbmeta/etc... partitions are going to be 
a set size. So even if my system image is 1.4 GB and vendor image 
is 500 mb. It'll still take up 6 GB if that's what was allocated 
to that partition. The device I'm working on currently has about 
4 GB for the system partition and 1 GB for the vendor partition.

So about 1.4 GB for system image, the largest  folders are for 
apps. The largest app is Webview totaling 108 MB (in app/). So 
just the webview alone can take half the space of the rest of the 
public apps. This is meant to be a minimal android build, so I 
wouldn't doubt a lot of that space does end up being taken up by 
pre-installed apps, and extra space for future updates.

12K	addon.d
225M	app
27M	bin
4.0K	build.prop
104K	compatibility_matrix.xml
5.9M	etc
20K	fake-libs
16K	fake-libs64
69M	fonts
217M	framework
149M	lib
222M	lib64
21M	media
233M	priv-app
8.0K	product
27M	usr
0	vendor
13M	xbin


 Windows 95 was 30MB. Today we have web pages heavier than that! 
 Windows 10 is 4GB, which is 133 times as big. But is it 133 
 times as superior? I mean, functionally they are basically the 
 same. Yes, we have Cortana, but I doubt it takes 3970 MB. But 
 whatever Windows 10 is, is Android really 150% of that?


Not sure why he thinks things taking up more spaces means they 
have to be better somehow? Developers have limited time, I'm sure 
they could squeeze out 500+ MB or something, but how much 
developer time would that take? Is it worth wasting the time to 
minimize it that much when people have 4 TB hdds? When they can 
download a 4 GB file in < 2 mins.

That's what he isn't getting. Doing these things isn't free. It 
takes development time to do all these things, development time 
that thanks to the hardware we have today, allows for it to be 
spent else where. Where it is more valuable. I remember using 
Windows 95, it's garbage in comparison to Windows 10. Comparing 
an OS based solely on it's file size, is just something someone 
incompetent would do.


 Modern text editors have higher latency than 42-year-old Emacs. 
 Text editors! What can be simpler? On each keystroke, all you 
 have to do is update a tiny rectangular region and modern text 
 editors can’t do that in 16ms. It’s a lot of time. A LOT. A 3D 
 game can fill the whole screen with hundreds of thousands (!!!) 
 of polygons in the same 16ms and also process input, 
 recalculate the world and dynamically load/unload resources. 
 How come?


I'll just assume he's talking about electron based editors here. 
They are built ontop of a web browser so yah they are going to be 
a bit more resource hungry. But if you take VS Code as an 
example. It is extremely easy to customize. There's no dozens of 
forks of it that modify little things to get certain features. 
There's more quality extensions for VS Code that integrate 
flawless, that aren't hacks than there are for Emacs, even though 
VS Code hasn't existed for nearly as long. There's a trade off 
for the ease of development and customizability. The latency also 
isn't that bad, it is pretty bad in Atom but that just shows the 
difference between the two.

Then he compares that to games and GPU rendering, just ugh. Maybe 
he didn't know it runs in a web browser, but he also goes on a 
rant about how web browsers don't render fast enough for him. Web 
browsers need to be secure, achieving performance a long side 
that is difficult. He seems to be in the mind set that security 
doesn't matter, or at the very least he probably doesn't think 
about it, as it seems to be more often than it should be. There's 
a reason why there's only so many web browsers. Hell even 
Microsoft gave up and uses Chromium's backend. Think about that, 
Microsoft, with a B.

Then the whole, oh we went to the moon with these slow computers. 
Yah going to the moon is pretty easy in comparison to some 
computer problems. As someone put, when a politician tried to 
make the same argument about going to the moon, it'd be akin to 
walking on the surface of the sun.

I could go on, this article is way too long and it's filled with 
misconceptions, terribly awful comparisons, and just so much more.
Jan 12 2020
parent reply user5678 <user5678 9012.sd> writes:
On Sunday, 12 January 2020 at 22:59:22 UTC, Arine wrote:

 On Sunday, 12 January 2020 at 20:29:59 UTC, aberba wrote:

 https://tonsky.me/blog/disenchantment/



 Wow, this person is really uninformed. They know just enough 
 about something to make a naive comment but not enough to 
 understand *why* it is the way it is.


 An Android system with no apps takes up almost 6 GB. Just 
 think for a second about how obscenely HUGE that number is. 
 What’s in there, HD movies? I guess it’s basically code: 
 kernel, drivers. Some string and resources too, sure, but 
 those can’t be big. So, how many drivers do you need for a 
 phone?


 The onboard memory on an android device is generally hardwired 
 into the system. That means the 
 system/vendor/boot/dtbo/vbmeta/etc... partitions are going to 
 be a set size. So even if my system image is 1.4 GB and vendor 
 image is 500 mb. It'll still take up 6 GB if that's what was 
 allocated to that partition. The device I'm working on 
 currently has about 4 GB for the system partition and 1 GB for 
 the vendor partition.

 So about 1.4 GB for system image, the largest  folders are for 
 apps. The largest app is Webview totaling 108 MB (in app/). So 
 just the webview alone can take half the space of the rest of 
 the public apps. This is meant to be a minimal android build, 
 so I wouldn't doubt a lot of that space does end up being taken 
 up by pre-installed apps, and extra space for future updates.

 12K	addon.d
 225M	app
 27M	bin
 4.0K	build.prop
 104K	compatibility_matrix.xml
 5.9M	etc
 20K	fake-libs
 16K	fake-libs64
 69M	fonts
 217M	framework
 149M	lib
 222M	lib64
 21M	media
 233M	priv-app
 8.0K	product
 27M	usr
 0	vendor
 13M	xbin


 Windows 95 was 30MB. Today we have web pages heavier than 
 that! Windows 10 is 4GB, which is 133 times as big. But is it 
 133 times as superior? I mean, functionally they are basically 
 the same. Yes, we have Cortana, but I doubt it takes 3970 MB. 
 But whatever Windows 10 is, is Android really 150% of that?


 Not sure why he thinks things taking up more spaces means they 
 have to be better somehow? Developers have limited time, I'm 
 sure they could squeeze out 500+ MB or something, but how much 
 developer time would that take? Is it worth wasting the time to 
 minimize it that much when people have 4 TB hdds? When they can 
 download a 4 GB file in < 2 mins.

 That's what he isn't getting. Doing these things isn't free. It 
 takes development time to do all these things, development time 
 that thanks to the hardware we have today, allows for it to be 
 spent else where. Where it is more valuable. I remember using 
 Windows 95, it's garbage in comparison to Windows 10. Comparing 
 an OS based solely on it's file size, is just something someone 
 incompetent would do.


 Modern text editors have higher latency than 42-year-old 
 Emacs. Text editors! What can be simpler? On each keystroke, 
 all you have to do is update a tiny rectangular region and 
 modern text editors can’t do that in 16ms. It’s a lot of time. 
 A LOT. A 3D game can fill the whole screen with hundreds of 
 thousands (!!!) of polygons in the same 16ms and also process 
 input, recalculate the world and dynamically load/unload 
 resources. How come?


 I'll just assume he's talking about electron based editors 
 here. They are built ontop of a web browser so yah they are 
 going to be a bit more resource hungry. But if you take VS Code 
 as an example. It is extremely easy to customize. There's no 
 dozens of forks of it that modify little things to get certain 
 features. There's more quality extensions for VS Code that 
 integrate flawless, that aren't hacks than there are for Emacs, 
 even though VS Code hasn't existed for nearly as long. There's 
 a trade off for the ease of development and customizability. 
 The latency also isn't that bad, it is pretty bad in Atom but 
 that just shows the difference between the two.

 Then he compares that to games and GPU rendering, just ugh. 
 Maybe he didn't know it runs in a web browser, but he also goes 
 on a rant about how web browsers don't render fast enough for 
 him. Web browsers need to be secure, achieving performance a 
 long side that is difficult. He seems to be in the mind set 
 that security doesn't matter, or at the very least he probably 
 doesn't think about it, as it seems to be more often than it 
 should be. There's a reason why there's only so many web 
 browsers. Hell even Microsoft gave up and uses Chromium's 
 backend. Think about that, Microsoft, with a B.

 Then the whole, oh we went to the moon with these slow 
 computers. Yah going to the moon is pretty easy in comparison 
 to some computer problems. As someone put, when a politician 
 tried to make the same argument about going to the moon, it'd 
 be akin to walking on the surface of the sun.

 I could go on, this article is way too long and it's filled 
 with misconceptions, terribly awful comparisons, and just so 
 much more.


Nah, the author of the article is right. The web and its techs 
(js) are completely retarded. I've myself observed on top of the 
usual slowness, a regression, on several major sites, during the 
latest months.

About the part on keyboard latency. This is based on another blog 
post I've read a few years ago. 
(https://hexus.net/tech/news/peripherals/113648-modern-computer-complexity-heavy-impac
-keyboard-latency/) and other stuff too.

The problem might be that you're so much into the web services 
that you don't even realize anymore how softs made in the classic 
way were much faster. But those softs failed to adapt to the web 
so developers have started to look elsewhere for a better 
interaction with the web.

At some point in the late 2000's compiled languages have lost. 
The article posted by the OP is fundamentally about that, if you 
read between the lines.
Jan 13 2020
next sibling parent reply Arine <arine123445128843 gmail.com> writes:
On Monday, 13 January 2020 at 11:54:19 UTC, user5678 wrote:

 About the part on keyboard latency. This is based on another 
 blog post I've read a few years ago. 
 (https://hexus.net/tech/news/peripherals/113648-modern-computer-complexity-heavy-impac
-keyboard-latency/) and other stuff too.


He wasn't talking about general latency such as that. Otherwise 
games would have the same problem, and he was specifically 
talking about editors.


 The article posted by the OP is fundamentally about that, if 
 you read between the lines.


You're just taking your own meaning from the article. If you have 
to "read between the lines", you aren't reading what the author 
actually wrote.
Jan 13 2020
parent reply Basile B. <b2.temp gmx.com> writes:
On Monday, 13 January 2020 at 17:20:05 UTC, Arine wrote:

 On Monday, 13 January 2020 at 11:54:19 UTC, user5678 wrote:

 About the part on keyboard latency. This is based on another 
 blog post I've read a few years ago. 
 (https://hexus.net/tech/news/peripherals/113648-modern-computer-complexity-heavy-impac
-keyboard-latency/) and other stuff too.


 He wasn't talking about general latency such as that. Otherwise 
 games would have the same problem, and he was specifically 
 talking about editors.


 The article posted by the OP is fundamentally about that, if 
 you read between the lines.


 You're just taking your own meaning from the article. If you 
 have to "read between the lines", you aren't reading what the 
 author actually wrote.


everything is slow because OS like windows, who deprecate their 
native UI
and prefer bloated apps that have to pass in a validation process 
making


But their OS, so supposedly made with their "fantastic tech", has 
to be patched every month, using an remarkably slow process that 
will monopolize your network.

At the same time everybody thinks that VScode is great.
There's a name for that

DIGITAL WASHING
Jan 14 2020
parent aria chris <ariachris56 gmail.com> writes:
I agree with the first comment.
Jan 16 2020
prev sibling next sibling parent reply Arine <arine123445128843 gmail.com> writes:
On Monday, 13 January 2020 at 11:54:19 UTC, user5678 wrote:

 (https://hexus.net/tech/news/peripherals/113648-modern-computer-complexity-heavy-impac
-keyboard-latency/) and other stuff too.


He's comparing two different technologies. If you want low input 
lag, get a TN panel gaming monitor with a high refresh rate. The 
thing is those cost $$$. All the while most of the devices he's 
testing are laptops. I'd love to a see a CRT display in a laptop. 
Read between the lines, that the author doesn't know what their 
doing.
Jan 13 2020
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Jan 13, 2020 at 05:40:08PM +0000, Arine via Digitalmars-d wrote:

 On Monday, 13 January 2020 at 11:54:19 UTC, user5678 wrote:

 (https://hexus.net/tech/news/peripherals/113648-modern-computer-complexity-heavy-impact-keyboard-latency/)
 and other stuff too.


 
 He's comparing two different technologies. If you want low input lag,
 get a TN panel gaming monitor with a high refresh rate. The thing is
 those cost $$$. All the while most of the devices he's testing are
 laptops. I'd love to a see a CRT display in a laptop. Read between the
 lines, that the author doesn't know what their doing.


You're totally missing the point.  The point is to take a step back at
the current state of things and evaluate just how much it (doesn't) make
sense:

1) Back in the 70's, we had 16 kHz CPUs and only up to 64KB of RAM.

2) Today we're in 2020, with multi-core CPUs running at speeds measured
in GHz, and RAM measured in GBs.

3) A word processor in the 70's runs horribly slowly with horrible lag
between input keystrokes.

4) Technologically speaking, today we have enough processing power to
run AAA games that process hundreds of thousands of objects per frame
running at 60 fps.  We're talking about things like *real-time
raytracing* here, something completely unimaginable in the 70's.

5) Yet a browser app of today, built with said modern technology with
modern processing power, still runs just as horribly slowly as a word
processor from the 70's running on ancient ultra-slow hardware, with
just as horrible a lag between input keystrokes.

Something isn't adding up.

Yes, all of this can be explained, and if you lose sight of the forest
for the trees, every step in the history of how this came about can be
logically explained. But when you step back and look at the forest as a
whole, the whole situation looks completely ridiculous.  The necessary
tech is all there to make things FAR more efficient. The development
methodologies are all there, and we have orders of magnitude more
manpower than in the 70's.  What a word processor has to compute is
peanuts compared to an AAA game with real-time raytracing running at 60
fps. Yet here we are, stuck with a completely insane web design
philosophy building horribly slow and unreliable apps that are barely a
step above an ancient word processor from the 70's.

The browser king wears no clothes, yet its proponents see invisible.


T

-- 
An elephant: A mouse built to government specifications. -- Robert Heinlein
Jan 13 2020
parent reply Arine <arine123445128843 gmail.com> writes:
On Monday, 13 January 2020 at 18:22:19 UTC, H. S. Teoh wrote:

 On Mon, Jan 13, 2020 at 05:40:08PM +0000, Arine via 
 Digitalmars-d wrote:

 On Monday, 13 January 2020 at 11:54:19 UTC, user5678 wrote:

 (https://hexus.net/tech/news/peripherals/113648-modern-computer-complexity-heavy-impac
-keyboard-latency/) and other stuff too.


 
 He's comparing two different technologies. If you want low 
 input lag, get a TN panel gaming monitor with a high refresh 
 rate. The thing is those cost $$$. All the while most of the 
 devices he's testing are laptops. I'd love to a see a CRT 
 display in a laptop. Read between the lines, that the author 
 doesn't know what their doing.


 You're totally missing the point.  The point is to take a step 
 back at the current state of things and evaluate just how much 
 it (doesn't) make sense:


It does make sense. Software back then wasn't complicated, it 
didn't have to be. Developer time has remained constant. Software 
companies failed because they were trying to shoot for 
perfection. You can't create a perfect piece of software. You 
have to use the limited developer time you have to and allocate 
that time effectively. Not trying to reduce file size cause some 
UX designed that doesn't know what he's doing or talking about 
rants about it on his blog.


 4) Technologically speaking, today we have enough processing 
 power to run AAA games that process hundreds of thousands of 
 objects per frame running at 60 fps.  We're talking about 
 things like *real-time raytracing* here, something completely 
 unimaginable in the 70's.

 Yes, all of this can be explained, and if you lose sight of the 
 forest for the trees, every step in the history of how this 
 came about can be logically explained. But when you step back 
 and look at the forest as a whole, the whole situation looks 
 completely ridiculous.  The necessary tech is all there to make 
 things FAR more efficient. The development methodologies are 
 all there, and we have orders of magnitude more manpower than 
 in the 70's.  What a word processor has to compute is peanuts 
 compared to an AAA game with real-time raytracing running at 60 
 fps.


Raytracing is just a marketing buzzword, it's exist for decades 
in games and it's been used in realtime for almost as long. 
That's the problem when you have people like you that don't 
understand what they are talking about, throwing things like. Oh 
we can do "raytracing" in real time then comparing that as if it 
means something because we can do that. GPUs have been doing 
operations like that for a long time, doing a lot simple tasks 
thousands at a time in parallel. But there's still a reason you 
can't run an operating system using a GPU. It's fundamentally 
difference.


 5) Yet a browser app of today, built with said modern 
 technology with modern processing power, still runs just as 
 horribly slowly as a word processor from the 70's running on 
 ancient ultra-slow hardware, with just as horrible a lag 
 between input keystrokes.

 Yet here we are, stuck with a completely insane web design 
 philosophy building horribly slow and unreliable apps that are 
 barely a step above an ancient word processor from the 70's.


I use VS Code and Discord (both made using electron btw) all the 
time, there's no lag. It's probably more responsive than most 
bloated IDEs that weren't built using electron. Bad programs are 
going to be bad.
Jan 16 2020
parent JN <666total wp.pl> writes:
On Thursday, 16 January 2020 at 19:38:21 UTC, Arine wrote:

 Raytracing is just a marketing buzzword, it's exist for decades 
 in games and it's been used in realtime for almost as long. 
 That's the problem when you have people like you that don't 
 understand what they are talking about, throwing things like. 
 Oh we can do "raytracing" in real time then comparing that as 
 if it means something because we can do that. GPUs have been 
 doing operations like that for a long time, doing a lot simple 
 tasks thousands at a time in parallel. But there's still a 
 reason you can't run an operating system using a GPU. It's 
 fundamentally difference.


That's not true. While I believe the current trend of 
'raytracing' is mostly hype build by NVidia to sell their RTX 
GPUs, real time raytracing wasn't viable in the past. It only 
worked for simple scenes with few cubes and spheres and was very 
low resolution/noisy. Now we have the performance to do that, 
also we can use machine learning to denoise the image in a much 
better way than the previous algorithms did.
Jan 16 2020
prev sibling parent user5678 <user5678 9012.sd> writes:
On Monday, 13 January 2020 at 11:54:19 UTC, user5678 wrote:

 On Sunday, 12 January 2020 at 22:59:22 UTC, Arine wrote:

 [...]


 Nah, the author of the article is right. The web and its techs 
 (js) are completely retarded. I've myself observed on top of 
 the usual slowness, a regression, on several major sites, 
 during the latest months.

 [...]


So instead of having your own system interacting with the web the 
situation is that the web tries to interact with your system, 
which is a completely crazy situation.
Jan 13 2020
prev sibling next sibling parent Ron Tarrant <rontarrant gmail.com> writes:
On Sunday, 12 January 2020 at 20:29:59 UTC, aberba wrote:

 https://tonsky.me/blog/disenchantment/

 Let's kill the bloat!!

 Software disenchantment
 =============================
 I’ve been programming for 15 years now. Recently, our 
 industry’s lack of care for efficiency, simplicity, and 
 excellence started really getting to me, to the point of me 
 getting depressed by my own career and IT in general.
 ...
 “why should we worry, computers are fast enough”:
 ...
 Look around: our portable computers are thousands of times more 
 powerful than the ones that brought man to the moon.
 ...
 Modern text editors have higher latency than 42-year-old Emacs.



This has bugged me for a while, too. Behind you all the way, 
Aberba.
Jan 13 2020
prev sibling parent reply Martin Tschierschke <mt smartdolphin.de> writes:
On Sunday, 12 January 2020 at 20:29:59 UTC, aberba wrote:

 https://tonsky.me/blog/disenchantment/

 Let's kill the bloat!!


And there is an other effect of this ever growing bloat.
I have two old iPads one iPad 1 and an iPad 2.
Both are in perfect hardware condition but, you can not use them
for much anymore, because of their small (256 and 512 MB) RAM the 
available browsers
are not able to render most of 'modern' webpages.

So the ever increasing need of memory for the simplest tasks is 
killing
old hardware.

The last computer, which software was optimized to the ultimate 
was probably the Commodore C64. After that the availability of 
more and more resources (CPU speed and RAM) has started building 
an ever increasing amount of additional layers between input and 
output.

Just look at a simple - statically linked - "hello world" DMD 
compilation result,
how many C64 times floppy discs (180KByte) you would need to 
store?

I think this process will not end as long as new storage and 
bandwidth is getting cheaper all the time.

But maybe I am wrong and the next generation of software 
engineers will bring the gain of Moors Law to us. (And the 
resources needed for computing world wide will stop increasing.)
Jan 16 2020
next sibling parent reply Gregor =?UTF-8?B?TcO8Y2ts?= <gregormueckl gmx.de> writes:
On Thursday, 16 January 2020 at 14:03:15 UTC, Martin Tschierschke 
wrote:

 But maybe I am wrong and the next generation of software 
 engineers will bring the gain of Moors Law to us. (And the 
 resources needed for computing world wide will stop increasing.)


It is especially the current young generation of coders that gets 
socialized with HTML+JS "GUIs" (yup, scare quotes!) and "apps" 
that are just services on somebody else's server.

There's just so many incentives pointing the wrong way:

- Cloud providers want to lock their customers in (Google, 
Amazon, MS)
- Software developers see how they can squeeze juicy subscription 
fees out of their customers when they don't sell installable 
software, but run it as a service
- Commercial users see shiny presentations that tell them that 
not running their software in-house is so much cheaper (and it's 
likely true until they lose access to their data or a critical 
3rd party service falls over)

I only see a single chance to get out of this particular hole: 
completely new devices that are more desirable than PCs, tablets 
or smartphones and for which the web as it exists today makes 
absolutely no sense. I see one chance of this happening if 
everyday augmented reality matures in about 5 to 10 years - and 
that's still a pretty big if.
Jan 16 2020
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Jan 16, 2020 at 03:08:47PM +0000, Gregor Mckl via Digitalmars-d wrote:
[...][

 There's just so many incentives pointing the wrong way:
 
 - Cloud providers want to lock their customers in (Google, Amazon, MS)


Yep, that's why I'm skeptical of this whole cloud hype. It's just like
the days of Java all over again: OO, which does have its uses, being
sold far beyond its scope of usefulness, and Java, which actually isn't
a *bad* language in certain respects, being sold as the panacea that
will solve all your programming problems and relieve you of the
necessity of thought. Only today, substitute OO with "cloud" and Java
with "webapps".

Cloud vendors want to lock you in, when the correct strategy is multiple
redundant systems (cf. Walter's rants about Boeing design). But you
can't have multiple redundant systems -- not in Walter's sense of
multiple *independent* systems that aren't running upon the same
principles that might fail *at the same time* -- if cloud vendors refuse
to let you interoperate with their competitors' systems, or only allows
arbitrarily restricted such interoperation, such that your redundancy is
essentially crippled and you might as well not bother.



 - Software developers see how they can squeeze juicy subscription fees
 out of their customers when they don't sell installable software, but
 run it as a service


Yeah, I have a lot of ideological problems with that. The first and
foremost being that your ability to use potentially mission-critical
functionality is now dependent on the state of some remote server farm
that's completely beyond your control.  Last year's AWS outage is just
the tip of the iceberg of what might happen if everyone becomes
dependent on the web (they already are) and the web becomes fragile
because of reliance on a small number of points of failure (already
happened: cloud providers), and something happens to one of these points
of failure.

(Well, you object, cloud providers have multiple redundant distributed
servers, so they're not vulnerable to single-point-of-failure problems.
Wrong, their *individual* servers can failover transparently, but
sometimes the *entire service* goes down for whatever reason -- faulty
software that all servers are running copies of, for instance. Or
targeted cybercriminal attacks on that service as a whole. Or the
company goes bust suddenly, who knows. Centralization of critical
services -- esp. on a 3rd party whose interests may not coincide with
yours -- is not a wise move.)



 - Commercial users see shiny presentations that tell them that not
 running their software in-house is so much cheaper (and it's likely
 true until they lose access to their data or a critical 3rd party
 service falls over)


[...]

Yeah, this is another major ideological problem I have with this whole
cloud hype. Your data doesn't belong to you anymore; it's sitting on the
hard drives of some 3rd party whose interests do not necessarily
coincide with yours. The accessibility of your mission-critical data is
dependent upon the availability of some remote service that isn't under
your control.  You're in trouble if the service goes down, or becomes
unavailable for whatever reason during the critical times when you most
need your data. You're in trouble if the 3rd party gets hacked and now
your supposedly private data is out in the open.  Or there's a serious
security flaw that you were never aware of, that has left your data that
you thought was securely stored open to the whole world.  And worst of
all, your data is in the hands of a 3rd party who has the power to do
what they want with it, and their interests may not coincide with yours.

How anyone could be comfortable with that idea is beyond me.


T

-- 
Why have vacation when you can work?? -- EC
Jan 16 2020
next sibling parent Rumbu <rumbu rumbu.ro> writes:
On Thursday, 16 January 2020 at 17:59:53 UTC, H. S. Teoh wrote:

 On Thu, Jan 16, 2020 at 03:08:47PM +0000, Gregor Mückl via 
 Digitalmars-d wrote: [...][

 There's just so many incentives pointing the wrong way:
 
 - Cloud providers want to lock their customers in (Google, 
 Amazon, MS)


 Yeah, this is another major ideological problem I have with 
 this whole cloud hype. Your data doesn't belong to you anymore; 
 it's sitting on the hard drives of some 3rd party whose 
 interests do not necessarily coincide with yours. The 
 accessibility of your mission-critical data is dependent upon 
 the availability of some remote service that isn't under your 
 control.  You're in trouble if the service goes down, or 
 becomes unavailable for whatever reason during the critical 
 times when you most need your data. You're in trouble if the 
 3rd party gets hacked and now your supposedly private data is 
 out in the open.  Or there's a serious security flaw that you 
 were never aware of, that has left your data that you thought 
 was securely stored open to the whole world.  And worst of all, 
 your data is in the hands of a 3rd party who has the power to 
 do what they want with it, and their interests may not coincide 
 with yours.

 How anyone could be comfortable with that idea is beyond me.


It depends. The business world is more dynamic today. As a 
startup company you have access to advanced technologies that you 
never dream of in a blink of time.

Last year I started a new company. In 30 minutes I had a fully 
fledged e-mail system, a communication platform, a secure 
environment and a nice pack of development software. Uploaded my 
databases, opened Visual Studio, load the project, changed some 
settings in the configuration file, hit Build, hit Publish 
button. Zbang, my web application is up and running in the wild. 
As a service, I don't even need a virtual machine for this.

The company doesn't even have a physical office, we are three 
partners and all we got are three laptops working from home. 300 
eur/month licenses and services.

Now imagine the same scenario years ago. Buy some servers, buy 
storage, buy firewall, configure, install. Set-up e-mail, setup 
network, have a server room, put some cables. 30k eur at least.

More than that, since I am working in the payroll industry, 
clients ask for security certifications. We cannot afford to buy 
such systems and services to meet their criteria. Instead I gave 
them the security certifications of the cloud provider, which are 
state of the art. I have access to secure technologies like data 
leaking prevention, audit, logging without any supplementary 
investment.
Jan 17 2020
prev sibling parent Chris <wendlec tcd.ie> writes:
On Thursday, 16 January 2020 at 17:59:53 UTC, H. S. Teoh wrote:

 Yeah, this is another major ideological problem I have with 
 this whole cloud hype. Your data doesn't belong to you anymore; 
 it's sitting on the hard drives of some 3rd party whose 
 interests do not necessarily coincide with yours. The 
 accessibility of your mission-critical data is dependent upon 
 the availability of some remote service that isn't under your 
 control.  You're in trouble if the service goes down, or 
 becomes unavailable for whatever reason during the critical 
 times when you most need your data. You're in trouble if the 
 3rd party gets hacked and now your supposedly private data is 
 out in the open.  Or there's a serious security flaw that you 
 were never aware of, that has left your data that you thought 
 was securely stored open to the whole world.  And worst of all, 
 your data is in the hands of a 3rd party who has the power to 
 do what they want with it, and their interests may not coincide 
 with yours.

 How anyone could be comfortable with that idea is beyond me.


 T


All valid points, but what do you suggest as an alternative? 
Create your own service from scratch? Can you guarantee your 
customers that your own software is secure and will not be hacked 
easily? All their personal data and financial transactions. The 
whole thing is just too big to roll your own. If you buy a car, 
you're "locked in", but does that mean you should build your own 
car? The market is about division of labor, else there wouldn't 
be progress. Gone are the romantic days of yore when people where 
farmers, thatchers and fishermen at the same time.
Jan 17 2020
prev sibling next sibling parent reply Chris <wendlec tcd.ie> writes:
On Thursday, 16 January 2020 at 14:03:15 UTC, Martin Tschierschke 
wrote:

 On Sunday, 12 January 2020 at 20:29:59 UTC, aberba wrote:

 https://tonsky.me/blog/disenchantment/

 Let's kill the bloat!!


 And there is an other effect of this ever growing bloat.
 I have two old iPads one iPad 1 and an iPad 2.
 Both are in perfect hardware condition but, you can not use them
 for much anymore, because of their small (256 and 512 MB) RAM 
 the available browsers
 are not able to render most of 'modern' webpages.

 So the ever increasing need of memory for the simplest tasks is 
 killing
 old hardware.

 The last computer, which software was optimized to the ultimate 
 was probably the Commodore C64. After that the availability of 
 more and more resources (CPU speed and RAM) has started 
 building an ever increasing amount of additional layers between 
 input and output.

 Just look at a simple - statically linked - "hello world" DMD 
 compilation result,
 how many C64 times floppy discs (180KByte) you would need to 
 store?

 I think this process will not end as long as new storage and 
 bandwidth is getting cheaper all the time.

 But maybe I am wrong and the next generation of software 
 engineers will bring the gain of Moors Law to us. (And the 
 resources needed for computing world wide will stop increasing.)


This was already known in the 80ies. It was called the 
hardware-software spiral or something like that. It's partly 
natural and partly by design to sell hardware and software. The 
more powerful the hardware the more software developers do (think 
of image and video editing), the more powerful the software, the 
slower the existing hardware, so you need to buy a new, more 
powerful machine, rinse and repeat...

As regards your iPads, Apple have always been mean with RAM and 
storage (unless you spend like $2000+). That's also by design, if 
Apple gives you 256/512MB RAM, they know exactly what they are 
doing, because they know that your iPad will soon be useless 
given the way the internet is evolving. They are aware of the 
hardware-software spiral.
Jan 17 2020
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 17 January 2020 at 12:19:18 UTC, Chris wrote:

 As regards your iPads, Apple have always been mean with RAM and 
 storage (unless you spend like $2000+). That's also by design, 
 if Apple gives you 256/512MB RAM, they know exactly what they 
 are doing, because they know that your iPad will soon be 
 useless given the way the internet is evolving. They are aware 
 of the hardware-software spiral.


Sadly, the iPad1 is quite capable, but is stuck on iOS5 and thus 
the browser will fail on many sites. In my experience RAM and CPU 
is not the main issue. I actually like the ergonomic shape of the 
iPad1 more than later models, but planned obsoletion is what you 
have to live with these days...  I still use it for wikipedia and 
pdfs, though :-)
Jan 17 2020
parent Chris <wendlec tcd.ie> writes:
On Friday, 17 January 2020 at 13:36:52 UTC, Ola Fosheim Grøstad 
wrote:

 Sadly, the iPad1 is quite capable, but is stuck on iOS5 and 
 thus the browser will fail on many sites. In my experience RAM 
 and CPU is not the main issue. I actually like the ergonomic 
 shape of the iPad1 more than later models, but planned 
 obsoletion is what you have to live with these days...  I still 
 use it for wikipedia and pdfs, though :-)


That's, of course, another trick to render devices useless. Make 
them un-updatable.
Jan 17 2020
prev sibling parent reply aberba <karabutaworld gmail.com> writes:
On Thursday, 16 January 2020 at 14:03:15 UTC, Martin Tschierschke 
wrote:

 On Sunday, 12 January 2020 at 20:29:59 UTC, aberba wrote:

 https://tonsky.me/blog/disenchantment/

 Let's kill the bloat!!





 Just look at a simple - statically linked - "hello world" DMD 
 compilation result,
 how many C64 times floppy discs (180KByte) you would need to 
 store?


That's the issue I wanted to address with the thread. Why is a 
simple 5 lines of code compiling to such a large binary. That's a 
typical example of bloat.
Jan 22 2020
parent reply IGotD- <nise nise.com> writes:
On Wednesday, 22 January 2020 at 10:38:54 UTC, aberba wrote:

 Just look at a simple - statically linked - "hello world" DMD 
 compilation result,
 how many C64 times floppy discs (180KByte) you would need to 
 store?


 That's the issue I wanted to address with the thread. Why is a 
 simple 5 lines of code compiling to such a large binary. That's 
 a typical example of bloat.


You have the same problem with C++ if you statically link the C 
and C++ library, then it will easily become over 2 MB. Many don't 
care because default is that you dynamically link those libraries 
in C++.

Problem is that D depends on the C library which can be 
statically linked and then it also becomes bigger. Even if you 
use just a very small portion of it, it has a tendency you just 
include everything anyway.

Many languages suffer from the C lib dependency which is kind of 
suboptimal. It is time to depreciate that dependency.
Jan 22 2020
next sibling parent reply FogD <hosszu outlook.com> writes:
On Wednesday, 22 January 2020 at 10:51:37 UTC, IGotD- wrote:

 Many languages suffer from the C lib dependency which is kind 
 of suboptimal. It is time to depreciate that dependency.


A recent comparison of languages from this perspective.

https://drewdevault.com/2020/01/04/Slow.html
Jan 22 2020
next sibling parent reply IGotD- <nise nise.com> writes:
On Wednesday, 22 January 2020 at 23:51:23 UTC, FogD wrote:

 On Wednesday, 22 January 2020 at 10:51:37 UTC, IGotD- wrote:

 Many languages suffer from the C lib dependency which is kind 
 of suboptimal. It is time to depreciate that dependency.


 A recent comparison of languages from this perspective.

 https://drewdevault.com/2020/01/04/Slow.html


It would be interesting to know what that huge number of system 
calls really do, especially when it comes to D which has around 
150.
Jan 22 2020
parent reply Johan Engelen <j j.nl> writes:
On Thursday, 23 January 2020 at 00:20:00 UTC, IGotD- wrote:

 On Wednesday, 22 January 2020 at 23:51:23 UTC, FogD wrote:

 A recent comparison of languages from this perspective.

 https://drewdevault.com/2020/01/04/Slow.html


 It would be interesting to know what that huge number of system 
 calls really do, especially when it comes to D which has around 
 150.


Indeed. Also to figure out why LDC's binary calls 31 more than 
DMD's binary.
Much appreciated if someone could repeat the test and post a list 
of all syscalls being made.

-Johan
Jan 24 2020
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/24/20 4:52 AM, Johan Engelen wrote:

 On Thursday, 23 January 2020 at 00:20:00 UTC, IGotD- wrote:

 On Wednesday, 22 January 2020 at 23:51:23 UTC, FogD wrote:

 A recent comparison of languages from this perspective.

 https://drewdevault.com/2020/01/04/Slow.html


 It would be interesting to know what that huge number of system calls 
 really do, especially when it comes to D which has around 150.


 
 Indeed. Also to figure out why LDC's binary calls 31 more than DMD's 
 binary.
 Much appreciated if someone could repeat the test and post a list of all 
 syscalls being made.


Most likely it's the runtime startup. Obviously sbrk quite a bit, but 
any runtime initialization (thread startup, mutex initialization, etc) 
are all going to go in there. Think of what the GC has to do!

A good test would be to do a betterC version with printf and see what 
the difference is (technically it should be the same as the C version).

-Steve
Jan 24 2020
next sibling parent IGotD- <nise nise.com> writes:
On Friday, 24 January 2020 at 13:06:33 UTC, Steven Schveighoffer 
wrote:

 Most likely it's the runtime startup. Obviously sbrk quite a 
 bit, but any runtime initialization (thread startup, mutex 
 initialization, etc) are all going to go in there. Think of 
 what the GC has to do!

 A good test would be to do a betterC version with printf and 
 see what the difference is (technically it should be the same 
 as the C version).

 -Steve


That makes perfectly sense, I didn't think about sbrk which needs 
to bump the heap quite a bit during the startup.
Jan 24 2020
prev sibling next sibling parent Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Friday, 24 January 2020 at 13:06:33 UTC, Steven Schveighoffer 
wrote:

 A good test would be to do a betterC version with printf and 
 see what the difference is (technically it should be the same 
 as the C version).


Indeed. I wrote an email to the author about that yesterday, 
though I haven't heard from him since.
Jan 24 2020
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Jan 24, 2020 at 08:06:33AM -0500, Steven Schveighoffer via
Digitalmars-d wrote:

 On 1/24/20 4:52 AM, Johan Engelen wrote:


[...]

 Indeed. Also to figure out why LDC's binary calls 31 more than DMD's
 binary.




[...]

 Most likely it's the runtime startup. Obviously sbrk quite a bit, but
 any runtime initialization (thread startup, mutex initialization, etc)
 are all going to go in there. Think of what the GC has to do!


[...]

It makes me wonder how much we can make all this startup stuff
pay-as-you-go.  I mean, IIRC, isn't the GC lazily initialized now? I
vaguely remember some PR along that direction. Or was it the pool
allocations?

I suppose thread startup would be hard to elide, unless there was a way
to initialize the thread stuff only on demand. Ditto for mutex inits.
But it might not be worth the effort for such minimal benefits in such a
marginal test case.


T

-- 
Those who've learned LaTeX swear by it. Those who are learning LaTeX swear at
it. -- Pete Bleackley
Jan 24 2020
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/24/20 1:24 PM, H. S. Teoh wrote:

 On Fri, Jan 24, 2020 at 08:06:33AM -0500, Steven Schveighoffer via
Digitalmars-d wrote:

 On 1/24/20 4:52 AM, Johan Engelen wrote:


 [...]

 Indeed. Also to figure out why LDC's binary calls 31 more than DMD's
 binary.




 [...]

 Most likely it's the runtime startup. Obviously sbrk quite a bit, but
 any runtime initialization (thread startup, mutex initialization, etc)
 are all going to go in there. Think of what the GC has to do!


 [...]
 
 It makes me wonder how much we can make all this startup stuff
 pay-as-you-go.  I mean, IIRC, isn't the GC lazily initialized now? I
 vaguely remember some PR along that direction. Or was it the pool
 allocations?


Yes, it is lazily initialized. It's kind of a cool mechanism too -- the 
"default" GC is a class that when used in a way where a "real" GC is 
needed (e.g. allocate some memory), figures out which one to create, 
creates it, and then replaces itself as the global handler with that new 
one.

But the GC is going to be initialized in a writeln call I think.

There's a few other things that are going to cause a lot of system calls 
too -- the static constructors and the cycle detection. At least the 
cycle detection we could rid ourselves of if we could make a 
post-compile step that runs the cycle detection algorithm and sets up 
the final ordering in the binary.


 I suppose thread startup would be hard to elide, unless there was a way
 to initialize the thread stuff only on demand. Ditto for mutex inits.
 But it might not be worth the effort for such minimal benefits in such a
 marginal test case.


I'm not sure why we need to exactly minimize the system calls, we should 
just be able to explain them. 150 calls isn't horrific, and trying to 
reduce an "artificial" metric like that really shouldn't be the goal. I 
know this is exactly what the author is complaining about, but there is 
a world of difference between a 50MB web site that can't scroll and 150 
system calls to do runtime startup + print hello world.

However, there could easily be an obvious candidate for removal if 
something looks like it's being called way too often. So explanation is 
still a good goal.

-Steve
Jan 24 2020
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2020-01-24 10:52, Johan Engelen wrote:


 Indeed. Also to figure out why LDC's binary calls 31 more than DMD's 
 binary.
 Much appreciated if someone could repeat the test and post a list of all 
 syscalls being made.


Not exactly the same as the original post, but here's some data I pulled 
out for macOS 10.14.6 with a Hello World compiled with DMD 2.088.0. This 
should give somewhat of an idea what's going on in the application. I've 
included the stacktrace for all syscalls NOT made by the system.

As you can see below, 10 calls are made by the application, all of the 
remaining calls are made by the system itself. Most of the calls made by 
the system are made by the dynamic loader. Only three calls originate 
from the D main function. Only one call into the C standard library is 
made from the D main function, which is the call to `fwrite`.

It's not our fault that the system does so many calls :).




total: 120, system: 110, app: 10

stat64, total: 40, system: 40, app: 0

-----------------------------------------------

mach_vm_map_trap, total: 8, system: 6, app: 2

    6 libsystem_malloc.dylib malloc
    7 foobar 
_D2rt5minfo11ModuleGroup9sortCtorsMFAyaZ6doSortMFmKAPyS6object10ModuleInfoZb
    8 foobar _D2rt5minfo11ModuleGroup9sortCtorsMFAyaZv
    9 foobar _D2rt5minfo11ModuleGroup9sortCtorsMFZv
   10 foobar 
_D2rt5minfo13rt_moduleCtorUZ14__foreachbody1MFKSQBu19sections_osx_x86_6412SectionGroupZi
   11 foobar rt_moduleCtor
   12 foobar rt_init
   13 foobar _D2rt6dmain212_d_run_main2UAAamPUQgZiZ6runAllMFZv
   14 foobar _D2rt6dmain212_d_run_main2UAAamPUQgZiZ7tryExecMFMDFZvZv
   15 foobar _d_run_main2
   16 foobar _d_run_main
   17 foobar main
   18 libdyld.dylib start



   10 libsystem_c.dylib fwrite
   11 foobar 
_D3std5stdio__T13trustedFwriteTaZQsFNbNiNePOS4core4stdcQBx7__sFILExAaZm 
~/.dvm/compilers/dmd-2.088.0/osx/bin/../../src/phobos/std/stdio.d:4322
   12 foobar _D3std5stdio4File17LockingTextWriter__T3putTAyaZQjMFNfMQlZv 
~/.dvm/compilers/dmd-2.088.0/osx/bin/../../src/phobos/std/stdio.d:2930
   13 foobar _D3std5stdio__T7writelnTAyaZQnFNfQjZv 
~/.dvm/compilers/dmd-2.088.0/osx/bin/../../src/phobos/std/stdio.d:3855
   14 foobar _Dmain ~/development/d/main.d:15
   15 foobar _D2rt6dmain212_d_run_main2UAAamPUQgZiZ6runAllMFZ9__lambda1MFZv
   16 foobar _D2rt6dmain212_d_run_main2UAAamPUQgZiZ7tryExecMFMDFZvZv
   17 foobar _D2rt6dmain212_d_run_main2UAAamPUQgZiZ6runAllMFZv
   18 foobar _D2rt6dmain212_d_run_main2UAAamPUQgZiZ7tryExecMFMDFZvZv
   19 foobar _d_run_main2
   20 foobar _d_run_main
   21 foobar main
   22 libdyld.dylib start


-----------------------------------------------
mprotect, total: 8, system: 8
mach_port_deallocate_trap, total: 4, system: 4
-----------------------------------------------
sigaction, total: 4, system: 0, app: 4

    0 libsystem_kernel.dylib __sigaction
    1 libsystem_platform.dylib __platform_sigaction
    2 foobar runModuleUnitTests
    3 foobar _D2rt6dmain212_d_run_main2UAAamPUQgZiZ6runAllMFZv
    4 foobar _D2rt6dmain212_d_run_main2UAAamPUQgZiZ7tryExecMFMDFZvZv
    5 foobar _d_run_main2
    6 foobar _d_run_main
    7 foobar main
    8 libdyld.dylib start

-----------------------------------------------
mach_reply_port, total: 4, system: 4, app: 0
host_self_trap, total: 3, system: 3, app: 0
mach_port_mod_refs_trap, total: 3, system: 3, app: 0
ioctl, total: 3, system: 3, app: 0
mach_port_construct_trap, total: 3, system: 3, app: 0
kdebug_typefilter, total: 2, system: 2, app: 0
thread_self_trap, total: 2, system: 2, app: 0
proc_info, total: 2, system: 2, app: 0
mach_port_destruct_trap, total: 2, system: 2, app: 0
csops, total: 2, system: 2, app: 0
getpid, total: 2, system: 2, app: 0
task_self_trap, total: 2, system: 2, app: 0
access, total: 1, system: 1, app: 0
close, total: 1, system: 1, app: 0
shared_region_check_np, total: 1, system: 1, app: 0
-------------------------------------------------------
fstat64, total: 1, system: 0, app: 1

    5 libsystem_c.dylib fwrite
    6 foobar 
_D3std5stdio__T13trustedFwriteTaZQsFNbNiNePOS4core4stdcQBx7__sFILExAaZm 
~/.dvm/compilers/dmd-2.088.0/osx/bin/../../src/phobos/std/stdio.d:4322
    7 foobar _D3std5stdio4File17LockingTextWriter__T3putTAyaZQjMFNfMQlZv 
~/.dvm/compilers/dmd-2.088.0/osx/bin/../../src/phobos/std/stdio.d:2930
    8 foobar _D3std5stdio__T7writelnTAyaZQnFNfQjZv 
~/.dvm/compilers/dmd-2.088.0/osx/bin/../../src/phobos/std/stdio.d:3855
    9 foobar _Dmain ~/development/d/main.d:15
   10 foobar _D2rt6dmain212_d_run_main2UAAamPUQgZiZ6runAllMFZ9__lambda1MFZv
   11 foobar _D2rt6dmain212_d_run_main2UAAamPUQgZiZ7tryExecMFMDFZvZv
   12 foobar _D2rt6dmain212_d_run_main2UAAamPUQgZiZ6runAllMFZv
   13 foobar _D2rt6dmain212_d_run_main2UAAamPUQgZiZ7tryExecMFMDFZvZv
   14 foobar _d_run_main2
   15 foobar _d_run_main
   16 foobar main
   17 libdyld.dylib start

-----------------------------------------------
mac_vm_allocate_trap, total: 1, system: 1, app: 0
open, total: 1, system: 1, app: 0
mac_syscall, total: 1, system: 1, app: 0
-----------------------------------------------
sysctl, total: 1, system: 0, app: 1

    3 libsystem_c.dylib sysconf
    4 foobar _D4core6thread26_sharedStaticCtor_L3685_C1FZv
    5 foobar _D4core6thread15__modsharedctorFZv
    6 foobar 
_D2rt5minfo__T14runModuleFuncsSQBdQBd11ModuleGroup8runCtorsMFZ9__lambda2ZQChMFAxPyS6object10ModuleInfoZv
    7 foobar _D2rt5minfo11ModuleGroup8runCtorsMFZv
    8 foobar 
_D2rt5minfo13rt_moduleCtorUZ14__foreachbody1MFKSQBu19sections_osx_x86_6412SectionGroupZi
    9 foobar rt_moduleCtor
   10 foobar rt_init
   11 foobar _D2rt6dmain212_d_run_main2UAAamPUQgZiZ6runAllMFZv
   12 foobar _D2rt6dmain212_d_run_main2UAAamPUQgZiZ7tryExecMFMDFZvZv
   13 foobar _d_run_main2
   14 foobar _d_run_main
   15 foobar main
   16 libdyld.dylib start


-----------------------------------------------
getentropy, total: 1, system: 1, app: 0
issetugid, total: 1, system: 1, app: 0
bsdthread_register, total: 1, system: 1, app: 0

-----------------------------------------------
write_nocancel, total: 1, system: 0, app: 1

    0 libsystem_kernel.dylib __write_nocancel
    1 libsystem_c.dylib _swrite
    2 libsystem_c.dylib __sflush
    3 libsystem_c.dylib fflush
    4 foobar _d_run_main2
    5 foobar _d_run_main
    6 foobar main
    7 libdyld.dylib start

-----------------------------------------------
exit, total: 1, system: 1, app: 0

-----------------------------------------------
getrlimit, total: 1, system: 0, app: 1

    8 libsystem_c.dylib fwrite
    9 foobar 
_D3std5stdio__T13trustedFwriteTaZQsFNbNiNePOS4core4stdcQBx7__sFILExAaZm 
~/.dvm/compilers/dmd-2.088.0/osx/bin/../../src/phobos/std/stdio.d:4322
   10 foobar _D3std5stdio4File17LockingTextWriter__T3putTAyaZQjMFNfMQlZv 
~/.dvm/compilers/dmd-2.088.0/osx/bin/../../src/phobos/std/stdio.d:2930
   11 foobar _D3std5stdio__T7writelnTAyaZQnFNfQjZv 
~/.dvm/compilers/dmd-2.088.0/osx/bin/../../src/phobos/std/stdio.d:3855
   12 foobar _Dmain ~/development/d/main.d:15
   13 foobar _D2rt6dmain212_d_run_main2UAAamPUQgZiZ6runAllMFZ9__lambda1MFZv
   14 foobar _D2rt6dmain212_d_run_main2UAAamPUQgZiZ7tryExecMFMDFZvZv
   15 foobar _D2rt6dmain212_d_run_main2UAAamPUQgZiZ6runAllMFZv
   16 foobar _D2rt6dmain212_d_run_main2UAAamPUQgZiZ7tryExecMFMDFZvZv
   17 foobar _d_run_main2
   18 foobar _d_run_main
   19 foobar main
   20 libdyld.dylib start

-----------------------------------------------
mach_timebase_info, total: 1, system: 1, app: 0
sysctlbyname, total: 1, system: 1, app: 0
csrctl, total: 1, system: 1, app: 0
thread_selfid, total: 1, system: 1, app: 0




-- 
/Jacob Carlborg
Jan 24 2020
parent Jacob Carlborg <doob me.com> writes:
On 2020-01-24 21:29, Jacob Carlborg wrote:


 total: 120, system: 110, app: 10


I would like to add that the number of calls varies between runs.

-- 
/Jacob Carlborg
Jan 24 2020
prev sibling parent reply Gregor =?UTF-8?B?TcO8Y2ts?= <gregormueckl gmx.de> writes:
On Friday, 24 January 2020 at 09:52:13 UTC, Johan Engelen wrote:

 Indeed. Also to figure out why LDC's binary calls 31 more than 
 DMD's binary.
 Much appreciated if someone could repeat the test and post a 
 list of all syscalls being made.

 -Johan


Here is a run of a statically linked hello world binary on Linux:

execve("./hello", ["./hello"], 0x7fffd9d9d0e0 /* 15 vars */) = 0
arch_prctl(0x3001 /* ARCH_??? */, 0x7ffff1a05c70) = -1 EINVAL 
(Invalid argument)
brk(NULL)                               = 0x1364000
brk(0x1365340)                          = 0x1365340
arch_prctl(ARCH_SET_FS, 0x1364a00)      = 0
uname({sysname="Linux", nodename="kangoroo", ...}) = 0
set_tid_address(0x1364cd0)              = 111
set_robust_list(0x1364ce0, 24)          = 0
rt_sigaction(SIGRTMIN, {sa_handler=0x453cb0, sa_mask=[], 
sa_flags=SA_RESTORER|SA_SIGINFO, sa_restorer=0x4531b0}, NULL, 8) 
= 0
rt_sigaction(SIGRT_1, {sa_handler=0x453d50, sa_mask=[], 
sa_flags=SA_RESTORER|SA_RESTART|SA_SIGINFO, 
sa_restorer=0x4531b0}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
prlimit64(0, RLIMIT_STACK, NULL, {rlim_cur=8192*1024, 
rlim_max=8192*1024}) = 0
readlink("/proc/self/exe", "/root/hello", 4096) = 11
brk(0x1386340)                          = 0x1386340
brk(0x1387000)                          = 0x1387000
clock_getres(CLOCK_MONOTONIC, {tv_sec=0, tv_nsec=100}) = 0
clock_getres(CLOCK_BOOTTIME, {tv_sec=0, tv_nsec=100}) = 0
clock_getres(CLOCK_MONOTONIC_COARSE, {tv_sec=0, tv_nsec=100}) = 0
clock_getres(CLOCK_MONOTONIC, {tv_sec=0, tv_nsec=100}) = 0
clock_getres(CLOCK_PROCESS_CPUTIME_ID, {tv_sec=0, 
tv_nsec=15625000}) = 0
clock_getres(CLOCK_MONOTONIC_RAW, {tv_sec=0, tv_nsec=100}) = 0
clock_getres(CLOCK_THREAD_CPUTIME_ID, {tv_sec=0, 
tv_nsec=15625000}) = 0
rt_sigaction(SIGUSR1, {sa_handler=0x42ac20, sa_mask=~[RTMIN 
RT_1], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x4531b0}, 
NULL, 8) = 0
rt_sigaction(SIGUSR2, {sa_handler=0x42ad20, sa_mask=~[RTMIN 
RT_1], sa_flags=SA_RESTORER, sa_restorer=0x4531b0}, NULL, 8) = 0
openat(AT_FDCWD, "/proc/self/maps", O_RDONLY|O_CLOEXEC) = 3
prlimit64(0, RLIMIT_STACK, NULL, {rlim_cur=8192*1024, 
rlim_max=8192*1024}) = 0
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
read(3, "00400000-00561000 r-xp 00000000 "..., 4096) = 488
close(3)                                = 0
brk(0x1386000)                          = 0x1386000
sched_getaffinity(111, 32, [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 
11]) = 32
clock_getres(CLOCK_MONOTONIC, {tv_sec=0, tv_nsec=100}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=469, tv_nsec=741732400}) = 
0
rt_sigaction(SIGSEGV, {sa_handler=0x4402b0, sa_mask=~[RTMIN 
RT_1], sa_flags=SA_RESTORER|SA_RESETHAND|SA_SIGINFO, 
sa_restorer=0x4531b0}, {sa_handler=SIG_DFL, sa_mask=[], 
sa_flags=SA_RESTORER, sa_restorer=0x7fa7e7f94fb0}, 8) = 0
rt_sigaction(SIGBUS, {sa_handler=0x4402b0, sa_mask=~[RTMIN RT_1], 
sa_flags=SA_RESTORER|SA_RESETHAND|SA_SIGINFO, 
sa_restorer=0x4531b0}, {sa_handler=SIG_DFL, sa_mask=[], 
sa_flags=SA_RESTORER, sa_restorer=0x7fa7e7f94fb0}, 8) = 0
rt_sigaction(SIGSEGV, {sa_handler=SIG_DFL, sa_mask=[], 
sa_flags=SA_RESTORER, sa_restorer=0x4531b0}, NULL, 8) = 0
rt_sigaction(SIGBUS, {sa_handler=SIG_DFL, sa_mask=[], 
sa_flags=SA_RESTORER, sa_restorer=0x4531b0}, NULL, 8) = 0
fstat(1, {st_mode=S_IFCHR|0660, st_rdev=makedev(0x4, 0x1), ...}) 
= 0
ioctl(1, TCGETS, {B38400 opost isig icanon echo ...}) = 0
write(1, "hello world\n", 12)           = 12
write(1, "\n", 1)                       = 1
exit_group(0)                           = ?
+++ exited with 0 +++

I linked it statically to exclude the huge number of system calls 
made by ld.so. There's little that could be done to improve on 
those, I think.

If I let strace dump stack traces for each call, the first 
mention of druntime is after the two consecutive brk calls. But 
I'm not sure if the traces are trustworthy. They seem cut short 
in a couple of instances.

So there's a couple of calls that should be fairly fast 
(clock_getres etc.). Why does the runtime need to read 
/proc/self/maps, though?
Jan 24 2020
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/24/20 5:20 PM, Gregor Mückl wrote:


 So there's a couple of calls that should be fairly fast (clock_getres 
 etc.). Why does the runtime need to read /proc/self/maps, though?


This is great stuff, thanks!

Note that /proc/self/maps appears nowhere in phobos or druntime. So I'm 
assuming that's some other function that's doing it (maybe in libc?)

The clock_getres calls are so you can use core.time.MonoTime. rt_init 
initializes those so they can be used early in the process. I would 
assume those are really fast, as they are constants in the kernel.

-Steve
Jan 25 2020
parent reply norm <norm.rowtree gmail.com> writes:
On Saturday, 25 January 2020 at 16:07:24 UTC, Steven 
Schveighoffer wrote:

 On 1/24/20 5:20 PM, Gregor Mückl wrote:


 So there's a couple of calls that should be fairly fast 
 (clock_getres etc.). Why does the runtime need to read 
 /proc/self/maps, though?


 This is great stuff, thanks!

 Note that /proc/self/maps appears nowhere in phobos or 
 druntime. So I'm assuming that's some other function that's 
 doing it (maybe in libc?)

 The clock_getres calls are so you can use core.time.MonoTime. 
 rt_init initializes those so they can be used early in the 
 process. I would assume those are really fast, as they are 
 constants in the kernel.

 -Steve


I was curious so I ran a quick strace hello world experiment 
using only printf, not writeln, compiled with D (DMD), C++ and C 
(gcc and clang). Only the D binary opens /proc/self/maps. Running 
`strace dmd --version` also opens /proc/self/maps, but I guess 
that makes sense since the compiler itself is now written in D.

===
// D hello world
import core.stdc.stdio;

int main() {
   printf("Hello world\n");
   return 0;
}

// CC and C hello world
#include <stdio.h>

int main() {
   printf("Hello world\n");
   return 0;
}
===


Cheers,
Norm
Jan 25 2020
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/25/20 6:41 PM, norm wrote:

 On Saturday, 25 January 2020 at 16:07:24 UTC, Steven Schveighoffer wrote:

 On 1/24/20 5:20 PM, Gregor Mückl wrote:


 So there's a couple of calls that should be fairly fast (clock_getres 
 etc.). Why does the runtime need to read /proc/self/maps, though?


 This is great stuff, thanks!

 Note that /proc/self/maps appears nowhere in phobos or druntime. So 
 I'm assuming that's some other function that's doing it (maybe in libc?)

 The clock_getres calls are so you can use core.time.MonoTime. rt_init 
 initializes those so they can be used early in the process. I would 
 assume those are really fast, as they are constants in the kernel.


 
 I was curious so I ran a quick strace hello world experiment using only 
 printf, not writeln, compiled with D (DMD), C++ and C (gcc and clang). 
 Only the D binary opens /proc/self/maps. Running `strace dmd --version` 
 also opens /proc/self/maps, but I guess that makes sense since the 
 compiler itself is now written in D.
 


Yeah, it's not being opened directly by druntime, but looks like 
pthread_getattr_np (man strace -k is useful!):

openat(AT_FDCWD, "/proc/self/maps", O_RDONLY|O_CLOEXEC) = 3
  > /lib/x86_64-linux-gnu/libc-2.27.so(__open_nocancel+0x41) [0x10fdb1]
  > /lib/x86_64-linux-gnu/libc-2.27.so(_IO_file_fopen+0x78d) [0x8cc3d]
  > /lib/x86_64-linux-gnu/libc-2.27.so(fopen+0x7a) [0x7eeaa]
  > /lib/x86_64-linux-gnu/libpthread-2.27.so(pthread_getattr_np+0x193) 
[0x95b3]
  > /mnt/hgfs/Documents/testd/teststrace(thread_init+0x250) [0x50d98]
  > /mnt/hgfs/Documents/testd/teststrace(rt_init+0x4c) [0x3f880]
  > 
/mnt/hgfs/Documents/testd/teststrace(_D2rt6dmain212_d_run_main2UAAamPUQg
iZ6runAllMFZv+0x14) 
[0x3c55c]
  > 
/mnt/hgfs/Documents/testd/teststrace(_D2rt6dmain212_d_run_main2UAAamPUQgZiZ7tr
ExecMFMDFZvZv+0x21) 
[0x3c4f9]
  > /mnt/hgfs/Documents/testd/teststrace(_d_run_main2+0x22e) [0x3c462]
  > /mnt/hgfs/Documents/testd/teststrace(_d_run_main+0xbe) [0x3c21e]
  > /mnt/hgfs/Documents/testd/teststrace(main+0x22) [0x3c136]
  > /lib/x86_64-linux-gnu/libc-2.27.so(__libc_start_main+0xe7) [0x21b97]
  > /mnt/hgfs/Documents/testd/teststrace(_start+0x2a) [0x3c01a]


So it's something to do with thread_init calling pthread_getattr_np. I 
don't see a direct call in there, so probably it's inlined or tail-calling.

And is that really using fopen? my goodness..

I'm not sure we can pay-as-you-go the low-level thread support. And we 
can't do anything about how pthreads use the OS to implement their 
mechanics.

-Steve
Jan 25 2020
parent reply Gregor =?UTF-8?B?TcO8Y2ts?= <gregormueckl gmx.de> writes:
On Sunday, 26 January 2020 at 00:18:49 UTC, Steven Schveighoffer 
wrote:

 On 1/25/20 6:41 PM, norm wrote:

 On Saturday, 25 January 2020 at 16:07:24 UTC, Steven 
 Schveighoffer wrote:

 On 1/24/20 5:20 PM, Gregor Mückl wrote:


 So there's a couple of calls that should be fairly fast 
 (clock_getres etc.). Why does the runtime need to read 
 /proc/self/maps, though?


 This is great stuff, thanks!

 Note that /proc/self/maps appears nowhere in phobos or 
 druntime. So I'm assuming that's some other function that's 
 doing it (maybe in libc?)

 The clock_getres calls are so you can use core.time.MonoTime. 
 rt_init initializes those so they can be used early in the 
 process. I would assume those are really fast, as they are 
 constants in the kernel.


 
 I was curious so I ran a quick strace hello world experiment 
 using only printf, not writeln, compiled with D (DMD), C++ and 
 C (gcc and clang). Only the D binary opens /proc/self/maps. 
 Running `strace dmd --version` also opens /proc/self/maps, but 
 I guess that makes sense since the compiler itself is now 
 written in D.
 


 Yeah, it's not being opened directly by druntime, but looks 
 like pthread_getattr_np (man strace -k is useful!):

 openat(AT_FDCWD, "/proc/self/maps", O_RDONLY|O_CLOEXEC) = 3
  > /lib/x86_64-linux-gnu/libc-2.27.so(__open_nocancel+0x41) 
 [0x10fdb1]
  > /lib/x86_64-linux-gnu/libc-2.27.so(_IO_file_fopen+0x78d) 
 [0x8cc3d]
  > /lib/x86_64-linux-gnu/libc-2.27.so(fopen+0x7a) [0x7eeaa]
  > 
 /lib/x86_64-linux-gnu/libpthread-2.27.so(pthread_getattr_np+0x193) [0x95b3]
  > /mnt/hgfs/Documents/testd/teststrace(thread_init+0x250) 
 [0x50d98]
  > /mnt/hgfs/Documents/testd/teststrace(rt_init+0x4c) [0x3f880]
  > 
 /mnt/hgfs/Documents/testd/teststrace(_D2rt6dmain212_d_run_main2UAAamPUQg
iZ6runAllMFZv+0x14) [0x3c55c]
  > 
 /mnt/hgfs/Documents/testd/teststrace(_D2rt6dmain212_d_run_main2UAAamPUQgZiZ7tr
ExecMFMDFZvZv+0x21) [0x3c4f9]
  > /mnt/hgfs/Documents/testd/teststrace(_d_run_main2+0x22e) 
 [0x3c462]
  > /mnt/hgfs/Documents/testd/teststrace(_d_run_main+0xbe) 
 [0x3c21e]
  > /mnt/hgfs/Documents/testd/teststrace(main+0x22) [0x3c136]
  > /lib/x86_64-linux-gnu/libc-2.27.so(__libc_start_main+0xe7) 
 [0x21b97]
  > /mnt/hgfs/Documents/testd/teststrace(_start+0x2a) [0x3c01a]


 So it's something to do with thread_init calling 
 pthread_getattr_np. I don't see a direct call in there, so 
 probably it's inlined or tail-calling.

 And is that really using fopen? my goodness..

 I'm not sure we can pay-as-you-go the low-level thread support. 
 And we can't do anything about how pthreads use the OS to 
 implement their mechanics.

 -Steve


You got some actually useful backtraces. Great! Looks like it is 
this implementation or very close to it:

https://code.woboq.org/userspace/glibc/nptl/pthread_getattr_np.c.html

So this opens the file if it doesn't have stack layout 
information readily available. If I read the code correctly, this 
information read form the file isn't even cached for the specific 
thread. So unless I'm missing something (and I hope I do...), 
pthread_getattr_np goes through that dance each and every time. 
Something about this feels odd.

I found two other references to /proc/self/maps, one of them as 
part of vfprintf when _FORTIFY_SOURCE is enabled (then the format 
string *must* be in read only page) and the other one in fatal 
error handling. Those shouldn't concern us.
Jan 25 2020
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/25/20 7:39 PM, Gregor Mückl wrote:

 If I read the code correctly, this information read form the file isn't 
 even cached for the specific thread. So unless I'm missing something 
 (and I hope I do...), pthread_getattr_np goes through that dance each 
 and every time.


If you read the notes, it looks like this is avoided for everything but 
the first thread. But I doubt we are going to be calling 
pthread_getattr_np more than once per thread. So probably this gets 
called only once per process.

Still, to use fopen for this seems really heavy. I doubt the maps file 
isn't so large you can't fit it in one page, just read once and get the 
data you need.

But it doesn't truly concern me to the point of worrying about it for 
D's sake.

-Steve
Jan 25 2020
prev sibling parent reply Laurent =?UTF-8?B?VHLDqWd1aWVy?= <laurent.treguier.sink gmail.com> writes:
On Wednesday, 22 January 2020 at 23:51:23 UTC, FogD wrote:

 On Wednesday, 22 January 2020 at 10:51:37 UTC, IGotD- wrote:

 Many languages suffer from the C lib dependency which is kind 
 of suboptimal. It is time to depreciate that dependency.


 A recent comparison of languages from this perspective.

 https://drewdevault.com/2020/01/04/Slow.html


I don't understand these numbers. When compiling on Linux x86_64 
myself with DMD and LDC, I get a 900k executable and a 692k 
executable, respectively. I also get different results with 
GCC/Glibc (dynamic: 7.7k, static: 761k), or with the assembly 
itself (760 bytes).
It's like all the sizes in the blog post are bigger somehow. How 
am I getting results so wildly different from the post ? I'm a 
bit puzzled
Jan 23 2020
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On Thursday, 23 January 2020 at 10:01:07 UTC, Laurent Tréguier 
wrote:

 On Wednesday, 22 January 2020 at 23:51:23 UTC, FogD wrote:

 On Wednesday, 22 January 2020 at 10:51:37 UTC, IGotD- wrote:

 Many languages suffer from the C lib dependency which is kind 
 of suboptimal. It is time to depreciate that dependency.


 A recent comparison of languages from this perspective.

 https://drewdevault.com/2020/01/04/Slow.html


 I don't understand these numbers. When compiling on Linux 
 x86_64 myself with DMD and LDC, I get a 900k executable and a 
 692k executable, respectively. I also get different results 
 with GCC/Glibc (dynamic: 7.7k, static: 761k), or with the 
 assembly itself (760 bytes).
 It's like all the sizes in the blog post are bigger somehow. 
 How am I getting results so wildly different from the post ? 
 I'm a bit puzzled


Did you include the size of the C standard library and other 
related libraries?

"The size of all files which must be present at runtime 
(interpreters, stdlib, libraries, loader, etc) are included".

--
/Jacob Carlborg
Jan 23 2020
parent Laurent =?UTF-8?B?VHLDqWd1aWVy?= <laurent.treguier.sink gmail.com> writes:
On Thursday, 23 January 2020 at 12:37:22 UTC, Jacob Carlborg 
wrote:

 Did you include the size of the C standard library and other 
 related libraries?

 "The size of all files which must be present at runtime 
 (interpreters, stdlib, libraries, loader, etc) are included".

 --
 /Jacob Carlborg


Ah, yes. I still have to learn to read things before writing...
Jan 23 2020
prev sibling parent kinke <noone nowhere.com> writes:
On Thursday, 23 January 2020 at 10:01:07 UTC, Laurent Tréguier 
wrote:

 I don't understand these numbers. When compiling on Linux 
 x86_64 myself with DMD and LDC, I get a 900k executable and a 
 692k executable, respectively. I also get different results 
 with GCC/Glibc (dynamic: 7.7k, static: 761k), or with the 
 assembly itself (760 bytes).
 It's like all the sizes in the blog post are bigger somehow. 
 How am I getting results so wildly different from the post ? 
 I'm a bit puzzled


I cannot reproduce them either. My sizes with official LDC 
v1.18.0 (same version as used by the author), on Ubuntu 18.04 x64 
(author: Arch):

Phobos writeln variant:
ldc2 -O:         990K (in the bog: 10305 KiB)
ldc2 -O -static: 3.0M (incl. glibc)

C puts variant:
ldc2 -O:         478K
ldc2 -O -static: 2.3M
Jan 23 2020
prev sibling parent user1234 <user1234 1234.de> writes:
On Wednesday, 22 January 2020 at 10:51:37 UTC, IGotD- wrote:

 [...]
 Many languages suffer from the C lib dependency which is kind 
 of suboptimal. It is time to depreciate that dependency.


make your syscall library in asm and handle optimally everything 
from there.
Jan 22 2020