www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Warning, ABI breakage from 2.074 to 2.075

reply Basile B. <b2.temp gmx.com> writes:
Static libraries that are
- compiled with dmd 2.074 (maybe previous versions too)
- call format() in their API

will be responsible for strange errors when used by programs 
compiled with dmd 2.075. People will see their software raising a 
FormatException (orphan argument) for no reason. When the static 
library will be recompiled with dmd 2.075 the errors will 
disappear.

I haven't investigated much since there's been a non trivial 
amount of change in std.format() during the this spring. I don't 
know what to do with this facts except to report them here.
May 25 2017
parent reply Joakim <dlang joakim.fea.st> writes:
On Thursday, 25 May 2017 at 10:42:44 UTC, Basile B. wrote:

 Static libraries that are
 - compiled with dmd 2.074 (maybe previous versions too)
 - call format() in their API

 will be responsible for strange errors when used by programs 
 compiled with dmd 2.075. People will see their software raising 
 a FormatException (orphan argument) for no reason. When the 
 static library will be recompiled with dmd 2.075 the errors 
 will disappear.

 I haven't investigated much since there's been a non trivial 
 amount of change in std.format() during the this spring. I 
 don't know what to do with this facts except to report them 
 here.


Why is this unexpected?  D has never committed to ABI stability 
across compiler versions or the different D compilers.
May 25 2017
next sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Thursday, May 25, 2017 13:23:57 Joakim via Digitalmars-d wrote:

 On Thursday, 25 May 2017 at 10:42:44 UTC, Basile B. wrote:

 Static libraries that are
 - compiled with dmd 2.074 (maybe previous versions too)
 - call format() in their API

 will be responsible for strange errors when used by programs
 compiled with dmd 2.075. People will see their software raising
 a FormatException (orphan argument) for no reason. When the
 static library will be recompiled with dmd 2.075 the errors
 will disappear.

 I haven't investigated much since there's been a non trivial
 amount of change in std.format() during the this spring. I
 don't know what to do with this facts except to report them
 here.


 Why is this unexpected?  D has never committed to ABI stability
 across compiler versions or the different D compilers.


Yeah, if you're relying on ABI compatibility across releases, then you're
likely to have some weird problems. It may actually work often enough that
you can get away with it a good chunk of the time (I don't know), but we
certainly don't promised ABI compatibility, and we do break it some
percentage of the time. Heck, we frequently do stuff like templatize a
function that wasn't templatized before (generally to make it more generic),
which breaks all code that was taking it's address to use as a function
pointer (but doesn't break most uses of the function). We do try to avoid
superfluous breakage, and _most_ code should be fine from release to
release, but corner cases do get broken on a regular basis and that's even
when you recompile all your code, not even trying to use newly compiled code
with code compiled with an older compiler. The sitation is _far_ better than
it once was, but we're definitely not ABI stable, and the nature of D
actually makes it pretty hard to be.

- Jonathan M Davis
May 25 2017
parent reply Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Thursday, 25 May 2017 at 14:36:43 UTC, Jonathan M Davis wrote:

 On Thursday, May 25, 2017 13:23:57 Joakim via Digitalmars-d 
 wrote:

 On Thursday, 25 May 2017 at 10:42:44 UTC, Basile B. wrote:

 Static libraries that are
 - compiled with dmd 2.074 (maybe previous versions too)
 - call format() in their API

 will be responsible for strange errors when used by programs 
 compiled with dmd 2.075. People will see their software 
 raising a FormatException (orphan argument) for no reason. 
 When the static library will be recompiled with dmd 2.075 
 the errors will disappear.

 I haven't investigated much since there's been a non trivial 
 amount of change in std.format() during the this spring. I 
 don't know what to do with this facts except to report them 
 here.


 Why is this unexpected?  D has never committed to ABI 
 stability across compiler versions or the different D 
 compilers.


 Yeah, if you're relying on ABI compatibility across releases, 
 then you're likely to have some weird problems. It may actually 
 work often enough that you can get away with it a good chunk of 
 the time (I don't know), but we certainly don't promised ABI 
 compatibility, and we do break it some percentage of the time. 
 Heck, we frequently do stuff like templatize a function that 
 wasn't templatized before (generally to make it more generic), 
 which breaks all code that was taking it's address to use as a 
 function pointer (but doesn't break most uses of the function). 
 We do try to avoid superfluous breakage, and _most_ code should 
 be fine from release to release, but corner cases do get broken 
 on a regular basis and that's even when you recompile all your 
 code, not even trying to use newly compiled code with code 
 compiled with an older compiler. The sitation is _far_ better 
 than it once was, but we're definitely not ABI stable, and the 
 nature of D actually makes it pretty hard to be.


Could someone please explain why people talk always of ABI 
compatibilty while what is described would imo better classified 
as API compatibilty. ABI is the way parameter are passed to 
functions i.e. which parameters go to which register and is 
defined by the platform, while the API is the set of signature of 
a library. I find it a bit disturbing that at each release the 
ABI would change, while that is generally something which is 
(should) be extremely stable.
The only time I have been confronted with an ABI (C) change 
within a platform was in the transition from Solaris 9 to Solaris 
10 where the way structs were returned from functions changed.
TL;DR is there a confusion between ABI and API or does the 
calling conventions change at every release?
May 25 2017
next sibling parent Joakim <dlang joakim.fea.st> writes:
On Thursday, 25 May 2017 at 16:16:19 UTC, Patrick Schluter wrote:

 On Thursday, 25 May 2017 at 14:36:43 UTC, Jonathan M Davis 
 wrote:

[...]


 Could someone please explain why people talk always of ABI 
 compatibilty while what is described would imo better 
 classified as API compatibilty. ABI is the way parameter are 
 passed to functions i.e. which parameters go to which register 
 and is defined by the platform, while the API is the set of 
 signature of a library. I find it a bit disturbing that at each 
 release the ABI would change, while that is generally something 
 which is (should) be extremely stable.
 The only time I have been confronted with an ABI (C) change 
 within a platform was in the transition from Solaris 9 to 
 Solaris 10 where the way structs were returned from functions 
 changed.
 TL;DR is there a confusion between ABI and API or does the 
 calling conventions change at every release?


The calling convention is only one aspect of an ABI:

https://dlang.org/spec/abi.html

For example, David notes in that Debian thread that every time 
attributes are added to druntime functions, their mangling 
changes, ie the ABI changes.
May 25 2017
prev sibling parent Basile B. <b2.temp gmx.com> writes:
On Thursday, 25 May 2017 at 16:16:19 UTC, Patrick Schluter wrote:

 Could someone please explain why people talk always of ABI 
 compatibilty while what is described would imo better 
 classified as API compatibilty.


Here the function mangle  is involved. Mangles/Names are part of 
the ABI in D specs: 
https://dlang.org/spec/abi.html#type_mangling, although you're 
right, generally speaking the ABI is rather what you described.

What's strange is that with a different mangle, the previous 
function that stand in the static library could still be linked 
but with different results. Usually this is not the case when the 
mangle changes. That's why this breakage is particularly 
"remarkable" i find.
May 25 2017
prev sibling parent reply Jason King via Digitalmars-d <digitalmars-d puremagic.com> writes:
That=E2=80=99s a fairly important requirement if it=E2=80=99s supposed to b=
e a systems
programming language, less so for application focused stuff.  I would hope
it=E2=80=99s at least an eventual goal even if it=E2=80=99s not quite the c=
ase today.


On May 25, 2017 at 8:26:04 AM, Joakim via Digitalmars-d (
digitalmars-d puremagic.com) wrote:

On Thursday, 25 May 2017 at 10:42:44 UTC, Basile B. wrote:

 Static libraries that are
 - compiled with dmd 2.074 (maybe previous versions too)
 - call format() in their API

 will be responsible for strange errors when used by programs
 compiled with dmd 2.075. People will see their software raising
 a FormatException (orphan argument) for no reason. When the
 static library will be recompiled with dmd 2.075 the errors
 will disappear.

 I haven't investigated much since there's been a non trivial
 amount of change in std.format() during the this spring. I
 don't know what to do with this facts except to report them
 here.


Why is this unexpected? D has never committed to ABI stability
across compiler versions or the different D compilers.
May 25 2017
next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 25/05/2017 4:02 PM, Jason King via Digitalmars-d wrote:

 That’s a fairly important requirement if it’s supposed to be a systems 
 programming language, less so for application focused stuff.  I would 
 hope it’s at least an eventual goal even if it’s not quite the case today.


You would be fighting the existing companies using D who are all saying 
"break our stuff". So highly unlikely that its going to happen.
May 25 2017
prev sibling next sibling parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Thursday, 25 May 2017 at 15:02:00 UTC, Jason King wrote:

 That’s a fairly important requirement if it’s supposed to be a 
 systems programming language, less so for application focused 
 stuff.  I would hope it’s at least an eventual goal even if 
 it’s not quite the case today.


The reason we don't have ABI compatibility is the same reason 
neither Rust or Go does, it's a lot of work for a minority of 
users and it stops the language from progressing.

Maybe D will have it eventually due to pressure from large D 
using companies, but I highly doubt it.
May 25 2017
parent reply Jason King via Digitalmars-d <digitalmars-d puremagic.com> writes:
Yes it is a lot of work, which I strongly suspect is a big reason why C
still reigns supreme at the systems level =E2=80=94 because it does have a =
stable
ABI which solves a lot of headaches from a systems point of view (obviously
momentum and history are also very big reasons).

If that=E2=80=99s the direction D wants to go in, there=E2=80=99s nothing w=
rong with that,
but it needs to be setting the correct expectations for users.  Not having
a stable ABI is perfectly fine for application level stuff, but it can be
rather (in some cases extremely) problematic for systems level stuff--that
needs to be understood both by the users and the people working on D (and I
haven=E2=80=99t really seen much recognition of it).

On May 25, 2017 at 10:25:59 AM, Jack Stouffer via Digitalmars-d (
digitalmars-d puremagic.com) wrote:

On Thursday, 25 May 2017 at 15:02:00 UTC, Jason King wrote:

 That=E2=80=99s a fairly important requirement if it=E2=80=99s supposed to=


 be a

 systems programming language, less so for application focused
 stuff. I would hope it=E2=80=99s at least an eventual goal even if
 it=E2=80=99s not quite the case today.


The reason we don't have ABI compatibility is the same reason
neither Rust or Go does, it's a lot of work for a minority of
users and it stops the language from progressing.

Maybe D will have it eventually due to pressure from large D
using companies, but I highly doubt it.
May 25 2017
next sibling parent Paulo Pinto <pjmlp progtools.org> writes:
On Thursday, 25 May 2017 at 15:36:38 UTC, Jason King wrote:

 Yes it is a lot of work, which I strongly suspect is a big 
 reason why C still reigns supreme at the systems level — 
 because it does have a stable ABI which solves a lot of 
 headaches from a systems point of view (obviously momentum and 
 history are also very big reasons).


That is a common misconception.

C only has a stable ABI on operating systems written in C, 
because the C ABI is actually the OS ABI.

In operating systems not written in C, like all the mainframe 
OSes before C got widespread out of UNIX and still in use 
nowadays (IBM i, z/OS, ClearPath), real time OSes written in Ada 
and quite a few other examples, the "C ABI" only has a meaning 
inside the POSIX emulation environment.

In fact, during the 80 and 90's it was common not being able to 
link object files from different C compilers on home OSes that 
were actually mostly written in Assembly.


--
Paulo
May 25 2017
prev sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Thursday, 25 May 2017 at 15:36:38 UTC, Jason King wrote:

 Yes it is a lot of work, which I strongly suspect is a big 
 reason why C still reigns supreme at the systems level — 
 because it does have a stable ABI which solves a lot of 
 headaches from a systems point of view (obviously momentum and 
 history are also very big reasons).

 If that’s the direction D wants to go in, there’s nothing wrong 
 with that,
 but it needs to be setting the correct expectations for users.  
 Not having
 a stable ABI is perfectly fine for application level stuff, but 
 it can be
 rather (in some cases extremely) problematic for systems level 
 stuff--that
 needs to be understood both by the users and the people working 
 on D (and I
 haven’t really seen much recognition of it).

 On May 25, 2017 at 10:25:59 AM, Jack Stouffer via Digitalmars-d 
 (
 digitalmars-d puremagic.com) wrote:

 On Thursday, 25 May 2017 at 15:02:00 UTC, Jason King wrote:

 That’s a fairly important requirement if it’s supposed to be a 
 systems programming language, less so for application focused 
 stuff. I would hope it’s at least an eventual goal even if 
 it’s not quite the case today.


 The reason we don't have ABI compatibility is the same reason 
 neither Rust or Go does, it's a lot of work for a minority of 
 users and it stops the language from progressing.

 Maybe D will have it eventually due to pressure from large D 
 using companies, but I highly doubt it.


There was a long thread last month about getting dmd into Debian, 
that discussed the ABI stability issue among others:

https://forum.dlang.org/thread/hhefnnighbowonxsnbdy forum.dlang.org

ABI stability is not promised, not now or anytime soon, not just 
from D but many languages, as Jack said.  It just doesn't make 
sense.
May 25 2017
parent reply Jason King via Digitalmars-d <digitalmars-d puremagic.com> writes:
And how many of those are claiming to be a systems programming language?

I have no problems with an unstable ABI, what I have a problem is with
claiming to be a systems programming language AND not having a stable ABI.
You realistically cannot have both =E2=80=94 it seems like D is trying to h=
ave it=E2=80=99s
cake and eat it too and I=E2=80=99m just pointing out that it=E2=80=99s goi=
ng to lead to
sadness.  If there are no plans to ever have a stable ABI, that=E2=80=99s f=
ine (may
even be good for the long term usage of the language), just drop the whole
systems programming language bit and focus more on application level, but
I=E2=80=99ve not really seen any recognition of that.

On May 25, 2017 at 11:41:52 AM, Joakim via Digitalmars-d (
digitalmars-d puremagic.com) wrote:

On Thursday, 25 May 2017 at 15:36:38 UTC, Jason King wrote:

 Yes it is a lot of work, which I strongly suspect is a big
 reason why C still reigns supreme at the systems level =E2=80=94
 because it does have a stable ABI which solves a lot of
 headaches from a systems point of view (obviously momentum and
 history are also very big reasons).

 If that=E2=80=99s the direction D wants to go in, there=E2=80=99s nothing=


 wrong

 with that,
 but it needs to be setting the correct expectations for users.
 Not having
 a stable ABI is perfectly fine for application level stuff, but
 it can be
 rather (in some cases extremely) problematic for systems level
 stuff--that
 needs to be understood both by the users and the people working
 on D (and I
 haven=E2=80=99t really seen much recognition of it).

 On May 25, 2017 at 10:25:59 AM, Jack Stouffer via Digitalmars-d
 (
 digitalmars-d puremagic.com) wrote:

 On Thursday, 25 May 2017 at 15:02:00 UTC, Jason King wrote:

 That=E2=80=99s a fairly important requirement if it=E2=80=99s supposed t=




o be a

 systems programming language, less so for application focused
 stuff. I would hope it=E2=80=99s at least an eventual goal even if
 it=E2=80=99s not quite the case today.


 The reason we don't have ABI compatibility is the same reason
 neither Rust or Go does, it's a lot of work for a minority of
 users and it stops the language from progressing.

 Maybe D will have it eventually due to pressure from large D
 using companies, but I highly doubt it.


There was a long thread last month about getting dmd into Debian,
that discussed the ABI stability issue among others:

https://forum.dlang.org/thread/hhefnnighbowonxsnbdy forum.dlang.org

ABI stability is not promised, not now or anytime soon, not just
from D but many languages, as Jack said. It just doesn't make
sense.
May 25 2017
next sibling parent reply Paulo Pinto <pjmlp progtools.org> writes:
On Thursday, 25 May 2017 at 17:04:10 UTC, Jason King wrote:

 And how many of those are claiming to be a systems programming 
 language?

 I have no problems with an unstable ABI, what I have a problem 
 is with
 claiming to be a systems programming language AND not having a 
 stable ABI.
 You realistically cannot have both — it seems like D is trying 
 to have it’s
 cake and eat it too and I’m just pointing out that it’s going 
 to lead to
 sadness.  If there are no plans to ever have a stable ABI, 
 that’s fine (may
 even be good for the long term usage of the language), just 
 drop the whole
 systems programming language bit and focus more on application 
 level, but
 I’ve not really seen any recognition of that.


 From that point of view, C++ and Ada aren't system programming 
languages.
May 25 2017
parent evilrat <evilrat666 gmail.com> writes:
On Thursday, 25 May 2017 at 17:10:01 UTC, Paulo Pinto wrote:

 On Thursday, 25 May 2017 at 17:04:10 UTC, Jason King wrote:

 And how many of those are claiming to be a systems programming 
 language?

 I have no problems with an unstable ABI, what I have a problem 
 is with
 claiming to be a systems programming language AND not having a 
 stable ABI.
 You realistically cannot have both — it seems like D is trying 
 to have it’s
 cake and eat it too and I’m just pointing out that it’s going 
 to lead to
 sadness.  If there are no plans to ever have a stable ABI, 
 that’s fine (may
 even be good for the long term usage of the language), just 
 drop the whole
 systems programming language bit and focus more on application 
 level, but
 I’ve not really seen any recognition of that.


 From that point of view, C++ and Ada aren't system programming 
 languages.


Exactly. Maybe I have holes in my memory, but I can't remember 
anything on "final C++ ABI standard". In fact I can even recall 
something like this - "by not doing so(ABI freeze, I mean) we are 
able to stay in touch with newer hardware"
May 25 2017
prev sibling parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 05/25/2017 01:04 PM, Jason King via Digitalmars-d wrote:

 
 I have no problems with an unstable ABI, what I have a problem is with
 claiming to be a systems programming language AND not having a stable ABI.
 You realistically cannot have both


Why?
May 25 2017
parent reply Jason King via Digitalmars-d <digitalmars-d puremagic.com> writes:
On May 26, 2017 at 12:11:09 AM, Nick Sabalausky (Abscissa) via
Digitalmars-d (digitalmars-d puremagic.com) wrote:

On 05/25/2017 01:04 PM, Jason King via Digitalmars-d wrote:

 I have no problems with an unstable ABI, what I have a problem is with
 claiming to be a systems programming language AND not having a stable ABI=


.


 You realistically cannot have both


Why?

Unless it=E2=80=99s a completely self contained system, if you are operatin=
g at the
systems level, you are going to be providing the foundations for other
people=E2=80=99s code to build on (vs. application level stuff where you=E2=
=80=99re already
at the top).  Just imagine if every time you applied Windows updates or
applied a security fix to your OS of choice you had to rebuild every piece
of software you have, and if providing it to others, provide corresponding
versions of your code with every update and fix that was released with your
OS =E2=80=94 not because anything of yours changed, but because just becaus=
e you
wanted to fix a problem with the underlying system.   Trying to build
something on top of an unstable ABI is building your foundations on sand.

All I=E2=80=99m saying is if no attention is going to be paid to this (it d=
oesn=E2=80=99t
mean you can=E2=80=99t change the ABI, but it needs to be managed it better=
 than
=E2=80=98whoops!=E2=80=99), just stop claiming the systems bit and stay up =
stack where this
isn=E2=80=99t a problem.
May 26 2017
parent Ola Fosheim Grostad <ola.fosheim.grostad gmail.com> writes:
On Friday, 26 May 2017 at 13:23:20 UTC, Jason King wrote:

 wanted to fix a problem with the underlying system.   Trying to 
 build
 something on top of an unstable ABI is building your 
 foundations on sand.

 All I’m saying is if no attention is going to be paid to this 
 (it doesn’t mean you can’t change the ABI, but it needs to be 
 managed it better than ‘whoops!’), just stop claiming the 
 systems bit and stay up stack where this isn’t a problem.


There is some truth to this as BeOS used C++, and ABI was a 
concern, but it really depends on the context. D has a too big 
runtime and too many runtime dependent features to be classified 
as a low level language anyway, though...
May 26 2017
prev sibling parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 5/25/17 11:02 AM, Jason King via Digitalmars-d wrote:

 That’s a fairly important requirement if it’s supposed to be a systems
 programming language, less so for application focused stuff.  I would
 hope it’s at least an eventual goal even if it’s not quite the case today.


Two large reasons why D is not focused on ABI compatibility:

1. Most of Phobos is templates. Template signatures can change easily 
from one version to the next, even if the template didn't change. C++ 
has the same issue.
2. D is generally statically compiled, not with shared libs. There is 
some progress on this front, but really, until the shared library 
support is mainstream, then having binary compatibility between 
releases, and therefore ABI compatibility, is not critical -- you have 
to recompile anyway.

-Steve
May 25 2017