www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - API changing, possible low lying fruit, random thought

reply Chad J <gamerChad _spamIsBad_gmail.com> writes:
Hi folks,

First, I'm shooting from the hip, so if you don't have time to sit and 
sift through these ramblings, maybe you shouldn't try.  You have been 
warned.

So I was driving to college this weekend while recalling some d ng 
conversation about memory management and how it was a "low lying fruit" 
which was plucked by writing a garbage collector, and how cool it would 
be if D could find another such "low lying fruit" to leverage on.  At 
about the same time, I was thinking about how much it freaking sucks 
that libraries and APIs in general are one-shot deals and can't have 
their interfaces rewritten at a later date without severe consequences. 
  It made me think - perhaps API interfaces are such a low lying fruit.

There are a couple problems with API interfaces:
1.)  People learn them.  There is a time investment.
2.)  Interface changes can cause code to need to be rewritten.  A LOT of 
code.

D addresses (1) a little bit.  There is the deprecated keyword.  It 
helps a little, but all it does is tell the lib user that a function, 
class, member, or whatnot is deprecated.  There is no way to tell the 
lib user what to use instead of why this thing was deprecated.  So there 
needs to be some way to deprecate old interface junk and give meaningful 
information to someone who is just now trying to compile with the new 
version of a library.  It would be impossible to infuse the user with 
all sorts of new knowledge at no cost, but when an API change should 
happen it would be nice if the user gets the information they need, when 
they need it, and as easily as possible.

It seems that D doesn't address (2).  I don't think very many languages, 
if any, address (2) at all.  But it can be handled, and perhaps using 
mostly language features that already exist in D.  It can be handled by 
creating some intuitive way for library/API writers to define changes or 
patches to their interface.  These changes would do stuff like rewrite 
writefln( "Hello World!" ); to Stdout( "Hello World!" ).newline;.  Even 
if this only handles trivial cases like rewriting function calls, it 
would catch a large portion of API changes.  It would be really nice 
though, if it could also edit the user's code, as in modify/add/remove, 
so that some levels of refactoring could be performed automatically. 
The new code (user's source tree) might be placed in a new directory or 
some such; an implementation detail.  I imagine the import mixin type 
stuff could be used to grab the user source code for analysis, and then 
you just need some way to write the result somewhere else.  As for the 
analysis itself, I'm not so sure about this.  I know it should be 
possible already, and talked about proposed features like AST macro type 
things seem to be pointed in this direction.

OK so let's assume we do the above two things.  That might still leave 
some issues.  Now we have library scripts rummaging through our code and 
changing various bits and pieces without our knowledge.  Maybe a little 
scary.  This could be handled by pragma(msg,"...")'ing some sort of 
description about changes that happened during a code update or patch. 
But it would also help to know that libraries aren't going to rewrite 
any user code until they are explicitly asked to do so, which might just 
mean versioning off of a compile switch.  Even better if, upon 
compilation, the library runs a script that checks for new versions of 
the library, and mentions to the user when those things are available. 
Perhaps even more practical would be to have the compiler keep a list of 
what versions of libraries are in use, and when a new one is tried it 
prompts to start the patching process.

Then there is a problem when refactoring requires some sort of decision 
from the library user.  This is where some compiler/language magic might 
be needed, in the form of better compile-time io.  Just some sort of 
scanf to pause compilation and retrieve user input.

All in all, I envision library users seeing a kind of installer like 
user interface whenever they try to upgrade to a new library version, 
and they have to do very little, if any, code refactoring of their own.

That's all I have for now.
- Chad
Apr 30 2007
next sibling parent Clay Smith <clayasaurus gmail.com> writes:
Chad J wrote:

 Hi folks,
 
 First, I'm shooting from the hip, so if you don't have time to sit and 
 sift through these ramblings, maybe you shouldn't try.  You have been 
 warned.
 
 So I was driving to college this weekend while recalling some d ng 
 conversation about memory management and how it was a "low lying fruit" 
 which was plucked by writing a garbage collector, and how cool it would 
 be if D could find another such "low lying fruit" to leverage on.  At 
 about the same time, I was thinking about how much it freaking sucks 
 that libraries and APIs in general are one-shot deals and can't have 
 their interfaces rewritten at a later date without severe consequences. 
  It made me think - perhaps API interfaces are such a low lying fruit.
 
 There are a couple problems with API interfaces:
 1.)  People learn them.  There is a time investment.
 2.)  Interface changes can cause code to need to be rewritten.  A LOT of 
 code.
 
 D addresses (1) a little bit.  There is the deprecated keyword.  It 
 helps a little, but all it does is tell the lib user that a function, 
 class, member, or whatnot is deprecated.  There is no way to tell the 
 lib user what to use instead of why this thing was deprecated.  So there 
 needs to be some way to deprecate old interface junk and give meaningful 
 information to someone who is just now trying to compile with the new 
 version of a library.  It would be impossible to infuse the user with 
 all sorts of new knowledge at no cost, but when an API change should 
 happen it would be nice if the user gets the information they need, when 
 they need it, and as easily as possible.
 
 It seems that D doesn't address (2).  I don't think very many languages, 
 if any, address (2) at all.  But it can be handled, and perhaps using 
 mostly language features that already exist in D.  It can be handled by 
 creating some intuitive way for library/API writers to define changes or 
 patches to their interface.  These changes would do stuff like rewrite 
 writefln( "Hello World!" ); to Stdout( "Hello World!" ).newline;.  Even 
 if this only handles trivial cases like rewriting function calls, it 
 would catch a large portion of API changes.  It would be really nice 
 though, if it could also edit the user's code, as in modify/add/remove, 
 so that some levels of refactoring could be performed automatically. The 
 new code (user's source tree) might be placed in a new directory or some 
 such; an implementation detail.  I imagine the import mixin type stuff 
 could be used to grab the user source code for analysis, and then you 
 just need some way to write the result somewhere else.  As for the 
 analysis itself, I'm not so sure about this.  I know it should be 
 possible already, and talked about proposed features like AST macro type 
 things seem to be pointed in this direction.
 
 OK so let's assume we do the above two things.  That might still leave 
 some issues.  Now we have library scripts rummaging through our code and 
 changing various bits and pieces without our knowledge.  Maybe a little 
 scary.  This could be handled by pragma(msg,"...")'ing some sort of 
 description about changes that happened during a code update or patch. 
 But it would also help to know that libraries aren't going to rewrite 
 any user code until they are explicitly asked to do so, which might just 
 mean versioning off of a compile switch.  Even better if, upon 
 compilation, the library runs a script that checks for new versions of 
 the library, and mentions to the user when those things are available. 
 Perhaps even more practical would be to have the compiler keep a list of 
 what versions of libraries are in use, and when a new one is tried it 
 prompts to start the patching process.
 
 Then there is a problem when refactoring requires some sort of decision 
 from the library user.  This is where some compiler/language magic might 
 be needed, in the form of better compile-time io.  Just some sort of 
 scanf to pause compilation and retrieve user input.
 
 All in all, I envision library users seeing a kind of installer like 
 user interface whenever they try to upgrade to a new library version, 
 and they have to do very little, if any, code refactoring of their own.
 
 That's all I have for now.
 - Chad


I agree the deprecated should take a string or something to say which 
function you should use instead.

~ Clay
Apr 30 2007
prev sibling next sibling parent reply janderson <askme me.com> writes:
Chad J wrote:

 Hi folks,
 
 First, I'm shooting from the hip, so if you don't have time to sit and 
 sift through these ramblings, maybe you shouldn't try.  You have been 
 warned.
 
 So I was driving to college this weekend while recalling some d ng 
 conversation about memory management and how it was a "low lying fruit" 
 which was plucked by writing a garbage collector, and how cool it would 
 be if D could find another such "low lying fruit" to leverage on.  At 
 about the same time, I was thinking about how much it freaking sucks 
 that libraries and APIs in general are one-shot deals and can't have 
 their interfaces rewritten at a later date without severe consequences. 
  It made me think - perhaps API interfaces are such a low lying fruit.
 
 There are a couple problems with API interfaces:
 1.)  People learn them.  There is a time investment.
 2.)  Interface changes can cause code to need to be rewritten.  A LOT of 
 code.
 
 D addresses (1) a little bit.  There is the deprecated keyword.  It 
 helps a little, but all it does is tell the lib user that a function, 
 class, member, or whatnot is deprecated.  There is no way to tell the 
 lib user what to use instead of why this thing was deprecated.  So there 
 needs to be some way to deprecate old interface junk and give meaningful 
 information to someone who is just now trying to compile with the new 
 version of a library.  It would be impossible to infuse the user with 
 all sorts of new knowledge at no cost, but when an API change should 
 happen it would be nice if the user gets the information they need, when 
 they need it, and as easily as possible.
 
 It seems that D doesn't address (2).  I don't think very many languages, 
 if any, address (2) at all.  But it can be handled, and perhaps using 
 mostly language features that already exist in D.  It can be handled by 
 creating some intuitive way for library/API writers to define changes or 
 patches to their interface.  These changes would do stuff like rewrite 
 writefln( "Hello World!" ); to Stdout( "Hello World!" ).newline;.  Even 
 if this only handles trivial cases like rewriting function calls, it 
 would catch a large portion of API changes.  It would be really nice 
 though, if it could also edit the user's code, as in modify/add/remove, 
 so that some levels of refactoring could be performed automatically. The 
 new code (user's source tree) might be placed in a new directory or some 
 such; an implementation detail.  I imagine the import mixin type stuff 
 could be used to grab the user source code for analysis, and then you 
 just need some way to write the result somewhere else.  As for the 
 analysis itself, I'm not so sure about this.  I know it should be 
 possible already, and talked about proposed features like AST macro type 
 things seem to be pointed in this direction.
 
 OK so let's assume we do the above two things.  That might still leave 
 some issues.  Now we have library scripts rummaging through our code and 
 changing various bits and pieces without our knowledge.  Maybe a little 
 scary.  This could be handled by pragma(msg,"...")'ing some sort of 
 description about changes that happened during a code update or patch. 
 But it would also help to know that libraries aren't going to rewrite 
 any user code until they are explicitly asked to do so, which might just 
 mean versioning off of a compile switch.  Even better if, upon 
 compilation, the library runs a script that checks for new versions of 
 the library, and mentions to the user when those things are available. 
 Perhaps even more practical would be to have the compiler keep a list of 
 what versions of libraries are in use, and when a new one is tried it 
 prompts to start the patching process.
 
 Then there is a problem when refactoring requires some sort of decision 
 from the library user.  This is where some compiler/language magic might 
 be needed, in the form of better compile-time io.  Just some sort of 
 scanf to pause compilation and retrieve user input.
 
 All in all, I envision library users seeing a kind of installer like 
 user interface whenever they try to upgrade to a new library version, 
 and they have to do very little, if any, code refactoring of their own.
 
 That's all I have for now.
 - Chad


I think it small doses this could be cool.  However I worry about the 
extra work placed on the library writer.  I mean, if they have to write 
code to fix their own interface in users code, how much longer will the 
user have to wait for an update of the library (or what will be 
sacrificed? Quality or features? )

It would also become increasing difficult to maintain across versions.

If it was really simple  for a library write to do and took them 
practically no time them I'd give a thumbs up.

Just taking about one aspect, I suppose the deprecated could be changed 
to something like:

deprecated("message")
{
	void foo();
}
auto_update(inout string codeinerror)
{
	//code in here
	string.replace(codeinerror, "foo()", "bar()");
}

And when D compiled (or something) it would ask at each one... "This has 
been depreciated do you want to apply this update."

I still think thats nasty and probably wouldn't be worth the effort 
unless you have one of those purposely slow moving projects (like stl or 
something).

-Joel
Apr 30 2007
parent Daniel Keep <daniel.keep.lists gmail.com> writes:
janderson wrote:

 Just taking about one aspect, I suppose the deprecated could be changed
 to something like:
 
 deprecated("message")
 {
     void foo();
 }
 auto_update(inout string codeinerror)
 {
     //code in here
     string.replace(codeinerror, "foo()", "bar()");
 }
 
 And when D compiled (or something) it would ask at each one... "This has
 been depreciated do you want to apply this update."
 
 I still think thats nasty and probably wouldn't be worth the effort
 unless you have one of those purposely slow moving projects (like stl or
 something).
 
 -Joel


That doesn't really seem like D's job, to be honest.

I think what we should do instead is to build a standardised set of code
tools for D.  Look at it like this: if there were one or two de facto
standard refactoring tools (like how bud and rebuild are the de facto
standard build tools for D), then you could fix this problem by
releasing semantic patches.

New, incompatible library release?  Release a set of scripts that use
these refactoring tools to make the changes safely and correctly.  That
way, we don't end up polluting the language with things it probably
shouldn't have.

Plus, I imagine it would be quite the selling point.

	-- Daniel

-- 
int getRandomNumber()
{
    return 4; // chosen by fair dice roll.
              // guaranteed to be random.
}

http://xkcd.com/

v2sw5+8Yhw5ln4+5pr6OFPma8u6+7Lw4Tm6+7l6+7D
i28a2Xs3MSr2e4/6+7t4TNSMb6HTOp5en5g6RAHCP  http://hackerkey.com/
Apr 30 2007
prev sibling parent gareis <dhasenan gmail.com> writes:
Your suggestion reminds me of autotools -- specifically, autoscan. But 
this would create a patch and a log of areas that may need programmer 
intervention.

It's an interesting idea, though there's still a lot of work when there 
are no simple mappings between APIs (GTK to QT, for example). Plus it's 
a lot of work, and not a tremendous amount less than would otherwise be 
required.

Chad J wrote:

 Hi folks,
 
 First, I'm shooting from the hip, so if you don't have time to sit and 
 sift through these ramblings, maybe you shouldn't try.  You have been 
 warned.
 
 So I was driving to college this weekend while recalling some d ng 
 conversation about memory management and how it was a "low lying fruit" 
 which was plucked by writing a garbage collector, and how cool it would 
 be if D could find another such "low lying fruit" to leverage on.  At 
 about the same time, I was thinking about how much it freaking sucks 
 that libraries and APIs in general are one-shot deals and can't have 
 their interfaces rewritten at a later date without severe consequences. 
  It made me think - perhaps API interfaces are such a low lying fruit.
 
 There are a couple problems with API interfaces:
 1.)  People learn them.  There is a time investment.
 2.)  Interface changes can cause code to need to be rewritten.  A LOT of 
 code.
 
 D addresses (1) a little bit.  There is the deprecated keyword.  It 
 helps a little, but all it does is tell the lib user that a function, 
 class, member, or whatnot is deprecated.  There is no way to tell the 
 lib user what to use instead of why this thing was deprecated.  So there 
 needs to be some way to deprecate old interface junk and give meaningful 
 information to someone who is just now trying to compile with the new 
 version of a library.  It would be impossible to infuse the user with 
 all sorts of new knowledge at no cost, but when an API change should 
 happen it would be nice if the user gets the information they need, when 
 they need it, and as easily as possible.
 
 It seems that D doesn't address (2).  I don't think very many languages, 
 if any, address (2) at all.  But it can be handled, and perhaps using 
 mostly language features that already exist in D.  It can be handled by 
 creating some intuitive way for library/API writers to define changes or 
 patches to their interface.  These changes would do stuff like rewrite 
 writefln( "Hello World!" ); to Stdout( "Hello World!" ).newline;.  Even 
 if this only handles trivial cases like rewriting function calls, it 
 would catch a large portion of API changes.  It would be really nice 
 though, if it could also edit the user's code, as in modify/add/remove, 
 so that some levels of refactoring could be performed automatically. The 
 new code (user's source tree) might be placed in a new directory or some 
 such; an implementation detail.  I imagine the import mixin type stuff 
 could be used to grab the user source code for analysis, and then you 
 just need some way to write the result somewhere else.  As for the 
 analysis itself, I'm not so sure about this.  I know it should be 
 possible already, and talked about proposed features like AST macro type 
 things seem to be pointed in this direction.
 
 OK so let's assume we do the above two things.  That might still leave 
 some issues.  Now we have library scripts rummaging through our code and 
 changing various bits and pieces without our knowledge.  Maybe a little 
 scary.  This could be handled by pragma(msg,"...")'ing some sort of 
 description about changes that happened during a code update or patch. 
 But it would also help to know that libraries aren't going to rewrite 
 any user code until they are explicitly asked to do so, which might just 
 mean versioning off of a compile switch.  Even better if, upon 
 compilation, the library runs a script that checks for new versions of 
 the library, and mentions to the user when those things are available. 
 Perhaps even more practical would be to have the compiler keep a list of 
 what versions of libraries are in use, and when a new one is tried it 
 prompts to start the patching process.
 
 Then there is a problem when refactoring requires some sort of decision 
 from the library user.  This is where some compiler/language magic might 
 be needed, in the form of better compile-time io.  Just some sort of 
 scanf to pause compilation and retrieve user input.
 
 All in all, I envision library users seeing a kind of installer like 
 user interface whenever they try to upgrade to a new library version, 
 and they have to do very little, if any, code refactoring of their own.
 
 That's all I have for now.
 - Chad
May 01 2007