www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.ldc - iOS progress

reply Dan Olson <zans.is.for.cans yahoo.com> writes:
Just a quick note.  I started again on LDC for iOS a month ago (new years
resolution!).  I bogged down in floating point and optimizer problems.  I think
I understand the major issues and have gotten all of phobos unittests to pass
with optimizer disabled except for some math unittests that are written for
80-bit floats and some that are off by LSB.

This weekend I'll cleanup the math fixes and put up a new repo on
https://github.com/smolt that helps glue it all together.
--
Dan
Feb 04 2015
next sibling parent reply "David Nadlinger" <code klickverbot.at> writes:
On Wednesday, 4 February 2015 at 16:37:05 UTC, Dan Olson wrote:

 This weekend I'll cleanup the math fixes and put up a new repo 
 on https://github.com/smolt that helps glue it all together.


Nice, great to hear!

David
Feb 04 2015
parent reply "Dan Olson" <zans4cans yahoo.com> writes:
On Wednesday, 4 February 2015 at 17:24:06 UTC, David Nadlinger 
wrote:

 On Wednesday, 4 February 2015 at 16:37:05 UTC, Dan Olson wrote:

 This weekend I'll cleanup the math fixes and put up a new repo 
 on https://github.com/smolt that helps glue it all together.


 Nice, great to hear!

 David


https://github.com/smolt/ldc-iphone-dev

This pulls all the pieces together as submodules.  I still need 
to cleanup and add my sample Xcode apps and add some docs, but 
this should be enough to build LDC iPhoneOS cross compiler and 
Phobos.  Someone with OS X should see if my build works for them.
Feb 14 2015
next sibling parent reply "Joakim" <dlang joakim.fea.st> writes:
On Sunday, 15 February 2015 at 03:40:41 UTC, Dan Olson wrote:

 https://github.com/smolt/ldc-iphone-dev

 This pulls all the pieces together as submodules.  I still need 
 to cleanup and add my sample Xcode apps and add some docs, but 
 this should be enough to build LDC iPhoneOS cross compiler and 
 Phobos.  Someone with OS X should see if my build works for 
 them.


Nice work.  Skimming your commits, it looks like you have a bunch 
of code to disable TLS altogether.  I guess that is just older 
work and you got TLS working now?  If so, how many tests are 
failing for you on druntime and phobos now?
Feb 15 2015
parent reply Dan Olson <zans.is.for.cans yahoo.com> writes:
"Joakim" <dlang joakim.fea.st> writes:


 On Sunday, 15 February 2015 at 03:40:41 UTC, Dan Olson wrote:

 https://github.com/smolt/ldc-iphone-dev

 This pulls all the pieces together as submodules.  I still need to
 cleanup and add my sample Xcode apps and add some docs, but this
 should be enough to build LDC iPhoneOS cross compiler and Phobos.
 Someone with OS X should see if my build works for them.


 Nice work.  Skimming your commits, it looks like you have a bunch of
 code to disable TLS altogether.  I guess that is just older work and
 you got TLS working now?  If so, how many tests are failing for you on
 druntime and phobos now?


Hi Joakim!

I added a -disable-tls option to help with bringing up D on other
targets that don't have TLS.  My thought is if you don't have threads,
say a bare-bones target, then you don't need TLS and should be
allowed to use existing D code without changing all TLS vars to
__gshared.

For iOS, TLS is working fine with the modification I made to LLVM last
spring.  Newer arm64 devices are supposed to natively support TLS, but I
don't have an arm64 device yet.  The iphonesim ironically does not
support TLS because of a check in ld to make sure TLS doesn't slip in.
Obviously it could support it.  I have ideas of how to get around the
check.

Unittests failures are not too bad.  All except one are math related.

std.csv - a couple double off-by-one LSB differences
std.internal.math.gammafunction - needs update for 64-bit reals
std.math - a real off-by-one LSB error in a few cases

Failures only with optimization on (-O1 or higher):

std.internal.math.errorfunction - erfc() NaN payload fails
std.math - acosh() not producing NaN in a couple cases
core.thread - a Fiber unittest crashes on multicore devices

I have all the failures marked in the druntime and phobos source with
versions that begin with "WIP_" to workaround the failure so rest of
test can run. Grep for "WIP_" to see all the details. It really is a
bummer that unittests use assert because it makes it difficult get
results for the rest of the unittest.
Feb 15 2015
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-02-15 20:19, Dan Olson wrote:


 For iOS, TLS is working fine with the modification I made to LLVM last
 spring.  Newer arm64 devices are supposed to natively support TLS, but I
 don't have an arm64 device yet.  The iphonesim ironically does not
 support TLS because of a check in ld to make sure TLS doesn't slip in.
 Obviously it could support it.  I have ideas of how to get around the
 check.


I had a look at the TLS commits as well. Is iOS using the 
"__tls_get_addr" symbol? On OS X "tlv_get_addr" is used.

-- 
/Jacob Carlborg
Feb 16 2015
parent reply Dan Olson <zans.is.for.cans yahoo.com> writes:
Jacob Carlborg <doob me.com> writes:


 On 2015-02-15 20:19, Dan Olson wrote:


 For iOS, TLS is working fine with the modification I made to LLVM last
 spring.  Newer arm64 devices are supposed to natively support TLS, but I
 don't have an arm64 device yet.  The iphonesim ironically does not
 support TLS because of a check in ld to make sure TLS doesn't slip in.
 Obviously it could support it.  I have ideas of how to get around the
 check.


 I had a look at the TLS commits as well. Is iOS using the
 "__tls_get_addr" symbol? On OS X "tlv_get_addr" is used.


Hi Jacob!  Yes, tlv_get_addr is used, but indirectly.  When I modified
LLVM, I had it emit a call to __tls_get_addr instead of to the thunk
which is initialized to tlv_get_addr.  It gave me a hook to work with
until I felt TLS was working, and so far it appears to be working.  My
__tls_get_addr just verifies the thunk and then calls tlv_get_addr().

https://github.com/smolt/iphoneos-apple-support/blob/master/threadLocalVariables.c#L487

It could be changed here,

https://github.com/smolt/llvm/blob/ios/lib/Target/ARM/ARMISelLowering.cpp#L2473

but I would need some help creating the correct LLVM code.
Feb 16 2015
parent reply Jacob Carlborg <doob me.com> writes:
On 2015-02-16 17:28, Dan Olson wrote:



 It could be changed here,

 https://github.com/smolt/llvm/blob/ios/lib/Target/ARM/ARMISelLowering.cpp#L2473

 but I would need some help creating the correct LLVM code.


Is calling tlv_get_addr any different from calling __tls_get_addr?

-- 
/Jacob Carlborg
Feb 16 2015
parent reply "Dan Olson" <zans4cans yahoo.com> writes:
On Monday, 16 February 2015 at 19:57:15 UTC, Jacob Carlborg wrote:

 On 2015-02-16 17:28, Dan Olson wrote:



 It could be changed here,

 https://github.com/smolt/llvm/blob/ios/lib/Target/ARM/ARMISelLowering.cpp#L2473

 but I would need some help creating the correct LLVM code.


 Is calling tlv_get_addr any different from calling 
 __tls_get_addr?


I think directly using tlv_get_addr may lead to weird errors if 
thread local descriptors have not been initialized by runtime.  
With __tls_get_addr there is a check that descriptor is 
initialized, and if I get around to changing llvm to call thunk 
directly, then uninitialized descriptor will have thunk set to 
_tlv_bootstrap and code will cleanly abort.
Feb 16 2015
parent reply Jacob Carlborg <doob me.com> writes:
On 2015-02-16 22:38, Dan Olson wrote:


 I think directly using tlv_get_addr may lead to weird errors if thread
 local descriptors have not been initialized by runtime. With
 __tls_get_addr there is a check that descriptor is initialized, and if I
 get around to changing llvm to call thunk directly, then uninitialized
 descriptor will have thunk set to _tlv_bootstrap and code will cleanly
 abort.


I see that you used a constructor to initialize everything. Won't that 
be enough to know that tlv_get_addr is correctly initialized? No D code 
will be able to run before that, as far as I understand.

On OS X tlv_initializer will be called by _dyld_initializer which is 
called by libSystem, if I understand the source code of dyld correctly. 
This doesn't happen on iOS?

-- 
/Jacob Carlborg
Feb 17 2015
parent reply Dan Olson <zans.is.for.cans yahoo.com> writes:
Jacob Carlborg <doob me.com> writes:


 On 2015-02-16 22:38, Dan Olson wrote:


 I think directly using tlv_get_addr may lead to weird errors if thread
 local descriptors have not been initialized by runtime. With
 __tls_get_addr there is a check that descriptor is initialized, and if I
 get around to changing llvm to call thunk directly, then uninitialized
 descriptor will have thunk set to _tlv_bootstrap and code will cleanly
 abort.


 I see that you used a constructor to initialize everything. Won't that
 be enough to know that tlv_get_addr is correctly initialized? No D
 code will be able to run before that, as far as I understand.


Actually I forgot that I switched to using a constructor. My original
solution had tlv_initializer called by D runtime but I decided to get
all Apple code out of druntime before I commited to git. You are
correct, descriptors should always be initialized. Only D code called in
__attribute__((constructor)) functions might run into problems, but that
would be unusual.


 On OS X tlv_initializer will be called by _dyld_initializer which is
 called by libSystem, if I understand the source code of dyld
 correctly. This doesn't happen on iOS?


iOS has a stub tlv_initializer() so nothing gets done automatically. I
had to change threadLocalVariables.c to include working tlv_initializer
for __arm__.
Feb 17 2015
parent reply Jacob Carlborg <doob me.com> writes:
On 2015-02-17 19:10, Dan Olson wrote:


 iOS has a stub tlv_initializer() so nothing gets done automatically. I
 had to change threadLocalVariables.c to include working tlv_initializer
 for __arm__.


You can't implement that stub because it's in dyld? Or is it possible to 
override?

-- 
/Jacob Carlborg
Feb 17 2015
parent Dan Olson <zans.is.for.cans yahoo.com> writes:
Jacob Carlborg <doob me.com> writes:


 On 2015-02-17 19:10, Dan Olson wrote:


 iOS has a stub tlv_initializer() so nothing gets done automatically. I
 had to change threadLocalVariables.c to include working tlv_initializer
 for __arm__.


 You can't implement that stub because it's in dyld? Or is it possible
 to override?


My thought would be that libdyld.dylib is statically linked to its
__arm__ stubbed tlv_initializer so my modified version can't override.

See

http://www.opensource.apple.com/source/dyld/dyld-353.2.1/src/dyldAPIsInLibSystem.cpp

and

http://www.opensource.apple.com/source/dyld/dyld-353.2.1/src/threadLocalVariables.c
Feb 18 2015
prev sibling parent reply "Joakim" <dlang joakim.fea.st> writes:
On Sunday, 15 February 2015 at 19:19:10 UTC, Dan Olson wrote:

 "Joakim" <dlang joakim.fea.st> writes:

 Nice work.  Skimming your commits, it looks like you have a 
 bunch of
 code to disable TLS altogether.  I guess that is just older 
 work and
 you got TLS working now?  If so, how many tests are failing 
 for you on
 druntime and phobos now?


 Hi Joakim!


Hey, good to see you're back working on this.


 I added a -disable-tls option to help with bringing up D on 
 other
 targets that don't have TLS.  My thought is if you don't have 
 threads,
 say a bare-bones target, then you don't need TLS and should be
 allowed to use existing D code without changing all TLS vars to
 __gshared.


Yes, that would certainly help when bringing D up on new targets.


 Unittests failures are not too bad.  All except one are math 
 related.

 std.csv - a couple double off-by-one LSB differences
 std.internal.math.gammafunction - needs update for 64-bit reals
 std.math - a real off-by-one LSB error in a few cases

 Failures only with optimization on (-O1 or higher):

 std.internal.math.errorfunction - erfc() NaN payload fails
 std.math - acosh() not producing NaN in a couple cases
 core.thread - a Fiber unittest crashes on multicore devices

 I have all the failures marked in the druntime and phobos 
 source with
 versions that begin with "WIP_" to workaround the failure so 
 rest of
 test can run. Grep for "WIP_" to see all the details.


I couldn't get most of the phobos unit tests to run on a 
linux/ARM hard-float board, Cubieboard2.  I'm guessing the issue 
is hard-float compatibility: I need to try softfp out and make 
sure.  Almost all the druntime tests passed on the armhf system 
though.


 It really is a
 bummer that unittests use assert because it makes it difficult 
 get results for the rest of the unittest.


Yes, Johannes had a PR last year that made tests continue 
running, among other changes, but he gave up on trying to get it 
merged:

https://github.com/D-Programming-Language/dmd/pull/3518
Feb 17 2015
parent reply Dan Olson <zans.is.for.cans yahoo.com> writes:
"Joakim" <dlang joakim.fea.st> writes:

 I couldn't get most of the phobos unit tests to run on a linux/ARM
 hard-float board, Cubieboard2.  I'm guessing the issue is hard-float
 compatibility: I need to try softfp out and make sure.  Almost all the
 druntime tests passed on the armhf system though.


What kinds of errors are you getting?  Are they all float related?  If
the linux C libs are using -float-abi=hard, then softfp probably won't
help. 

If you do decide to look at softfp, I have this commit to get the D
versions correct:

https://github.com/smolt/ldc/commit/5a19080

Without that change, -float-abi=softfp sets version ARM_Soft instead of
ARM_SoftFP so std.math ends up incorrectly taking non-FPU paths.


 It really is a
 bummer that unittests use assert because it makes it difficult get
 results for the rest of the unittest.


 Yes, Johannes had a PR last year that made tests continue running,
 among other changes, but he gave up on trying to get it merged:

 https://github.com/D-Programming-Language/dmd/pull/3518


Looks useful.
Feb 17 2015
parent "Joakim" <dlang joakim.fea.st> writes:
On Wednesday, 18 February 2015 at 07:10:18 UTC, Dan Olson wrote:

 "Joakim" <dlang joakim.fea.st> writes:

 I couldn't get most of the phobos unit tests to run on a 
 linux/ARM
 hard-float board, Cubieboard2.  I'm guessing the issue is 
 hard-float
 compatibility: I need to try softfp out and make sure.  Almost 
 all the
 druntime tests passed on the armhf system though.


 What kinds of errors are you getting?  Are they all float 
 related?  If
 the linux C libs are using -float-abi=hard, then softfp 
 probably won't
 help.


It's mostly segfaults, several modules segfault when their tests 
are run.  I'm guessing this is because of some ABI issue.  I 
believe most Android/ARM systems are softfp, so testing on 
linux/ARM-hardfp is probably worthless anyway.  I need to try 
softfp on a softfp system and see what happens.


 If you do decide to look at softfp, I have this commit to get 
 the D
 versions correct:

 https://github.com/smolt/ldc/commit/5a19080

 Without that change, -float-abi=softfp sets version ARM_Soft 
 instead of
 ARM_SoftFP so std.math ends up incorrectly taking non-FPU paths.


OK, will look at that when I get the tests not to segfault. :)
Feb 24 2015
prev sibling next sibling parent "Szymon Gatner" <noemail gmail.com> writes:
On Sunday, 15 February 2015 at 03:40:41 UTC, Dan Olson wrote:

 On Wednesday, 4 February 2015 at 17:24:06 UTC, David Nadlinger 
 wrote:

 On Wednesday, 4 February 2015 at 16:37:05 UTC, Dan Olson wrote:

 This weekend I'll cleanup the math fixes and put up a new 
 repo on https://github.com/smolt that helps glue it all 
 together.


 Nice, great to hear!

 David


 https://github.com/smolt/ldc-iphone-dev

 This pulls all the pieces together as submodules.  I still need 
 to cleanup and add my sample Xcode apps and add some docs, but 
 this should be enough to build LDC iPhoneOS cross compiler and 
 Phobos.  Someone with OS X should see if my build works for 
 them.


Fantastic news! When I am back from holidays next week I will 
definitely try it.
Feb 16 2015
prev sibling parent Dan Olson <zans.is.for.cans yahoo.com> writes:
"Dan Olson" <zans4cans yahoo.com> writes:

 https://github.com/smolt/ldc-iphone-dev

 This pulls all the pieces together as submodules.  I still need to
 cleanup and add my sample Xcode apps and add some docs, but this
 should be enough to build LDC iPhoneOS cross compiler and Phobos.
 Someone with OS X should see if my build works for them.


I added two Xcode projects to https://github.com/smolt/ldc-iphone-dev.

- helloD to simple demos of D on iOS device
- unittester to run druntime/phobos unittests

I also add more words to the repo README that I hope better explains
things.
--
Dan
Feb 26 2015
prev sibling next sibling parent reply "Laeeth Isharc" <nospamlaeeth nospam.laeeth.com> writes:
On Wednesday, 4 February 2015 at 16:37:05 UTC, Dan Olson wrote:

 Just a quick note.  I started again on LDC for iOS a month ago 
 (new years resolution!).  I bogged down in floating point and 
 optimizer problems.  I think I understand the major issues and 
 have gotten all of phobos unittests to pass with optimizer 
 disabled except for some math unittests that are written for 
 80-bit floats and some that are off by LSB.

 This weekend I'll cleanup the math fixes and put up a new repo 
 on https://github.com/smolt that helps glue it all together.
 --
 Dan


This is great news.  I realize the magnitude of the
accomplishment just in getting this to work, but if/when you have
time would love to get a sense for what can and can't be done as
far as accessing the iOS SDK/Cocoa given that the Objective C
support is not yet complete.

https://michelf.ca/projects/d-objc/

What is it that makes this iphone specific, and what plans are
there to enable ipad-adapted apps?

https://github.com/smolt/ldc-iphone-dev

Sorry for the basic questions - I have no experience of iOS
development, but I figured the answer might be of interest to
others.
Feb 17 2015
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2015-02-17 21:53, Laeeth Isharc wrote:


 This is great news.  I realize the magnitude of the
 accomplishment just in getting this to work, but if/when you have
 time would love to get a sense for what can and can't be done as
 far as accessing the iOS SDK/Cocoa given that the Objective C
 support is not yet complete.

 https://michelf.ca/projects/d-objc/


It's possible to access the complete Objective-C language from its 
runtime, which has a C API, meaning it can be accessed from D. It's 
quite cumbersome and verbose to use but can be used as a start to test.

The better alternative is, as you wrote above, implement language 
support for linking with Objective-C. The latest code and pull request 
for DMD is available here [1]. The pull request is the first step and 
only implements the bare minimum to call an Objective-C instance method.

Hopefully the code should work for iOS as well, the only thing that I 
know of that needs some adjustment is selecting the correct 
objc_msgSend_* function which is different for each architecture. When 
this pull request has been accepted and later implemented in LDC, it 
should be enough as well to start testing.

DMD and LDC share the same front end but the glue code is specific for 
each compiler and needs to be adopted for LDC.

[1] https://github.com/D-Programming-Language/dmd/pull/4321

-- 
/Jacob Carlborg
Feb 17 2015
prev sibling parent reply Dan Olson <zans.is.for.cans yahoo.com> writes:
"Laeeth Isharc" <nospamlaeeth nospam.laeeth.com> writes:

 This is great news.  I realize the magnitude of the
 accomplishment just in getting this to work, but if/when you have
 time would love to get a sense for what can and can't be done as
 far as accessing the iOS SDK/Cocoa given that the Objective C
 support is not yet complete.

 https://michelf.ca/projects/d-objc/


I can't wait for that change to get in dmd. I believe Jacob is pushing
on it. I have thought of putting that change directly into this iOS
branch, but that might make it difficult when I start making pull
requests to get the iOS support into LDC and DMD.

There are some D ObjC bridges out there that might work, I think Jacob
wrote one. Then there was this post by Yuriy:

http://forum.dlang.org/thread/m2d2h15ao3.fsf comcast.net?page=2#post-ftzfashwfgpiverwkyeo:40forum.dlang.org

I have not used any myself. I have my own hackish objc wrappers I use.

Probably today the best use of this iOS LDC toolchain is to compile
existing D libraries and do Cocoa calls with objc or swift.


 What is it that makes this iphone specific, and what plans are
 there to enable ipad-adapted apps?

 https://github.com/smolt/ldc-iphone-dev


It works same for iPad and iPhone. I actually do most of my testing on
an iPad-mini with a cortex-a9.


 Sorry for the basic questions - I have no experience of iOS
 development, but I figured the answer might be of interest to
 others.


Yeah there is a lot more I need to document.  It is coming.  I also have
more sample apps to add to https://github.com/smolt/ldc-iphone-dev.
-- 
Dan
Feb 17 2015
parent reply Jacob Carlborg <doob me.com> writes:
On 2015-02-18 08:49, Dan Olson wrote:


 I can't wait for that change to get in dmd. I believe Jacob is pushing
 on it.


Yes, see my post just before this one [1]

[1] http://forum.dlang.org/post/mc1ft5$9m6$1 digitalmars.com

-- 
/Jacob Carlborg
Feb 18 2015
parent "Laeeth Isharc" <laeethnospam nospamlaeeth.com> writes:
Thank you for the help, Jacob and Dan - I appreciate it.
Feb 18 2015
prev sibling parent reply "Kai Nacke" <kai redstar.de> writes:
Hi Dan!

On Wednesday, 4 February 2015 at 16:37:05 UTC, Dan Olson wrote:

 Just a quick note.  I started again on LDC for iOS a month ago 
 (new years resolution!).  I bogged down in floating point and 
 optimizer problems.  I think I understand the major issues and 
 have gotten all of phobos unittests to pass with optimizer 
 disabled except for some math unittests that are written for 
 80-bit floats and some that are off by LSB.

 This weekend I'll cleanup the math fixes and put up a new repo 
 on https://github.com/smolt that helps glue it all together.
 --
 Dan


I looked at your sources. I think you should turn the softfp 
changes and the iOS-ABI into pull requests for ldc.

Regards,
Kai
Feb 21 2015
parent Dan Olson <zans.is.for.cans yahoo.com> writes:
"Kai Nacke" <kai redstar.de> writes:


 I looked at your sources. I think you should turn the softfp changes
 and the iOS-ABI into pull requests for ldc.

 Regards,
 Kai


The softfp change ok, the iOS abi, not yet.  There is at least a C
calling convention problem to fix first.  A small struct (e.g. struct A
{int x;}) should be returned in r0 instead of as an sret.

I still owe you the longdouble changes I needed to get it to compile in
my env.  I am not using it in my ios branch right now, but I plan to
swap to it now that I have a unittest pass/fail baseline.
--
Dan
Feb 21 2015