www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.ldc - Automated testing on iOS

reply Jacob Carlborg <doob me.com> writes:
So far I've been testing on iOS by using a Xcode skeleton 
project. Since Xcode cannot build D code I've been building the D 
code outside of Xcode and inserted the binary into the Xcode 
project. Then I run the binary on the device through Xcode.

This is not really a good approach for automated testing and CI 
pipelines. There's a command line tool for building Xcode 
projects, which will help with automation. The problem is that 
this tool doesn't allow to run a project. What seems to be the 
only way to run code on a device using the command line is to run 
a test. To run a test it seems that one needs to use the XCTest 
framework that comes with Xcode.

I'm thinking that easiest way to do this is to generate/have a 
Xcode project with a single XCTest test implemented in 
Objective-C which initializes druntime and calls [1] (for 
druntime) to run the tests.

But I'm not sure how to best integrate this with the existing 
build/test infrastructure. I'm not sure if the project needs to 
be generated or if it's static enough to be committed to the 
repository. If needs to be generated if would be nice if 
`ldc-build-runtime --testrunners` generated it.

Or perhaps this is possible to do with CMake?

[1] 
https://github.com/dlang/druntime/blob/ec1b76acb68d460eaa66103b6cc7774f231c2d7f/src/test_runner.d#L11

--
/Jacob Carlborg
Jan 21 2020
parent reply Dan Olson <gorox comcast.net> writes:
Jacob Carlborg <doob me.com> writes:


 So far I've been testing on iOS by using a Xcode skeleton
 project. Since Xcode cannot build D code I've been building the D code
 outside of Xcode and inserted the binary into the Xcode project. Then
 I run the binary on the device through Xcode.

 This is not really a good approach for automated testing and CI
 pipelines. There's a command line tool for building Xcode projects,
 which will help with automation. The problem is that this tool doesn't
 allow to run a project. What seems to be the only way to run code on a
 device using the command line is to run a test. To run a test it seems
 that one needs to use the XCTest framework that comes with Xcode.

 I'm thinking that easiest way to do this is to generate/have a Xcode
 project with a single XCTest test implemented in Objective-C which
 initializes druntime and calls [1] (for druntime) to run the tests.

 But I'm not sure how to best integrate this with the existing
 build/test infrastructure. I'm not sure if the project needs to be
 generated or if it's static enough to be committed to the
 repository. If needs to be generated if would be nice if
 `ldc-build-runtime --testrunners` generated it.

 Or perhaps this is possible to do with CMake?

 [1]
 https://github.com/dlang/druntime/blob/ec1b76acb68d460eaa66103b6cc7774f231c2d7f/src/test_runner.d#L11

 --
 /Jacob Carlborg


Hi Jacob, nice progress on getting iOS support into ldc!  I peek at the
forums every now and then. Your approach of using an xcode test was how
I ran from the command line on an iOS device.  The idea to generate an
Xcode project with test seems good to me.

I assume you know about setupEnvAndRunTests() in

https://github.com/smolt/ldc-iphone-dev/blob/master/unittester/unittest.d

Is that still needed to pass all druntime/phobos unittests?
-- 
Dan
Feb 08 2020
parent Jacob Carlborg <doob me.com> writes:
On 2020-02-08 20:54, Dan Olson wrote:


 Hi Jacob, nice progress on getting iOS support into ldc!  I peek at the
 forums every now and then. Your approach of using an xcode test was how
 I ran from the command line on an iOS device.  The idea to generate an
 Xcode project with test seems good to me.
 
 I assume you know about setupEnvAndRunTests() in
 
 https://github.com/smolt/ldc-iphone-dev/blob/master/unittester/unittest.d


No, I haven't seen that.


 Is that still needed to pass all druntime/phobos unittests?


The unit tests for druntime pass without that. For the manual tests, 
I've been using the binaries that are built using `ldc-build-runtime 
--testrunners`.

For automated testing I have an Xcode project which contains a single 
XCTest implemented in Objective-C. This is linked with `test_runner.o` 
and `libdruntime-ldc-unittest.a` which are generated by the above 
command. The test calls `rt_init`, then `runModuleUnitTests` and asserts 
that the return value of `runModuleUnitTests` is whatever it should be. 
Then it calls `rt_term`.

The unit tests for Phobos can be run in the same way. Currently not all 
of them pass. Most of the failures are in allocator and math related 
modules. I suspect the allocator tests that fail might be due to a 
different page size, but I'm not sure. For the math related tests I 
haven't investigated that yet.

I see that `setupEnvAndRunTests` changes the directory to the temp 
directory and then configures the FPU. I've modified the tests that 
write files (those that I have found so far) to write to the temp 
directory instead. Configuring the FPU might help with the math related 
tests. I'll take a look at that.

Thanks.

-- 
/Jacob Carlborg
Feb 12 2020