www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - [SAoC] MLIR Support for LDC

reply Roberto Rosmaninho <Robertogrosmaninho gmail.com> writes:
Hi everyone, I'm responsible to implement MLIR support to LDC, 
just in case you missed my presentation post about the project 
this is the post that I wrote last week: 
https://forum.dlang.org/thread/hinceoqtdfcxhahbuwru forum.dlang.org

This first week I finished the first topic of Milestone 1:
   Milestone 1. Compilation of D through the LLVM dialect of MLIR:
     - Build and set up as the newest version available: LLVM 
(10.0.0svn), MLIR, LDC (1.17.0) and DMD (v2.087.1).

Thanks to my mentor Nicholas Wilson and other LDC developers I'm 
now able to work with LDC and LLVM 10, which is the version that 
MLIR it's built.

For now, I have just two questions:
I've already started to play with LDC source code but I still 
don't know how to run and test the modifications that I writing. 
How can I see a "printf" that I put in a file just to test a 
variable?
How is the best way to debug LDC on MacOS? And how to enable all 
flags of debug and log?

I'm currently working on irstate.* and declarations.cpp, any tips 
or warnings about these files?

Regards,
Roberto Rosmaninho
Sep 24 2019
next sibling parent reply Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Wednesday, 25 September 2019 at 01:47:43 UTC, Roberto 
Rosmaninho wrote:

 Hi everyone, I'm responsible to implement MLIR support to LDC, 
 just in case you missed my presentation post about the project 
 this is the post that I wrote last week: 
 https://forum.dlang.org/thread/hinceoqtdfcxhahbuwru forum.dlang.org


IMO, it's better to keep updates on one thread (e.g. your 
presentation thread).
That way, one can open only one thread and see all the history.


 For now, I have just two questions:
 I've already started to play with LDC source code but I still 
 don't know how to run and test the modifications that I 
 writing. How can I see a "printf" that I put in a file just to 
 test a variable?
 How is the best way to debug LDC on MacOS? And how to enable 
 all flags of debug and log?


So, if you just want to print something for debugging purposes,
you can just use printf. However, I would recommend getting used 
to
Logger utilities. Let me explain:
You can see across LDC this "Logger" thing everywhere
There is code like:
IF_LOG Logger::println("InterfaceDeclaration::codegen: '%s'",
                        decl->toPrettyChars())

(That specific code is in visit(InterfaceDeclaration) in 
declarations.cpp).

These work like printf but with some added features. First, if 
you put
this IF_LOG thing in the front, then the message is printed only 
if logging
is enabled (don't worry, I will explain how to enable logging). 
Otherwise,
always of course. That helps so that you can leave logging info 
around
(when done debugging and you feel that there's something that is 
useful
to be printed when one requests a log).

The second thing which for me is super smart and useful is that 
these
utilities have a sense of scope. Think of this example:

void b() {
   Logger::println("roberto2");
}

void a() {
   Logger::println("roberto1");
   b();
}

With normal printf, you would see:
roberto1
roberto2

But with Logger::println, you see:
roberto1
* roberto2

Notice the indentation and the star. ;)
That is super useful because it decouples messages and helps you 
pinpoint
where your message is in the bigger picture.

(That thing btw is controlled with LOG_SCOPE which you will also 
see
around and you can learn more about how it works either by just 
grepping
it or reading this [1]).

So, now that you have a nicer way to print, how one enables 
logging?

Logging is of course useful not in order to see _your_ message, 
but all the
other messages and how yours is placed among them.
There are 2 kinds of logging (that I am aware of. I'm pretty 
happy with the second
so I didn't search more).
The one is -v which is "Verbose". What that means basically is it 
logs
info not for the LDC programmer but for the LDC user, who happens 
to care
to see some specific info about what LDC does.
There is also -vv which has a weird description but basically 
means: "Info
for the LDC developers that care about the internal things that 
happen".

You can see these and all the other options here [2]

You probably want the second, -vv. Since we're on the subject, 
the other
thing you probably will like is -output-ll which outputs the LLVM 
IR.
You may even end up to put a specific flag for MLIR. But for the 
time being,
seeing the LLVM IR I think will help.

Lastly, the usual workflow for LDC is Johan's workflow [3].
You don't necessarily want to follow this for _every single 
change_. That will
kill productivity and it's already kind of boring to wait for 
compilation
when you change even a single variable.
What you probably want though is having a specific (or a bunch of 
specific)
test case that you run (instead of running the whole suite) 
continuously.
and _definitely_, you run the whole suite when you're done. I'd 
suggest
to run it every so often, I usually do it every 10-20 "small" 
changes.

Now, notice this note I have added on the bottom of the page:

Note: `ninja` builds the whole LDC suite. Most of the time, 
you'll want to only rebuild the ldc2 binary. You can do that with 
`ninja ldc2`.

So, when you do a change on a single file, you want to do `ninja 
ldc2`. Trust me,
this is day and night in how much time it takes to compile.

Unfortunately, I can't give you any info about MacOS because I 
have not used
one for any serious programming. I _think_ David Nadlinger
uses Mac but I may very well be wrong.

AFAIK, you can have GDB. My guess is that Mac will have a more 
Visual Studio kind
of debugger because the terminal interface of GDB is just 
nerve-racking.
But since it's better than nothing for some situations, if I 
can't target
something easily, I do "info functions" while first I have made 
gdb output
to a file and then I grep this file for function that I want. 
This is
more complicated than you would expect because of name mangling 
and stuff.
I described this very roughly since I don't know if you're 
interested. Let
me know if you, but I would also recommend to ask Mac users for 
more info.

Lastly, make sure you have read this page [4] and that you have 
installed
LLVM and LDC with -DCMAKE_BUILD_TYPE=RelWithDebInfo. And bear in 
mind
that LDC debug info is not the best :).

Best of luck,
Stefanos Baziotis


 I'm currently working on irstate.* and declarations.cpp, any 
 tips or warnings about these files?

 Regards,
 Roberto Rosmaninho


[1] 
https://forum.dlang.org/post/xbfsztzkrqtxmfgcflug forum.dlang.org
[2] https://wiki.dlang.org/Using_LDC
[3] 
https://wiki.dlang.org/LDC_Lit-based_testsuite#My_workflow_when_working_on_LDC_.28Johan.29
[4] 
https://wiki.dlang.org/Building_LDC_from_source#Building_LLVM_manually
Sep 25 2019
parent reply Roberto Rosmaninho <Robertogrosmaninho gmail.com> writes:
On Wednesday, 25 September 2019 at 10:42:46 UTC, Stefanos 
Baziotis wrote:


 IMO, it's better to keep updates on one thread (e.g. your 
 presentation thread).
 That way, one can open only one thread and see all the history.


All right, I'll keep updating this thread them, thanks!


 So, if you just want to print something for debugging purposes,
 you can just use printf. However, I would recommend getting 
 used to
 Logger utilities.
 . . .

 There is also -vv which has a weird description but basically 
 means: "Info
 for the LDC developers that care about the internal things that 
 happen".

 You can see these and all the other options here [2]

 You probably want the second, -vv. Since we're on the subject, 
 the other


Thanks a lot!! It'll make my work much easier!!


 thing you probably will like is -output-ll which outputs the 
 LLVM IR.
 You may even end up to put a specific flag for MLIR. But for 
 the time being,
 seeing the LLVM IR I think will help.


Yeah! The plan is to have a -output-mlir at the end of the 
project!


 Lastly, the usual workflow for LDC is Johan's workflow [3].


This workflow makes sense even for this project that I still have 
to change a lot of files to get some output, just to know if I'm 
crashing something. Thanks again!


 Note: `ninja` builds the whole LDC suite. Most of the time, 
 you'll want to only rebuild the ldc2 binary. You can do that 
 with `ninja ldc2`.
 So, when you do a change on a single file, you want to do 
 `ninja ldc2`. Trust me,
 this is day and night in how much time it takes to compile.


This seems powerful due the huge time to compile LDC just because 
of single modification. I'll try it!


 Unfortunately, I can't give you any info about MacOS because I 
 have not used
 one for any serious programming. I _think_ David Nadlinger
 uses Mac but I may very well be wrong.


I have set up my project on Ubuntu as well, so if you know 
something about debugging on Linux and can explain it in greater 
detail it'll help anyway!


 Lastly, make sure you have read this page [4] and that you have 
 installed
 LLVM and LDC with -DCMAKE_BUILD_TYPE=RelWithDebInfo. And bear 
 in mind
 that LDC debug info is not the best :).


Done!


 Best of luck,
 Stefanos Baziotis


Thanks again,
Roberto Rosmaninho
Sep 25 2019
parent Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Wednesday, 25 September 2019 at 13:30:20 UTC, Roberto 
Rosmaninho wrote:

 On Wednesday, 25 September 2019 at 10:42:46 UTC, Stefanos 
 Baziotis wrote:
 Thanks a lot!! It'll make my work much easier!!


I'm glad!


 This seems powerful due the huge time to compile LDC just 
 because of single modification. I'll try it!


Yes, for me it was crucial. You can do the same thing for more 
changes.
Basically, as long as you change only LDC files.


 I have set up my project on Ubuntu as well, so if you know 
 something about debugging on Linux and can explain it in 
 greater detail it'll help anyway!


IMO, for the most part the debugging experience on Linux is 
horrible. :)
If you're lucky, because LDC is mostly C++, you may be able to get
CLion to work. Or make Visual Studio Code work as a GUI for GDB 
(I have not
done either of those, so I can't provide further info, but 
there's info
on the Internet).
The common case though is that you're stuck with GDB, which while 
powerful,
its terminal interface personally drives me crazy.

But, enough with bashing on GDB. I debug LDC mostly with printfs, 
but when
these don't work, here's what I do:

Note: I will cover some basics in case you know nothing about GDB.
I'm not a GDB guru anyway, but these will help you in most 
situations.

== Case 1 - You want to break in LDC / LLVM code ==

This is the easiest case. You just run the executable through 
gdb, then
break on a specific line by specifying the _full_ path. For 
example, I
have my ldc repo in: /home/stefanos/dlang/ldc/

So:
$ gdb /home/stefanos/dlang/ldc/build/bin/ldc2
... GDB opens. Inside GDB:
$ break /home/stefanos/dlang/ldc/gen/statements.cpp:289

to break on line 289 on gen/statements.cpp

You can similarly break into a function, e.g.

$ break ToIRVisitor::visit(IfStatement*)

using tab for autocomplete.

Similarly, you can break in LLVM code, e.g.:

$ break llvm::BasicBlock::Create(llvm::LLVMContext&, llvm::Twine 
const&, llvm::Function*, llvm::BasicBlock*)

again using autocomplete.

== Case 2 - You want to break in DMD code ==

For that, I'd recommend using the dmd executable, built with dmd.
Basically, doing what you would do if you were working on DMD. 
So, you download
the DMD version you want and do the same debugging steps.
You just have better debugging experience when DMD is built with 
DMD.

For now, you can forget what I said about listing functions and 
stuff. I can't remember why I had needed that.

Best regards,
Stefanos
Sep 25 2019
prev sibling next sibling parent reply Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Wednesday, 25 September 2019 at 01:47:43 UTC, Roberto 
Rosmaninho wrote:

 For now, I have just two questions:
 I've already started to play with LDC source code but I still 
 don't know how to run and test the modifications that I 
 writing. How can I see a "printf" that I put in a file just to 
 test a variable?
 How is the best way to debug LDC on MacOS? And how to enable 
 all flags of debug and log?


I forgot to remind you: Don't hesitate to ask LDC-specific 
questions in
the LDC gitter [1]. Apart from the fact that it's very welcome,
you also have a better chance that seasoned developers will see 
it and/or
answer like kinke and Johan (although they will probably see it 
here too).


 I'm currently working on irstate.* and declarations.cpp, any 
 tips or warnings about these files?


I personally don't have any experience with those files. Bear in 
mind that
in my experience, questions about constrained situations (like, 
oh in the visit of
ClassDeclaration I don't understand the x) are easier to answer 
than broad ones.

Now that you mentioned it though, you reminded me that LDC does 
not have
a header description for a file (like on the top of every file, 
describing
what the file does roughly). I find this useful. I also find 
useful
a source guide like DMD's [2]. I'm trying to construct one on my 
own bit by bit.
If along the course of your project you're interested to 
contribute to the
documentation, let me know.

For now, you can safely ignore the last paragraph, I just left it 
there
for future reference. One thing to have in mind though is that 
you probably
will need to get somewhat comfortable with the DMD source code, 
so this
DMD guide might help.

Best regards,
Stefanos Baziotis

[1] http://gitter.im/ldc-developers/main
[2] https://wiki.dlang.org/DMD_Source_Guide
Sep 25 2019
parent reply Roberto Rosmaninho <Robertogrosmaninho gmail.com> writes:
On Wednesday, 25 September 2019 at 11:02:35 UTC, Stefanos 
Baziotis wrote:


 Now that you mentioned it though, you reminded me that LDC does 
 not have
 a header description for a file (like on the top of every file, 
 describing
 what the file does roughly). I find this useful. I also find 
 useful
 a source guide like DMD's [2]. I'm trying to construct one on 
 my own bit by bit.
 If along the course of your project you're interested to 
 contribute to the
 documentation, let me know.


Yeah, I'm trying to understand the code and without some 
explanation about the files it has been really difficult, this 
lack of documentation is annoying but certainly, I'd contribute 
to improving this, I'm on Slack and Gitter just ping me there!


Regards,
Roberto Rosmaninho
Sep 25 2019
parent Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Wednesday, 25 September 2019 at 14:16:36 UTC, Roberto 
Rosmaninho wrote:

 Yeah, I'm trying to understand the code and without some 
 explanation about the files it has been really difficult


There's not been much time since I started contributing to LDC so 
I know
how it feels. Don't hesitate to ask questions! I hope I will be 
able to
provide some feedback if kinke, johan etc. don't have time.

LDC is kind of a beast, with minimal commenting (for my taste), 
so just
stick with it. :)


 this lack of documentation is annoying but certainly, I'd 
 contribute to improving this, I'm on Slack and Gitter just ping 
 me there!


I'm on both Slack and Gitter too, so you can reach me if you have 
any problem.
For now, I think the best things to do if you want to help with 
doc are (these things are the ones I do as well and that require 
minimal effort so they
shouldn't take valuable time):
a) Ask about what you don't understand! It's obvious that to be 
selected
for SAoC, you're good. Meaning, if you don't understand 
something, it's probably
not something that is obvious and / or not something that one 
would understand
easily (or at all) from context. So, IMO, we better add some 
clarification
for future contributors like you and me.

b) Whenever you feel you have understood something non-trivial, 
add a clarification
comment. Even if it's wrong, it will be caught in PR review (it's 
better to
actually pinpoint to the comment on github so that reviewers can 
spot it easily
and verify it).

What I plan to do other than those micro-changes is roughly:
a) Add a header description in files.
b) Write some articles describing the architecture of LDC, maybe 
some diagrams
etc. Sort of like the DMD guide, but more visual and extended.

These actions require 2 things that I currently don't have:
a) Knowing more about LDC.
b) Time.

When I get around to do any of what I described, I hope that LDC 
people
will have the time to review them. In any case, if you feel that 
in the course
of your project, you can provide any contributions in what is 
described above,
be my guest!

Kind regards,
Stefanos Baziotis
Sep 25 2019
prev sibling next sibling parent reply Roberto Rosmaninho <Robertogrosmaninho gmail.com> writes:
Hi everyone,
I'm here just to report the status of my project on its second 
week:

  - I Read all documentation from MLIR
  - I Started to implement mlirstate.h and mlirstate.cpp
  - I Started to implement mlircodegenerator.h 
mlircodegenerator.cpp

The main issue that I stuck this week was to compile ldc with my 
modifications due to problems with mlir headers and dmd/globals.h.
So I had these 2 questions and I hope someone could help me:

  - What is the best way to refer mlir (a project inside llvm, 
like clang) into ldc CMakeLists.txt? I need the headers provided 
by this project to write my codes.

  - I'm trying to do the same things that ldc does to get LLVM IR, 
so I tried to create a "DArray<const char> mlir_ext;" in 
dmd/globals.h:296, but when I compile ldc with -vv it stops at " 
Targeting 'x86_64-apple-macosx' (CPU 'core2' with features '') " 
and tell me that the process finished with exit code 1. How can I 
fix this? And if I'm doing something wrong, what is the best way 
to get these variables for ldc and what is its function on the 
program? I thought it was setting the file extension, but I'm not 
sure.

Regards,
Roberto Rosmaninho
Oct 02 2019
parent reply Johan Engelen <j j.nl> writes:
On Wednesday, 2 October 2019 at 18:43:14 UTC, Roberto Rosmaninho 
wrote:

 Hi everyone,
 I'm here just to report the status of my project on its second 
 week:

  - I Read all documentation from MLIR
  - I Started to implement mlirstate.h and mlirstate.cpp
  - I Started to implement mlircodegenerator.h 
 mlircodegenerator.cpp

 The main issue that I stuck this week was to compile ldc with 
 my modifications due to problems with mlir headers and 
 dmd/globals.h.
 So I had these 2 questions and I hope someone could help me:

  - What is the best way to refer mlir (a project inside llvm, 
 like clang) into ldc CMakeLists.txt? I need the headers 
 provided by this project to write my codes.


As far as I can tell, if you checkout MLIR into your local LLVM 
checkout [1], then it is automatically built with LLVM and the 
header files will be part of the LLVM install. So to get to the 
headers you probably don't have to add anything in LDC's 
CMakeLists.txt.
For linking correctly, you probably have to add similar CMake 
code as is done for SPIR-V, see the cmake code around 
`LLVM_SPIRV_FOUND`.


  - I'm trying to do the same things that ldc does to get LLVM 
 IR, so I tried to create a "DArray<const char> mlir_ext;" in 
 dmd/globals.h:296, but when I compile ldc with -vv it stops at 
 " Targeting 'x86_64-apple-macosx' (CPU 'core2' with features 
 '') " and tell me that the process finished with exit code 1. 
 How can I fix this? And if I'm doing something wrong, what is 
 the best way to get these variables for ldc and what is its 
 function on the program? I thought it was setting the file 
 extension, but I'm not sure.


globals.h _must exactly_ match with globals.d and there is no 
tool that will check that for you ('.h' is used by the C++ 
compiler, '.d' by the D compiler and they don't talk to each 
other). One way to look at it is that `globals.d` is the `.cpp` 
file belonging to `globals.h`.
For your particular issue: is you add `mlir_ext` to `globals.h`, 
you have to add it to `globals.d` in _exactly the same_ location 
in the `Global` struct.

Do you aim to take a similar approach as the SPIR-V/dcompute 
mechanism?

cheers,
   Johan

[1] https://github.com/tensorflow/mlir
Oct 02 2019
parent reply Roberto Rosmaninho <Robertogrosmaninho gmail.com> writes:
On Wednesday, 2 October 2019 at 20:31:02 UTC, Johan Engelen wrote:

 As far as I can tell, if you checkout MLIR into your local LLVM 
 checkout [1], then it is automatically built with LLVM and the 
 header files will be part of the LLVM install.


Yes! I found a solution: create a cmake find for mlir, ldc have 
to mlir and it fixes the problem: http://bit.ly/2n7xEJS


 globals.h _must exactly_ match with globals.d and there is no 
 tool that will check that for you ('.h' is used by the C++ 
 compiler, '.d' by the D compiler and they don't talk to each 
 other). One way to look at it is that `globals.d` is the `.cpp` 
 file belonging to `globals.h`.
 For your particular issue: is you add `mlir_ext` to 
 `globals.h`, you have to add it to `globals.d` in _exactly the 
 same_ location in the `Global` struct.


Works here! Thanks!!


 Do you aim to take a similar approach as the SPIR-V/dcompute 
 mechanism?


I'm new to ecosystem D, so I don't know these approaches in D, do 
you mind explaining me about it or giving me a guide to study?

Best regards,
Roberto Rosmaninho
Oct 02 2019
next sibling parent reply Roberto Rosmaninho <Robertogrosmaninho gmail.com> writes:
On Wednesday, 2 October 2019 at 23:45:03 UTC, Roberto Rosmaninho 
wrote:


 Yes! I found a solution: create a CMake find for mlir (ldc have 
 one for LLVM) and it fixes the problem: http://bit.ly/2n7xEJS


Just fixing te sentence...
Oct 02 2019
parent reply Roberto Rosmaninho <Robertogrosmaninho gmail.com> writes:
Hi everyone,
Just a kick updates about the project:

1. Compilation of D through the LLVM dialect of MLIR:
  - Model all core IR structures in LLVM: Instructions, Globals, 
Modules, etc.
  -- Status: Module and Function already implemented, working now 
on operations.

  - Traverse the AST of D to emit MLIR code.
  -- Status: Working on simple examples with integers and 
BinOps(+,-,x,/) so far, I'll work this last week to cover more 
types and operations.

CMake Status: I'm trying to compile the project on other machines 
to make sure that each modification doesn't crash the compiler.

Best regards,
Roberto Rosmaninho
Oct 09 2019
parent reply Roberto Rosmaninho <Robertogrosmaninho gmail.com> writes:
Hi everyone, I'm here for the last update of this 1st Milestone!

So, this week I refactored the code! Unfortunately, I wrote a 
bunch of useless code while I was trying to understand how ldc 
does some things, but that ok...the code is much better readable 
now, so if you have some interest in review the code I'm still 
updating this branch:
https://github.com/Robertorosmaninho/ldc/tree/fix-llvm-10

Since my last update I worked on the following tasks:
  - Refactoring the code.
     - I was doing some weird stuff to get access to all 
statements, so now it's possible to declare more than 1 function 
on a program.
     - I changed the name of the functions for the mangled name.
     - I had some problems with function parameters as well, but 
finally, I could fix.
  - I made mlir less verbose (HUGE improvements here!!)
  - Implement calls
  - Implement one-dimensional tensors (as a static array)

So far my implementation can deal with:
  - Simple VarDeclarations
  - Operations: +, - , x, /
  - Declare undimensional vectors
  - Function calls
  - All types are integer except the output of arrays

Issues:
  - MLIR is growing too fast I am more than 150 commits behind 
HEAD and at least yesterday wasn't compiling on my Linux with 
LLVM trunk.
  - LLVM trunk changed some stuff and I wasn't able to compile 
with my version of MLIR.

Where I'm at each project:
- LLVM: commit f79d8a064ce4d8846377e4abbc9a389b62f90d43 |  Wed 
Aug 28 15:40:34 2019.
- MLIR: I'm working on fix-llvm-10 branch so I'm behind commit 
b50775af52281d9f38aae6c359b72dc715408df0 of Aug 30.

This should be fixed next week, I will try to port my code to the 
next stable version of each project.

For the next steps:
  - Make CMake FindMLIR more general, it seems like it's not 
possible to reach projects directory by LLVM_ROOT_DIR on Linux so 
I have to figure out a new way to find MLIR.
  - Extend the size of tensor and work on support for all types...
  - Create the LDC Dialect
Oct 15 2019
parent reply Johan <j j.nl> writes:
On Tuesday, 15 October 2019 at 23:09:06 UTC, Roberto Rosmaninho 
wrote:

 Hi everyone, I'm here for the last update of this 1st Milestone!


Hey Roberto,
   It's fun to read your updates, thanks :)


 So, this week I refactored the code! Unfortunately, I wrote a 
 bunch of useless code while I was trying to understand how ldc 
 does some things, but that ok...the code is much better 
 readable now, so if you have some interest in review the code 
 I'm still updating this branch:
 https://github.com/Robertorosmaninho/ldc/tree/fix-llvm-10


I hope to get the LLVM 10 fixes into LDC master soon. Then you 
can rebase on top of that to make the specific MLIR changes more 
obvious and easier to view.


 Since my last update I worked on the following tasks:
  - Refactoring the code.
     - I was doing some weird stuff to get access to all 
 statements, so now it's possible to declare more than 1 
 function on a program.
     - I changed the name of the functions for the mangled name.
     - I had some problems with function parameters as well, but 
 finally, I could fix.
  - I made mlir less verbose (HUGE improvements here!!)
  - Implement calls
  - Implement one-dimensional tensors (as a static array)

 So far my implementation can deal with:
  - Simple VarDeclarations
  - Operations: +, - , x, /
  - Declare undimensional vectors
  - Function calls
  - All types are integer except the output of arrays


(I didn't check if you already did this) I find it very helpful 
to look at test cases to see what functionality you are adding. 
The lit-based testsuite is very convenient for checking IR 
output. Did you already add MLIR tests there?


 Issues:
  - MLIR is growing too fast I am more than 150 commits behind 
 HEAD and at least yesterday wasn't compiling on my Linux with 
 LLVM trunk.
  - LLVM trunk changed some stuff and I wasn't able to compile 
 with my version of MLIR.


Don't stress out too much about not following LLVM trunk exactly. 
But also don't lag too long behind, such that the changes you 
need to make are not too big and that you keep up to date on 
impactful MLIR changes. So perhaps only pull MLIR trunk once in 2 
weeks or so.
You can try to offload some of the general LLVM trunk compile 
issues to others, by making us aware of LLVM trunk compile 
issues. I can try to help with that.


 Where I'm at each project:
 - LLVM: commit f79d8a064ce4d8846377e4abbc9a389b62f90d43 |  Wed 
 Aug 28 15:40:34 2019.
 - MLIR: I'm working on fix-llvm-10 branch so I'm behind commit 
 b50775af52281d9f38aae6c359b72dc715408df0 of Aug 30.

 This should be fixed next week, I will try to port my code to 
 the next stable version of each project.


LLVM trunk compilation fixes (not related to MLIR) you can submit 
as PRs to LDC. That way your branch is solely about MLIR changes, 
and not cluttered by the general LLVM changes needed.


 For the next steps:
  - Make CMake FindMLIR more general, it seems like it's not 
 possible to reach projects directory by LLVM_ROOT_DIR on Linux 
 so I have to figure out a new way to find MLIR.


I was hoping you had solved these issues. I'm still confused by 
"reach projects directory", is it not possible to `ninja install` 
MLIR ?


  - Extend the size of tensor and work on support for all 
 types...
  - Create the LDC Dialect


Looking forward to the next update.

cheers,
   Johan
Oct 17 2019
parent reply Roberto Rosmaninho <Robertogrosmaninho gmail.com> writes:
On Thursday, 17 October 2019 at 22:05:30 UTC, Johan wrote:

 Hey Roberto,
   It's fun to read your updates, thanks :)


Hey Johan, I'm glad to hear that, thank you!!


 I hope to get the LLVM 10 fixes into LDC master soon. Then you 
 can rebase on top of that to make the specific MLIR changes 
 more obvious and easier to view.


Nic told me about your PR and I'm already trying to rebase my 
code on it!


 (I didn't check if you already did this) I find it very helpful 
 to look at test cases to see what functionality you are adding. 
 The lit-based testsuite is very convenient for checking IR 
 output. Did you already add MLIR tests there?


Not yet, I'll use this test suite on this next phase, maybe I can 
put some tests that I'm already writing there. :)


 Don't stress out too much about not following LLVM trunk 
 exactly. But also don't lag too long behind, such that the 
 changes you need to make are not too big and that you keep up 
 to date on impactful MLIR changes. So perhaps only pull MLIR 
 trunk once in 2 weeks or so.


Great idea, I'll follow it, thanks!


 You can try to offload some of the general LLVM trunk compile 
 issues to others, by making us aware of LLVM trunk compile 
 issues. I can try to help with that.


Ok!


 LLVM trunk compilation fixes (not related to MLIR) you can 
 submit as PRs to LDC. That way your branch is solely about MLIR 
 changes, and not cluttered by the general LLVM changes needed.


I'm currently trying to use Linux to update the project and find 
the projects directory to improve my FindMLIR.cmake but I'm 
having the following problem over and over and I coudn't figure 
out what I'm doing wrong:

Commands:
$ git clone http://github.com/ldc-developers/ldc
$ cd ldc
$ git checkout llvm10cmake
$ mkdir build && cd build
$ export DMD=~/dlang/dmd-2.087.1/linux/bin64/dmd
$ cmake -G Ninja ..  -DCMAKE_BUILD_TYPE=Release

here later if I request.
$ ninja
$ ninja install
#It gives me the following error:

CMake Error at cmake_install.cmake:44 (file):
   file INSTALL cannot copy file 
"/home/roberto/dlang/ldc/build/bin/ldc2" to
   "/usr/local/bin/ldc2".


FAILED: CMakeFiles/install.util
cd /home/roberto/dlang/ldc/build && /usr/bin/cmake -P 
cmake_install.cmake
ninja: build stopped: subcommand failed.

#When I try to compile a simple program: int main(){int a = 10; 
return 0;}
$ ~/ldc/build/bin/ldc2 sum.d

Error: cannot find source code for runtime library file 'object.d'
        ldc2 might not be correctly installed.
        Please check your ldc2.conf configuration file.
        Installation instructions can be found at 
http://wiki.dlang.org/LDC.
Specify path to file 'object.d' with -I switch


 I was hoping you had solved these issues. I'm still confused by 
 "reach projects directory", is it not possible to `ninja 
 install` MLIR ?


Well, the instructions on MLIR git just ask for a "cmake --build 
. --target check-mlir" when you're compiling LLVM with MLIR.

The problem is that doesn't exist any flags such as 
$LLVM_SOURCE_DIR or $LLVM_PROJECTS_DIR. So unless I pass the path 
of $MLIR_ROOT_DIR BY command line I'm don't know any way to reach 
the source code of llvm to get MLIR files. Unfortunately, the 
project's dir inside llvm build doesn't provide all the files 
that I need.

Just to illustrate my problem with LLVM_ROOT_DIR I added the 
following line on my cmake:
       message("LLVM ROOT DIR IS: ${LLVM_ROOT_DIR}")
Then when I run the CMake I get the following result:
       LLVM ROOT DIR IS: /home/roberto/llvm-project/build


Hope I can solve this soon!

Best regards,
Roberto
Oct 18 2019
parent reply Johan Engelen <j j.nl> writes:
On Friday, 18 October 2019 at 21:30:16 UTC, Roberto Rosmaninho 
wrote:

 On Thursday, 17 October 2019 at 22:05:30 UTC, Johan wrote:

 Hey Roberto,
   It's fun to read your updates, thanks :)


 Hey Johan, I'm glad to hear that, thank you!!


 I hope to get the LLVM 10 fixes into LDC master soon. Then you 
 can rebase on top of that to make the specific MLIR changes 
 more obvious and easier to view.


 Nic told me about your PR and I'm already trying to rebase my 
 code on it!


I just merged it into LDC master, so you can rebase on LDC master 
again.


 (I didn't check if you already did this) I find it very 
 helpful to look at test cases to see what functionality you 
 are adding. The lit-based testsuite is very convenient for 
 checking IR output. Did you already add MLIR tests there?


 Not yet, I'll use this test suite on this next phase, maybe I 
 can put some tests that I'm already writing there. :)


Exactly. I find it very helpful to immediately add the tests in 
the lit test suite while implementing things.


 LLVM trunk compilation fixes (not related to MLIR) you can 
 submit as PRs to LDC. That way your branch is solely about 
 MLIR changes, and not cluttered by the general LLVM changes 
 needed.


 I'm currently trying to use Linux to update the project and 
 find the projects directory to improve my FindMLIR.cmake but 
 I'm having the following problem over and over and I coudn't 
 figure out what I'm doing wrong:

 Commands:
 $ git clone http://github.com/ldc-developers/ldc
 $ cd ldc
 $ git checkout llvm10cmake
 $ mkdir build && cd build
 $ export DMD=~/dlang/dmd-2.087.1/linux/bin64/dmd
 $ cmake -G Ninja ..  -DCMAKE_BUILD_TYPE=Release

 CMake here later if I request.
 $ ninja
 $ ninja install


For LDC, you don't have to do a `ninja install`. If you do, you 
should specify where to install it when running cmake. What I do 
is have a `run_cmake.sh` that runs cmake for me so I can remember 
what options I passed to cmake. For reference, this is my 
`run_cmake.sh` for building with LLVM trunk:
```
cmake -G Ninja -DRT_SUPPORT_SANITIZERS=ON 
-DLDC_DYNAMIC_COMPILE=False 
-DD_COMPILER="/Users/johan/ldc/ldc2-1.12.0-osx-x86_64/bin/ldmd2" 
-DCMAKE_BUILD_TYPE="Debug" 
-DCMAKE_INSTALL_PREFIX="/Users/johan/ldc/johan/installtrunk" 
-DLLVM_ROOT_DIR="/Users/johan/llvm/llvmtrunkinstall" ..
```

Note that I explicitly pass LLVM_ROOT_DIR where my LLVM trunk 
build is installed.


 I was hoping you had solved these issues. I'm still confused 
 by "reach projects directory", is it not possible to `ninja 
 install` MLIR ?


 Well, the instructions on MLIR git just ask for a "cmake 
 --build . --target check-mlir" when you're compiling LLVM with 
 MLIR.


The instructions don't say it, because I think most people will 
already know what to do and there are many cases where one would 
_not_ want to install LLVM.
This is what my `run_cmake.sh` looks like for building LLVM:
```
cmake -G Ninja 
-DLLVM_ENABLE_PROJECTS="clang;compiler-rt;lld;polly" 
-DCMAKE_INSTALL_PREFIX="/Users/johan/llvm/llvmtrunkinstall" 
-DLLVM_TARGETS_TO_BUILD="AArch64;ARM;Mips;MSP430;NVPTX;PowerPC;X86"
-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD="RISCV;WebAssembly"
-DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_APPEND_VC_REV=ON
-DLLVM_INSTALL_UTILS=ON ..
```
Note the "-DCMAKE_INSTALL_PREFIX", that's where LLVM will be 
installed. And that is what you need to pass to LDC CMake 
"-DLLVM_ROOT_DIR".

Hope that helps,
   Johan
Oct 18 2019
parent reply Roberto Rosmaninho <Robertogrosmaninho gmail.com> writes:
On Friday, 18 October 2019 at 21:43:53 UTC, Johan Engelen wrote:

 I just merged it into LDC master, so you can rebase on LDC 
 master again.


Great, I already did the git pull here.


 For LDC, you don't have to do a `ninja install`. If you do, you 
 should specify where to install it when running cmake. What I 
 do is have a `run_cmake.sh` that runs cmake for me so I can 
 remember what options I passed to cmake. For reference, this is 
 my `run_cmake.sh` for building with LLVM trunk:
 ```
 cmake -G Ninja -DRT_SUPPORT_SANITIZERS=ON 
 -DLDC_DYNAMIC_COMPILE=False 
 -DD_COMPILER="/Users/johan/ldc/ldc2-1.12.0-osx-x86_64/bin/ldmd2"
-DCMAKE_BUILD_TYPE="Debug"
-DCMAKE_INSTALL_PREFIX="/Users/johan/ldc/johan/installtrunk"
-DLLVM_ROOT_DIR="/Users/johan/llvm/llvmtrunkinstall" ..
 ```

 Note that I explicitly pass LLVM_ROOT_DIR where my LLVM trunk 
 build is installed.


I just tried to compile it here using your script as baseline:

$ cmake -G Ninja -DRT_SUPPORT_SANITIZERS=ON 
-DLDC_DYNAMIC_COMPILE=False 
-DD_COMPILER="/home/roberto/dlang/dmd-2.087.1/linux/bin64/dmd" 
-DCMAKE_BUILD_TYPE="Debug" 
-DCMAKE_INSTALL_PREFIX="/home/roberto/dlang/ldc" 
-DLLVM_ROOT_DIR="/home/roberto/llvm-project" ..

$ ninja

Then I tried to compile this example: 
https://run.dlang.io/is/ulDVcv

$ ~/dlang/ldc/build/bin/ldc2 test.d

And get the same error:

Error: cannot find source code for runtime library file 'object.d'
        ldc2 might not be correctly installed.
        Please check your ldc2.conf configuration file.
        Installation instructions can be found at 
http://wiki.dlang.org/LDC.
Specify path to file 'object.d' with -I switch

I'm not trying my code yet, this is just the master branch that 
I'm trying to use.
Any ideas or should I report this as a issue?


 I was hoping you had solved these issues. I'm still confused 
 by "reach projects directory", is it not possible to `ninja 
 install` MLIR ?


 Well, the instructions on MLIR git just ask for a "cmake 
 --build . --target check-mlir" when you're compiling LLVM with 
 MLIR.


 The instructions don't say it, because I think most people will 
 already know what to do and there are many cases where one 
 would _not_ want to install LLVM.


Right!


 This is what my `run_cmake.sh` looks like for building LLVM:
 ```
 cmake -G Ninja 
 -DLLVM_ENABLE_PROJECTS="clang;compiler-rt;lld;polly" 
 -DCMAKE_INSTALL_PREFIX="/Users/johan/llvm/llvmtrunkinstall" 
 -DLLVM_TARGETS_TO_BUILD="AArch64;ARM;Mips;MSP430;NVPTX;PowerPC;X86"
-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD="RISCV;WebAssembly"
-DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_APPEND_VC_REV=ON
-DLLVM_INSTALL_UTILS=ON ..
 ```
 Note the "-DCMAKE_INSTALL_PREFIX", that's where LLVM will be 
 installed. And that is what you need to pass to LDC CMake 
 "-DLLVM_ROOT_DIR".


I will test it as soon as I can compile LDC without my code!
Best regards,
Roberto
Oct 18 2019
parent reply kinke <noone nowhere.com> writes:
On Friday, 18 October 2019 at 22:18:29 UTC, Roberto Rosmaninho 
wrote:

 $ ~/dlang/ldc/build/bin/ldc2 test.d

 And get the same error:

 Error: cannot find source code for runtime library file 
 'object.d'
        ldc2 might not be correctly installed.
        Please check your ldc2.conf configuration file.
        Installation instructions can be found at 
 http://wiki.dlang.org/LDC.
 Specify path to file 'object.d' with -I switch

 I'm not trying my code yet, this is just the master branch that 
 I'm trying to use.
 Any ideas or should I report this as a issue?


Looks like you simply haven't cloned the git submodules. Add 
--recursive to the git clone command, or do a `git submodule 
update --init` in an existing LDC repo. Note that CMake spits out 
a warning when the druntime isn't found and informs that only the 
compiler will be built.
Oct 18 2019
parent Roberto Rosmaninho <Robertogrosmaninho gmail.com> writes:
On Friday, 18 October 2019 at 22:25:00 UTC, kinke wrote:

 Looks like you simply haven't cloned the git submodules. Add 
 --recursive to the git clone command, or do a `git submodule 
 update --init` in an existing LDC repo. Note that CMake spits 
 out a warning when the druntime isn't found and informs that 
 only the compiler will be built.


Oh God, I knew it was just a small problem, but I wasn't expected 
a dumb one!!
So sorry and thanks Kinke and Johan!
Oct 18 2019
prev sibling parent reply Johan Engelen <j j.nl> writes:
On Wednesday, 2 October 2019 at 23:45:03 UTC, Roberto Rosmaninho 
wrote:

 On Wednesday, 2 October 2019 at 20:31:02 UTC, Johan Engelen 
 wrote:

 As far as I can tell, if you checkout MLIR into your local 
 LLVM checkout [1], then it is automatically built with LLVM 
 and the header files will be part of the LLVM install.


 Yes! I found a solution: create a cmake find for mlir, ldc have 
 to mlir and it fixes the problem: http://bit.ly/2n7xEJS


This doesn't look quite right.
You should `ninja install` LLVM (set an install path using 
-DCMAKE_INSTALL_PREFIX="..." when running cmake to generate the 
ninja build file), such that you get a local LLVM installation 
that contains also the MLIR stuff. Are you sure that MLIR is 
built when building LLVM? Perhaps cmake argument 
-DLLVM_ENABLE_PROJECTS="mlir;compiler-rt;lld" is needed.

-Johan
Oct 02 2019
parent reply Roberto Rosmaninho <Robertogrosmaninho gmail.com> writes:
On Thursday, 3 October 2019 at 06:22:35 UTC, Johan Engelen wrote:


 This doesn't look quite right.
 You should `ninja install` LLVM (set an install path using 
 -DCMAKE_INSTALL_PREFIX="..." when running cmake to generate the 
 ninja build file), such that you get a local LLVM installation 
 that contains also the MLIR stuff. Are you sure that MLIR is 
 built when building LLVM? Perhaps cmake argument 
 -DLLVM_ENABLE_PROJECTS="mlir;compiler-rt;lld" is needed.

 -Johan


Actually, this is a complex question. Today, to get MLIR, you 
must clone LLVM and MLIR from different repositories by cloning 
MLIR into the LLVM projects[1]. The expectation is to have MLIR 
into LLVM 11 and then you will be able to get the project using 
the -DLLVM_ENABLE_PROJECTS, which will download the project on 
the same projects directory used today and referred on my CMake. 
I believe your point is that if users don't get this project on 
they LLVM, then they won't be able to use LDC, please correct me 
if I get it wrong. Well, the thing is I have to use some header 
files to implement MLIR to LDC, maybe I can put some Macros to 
not compile those files or codes that use MLIR if that's the main 
issue.

Regards,
Roberto Rosmaninho

[1] https://github.com/tensorflow/mlir#getting-started-with-mlir
Oct 03 2019
parent reply Johan <j j.nl> writes:
On Thursday, 3 October 2019 at 12:58:38 UTC, Roberto Rosmaninho 
wrote:

 On Thursday, 3 October 2019 at 06:22:35 UTC, Johan Engelen 
 wrote:


 This doesn't look quite right.
 You should `ninja install` LLVM (set an install path using 
 -DCMAKE_INSTALL_PREFIX="..." when running cmake to generate 
 the ninja build file), such that you get a local LLVM 
 installation that contains also the MLIR stuff. Are you sure 
 that MLIR is built when building LLVM? Perhaps cmake argument 
 -DLLVM_ENABLE_PROJECTS="mlir;compiler-rt;lld" is needed.

 -Johan


 Actually, this is a complex question. Today, to get MLIR, you 
 must clone LLVM and MLIR from different repositories by cloning 
 MLIR into the LLVM projects[1]. The expectation is to have MLIR 
 into LLVM 11 and then you will be able to get the project using 
 the -DLLVM_ENABLE_PROJECTS, which will download the project on 
 the same projects directory used today and referred on my CMake.


I know how MLIR is to be included into the LLVM source directory. 
This is the same method that compiler-rt and clang used to be 
before the single big LLVM repo happened recently.
But I think that also means that mlir will be installed in the 
LLVM installation dir. So instead of searching in 
"${LLVM_ROOT_DIR}/../projects/mlir" (which I think will only work 
with your directory layout), you should be able to find the 
include files in "${LLVM_ROOT_DIR}/include/mlir" no?
The benefit of that is that your MLIR branch will work for others 
that have mlir too (even if a distribution carries MLIR, without 
a user having to compiler it).


 I believe your point is that if users don't get this project on 
 they LLVM, then they won't be able to use LDC, please correct 
 me if I get it wrong. Well, the thing is I have to use some 
 header files to implement MLIR to LDC, maybe I can put some 
 Macros to not compile those files or codes that use MLIR if 
 that's the main issue.


Separate issue, but indeed. Simply do something like this in 
CMake and #ifdef code that cannot be compiled without MLIR:
if(LLVM_MLIR_FOUND)
     message(STATUS "Building with MLIR support")
     add_definitions("-DLDC_LLVM_MLIR_ENABLED")
endif()

(that's exactly how SPIR-V support is added to LDC)

cheers,
   Johan
Oct 03 2019
parent reply Roberto Rosmaninho <Robertogrosmaninho gmail.com> writes:
On Thursday, 3 October 2019 at 18:38:35 UTC, Johan wrote:


 I know how MLIR is to be included into the LLVM source 
 directory. This is the same method that compiler-rt and clang 
 used to be before the single big LLVM repo happened recently.
 But I think that also means that mlir will be installed in the 
 LLVM installation dir. So instead of searching in 
 "${LLVM_ROOT_DIR}/../projects/mlir" (which I think will only 
 work with your directory layout), you should be able to find 
 the include files in "${LLVM_ROOT_DIR}/include/mlir" no?
 The benefit of that is that your MLIR branch will work for 
 others that have mlir too (even if a distribution carries MLIR, 
 without a user having to compiler it).


I see, but neither clang nor compiler-rt has any code in 
"${LLVM_ROOT_DIR}/include/".  I think the problem here is that 
${LLVM_ROOT_DIR} has the following path 
"~/llvm-project/llvm/build" (at least this is the path that I got 
here - this llvm-project is the new pattern), so to get separate 
projects like clang, compiler-rt , and mlir, we need to go back 
to llvm directory, enter the project directory and select the 
project we want. As far as I know, this is the build pattern for 
llvm, so anyone who has already installed mlir will benefit from 
it.


 Separate issue, but indeed. Simply do something like this in 
 CMake and #ifdef code that cannot be compiled without MLIR:
 if(LLVM_MLIR_FOUND)
     message(STATUS "Building with MLIR support")
     add_definitions("-DLDC_LLVM_MLIR_ENABLED")
 endif()


Done! Thanks again!

Regards,
Roberto Rosmaninho
Oct 04 2019
parent reply Johan Engelen <j j.nl> writes:
On Friday, 4 October 2019 at 13:07:56 UTC, Roberto Rosmaninho 
wrote:

 On Thursday, 3 October 2019 at 18:38:35 UTC, Johan wrote:

 I see, but neither clang nor compiler-rt has any code in 
 "${LLVM_ROOT_DIR}/include/".  I think the problem here is that 
 ${LLVM_ROOT_DIR} has the following path 
 "~/llvm-project/llvm/build" (at least this is the path that I 
 got here - this llvm-project is the new pattern), so to get 
 separate projects like clang, compiler-rt , and mlir, we need 
 to go back to llvm directory, enter the project directory and 
 select the project we want. As far as I know, this is the build 
 pattern for llvm, so anyone who has already installed mlir will 
 benefit from it.


You should "install" the LLVM that you build yourself, e.g. by 
make install or ninja install. And then point LDC to that 
installation of LLVM instead of to the build dir. LLVM_ROOT_DIR 
should point to the installation dir, not to your build dir. That 
is the expected setup when building LDC. Can you try that?

-Johan
Oct 04 2019
parent reply Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Friday, 4 October 2019 at 14:55:08 UTC, Johan Engelen wrote:

 On Friday, 4 October 2019 at 13:07:56 UTC, Roberto Rosmaninho 
 wrote:

 On Thursday, 3 October 2019 at 18:38:35 UTC, Johan wrote:

 I see, but neither clang nor compiler-rt has any code in 
 "${LLVM_ROOT_DIR}/include/".  I think the problem here is that 
 ${LLVM_ROOT_DIR} has the following path 
 "~/llvm-project/llvm/build" (at least this is the path that I 
 got here - this llvm-project is the new pattern), so to get 
 separate projects like clang, compiler-rt , and mlir, we need 
 to go back to llvm directory, enter the project directory and 
 select the project we want. As far as I know, this is the 
 build pattern for llvm, so anyone who has already installed 
 mlir will benefit from it.


 You should "install" the LLVM that you build yourself, e.g. by 
 make install or ninja install. And then point LDC to that 
 installation of LLVM instead of to the build dir. LLVM_ROOT_DIR 
 should point to the installation dir, not to your build dir. 
 That is the expected setup when building LDC. Can you try that?

 -Johan


I'd like to point out that that the wiki [1] seems confuse people.
If one follows the standard procedure described (the one I had 
initially
followed and that Roberto followed if I understood correctly),
the instructions make the user not install LLVM. Then, it makes 
them
install LDC using a temporary location of LLVM (check especially
the highlighted snippets). That in turn seems to cause problems.

Should we change that ?

[1] https://wiki.dlang.org/Building_LDC_from_source
Oct 04 2019
parent reply Johan <j j.nl> writes:
On Friday, 4 October 2019 at 17:37:23 UTC, Stefanos Baziotis 
wrote:

 
 I'd like to point out that that the wiki [1] seems confuse 
 people.
 If one follows the standard procedure described (the one I had 
 initially
 followed and that Roberto followed if I understood correctly),
 the instructions make the user not install LLVM. Then, it makes 
 them
 install LDC using a temporary location of LLVM (check especially
 the highlighted snippets). That in turn seems to cause problems.

 Should we change that ?


Ah indeed. Yes please!
Oct 04 2019
parent Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Friday, 4 October 2019 at 18:32:55 UTC, Johan wrote:

 On Friday, 4 October 2019 at 17:37:23 UTC, Stefanos Baziotis 
 wrote:

 
 I'd like to point out that that the wiki [1] seems confuse 
 people.
 If one follows the standard procedure described (the one I had 
 initially
 followed and that Roberto followed if I understood correctly),
 the instructions make the user not install LLVM. Then, it 
 makes them
 install LDC using a temporary location of LLVM (check 
 especially
 the highlighted snippets). That in turn seems to cause 
 problems.

 Should we change that ?


 Ah indeed. Yes please!


Ok, done!
Oct 04 2019
prev sibling parent reply Roberto Rosmaninho <Robertogrosmaninho gmail.com> writes:
Hi everyone, here goes another update from my project!

This week I was able to build the support for more BinOps:
  - ModExp
  - AndExp
  - OrExp
  - XorExp
  - EqualExp
  - CmpExp(slt,sle,sgt,sge,ult,ule,ugt,uge,neq,eq)

And the last three days I worked to implement Ifstatement witch 
cost me a lot of time to figure out the right way to do that due 
the lack of documentation of implementation (like a doxygen) and 
none examples provided by MLIR. The next week I plan to finish 
those BinOps, replace the opaque BinOps for those provided by a 
StandardDialect from MLIR and start to build the D Dialect. My 
primary goal for now it will be build the ForStatement support so 
we will be able to have some cool tests like matrix 
multiplication or some like that to put on D test suit.

Hopefully next week I'll bring this test to show you, but for now 
you can track my work on 
https://github.com/Robertorosmaninho/ldc/tree/fix-llvm-10 and 
maybe provide some ideas or show me something that I'm not doing 
right.

About the cmake problem on FindMLIR.cmake the one way that I 
could think about solve was passing the MLIR path as cmake 
argument with -DMLIR_ROOT_DIR=/path/to/dir and then all the 
directories necessaries for MLIR works will be set.

Best regards,
Roberto Rosmaninho
Oct 25 2019
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Saturday, 26 October 2019 at 00:00:51 UTC, Roberto Rosmaninho 
wrote:

 Hi everyone, here goes another update from my project!

 About the cmake problem on FindMLIR.cmake the one way that I 
 could think about solve was passing the MLIR path as cmake 
 argument with -DMLIR_ROOT_DIR=/path/to/dir and then all the 
 directories necessaries for MLIR works will be set.


I wouldn't waste too much more time on that, MLIR will soon be 
moving to the mono-repo[1], and so will pretty much always be in 
the same place.

[1]: https://github.com/llvm/llvm-project
Oct 25 2019
parent reply Roberto Rosmaninho <Robertogrosmaninho gmail.com> writes:
Hi everyone, here I know that I'm not writing for a while and I 
apologize for that, but here it's what I've been working on the 
past few weeks:

  - Add the notion of blocks and control flow: Support to 
IfStatement and ForStatement
  - Add support to fill different blocks
  - Refactoring of MLIRDeclaration to infer the type of each 
declaration as is used on MLIRStatement
  - Support for Template Interface
  - Improving on AssignExp
  - Support for PostExp: MinusMinus and PlusPlus
  - Support for Assign Bin Ops
  - Introduction to D Dialect: Creation of Dialect Class
  - Dialect Registration
  - Creation of Ops.td a file written in mlir-tablegen style 
(ODS[1]) to define Ops on D Dialect, for now only AddExp and 
MulExp are implemented for int{8,16,32,64} and float{16,32,64} 
types
  - Support to bool, float and different sizes of int on the 
entire project, other types depend on the creation of new types, 
MLIR doesn't support char, null, none by default.

Hopefully, I'll finish writing the Ops.td until next week, then 
I'll try to build my first MLIR pass to improve ForStatement with 
arguments on the block which replace the phi functions of LLVM.

Suggestions, ideas, and code reviews are more than welcome!
[1]Table-driven Operation Definition Specification (ODS) - 
https://github.com/tensorflow/mlir/blob/master/g3doc/OpDefinitions.md#table-driven-operation-definition-specification-ods

GitHub of the project: 
https://github.com/Robertorosmaninho/ldc/tree/fix-llvm-10

Thanks for your attention,
Roberto Rosmaninho
Nov 20 2019
parent Roberto Rosmaninho <Robertogrosmaninho gmail.com> writes:
Hi everyone,

Sorry for not writing updates in a while, on this stage, I'm more 
fixing bugs than making advances on the project. However, 
finally, all opaque operations are now dialect operations!! It 
means that the operations are no more just string, they have 
builders, types, verifiers and can return error messages if you 
try to build something wrong. More than that, now it's possible 
to start a new phase that we can translate D to other dialects, 
including those ones used by TensorFlow. For the end of this 
project, I may try to build the code that will translate D 
Dialect for the LLVM IR dialect and see how correlated it is with 
LLVM IR.

For now, I have just 2 more bugs to fix: IfStatement that 
recently had some updates on LDC and generates arguments for 
blocks to work as phi nodes for loops and branches.

Some tests are available and using LIT with FileCheck, so if you 
want to how D Dialect looks like the tests ate under the 
ldc/tests/codegen_mlir directory on this repository: 
https://github.com/Robertorosmaninho/ldc/tree/support-for-MLIR

The project works with the last version of LLVM, MLIR and LDC - 
with commits until 12/14/20

Regards,
Roberto Rosmaninho
Dec 16 2019