Difference between revisions of "GDC Development"

From D Wiki
Jump to: navigation, search
(→‎GDC's glue code)
m (Fix "DMD_Source_Guide" link title)
 
(15 intermediate revisions by 5 users not shown)
Line 1: Line 1:
−
= Developer Documentation =
+
== Developer Documentation ==
  
 
At the moment, documentation for GDC, especially the internals, is sparse.
 
At the moment, documentation for GDC, especially the internals, is sparse.
Line 5: Line 5:
 
This page will hopefully help provide insight on GDC's internals.
 
This page will hopefully help provide insight on GDC's internals.
  
−
----
+
=== GDC Internals ===
−
 
 
−
== GDC Internals ==
 
  
 
GDC is essentially divided into 3 parts.
 
GDC is essentially divided into 3 parts.
Line 15: Line 13:
 
* GDC's glue code
 
* GDC's glue code
  
−
=== The DMD frontend ===
+
==== The DMD frontend ====
  
−
* The dmd and dmd2 subfolders contain D1 and D2 frontend source code respectively.
+
* [[DMD Source Guide]]
−
* On Wiki4D, there is the start of a  [//prowiki.org/wiki4d/wiki.cgi?DMDSourceGuide DMD Source Guide] to look at for now.
+
* In DMD, src/mars.c holds the main function for compiler. The main for GCC function can be found gcc/main.c. It calls the toplev::main() in gcc/toplev.c, which uses a generic main function for GCC front ends that don't implement their own.
−
* In DMD, mars.c holds the main function for compiler. In GDC, this is surrounded #ifndef IN_GCC, meaning that if we are using GCC, don't compile that part in. The main for GDC function can be found gcc/main.c. It calls the toplev_main() function in gcc/toplev.c, which uses a generic main function for GCC front ends that don't implement their own.
 
  
−
=== GCC's Internals ===
+
==== GCC's Internals ====
  
 
* http://gcc.gnu.org/wiki/GettingStarted
 
* http://gcc.gnu.org/wiki/GettingStarted
  
−
=== GDC's glue code ===
+
==== GDC's glue code ====
  
 
* This is the bridge between the DMD frontend and GCC
 
* This is the bridge between the DMD frontend and GCC
−
* Read the [[http://prowiki.org/wiki4d/wiki.cgi?DMDSourceGuide | DMD Source Guide]], especially section Intermediate representation as a start.
+
* Read the [//prowiki.org/wiki4d/wiki.cgi?DMDSourceGuide DMD Source Guide], especially section Intermediate representation as a start.
 
* Also look at the [[GDC/Hacking|GDC/Hacking]] entry for information about GDC's glue code.
 
* Also look at the [[GDC/Hacking|GDC/Hacking]] entry for information about GDC's glue code.
  
−
==== Files ====
+
===== Files =====
  
 
* d-codegen[.cc/.h]:
 
* d-codegen[.cc/.h]:
Line 42: Line 39:
 
** Simliar to irstate[.c/.h] in DMD's source, but not quite. In DMD, irstate[.c/.h] are only for the struct IRState, and do not inherit from any other structs. In GDC, d-irstate[.cc/.h] implements "IRBase", which inherits from Object. In d-codegen[.cc/.h], IRState is then implemented.
 
** Simliar to irstate[.c/.h] in DMD's source, but not quite. In DMD, irstate[.c/.h] are only for the struct IRState, and do not inherit from any other structs. In GDC, d-irstate[.cc/.h] implements "IRBase", which inherits from Object. In d-codegen[.cc/.h], IRState is then implemented.
  
−
----
+
=== Updating GDC ===
  
−
== Updating GDC ==
+
==== Merging a new DMD/Phobos version ====
 +
 
 +
When a new DMD version is out, we will try to get GDC up to date as soon as possible. The instructions are as follows: (From David Friedman)
  
−
=== Merging a new DMD/Phobos version ===
+
<blockquote>
 +
1. Fetch the sources.
 +
<syntaxhighlight lang="bash">
 +
git clone https://github.com/D-Programming-GDC/GDC
 +
git clone https://github.com/D-Programming-Language/dmd
 +
git clone https://github.com/D-Programming-Language/phobos
 +
git clone https://github.com/D-Programming-Language/druntime
 +
</syntaxhighlight>
  
−
When a new DMD version is out, we will try to get GDC up to date as soon as possible. The instructions are as follows: (From David Friedman)
+
2.
 +
Diff the DMD, Druntime and Phobos sources from the previously merged version:
 +
<syntaxhighlight lang="bash">
 +
(
 +
  cd dmd/src/root
 +
  git diff v2.066.1..v2.067.0 --relative > ../../../dmd.patch
 +
  cd ..
 +
  git diff v2.066.1..v2.067.0 --relative *{c,h,txt} >> ../../dmd.patch
 +
  cd ..
 +
  git diff v2.066.1..v2.067.0 test > ../dmd-test.patch
 +
)
 +
(cd druntime; git diff v2.066.1..v2.067.0 src > ../druntime.patch)
 +
(cd phobos; git diff v2.066.1..v2.067.0 etc/c/*.d std > ../phobos.patch)
 +
</syntaxhighlight>
  
−
<pre>
+
3. Do steps 4 and 5 first for the compiler, then for Druntime and then for Phobos.  It helps to work on these parts separately.
−
1. Extract the new dmd-xxx.zip package.
+
Sometimes compiler changes will require the new runtime library to work correctly, however.
 +
Also, I don't merge too many versions at a time. For example, I would probably do 2.066+2.067, and then 2.068.  
  
−
2. Diff the compiler source from the previously merged dmd version (not
+
4. Depending on the nature of the changes, it may be possible to simply apply the changes to the gdc version as a patch. Otherwise, make changes manually.
−
gdc's d/dmd or d/dmd2 directory).  You may have to fix line endings in
+
Because some of the changes I made a pretty drastic, it will probably be easier to applies these incremental changes to gdc rather than try to re-apply my changes to the original dmd code.
−
the new dmd source to get a clean diff.
 
  
−
3. Depending on the nature of the changes, it may be possible to simply
+
<syntaxhighlight lang="bash">
−
apply the changes to the gdc version as a patch. Otherwise, make
+
# For DMD:
−
changes manually.
+
(
 +
  cd GDC/gcc/d/dfrontend
 +
  patch -p1< ../../../../dmd.patch -l --verbose
 +
  cd ../../testsuite/gdc.test
 +
  patch -p2< ../../../../dmd-test.patch -l --verbose
 +
)
  
−
Because some of the changes I made a pretty drastic, it will probably be
+
# For Druntime:
−
easier to applies these incremental changes to gdc rather than try to
+
(cd GDC/libphobos/libdruntime; patch -p2< ../../../druntime.patch -l --verbose)
−
re-apply my changes to the original dmd code.
 
  
−
4. Get the new compiler working and run regression tests.
+
# For Phobos:
 +
(cd GDC/libphobos/src; patch -p1< ../../../phobos.patch -l --verbose)
 +
</syntaxhighlight>
  
−
5. Do steps 2 and 3 for Phobos.  It helps to work on the compiler and  
+
It will patch all the files it can, and will place all the statement that can't be merged in **file.rej** where file is a filename.
−
Phobos separately. Sometimes compiler changes will require the new
 
−
runtime library to work correctly, however.
 
  
−
Also, I don't merge too many versions at a time. For example, I would
+
5. Get the component working and run regression tests.
−
probably do  2.015, 2.016+2.017, and then 2.018.  The array operations
 
−
in 2.018 should well tested before moving on.
 
  
 
I don't really have any other documentation.  I do have some scripts to  
 
I don't really have any other documentation.  I do have some scripts to  
Line 79: Line 100:
  
 
David
 
David
−
</pre>
+
</blockquote>
−
 
 
−
If you are using D2, you will also need to repeat the process for Druntime.
 
−
 
 
−
You can find all the DMD, Phobos, and Druntime versions in the [[http://ftp.digitalmars.com/|Digitalmars FTP]].
 
−
 
 
−
To create a more readable diff you can use this command:
 
−
<syntaxhighlight lang="bash">
 
−
diff -Naruw dmdX dmdX+1 > dmdX-to-dmdX+1.patch
 
−
</syntaxhighlight>
 
−
 
 
−
Where **X** is a dmd version.
 
−
 
 
−
To apply the generated patch in the most automatic way run patch as follows in the corresponding DMD or Phobos dir:
 
−
<syntaxhighlight lang="bash">
 
−
patch -p4 -l -F 3 --verbose < dmdX-to-dmdX+1.patch
 
−
</syntaxhighlight>
 
−
 
 
−
It will patch all the files it can, and will place all the statement that can't be merged in **file.rej** where file is a filename.
 
  
 
=== Merging a new GCC version ===
 
=== Merging a new GCC version ===
Line 103: Line 106:
 
New GCC versions come out on occasion as well, but can require a bit more work to set up than a new DMD version. Usually, you only need to make patches for a 4.x version, and not for every 4.x.x version. Here are the instructions given from David Friedman:
 
New GCC versions come out on occasion as well, but can require a bit more work to set up than a new DMD version. Usually, you only need to make patches for a 4.x version, and not for every 4.x.x version. Here are the instructions given from David Friedman:
  
−
<pre>
+
<blockquote>
 
The first thing to do is modify d/setup-gcc.sh to allow building with  
 
The first thing to do is modify d/setup-gcc.sh to allow building with  
 
the new version.
 
the new version.
Line 128: Line 131:
 
'patch-toplev-<version>'.  The other is for thee gcc/ directory  
 
'patch-toplev-<version>'.  The other is for thee gcc/ directory  
 
'patch-gcc-<version>'.
 
'patch-gcc-<version>'.
−
</pre>
+
</blockquote>
  
 
You can find GCC versions in the [//ftp.gnu.org/gnu/gcc GNU FTP].
 
You can find GCC versions in the [//ftp.gnu.org/gnu/gcc GNU FTP].
−
 
−
----
 
  
 
== Repository Guidelines ==
 
== Repository Guidelines ==
Line 138: Line 139:
 
These are the repository guidelines that a committer should follow while committing to the GDC repository.
 
These are the repository guidelines that a committer should follow while committing to the GDC repository.
  
−
The '''default''' branch must never be broken but should contain a working version of the compiler. If you want to develop a new feature or do a merging with a newer GCC or DMD or Phobos, it's strongly encuraged that you create your own branch. If you don't know about mercurial branches, [[this small tutorial will fill all the knowledge requirements to create a new branch|http://hgbook.red-bean.com/read/managing-releases-and-branchy-development.html]].  
+
The '''default''' branch must never be broken but should contain a working version of the compiler. If you want to develop a new feature or do a merging with a newer GCC or DMD or Phobos, it's strongly encuraged that you create your own branch. If you don't know about mercurial branches, [//hgbook.red-bean.com/read/managing-releases-and-branchy-development.html this small tutorial] will fill all the knowledge requirements to create a new branch.  
  
 
It's better if you give to your branch a short name representing the topic of that branch. You can commit in anytime to that branch and broke it, people interested in the same feature will work in the same branch. When a feature is fully developed and stable, where stable means that it compiles, it can be merged to the '''default''' branch. After the merging, before the default branch is committed to the central repository, you have to build and test that the '''default''' branch compiles too.
 
It's better if you give to your branch a short name representing the topic of that branch. You can commit in anytime to that branch and broke it, people interested in the same feature will work in the same branch. When a feature is fully developed and stable, where stable means that it compiles, it can be merged to the '''default''' branch. After the merging, before the default branch is committed to the central repository, you have to build and test that the '''default''' branch compiles too.
Line 144: Line 145:
 
After that you can delete your branch from the repository.
 
After that you can delete your branch from the repository.
  
−
----
+
== See also ==
 +
* [[GDC]]
 +
* [[GDC/Hacking|GDC Hacking]]
  
 
== External Links ==
 
== External Links ==
Line 150: Line 153:
 
For what may not have been explained here well enough, here are some other links that have some useful information:
 
For what may not have been explained here well enough, here are some other links that have some useful information:
  
−
[//prowiki.org/wiki4d/wiki.cgi?GdcHacking GDC Hacking]
+
* [//prowiki.org/wiki4d/wiki.cgi?DMDSourceGuide DMD Source Guide]
−
 
+
* [//gcc.gnu.org/wiki GCC Wiki]
−
[//prowiki.org/wiki4d/wiki.cgi?DMDSourceGuide DMD Source Guide]
+
* [//blog.lxgcc.net/?page_id=116 GCC front-end guide]
−
 
 
−
[//gcc.gnu.org/wiki GCC Wiki]
 
  
−
[//blog.lxgcc.net/?page_id=116 GCC front-end guide]
+
[[Category:GDC Compiler]]

Latest revision as of 12:32, 21 February 2016

Developer Documentation

At the moment, documentation for GDC, especially the internals, is sparse. The DMD frontend and the GCC internals aren't very well documented either. This page will hopefully help provide insight on GDC's internals.

GDC Internals

GDC is essentially divided into 3 parts.

  • The DMD Frontend
  • GCC's Internals
  • GDC's glue code

The DMD frontend

  • DMD Source Guide
  • In DMD, src/mars.c holds the main function for compiler. The main for GCC function can be found gcc/main.c. It calls the toplev::main() in gcc/toplev.c, which uses a generic main function for GCC front ends that don't implement their own.

GCC's Internals

GDC's glue code

  • This is the bridge between the DMD frontend and GCC
  • Read the DMD Source Guide, especially section Intermediate representation as a start.
  • Also look at the GDC/Hacking entry for information about GDC's glue code.
Files
  • d-codegen[.cc/.h]:
    • Implements IRState.
  • d-glue.cc:
    • Implements all the methods described there in the DMD source guide, including all Statement::toIR and Expression::toElem methods thus, replacing dmd's toelem.c, e2ir.c, s2ir.c, etc.
  • d-irstate[.cc/.h]:
    • Simliar to irstate[.c/.h] in DMD's source, but not quite. In DMD, irstate[.c/.h] are only for the struct IRState, and do not inherit from any other structs. In GDC, d-irstate[.cc/.h] implements "IRBase", which inherits from Object. In d-codegen[.cc/.h], IRState is then implemented.

Updating GDC

Merging a new DMD/Phobos version

When a new DMD version is out, we will try to get GDC up to date as soon as possible. The instructions are as follows: (From David Friedman)

1. Fetch the sources. 

git clone https://github.com/D-Programming-GDC/GDC
git clone https://github.com/D-Programming-Language/dmd
git clone https://github.com/D-Programming-Language/phobos
git clone https://github.com/D-Programming-Language/druntime

2. Diff the DMD, Druntime and Phobos sources from the previously merged version: 

(
  cd dmd/src/root
  git diff v2.066.1..v2.067.0 --relative > ../../../dmd.patch
  cd ..
  git diff v2.066.1..v2.067.0 --relative *{c,h,txt} >> ../../dmd.patch
  cd ..
  git diff v2.066.1..v2.067.0 test > ../dmd-test.patch
)
(cd druntime; git diff v2.066.1..v2.067.0 src > ../druntime.patch)
(cd phobos; git diff v2.066.1..v2.067.0 etc/c/*.d std > ../phobos.patch)

3. Do steps 4 and 5 first for the compiler, then for Druntime and then for Phobos. It helps to work on these parts separately. Sometimes compiler changes will require the new runtime library to work correctly, however. Also, I don't merge too many versions at a time. For example, I would probably do 2.066+2.067, and then 2.068.

4. Depending on the nature of the changes, it may be possible to simply apply the changes to the gdc version as a patch. Otherwise, make changes manually. Because some of the changes I made a pretty drastic, it will probably be easier to applies these incremental changes to gdc rather than try to re-apply my changes to the original dmd code. 

# For DMD:
(
  cd GDC/gcc/d/dfrontend
  patch -p1< ../../../../dmd.patch -l --verbose
  cd ../../testsuite/gdc.test
  patch -p2< ../../../../dmd-test.patch -l --verbose
)

# For Druntime:
(cd GDC/libphobos/libdruntime; patch -p2< ../../../druntime.patch -l --verbose)

# For Phobos:
(cd GDC/libphobos/src; patch -p1< ../../../phobos.patch -l --verbose)

It will patch all the files it can, and will place all the statement that can't be merged in **file.rej** where file is a filename.

5. Get the component working and run regression tests.

I don't really have any other documentation. I do have some scripts to help with the diffing process.

David

Merging a new GCC version

New GCC versions come out on occasion as well, but can require a bit more work to set up than a new DMD version. Usually, you only need to make patches for a 4.x version, and not for every 4.x.x version. Here are the instructions given from David Friedman:

The first thing to do is modify d/setup-gcc.sh to allow building with the new version.

Next, you should try to apply the set of patches in d/patches/ for the previous version. For the top-level-xxx patch, just manually apply the changes to Makefile.def and configure.ac. You will need to get the autogen package to rebuild Makefile.in. Rebuilding the top-level 'configure' script with autoconf may be tricky, so you just apply the one-line change directly to 'configure'.

You probably will need to create a d-bi-attrs-4x.h. This is select portion of gcc/c-common.c modified to for D. (Conditionally) included it in d-builtins.c.

Most of the work is then getting GDC to work correctly. Any changes you make that are version specific should probably involve the use of the D_GCC_VER macro. Search for "D_GCC_VER" to get a feel for what you will need to do.

Once you gotten everything working and are done making changes to the GCC sources, you need to create a set of patches for the new version. The for FSF GCC, there are two patches: One is the top-level, 'patch-toplev-<version>'. The other is for thee gcc/ directory 'patch-gcc-<version>'.

You can find GCC versions in the GNU FTP.

Repository Guidelines

These are the repository guidelines that a committer should follow while committing to the GDC repository.

The default branch must never be broken but should contain a working version of the compiler. If you want to develop a new feature or do a merging with a newer GCC or DMD or Phobos, it's strongly encuraged that you create your own branch. If you don't know about mercurial branches, this small tutorial will fill all the knowledge requirements to create a new branch.

It's better if you give to your branch a short name representing the topic of that branch. You can commit in anytime to that branch and broke it, people interested in the same feature will work in the same branch. When a feature is fully developed and stable, where stable means that it compiles, it can be merged to the default branch. After the merging, before the default branch is committed to the central repository, you have to build and test that the default branch compiles too.

After that you can delete your branch from the repository.

See also

External Links

For what may not have been explained here well enough, here are some other links that have some useful information:

Retrieved from "https://wiki.dlang.org/?title=GDC_Development&oldid=7270"