Difference between revisions of "GDC Development"

From D Wiki
Jump to: navigation, search
m (O3o moved page GDC/Development to GDC Development)
m (Fix "DMD_Source_Guide" link title)
 
(2 intermediate revisions by the same user not shown)
Line 15: Line 15:
 
==== The DMD frontend ====
 
==== The DMD frontend ====
  
* The dmd and dmd2 subfolders contain D1 and D2 frontend source code respectively.
+
* [[DMD Source Guide]]
* On Wiki4D, there is the start of a  [//prowiki.org/wiki4d/wiki.cgi?DMDSourceGuide DMD Source Guide] to look at for now.
+
* In DMD, src/mars.c holds the main function for compiler. The main for GCC function can be found gcc/main.c. It calls the toplev::main() in gcc/toplev.c, which uses a generic main function for GCC front ends that don't implement their own.
* In DMD, mars.c holds the main function for compiler. In GDC, this is surrounded #ifndef IN_GCC, meaning that if we are using GCC, don't compile that part in. The main for GDC function can be found gcc/main.c. It calls the toplev_main() function in gcc/toplev.c, which uses a generic main function for GCC front ends that don't implement their own.
 
  
 
==== GCC's Internals ====
 
==== GCC's Internals ====
Line 47: Line 46:
  
 
<blockquote>
 
<blockquote>
1. Extract the new dmd-xxx.zip package.
+
1. Fetch the sources.
 +
<syntaxhighlight lang="bash">
 +
git clone https://github.com/D-Programming-GDC/GDC
 +
git clone https://github.com/D-Programming-Language/dmd
 +
git clone https://github.com/D-Programming-Language/phobos
 +
git clone https://github.com/D-Programming-Language/druntime
 +
</syntaxhighlight>
 +
 
 +
2.
 +
Diff the DMD, Druntime and Phobos sources from the previously merged version:
 +
<syntaxhighlight lang="bash">
 +
(
 +
  cd dmd/src/root
 +
  git diff v2.066.1..v2.067.0 --relative > ../../../dmd.patch
 +
  cd ..
 +
  git diff v2.066.1..v2.067.0 --relative *{c,h,txt} >> ../../dmd.patch
 +
  cd ..
 +
  git diff v2.066.1..v2.067.0 test > ../dmd-test.patch
 +
)
 +
(cd druntime; git diff v2.066.1..v2.067.0 src > ../druntime.patch)
 +
(cd phobos; git diff v2.066.1..v2.067.0 etc/c/*.d std > ../phobos.patch)
 +
</syntaxhighlight>
  
2. Diff the compiler source from the previously merged dmd version (not
+
3. Do steps 4 and 5 first for the compiler, then for Druntime and then for PhobosIt helps to work on these parts separately.
gdc's d/dmd or d/dmd2 directory)You may have to fix line endings in
+
Sometimes compiler changes will require the new runtime library to work correctly, however.
the new dmd source to get a clean diff.
+
Also, I don't merge too many versions at a time. For example, I would probably do 2.066+2.067, and then 2.068.  
  
3. Depending on the nature of the changes, it may be possible to simply  
+
4. Depending on the nature of the changes, it may be possible to simply apply the changes to the gdc version as a patch. Otherwise, make changes manually.
apply the changes to the gdc version as a patch. Otherwise, make  
+
Because some of the changes I made a pretty drastic, it will probably be easier to applies these incremental changes to gdc rather than try to re-apply my changes to the original dmd code.
changes manually.  
 
  
Because some of the changes I made a pretty drastic, it will probably be
+
<syntaxhighlight lang="bash">
easier to applies these incremental changes to gdc rather than try to
+
# For DMD:
re-apply my changes to the original dmd code.
+
(
 +
  cd GDC/gcc/d/dfrontend
 +
  patch -p1< ../../../../dmd.patch -l --verbose
 +
  cd ../../testsuite/gdc.test
 +
  patch -p2< ../../../../dmd-test.patch -l --verbose
 +
)
  
4. Get the new compiler working and run regression tests.
+
# For Druntime:
 +
(cd GDC/libphobos/libdruntime; patch -p2< ../../../druntime.patch -l --verbose)
  
5. Do steps 2 and 3 for Phobos. It helps to work on the compiler and
+
# For Phobos:
Phobos separately. Sometimes compiler changes will require the new
+
(cd GDC/libphobos/src; patch -p1< ../../../phobos.patch -l --verbose)
runtime library to work correctly, however.
+
</syntaxhighlight>
  
Also, I don't merge too many versions at a time. For example, I would
+
It will patch all the files it can, and will place all the statement that can't be merged in **file.rej** where file is a filename.
probably do  2.015, 2.016+2.017, and then 2.018.  The array operations
+
 
in 2.018 should well tested before moving on.
+
5. Get the component working and run regression tests.
  
 
I don't really have any other documentation.  I do have some scripts to  
 
I don't really have any other documentation.  I do have some scripts to  
Line 76: Line 101:
 
David
 
David
 
</blockquote>
 
</blockquote>
 
If you are using D2, you will also need to repeat the process for Druntime.
 
 
You can find all the DMD, Phobos, and Druntime versions in the [//ftp.digitalmars.com Digitalmars FTP].
 
 
To create a more readable diff you can use this command:
 
<syntaxhighlight lang="bash">
 
diff -Naruw dmdX dmdX+1 > dmdX-to-dmdX+1.patch
 
</syntaxhighlight>
 
 
Where **X** is a dmd version.
 
 
To apply the generated patch in the most automatic way run patch as follows in the corresponding DMD or Phobos dir:
 
<syntaxhighlight lang="bash">
 
patch -p4 -l -F 3 --verbose < dmdX-to-dmdX+1.patch
 
</syntaxhighlight>
 
 
It will patch all the files it can, and will place all the statement that can't be merged in **file.rej** where file is a filename.
 
  
 
=== Merging a new GCC version ===
 
=== Merging a new GCC version ===

Latest revision as of 12:32, 21 February 2016

Developer Documentation

At the moment, documentation for GDC, especially the internals, is sparse. The DMD frontend and the GCC internals aren't very well documented either. This page will hopefully help provide insight on GDC's internals.

GDC Internals

GDC is essentially divided into 3 parts.

  • The DMD Frontend
  • GCC's Internals
  • GDC's glue code

The DMD frontend

  • DMD Source Guide
  • In DMD, src/mars.c holds the main function for compiler. The main for GCC function can be found gcc/main.c. It calls the toplev::main() in gcc/toplev.c, which uses a generic main function for GCC front ends that don't implement their own.

GCC's Internals

GDC's glue code

  • This is the bridge between the DMD frontend and GCC
  • Read the DMD Source Guide, especially section Intermediate representation as a start.
  • Also look at the GDC/Hacking entry for information about GDC's glue code.
Files
  • d-codegen[.cc/.h]:
    • Implements IRState.
  • d-glue.cc:
    • Implements all the methods described there in the DMD source guide, including all Statement::toIR and Expression::toElem methods thus, replacing dmd's toelem.c, e2ir.c, s2ir.c, etc.
  • d-irstate[.cc/.h]:
    • Simliar to irstate[.c/.h] in DMD's source, but not quite. In DMD, irstate[.c/.h] are only for the struct IRState, and do not inherit from any other structs. In GDC, d-irstate[.cc/.h] implements "IRBase", which inherits from Object. In d-codegen[.cc/.h], IRState is then implemented.

Updating GDC

Merging a new DMD/Phobos version

When a new DMD version is out, we will try to get GDC up to date as soon as possible. The instructions are as follows: (From David Friedman)

1. Fetch the sources.

git clone https://github.com/D-Programming-GDC/GDC
git clone https://github.com/D-Programming-Language/dmd
git clone https://github.com/D-Programming-Language/phobos
git clone https://github.com/D-Programming-Language/druntime

2. Diff the DMD, Druntime and Phobos sources from the previously merged version:

(
  cd dmd/src/root
  git diff v2.066.1..v2.067.0 --relative > ../../../dmd.patch
  cd ..
  git diff v2.066.1..v2.067.0 --relative *{c,h,txt} >> ../../dmd.patch
  cd ..
  git diff v2.066.1..v2.067.0 test > ../dmd-test.patch
)
(cd druntime; git diff v2.066.1..v2.067.0 src > ../druntime.patch)
(cd phobos; git diff v2.066.1..v2.067.0 etc/c/*.d std > ../phobos.patch)

3. Do steps 4 and 5 first for the compiler, then for Druntime and then for Phobos. It helps to work on these parts separately. Sometimes compiler changes will require the new runtime library to work correctly, however. Also, I don't merge too many versions at a time. For example, I would probably do 2.066+2.067, and then 2.068.

4. Depending on the nature of the changes, it may be possible to simply apply the changes to the gdc version as a patch. Otherwise, make changes manually. Because some of the changes I made a pretty drastic, it will probably be easier to applies these incremental changes to gdc rather than try to re-apply my changes to the original dmd code.

# For DMD:
(
  cd GDC/gcc/d/dfrontend
  patch -p1< ../../../../dmd.patch -l --verbose
  cd ../../testsuite/gdc.test
  patch -p2< ../../../../dmd-test.patch -l --verbose
)

# For Druntime:
(cd GDC/libphobos/libdruntime; patch -p2< ../../../druntime.patch -l --verbose)

# For Phobos:
(cd GDC/libphobos/src; patch -p1< ../../../phobos.patch -l --verbose)

It will patch all the files it can, and will place all the statement that can't be merged in **file.rej** where file is a filename.

5. Get the component working and run regression tests.

I don't really have any other documentation. I do have some scripts to help with the diffing process.

David

Merging a new GCC version

New GCC versions come out on occasion as well, but can require a bit more work to set up than a new DMD version. Usually, you only need to make patches for a 4.x version, and not for every 4.x.x version. Here are the instructions given from David Friedman:

The first thing to do is modify d/setup-gcc.sh to allow building with the new version.

Next, you should try to apply the set of patches in d/patches/ for the previous version. For the top-level-xxx patch, just manually apply the changes to Makefile.def and configure.ac. You will need to get the autogen package to rebuild Makefile.in. Rebuilding the top-level 'configure' script with autoconf may be tricky, so you just apply the one-line change directly to 'configure'.

You probably will need to create a d-bi-attrs-4x.h. This is select portion of gcc/c-common.c modified to for D. (Conditionally) included it in d-builtins.c.

Most of the work is then getting GDC to work correctly. Any changes you make that are version specific should probably involve the use of the D_GCC_VER macro. Search for "D_GCC_VER" to get a feel for what you will need to do.

Once you gotten everything working and are done making changes to the GCC sources, you need to create a set of patches for the new version. The for FSF GCC, there are two patches: One is the top-level, 'patch-toplev-<version>'. The other is for thee gcc/ directory 'patch-gcc-<version>'.

You can find GCC versions in the GNU FTP.

Repository Guidelines

These are the repository guidelines that a committer should follow while committing to the GDC repository.

The default branch must never be broken but should contain a working version of the compiler. If you want to develop a new feature or do a merging with a newer GCC or DMD or Phobos, it's strongly encuraged that you create your own branch. If you don't know about mercurial branches, this small tutorial will fill all the knowledge requirements to create a new branch.

It's better if you give to your branch a short name representing the topic of that branch. You can commit in anytime to that branch and broke it, people interested in the same feature will work in the same branch. When a feature is fully developed and stable, where stable means that it compiles, it can be merged to the default branch. After the merging, before the default branch is committed to the central repository, you have to build and test that the default branch compiles too.

After that you can delete your branch from the repository.

See also

External Links

For what may not have been explained here well enough, here are some other links that have some useful information: