Difference between revisions of "GDC/Development/DevelopmentEnvironment"

From D Wiki
Jump to: navigation, search
(Added intro)
(Update gcc version to 7 (from 4!!))
 
(19 intermediate revisions by 4 users not shown)
Line 2: Line 2:
  
 
== Introduction ==
 
== Introduction ==
This page provides instructions on how to set up a development environment for hacking the GDC compiler.  
+
This page provides instructions on how to set up a development environment for hacking the GDC compiler.  If you encounter any problems while following these instructions, you may be able to get help at the [http://forum.dlang.org/group/D.gnu D.GNU] forum. If you can improve these instructions, please do so by making an account on this wiki and authoring changes.
  
 
== Prerequesites ==
 
== Prerequesites ==
Line 21: Line 21:
  
 
== Creating a Directory Structure ==
 
== Creating a Directory Structure ==
These instructions will do a native build.  If you wish to build a cross-compiler, see [[GDC/Cross_Compiler]] and modify these scripts and instructions per your target's requirements.
+
These instructions will do a native Linux build.  If you wish to build a cross-compiler, see [[GDC/Cross_Compiler]] and modify these scripts and instructions per your target's requirements.
  
 
These instructions will use the following directory structure.
 
These instructions will use the following directory structure.
 
<pre>
 
<pre>
 
$HOME/
 
$HOME/
  gdc-src/
+
. gdc-src/
    gdc/
+
. . gcc/
    gcc/
+
. . gdc/
    build/
+
. . build/
  usr/
+
. usr/
 
</pre>
 
</pre>
  
* <code>$HOME/gdc-src/gdc/</code> - This directory will contain the GDC source code downloaded from a GCC mirror
+
* <code>$HOME/gdc-src/gcc/</code> - This directory will contain the GCC source code downloaded from a GCC mirror
* <code>$HOME/gdc-src/gcc/</code> - This directory will contain the GCC source code clone from the GDC GitHub repository
+
* <code>$HOME/gdc-src/gdc/</code> - This directory will contain the GDC source code cloned from the GDC GitHub repository
 
* <code>$HOME/gdc-src/build/</code> - <code>configure</code> should not be run within the <code>$HOME/gdc-src/gcc/</code> directory, so <code>$HOME/gdc-src/build/</code> will be used to for running <code>configure</code> and <code>make</code>
 
* <code>$HOME/gdc-src/build/</code> - <code>configure</code> should not be run within the <code>$HOME/gdc-src/gcc/</code> directory, so <code>$HOME/gdc-src/build/</code> will be used to for running <code>configure</code> and <code>make</code>
 
* <code>$HOME/usr/</code> - This directory will contain the final GDC toolchain binaries, libraries, etc...
 
* <code>$HOME/usr/</code> - This directory will contain the final GDC toolchain binaries, libraries, etc...
Line 41: Line 41:
 
<syntaxhighlight lang="bash">
 
<syntaxhighlight lang="bash">
 
mkdir $HOME/gdc-src/
 
mkdir $HOME/gdc-src/
 +
mkdir $HOME/gdc-src/gcc/
 
mkdir $HOME/gdc-src/gdc/
 
mkdir $HOME/gdc-src/gdc/
mkdir $HOME/gdc-src/gcc/
 
 
mkdir $HOME/gdc-src/build/
 
mkdir $HOME/gdc-src/build/
 
mkdir $HOME/usr/
 
mkdir $HOME/usr/
 
</syntaxhighlight>
 
</syntaxhighlight>
  
== Obtain the GCC Source Code and Extract it to <code>$HOME/gdc-src/gcc/</code> ==
+
== Obtain the GCC Source Code and Extract it ==
 
<syntaxhighlight lang="bash">
 
<syntaxhighlight lang="bash">
 
cd $HOME/gdc-src/
 
cd $HOME/gdc-src/
 
export GCC_MIRROR=http://ftp.tsukuba.wide.ad.jp/software/gcc/releases
 
export GCC_MIRROR=http://ftp.tsukuba.wide.ad.jp/software/gcc/releases
export GCC_NAME=gcc-4.9.1
+
export GCC_NAME=gcc-7.1.0
 
export GCC_SOURCE_ARCHIVE=$GCC_NAME.tar.bz2
 
export GCC_SOURCE_ARCHIVE=$GCC_NAME.tar.bz2
 
wget $GCC_MIRROR/$GCC_NAME/$GCC_SOURCE_ARCHIVE
 
wget $GCC_MIRROR/$GCC_NAME/$GCC_SOURCE_ARCHIVE
Line 64: Line 64:
 
cd $HOME/gdc-src/gdc/
 
cd $HOME/gdc-src/gdc/
 
git clone https://github.com/D-Programming-GDC/GDC.git .
 
git clone https://github.com/D-Programming-GDC/GDC.git .
git checkout gdc-4.9
+
git checkout gdc-7
 
</syntaxhighlight>
 
</syntaxhighlight>
  
If you plan on submitting pull requests, may want to first fork the GDC GitHub repository, and then clone from your fork.
+
If you plan to submit pull requests, you may want to first fork the GDC GitHub repository, and then clone from your fork.
  
 
== Add GDC to GCC ==
 
== Add GDC to GCC ==
Line 115: Line 115:
 
</syntaxhighlight>
 
</syntaxhighlight>
  
== Modifying the Source Code ==
+
== Modifying the Source Code and Doing an Incremental Build ==
GDC source code will still remain in <code>$HOME/gdc-src/gdc</code> and GCC source code will still be in <code>$HOME/gdc-src/gcc</code>.  Hack away!  When you're finished modifying the source code, you can do an incremental build by simply running the above make command, with the following caveats:
+
GDC source code will remain in <code>$HOME/gdc-src/gdc</code> and GCC source code will remain in <code>$HOME/gdc-src/gcc</code>.  Hack away!   
 
+
 
Adding/Removing files from libphobos or libdruntime require you to re-run the <code>setup-gcc.sh</code> script
+
When you're finished modifying the source code, you can do an incremental build by simply running the above <code>make</code> command, with the following caveats:
<syntaxhighlight lang="bash">
 
cd $HOME/gdc-src/gdc/
 
./setup-gcc.sh --update ../gcc
 
</syntaxhighlight>
 
  
 
In certain circumstances (updating the frontend mostly), changes to <code>dfrontend/idgen.c</code> or <code>dfrontend/impcnvgen.c</code> sometimes do not trigger rebuilds of other sources.  When this occurs, crashes or errors will look rather confusing.  That's because the symbol table between two object files differ. Clean out all objects from the gdc build directory and re-make it.
 
In certain circumstances (updating the frontend mostly), changes to <code>dfrontend/idgen.c</code> or <code>dfrontend/impcnvgen.c</code> sometimes do not trigger rebuilds of other sources.  When this occurs, crashes or errors will look rather confusing.  That's because the symbol table between two object files differ. Clean out all objects from the gdc build directory and re-make it.
  
Changes to just GDC only require a rebuild of <code>cc1d</code> - or the 'D compiler proper':
+
Changes to just GDC only require a rebuild of the D front-end components:
 
<syntaxhighlight lang="bash">
 
<syntaxhighlight lang="bash">
 
cd $HOME/gdc-src/build/
 
cd $HOME/gdc-src/build/
make -C gcc cc1d
+
make -C gcc d
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Line 136: Line 132:
 
cd $HOME/gdc-src/build/
 
cd $HOME/gdc-src/build/
 
make all-target-libphobos
 
make all-target-libphobos
 +
</syntaxhighlight>
 +
 +
Adding/Removing files from libphobos or libdruntime require you to re-run the <code>setup-gcc.sh</code> script
 +
<syntaxhighlight lang="bash">
 +
cd $HOME/gdc-src/gdc/
 +
./setup-gcc.sh --update ../gcc
 
</syntaxhighlight>
 
</syntaxhighlight>
  
 
== Cleaning ==
 
== Cleaning ==
To clean out gdc/cc1d only from $HOME/gdc-src/build:
+
To clean out <code>gdc/cc1d</code> only from <code>$HOME/gdc-src/build</code>:
 
<syntaxhighlight lang="bash">
 
<syntaxhighlight lang="bash">
 
cd $HOME/gdc-src/build/
 
cd $HOME/gdc-src/build/
rm gcc/gdc gcc/d/*
+
make -C gcc d.mostlyclean
 
</syntaxhighlight>
 
</syntaxhighlight>
 
    
 
    
Line 152: Line 154:
  
 
== Debugging GDC ==
 
== Debugging GDC ==
Though GCC does allow you to debug the compiler using <code>-wrapper</code>, it may be quicker to obtain the actual command passed to the cc1d and pass that to the debugger.  To obtain that command run:
+
Though GCC does allow you to debug the compiler using <code>-wrapper</code>, it may be quicker to obtain the actual command passed to <code>cc1d</code> and pass that to the debugger.  To obtain that command run:
 
<syntaxhighlight lang="bash">
 
<syntaxhighlight lang="bash">
 
$HOME/usr/bin/gdc -pipe -c --verbose program.d
 
$HOME/usr/bin/gdc -pipe -c --verbose program.d
Line 171: Line 173:
  
 
See also [https://gcc.gnu.org/wiki/DebuggingGCC Debugging GCC] on the GCC wiki.
 
See also [https://gcc.gnu.org/wiki/DebuggingGCC Debugging GCC] on the GCC wiki.
 +
 +
== Regenerating GCC patches ==
 +
The GCC patches need to be regenerated when additional changes are introduced in the patches or when the patches don't match the GCC source anymore.
 +
* Use a checkout of the GCC git repository
 +
* Checkout the correct tag, e.g. for gcc-7.1.0
 +
* Apply old patch
 +
** make sure to apply patches in the same order as in setup-gcc.sh
 +
** e.g. when modifying patch-gcc-7.1.x
 +
*** first apply patch-toplev-7.1.x
 +
*** commit
 +
*** then apply patch-gcc-7.1.x
 +
* Make modifications
 +
* Generate patch, see below:
 +
 +
<syntaxhighlight lang="bash">
 +
# Check for whitespace errors
 +
git diff --check
 +
 +
# Generate patch
 +
git diff --no-prefix > /path/to/patch
 +
 +
# (Only look in gcc dir)
 +
git diff --no-prefix gcc > /path/to/patch
 +
 +
# Remove git index lines
 +
sed '/^index /d' -i /path/to/patch
 +
 +
# Manually add back the patch description at the top of the file
 +
$EDITOR /path/to/patch
 +
</syntaxhighlight>
 +
 +
 +
[[Category:GDC Compiler]]

Latest revision as of 19:59, 22 June 2017


Introduction

This page provides instructions on how to set up a development environment for hacking the GDC compiler. If you encounter any problems while following these instructions, you may be able to get help at the D.GNU forum. If you can improve these instructions, please do so by making an account on this wiki and authoring changes.

Prerequesites

The minimum system requirements you need to obtain and build GDC are:

  • gcc and g++
  • libmpc-dev
  • libmpfr-dev
  • libgmp3-dev
  • autoconf and automake
  • flex and bison
  • patch
  • git

For cross-compiling/multilib builds, you should also have installed:

  • gcc-multilib
  • (maybe g++-multilib)

Creating a Directory Structure

These instructions will do a native Linux build. If you wish to build a cross-compiler, see GDC/Cross_Compiler and modify these scripts and instructions per your target's requirements.

These instructions will use the following directory structure.

$HOME/
. gdc-src/
. . gcc/
. . gdc/
. . build/
. usr/
  • $HOME/gdc-src/gcc/ - This directory will contain the GCC source code downloaded from a GCC mirror
  • $HOME/gdc-src/gdc/ - This directory will contain the GDC source code cloned from the GDC GitHub repository
  • $HOME/gdc-src/build/ - configure should not be run within the $HOME/gdc-src/gcc/ directory, so $HOME/gdc-src/build/ will be used to for running configure and make
  • $HOME/usr/ - This directory will contain the final GDC toolchain binaries, libraries, etc...

Run the following commands to create the directory structure

mkdir $HOME/gdc-src/
mkdir $HOME/gdc-src/gcc/
mkdir $HOME/gdc-src/gdc/
mkdir $HOME/gdc-src/build/
mkdir $HOME/usr/

Obtain the GCC Source Code and Extract it

cd $HOME/gdc-src/
export GCC_MIRROR=http://ftp.tsukuba.wide.ad.jp/software/gcc/releases
export GCC_NAME=gcc-7.1.0
export GCC_SOURCE_ARCHIVE=$GCC_NAME.tar.bz2
wget $GCC_MIRROR/$GCC_NAME/$GCC_SOURCE_ARCHIVE
tar xjfv $GCC_SOURCE_ARCHIVE -C gcc
rm $GCC_SOURCE_ARCHIVE

You may wish to use an alternate GCC mirror.

Obtain the GDC Source Code

cd $HOME/gdc-src/gdc/
git clone https://github.com/D-Programming-GDC/GDC.git .
git checkout gdc-7

If you plan to submit pull requests, you may want to first fork the GDC GitHub repository, and then clone from your fork.

Add GDC to GCC

cd $HOME/gdc-src/gdc/
./setup-gcc.sh ../gcc

Configure GCC

cd $HOME/gdc-src/build/
../gcc/configure         \
  --enable-languages=d   \
  --disable-bootstrap    \
  --prefix=/usr          \
  --disable-multilib     \
  --disable-libgomp      \
  --disable-libmudflap   \
  --disable-libquadmath

Other configure arguments you might want to use:

--[enable|disable]-nls          # Native language support (NLS).  Lets GDC output diagnostics in languages other than English.
--[enable|disable|-libssp       # Build GCC with stack smashing protection.
--[enable|disable]-lto          # Support link-time optimization.
--[enable|disable]-ld           # Build the ld linker.
--[enable|disable]-gold         # Build the gold linker.
--[enable|disable]-multilib     # Allow building both 32-bit and 64-bit apps/libs.
--[enable|disable]-shared       # Build shared libraries.
--with-multilib-list=mXX,mYY    # Select multilibs, eg: --with-multilib-list=m32,m64,mx32 (AArch64 and x86-64 only)

Documentation on the various configure options can be found in GCC's documentation here

Build GCC

cd $HOME/gdc-src/build/
make CXXFLAGS="-g3 -O0" -j2 2>&1 | tee build.log

Install GCC

cd $HOME/gdc-src/build/
export DESTDIR=$HOME
make install

Modifying the Source Code and Doing an Incremental Build

GDC source code will remain in $HOME/gdc-src/gdc and GCC source code will remain in $HOME/gdc-src/gcc. Hack away!

When you're finished modifying the source code, you can do an incremental build by simply running the above make command, with the following caveats:

In certain circumstances (updating the frontend mostly), changes to dfrontend/idgen.c or dfrontend/impcnvgen.c sometimes do not trigger rebuilds of other sources. When this occurs, crashes or errors will look rather confusing. That's because the symbol table between two object files differ. Clean out all objects from the gdc build directory and re-make it.

Changes to just GDC only require a rebuild of the D front-end components:

cd $HOME/gdc-src/build/
make -C gcc d

Changes to libphobos/libdruntime can be done using make:

cd $HOME/gdc-src/build/
make all-target-libphobos

Adding/Removing files from libphobos or libdruntime require you to re-run the setup-gcc.sh script

cd $HOME/gdc-src/gdc/
./setup-gcc.sh --update ../gcc

Cleaning

To clean out gdc/cc1d only from $HOME/gdc-src/build:

cd $HOME/gdc-src/build/
make -C gcc d.mostlyclean

To clean out libphobos/libdruntime:

cd $HOME/gdc-src/build/
make clean-target-libphobos

Debugging GDC

Though GCC does allow you to debug the compiler using -wrapper, it may be quicker to obtain the actual command passed to cc1d and pass that to the debugger. To obtain that command run:

$HOME/usr/bin/gdc -pipe -c --verbose program.d

Noteworthy functions to break at to catch ICE's:

internal_error
gcc_unreachable

Noteworthy functions for debugging trees/gimple:

debug_tree
debug_generic_expr
debug_gimple_stmt

See also Debugging GCC on the GCC wiki.

Regenerating GCC patches

The GCC patches need to be regenerated when additional changes are introduced in the patches or when the patches don't match the GCC source anymore.

  • Use a checkout of the GCC git repository
  • Checkout the correct tag, e.g. for gcc-7.1.0
  • Apply old patch
    • make sure to apply patches in the same order as in setup-gcc.sh
    • e.g. when modifying patch-gcc-7.1.x
      • first apply patch-toplev-7.1.x
      • commit
      • then apply patch-gcc-7.1.x
  • Make modifications
  • Generate patch, see below:
# Check for whitespace errors
git diff --check

# Generate patch
git diff --no-prefix > /path/to/patch

# (Only look in gcc dir)
git diff --no-prefix gcc > /path/to/patch

# Remove git index lines
sed '/^index /d' -i /path/to/patch

# Manually add back the patch description at the top of the file
$EDITOR /path/to/patch