GDC/CurrentReleaseTasks

This is a list of project ideas for GDC - potential improvements, new features, et cetera. that are in relation to a release merge.

DMD 2.067

Overview

The release of D 2.067 marks the half-way point in DMD's development that moves towards having a frontend compiler that is entirely self-hosted, and has many glue methods stripped out and converted into either flat functions or double-dispatch Visitors.

From GDC's perspective, this means that many symbols that had to be provided for the sake of source compatibility are no longer required. So rather than the straight-forward conversion opted by upstream (examples including Statement::toIR -> Statement_toIR, Declaration::toDebug -> toDebug), instead the end-goal is to jump at the opportunity to rewrite many parts of the glue to be free of DMD-isms. This results in the following:

• Less code, no more horrible stubs.
• Frees us from the DMD naming conventions, such as toElem, toDebug, toImport.
• Potentially re-writing major parts to follow upstream GCC C++ conventions - as at the end of the day, a merge with GCC is still on the table, and on a personal note, I don't feel up to doing this again without ensuring this is done.
• Allows a logic/flow that is more simplified and agrees more with our orientation around GCC trees.
• Did I already mention less code?

Remove DMD backend structures

• struct IRState: (done)
  • Move methods to either IRVisitor in toir.c or as flat codegen routines in d-codegen.cc.
  • Move members of IRState to cfun->language, as there is only one IRState per function (unlike DMD).
  • Move members of struct Flow into struct binding_level as there is no distinction between the two for codegen purposes.
  • Calls to ::toElem should be replaced with 'toElem(NULL)' once all dependencies on the irs parameter have been removed.
• 'struct block: (done)
  • Needs upstream co-operation: https://github.com/D-Programming-Language/dmd/pull/4854
  • Logical place to put all labels (both user and case) is in cfun->language
• struct dt_t:
  • Should just be a case of replacing all instances in d-todt.cc with tree
• struct Symbol:
  • Move members to struct lang_decl
• struct elem:
  • Should just be a case of replacing all instances in d-elem.cc with tree

Visitor conversions

• Not necessarily a straight conversion, must ensure that code follows GCC C++ coding conventions: https://gcc.gnu.org/codingconventions.html#Cxx_Conventions
• Names of visitors should match what it is generating code for, which may not necessarily be the name DMD gives it, some examples:
  •  ::toIR -> StatementVisitor -> stmt.cc
  •  ::toCtype -> TypeVisitor -> types.cc
  •  ::toElem -> ExpressionVisitor -> expr.cc
  •  ::toSymbol -> DeclarationVisitor -> decl.cc
• Some D frontend structures (Type, Declaration) have fields that are intended for backends to cache generated code, it should be possible to instead use a hash_map to cache these, allowing us to remove these backend-specific fields from the frontend.
  • Needs upstream co-operation: https://github.com/D-Programming-Language/dmd/pull/4569
• Provide a build_* function as the entry point for backend visitors.

Compiler

GDC

• Integrate GDC Visitors and structures into the GCC garbage collector.
  • Remove current workaround of using d_keep as a global bucket-list of all generated trees.
• Identify where functions need to be declared because they are called from the D frontend (d-glue.cc, d-typinf.cc).
  • Functions that don't touch any part of GDC/GCC should be moved into the frontend so as to be part of DDMD.
  • Functions that do touch GDC/GCC should be considered candidates for Target, however some exceptions to this are error routines.
• GDC sources have their dependencies managed automatically by the build systems. The D frontend sources should be fixed to work in the same way. (done)
• d-dmd-gcc.h:
  • This needs to go, all GDC defined functions should have been moved to Target, all DMD defined functions need to be declared properly in frontend headers.
• d-todt.cc:
  • Remove the two-tier construction of initializers. This should be as straight-forward as possible, using CONSTRUCTOR_APPEND_ELT(elt, NULL, init) directly, rather than going through the dt_* helpers.

DMD/Frontend

• Pool together a list of upstream changes that depend on backend codegen modifications.
• Merge all changes between 2.066.1 -> 2.067.1 in upstream into dfrontend
• Independent sources or changes can be done ahead of time whilst GDC is preparing for the visitor switch-over.
  • rmem.c: Possibly other root library sources too.
  • cppmangle.c: Needed in order to build DDMD converted sources.
• Merge in changes to the D2 testsuite, being mindful not to remove GDC tests (eg: asm { })
• Review uses of IN_GCC: If changes have been (or can be) made which allow for their removal, this should be done. However relevant upstream PRs should be raised to ensure that our copy is as close to being aligned with upstream as possible.

Library

• Improve build scripts for libphobos
• Allow for building libphobos outside the normal GCC build (still requires GCC sources). (done)
• Remove all *.di handling in build scripts, don't install modules as *.di in the system. (done)
• Identify and remove modules from the build that don't actually need compiling, mostly this is just gcc and core.stdc bindings.
  • Not sure if this is a good idea, if upstream adds template helpers to these modules later on things could break: See discussion at https://github.com/D-Programming-Language/dmd/pull/4851 Johannes Pfau (talk)
  • I think it should be fine. We don't generate any helpers, only the moduleinfo function which builds up the linked list of runtime modules. So for us it only makes sense to build in with the library if modules have any static ctors, static dtors, unittests, or functions. This actually has me thinking, how dependent on are we on 'genmoduleinfo' in the absence of any functions/classes (could we make reasonable guesses for useless moduleinfo objects, I think so). In any case with regards to changes in upstream, I tend to try to keep the lists of compiled modules in sync. Iain Buclaw (talk)

Platform Testing

• Test builds on all primary supported platforms, secondary platforms are also nice: https://gcc.gnu.org/gcc-5/criteria.html
• Actively tested platforms:
  • i686-pc-linux-gnu
  • x86_64-unknown-linux-gnu
• Platform that should already be widely tested (binary downloads are released):
  • arm-linux-gnueabi
• Platforms that need confirmed support:
  • For these platforms, virtual machines should be possible set-up using https://gmplib.org/~tege/qemu.html
    • i386-unknown-freebsd
    • mipsisa64-elf
    • powerpc64-unknown-linux-gnu
  • For these, virtual machine seems to only work with older versions (eg: http://tyom.blogspot.de/2009/12/solaris-under-qemu-how-to.html) - using Linux as per above is likely the easier option.
    • sparc-sun-solaris2.10
• Other Debian supported platforms (listed in order of priority):
  • These should all be possible to get working in a virtual machine except for s390x. Where libphobos is already being built and distributed, need to ensure that we haven't broken builds for them.
    • armel-linux (has libphobos package)
    • armhf-linux (has libphobos package)
    • aarch64-linux
    • mips-linux
    • mipsel-linux
    • powerpc-linux
    • s390x-linux
• Prebuilt qemu images
  • https://people.debian.org/~aurel32/qemu/ (armel, armhf, mips, mipsel, kfreebsd, powerpc, sparc (etch), sh4 (old debian sid))
  • I've built a sparc64 wheezy image, not yet published Johannes Pfau (talk)
  • https://npmccallum.fedorapeople.org/qemu/ppc64/debian/ (ppc64)
  • Instructions for arm64/aarch64: https://gist.github.com/philipz/04a9a165f8ce561f7ddd (Should build & upload image)

General Tidy

Some low priority tasks that don't stop anything if not done.

• Source files should have clearer names, possibly removing the 'd-' prefix on sources that are frontend-specific, keeping the prefix for common frontend sources.
• Use .cc for sources that interact with the frontend, and .c for sources that are independent to GDC code generation.
• Add a namespace gcc { namespace d { } } for common codegen routines.
• Encapsulating codegen routines into Visitor classes where it makes sense.