GSOC 2014 Ideas

Refer to our GSoC 2014 page for details about this project.

GSoC idea pages from past years (GSoC 2013 (not accepted), GSoC 2012, and GSoC 2011).

Plenty of challenging and important projects exist in the D worlds. They range from writing new or improving existing modules of D's standard library (Phobos), working on its compilers (Compilers), bringing D to mobile devices, shaping GUI libraries for D, integrating D with other languages and more. Happy hacking!

This is the most important part of our application, and where you can significantly improve our chance of being accepted this year. Please add your ideas to the list.

Phobos Modules

std.i18n

Design and implement a basic internationalization framework. It should provide at least the following functionality:

• A locale part, std.i18n.locale
  • detect a users default language
  • select a global application language
  • types to describe and work with locales
• A text translation part, std.i18n.translation
  • Gettext compatible
  • Preferably using the gettext serialized hashtables in .mo files
  • Low level, gettext like API and high level, boost::locale like API
• A tool to extract strings which need to be translated from D code
  • Prefereably based on DScanner (https://github.com/Hackerpilot/Dscanner)
  • Alternatively as a D program using regular expressions
  • Optionally adding support for D code to xgettext
  • It might also be possible to do with pragma(msg)... proof of concept http://arsdnet.net/dcode/i18n.d

Internationalization is required in all kinds of programs, therefore std.i18n must be as portable as possible. This especially means it must work on systems without a Garbage Collector and it must be easy to adapt the 'current locale detection' code and all code accessing the filesystem.

Think 'std.i18n should be portable to a system with 4MB Ram, no GC and no usual OS (e.g. game consoles)'

Large parts of the code can be ported from boost::locale.

std.compression

(Please insert requirements)

std.event

Design and implement a event IO system with switchable backends. At least on backend which works on all OSs D supports should be implemented (libevent, libev). A D backend which directly uses OS functions (epoll, ...) should be started. It should be considered how another event loop (gtk,...) can be integrated with this D event loop in a efficient way. For more requirements, see http://wiki.dlang.org/Event_system

std.xml and std.json replacement

Requirements for std.json:

• A pull parser (tokenizer)
  • Preferably based on std.lexer (http://hackerpilot.github.io/experimental/std_lexer/phobos/lexer.html)
  • Must work without any dynamic allocations
• A JSONWriter
  • Must accept JSONTokens produced by the tokenizer
  • Must provide a higher level API, JSONWriter.beginArray() ...
  • Must work without any dynamic allocations
  • Must provide basic pretty printing either in a different type (PrettyJSONWriter) or integrated into JSONWriter
• A DOM which does parsing/formatting via JSONTokenizer/JSONWriter
• Convenience functions
  • JSONObject parseJSON(Range input)
  • string printJSON(JSONObject)
  • void printJSON(JSONObject, Sink)
• simple! (de)serialization using JSONTokenizer/JSONWriter
  • Must not allocate any memory
  • struct MyStruct; void printJSON(T, Sink)
  • T parseJSON(T)(Range)

For XML a pull parser and a XMLWriter should be provided. If there's time left, a XML DOM and XML validation can be implemented. See also: http://wiki.dlang.org/Wish_list/std.json

std.socket

The existing std.socket module needs an overhaul. This includes fixing existing issues and improving its API. Keeping the module backward compatible and avoiding code breaking changes is important.

std.units

This projects seeks to provide physical units as part of D's standard library. Allowing the use of units simplifies programming and helps avoiding bugs. It has been a success in F# see Units of Measure in F#. For C++ Boost.Units is well known.

David Nadlinger (github) and Arlen Aghakians (github) have written std.units modules which can be of inspiration.

std.log

Having a simple and standard way of logging in D will help the development/debugging of D applications. The framework should be easy to use to foster its adoption by D developers. Google's Logging library for C++ can be of some inspiration.

There are two proposals: std.logger and std.log (see Review Queue). In accordance with the community a module should be finalized.

std.benchmark

Defining good benchmarks is a difficult task requiring understanding of low-level, fine-grained timing mechanisms that are OS specific. (It also requires a good strategy of what to benchmark, but unfortunately that is difficult to automate.) A benchmarking framework could just require the user to define benchmark functions (e.g. starting with 'benchmark_') and would take care of timing execution, doing multiple iterations etc.

Reviving the proposed std.benchmark (see Review Queue) and finishing it up for inclusion in Phobos is one option.

std.numeric.matrix

Cristi Cobzarenco created a fork of Lars Kyllingstad's SciD library. The goal was to create a matrix/linear algebra library that could be integrated into Phobos. Significant progress was made but the goals weren't fully met as the project was probably too ambitious for one summer. The goals for this summer would be the following or some subset thereof:

• Fixed-size matrices/vectors
• Sparse matrices and better support for packed matrices
• Better Lapack wrappers
• Most importantly, polish, debugging and submission for review and inclusion in Phobos within GSoC.

D Compilers

GDC

GDC is a D compiler based on the GCC backend. There are a few outstanding issues in GDC that have ended up in the backlog of known issues to fix, as well as changes that could be beneficial to upstream and other compilers.

Compiler Improvements

• Implement support for compiling with LTO (-flto) turned on.
• There are various XBUG markers in GDC's testsuite, some haven't been looked at in a long time. There should be fewer!
• Internal Documentation. Ideally every function should have a comment describing its use and input values. As the bulk of what happens in the glue is just providing for backend members declared in the front-end. Most should be pretty trivial to work through.
• Debugging Support. Enumerals of non-scalar types currently emit no debugging information. A solution should be proposed and implemented for this.
• Improving codegen for better optimisations. Currently all shared types are marked volatile, which is a workaround for not being able to prevent shared decls from being cached into registers by other means. Improvements could also be done around parameter passing, taking advantage of D storage attributes.

Library Improvements

• Get shared library support working. The current implementation upstream in DM druntime has not been tested or merged in, and it is not clear what changes are required to support it, or whether is even going to be feasibly possible with GDC.
• GDC EH support is lacking the ability to chain multiple thrown exceptions together in its deh personality routines.
• Now GCC has __atomic primitives defined for the new C++ memory model. We should take advance of these and update the API in gcc.atomics which hash out the old __synchronise primitives. The new __atomic primitives having a one-to-one mapping with core.atomic being just one of the benefits of this.
• More targets are expected to land in GDC pretty soon. In preparation for this, we should investigate an alternate layout of druntime before the long lists of version conditions becomes a problem for port maintainers (See #11666).

LDC

LDC is based on LLVM, so this suggests several small project ideas, like porting some of the LLVM tools/features designed for C/C++ to the D-LDC world. Things like:

Port LDC to a new platform

Requirements are:

• Update CMake files to support cross-compilation
• Make sure that code generation works for the choosen platform. This includes fixing blocking LLVM bugs discovered during porting.
• Platform-specific requirements:
  • For a traditional Linux platform (e.g. MIPS, SPARC, SystemZ):
    • Check and update the Posix bindings in Druntime.
    • Make sure the compiler test suite and Phobos/Druntime unit tests passes.
  • For a GPU platform (NVPTX, R600):
    • Create and implement a concept how to mix and compile kernel code with standard D. (A look at A simple tutorial on generating PTX assembler out of Ada source code using LLVM NVPTX backend might help here.)
    • Evaluate which runtime can be provided. (E.g. is the GC usable? Are the arrayOp functions usable?)
    • Based on your evaluation, port an existing runtime or create a minimalistic one.
    • Make sure the compiler test suite passes.
  • For a complete new platform (AIX on POWER7):
    • Check and update the OS support of LLVM. Compile with a recent gcc before trying to switch to platform-vendor compiler.
    • Contribute new OS binding files to Druntime.
    • Make sure the compiler test suite and Phobos/Druntime unit tests passes.
• The compiler should be usable as native and cross-compiler.

Improve optimization using D specific knowledge

Requirements are:

• Research new library optimizations and new code improvement patterns and implement them in SimplifyDRuntime pass. (Examples can be found in this forum thread)
• Improve code generation by using more stack allocation instead of heap allocation. This can be done by exploiting scope parameters or by improving the GarbageCollect2Stack pass
• Improve IR generation
  • Check alignment of allocations and enforce usage (see issue #531)
  • Currently, all parameters get some space on the stack allocated. Identify conditions when this is not needed and implement them.
• Create test cases to show the impact of the new optimizations.

Other ideas

• Perform more static analysis, like abstract compilation or linting (Clang does both);
• Develop a simple way to compile D for the browser with Emscripten to "asm.js" (this is a very small project);
• Perform other run-time tests as done by Clang;
• Use some of the D semantics to improve D code optimization (Walter talks about this all the time);
• Implement better high level optimizations for D code that uses array ops.

SDC

SDC is a work in progress in order to implement a D frontend in D. The lexing/parsing/semantic part is available as a lib in order to allow other tools to be built on top of it. The long term goal is to improve the tooling around D in general.

If SDC is to handle the most advanced features of D (CTFE, templates, static ifs, mixin), many things remain to do. They include:

• Full support of D's OOP capabilities.
• Context pointers creation (closures and delegate passed by alias).
• pure, @safe
• shared/synchronized.
• and much more!

A lot of work, but the project has a solid base to build upon.

SDC is leveraging LLVM in order to do JIT CTFE and for codegen. It is probably possible to mutually work around LLVM with LDC (for instance Emscripten and asm.js ).

Integration with other Languages

PyD

Once remaining issues with shared libraries are resolved, it becomes feasible to use D code from dynamic languages. Clearly it's vital to provide easy path for calling it from Python, as it's used by many scientists, amongst others. Although there's PyD, the last few years saw rising interest in PyPy because it shows much better performance than CPython. For PyPy, the way to go seems to be generating extern(C) wrappers for classes/functions/etc. + CFFI bindings for that C code. First part of this can be reused for producing C++ wrapping code -- this is similar in goals to dtoh but here entities to be wrapped should be explicitly mentioned. Extra benefit of producing C++ bindings is ability to call SWIG on them to support other languages. So, perhaps, C++ is the number one target in this respect.

Objective-C support

For D to be usable on iOS it's basically essential to be able to easily use the Objective-C system libraries and frameworks of the platform. GUI programming on Mac OS X would also get a big boost of this. The best way to do this is to add direct support for Objective-C in the compiler. Michel Fortin has already stared a project for this but it needs to be finished, updated with the latest compiler changes and merged back into the mainline.

D on Mobile/Embedded Devices

ARM support

The ARM architecture is predominant on mobile devices. Enabling D developement on those requires support from the compiler, from druntime and phobos and documentation on toolchain setup. There are existing approaches for GDC/LDC. The goal of this project would be to take up the existing work and get GDC/LDC to pass their testsuites on Linux and Android. Building a cross-compiler for these targets should become a simple and documented process. Additional time could be spent on exploring bare-metal targets or Windows support.

iPhone support

This should also exist alongside the Android support. The needs and objectives are probably similar, though less discussed.

Bare Metal D

This is a project that goes hand in hand with ARM support in D but focuses on getting bare metal cross-compiler and appealing eco-system to surround it. The goal is to raise the awareness of D in the open hardware community at large and prove D's systems programming lanugage title along the way.

It's hard not to notice the growing momentum of the open hardware movement. Yesterday's newbies awkwardly messing with theirs first Arduinos are today's mature embedded systems hackers. The hanger for ever more powerful systems takes them en mass towards ARM development boards. These systems are cheap, have 32bit CPUs and plenty of RAM/ROM so that D certainly can be a compelling offering in this setting (compared to C/C++ in usability).

The project is two-fold and can be geared towards any of the 2 major parts:

• Getting a bare-metal D runtime support on Cortex-M3/M4 wtih as many language features as possible. A minor victory would be having a minimal thread-less and GC-less runtime. A more ambitious plan here is creating a small-ish RTOS in D, to provide full runtime with threads, mutexes etc.

• Create a Wiring look alike library in D for a select family of popular chips (and consequently a range of boards). Alternativly instead of blindly copying a popular API is to design your own hardware abstraction library. The challenge here is to tackle things like GPIOs, USARTs, I2C, SPIs, PWM, ADC and other peripherals commonly found in MCUs in a future-proof way.

Cool demos (including public videos) are an obligatory part of the project ;)

For a starting point see e.g. this project (some D working on STM32 boards): https://bitbucket.org/timosi/minlibd

Third party (expected) libraries

Developers commonly expect a good set of libraries for common functionality. In the case of D, there is need for standard implementations of these. These don't necessarily need to be pushed into phobos however they should be almost de facto in nature.

GUI Library

D really needs an official, or de facto standard, GUI library. There are already various GUI libraries available, with different pros and cons and in different states. What needs to be done varies quite much depending on which library is chosen. A few of the existing libraries are: QtD, GtkD, DWT, wxD.

Of these four libraries, only DWT and wxD use native operations to draw their widgets. Of these libraries only DWT is completely written in D. The rest of the libraries are only bindings to C/C++ libraries. There have been many discussions about which library would be best for D. My suggestion is to use DWT because it's completely written in D and it uses native operators to draw its widgets.

What can be done for DWT:

• Finish the Mac OS X port
• Port to 64bit
• Update to later versions of SWT
• Port to Qt, QtWebkit

Content Management System

One of D's current main targets is web development. A CMS in of itself is not doable for a project. However broken down into sub projects it is. Sub components that are required are:

• Routing
• ORM
• Form validation
• XSS filtering
• Email recieve / sending
• File uploading
• Localization
• Pagination
• Caching
• Barcode generator
• Qr code generator
• Captcha
• Mysql, Postgresql, sqlite, Oracle
• Rss
• Ldap auth
• SOAP/wsdl
• Sitemap generator
• Mime handling
• Theming
• Oauth/OpenID consumer/provider
• XMPP
• Menus
• Editable pages
• Wiki
• Templating mechanism
• Reloadable routes/models and templates (most likely requires shared libraries)

This list was created based upon common denominators in PHP web service frameworks and libraries. Java and Ruby on Rails also follow this trend.

Some of these are already worked on by Vibe. This includes email sending, file uploading, mime handling and routing. There is also other projects towards these aims. These include Cmsed, dvorm, hibernated and vibe_recaptcha. However some of these need some work to make them fully usable in their goals. Some of these modules would be better suited towards phobos modules. This includes localization and perhaps couple others.