View source for SAOC 2018 ideas

This is a list of potential project ideas for applicants to the Symmetry Autumn of Code. Projects on this page are simply suggestions, not detailed proposals. It is up to any applicant who chooses one to flesh out the idea to a degree suitable for a project proposal as part of the SAoC application.

=== HTTP/2 for Vibe.d ===
----

HTTP/2 is a major revision of the HTTP network protocol. Among its advantages belong decreased latency due to (e.g. due to compressed HTTP headers or pipelined requests)
or server-push from the server which allows the server to send resources to the client before they are requested.
Therefore, [http://vibed.org Vibe.d] - D's flagship web application framework - would vastly profit from implementing HTTP/2.

=== SChannel (Windows CryptoAPI) for Vibe.d ===
----

For SSL/TLS Vibe.d supports OpenSSL and Botan. It would be great if also SChannel from Windows CryptoAPI could be supported. There are major benefits of using SChannel. In case a security issue is detected, Microsoft will solve this issue using Windows Updates. For OpenSSL/Botan developers needs to ship new versions of the application with the fixed Dlls.

You also need no additional Dlls to ship with your application.

=== Graphics library for resource constrained embedded systems ===
----

Create a 2D rasterizer, rich drawing primitives, and 2D graphics library suitable for resource constrained embedded systems (e.g. ARM Cortex-M) to be used in industrial controls, home appliances, medical devices, consumer electronics, and IoT just to name a few.  The end goal would be something similar to [https://www.segger.com/products/user-interface/emwin Segger's emWin].  The library would be used to drive LDCs similar to https://www.adafruit.com/product/3396
Requirements:

*  Hardware agnostic; should simply render to a frame buffer
*  No dependencies (No Phobos, no C standard library, and no official D runtime).
*  Consider using -betterC, but a custom minimal D runtime is also a viable option
 
Related work:

*  https://github.com/TurkeyMan/color
*  https://bitbucket.org/timosi/minlibd
*  https://github.com/JinShil/stm32f42_discovery_demo

References:

* [https://medium.com/@raphlinus/inside-the-fastest-font-renderer-in-the-world-75ae5270c445 Inside the fastest font renderer in the world]
* [http://nothings.org/gamedev/rasterize/ How the stb_truetpe Anti-Aliased Software Rasterizer v2 Works]
* [http://angusj.com/delphi/clipper.php Clipper - an open source freeware library for clipping and offsetting lines and polygons]

=== DUB: D's package manager ===
----

DUB - D's package manager - is one of the central infrastructure tools in the D world as a convenient build and package management tool for the D community.
With D gaining more and more popularity, it's also a key tool to guarantee D's further adoption.
Thus there are many great ideas to improve DUB:

* [https://github.com/dlang/dub/wiki/DEP2 separate compilation and caching]
* [https://github.com/dlang/dub/wiki/DEP3 extensibility through DUB]
* [https://github.com/dlang/dub/wiki/DEP4 control flow directives]
* [https://github.com/dlang/dub/wiki/DEP5 build system support for C/C++]
* [https://github.com/dlang/dub/wiki/DEP6 git support]
* support dub install for distributions
* support building a {snap, Debian, Arch, ...} package out of DUB package
* Add dub watch command to e.g. automatically restart vibe.d application if coding changes. PR was started 2015 but never completed (https://github.com/dlang/dub/pull/446)

=== Multi IDE debugger support (for windows) ===
----

Mago, [http://rainers.github.io/visuald/visuald/StartPage.html the debug engine used in VisualD], includes a tool called Mago-MI, which has a GDB compatible interface. Therefore, you can use Mago-MI on Windows as replacement for GDB. Several IDEs uses this feature to enable debugging with 1 code line for Windows/Linux/MacOS. It is used in an experimental state in IntelliJ. There is [https://code.visualstudio.com/ also support in Visual Studio Code] and [https://github.com/buggins/dlangide in DLangIDE for which it was originally built].

There are several issues which could be addressed in Symmetry Autumn of Code:

* Mago-MI is written in C++. This makes bug fixing hard. Rewriting it in D might make sense.

* While the installation of Mago-MI is easy if you want to debug OMF executables, it is very hard if you want to debug COFF executables. You need another executable from Mago, you have to register DLLs via regserv and you have to manually create a registry entry. An installation procedure for installing Mago-MI would be great.

* There are some bugs in Mago-MI and DMD (wrong debug information) which makes debugging hard ([https://github.com/rainers/mago/issues/21 see this issue] and [https://github.com/rainers/mago/issues/23 this one]). Also [https://github.com/rainers/mago/issues/14 Mago-Mi is missing some features]. There are more bugs but not investigated so far.

* As Visual Studio Code is already supported by the D Language Foundation, using this as a reference client of Mago-MI would make sense.

While this proposal seems only related to Windows, the nature of Mago-Mi is to enable IDEs having 1 code line for debugging on Windows/Linux/MacOS. Therefore, overall investment in this project is good for all platforms.

=== Convert DMD, DRuntime, and Phobos Makefiles to D ===
----

Building DMD, DRuntime, and Phobos on Windows is too cumbersome.  It appears to require the user to first set up a Posix environment and then battle environment configuration and other vague dependencies such as whether you are building for 32-bit or 64-bit and whether Visual Studio is installed.

Sometimes there are bugs that only manifest themselves on Windows, so they need to be investigated in a Windows environment.  To do that that we need to be able to build and troubleshoot dlang repositories on Windows without hassle.

It would be nice if the requirement for a Posix environment was removed by porting all Makefiles to D, so the same code (with appropriate <code>version</code>ing of course) could be used to compile and build dlang repositories on all platforms without a lot of environment setup and configuration.  All that would be required would be a recent D compiler in one's <code>PATH</code> and D's intrinsic dependencies (e.g. a C standard library).

Work on this has already been started at https://github.com/dlang/dmd/pull/8293.  It just needs to be taken to completion and duplicated to other dlang repositories (e.g. DRuntime and Phobos).

=== Re-implement Software Building Blocks (e.g. <code>memcpy</code>, <code>memset</code>, <code>memcmp</code>, <code>malloc</code>, <code>free</code>, etc.) in D ===
----

==== Statement of the problem ====
It is currently possible to dynamically set the length of an array in <code>@safe</code>, <code>pure</code> and <code>nothrow</code> D:

<syntaxhighlight lang=D>
void main() @safe pure nothrow
{
    int[] a;
    a.length = 10;  // No problem, but beware, the compiler is tricksy and false.
}
</syntaxhighlight>

That code gets lowered to a runtime hook [https://github.com/dlang/druntime/blob/cb5efa9854775c5a72acd6870083b16e5ebba369/src/rt/lifetime.d#L1442 _d_arraysetlengthT].  However that function is neither <code>@safe</code>, <code>pure</code> or <code>nothrow</code>.  The compiler is being deceitful.  This is just one of *many* such problems in the compiler/runtime interface.

Another problem with those runtime hooks is that they almost always require <code>TypeInfo</code>, a runtime entity. The primary reason for requiring <code>TypeInfo</code> is to get the size of the type.  The size of the type is a value that is known at compile-time, so there is no need to defer obtaining that information to runtime, but the runtime hooks were created before D had templates and many other meta-programming facilities.

==== Solution: Use templates ====
The solution to the problems stated above is to convert the runtime hooks to templates.  Also, having the runtime hooks available as templates would allow the language features they implement to be used in -betterC code, as the implementations will be generated at compile-time; no longer requiring linking in a pre-compiled runtime implementation.

==== Problem with the solution ====
If a runtime hook is converted to a template, and the compiler lowers expressions to that template, then the code will be verified in the semantic pass of the compiler.  That will force the the compiler to be honest; it will no longer be able to lie about <code>@safe</code>, <code>pure</code> and <code>nothrow</code>, as the semantic pass will verify it.

Unfortunately, this means that the runtime hooks (now templates) need to rewritten in a manner to ensure they are <code>@safe</code>, <code>pure</code> and <code>nothrow</code>.  That is quite difficult as many of the software building blocks employed by implementation of those runtime hooks (e.g. <code>memcpy</code>, <code>memset</code>, and <code>memcmp</code>) are not written to be used in a memory-safe language.  For example, to copy blocks of memory in D, one would likely use a function declared as <code>void memcpy(T[] dest, const T[] src)</code> not <code>void* memcpy(void* dest, const void* src, size_t n)</code>.

Those software building blocks are currently taken from the C standard library.  The fact that D requires the C standard library is an artificial dependency.  D, being a suitable alternative to C, should not need to rely on toolchains and libraries written in other languages.  It might be expedient to leverage such implementations, but it is not good design as it prevents D from building freestanding software, and complicates the software development environment as it becomes necessary to obtain and configure a C toolchain in addition to a D toolchain.

==== Solution to the problem with the solution ====
Therefore, to help address all of the issues above, it would be ideal to rewrite those software building blocks in <code>@safe</code>, <code>pure</code>, <code>nothrow</code>, typesafe, idiomatic D, so they can be used to implement the runtime hooks, compiler intrinsics, freestanding software, and more, while simplifying the development environment setup required to use D.

==== Other thoughts ====
It may be more feasible to choose one of two of these building block functions to keep the project within the scope and duration of the event.

<code>malloc</code>, <code>free</code>, and friends may be able to take implementation details from [https://dlang.org/phobos/std_experimental_allocator.html std.experimental.allocator].

==== References ====
* [https://github.com/JinShil/memcpyD A preliminary investigation of implementing <code>memcpy</code> in D]
* Agner Fog's [https://www.agner.org/optimize/optimizing_assembly.pdf Optimizing subroutines in assembly language, 17.9 Moving blocks of data].

=== Implement AVX2 and AVX-512 in DMD/DRuntime ===
----

[https://en.wikipedia.org/wiki/Advanced_Vector_Extensions AVX2 and AVX-512] are suspiciously absent from [https://github.com/dlang/druntime/blob/master/src/core/simd.d core.simd].  Implement them.  They will likely also be useful, if not necessary, to implement optimized block memory functions in the [https://wiki.dlang.org/SAOC_2018_ideas#Re-implement_Software_Building_Blocks_.28e.g._memcpy.2C_memset.2C_memcmp.2C_malloc.2C_free.2C_etc..29_in_D software building blocks proposal].

==== References ====
* [https://code.dlang.org/packages/simd std.experimental.simd implementation]
* [https://code.dlang.org/packages/intel-intrinsics intel-intrinsics DUB package]

=== Re-implement Software Building Blocks (e.g. <code>memcpy</code>, <code>memset</code>, <code>memcmp</code>, <code>malloc</code>, <code>free</code>, etc.) in D for the ARM Cortex-M ===
----

[https://github.com/JinShil/stm32f42_discovery_demo Work is currently underway] to use D as an alternative for microcontroller firmware programming for the ARM Cortex-M target.  However, like all targets, we need the fundamental software building blocks created.  

This project would implement one or more of <code>memcpy</code>, <code>memset</code>, <code>memcmp</code>, <code>malloc</code>, <code>free</code> in D so they can be used from <code>@safe</code>, <code>pure</code>, and <code>nothrow</code> code.  They  should be highly optimized, and will likely require some ARM Thumb inline assembly potentially leveraging [http://www.keil.com/support/man/docs/armasm/armasm_dom1359731150961.htm SIMD instructions] for the targets that support them.

=== Make WebAssembly a First-Class Target for D ===
----

Some progress has been made towards making WebAssembly a first-class target for D.  See [https://wiki.dlang.org/Generating_WebAssembly_with_LDC Generating WebAssembly with LDC].  However, it's just a start and a lot more work needs to be done.

=== Performance Improvements for DMD ===
----

Over the years, DMD's performance has degraded and currently requires excessive amounts of memory.  Profile DMD, suggest improvements, and implement them.

==== References ====
* [https://forum.dlang.org/post/ehyfilopozdndjdahdim@forum.dlang.org Forum Post - Compilation is taking a ton of memory]
* [https://forum.dlang.org/post/furvogninkjycltwjkzv@forum.dlang.org Forum Post - D compiles fast, right? Right?]]

=== Adding D support to gRPC ===
----

D support for gRPC should be added. The maintainers are willing to include D into their repository (https://github.com/grpc/grpc/tree/master/src).
Python, C++ or the higher level c wrapper (https://github.com/Juniper/grpc-c) could be used as blueprint.

==== References ====
* [https://github.com/andre2007/grpc/tree/feature/d/src/d Untested C wrapper]
* [https://github.com/grpc/grpc/issues/15786 Information regarding including D into gRPC repository]

=== Create a performance benchmarking and monitoring website ===
----
Create a performance benchmarking and monitoring website for tracking D's performance metrics over time.

Metrics may include.
* Compilation speed
* Binary size
* Compiler memory consumption
* Various runtime benchmarks stressing CPU, memory, or both
* Performance metrics for various projects/libraries in the D ecosystem

==== References ====
* [https://areweslimyet.com/ Mozilla's 'Are We Slim Yet']
* [http://blog.thecybershadow.net/2015/05/05/is-d-slim-yet/ An attempt to mimic 'Are We Slim Yet' in D]
* [https://benchmarksgame-team.pages.debian.net/benchmarksgame/ The benchmarks game]

=== Automate the collection and publishing of data for monitoring D's progress as a software development project ===
----

Find a way to automatically collect data from the D ecosystem and publish it as a set of metrics for measuring D's progress as a software development project.

Metrics may include.
* Number of pull requests per day/week/month/year
* Number of issues fixed per day/week/month/year
* Number of commits to any repository on GitHub or other source code repository websites
* Forum/Twitter/Facebook/Reddit/Hacker News/Search Engine and other social media activity

==== References ====
* [https://auto-tester.puremagic.com/chart.ghtml?projectid=1 Current metrics on D's autotester]
* [https://insights.stackoverflow.com/trends?tags=d Stack Overflow trends]
* [http://erdani.com/d/downloads.daily.png D's daily downloads]
* [https://api.github.com/search/repositories?q=stars:%3E0+forks:%3E0+size:%3E10+language:%22D%22+pushed:2017-01-01..2017-12-31 GitHub API - Repositories using D]

=== Port DRuntime to Priority Platforms for GDC ===
----
GDC needs [https://github.com/D-Programming-GDC/GDC/tree/master/libphobos/libdruntime it's druntime code] ported to more targets for inclusion in GCC.

Areas that need some love:
* <code>[https://github.com/D-Programming-GDC/GDC/blob/master/libphobos/libdruntime/core/internal/hash.d core.internal.hash]</code> (CTFE `real` hash)
* <code>[https://github.com/D-Programming-GDC/GDC/blob/master/libphobos/libdruntime/core/thread.d core.thread]</code> (notably fiber support in <code>[https://github.com/D-Programming-GDC/GDC/blob/master/libphobos/libdruntime/core/threadasm.S threadasm.S]</code>)
* <code>[https://github.com/D-Programming-GDC/GDC/blob/master/libphobos/src/std/math.d std.math]</code> (FP control and ieeeFlags)
* [https://buildbot.dgnu.org/#/ GDC Builtbot Platform Status] 

==== References ====
* [https://www.gnu.org/software/gcc/gcc-9/criteria.html GCC Primary Platform List]
* [https://wiki.debian.org/SupportedArchitectures Debian Supported Architectures]

=== Create a script to produce a Raspberry Pi cross-compiler ===
----

Create a utility or script, in D, with as few dependencies as possible, that will download and build an LDC compiler that can be run on Windows and Linux hosts, but can generate code for the Raspberry Pi 3 running Raspbian Linux.

==== References ====
* [https://wiki.dlang.org/Building_LDC_runtime_libraries Cross-compiling the druntime]
* [http://d-land.sepany.de/tutorials/einplatinenrechner/einstieg-in-die-raspberry-pi-entwicklung-mit-ldc/ D Windows toolchain for Raspberry Pi (German)]
* [https://visualgdb.com/tutorials/raspberry/qt/embedded/ A utility that produces an SDK for Qt (a similar principle)]
* [https://github.com/JinShil/arm-none-eabi-gdc A similar script that produces a GDC cross-compiler for generated ARM Cortex-M code from a Linux host]
* [https://forum.dlang.org/post/siznumnaitvcnyqpleeh@forum.dlang.org A forum discussion on the topic]
* [https://forum.dlang.org/post/ppuiojvktlnnvoqfdtuc@forum.dlang.org Another forum discussion on the topic]

=== Implement a borrow checker in D ===
----

Explore the possibility of implementing a borrow checker in D.

==== References ====
* [https://gist.github.com/foonathan/023ff0fe923c6b0312dfc15e17ebb595 Borrow checker implemented in C++]
* [https://www.youtube.com/watch?v=Lj1GppqNr8c Presentation demonstrating a borrow checker implemented in C++]

=== Implementation of RYU to convert floats to strings ===
----

Copied from issue 19124 (https://issues.dlang.org/show_bug.cgi?id=19124)

Relying on the c standard library to do that is sometimes an issue. It's been encountered several time, 
for example in std.json because snn.lib give different results that the gnu c library in some corner cases. 
It would also allow to get to!string, format, and such functions working at CTFE for float and double.

==== References ====
* [https://dl.acm.org/ft_gateway.cfm?id=3192369&type=pdf Ryu: Fast Float-to-String Conversion]
* [https://github.com/ulfjack/ryu Implementation in C and Java]


=== Implement a FAT/FAT32/EXFAT file system for use in embedded systems ===
----

It is hoped that some day D will be a formidable alternative to C/C++ for programming microcontrollers and other embedded systems.  Many embedded systems employ SD cards for loading and storing data, but file system drivers will be needed to access them.

==== References ====
* [http://www.elm-chan.org/fsw/ff/00index_e.html FatFs - A popular implementation of FAT file systems in C]
* [https://www.adafruit.com/product/2796 Adafruit Feather M0 Adalogger - A typical ARM Cortex-M embedded system with an SD card]


=== Fork() based Garbage collector  === 
----
Gustavo Rodriguez-Rivera and Vincent Russo authored a [https://www.cs.purdue.edu/homes/grr/snapshot-gc.pdf paper] on an interesting take on a garbage collector: leverage efficient <tt>fork()</tt> implementations, which in many Unix systems take advantage of hardware-provided copy-on-write semantics for duplicated memory pages across processes.

This idea is already leveraged in the high-performance garbage collector for D implemented and used by Sociomantic. (A lingering issue is <tt>fork()</tt> and <tt>malloc()</tt> share a lock, which is [http://www.dsource.org/projects/tango/ticket/2087 problematic].) Leandro Lucarella, the engineer who wrote the implementation, has open sourced it [https://git.llucax.com.ar/w/software/dgc/cdgc.git here], but that is a bitrotten version that has fallen on the wayside.

=== Add crypto functions to Phobos  === 
----

The idea is to add crypto functionality to standard library Phobos by using existing libraries / operation system APIs. It should be possible to select a crypto provider like OpenSSL / CryptoAPI on Windows / Linux Kernel API. Then you can call crypto functions like RSA Digital Signature Validation and the chosen provider is used. To be clear, the idea is not to write crypto functions native in D but provide high level wrapper for existing libraries / OS APIs.

== External links ==
* [https://dlang.org/blog/symmetry-autumn-of-code/ Symmetry Autumn of Code]

[[Category:Student_projects]]