SAOC 2018 ideas

This is a list of potential project ideas for applicants to the Symmetry Autumn of Code. Projects on this page are simply suggestions, not detailed proposals. It is up to any applicant who chooses one to flesh out the idea to a degree suitable for a project proposal as part of the SAoC application.

HTTP/2 for Vibe.d

---

HTTP/2 is a major revision of the HTTP network protocol. Among its advantages belong decreased latency due to (e.g. due to compressed HTTP headers or pipelined requests) or server-push from the server which allows the server to send resources to the client before they are requested. Therefore, Vibe.d - D's flagship web application framework - would vastly profit from implementing HTTP/2.

SChannel (Windows CryptoAPI) for Vibe.d

---

For SSL/TLS Vibe.d supports OpenSSL and Botan. It would be great if also SChannel from Windows CryptoAPI could be supported. There are major benefits of using SChannel. In case a security issue is detected, Microsoft will solve this issue using Windows Updates. For OpenSSL/Botan developers needs to ship new versions of the application with the fixed Dlls.

You also need no additional Dlls to ship with your application.

Graphics library for resource constrained embedded systems

---

Create a 2D rasterizer, rich drawing primitives, and 2D graphics library suitable for resource constrained embedded systems (e.g. ARM Cortex-M) to be used in industrial controls, home appliances, medical devices, consumer electronics, and IoT just to name a few. The end goal would be something similar to Segger's emWin. The library would be used to drive LDCs similar to https://www.adafruit.com/product/3396 Requirements:

• Hardware agnostic; should simply render to a frame buffer
• No dependencies (No Phobos, no C standard library, and no official D runtime).
• Consider using -betterC, but a custom minimal D runtime is also a viable option

Related work

• https://github.com/TurkeyMan/color
• https://bitbucket.org/timosi/minlibd
• https://github.com/JinShil/stm32f42_discovery_demo

References

• Inside the fastest font renderer in the world
• Scanline edge-flag algorithm for antialiasing
• How the stb_truetpe Anti-Aliased Software Rasterizer v2 Works
• Clipper - an open source freeware library for clipping and offsetting lines and polygons
• Anti-Grain Geometry - High Fidelity 2D Graphics - A High Quality Rendering Engine for C++

DUB: D's package manager

---

DUB - D's package manager - is one of the central infrastructure tools in the D world as a convenient build and package management tool for the D community. With D gaining more and more popularity, it's also a key tool to guarantee D's further adoption. Thus there are many great ideas to improve DUB:

• separate compilation and caching
• extensibility through DUB
• control flow directives
• build system support for C/C++
• git support
• support dub install for distributions
• support building a {snap, Debian, Arch, ...} package out of DUB package
• Add dub watch command to e.g. automatically restart vibe.d application if coding changes. PR was started 2015 but never completed (https://github.com/dlang/dub/pull/446)

Multi IDE debugger support (for windows)

---

Mago, the debug engine used in VisualD, includes a tool called Mago-MI, which has a GDB compatible interface. Therefore, you can use Mago-MI on Windows as replacement for GDB. Several IDEs uses this feature to enable debugging with 1 code line for Windows/Linux/MacOS. It is used in an experimental state in IntelliJ. There is also support in Visual Studio Code and in DLangIDE for which it was originally built.

There are several issues which could be addressed in Symmetry Autumn of Code:

• Mago-MI is written in C++. This makes bug fixing hard. Rewriting it in D might make sense.

• While the installation of Mago-MI is easy if you want to debug OMF executables, it is very hard if you want to debug COFF executables. You need another executable from Mago, you have to register DLLs via regserv and you have to manually create a registry entry. An installation procedure for installing Mago-MI would be great.

• There are some bugs in Mago-MI and DMD (wrong debug information) which makes debugging hard (see this issue and this one). Also Mago-Mi is missing some features. There are more bugs but not investigated so far.

• As Visual Studio Code is already supported by the D Language Foundation, using this as a reference client of Mago-MI would make sense.

While this proposal seems only related to Windows, the nature of Mago-Mi is to enable IDEs having 1 code line for debugging on Windows/Linux/MacOS. Therefore, overall investment in this project is good for all platforms.

Convert DMD, DRuntime, and Phobos Makefiles to D

---

Building DMD, DRuntime, and Phobos on Windows is too cumbersome. It appears to require the user to first set up a Posix environment and then battle environment configuration and other vague dependencies such as whether you are building for 32-bit or 64-bit and whether Visual Studio is installed.

Sometimes there are bugs that only manifest themselves on Windows, so they need to be investigated in a Windows environment. To do that that we need to be able to build and troubleshoot dlang repositories on Windows without hassle.

It would be nice if the requirement for a Posix environment was removed by porting all Makefiles to D, so the same code (with appropriate versioning of course) could be used to compile and build dlang repositories on all platforms without a lot of environment setup and configuration. All that would be required would be a recent D compiler in one's PATH and D's intrinsic dependencies (e.g. a C standard library).

Work on this has already been started at https://github.com/dlang/dmd/pull/8293. It just needs to be taken to completion and duplicated to other dlang repositories (e.g. DRuntime and Phobos).

Re-implement Software Building Blocks (e.g. memcpy, memset, memcmp, malloc, free, etc.) in D

---

Statement of the problem

It is currently possible to dynamically set the length of an array in @safe, pure and nothrow D:

void main() @safe pure nothrow
{
    int[] a;
    a.length = 10;  // No problem, but beware, the compiler is tricksy and false.
}

That code gets lowered to a runtime hook _d_arraysetlengthT. However that function is neither @safe, pure or nothrow. The compiler is being deceitful. This is just one of *many* such problems in the compiler/runtime interface.

Another problem with those runtime hooks is that they almost always require TypeInfo, a runtime entity. The primary reason for requiring TypeInfo is to get the size of the type. The size of the type is a value that is known at compile-time, so there is no need to defer obtaining that information to runtime, but the runtime hooks were created before D had templates and many other meta-programming facilities.

Solution: Use templates

The solution to the problems stated above is to convert the runtime hooks to templates. Also, having the runtime hooks available as templates would allow the language features they implement to be used in -betterC code, as the implementations will be generated at compile-time; no longer requiring linking in a pre-compiled runtime implementation.

Problem with the solution

If a runtime hook is converted to a template, and the compiler lowers expressions to that template, then the code will be verified in the semantic pass of the compiler. That will force the the compiler to be honest; it will no longer be able to lie about @safe, pure and nothrow, as the semantic pass will verify it.

Unfortunately, this means that the runtime hooks (now templates) need to rewritten in a manner to ensure they are @safe, pure and nothrow. That is quite difficult as many of the software building blocks employed by implementation of those runtime hooks (e.g. memcpy, memset, and memcmp) are not written to be used in a memory-safe language. For example, to copy blocks of memory in D, one would likely use a function declared as void memcpy(T[] dest, const T[] src) not void* memcpy(void* dest, const void* src, size_t n).

Those software building blocks are currently taken from the C standard library. The fact that D requires the C standard library is an artificial dependency. D, being a suitable alternative to C, should not need to rely on toolchains and libraries written in other languages. It might be expedient to leverage such implementations, but it is not good design as it prevents D from building freestanding software, and complicates the software development environment as it becomes necessary to obtain and configure a C toolchain in addition to a D toolchain.

Solution to the problem with the solution

Therefore, to help address all of the issues above, it would be ideal to rewrite those software building blocks in @safe, pure, nothrow, typesafe, idiomatic D, so they can be used to implement the runtime hooks, compiler intrinsics, freestanding software, and more, while simplifying the development environment setup required to use D.

Other thoughts

It may be more feasible to choose one of two of these building block functions to keep the project within the scope and duration of the event.

malloc, free, and friends may be able to take implementation details from std.experimental.allocator.

References

• A preliminary investigation of implementing memcpy in D
• Agner Fog's Optimizing subroutines in assembly language, 17.9 Moving blocks of data.

Implement AVX2 and AVX-512 in DMD/DRuntime

---

AVX2 and AVX-512 are suspiciously absent from core.simd. Implement them. They will likely also be useful, if not necessary, to implement optimized block memory functions in the software building blocks proposal.

References

• std.experimental.simd implementation
• intel-intrinsics DUB package

Re-implement Software Building Blocks (e.g. memcpy, memset, memcmp, malloc, free, etc.) in D for the ARM Cortex-M

---

Work is currently underway to use D as an alternative for microcontroller firmware programming for the ARM Cortex-M target. However, like all targets, we need the fundamental software building blocks created.

This project would implement one or more of memcpy, memset, memcmp, malloc, free in D so they can be used from @safe, pure, and nothrow code. They should be highly optimized, and will likely require some ARM Thumb inline assembly potentially leveraging SIMD instructions for the targets that support them.

Make WebAssembly a First-Class Target for D

---

Some progress has been made towards making WebAssembly a first-class target for D. This has been some progress, but, it's just a start and a lot more work needs to be done.

References

• Generating WebAssembly with LDC
• WebAssembly Image Dithering Example

Performance Improvements for DMD

---

Over the years, DMD's performance has degraded and currently requires excessive amounts of memory. Profile DMD, suggest improvements, and implement them.

References

• Forum Post - Compilation is taking a ton of memory
• Forum Post - D compiles fast, right? Right?]

Adding D support to gRPC

---

D support for gRPC should be added. The maintainers are willing to include D into their repository (https://github.com/grpc/grpc/tree/master/src). Python, C++ or the higher level c wrapper (https://github.com/Juniper/grpc-c) could be used as blueprint.

References

• Untested C wrapper
• Information regarding including D into gRPC repository

Create a performance benchmarking and monitoring website

---

Create a performance benchmarking and monitoring website for tracking D's performance metrics over time.

Metrics may include.

• Compilation speed
• Binary size
• Compiler memory consumption
• Various runtime benchmarks stressing CPU, memory, or both
• Performance metrics for various projects/libraries in the D ecosystem

References

• Mozilla's 'Are We Slim Yet'
• An attempt to mimic 'Are We Slim Yet' in D
• The benchmarks game

Automate the collection and publishing of data for monitoring D's progress as a software development project

---

Find a way to automatically collect data from the D ecosystem and publish it as a set of metrics for measuring D's progress as a software development project.

Metrics may include.

• Number of pull requests per day/week/month/year
• Number of issues fixed per day/week/month/year
• Number of commits to any repository on GitHub or other source code repository websites
• Forum/Twitter/Facebook/Reddit/Hacker News/Search Engine and other social media activity

References

• Forum post from Andrei with similar similar vision
• Current metrics on D's autotester
• Stack Overflow trends
• D's daily downloads
• GitHub API - Repositories using D
• D IRC statistics

Port DRuntime to Priority Platforms for GDC

---

GDC needs it's druntime code ported to more targets for inclusion in GCC.

Areas that need some love:

• core.internal.hash (CTFE `real` hash)
• core.thread (notably fiber support in threadasm.S)
• std.math (FP control and ieeeFlags)

References

• GCC Primary Platform List
• Debian Supported Architectures
• GDC Builtbot Platform Status

Create a script to produce a Raspberry Pi cross-compiler

---

Create a utility or script, in D, with as few dependencies as possible, that will download and build an LDC compiler that can be run on Windows and Linux hosts, but can generate code for the Raspberry Pi 3 running Raspbian Linux.

References

• Cross-compiling the druntime
• D Windows toolchain for Raspberry Pi (German)
• A utility that produces an SDK for Qt (a similar principle)
• A similar script that produces a GDC cross-compiler for generated ARM Cortex-M code from a Linux host
• A forum discussion on the topic
• Another forum discussion on the topic

Implement a borrow checker in D

---

Explore the possibility of implementing a borrow checker in D.

References

• Borrow checker implemented in C++
• Presentation demonstrating a borrow checker implemented in C++

Implementation of RYU to convert floats to strings

---

Copied from issue 19124 (https://issues.dlang.org/show_bug.cgi?id=19124)

Relying on the c standard library to do that is sometimes an issue. It's been encountered several time, for example in std.json because snn.lib give different results that the gnu c library in some corner cases. It would also allow to get to!string, format, and such functions working at CTFE for float and double.

References

• Ryu: Fast Float-to-String Conversion
• Implementation in C and Java

Implement a FAT/FAT32/EXFAT file system for use in embedded systems

---

It is hoped that some day D will be a formidable alternative to C/C++ for programming microcontrollers and other embedded systems. Many embedded systems employ SD cards for loading and storing data, but file system drivers will be needed to access them. The general idea is to create an alternative to FatFs in D.

References

• FatFs - A popular implementation of FAT file systems in C
• Adafruit Feather M0 Adalogger - A typical ARM Cortex-M embedded system with an SD card

Fork() based Garbage collector

---

Gustavo Rodriguez-Rivera and Vincent Russo authored a paper on an interesting take on a garbage collector: leverage efficient fork() implementations, which in many Unix systems take advantage of hardware-provided copy-on-write semantics for duplicated memory pages across processes.

This idea is already leveraged in the high-performance garbage collector for D implemented and used by Sociomantic. (A lingering issue is fork() and malloc() share a lock, which is problematic.) Leandro Lucarella, the engineer who wrote the implementation, has open sourced it here, but that is a bitrotten version that has fallen on the wayside.

Add crypto functions to Phobos

---

The idea is to add crypto functionality to standard library Phobos by using existing libraries / operation system APIs. It should be possible to select a crypto provider like OpenSSL / CryptoAPI on Windows / Linux Kernel API. Then you can call crypto functions like RSA Digital Signature Validation and the chosen provider is used. To be clear, the idea is not to write crypto functions native in D but provide high level wrapper for existing libraries / OS APIs.

External links

• Symmetry Autumn of Code
• Symmetry Autumn of Code is Underway