Difference between revisions of "SAOC 2018 ideas"

From D Wiki
Jump to: navigation, search
(→‎Automate the collection and publishing of data for monitoring D's progress as a software development project)
(→‎Port DRuntime to Priority Platforms for GDC)
Line 204: Line 204:
 
* <code>[https://github.com/D-Programming-GDC/GDC/blob/master/libphobos/libdruntime/core/thread.d core.thread]</code> (notably fiber support in <code>[https://github.com/D-Programming-GDC/GDC/blob/master/libphobos/libdruntime/core/threadasm.S threadasm.S]</code>)
 
* <code>[https://github.com/D-Programming-GDC/GDC/blob/master/libphobos/libdruntime/core/thread.d core.thread]</code> (notably fiber support in <code>[https://github.com/D-Programming-GDC/GDC/blob/master/libphobos/libdruntime/core/threadasm.S threadasm.S]</code>)
 
* <code>[https://github.com/D-Programming-GDC/GDC/blob/master/libphobos/src/std/math.d std.math]</code> (FP control and ieeeFlags)
 
* <code>[https://github.com/D-Programming-GDC/GDC/blob/master/libphobos/src/std/math.d std.math]</code> (FP control and ieeeFlags)
−
* [https://buildbot.dgnu.org/#/ GDC Builtbot Platform Status]
 
  
 
==== References ====
 
==== References ====
 
* [https://www.gnu.org/software/gcc/gcc-9/criteria.html GCC Primary Platform List]
 
* [https://www.gnu.org/software/gcc/gcc-9/criteria.html GCC Primary Platform List]
 
* [https://wiki.debian.org/SupportedArchitectures Debian Supported Architectures]
 
* [https://wiki.debian.org/SupportedArchitectures Debian Supported Architectures]
 +
* [https://buildbot.dgnu.org/#/ GDC Builtbot Platform Status]
  
 
=== Create a script to produce a Raspberry Pi cross-compiler ===
 
=== Create a script to produce a Raspberry Pi cross-compiler ===

Revision as of 01:39, 8 August 2018

This is a list of potential project ideas for applicants to the Symmetry Autumn of Code. Projects on this page are simply suggestions, not detailed proposals. It is up to any applicant who chooses one to flesh out the idea to a degree suitable for a project proposal as part of the SAoC application.

Contents

HTTP/2 for Vibe.d

HTTP/2 is a major revision of the HTTP network protocol. Among its advantages belong decreased latency due to (e.g. due to compressed HTTP headers or pipelined requests) or server-push from the server which allows the server to send resources to the client before they are requested. Therefore, Vibe.d - D's flagship web application framework - would vastly profit from implementing HTTP/2.

SChannel (Windows CryptoAPI) for Vibe.d

For SSL/TLS Vibe.d supports OpenSSL and Botan. It would be great if also SChannel from Windows CryptoAPI could be supported. There are major benefits of using SChannel. In case a security issue is detected, Microsoft will solve this issue using Windows Updates. For OpenSSL/Botan developers needs to ship new versions of the application with the fixed Dlls.

You also need no additional Dlls to ship with your application.

Graphics library for resource constrained embedded systems

Create a 2D rasterizer, rich drawing primitives, and 2D graphics library suitable for resource constrained embedded systems (e.g. ARM Cortex-M) to be used in industrial controls, home appliances, medical devices, consumer electronics, and IoT just to name a few. The end goal would be something similar to Segger's emWin. The library would be used to drive LDCs similar to https://www.adafruit.com/product/3396 Requirements:

  • Hardware agnostic; should simply render to a frame buffer
  • No dependencies (No Phobos, no C standard library, and no official D runtime).
  • Consider using -betterC, but a custom minimal D runtime is also a viable option

Related work

References

DUB: D's package manager

DUB - D's package manager - is one of the central infrastructure tools in the D world as a convenient build and package management tool for the D community. With D gaining more and more popularity, it's also a key tool to guarantee D's further adoption. Thus there are many great ideas to improve DUB:

Multi IDE debugger support (for windows)

Mago, the debug engine used in VisualD, includes a tool called Mago-MI, which has a GDB compatible interface. Therefore, you can use Mago-MI on Windows as replacement for GDB. Several IDEs uses this feature to enable debugging with 1 code line for Windows/Linux/MacOS. It is used in an experimental state in IntelliJ. There is also support in Visual Studio Code and in DLangIDE for which it was originally built.

There are several issues which could be addressed in Symmetry Autumn of Code:

  • Mago-MI is written in C++. This makes bug fixing hard. Rewriting it in D might make sense.
  • While the installation of Mago-MI is easy if you want to debug OMF executables, it is very hard if you want to debug COFF executables. You need another executable from Mago, you have to register DLLs via regserv and you have to manually create a registry entry. An installation procedure for installing Mago-MI would be great.
  • As Visual Studio Code is already supported by the D Language Foundation, using this as a reference client of Mago-MI would make sense.

While this proposal seems only related to Windows, the nature of Mago-Mi is to enable IDEs having 1 code line for debugging on Windows/Linux/MacOS. Therefore, overall investment in this project is good for all platforms.

Convert DMD, DRuntime, and Phobos Makefiles to D

Building DMD, DRuntime, and Phobos on Windows is too cumbersome. It appears to require the user to first set up a Posix environment and then battle environment configuration and other vague dependencies such as whether you are building for 32-bit or 64-bit and whether Visual Studio is installed.

Sometimes there are bugs that only manifest themselves on Windows, so they need to be investigated in a Windows environment. To do that that we need to be able to build and troubleshoot dlang repositories on Windows without hassle.

It would be nice if the requirement for a Posix environment was removed by porting all Makefiles to D, so the same code (with appropriate versioning of course) could be used to compile and build dlang repositories on all platforms without a lot of environment setup and configuration. All that would be required would be a recent D compiler in one's PATH and D's intrinsic dependencies (e.g. a C standard library).

Work on this has already been started at https://github.com/dlang/dmd/pull/8293. It just needs to be taken to completion and duplicated to other dlang repositories (e.g. DRuntime and Phobos).

Re-implement Software Building Blocks (e.g. memcpy, memset, memcmp, malloc, free, etc.) in D

Statement of the problem

It is currently possible to dynamically set the length of an array in @safe, pure and nothrow D: 

void main() @safe pure nothrow
{
    int[] a;
    a.length = 10;  // No problem, but beware, the compiler is tricksy and false.
}

That code gets lowered to a runtime hook _d_arraysetlengthT. However that function is neither @safe, pure or nothrow. The compiler is being deceitful. This is just one of *many* such problems in the compiler/runtime interface.

Another problem with those runtime hooks is that they almost always require TypeInfo, a runtime entity. The primary reason for requiring TypeInfo is to get the size of the type. The size of the type is a value that is known at compile-time, so there is no need to defer obtaining that information to runtime, but the runtime hooks were created before D had templates and many other meta-programming facilities.

Solution: Use templates

The solution to the problems stated above is to convert the runtime hooks to templates. Also, having the runtime hooks available as templates would allow the language features they implement to be used in -betterC code, as the implementations will be generated at compile-time; no longer requiring linking in a pre-compiled runtime implementation.

Problem with the solution

If a runtime hook is converted to a template, and the compiler lowers expressions to that template, then the code will be verified in the semantic pass of the compiler. That will force the the compiler to be honest; it will no longer be able to lie about @safe, pure and nothrow, as the semantic pass will verify it.

Unfortunately, this means that the runtime hooks (now templates) need to rewritten in a manner to ensure they are @safe, pure and nothrow. That is quite difficult as many of the software building blocks employed by implementation of those runtime hooks (e.g. memcpy, memset, and memcmp) are not written to be used in a memory-safe language. For example, to copy blocks of memory in D, one would likely use a function declared as void memcpy(T[] dest, const T[] src) not void* memcpy(void* dest, const void* src, size_t n).

Those software building blocks are currently taken from the C standard library. The fact that D requires the C standard library is an artificial dependency. D, being a suitable alternative to C, should not need to rely on toolchains and libraries written in other languages. It might be expedient to leverage such implementations, but it is not good design as it prevents D from building freestanding software, and complicates the software development environment as it becomes necessary to obtain and configure a C toolchain in addition to a D toolchain.

Solution to the problem with the solution

Therefore, to help address all of the issues above, it would be ideal to rewrite those software building blocks in @safe, pure, nothrow, typesafe, idiomatic D, so they can be used to implement the runtime hooks, compiler intrinsics, freestanding software, and more, while simplifying the development environment setup required to use D.

Other thoughts

It may be more feasible to choose one of two of these building block functions to keep the project within the scope and duration of the event.

malloc, free, and friends may be able to take implementation details from std.experimental.allocator.

References

Implement AVX2 and AVX-512 in DMD/DRuntime

AVX2 and AVX-512 are suspiciously absent from core.simd. Implement them. They will likely also be useful, if not necessary, to implement optimized block memory functions in the software building blocks proposal.

References

Re-implement Software Building Blocks (e.g. memcpy, memset, memcmp, malloc, free, etc.) in D for the ARM Cortex-M

Work is currently underway to use D as an alternative for microcontroller firmware programming for the ARM Cortex-M target. However, like all targets, we need the fundamental software building blocks created.

This project would implement one or more of memcpy, memset, memcmp, malloc, free in D so they can be used from @safe, pure, and nothrow code. They should be highly optimized, and will likely require some ARM Thumb inline assembly potentially leveraging SIMD instructions for the targets that support them.

Make WebAssembly a First-Class Target for D

Some progress has been made towards making WebAssembly a first-class target for D. See Generating WebAssembly with LDC. However, it's just a start and a lot more work needs to be done.

Performance Improvements for DMD

Over the years, DMD's performance has degraded and currently requires excessive amounts of memory. Profile DMD, suggest improvements, and implement them.

References

Adding D support to gRPC

D support for gRPC should be added. The maintainers are willing to include D into their repository (https://github.com/grpc/grpc/tree/master/src). Python, C++ or the higher level c wrapper (https://github.com/Juniper/grpc-c) could be used as blueprint.

References

Create a performance benchmarking and monitoring website

Create a performance benchmarking and monitoring website for tracking D's performance metrics over time.

Metrics may include.

  • Compilation speed
  • Binary size
  • Compiler memory consumption
  • Various runtime benchmarks stressing CPU, memory, or both
  • Performance metrics for various projects/libraries in the D ecosystem

References

Automate the collection and publishing of data for monitoring D's progress as a software development project

Find a way to automatically collect data from the D ecosystem and publish it as a set of metrics for measuring D's progress as a software development project.

Metrics may include.

  • Number of pull requests per day/week/month/year
  • Number of issues fixed per day/week/month/year
  • Number of commits to any repository on GitHub or other source code repository websites
  • Forum/Twitter/Facebook/Reddit/Hacker News/Search Engine and other social media activity

References

Port DRuntime to Priority Platforms for GDC

GDC needs it's druntime code ported to more targets for inclusion in GCC.

Areas that need some love:

References

Create a script to produce a Raspberry Pi cross-compiler

Create a utility or script, in D, with as few dependencies as possible, that will download and build an LDC compiler that can be run on Windows and Linux hosts, but can generate code for the Raspberry Pi 3 running Raspbian Linux.

References

Implement a borrow checker in D

Explore the possibility of implementing a borrow checker in D.

References

Implementation of RYU to convert floats to strings

Copied from issue 19124 (https://issues.dlang.org/show_bug.cgi?id=19124)

Relying on the c standard library to do that is sometimes an issue. It's been encountered several time, for example in std.json because snn.lib give different results that the gnu c library in some corner cases. It would also allow to get to!string, format, and such functions working at CTFE for float and double.

References


Implement a FAT/FAT32/EXFAT file system for use in embedded systems

It is hoped that some day D will be a formidable alternative to C/C++ for programming microcontrollers and other embedded systems. Many embedded systems employ SD cards for loading and storing data, but file system drivers will be needed to access them. The general idea is to create an alternative to FatFs in D.

References

Fork() based Garbage collector

Gustavo Rodriguez-Rivera and Vincent Russo authored a paper on an interesting take on a garbage collector: leverage efficient fork() implementations, which in many Unix systems take advantage of hardware-provided copy-on-write semantics for duplicated memory pages across processes.

This idea is already leveraged in the high-performance garbage collector for D implemented and used by Sociomantic. (A lingering issue is fork() and malloc() share a lock, which is problematic.) Leandro Lucarella, the engineer who wrote the implementation, has open sourced it here, but that is a bitrotten version that has fallen on the wayside.

Add crypto functions to Phobos

The idea is to add crypto functionality to standard library Phobos by using existing libraries / operation system APIs. It should be possible to select a crypto provider like OpenSSL / CryptoAPI on Windows / Linux Kernel API. Then you can call crypto functions like RSA Digital Signature Validation and the chosen provider is used. To be clear, the idea is not to write crypto functions native in D but provide high level wrapper for existing libraries / OS APIs.

External links

Retrieved from "https://wiki.dlang.org/?title=SAOC_2018_ideas&oldid=9331"