Event system

This page describes a proposal for a standard D framework/API for event loops and handling events.

Motivation

• Allow writing high-performance network programs in D, which match the performance of existing native-code solutions
• Allow multiple networking libraries / web frameworks to co-exist, by removing the requirement that each library implements its own event loop / protocol implementation / etc.
• Strengthen the D standard library by introducing any missing prerequisites
• Provide the requirements for implementing network protocols in the D standard library
• In the long run, allow D to compete with Node.js / Go for network applications, without relying on third-party libraries / dependencies

Existing work

Using D

The Vibe.d framework

Vibe.d (http://vibed.org/) is currently the most popular solution for writing web applications in D. Vibe uses an asynchronous model and a fiber-based API, exclusively.

ae library

ae (https://github.com/CyberShadow/ae) is a generic D library, which also includes support for asynchronous networking. Originally written in D1, it mostly uses an OOP API. It can use select() and libevent. It is notable mainly because it was used in multiple diverse network applications, most notably DFeed - a web frontend, NNTP/IRC/REST client, TCP server, and RSS aggregator.

libasync

• Announcement: http://forum.dlang.org/post/lvug1u$22vi$1@digitalmars.com

Dependencies

Both libraries also contain miscellaneous code which is not specific to networking / event loops:

• Memory management: vibe.utils.memory, ae.sys.data / ae.utils.alloc
• Efficient containers: vibe.utils.array / vibe.utils.hashmap, ae.utils.container

In a few cases, they are reimplementations of Phobos modules, presumably because the Phobos code was lacking.

• std.path: vibe.inet.path
• std.json: vibe.data.json, ae.utils.json

This indicates a necessity to improve the Phobos implementations.

Outside D

• Node.js uses the V8 JavaScript engine and a callback-based asynchronous, single-threaded model.

  Third-party additions exist to allow using multiple (isolated) threads, as well as a fiber-based model.

• Rust provides an IoFactory trait to abstract all I/O functionality.
  • It currently provides two backends, one based on libuv (cross-platform async/event based I/O), and one using the native, blocking, OS API. The runtime will initialise each backend as appropriate at start up - switching between these backends is as simple as adding a line of code to the program.
  • The API exposed to the end user is blocking, regardless of what the backend does.
  • The definition of IoFactory can be found here[1]. Additional functionality which could lead to blocking on I/O must be added to this trait and implemented for each backend.
  • The IoFactory trait provides a very simple wrapper around what the backend provides, higher level abstractions are built on top of it.
  • Multiple backends may be used at the same time by manually initialising them.

Goals

General design

• Multi-threaded (take advantage of D concurrency capabilities)
• Clean, structured API (e.g. layers: abstract event loop, network, protocol)
• Extensible (allow users to implement their own event loops, that can be used by the standard library)
• Do not limit the API to network events - provide abstractions that allow implementing event loops for other systems, e.g. SDL
• Allow multiple event loops (e.g. a network and an SDL event loop), and facilitate such usage (e.g. by allowing to designate one interruptible master event loop, and satellite event loops which forward events to the master event loop thread)
• Allow using both a synchronous (single-task or fiber) API or an asynchronous (callback / delegate-based) API (one API could be implemented as a layer on top of the other)
  • Do not limit this to network events either. For example, allow implementing a Windows event object event loop, which provides a call equivalent to WaitForSingleObject, which also uses a synchronous API, but instead of blocking the current thread, yields the current fiber and resumes it from the main event loop.

Networking

• High-performance
• Support native OS high-performance event loop capabilities, such as Input/Output Completion Ports and Grand Central Dispatch
• Load abstracted-event-loop libraries (libevent, libev) dynamically (to allow dynamic fallback at runtime and avoid static linking dependencies)
• Allow using the blocking API without an event loop at all (that is, no asynchronicity at all - use blocking calls)
  • Can be used to make one simple client connection, or a one-thread-per-connection server model (although the latter shouldn't be encouraged)
• The design should allow connection "adapters" which wrap a connection. Examples:
  • SSL
  • proxies (incl. chaining them)
  • TURN wrapper
  • HTTP Transfer-Encoding: chunked

Practical

• Design and implementation must satisfy the requirements of existing D frameworks / network applications (e.g. Vibe / DFeed)
• Consider requirements of the next-generation std.stdio replacement (std.io?)
• Aim for full automatic test coverage

Prerequisites

• Dynamic loading of libraries
• Manual memory management (custom allocators) may be required to match Vibe performance
• Fiber-local storage is required for transparent arbitrary task scheduling across multiple threads

Design

API blocks

API blocks are interfaces which indicate that an event loop supports waiting on certain kinds of events.

Examples:

• Sockets (network communication)
• Files (disk access)
• DNS lookups
• Timers
• Directory change notifications
• POSIX signals
• Windows messages
• Idle event (signaled when no other events are in the queue)
• Cross-thread event (interruptible event loop that can be asked to stop from another thread, and execute a function/delegate immediately)

API providers

API providers implement the API block interfaces.

Aside from event loops (which are API providers), some default ones can be:

• Straight-forward blocking calls (no asynchronicity)
• Threaded - auto-generated wrapper around a blocking provider, which runs blocking calls in a thread pool unrestricted in size
  • Example: Not every event loop implementation can be capable of performing asynchronous DNS lookup. Thus, the lookup would need to happen either synchronously (by blocking the calling thread), or in a separate thread.

Extending event loop APIs

It should be possible to "plug in" an API provider into an existing event loop, without the event loop code knowing about the API provider. The event loop would simply provide a private interface through which it can be extended.

Examples:

• Socket events can be added unto a standard Windows message loop
• SDL can issue network events through its main event loop when SDL_net is used, but the SDL event loop code shouldn't know anything about SDL_net specifically
• Interruptability can be added to an uninterruptible network event loop (such as select()) by using socketpair

Included event loops

Aside from wrappers around abstract-event-loop libraries, here's some event loops that can be included with the implementation:

• std.socket.select() (use std.socket)

  Covers a lot of common cases, has no additional requirements, and is very portable

• Thread.sleep (timers only)

  If the main event loop does not provide a timer API block, but is interruptible, the timer event loop can be run in its own thread.

• std.concurrency.receive (cross-thread event only)

  Multiple non-interruptible event loops can be "joined" by having each in its own thread, and having the std.concurrency event loop as the main one.

Plan

1. Establish requirements
2. Draft API, implementation
3. Port an existing network application which makes versatile use of networking (nomination: DFeed)
   • Avoid hacks and workarounds, as these indicate a deficiency in the API.
4. Submit for inclusion in Phobos/Druntime

Implementation notes

Components

• std.event.model.*: Asynchronous API block declarations (interfaces)

• std.event.loop.*: Event loop abstractions, which implement some asynchronous API block declarations

• std.block.model.*: Synchronous API block declarations

• Synchronous API block implementations:
  • std.block.fiber: Fiber-based wrappers around asynchronous block interfaces
  • std.block.direct: Straight-forward implementations

Initialization

• Interface declarations contain static field which points to implementation of the current interface.
  • getInstance() static method, which returns an existing instance if it was registered, or creates one if not
  • createInstance() static method, which examines available options, then chooses and creates one implementation
    • Can we make createInstance() templated or something, so that if the user specifies an event loop implementation explicitly, we don't pull in dependencies for all the implementations we'd try to create?
• For the synchronous API, use the blocking implementations by default, and switch to the fiber-based implementations as soon as an event loop is started.
• Start an event loop automatically when a fiber is created?
  • Wrap Fiber class in Druntime into some kind of Task object? Expand Fiber class? Expand Task class from std.concurrency?

Satellite event loops

• We want to give users the option to run event loop handlers of satellite event loops in the satellite event loop's thread, rather than the main thread, in case the underlying system requires a certain thread context (e.g. TLS or OpenGL contexts).
  • Do we lock a global state mutex ourselves, or force the users to do that?
• Have wrappers which take an interface, and create an implementation of that interface (using std.typecons.AutoImplement) which:
  • InterruptMethodForwarder: forwards method calls to another thread via an IInterruptible (same as the target loop)
  • InterruptCallbackForwarder: forwards callbacks (delegates in interface method parameters) to a main event loop via its IInterruptible
  • We can avoid the overhead of an indirect call in every layer by specifying the target type directly as a template parameter for the wrapper.

Initialization:

• When an API block can be satisfied by spawning a satellite event loop, specifically:
  • the main event loop does not implement the desired interface, but is interruptible
    • (the main event loop needs to be interruptible to proxy event handlers into the main event loop's thread)
  • we know an event loop that implements the desired interface and is also interruptible
    • (the satellite event loop needs to be interruptible so that we can register event handlers that might fire before the first previously-scheduled event)
• then:
  • create an instance of the event loop implementing the desired interface
  • IInterruptible-wrap it both ways (InterruptMethodForwarder and InterruptCallbackForwarder)
  • launch the event loop in its own thread
  • return our wrapped implementation

Questions:

• Should interrupt requests be synchronous (wait for the event loop to stop and run the method call), or asynchronous (add it to a queue that the event loop thread will process when it's actually interrupted)?
  • Asynchronous requests will require memory allocation.
• Do we need to allow API block methods to call the callbacks immediately (before exiting the method)?
  • If yes, and we use synchronous interrupts, then we need to be careful about deadlocks.

Other

• See libeio for asynchronous disk access

Questions

• Some high-performance event loops which operate with thread pools, such as IOCP, select threads to handle an event arbitrarily. This can cause tasks that begin waiting for an event to resume in another thread, and thus "see" other data in its TLS (global and static variables).
• WinRT support? Mentioned as a Vibe requirement. Can D target WinRT at all?
• std.event or core.event?
• Package name for blocking API?