Wish list/std.json

This page describes the current problems of std.json and what would be expected of an ideal replacement module.

Problems with std.json

• It's API is not up to current phobos standards
• there are some - probably serious - bugs
• no interaction with ranges
• doesn't use @safe const, nothrow IIRC
• inherently unsafe API (accessing the union part)
• could be (should be) much faster

Design guidelines for std.json2

Input API:

1. A pull-parser/tokenizer/lexer
   • Should implement a InputRange API with ElementType JSONToken.
   • Should be as fast as possible
   • shouldn't allocate memory
   • shouldn't even convert strings to numbers. Instead JSONToken should have a 'type' field (ObjectStart, ArrayStart, String,Number, ...) and a value field which is always a (raw!) w/d/string. A templated T get!(JSONType) helper which could also verify the type would be useful. Any decoding of JSON strings should be done in the get function. Get may optionally use some caching.
   • It should be specialized for input arrays (const/immutable/mutable) of char/wchar/dchar which are completely in-memory and use slicing in these cases.
   • It should work with arbitrary ranges of w/d/char. To avoid memory allocation a fixed-length internal buffer should be used and token values should be slices to that buffer. (It would be good if the user can query whether values are slices of the original input which can be reused, or sliced of the iternal buffer which have to be .dup ed to keep them)
2. DOM-API
   • Should be based on 1)
   • Same principle as current std.json, but in modern D
   • API similar to std.variant, DYAML

optional, nice-to have:

1. A Sax-style API (based on 1)
2. A simple deserialization API
   • basically T fromJSON(T, CHAR)(CHAR input) (supporting all inputs supported by 1)
   • Shouldn't allocate any memory
   • Optionally skipping fields which are in the JSON data but not part of T and the other way round.
   • Usage: auto artist = fromJSON(Artist, `{"name": "", "songs":42}´);

Output API:

1. A OutputRange of JSONTokens
   • Shouldn't allocate
   • Should work on top of other OutputRanges
   • should accept the tokens from the pull-parser, so we can do copy(JSONTokenizer(json), JSONWriter(stdout));
   • Additionally provide an easier user API: JSONWriter.startObject(), JSONWriter.startArray(), JSONWriter.writeField(name, T),...
2. DOM API
   • It should of course be possible to manipulate the DOM, then write it back
   • Should be the same as the input DOM API
   • Should be based on 1)

optional:

1. A simple serialization API
   • counterpart to the deserialization API
   • Shouldn't allocate any memory
   • Based on output-ranges and 1)
   • Usage: myvariable.toJSON(stdout);
   • Helper function using Appender: string json = myvariable.toJSON();

Another questions is if and how this should interact with streams, if we ever get them.