Here’s a couple of things hanging around in my pre-SRFIs repo which I think are particularly interesting but which, owing to recent events, I’m unlikely to take any further within the context of Scheme. They might form part of a future programming language adventure, but in the meanwhile I’d like to document them here in the hope that some of them might be interesting to passers-by who may like to take them up.
Record systems are a design nightmare. Existing record type systems (especially R6RS’s) are featureful, but the features they provide sometimes make it impossible for users to add their own features beyond what the system designer envisaged. For example, R6RS’s design makes it impossible to offer a general record copy or functional field updater system, because its combination of inheritance and opacity means you maybe can’t get an instances’ field values defined by its type’s parent. There are many more examples on the linked page on my personal wiki.
Since Scheme is happily a do-it-yourself language, it would be nice to find a record type system which provided the right set of features to be both efficient and extensible. Ideally, it should be possible for Scheme users to extend the record system in any way they like without breaking existing record system features.
Turtle records are my latest attempt at this. The pre-SRFI API is crap, and I would revise it significantly (likely preferring a procedural interface as the base layer, with syntactic sugar on top, as there no longer seems to be any controversy over their procedural record definitions’ efficiency). But the idea – of explicitly defining multiple record systems and requiring the record system definition to be available in order to subtype a type or inspect an instance – is strong. Fundamentally, it’s an application of capability security to record type encapsulation, using lexical scope as the capability mechanism. The usual way encapsulation is done in such cases is using sealers – boxes which control and forward messages to the object contained within them – but that presumes an OO environment and adds extra memory indirection, so instead we do it here with a key-based system where you need the key in scope in order to break the encapsulation.
Performance-wise, the goal is modest but gets most of the way there for most Scheme programs:
- the basic features provided by SRFI 99 should be supported at the same level of optimization as SRFI 99 supports: this means mostly that subtyping incurs no performance penalty (including for type checks, tyvm) and uses of immutable record fields can be aggressively coalesced by common subexpression elimination – there’s nothing in the encapsulation system which prevents these simple optimizations
- the optimizations, such as they may be, provided by any features supported by R6RS but not by SRFI 99 are left as an exercise to clever compiler writers; we perceive the main value of those R6RS features as being in the improved possibilities for abstraction and encapsulation they offer, and any optimization possibility is incidental
Using the encapsulation features provided by turtle records, it’s possible to implement the entirety of R6RS’s record system, or an improved variant of it such as SRFI 237. It’s also possible to implement systems like CLOS, Chez SOOP, SRFI 57, SRFI 99, etc.
It would be an interesting research project to try to get all the performance benefits of natively supported type sealing and instance opacity within a system which implements them in terms of lexical scope.
See also: SRFI 259, the other half of the project to let programmers create their own record systems, which lets such systems define records which are applicable procedures.
Ports have a similar problem to records. R6RS provides an inflexible interface for creating custom ports. It’s not possible, for example, for a port to record the file path it’s open to, or for a port which is talking to a network socket to remember the IP address at the other end, and so on.
Faceted ports fix this with a combination of composition and inheritance similar to that which R6RS uses for conditions. First, the set of operations on a port is arbitrarily extensible by composition: a port can save useful metadata like the name of the thing that’s at the other end, or you could add facilities for non-blocking I/O and such. Second, ports can be stacked up in layers and these extra properties of the underlying ports apply to the overlaid ports. You could imagine having a port which gzip compresses everything written to it; this is overlaid on top of a port which actually does the task of writing the (now compressed) data to the underlying file, but that in turn is probably overlaid by a buffering layer between the compression procedure and the actual operating system interface. Nonetheless, the overlaid ports can still report the name of the file they are talking to through a so-called ‘informative facet’ which inherits from one layer to the next.
I would likely revise the API for creating facets and defining facet types; I don’t think it’s necessary for a facet type to be reified as an object in itself.
I’m not sure about the use of SRFI 158 generators and accumulators here, although I/O related applications such as this are one of the few cases where those are unquestionably applicable (because they are inherently stateful and limited in the types of data that pass through them, so the problem of their non-functional interface and abuse of the EOF object is inconsequential). The problem is that they require one-figure-at-a-time writing when it might well be possible to do better copying entire chunks at once. The security and design advantages mentioned in the pre-SRFI still apply, though; a potential improvement should try to maintain those.
There are various desirable properties of an iteration protocol: it should be fast; it should be pure functional; it should be replayable; it should be possible to use the call stack to traverse trees instead of building an explicit stack; etc.
Gold iterators (Elgot algebras, possibly) are an attempt to provide something which satisfies as many of these desires as possible. Unfortunately my investigations showed that Scheme compilers aren’t quite as good at optimizing this pattern as I hoped, but I think it’s certainly plausible that these could be made very fast.
They’re a bit tricky to write and use correctly, but the idea here is that this would be a low-level protocol and that for most cases you would use a looping macro to consume sequences and a generator (such as gold-generate in the pre-SRFI) to produce them. (I have most of a suitable looping macro designed in my head but have never yet written it down in either code or prose.)
Rather pedestrian but still missing from standard Scheme.
The only language I know of which has basically avoided the ignominy of several incompatible date and time representations is Python. This library attempts to repeat that minor miracle with a design very closely based on the Python one, with explicit reference to ISO 8601 and the standard tz database as similarly successful long-term designs. It’s also adapted for R7RS Small’s (correct, in my view) choice of TAI as its standard timescale.
Thanks so much to Arvydas Silanskas for writing an implementation; I know it can’t have been easy.
John Cowan’s comparator abstraction is a nice way to provide standard data structures with the equality, ordering, and hash functions they need.
Unsurprisingly for a new design, it’s taken several tries to get it right, and this is my attempt to push the design a little further along. Apart from encouraging users to avoid an unnecessary extra memory indirection when accessing the procedures used by their data structure implementations, it also makes the procedures for implementing one’s own hash functions considerably more useful (including adding ones to avoid silly performance errors), defines the hash bound more helpfully, provides a useful definition of a hash salt, and adds hash functions corresponding to bound-identifier=? and free-identifier=?. It also supports the notion of an ‘unstable hash function’ to support transport cell guardians, in case some part of a hash function depends on a pointer value which might change when the garbage collector compacts the heap.
It’s possible to implement the SRFI 128 API in terms of this one, and that should be done. Ideally SRFIs 128, 162, and 228 would be withdrawn afterwards in favour of this one, but it would require John’s consent to do that (and mine, in the latter case).
Identifier properties offer an attractive means of creating extensible syntactic DSLs in Scheme, but it’s not possible to store a transformer in a property and hygienically apply it to expand to an embedded sublanguage without going through continuation-passing style.
This scrappy proposal adds a means of doing that, making it a lot easier to create extensible DSLs.
The R7RS Large WG resolved in 2023 not to add keyword arguments, although there was rough consensus that if we did add them, it would be best to do them in the Racket style where keywords are not datums which evaluate to themselves.
This proposal adds minimal facilities for supporting this (including, unlike Racket, support for using keywords for interfaces where the order is significant – like in Objective-C FFIs) and explains how Schemes (like Guile, Chicken etc.) which already went down the route of self-evaluating keyword objects can support both styles.