~

Roadmap

Future Projects

Projects

Future

+mink Stack Trace Determinism

duration

TBD

If code run inside the `+mink` metacircular evaluation function in the Hoon standard library, which runs all userspace code, errors out deterministically (i.e. hits a trivial infinite loop in the Nock spec), then `+mink` catches the error and retursns a stack trace.

Future

Add %pine Query-At-Latest Protocol

duration

1-2 Months

Remote scry on its own doesn't let one ship determine the latest state of a publication on another ship. This is solved by adding another protocol alongside the remote scry network protocol to implement `%pine` query-at-latest requests over the network, as pure reads.

Future

Add +on-rift or Similar For Userspace Breach Handling

duration

1 Month

Applications often have bugs when a peer breaches. Modifying the userspace/kernelspace API to give Gall agents clearer information about breaches should improve application reliability and developer experience.

Future

Add Urth-to-Urth Network Push Sessions

duration

4-6 Months

Solid-state publications that need low latency, such as chat, can't use the remote scry protocol without adding a new protocol to "push" updates to subscribers as soon as they are created. This protocol is Urth-to-Urth, and opt-in, to ensure naive runtimes still work without it (both as publisher and subscriber).

Future

Arvo/Vere Error Handling

duration

TBD

There are a number of cases where Arvo can crash in a way where it gets into an inconsistent state (e.g. Clay does not always handle Ames crashes properly) or gets stuck on something and fails to continue to make progress.

Future

Automatic Binary Upgrades

duration

TBD

Arvo updates itself over the air, but current Vere does not. We should let the binary update itself by downloading a new version and replacing itself with that, instead of forcing the user to run terminal commands to stay up to date.

Future

Automatic Error Handling on Replay

duration

TBD

If Vere hits a nondeterministic error on replay, it should retry instead of giving up and crashing.

Future

Automatic Memory Pack/Trim/Meld

duration

TBD

Future

Better Arvo/Vere Version Negotiation

duration

TBD

Improving the Arvo/Vere version negotiation will increase release rate by reducing the amount of work it takes to release a new Kelvin version.

Future

Clay Sync Error Handling

duration

TBD

The filesystem I/O driver doesn't handle filesystem errors properly, which can cause desynchronization between Unix and Clay. Fixing this would improve developer experience and general reliability.

Future

Clean up Clay %cares

duration

2 Months

Clay's filesystem querying API is cumbersome to use and difficult to implement properly. A cleaner API would support remote scry better, would be easier for developers to integrate with, and would be less bug-prone.

Future

Commit Before Compute

duration

1-3 Months

Current Vere needs to run Nock on an event before it can write it to disk. This places a lower bound on event latency, defined as time between receiving an event and performing its effects, at `D + N`, i.e. disk write latency (`D`) plus nock execution time (`N`). Commit-before-compute has amortized latency `max(D, N)`, which is usually significantly better.

Future

Consolidate Packet Re-Send Timers

duration

1 Month

Ames sets a lot of timers to remind it to send packets again if they don't get acknowledged fast enough. Reducing the number of timers lowers disk write usage, improves quiescence (which should eventually let hosting providers use less RAM and thereby lower costs), and should improve overall performance on publisher ships.

Future

Dependent Hoon Demo

duration

3 Months

A version of Hoon with a dependent type system could make the type system easier to understand, improving developer experience.

Future

Desk-Based Namespacing

duration

3-6 Months

Desk-based namespacing should improve reliability, maintenance, and developer experience.

Future

Dropping Privileges

duration

TBD

According to the principle of least privilege, each Vere process should only have privileges from the host operating system that it needs for normal operation. This is easier to understand, and likely easier to implement well, with multiprocess I/O.

Future

Encrypted Remote Scry

duration

2-3 Months

Arvo needs to encrypt scry paths and the values bound to them in order to use the remote scry protocol for private data such as group chats. This requires changes to the kernel to distribute encryption keys and let applications determine which other ships should have access to data in which publications.

Future

Exit Code Scheme for Mars

duration

1 Month

Adding a scheme of Unix process exit codes to the Mars process, and handling them in the Urth process (sometimes by printing them with more detail), will make the system easier to debug and maintain.

Future

Forward Secrecy Ratchet in Ames

duration

3-6 Months

Ames has no forward secrecy other than manual on-chain key rotation. This means if an attacker finds the symmetric key between two ships or one of the private keys, they could decrypt the whole history of communication between those ships.

Future

Generalized Deferral Mechanism

duration

TBD

Arvo's Behn vane (kernel module) currently serves two purposes: setting timers, and deferring tasks to later events. Deferral could be split out into a separate feature, which could aid both in refactoring Behn to be easier to verify and optimize.

Future

HTTP Scrying

duration

2-3 Months

The scry namespace should be made available over HTTP, to improve developer experience and performance for Urbit clients ("airlocks").

Future

duration

1 Week

The way ships ping their sponsors doesn't always work well behind home routers or other NATs, causing flaky connections.

Future

libames

duration

TBD

`libames` would aid in integrating Urbit with other systems, writing iOS clients, and allowing Urbit to be used in internet-of-things applications.

Future

Loom Tool Scaling

duration

2-4 Months

Once there are 8GB or 16GB in the loom, it will be common for there to be more data in Urbit than fits in RAM. This presents several new problems, and demands solutions.

Future

More Revision Control Tools

duration

2-3 Months

At some point, we want to stop using Git and use Clay instead. Most of what's missing from that is building more tools in Clay that replicate the various Git tools.

Future

Multi-Session Terminal

duration

1 Month

Dill, in the Arvo kernel, supports multiple sessions, but the current Urbit terminal client only uses one session. To make the Urbit command-line more usable, the client should support multiple sessions too.

Future

Multiprocess Event Log

duration

TBD

Similarly to breaking up the Urth process, we should split event log reading and writing into its own process, which will communicate with the Mars Nock runner process.

Future

Multiprocess I/O

duration

2-3 Months

We intend to split the Urth I/O process into multiple processes -- one dispatcher process and one process for each I/O driver.

Future

Nan Madol

duration

TBD

A far-out proposal is to have the runtime perform serialization, packetization, encryption, and congestion control, instead of Arvo.

Future

Native UI Research

duration

TBD

One of the main barriers to writing an Urbit app is that the standard way to present a user interface is by writing it in JavaScript that runs in a web browser. This has advantages, but it requires writing code in two languages, and it means you can't stay in Urbit entirely when writing an app.

Future

Network DoS Protection

duration

TBD

A standard protection against DDoS attacks is to track and rate-limit IP addresses sending malicious packets. A basic version of this functionality could be added to Vere without too much trouble and give Urbit a modicum of DoS resilience.

Future

New Mars Snapshot Design

duration

2 Months

New Mars has a novel scheme for ensuring copy-on-write semantics for memory pages in a way that minimizes disk write amplification when taking an incremental snapshot.

Future

Outer HMACs on Packets for DDoS Protection

duration

1-3 Months

Ames packets need to be decrypted to be authenticated. Wrapping the packet in an HMAC would let the receiver discard unauthenticated packets faster, improving DoS resilience.

Future

Pointer Compression: 16GB Loom

duration

1 Month

Once cells have all been reduced to two 64-bit words, Vere could align all allocations at 128 bits instead of 64. This would double the amount of addressable memory to 16GB.

Future

Refactor Ames Vane

duration

1-2 Months

The Ames vane could be shorter, easier to read, more performant, and easier to prove correct.

Future

Separating IPC Streams

duration

1 Month

Urth/Mars communication all happens within a single IPC connection. Separating communication that's unrelated to the event loop (such as logging, scry requests and responses, spinner status notifications, and snapshot notifications) into separate IPC streams should improve performance and increase flexibility for alternate Mars implementations.

Future

Shared Memory IPC

duration

2-3 Months

Vere's two processes, Urth (the I/O process) and Mars (the Nock worker process), communicate using a custom noun-based interprocess communication (IPC) protocol. This currently uses the Unix stdin and stdout, but using shared memory instead would make IPC significantly faster, reducing event processing latency and improving overall data throughput.

Future

Symmetric Routing

duration

3-6 Months

Symmetric routing should improve multiple things about Ames networking, enabling star packet forwarding, as well as improving firewall flakiness and peer discovery.

Future

Timer Improvements

duration

TBD

Urbit's timer system could be better in several ways.

Future

Tune %clog Backpressure Time and Space Constants

duration

1 Month

Tuning the constants in Ames's `%clog` system should reduce network flakiness, reduce the number of Arvo events, and improve publication bandwidth, especially until chat uses remote scry and solid-state subscriptions.

Future

Typed Interface to Solid-State Publications

duration

4-8 Months

A typed interface to solid-state publications should improve developer experience and performance (by avoiding runtime typechecks and type coercions).

Future

Use Nouns over HTTP

duration

1 Month

Eyre, Urbit's HTTP server, only supports communicating using JSON at the moment. Using "jammed" (serialized) nouns over HTTP would move JSON conversion logic from the server to the client, allowing a fully featured Gall agent to be written, including communication with a web client, without needing to write JSON conversions in Hoon.

Future

Vere Error Handling

duration

1 Month

There are a number of places where Vere crashes where it should instead handle the error and continue.

Future

Versioned Pokes and Subscriptions

duration

4-8 Months

Standardizing application protocol versioning in the kernel should improve developer experience and lead to more applications behaving correctly across upgrades, thereby improving user experience too.

Future

Zapgal Security

duration

1 Month

Turning off the `!<` ("zapgal") rune in userspace will reduce the security attack surface of the kernel, reducing the kernel's vulnerability to attack by malicious applications.