Tools

The compiler is a query processor

The foundational decision, from which the rest of the tooling falls out, is that the compiler is not a batch (parse, check, emit, all at once, and die), it is an incremental library of queries, in the rust-analyzer model. The reason is that almost every tool is a consumer of the understanding the compiler has of the code: the LSP asks “what is the type of X?” and “where is Y defined?”; the linter asks for the semantic info; the doc generator asks for the signatures; the migration tool rewrites the AST. If each tool re-implemented the frontend, there would be six divergent parsers. Instead, the compiler is the engine, and the tools are thin frontends over it, consistent with the single-binary decision (no separate gopls/zls).

The incrementality is at the declaration level (the finest scope): when you edit a function, the compiler re-checks that declaration and what depends on it, not the file nor the project. It is what makes the LSP respond on each keystroke without reprocessing everything. This engine is the foundation of LSP, linter, formatter, doc gen and migration: all ask the same brain.

Build, dependencies and tasks

The package manager and the registry are ours, without depending on npm nor on crates. The mechanics is Go style: each mk.mod (lib) and the mk.project (bin) declare their deps, the mk.sum (section 15) is the lockfile with the auto-generated checksums, and the semver versioning is resolved by MVS (minimal version selection, reproducible, with no solver). And the sharp point: packages are language-only, they do not embed C nor JS. Want C? you bring it (compile and link, section 17). Want JS? it comes from npm (the @jsimport reads it, section 20). The registry stays clean (language only), without trying to be a polyglot manager that repackages everyone else’s world.

The build drives the compilation per target via --target: the native architectures (LLVM emits), WASM, and JS (the transpile of the .mkoui, section 20). The C of the @cimport compiles and links together (section 17); the JS target is the transpiler (the JS target spits out JS). The backend is LLVM for now (own backend later, section 18), and the frontend (lexer/parser) is a separate layer.

The task runner is Turborepo/CMake style: tasks are “what to do” and their dependencies. You declare that before the build it runs formatting, lint and tests, and the runner orchestrates (what today you would write in loose shell scripts). The tasks live in the mk.project (section 15), not in a separate build file.

Two modes of the compiler: portable and installed

The compiler is a single binary, for trivial transport (one file on the PATH, portable mode). But a single binary is hard to update (the TS parser changed? it would downgrade everything), so there is a second mode: mk install host explodes the binary into a pipeline of N files installed in a directory, where mk run/mk build become the entry of that pipeline. They are the two modes, and you choose: the portable one is the default (one file, zero installation), the installed one is opt-in for whoever wants incremental updates. (Consistent with the opt-in test: you do not pay for the pipeline’s structure until you ask for install host.)

The gain of the installed pipeline is partial updating: each stage (lexer, parser, type-checker, the TS parser of the @jsimport and the others) is a versionable unit of stable boundary, the same discipline of decoupled interfaces that the language requires of you (MemoryManager, MemorySource, Serializable) applied to the compiler itself. So updating the TS parser downloads only that stage, not the whole compiler. (This presupposes versionable stage boundaries, which the query engine already pushes, since queries have interfaces; the pipeline makes that decoupling an observable property, not just an internal one.)

And this structure pays off in version coexistence. Several versions of the language coexist (each project pins its own in the mk.project, section 15), and since a version is a set of stages, the difference between two versions is the delta of the pipeline: versions that share 80% of the stages download only the 20% different. The same holds for the stdlib: a new version downloads the diff, not the whole library. Ten projects in ten versions cost base plus ten small deltas, not ten whole toolchains side by side (what rustup and gvm do today).

Auto-update: always checks, applies when you tell it

The compiler checks for updates (default-on), of the toolchain and of the language (especially relevant pre-1.0, but also after: 1.1 was released, it warns). But the application is opt-in and non-intrusive: it does not update on its own, it informs (“there is a new version of Y, Z, W; run X to update”) and you decide. This reconciles with the slow-core (section 1): the auto-update warns you of the new version, it does not push you, and you move up when you want, with the mk.project pinning the project’s version until then. In CI and a reproducible build, the project’s pinned version is what commands, and the warning never changes a build under your feet. (The separation is clean: the toolchain of a version receives continuous patches; the language version is fixed per project and moves up by your decision, with the migration tool covering the breaks.)

Tests and benchmarks

Tests have two ways to live in the project: the @test decorator for loose tests next to the code (better a decorator for one test than a whole file), and dedicated files, with file_test.mko next to the code for local tests, and the tests/ folder (with sub-modules and subfolders) for an extensive suite.

The execution mode is a matrix of two orthogonal axes, via @testmode(order, state):

• order: sequential (in order) or parallel (concurrent).
• state: isolated (each test in a clean process) or shared (state carries between tests).

The four points of the matrix catch different classes of bug: (sequential, isolated) is the common one; (parallel, isolated) is fast and safe; (sequential, shared) carries the final state of one test into the next (left-over transition bugs, a timeline); and (parallel, shared) is concurrent shared state, where race conditions live. Note the sense of “shared”: the language is shared-nothing and prevents memory races (section 6), so this mode hunts races in logical or external state (a database, a file, a registered process that serves state), not in memory; “several tests touching the same state” is that external and logical state, not a shared heap (which does not exist). The default, with no annotation, is the point (sequential, isolated), the expected and safe one. The @testmode is at the file scope (the matrix applies to the suite; the loose @test inherits or uses the default), because per-test it would become ungovernable.

Each position accepts a value or a list, and the runner runs the cartesian product, in the order the values appear:

@testmode(sequential, [isolated, shared])
// runs: (sequential, isolated), then (sequential, shared)

@testmode([parallel, sequential], isolated)
// runs: (parallel, isolated), then (sequential, isolated)

@testmode([parallel, sequential], [isolated, shared])
// runs: (parallel,isolated), (parallel,shared), (sequential,isolated), (sequential,shared)

The order of the values in the list is significant (want parallel first? put it in front). And the report is per combination: each result is labeled with its pair (order, state), because “test X failed” is useless without saying under which condition. A test can pass isolated and fail in (parallel, shared), and that is exactly the signal you want.

Cost, and read this before using the full matrix. The configurable nature of the matrix is powerful and expensive: it multiplies the suite’s time by the number of combinations. @testmode([parallel, sequential], [isolated, shared]) runs the whole suite four times, that is 4x the time, which in CI is significant. The full matrix is a tool to hunt a specific bug (a race you suspect, a transition bug you chase), not the default mode of every suite. Use a single point day to day (the default (sequential, isolated) already covers the common case), and open the matrix when you are investigating: it is opt-in precisely because the cost is yours to choose when to pay.

Benchmarks follow the Go model (any test can be a benchmark), via @bench. There are two pieces with opposite decisions. The runner is configurable (the effort: time, iterations, repetitions, like -benchtime/-count), because you adjust how much to run; but the comparator is fixed (the statistic that decides “regression vs noise”, variance and significance, inspired by Go’s benchstat). The comparator is not configurable on purpose: an adjustable significance threshold invites p-hacking (lowering the threshold until the regression “vanishes”). You control the effort of the measurement, not the ruler that judges the result, consistent with the ethos of one right way.

The linter is closed (Go style): a set of rules you do not extend. You can turn it off, but it is not recommended. The point of the closed linter, together with the formatter, is to end the format-war (tabs vs spaces, brace position, all the sterile fights). You program however you want; the language normalizes.

Formatter, LSP and documentation

The formatter is zero-config (gofmt style): a single canonical format, with no options. There is no .prettierrc, there is no style bikeshedding; everyone’s code looks the same, and the energy that would go into discussing formatting goes into the work. It is the other half of “the language normalizes”.

The LSP is built over the query engine (above), which makes it fast (it re-checks per declaration, not the project). It delivers the expected (completion, hover, go-to-definition, rename, diagnostics, refactors) and understands the .mkoui extension (completing tags, props, seeing the bindings).

The documentation uses doc comments with ///, which work both in the language and inside the .mkoui files (you document a component where it lives). The doc generator renders to three targets: HTML, Markdown, and a site written in the UI extension itself, where the extension documents itself with the extension (docs as an app, since it serves to build sites).

The migration tool applies mechanical fixes for breaking changes (in the spirit of go fix and rustfix): when the core changes in a way that breaks code, the migrator rewrites yours automatically. But this is not just a tool, it is the mechanism that sustains a promise about the language (the slow-core policy, section 1): we change little, and when we change, we give the migrator.

Runtime: profiler, observer, debugger

These three tools rely almost entirely on observability (section 19), because the runtime already exposes what they consume.

The profiler measures what you configure and expose (section 19): it is opt-in, with continuous capture and tracing or sampling (and an adjustable sampling rate). The visualizer is not just a flamegraph; besides it, other graphs and, deliberately, a structured textual output (in the age of LLMs, a model reads the profile as text and reasons about it). The filters hide_runtime/hide_tracer (section 19) let you focus only on your project, hiding the noise of the runtime itself and of the tracer.

The runtime observer is the frontend of runtime.processes()/trace() (section 19), the equivalent of BEAM’s :observer/dbg (from whom we inherited the model). The API already exists; the observer presents it (supervision tree, live processes, channels, the |e| of the deaths).

The step-debugger is the only genuinely new runtime piece, because debugging a system of isolated processes is not debugging a single call stack. It has two breakpoint levels, process-only (pauses only that process, the rest of the application continues) and everything (pauses the whole application, all the processes), and a system of linked breakpoints: you connect breakpoints (ABC), and firing one fires the others. This is designed for the actor model: you link breakpoints in processes that interact via a channel and catch them together, at the moment of interaction, exactly what a single-call-stack debugger cannot do (it sees one process at a time, not the interaction between them).

All of this also holds for the UI extension (core/markup)

Each tool has a core half and a markup half: the LSP understands .mko and .mkoui, the formatter formats markup, the doc gen documents components, the linter lints markup. The core/markup structure cuts across the whole tooling: every decision above duplicates for the UI extension, because it is an official extension (section 20), not an annex.