A Closer Look at the Details Behind the Go Port of the TypeScript Compiler

Relative Materials

A closer look at the details behind the Go port of the TypeScript compiler

Background Introduction

Currently, there is no tool that can fully replace the comprehensive type checking capabilities provided by tsc

Microsoft announced today:

[...] We have begun a native port of the TypeScript compiler and tools. This native implementation will significantly improve editor startup time, reduce most build times by approximately 10x, and significantly lower memory usage.

This article will delve into the technical details behind this major announcement.

Codebases: JavaScript vs. Native Implementation

To avoid confusion, I will use the following terms:

• JavaScript codebase: The current TypeScript codebase — actually written in TypeScript.
• Native codebase: The new codebase. I use the term "native" because that's what the TypeScript team calls it - actually written in Go.

Why This is a Major Breakthrough

Type checking is the only task that external tools cannot accomplish:

• Generating .js files has become faster — thanks to native tools and type stripping.
• Generating .d.ts files has become faster — thanks to native tools and isolated declarations.

Therefore, making type checking faster as well is a significant advancement.

Project Timeline

The current TypeScript version is 5.8.

TypeScript 6 (JavaScript): The JavaScript codebase will continue to be used for the 6.x series, with version 6.0 introducing some breaking changes to align with the native codebase.

• TypeScript original codename: Strada

TypeScript 7 (Native): Once the native codebase achieves sufficient parity with the JavaScript codebase, it will be released as TypeScript 7.0.

• Native codebase codename: Corsa

The two codebases will coexist for a long time.

What Needs to be Migrated

It's helpful to understand which parts of the TypeScript ecosystem need to be migrated:

• The command-line TypeScript compiler
• The TypeScript language server (helps editors support TypeScript)
  • The JavaScript codebase predates the widely used language server protocol, so it doesn't use it. The new language server will use this protocol, which should make it easier for editors to support TypeScript.
• Tools that use the TypeScript codebase
  • Interacting with the Go codebase requires a completely new approach:
    • Internal component exposure will be reduced
    • Interactions now occur across processes

When Will the Native Version Be Available to the Public

Current status: tsc is available. Still missing:

• JSX
• Types via JSDoc
• Build mode (project references)

Mid-2025: tsc with JSX and JSDoc (without build mode) End of 2025: Complete tsc and language server

Source: "A 10x Faster TypeScript"

When Did the Project Begin

First prototype developed by Anders Hejlsberg:

• Started in August 2024
• Scanner and parser took 1-1.5 months to write
• Initial approach: Manual code writing
• Later: Tools to automatically convert TypeScript code to Go code
• Ported code worked well (required some manual intervention)
• Data structure porting could only be done manually — because JavaScript objects (with flexible types) and Go structs (with highly configurable data layouts) are very different. Additionally, they now must work in a concurrent environment — for example: The JavaScript codebase orders types by giving each type a sequence number when it's created. This approach doesn't work in the Go codebase because the order of type creation is no longer deterministic due to multi-threading.

Why Choose Go Over Other Programming Languages

The TypeScript team wanted to (primarily) port the JavaScript codebase rather than rewrite it in a different language — for two reasons:

1. The new codebase must be (primarily) a drop-in replacement for the old codebase. This is difficult to achieve through rewriting.
2. Rewriting would take longer.

If we look at the requirements for the programming language used in the new codebase, some of them stem from the decision to port:

• Support for cyclic data structures — heavily used in the TypeScript codebase.
• Garbage collection. The codebase assumes this feature.
• The JavaScript codebase's style is more functional than object-oriented programming, not frequently using classes. This style is similar to how Go is written.

The remaining requirements are driven by performance and ease of use (developer experience):

• Good native code support on all major platforms.
• The language should be simple to learn.
• The language should have good tooling support.
• Control over in-memory data structure layout. With Go, you can use structs and create an array of structs with just one allocation (compared to multiple allocations in JavaScript).
• Good support for shared memory concurrency — this is an important element in making the code faster (more details below).

Why Not Choose C#

When asked "Why not C#", Anders Hejlsberg mentioned the following points:

• Go is lower-level than C#.
• Go has better support for generating native code (including specifying data structure layouts).
• Go is better suited for the non-object-oriented programming style used in the JavaScript codebase.

Where Does the 10x Performance Improvement Come From

• Half of the speed improvement comes from shared memory concurrency and using multiple cores.
• The other half comes from native code: JavaScript must be just-in-time compiled; it must provide great flexibility for its objects; it cannot inline objects (one allocation per array element vs. one allocation for the entire array); and so on.

JavaScript does have concurrency capabilities through Web Workers, but memory sharing is very limited (see SharedArrayBuffer).

TypeScript compilation has the following phases:

1. Parsing: Generate Abstract Syntax Tree (AST)
2. Binding: Create "symbol tables" for declarations, set up control flow graphs, etc.
3. Type checking
4. Emit: Code generation

In the native codebase, parsing and binding can be done independently (no memory sharing needed). Then, the data structures are immutable and can be easily shared between threads.

Parsing, binding, and emitting speed scales linearly with the number of cores used. Together, they account for about one-third of the total compilation time.

Type checking takes up the remaining two-thirds and is less easily parallelizable. Therefore, the following tricks are used:

• Type checking works on individual files — it lazily loads more information as needed.
• Technique: Run multiple type checkers, assigning parts of the files to each checker.
• Thread safety requirements are not high: checkers only share immutable ASTs. There is some duplicate work, but not much, as most type information is local.
• On the other hand, threads cannot operate completely independently — for example, error messages should not be duplicated and should be displayed in a deterministic order.
• Using 4 checkers (current hardcoded number), memory usage increases by 20% due to work duplication, but checking speed improves by 2-3x.
• Note that this 20% is relative to single-checker Go — and single-checker Go uses only half the memory of the JavaScript codebase.
• In testing, using 8 checkers only brought an additional 20% speed improvement (source).

Can the Native Codebase Run in WebAssembly

Supporting WebAssembly is important because it enables use cases like online TypeScript playgrounds. It's actually already supported.

Kevin Deng wrote a "TypeScript Go Playground" using the new codebase compiled to WebAssembly.

Work is ongoing to improve Go's WebAssembly output size and performance — that is: they can and will get better. Related discussion on GitHub: "Go Wasm performance"

Conclusion: An Impressive Achievement

I'm impressed by how quickly the TypeScript team was able to port the JavaScript codebase to Go. In a podcast (see "Sources" below), Hejlsberg said "I started prototyping in August". I didn't expect he meant 2024, but rather 2023 or even earlier.

This speed proves the cleverness of the team's approach: if they had rewritten the JavaScript codebase instead of porting it, it might have taken years and led to many inconsistencies between the codebases.

An interesting concern expressed by Anders Hejlsberg is that as type checking becomes faster, people might stop trying to write types that can be computed quickly — for example, it's easy to create types using template literal types that require significant computational power.

Perhaps we'll eventually get tools to analyze type performance. Type-level debugging also seems useful.

Sources

• Video "Syntax podcast: Typescript Just Got 10× Faster", hosted by Wes Bos and Scott Tolinski, featuring Anders Hejlsberg and Daniel Rosenwasser
• Video "TypeScript is being ported to Go | interview with Anders Hejlsberg", produced by Dimitri Mitropoulos for Michigan TypeScript
• Blog post "A 10x Faster TypeScript", by Anders Hejlsberg
• Video "Anders Hejlsberg on TypeScript's Go Port", hosted by Matt Pocock
• GitHub discussion: "Why Go?"

Contributors

XiSenao

Changelog

Last edited 5 months ago

View full history

3bb29bb-feat: i18n

e0a5c66-doc(story-tech_pulse): a closer look at the details behind the go port of the typescript compiler