I'm Morgan McGuire (@CasualEffects). I've been working on computer graphics and games for 20 years at great places including NVIDIA, University of Waterloo, Williams College, Brown University, Roblox, Unity, and Activision.

See my home page for a full index of my blog posts, books, research, and projects.

Monday, December 2, 2013

JavaScript Desiderata

Why JavaScript?

I teach the the programming portion of Williams CS107: Creating Games using codeheart.js, which is a minimal game framework for JavaScript. I use JavaScript because it has several practical advantages over other currently available languages for beginning students (as well as for my own jam games and prototypes):

  1. Requires no software IDE, compiler, etc. to be installed. Anyone with a web browser and a text editor can immediately begin programming, regardless of what user privileges they have on a machine
  2. Allows creation of web games, including mobile games
  3. Allows easily sharing your creations with others
  4. Minimal boilerplate in the syntax
  5. Supports first-class functions, which enable important and elegant CS concepts (e.g., lexical scope, iterators, anonymous functions, abstraction, functional programming)
  6. Easy access to 2D graphics and GUI features
  7. Supports network, threading, audio, gamepad, and OpenGL for the sufficiently motivated student (albeit awkwardly)
  8. C-like syntax transitions students quickly to Python, Java, and C++ for upper-level courses
  9. Surprisingly fast for numerically-dense code (memory allocation and garbage collection remain slow in all implementations I've used.)
  10. Interpreted, so that code can be interacted with from the console ("REPL")
  11. Debuggers and profilers are readily available (if somewhat flakey)
  12. Is also supported by non-browser environments, e.g., node.js and rhino

A Near-Minimal Subset

To reduce both the students' and my own confusion, I use a near-minimal subset of the language:
  1. IF and WHILE (many students will quickly pick up FOR on their own)
  2. Function declaration
  3. Variable declaration
  4. Essential standard operators (omitting ++, ===, etc.)
  5. Arrays
  6. Objects as structures (not classes)
codeheart.js provides the structure that allows students to use this minimal subset:
  1. Function-based wrapper for essential data structure functionality
  3. Canvas-based graphics setup with a standardized resolution
  4. A more useful and intuitive Array iterator
  5. Argument checking for its own built-in functions
  6. A unified event model for mouse, touch, and keyboard
  7. Abstraction of some browser peculiarities and differences
  8. A generic math library suitable for scalars or vectors
Part of the point of using small framework and a common language is that students are welcome to go beyond the minimal subset that I teach (and use myself), since any JavaScript that they learn can be applied. 

Why Not JavaScript?

JavaScript is limited by some really poor design choices and certain missing features, a few of which
are intentionally missing in the name of browser security but really provide little additional security. Limitations that are impossible to work around within the language include:

  1. No built-in module or scoped variable ("let") support. One has to use (function() { ... })() to perform the equivalent of a LET statement. A strangely non-functional LET is in JavaScript 1.7, which Apple does not support in Safari since they are using the 13-year old JavaScript 1.5 specification.
  2. No built-in INCLUDE statement. There are various workarounds for this (and codeheart.js has one), but they all lead to serious confusion in the presence of program errors.
  3. The threading API precludes efficient data transfer between threads, forcing all data to be marshalled through strings.
  4. Due to ambiguity in the syntax, whitespace is significant within a RETURN statement and anonymous function calls require an extra set of surrounding parentheses.
  5. The FileSystem and Clipboard APIs are near useless since they don't really provide access to those features.
  6. The gamepad API is only supported on Chrome
  7. "Cross-site" scripting limitations make simple actions like looking at the pixels of an image or other data loaded from a different host impossible to perform in a generic way
  8. No peer-to-peer network API (for browsers); one has to relay everything through a server. No actual security is gained by this because the same communication is possible, but all networking becomes slower and substantially more awkward.
  9. The serialization API (JSON) is missing:
    1. Support for comments and whitespace
    2. A way to serialize and deserialize floating point specials such as Infinity and NaN
    3. A way to serialize the undefined value (which itself weirdly has no literal syntax)
    4. A way to specify the deserializer for an object without parsing it. E.g., one can't parse "Foo {a: 1, b: 2}", one has to make up a special convention such as "{type: 'Foo'; a: 1, b:2}", which is obviously not generic or suitably abstract and extensible
  10. No support for multiline strings (like Python's """)
  11. No support for synchronous loading of large data files (WebGL demos embed giant 3D models as code literals!)
  12. No support for #file and #line pragmas (see the next section for why these are important)
  13. No support for operator overloading (nearly essential when writing graphics and game code)
  14. No (universal) support for constant declarations; some limited object freezing
  15. No support for static argument checking (e.g., types, number of arguments)

The last point makes it almost impossible to write large JavaScript programs because refactoring requires manual checking of all potentially affected code and explicit test coverage. Dynamically-typed variables are a great feature; they elegantly enable polymorphism, avoid clutter, and ease new programmers into a language. However, type inference and/or compile-time argument checking are essential for checking contracts and static error detection in software engineering. After about 1000 lines, I find it impossible to maintain a JavaScript program. Some of my larger web games are about this size because they've saturated.

Three Dangerous Roads Forward

For those morally insulted by the limitations listed above or seeking to use JavaScript more seriously in a CS curriculum or development environment, I see three ways to address JavaScript's limitations while keeping most of its benefits:

  1. Just wait. The JavaScript (ECMAScript) specification is evolving quickly. However, waiting is not particularly attractive because: 
    1. to capture the "runs everywhere" benefit, one must use the version supported by the least functional major browser (iOS Safari seems to be this), and
    2. the specification is evolving crazy new syntax and OOP features rather than addressing the fundamental limitations of the language...just like Python, C++, Java, and other languages are going off the rails by chasing features instead of ensuring good design.
  2. Use a language that compiles to JavaScript. This allows a more advanced, and often better-thought-out language while retaining the JavaScript runtime. ASM.js also provides an assembly-language JavaScript compilation target that JIT compiles to native code on many browsers. There are many languages implemented this way. The primary new drawbacks of this are:
    1. adding a compilation step destroys the "work without installing software" and "interactive debugging" features
    2. none of these languages are sufficiently popular that I'd want to depend on their implementations for long-term courseware
    3. the languages that I've investigated all introduce new syntax, which might be superior but is a new learning (and remembering) hurdle
    4. it is really hard to debug programs when errors occur in generated code
  3. Extend JavaScript with a self-interpreter. This is the classic Scheme-like approach to language development, and JavaScript is sufficiently Scheme-like that it seems viable. The idea is to write a JavaScript interpreter in JavaScript so that everything still runs dynamically and in the browser, and try to keep the generated code sufficiently close to the base language that one can still interact usefully at the console with it. Some features of HTML and JavaScript that make this approach plausible are:
    1. JavaScript has an EVAL statement
    2. JavaScript has first-class functions, so one can compile code to functions
    3. HTML allows embedding arbitrary code in <script> tags with custom MIME-types. These will be ignored by the browser but are accessible to JavaScript itself as source
    4. JavaScript can insert more JavaScript into HTML as additional <script> tags (I use this to implement INCLUDE())
    5. Critically, Ariya Hidayat and others have already written an open source JavaScript parser in JavaScript: Esprima. This is the hardest part of writing an interpreter and it is already done.
For example, LET support can be added to a legacy JavaScript implementation by rewriting code, as in this example:

if (x > 3) {
   let y = 2;

if (x > 3) {
   (function(y) {

provided that one is very careful about RETURN, CONTINUE, and BREAK statements, which trigger non-local control flow that is altered by a function. More aggressively, one could directly interpret JavaScript and EVAL functions into the native interpreter as needed.

I'm not aware of any JavaScript + features implementations in JavaScript, although I suspect that a few exist (and I welcome hearing about them in the comment thread below!). The basic idea was already proven by Flapjax and ASM.js. The features that I'd most want in such a language in order of decreasing importance are full JavaScript compatibility, optional type declarations, optional argument count checking, CONST declarations, functional LET and modules, and operator overloading. I can work around most of the other limitations given these to abstract the solutions.

The first two approaches that I suggested above are "dangerous roads" because they lose features and move one towards an increasingly arcane language. The third approach is dangerous because writing an interpreter is often how a computer scientist (e.g., Knuth) manages to spend a lifetime building infrastructure to write a very nice 100-line program.

Morgan McGuire is a professor of Computer Science at Williams College, visiting professor at NVIDIA Research, and a professional game developer. He is the author of the Graphics Codex, an essential reference for computer graphics now available in iOS and Web Editions.