In Defense of the Open Plan Office Layout

Every so often some meme seems to fly around the internet (or at least around the parts that I tend to frequent) that explores some variant of “engineers need offices (or cubes) so they don’t get interrupted” or “physical proximity doesn’t matter.” Now, I’m a firm believer in the theory that if it works for you, keep doing it, and that different things work for different people, but it seems like plenty of people are not so charitable and assume that there’s one right way to do software development, and that what works for their team must be the right thing for everyone else as well.

Here at Guidewire, our development teams use an open plan layout, and we do it because it works for us. We’ve got clusters of desks all around our engineering floor, with developers, product managers, and QA all mixed together in small groups that we call “pods” which collectively form our application and platform teams. The company started with an open plan setup from the beginning, and we’ve been anti-cube on the development side from the very beginning, even going so far as to pay to disassemble and store the cubes that were already set up in our current offices when we moved in. At first the open plan takes a bit of getting used, but over the years I’ve really come to like it, and I really have a hard time imagining working another way.

In my opinion, communication is actually one of the hardest problems to solve in software development, at least once you have a team of any decent size where knowledge is spread across a number of individuals. That problem is magnified when you’re in an industry-specific business like we are, where the developers aren’t experts in the field the software is targeting. As a result, we have to rely on product managers and subject matter experts to make decisions about how features should work and what should be prioritized; while I might be able to make reasonable decisions for myself about how a feature of an e-mail client should work, since I use one all the time, I can’t on my own make reasonable decisions about how a policy administration system should function, and it’s important that I talk to someone who does know what to do. Furthermore, that communication channel needs to be high-bandwidth and constant: you can’t just decide on requirements up front and then go work for a month, because there are a million small decisions that need to be made every day and many of them should really be made by an expert and not arbitrarily by someone who isn’t an expert user of the system.

The communication between the other members of the development team is also critical, though. The QA team needs to know as much about the product as the product managers or the developers, and they need to be able to understand the customer perspective in order to properly exercise the features and to understand the design of the feature so they can know if what’s happen is correct and intentional. They need to be able to ask questions of the product managers and developers as they go, and they need to be looped in as even small-scale decisions are being made. The developers also need to talk to each other on any reasonably-sized system in order to share knowledge about areas of the code, transmit best practices, and help get each other unstuck.

A lot of failures of software projects are failures of communication. With a team of highly competent engineers, the chance that any one person will do anything catastrophically stupid in isolation is pretty much 0. If I go off to implement a hash table, I can pretty well make sure that the thing works before I hand it off to other people. The far, far more likely causes of failure lie on the boundaries between people: things don’t get done because of a communication breakdown where two people each thought the other person was doing it, or the wrong features get built because the developer didn’t understand the use cases and product direction, or the team can’t expand enough because there’s not enough knowledge sharing, or the features themselves are right but the work wasn’t prioritized correctly and too much time was spent on relatively inconsequential things. Mitigating most of those risks requires optimizing for communication between the right parties, and part of ensuring that communication happens is setting up the right environment.

An open office plan is one way to optimize for that kind of communication. It’s not the only way to do so, of course, but if you have the luxury of physical proximity it can work well. Physical proximity has another benefit as well: you develop better relationships with the people around you. Cubes and offices can be very isolating, depending on the environment, and simply being around the other people on your team tends to lead to better working relationships, which improves both the work environment and makes communication even better. If you talk with people a little bit all day, asking questions and brainstorming and making jokes, assuming you like those people it can lead to a really good work environment.

The most common concern I’ve heard about the open plan office is that it’s too easy to get distracted; that concern is usually then followed up with some reference to how it takes 15 minutes to achieve “flow” again after being interrupted, so if you’re interrupted just once per hour that kills 2 hours out of your daily productivity. Of course, with an open office with conversations going on all the time and the ease of asking questions of other people, we all must be getting interrupted all the time, right? How can we possibly get anything done? We must be killing ourselves! If only we all worked in splendid isolation in offices, with very rare, pre-arranged meetings to hash out details, we’d certainly be way more productive!

But the reality is, it doesn’t actually work out that way, at least not for me. It’s true that interrupting someone can remove them from “flow,” but not all interruptions are created equally, and not everyone is in a state of “flow” all the time anyway. But supposing even then that a five minute conversation does result in a 15 minute loss of productivity for the person being interrupted: even then, it’s worth it if that 5 minute conversation saves someone a couple of hours of fumbling around on their own (“don’t reinvent the wheel, I think Bob did something similar last month, so just go talk to him and see what he did”) or, even more dramatically, saves someone from several days of going off in the wrong direction on something. It’s very easy to feel productive when you’re just writing code, but it’s a poor measure of productivity if you’re doing stuff that doesn’t need to be done or doing it sub-optimally. Most people also eventually develop some strategy for dealing with the distractions: they wear headphones or earplugs much of the time, or they work remotely sometimes so they have fewer distractions. Techniques like pair programming also help reduce the impact of distractions; one member of the pair can answer questions while the other stays focused, and when the interrupted developer returns to the task at hand their partner can help them context switch back much more quickly.

I’m sure it’s also the case that some companies set up gigantic factory-like warehouse floors full of anonymous coder units that are conveniently herded together so they can be more easily lambasted en masse by some dictatorial manager . . . but that’s not exactly how we roll over here, and many companies like us choose open plans intentionally and thoughtfully because we really feel like it’s the best way for us to develop. Again, that’s not to say that this is the right thing for every team or every product, or that it’s the only way to do things. But it’s often a good way to do things, and it’s a well thought out way that’s intentionally set up to optimize for communication (and collegiality).


JVM Compilation Basics

Long ago, seemingly in another life, I wrote about compiling try/catch/finally statements for the JVM. Some people seemed interested in that, but I never followed it up with anything else about compiling to the JVM. Well, better late than never I figure. I’m probably going to attempt to write a series of posts describing how we implement certain Gosu features in terms of bytecode, but before those will make sense to anyone who hasn’t spent quality time with the JVM’s instruction set, I figured it would be worthwhile to go over some basics about what bytecode looks like on the JVM and how one goes about generating classes on the fly.

JVM Basics

As most Java programmers are probably at least vaguely aware, the Java Virtual Machine is a virtualized stack machine with a relatively simple instruction set. Rather than specifying registers as the targets of operations, each instruction operates on the contents of the stack, either pushing or popping elements off the stack as appropriate. Getting a field, for example, will push the value of that field onto the stack, adding two numbers will pop two elements off of the stack, and calling a method will pop off as many elements on the stack as there are arguments to the method (plus one for the this pointer if it’s an instance method) and then push on the result (if the method is non-void). Local variables are stored in numbered virtual registers (more or less), of which there are an unlimited number, and you can store to those register from the stack or load them back to the top of the stack. There are plenty of other details to get into, but that’s the big picture: a stack machine where you push stuff on, pop stuff off, and load/store to virtualized registers as need be.

The instruction set for the JVM is very simple, which is mostly a good thing. Constructs like if statements and loops aren’t present in the bytecode, and instead you need to implement them yourself in terms of GOTOs and conditional branches, which isn’t all that difficult. Features like exceptions and try/catch blocks are baked into the VM, but checked exceptions and finally blocks are entirely constructs of the compiler. (It’s sometimes surprising once you get into things to see the which Java language features have direct support in bytecode and which are essentially implemented by the Java compiler.)

That simplicity means that it’s actually fairly simple to compile to. Most importantly, there’s no real register allocation to worry about, no register spilling, and no stack frame pointer or return pointer to worry about managing. Compilation is essentially just a recursive tree traversal then: to call a method with two arguments, you compile the expression representing the root, compile the expression representing the first argument, compile the expression representing the second argument, and then emit the method call instruction. It doesn’t matter if that first argument is an identifier or a horrifically complicated chain of method calls, because after all the instructions for that expression execute there will be one new value pushed on the stack in the right position. Of course, there are some nasty gotchas in there (the words “bridge method” bring on an involuntary cringe) . . . but the basics at least are straightforward.

Code Generation Tools

Suppose you’re thinking you want to generate classes on the fly. What library should you use to do it? I’ll save you some trouble and say just go download ASM (http://asm.ow2.org/). If you’re serious about writing a compiler, you want something as fast and low-level as possible, and ASM fits the bill: it handles some nasty bits like constant pools, label hookups, and max frame size counting, but otherwise pretty much each method call you make to ASM corresponds directly to a single bytecode instruction. It also includes an awesome Eclipse plugin, which will let you write Java code, compile it, and then view both a pretty-printed view of the bytecode and the ASM calls necessary to generate that bytecode. Playing around like that is by far the best way to learn about how certain constructs can be implemented on the JVM.

Basic Construct #1 – Method Calls

I won’t get into how every Java construct compiles into bytecode, but a couple of examples of some common constructs goes a long way. First up: the trusty method call. All method calls work basically the same way: you push all the arguments onto the stack, with the implied this pointer for an instance method first followed by all the explicit arguments, and then you call one of the method invocation instructions, giving it the name of the class the method is on followed by the method’s signature, which includes the name of the method, the name of all parameter types, and the return type (which is interesting, because the return type is not necessary to uniquely identify the method; I’ve always assumed it was there for the sake of the verifier). In Java, there are actually four different instructions for making method calls: invokevirtual, invokeinterface, invokestatic, and invokespecial. (JDK7 adds in invokedynamic, which is a whole ‘nother topic).

invokevirtual has your standard OO dynamic dispatch behavior, where the method is assumed to be on the class in question or perhaps overridden on a subclass. invokevirtual can only be used for methods on classes and not for interface methods. In implementation terms, an invokevirtual call can be directly mapped to an offset into the class’s vtable, since all subclasses should have the same function signature in that slot. invokeinterface, on the other hand, is used to invoke methods that are defined on interfaces, and it requires a search at execution time through the root object’s classes to look for the method definition. Since the interface could be implemented by many different, unrelated classes, the method isn’t guaranteed to be at the same vtable offset in all those places, so it can’t be linked at class load time or after the first method execution, and instead has to be looked up dynamically. (Of course, once the JIT kicks in, if there’s only one common implementation of an interface, presumably that will all get compiled down along with some guard clauses that never fail, and the performance will presumably be analogous.) Even so, it’s still good practice to use invokevirtual instead of invokeinterface when you have the choice; for example, if you know that an object is an ArrayList and not just a List, you want to compile calls to the get(int) method call as an invokevirtual call against the method defined on ArrayList, not as an invokeinterface call against the method defined on List. In addition, for virtual calls it matters what class you invoke the method against: if you have a method foo() on MySuperClass and overridden on MySubClass, and you then invoke foo() on an instance statically determined to be a MySubClass object, you want to make sure to invoke it as such and use MySubClass as the target in the invoke instruction. Even though the method will dynamically dispatch whether you invoke it against MySuperClass or MySubClass, the verifier and access controller can have different behavior depending on which one you list: for example, if MySuperClass is in the package1 package, MySubClass is in the package2 package, the method is protected, and SomeOtherClass in package2 invokes foo() against a MySubClass instance, that’s legal, but if the invokevirtual call is compiled against MySuperClass, you’ll get an illegal access exception at runtime because SomeOtherClass can’t see protected methods declared on MySuperClass. (Not that we had that exact bug in our compiler at some point or anything . . . what kind of idiots would do that?)

invokestatic is basically exactly what you’d think it is: it statically invokes a method. In that case, there’s no implicit “this” pointer to push on prior to the method call, and no dynamic dispatch happens. The class and method in the instruction is exactly what gets invoked.

invokespecial is used for static dispatch of instance methods. Out of the box, as it were, that’s used in two situations: constructors and super.foo() method calls. Constructors are compiled as instance methods with the special name <init>, but you can’t have the constructor dynamically dispatch to some other method, so the compiler explicitly specifies that the call shouldn’t dynamic dispatch and should go exactly against the class listed. A similar case applies to super.foo() calls; in that case, you’re explicitly stating your intention to invoke a method on the superclass even though it’s overridden on the subclass, so the compiler needs to flag those as statically dispatching.

Basic Construct #2 – If Statements and Other Control Flow Constructs

As I mentioned above, control flow constructs on the JVM are all implemented in terms of conditional branches and gotos. The most basic branch instructions are IFEQ and IFNE which mean, respectively, jump the specified label if the top value on the stack is equal to 0 or not equal to 0 (note that boolean “false” is 0, and any non-zero value is “true.”) There are lots of other options as well, though: the primitive integer relational operators have branch variants, i.e. branch if the element just below the top of the stack is less than the element at the top of the stack. Note that, in the simplest compiler implementation, you’ll basically always use IFEQ and IFNE: since the test for the if statement is an expression, you’ll probably want to compile that expression the same way you would compile it if it were an argument to a method call or an assignment statement, which will leave you with either a 0 or a non-zero value on the stack. Implementing the if construct directly in terms of some of the other conditional branches compacts the bytecode a tiny bit and makes it more readable, but doesn’t affect performance much (if at all) in practice, so for the most part we haven’t bothered implementing that yet in Gosu.

To compile an if-else statement, for example, you essentially do a test on the if condition, and invert it: if the condition is false, skip over the body of the if statement, otherwise fall through and execute the body. If there’s an else block, at the end of the if statement body skip over the else block using a GOTO.

Essentially, to compile:

if (foo) {
  bar();
} else {
  baz();
}

You’ll end up with bytecode that looks like:

. . . push foo on the stack . . .
IFEQ L1
. . . call bar . . .
GOTO L2
L1
 . . . call baz() . . .
L2

So if foo is false, we skip over the call to bar and go to L1, which starts the body of the else statement. After the call to bar from within the if statement, we instead skip to L2, which is after the body of the else statement.

Loop constructs are basically the same thing, with the body of the loop ending with a backwards conditional branch or straight goto back to the start of the loop (depending on the kind of loop and how you decide to implement them).

An Example To Put It Together

To put that all together, here’s more of a full-featured example of how some simple Gosu code compiles down to Java bytecode. (The code in question is for the Josephus microbenchmark as described at http://blog.dhananjaynene.com/2008/07/performance-comparison-c-java-python-ruby-jython-jruby-groovy/).

Here’s the Gosu code for the “GosuPerson” class. It should be straightforward even if you don’t know Gosu:

(Note that the “ as Count,” etc. tokens after the instance variable declarations declare read/write properties to wrap those instance variables, which make down to getCount()/setCount() methods in bytecode).

package gw.internal.gosu.compiler.sample.benchmark.josephus

public class GosuPerson {
  var _count : int as Count
  var _prev : GosuPerson as Prev
  var _next : GosuPerson as Next

  construct( c : int ) {
    _count = c
  }

  function shout( shout : int, deadif : int ) : int {
    if( shout < deadif ) return shout + 1
    Prev.Next = Next
    Next.Prev = Prev
    return 1
  }
}

And here’s what that raw, unadulterated output looks like in ASM’s printout of that class (note that we’re having ASM compute MAXSTACK and MAXLOCAL so our calls to those methods always have an argument value of 0, so that’s what ASM prints out even though that’s not what will end up in the bytecode). See if you can do the mapping from source to bytecode yourself. A couple important hints to keep in mind. First of all, the LOAD/STORE instructions are used to load/store to the virtual registers. Method calls start with their arguments already in the appropriate registers, with the “this” pointer in register 0 followed by all the arguments. Secondly, many bytecode instructions have a prefix of I, F, D, L, or A to indicate what type they operate on: I means integer (i.e. boolean, byte, short, char, or int), F means float, D means double, L means long, and A means Object. Lastly, Object type names in Java start with an L and end with a ; while primitive type names are all single character, with most of the mappings being obvious with the exception of Z, which means “boolean.”.

(Note that our wordpress theme will cut off the right-hand-side of the code listing here, and it’s difficult to wrap it correctly, so if you want to read through the code listing in its entirety I’d suggest selecting all the text, copying, and pasting it into a text editor. My apologies for not taking the time to format this more readably.)

// class version 49.0 (49)
// access flags 33
public class gw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson implements gw/lang/reflect/gs/IGosuClassObject  {

  // compiled from: GosuPerson.gs

  // access flags 0
  I _count

  // access flags 0
  Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson; _prev

  // access flags 0
  Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson; _next

  // access flags 1
  public (I)V
   L0
    ALOAD 0
    INVOKESPECIAL java/lang/Object. ()V
   L1
    LINENUMBER 9 L1
    ALOAD 0
    ILOAD 1
    PUTFIELD gw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson._count : I
   L2
    RETURN
   L3
    LOCALVARIABLE this Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson; L0 L3 0
    LOCALVARIABLE c I L0 L3 1
    MAXSTACK = 0
    MAXLOCALS = 0

  // access flags 1
  public getCount()I
   L0
    ALOAD 0
    GETFIELD gw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson._count : I
    IRETURN
   L1
    LOCALVARIABLE this Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson; L0 L1 0
    MAXSTACK = 0
    MAXLOCALS = 0

  // access flags 1
  public setCount(I)V
   L0
    ALOAD 0
    ILOAD 1
    PUTFIELD gw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson._count : I
   L1
    RETURN
   L2
    LOCALVARIABLE this Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson; L0 L2 0
    LOCALVARIABLE __value_ I L0 L2 1
    MAXSTACK = 0
    MAXLOCALS = 0

  // access flags 1
  public getPrev()Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson;
   L0
    ALOAD 0
    GETFIELD gw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson._prev : Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson;
    ARETURN
   L1
    LOCALVARIABLE this Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson; L0 L1 0
    MAXSTACK = 0
    MAXLOCALS = 0

  // access flags 1
  public setPrev(Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson;)V
   L0
    ALOAD 0
    ALOAD 1
    PUTFIELD gw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson._prev : Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson;
   L1
    RETURN
   L2
    LOCALVARIABLE this Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson; L0 L2 0
    LOCALVARIABLE __value_ Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson; L0 L2 1
    MAXSTACK = 0
    MAXLOCALS = 0

  // access flags 1
  public getNext()Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson;
   L0
    ALOAD 0
    GETFIELD gw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson._next : Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson;
    ARETURN
   L1
    LOCALVARIABLE this Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson; L0 L1 0
    MAXSTACK = 0
    MAXLOCALS = 0

  // access flags 1
  public setNext(Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson;)V
   L0
    ALOAD 0
    ALOAD 1
    PUTFIELD gw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson._next : Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson;
   L1
    RETURN
   L2
    LOCALVARIABLE this Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson; L0 L2 0
    LOCALVARIABLE __value_ Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson; L0 L2 1
    MAXSTACK = 0
    MAXLOCALS = 0

  // access flags 1
  public shout(II)I
   L0
    LINENUMBER 13 L0
    ILOAD 1
    ILOAD 2
    IF_ICMPLT L1
    ICONST_0
    GOTO L2
   L1
    ICONST_1
   L2
    IFEQ L3
   L4
    LINENUMBER 13 L4
    ILOAD 1
    ICONST_1
    IADD
    IRETURN
   L3
   L5
    LINENUMBER 14 L5
    ALOAD 0
    INVOKEVIRTUAL gw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson.getPrev ()Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson;
    ALOAD 0
    INVOKEVIRTUAL gw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson.getNext ()Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson;
    INVOKEVIRTUAL gw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson.setNext (Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson;)V
   L6
    LINENUMBER 15 L6
    ALOAD 0
    INVOKEVIRTUAL gw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson.getNext ()Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson;
    ALOAD 0
    INVOKEVIRTUAL gw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson.getPrev ()Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson;
    INVOKEVIRTUAL gw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson.setPrev (Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson;)V
   L7
    LINENUMBER 16 L7
    ICONST_1
    IRETURN
   L8
    LOCALVARIABLE this Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson; L0 L8 0
    LOCALVARIABLE shout I L0 L8 1
    LOCALVARIABLE deadif I L0 L8 2
    MAXSTACK = 0
    MAXLOCALS = 0

  // access flags 4097
  public getIntrinsicType()Lgw/lang/reflect/IType;
   L0
    ALOAD 0
    INVOKESTATIC gw/internal/gosu/runtime/GosuRuntimeMethods.getType (Ljava/lang/Object;)Lgw/lang/reflect/IType;
    ARETURN
   L1
    LOCALVARIABLE this Lgw/internal/gosu/compiler/sample/benchmark/josephus/GosuPerson; L0 L1 0
    MAXSTACK = 0
    MAXLOCALS = 0

  // access flags 9
  public static $evalAnnotations()Ljava/util/Map;
   L0
    NEW gw/internal/gosu/annotations/AnnotationMap
    DUP
    INVOKESPECIAL gw/internal/gosu/annotations/AnnotationMap. ()V
    ASTORE 0
   L1
    ALOAD 0
    INVOKEVIRTUAL gw/internal/gosu/annotations/AnnotationMap.getAnnotations ()Ljava/util/Map;
    ARETURN
   L2
    LOCALVARIABLE builder Lgw/internal/gosu/annotations/AnnotationMap; L0 L2 0
    MAXSTACK = 0
    MAXLOCALS = 0
}

The Effect of Language Design on Tooling

As another reflection on the JVM Language Summit (which was last week), one thing that I noticed both last year and this year is that very few language developers seem to talk much about tooling or integration with existing tools (there was one excellent talk about the Groovy plugin for Eclipse, however). Most of the language discussions get focused on things like syntax/semantics and performance. It’s great that you have some ultra-flexible syntax for your new language and all . . . but does it actually have the ecosystem around it to make it usable in production environments by non-experts? Is there IDE integration? Can I auto-complete, find usages of methods, or refactor things easily? Is there a usable debugger? Can it integrate cleanly with existing libraries written in Java? Is the error reporting reasonable? Are the stack traces readable? Can I profile it? Does it work with existing third-party tools like code coverage tools, or does it require a custom toolkit?

I ask those not to be totally pedantic and annoying, but because they’re important to actual users, and until those questions have reasonable answers the language itself is far less attractive. A language with the cleanest, most powerful syntax in the world isn’t particularly usable if the tools aren’t there.

It seems like the attitude of the language designers is often that those questions are unrelated to the design of the language; designing a great language is one problem, and making sure it has great tools is an entirely orthogonal problem to be solved by someone else at some point down the road. That might be true to a certain degree in a world where every language had infinite development resources at its disposal, but in reality the design of the language has a huge impact on the ability to take advantage of existing tools or to build new ones. If the language is dynamically typed, a good IDE is going to be hard. If it’s a JVM language that’s interpreted at runtime or compiles down to bytecode that doesn’t look much like a standard Java class with normal names and line numbers, standard Java debuggers and profilers won’t understand your code, Java integration will be difficult, and stack traces will be jumbled. Some of those problems can be patched up to some extent by the language itself (like stack traces and Java integration), while others (like needing a custom debugger) require a huge amount of work to overcome.

The thing is, developing that ecosystem of tools is hard and time consuming. Writing a parser and a compiler is a relatively small amount of work relative to the work needed to also create a good IDE, a good debugger, a good profiler, and all those other tools. The work there shouldn’t be trivialized, it shouldn’t be seen as an afterthought, and because the tool-ability of a language is so heavily influenced by the language’s design, it’s something that needs to be taken into account from the very beginning.

But for whatever reason, I don’t think I’ve ever seen that happen. I’ve seen plenty of “announcing my new language” presentations (and read plenty of such blogs), and I don’t think I’ve ever once seen anyone say “we designed it this way so we could re-use the existing Java debugging tools” or “we designed it this way so we could provide a really great IDE experience.” Is it because the people developing the languages don’t really use those tools themselves and so don’t think about them? Is it because people see tool development as a separate problem that should be solved separately from language design? Is there something else going on?

I know someone out there will try to make the argument that good languages don’t need tools beyond Emacs and a REPL and that only blurb languages like Java need tools to make them usable, but I think that’s an argument for another time. That argument can apply to IDEs, but I don’t think it applies to other sorts of tools: no matter how good the language, everyone needs a good debugger, a good profiler, good error reporting, or integration to other libraries from time to time. Those things are all often just as important as the language’s syntax and capabilities when it comes to developer adoption and productivity, so they should be thought of at language design time, rather than after the fact.