Using Data In-Place

Russ Ross posted a question to the LL1 list, asking about any research people have done into allowing higher-level languages to use data structures from other higher-level languages in place. He points out that this is a strength of C -- you can take a pointer to the other language's memory, cast it to whatever type you want, and then go at it. But most high-level languages have their own requirements for how memory is laid out at runtime, and these requirements are almost always subtly different for each implementation.

There are several ways to address this issue. You can convert the data structures from one format to another, by making a copy. Unfortunately in order to do this you probably have to write your converter in C, because the source and target languages probably don't offer enough control over memory layout.

Another technique is to use a common virtual machine, like the JVM or Parrot. If the VM has sufficiently high-level semantics, then language implementations won't have to think up their own (incompatible) implementation technique, and they will be able to interoperate. You can think of C as being a degenerate version of this technique, where the VM is the CPU itself. That's why C is so good at handling data structures from other languages -- they both use the same CPU model.

A third technique is the one that Russ is asking about: find some way of explaining the foreign semantics of the source language in a way that the target language can understand and use directly. This would add a whole lot of sophistication to your language implementation, and quite possibly make it too much of a headache to bother with. It would impact your compiler as well as your runtime libraries, and it would require that the compiler make a distinction between the behavior of a program, and how that maps to runtime implementation. It might even require that you have multiple versions of each runtime implementation, and that all the versions are capable of interoperating in the same address space. This is clearly not trivial.

Even assuming you can handle the implementation difficulties, you still have the difficulty of finding a way to describe the semantics of a foreign language implementation. Can you imagine explaining Haskell's implicit lazy evaluation to an imperative language like Scheme? How about explaining Parrot's continuation passing implementation to a stack-based implementation like Python? Explaining garbage collection to a language that uses explicit memory management?