OpenRecomp: Fred Day on static recompilation for legacy software

When we talk about software preservation, emulation is inevitably the first thing that comes to mind. For decades, emulators have allowed us to reproduce entire platforms and keep thousands of programs running on hardware completely different from the machines they were originally designed for.

OpenRecomp is a project looking for another path. The aim is not to reproduce the old machine's hardware in software, but to recompile software so that it can run natively on modern hardware.

Obviously, the process is far from automatic. An old binary contains code written for a specific CPU, depends on precise conventions, and interacts with the memory and hardware of that particular system. Running old software on modern hardware can lead to behaviour that did not exist, or was never expected, in its original environment.

This is where OpenRecomp comes in, a project developed by Fred Day around the concept of deterministic static recompilation. Its pipeline transforms the original binary into an architecture-neutral intermediate representation and from there generates code for the host machine. For Fred, however, a result is not satisfactory simply because the experiment "works": it has to be verifiable, reproducible and comparable against a reference.

We spoke with him about exactly that: different CPU architectures, indirect control flow, platform adapters, preservation, and above all what would need to happen for OpenRecomp to eventually move beyond being primarily a research project and become a real platform.

OpenRecomp is currently crowdfunding the next stage of the project: strengthening MIPS32 support and moving it from CANDIDATE towards PROVEN through development, validation, reproducible testing and public documentation.

Support the OpenRecomp MIPS32 campaign on Crowdfunder

OpenRecomp is built around deterministic static recompilation. For readers more familiar with traditional emulation, what does “deterministic” mean in this context, and why is it important to the way you want the project to be tested and trusted?

In OpenRecomp, “deterministic” refers primarily to the translation pipeline rather than to the behaviour of the guest program itself. A recompiled program can still use randomness, timing or external input like any other program.

What I want to be deterministic is the transformation from the original binary into OpenRecomp’s intermediate representation and then into host-side code. Given the same input binary, the same contracts and the same defined toolchain, the pipeline should produce the same translation rather than making translation decisions dynamically while the program is running.

That matters because it makes the evidence reproducible. Instead of saying “I ran it and it seemed correct”, I can say: run this input through this pipeline, execute the result against this reference, capture these observations and compare them. Someone else should be able to repeat that process independently and obtain the same result.

A JIT-based emulator can of course also be deterministic and extremely accurate. The distinction is that OpenRecomp aims to make the translation itself a stable artifact that can be inspected, versioned and verified before runtime. That is what allows trust to be built from reproducible evidence rather than only from cumulative play-testing.

One of the central ideas behind OpenRecomp is that different guest architectures can feed a common intermediate representation. Where do you draw the boundary between what can genuinely be shared across architectures and what will inevitably remain specific to a CPU, platform or individual piece of software?

I think of it as three layers.

The shared core contains the pipeline structure, the intermediate representation, control-flow representation, memory operations, contract validation, host-code generation and evidence machinery. That layer should not need to know whether the original program came from MIPS32, RV32I or some future architecture.

Architecture-specific behaviour belongs in the frontend. Instruction encodings, register files, endianness, calling conventions, delay slots, flags and other CPU-specific rules have to be translated into explicit generic IR semantics. A MIPS delay slot, for example, should become an explicit sequence of operations in the IR so the backend does not need to know that MIPS delay slots exist.

Then there is a third layer that is neither fully architectural nor fully generic: software-specific behaviour. A compiler may lay out a jump table in an unusual way, a program may rely on an undocumented quirk, or it may use self-modifying code. No intermediate representation makes those problems disappear.

The aim is therefore not to pretend everything can be architecture-neutral. It is to keep the genuinely shared machinery shared, push CPU-specific semantics into frontends, and keep whatever software-specific residue remains as small and as clearly bounded as possible.

You place a strong emphasis on evidence and on distinguishing CANDIDATE results from PROVEN ones. What does a result have to demonstrate before you are personally comfortable calling it proven, and how do you guard against a recompiler producing output that appears correct but fails in less obvious cases?

I use CANDIDATE and PROVEN as descriptions of evidence strength, not as labels for how far along the implementation is.

A result can compile, execute and look correct and still remain CANDIDATE if the claim has not yet been independently checked against a defined contract. PROVEN means I can point to the precise claim being made, the evidence supporting it and a reproducible verification process that does not depend on my interpretation of the result.

Where possible, that includes behavioural comparison between a reference path and generated native execution: registers, memory state, return values, checksums or other externally observable state relevant to the test.

The important point is that a PASS is not sufficient by itself. I also want to test the places where the pipeline should fail: malformed inputs, out-of-bounds behaviour, stale assumptions, integrity checks and cases where two supposedly equivalent execution paths disagree.

That approach has already mattered in OpenRecomp. Auditing the pipeline has found issues that would not have appeared in a happy-path demonstration, including validation or bounds assumptions that were weaker than they appeared to be. If evidence later shows that a claim does not meet the bar I originally gave it, I would rather downgrade that claim than protect the label.

For me, that willingness to move a result backwards as well as forwards is part of what makes the evidence classification useful.

You identified indirect control flow as one of the biggest obstacles between synthetic fixtures and real software. Why are jump tables, function pointers and computed branches particularly difficult for a static recompiler, and what kind of solution do you think OpenRecomp will eventually need?

A direct branch or call tells the recompiler where execution is going. An indirect branch does not. Its destination may come from a register, a table lookup, a function pointer or a virtual-function slot and may only become known at runtime.

That breaks one of the most useful assumptions in static recompilation: that you can construct the program’s control-flow graph completely before the program runs.

An emulator gets around this naturally because the original guest code remains available. If execution jumps to address X, the emulator can simply decode whatever is at address X. A static recompiler cannot assume that safety net exists. If it has replaced the original program with translated host code, it needs to know that every legitimate destination has a translated counterpart.

I expect the solution to have at least two parts.

First, recover as much as possible statically. Compilers tend to produce recognisable structures for switch tables, vtables and other forms of indirect control flow, so a meaningful portion can potentially be resolved ahead of time.

For destinations that genuinely remain dynamic, I expect OpenRecomp will need some bounded runtime-dispatch mechanism that maps known guest addresses to their translated host functions.

The important design question is what happens when a destination is not in that map. My current preference is for that case to fail explicitly rather than silently falling back to arbitrary execution. I do not want a general-purpose interpreter to become a hidden safety net, because that would gradually undermine the static-recompilation model the project is trying to test.

Even with CPU recompilation working correctly, a historical console still brings graphics, audio, interrupts, DMA, timing and other hardware behaviour with it. How do you imagine OpenRecomp’s platform adapters handling that boundary without turning the core project into a different emulator for every machine?

I want CPU recompilation and platform behaviour to meet at a very strict boundary.

The core should understand CPU semantics. A platform adapter should understand the environment that CPU expected to talk to.

Once you move beyond the processor, a real console program interacts with memory-mapped registers, graphics command streams, DMA, interrupts, audio systems, timers and other stateful hardware. Some of those interactions may map relatively cleanly onto modern APIs; others may require much deeper modelling.

The important architectural rule is that those details should not leak back into the architecture-neutral recompilation core.

A platform adapter therefore does not necessarily mean that every hardware write becomes one simple API call. It may need to maintain substantial platform state. But that complexity should remain outside the core translator.

One direction I have already explored is redirecting hardware-facing behaviour into a modern engine runtime instead of recreating the original silicon inside the recompiler itself. In that model, translated CPU logic remains native code, while the platform layer translates relevant hardware-facing intent into services provided by the host environment.

The exact balance will vary greatly by platform, but the separation matters: OpenRecomp should not have to become a different CPU recompiler every time a different console is targeted.

The next major milestone you described is running a small, rights-safe real MIPS program through the complete pipeline. What would constitute a successful demonstration for you? Is simply reaching native execution enough, or would you want reproducible behavioural comparisons similar to the evidence you are producing today?

Native execution would be necessary, but it would not be enough.

If a real MIPS binary reaches a Windows executable and starts running, that proves the pipeline did not immediately fail. It does not prove that the translation is correct.

The demonstration I want is a small, legally distributable MIPS binary that was not written specifically for OpenRecomp, carried through binary ingestion, frontend translation, the common IR, AOT code generation and native execution.

Then I want the meaningful observable behaviour of the native result compared against a defined reference. Depending on the program, that might mean return values, register state, memory contents, generated buffers, hashes or other deterministic outputs.

Most importantly, the process should be reproducible by someone other than me.

The real advance would therefore not be “OpenRecomp executed a MIPS binary.” It would be: “OpenRecomp executed a real, independently produced MIPS binary and there is reproducible evidence that the translated behaviour agrees with the reference within a clearly stated scope.”

That would be substantially stronger evidence than another synthetic fixture because the real program would have been produced by a normal toolchain and would not have been designed specifically to make the recompiler’s job easy.

From a preservation perspective, what can static recompilation potentially offer that conventional emulation already does extremely well? Do you see OpenRecomp primarily as another way of keeping old software executable, or could it eventually solve preservation problems that emulation alone cannot?

Emulation is already one of the most successful preservation technologies we have, and I do not see OpenRecomp as a replacement for it.

A good emulator can preserve an entire hardware platform and thousands of pieces of software. Static recompilation, especially when individual titles need platform-specific work, is unlikely to match that breadth and should not try to.

What recompilation potentially offers is a different preservation trade-off for particular pieces of software where the additional engineering effort is justified.

A recompiled artifact can depend less directly on continued reproduction of the original hardware environment. Its generated host code can potentially be rebuilt using future toolchains, examined with ordinary development tools and deliberately connected to modern host APIs.

That may make certain programs easier to maintain or adapt over very long periods, while emulation remains the stronger answer for library-wide compatibility and accurate preservation of the original machine.

There can also be useful legal and distribution models where original copyrighted binaries or assets are not redistributed and the end user supplies their own legally obtained material.

So I see the two approaches as complementary rather than competitive.

A simple way of putting it is: emulation preserves the machine; static recompilation may provide another way to preserve particular pieces of software.

Looking further ahead, what would make you consider OpenRecomp a mature project rather than an experimental research effort? Is there a particular technical milestone or real-world use case that would tell you the underlying idea has genuinely worked?

I would stop thinking of OpenRecomp primarily as an experimental research project when someone other than me can take the framework, add support for an architecture or platform I did not build personally, carry a real program through the full evidence chain and produce an independently reproducible result without needing me to guide every stage.

One real program running successfully would prove that the idea can work.

Maturity requires more than one demonstration. I would want a track record: several real programs, ideally spanning more than one guest architecture, reproducible builds, well-defined contracts, explicit known limitations and enough documentation that the project can continue without depending on its original author.

A commercial game successfully running through the pipeline would obviously be an important milestone, but I do not think a single spectacular demo would prove architecture neutrality or maturity on its own.

The deeper test is repeatability.

One impressive result is evidence. A repeatable process is a platform.

───────────────────────────────────────────

Then perhaps it is precisely that final sentence that explains better than any definition what Fred Day is trying to build.

OpenRecomp is still an evolving project, and the path towards real software, different architectures and complete platforms presents problems that Fred himself makes no attempt to hide. In fact, the distinction between what has already been demonstrated and what still needs to be demonstrated is a fundamental part of the project.

And that is also what makes it interesting.

Static recompilation does not aim to replace emulators, nor would it make sense to expect it to. An emulator still has an enormous advantage when the goal is to preserve an entire machine and its catalogue. OpenRecomp is taking a different path. It is trying to understand how far an old program can be made independent from the hardware it was written for, translated in a verifiable way and made to run on modern machines.

The next step is to take a small, real, legally distributable MIPS program that was not written specifically for OpenRecomp and carry it through the entire pipeline to native execution. But the real goal is to produce concrete results and, above all, make it possible for someone else to reproduce them without Fred's guidance. That will tell us how far this idea can go.