Bytecode VMs

The JVM isn't the only virtual machine you will find. Some obfuscators use virtualization which works by building a custom VM and then transpiling JVM instructions to the VM's instruction set and executing them.

The obfuscator "hides" code in this way and without de-virtualizing the VM or at least figuring out the architecture the reverse-engineer won't get far.

In this page we will write a simple VM in C to demonstrate how they work and how common instructions are executed.

Now our VM will be as I said simple and we will not transpile JVM instructions because as you can imagine making a VM capable of executing the instructions a JVM can isn't exactly the easiest task.

A JVM at it's core takes in bytes and interprets them as instructions then executes them, we will do the same only our instructions and VM architecture will be a lot more simple.

Architecture

Now the JVM has a stack, constant pool, local variables and types which again is too much for us to make, so we will cheap out a bit.

Our VM will have a stack and a constant pool supporting only strings and we will have only a 32-bit signed integer type which everything will revolve around including our pool.

The instruction set

We will support unconditional and conditional jumps those being: JMP, JNE, JE. We will also have basic arithmetic operations: ADD, SUB, MUL, DIV. For pushing onto the stack and popping from it we will have: PUSH, POP.

And just for a little fun we will include DPRINT for printing our 32-bit signed integers and SPRINT for printing our string constants.

Definitions

So first we must define our instructions for this will we use the C pre-processor:

#define PUSH 0x00
#define POP 0x01
#define ADD 0x02
#define SUB 0x03
#define MUL 0x04
#define DIV 0x05
#define JMP 0x06
#define JNE 0x07
#define JE 0x08
#define DPRINT 0x9
#define SPRINT 0xA
#define END 0xB

Our 32-bit signed integer type will come from the stdint header from the C standard library with the name int32_t.

Stack

So as we know we need a stack in order to do that we need to define one and write utility functions for the push and pop actions.

So first define the stack and the stack pointer using int32_t, now we will set the stack size as 256 for simplicity:

int32_t stack[256];
int sp = 0;

And then we write the push utility function:

// ask for the stack and stack pointer in case the functions must be used for multiple stacks.
void push(int32_t* stack, int* sp, int value) {
	// set our value in the index that the pointer points to
	stack[*sp] = value; 
	// increment the stack pointer to prepare room for the next value
	(*sp)++;
}

Then go on to write the pop utility function:

int pop(int32_t* stack, int* sp) {
	(*sp)--; // decrement the stack pointer
	return stack[*sp]; // return the stack element pointed to
}

Pool

Now we must define our pool, to make it a little less annoying to look at we can define a struct for our constant:

typedef struct {
	int location; // the index in the pool
	char* value; 
} const_t;

And then define our pool as an array of our structs as well as add our first entry to prepare the pool for our SPRINT:

const_t pool[128];
pool[0] = (const_t){0, "github.com/ethaanol/re-manual"};
pool[1] = (const_t){1, "Hello from the manual VM!"};

Execution

This is where the main part goes, we must write a function to execute our instructions, so let's begin by supporting SPRINT so we can print out the awesome pool constants, we will use int8_t from the C standard library to represent a byte:

void exec(int32_t* stack, int* sp, const_t* pool, int8_t* code) {
	int pc = 0; // define our program counter
	size_t size = sizeof(code) / sizeof(code[0]);

	// make our loop
	while (pc < size) {
		// match opcodes
		int opcode = code[pc];
		switch (opcode) {
			// SPRINT will take 2 bytes for the entry index
			case SPRINT: {
				int b1 = code[++pc];
				int b2 = code[++pc];

				// construct the index
				int idx = (b1 << 8) | b2;
				// get the constant from the pool using the index
				const_t entry = pool[idx];

				printf("%s\n", entry.value);
				pc++; // advance pc to the next instruction
				break;
			}
		    // finish executing the program
			case END:
				return;
			default:
				break;
		}
	}
}

Now we can use the exec function to run our program:

int32_t stack[256];
int sp = 0;

const_t pool[128];
pool[0] = (const_t){0, "github.com/ethaanol/re-manual"};
pool[1] = (const_t){1, "Hello from the manual VM!"};

int8_t code[] = {
	// sprint #0
	// sprint #1
	// end
	SPRINT, 0x00, 0x00,
	SPRINT, 0x00, 0x01,
	END
};
exec(stack, &sp, pool, code);

And congratulations, you have written your first virtual machine! Test it.

0xb

Now I don't have the time to finish this VM sadly but maybe soon it will be done. You should somewhat understand now how a virtual machine works.