squaresLab/codealigneval: A repository for reproducing the experiments used to evaluate Codealign (https://github.com/squaresLab/codealign) · GitHub

This replication package contains the resources needed to replicate the experiments used to evaluate Codealign, a technique for evaluating neural decompilers introduced in Fast, Fine-Grained Equivalence Checking for Neural Decompilers.

For Codealign itself, see the main tool repository.

To install dependencies using conda, run

$ conda env create -f environment.yml
$ conda activate codealigneval

It may a while for conda to solve the dependencies.

Many of the dependencies are present to support the neural decompilation experiments. The other experiments require four packages:

• z3-solver
• tree_sitter==0.20.4
• tqdm
• numpy

For the remaining steps, you'll need pip. Make sure you're using the pip from the conda environment, not the system pip. To check that you're using the right one, run

$ which pip

If the output isn't <your anaconda install location>/anaconda3/envs/codealigneval/bin/pip, find where that pip is located and run it directly using the absolute path.

If you'd like to generate VarCLR scores in the neural decompilation experiment, then clone a copy of the repository:

$ git clone https://github.com/squaresLab/VarCLR.git
$ cd VarCLR
$ pip install .

If you do not have a rust compiler installed, you will need to modify VarCLR's setup.py to use any version of the Transformers library, not just the one specified. Change "transformers==4.5.1", (line 22) to "transformers", Then you can

$ pip install .
$ cd ..

Then install codealign with

git clone https://github.com/squaresLab/codealign.git
cd codealign
$ pip install .
cd ..

Instructions for running individual experiments are located in directory for each experiment:

• symexe: for the symbolic execution alignment comparison
• runtime: for runtime performance evaluation
• neural_decompilation: for the neural decompilation experiment.