Skip to content

SoB Week 9 and 10

Overview

Because progress was somewhat scattered recently, I decided to merge the updates for the last couple of weeks into a single post. Over this two-week span, I primarily focused on advancing the research paper and finally kicking off the evaluation experiments.

Goals Accomplished

  • Fleshed out the research paper: I drafted additional sections of the paper, bringing the first draft to near completion. However, the current version is still too wordy and far exceeds the acceptable page-count limit. I now need to refine each section by cutting the clutter and replacing dense paragraphs with space-saving alternatives like tables and figures.

  • Got the orchestration script merged: The developer-facing orchestration script turned out to be a bigger hurdle than I anticipated. I had to completely scrap my initial approach: using smitebot run for a single trial, and replace it with a custom invocation of a single afl-fuzz instance, as the previous method introduced too many complications. Testing the script locally caused breakages on certain targets, but with some help, I confirmed this was due to my local compute limits rather than bugs in the script itself. The script is now finalized and merged into the master branch (you can view the PR here).

  • Reviewed some PRs: I had some extra time within these past few weeks, so I decided to review some of the open PRs for Smite right now. I learnt that it is very helpful in keeping up with the changes in the tool’s behavior while lets me then improve my PR’s as well.

  • Sourced the experimentation server: I finally rented and configured the dedicated server for my experiments. It is a beefy machine, featuring 32 identical cores, 64 threads, and 128 GB of RAM. I secured a great deal on SpinServers.com and even used a discount code to lower the price further. I am hoping to finish all necessary experimental runs within my one-month rental window.

  • Conducted the SpliceMutator experiment: I have already put the new server to work, running the experiment to determine the efficacy of the SpliceMutator. This is the final piece of the puzzle required to decide whether this mutator is worth merging, and I am highly optimistic that it is. You can view the results here.

  • Started preparations for the survival analysis: The main attraction of my formal evaluation script is the ground-truth survival analysis. However, before running the actual experiment, I need to streamline the Docker scripts required to orchestrate the trials. I have already done this for CLN and can tell the rest will be a grueling process; hopefully, I can get past it sooner rather than later.

Next Week’s Goals

  • Finish preparations for the survival analysis: Getting the Docker images to successfully compile every single target alongside its respective vulnerabilities is the biggest hurdle remaining for the survival analysis. Fixing these Dockerfiles so that each vulnerability can run without a hitch is my absolute top priority for next week.

  • Refine the draft paper: While the first draft is complete, it is still a long way from being presentable. A substantial amount of material needs to be added, cut, or updated before I can call it finished. I have already streamlined several sections and will continue refining the rest in the upcoming week.

  • Get SpliceMutator merged: I believe the experimental evaluation provides the final justification needed to get this mutator merged. I will definitely nudge the PR if there is no activity on it in the coming week.

  • Triage the CLN memory crash: I couldn’t find the time for this (yet again), so I am carrying it over to next week (yet again). To be completely honest, I doubt I will actually get to it. I have been entirely consumed by the final stages of the project and the paper. Furthermore, the crash is incredibly hard to reproduce; I couldn’t trigger it even once on my new rented server.

Challenges

I was naïve to believe that my work for the survival analysis ended with sourcing the vulnerabilities and creating the flag patches. I am currently kicking myself for not testing whether the per-bug sharedirs built correctly earlier on. Still, there is no use crying over spilled milk; I just need to get on top of it and get these Dockerfiles working as soon as possible.

Till next time,

Chandra.