Skip to content

SoB Week 11 and 12

Overview

Similar to my previous update, my work was quite scattered recently, so I decided to combine the updates for the last couple of weeks into a single post. Over this two-week span, I focused primarily on running the coverage experiments and preparing to start the survival experiments.

Goals Accomplished

  • Made VM memory allocation configurable: Something that bothered me a lot when running the coverage trials was having to manually edit the VM memory allocation for a single QEMU instance every time I wanted to run a new campaign. I wasted quite a few CPU hours because I forgot to decrease the VM image size from 4 GB to 2 GB, causing trials to crash. I discussed the issue with Matt and Nishant, and we agreed that setting a default value of 2 GB alongside an option to override it via an environment variable was a much better alternative. I created a PR for it, which was subsequently merged. You can view it here.

  • Add exec timeout in the orchestrators, started ablation campaign: During an earlier meeting, Nishant mentioned that the IR scenario needs a much longer timeout than the default AFL++ value because our IR programs take quite some time to execute against a live target. I modified our coverage and survival evaluation orchestration scripts to include this extended timeout. With that in place, I finally ran the first ablation trial: ir-full-stack vs ir-splice. I had run this campaign previously, but I needed to rerun it to account for the new changes; although the results didn’t differ all that much. You can see the new results here. I have also started running the other ablation trials, which should finish up in the coming week.

  • Finished the vulnerability fixes: This was the biggest roadblock to my paper, and I am incredibly happy to say I finally got it sorted out. All the per-vulnerability Docker images now build correctly and can be managed using our orchestration script. The only thing left to do is run the final trials.

  • Finished building the required docker images: With the vulnerabilities fixed, I went ahead and built the Docker images that will be used for the actual trials, as this step takes quite a bit of time to complete. All the images are now built, and I have kicked off a pilot campaign to help us determine which vulnerabilities are worth keeping and which should be discarded from the final set.

  • Advanced the paper: I refined the paper even further. Since the primary coverage results have already been determined, I integrated those findings into the draft. A couple of sections remain, but the majority of the remaining work is just dropping in the final results as they arrive.

  • Triaged a crash in the campaign: While running the ablation trials, I encountered what looked like a memory exhaustion bug for CLN. I triaged it and determined it to be a false positive, but I want to discuss it with Matt and Nishant next week just to be absolutely sure.

Next Week’s Goals

  • Modify the developer-facing orchestrator script: The developer-facing orchestration script still relies on the default AFL++ timeouts instead of a custom one. I need to create a PR to fix this. Additionally, I want to find a way to automatically output the Smite commit hash used during a run into the final evaluation report. I’ve had multiple instances where I had to discard coverage data because I accidentally used an older version of Smite.

  • Finish the ablation trial reports: As the ablation trials finish up, I will continue posting the results to the corresponding GitHub issue. This is mostly hands-off work and shouldn’t take long to complete.

  • Possibly redesign SpliceMutator: While the SpliceMutator isn’t particularly useful in its current form, I still believe the core idea of splicing together multiple corpus entries holds value for our fuzzer. If time permits, I would like to make the SpliceMutator leaner and formally evaluate it using our established pipeline.

  • Start the ground-truth campaign: This should have been done a long time ago, but alas, better late than never. I plan to start the survival analysis campaign for the ground-truth evaluation as soon as possible. I will have to cut some corners to save time, but there isn’t much I can do about that now.

Challenges

Fixing the vulnerabilities took much longer and was a far more grueling process than I expected. It was honestly stupid of me not to get on top of it earlier, and that delay is exactly why my campaign has to be trimmed down now. I also have to write the final SoB report next week, so I will need to set aside dedicated time for that as well.

On a different note, I appeared on the Bitcoin Optech Recap Podcast (episode #416) to talk about the two memory exhaustion vulnerabilities I discovered in CLN last year with Summer of Bitcoin. It was a really fun experience, and I think I did much better than my first time on a podcast.

I’ve also asked Matt to be a co-author on the research paper, since a large part of Smite is his brainchild. Hopefully, he agrees to it!

Till next time,

Chandra.