StorFuzz: Using Data Diversity to Overcome Fuzzing Plateaus

Fuzzing is widely used to discover software bugs and vulnerabilities. Unfortunately, real-world long-running fuzzing campaigns often plateau and no progress can be made anymore, leaving code areas untested. State-of-the-art fuzzers leverage code coverage to measure progress and reach new areas, but this is insufficient to capture all program behavior, as code coverage may be the same for different behaviors, thus preventing progress and masking bugs.

In this paper, we introduce StorFuzz, a novel technique to overcome fuzzing plateaus and improve on code coverage by leveraging our new data coverage. StorFuzz automatically identifies and instruments memory stores to capture changes in program behavior invisible to control flow, which it uses to diversify the saturated corpora of plateaued campaigns. StorFuzz leverages this diversified corpus of test cases that changed internal states to improve navigation of the input space, which also enables conventional fuzzers to improve their code coverage. We implement StorFuzz in LibAFL and evaluate on FuzzBench, starting from a corpus that is saturated by multi-month OSS-Fuzz fuzzing campaigns and LibAFL.

We show that StorFuzz successfully generates new coverage for plateauing campaigns of widely-used and well-fuzzed software, leading to the discovery of 50 new bugs in 7 OSS-Fuzz projects, like VLC and PHP, with some bugs having been present in the code for 14 years. Our approach significantly outperforms both the state-of-the-art fuzzer LibAFL and data-guided fuzzer DDFuzz in 11 of 23 FuzzBench benchmarks, while performing equally on all others. StorFuzz is also complementary to WingFuzz, an approach guided by static data, as both fuzzers cover distinct code regions.

Source Code and Data: github.com/rub-softsec/StorFuzz

Source Code

StorFuzz is based on LibAFL revision bb579e6 (v0.13.1), the source code along with usage instructions can be found in rub-softsec/StorFuzz-LibAFL.

FuzzBench

We include all fuzzers and configurations used in our evaluation as FuzzBench fuzzers: rub-softsec/StorFuzz-FuzzBench.

The repository includes information on how to replicate the experiments presented in the paper.

LibAFL fuzzbench fuzzer patch

The fuzzbench fuzzer for LibAFL is based on revision bb579e6. The patch applied to the fuzzer can be found in fuzzbench_stats.patch.

DDFuzz Fuzzer Implementation

The DDFuzz fuzzer for LibAFL is implemented on top of revision bb579e6. The patch file can be found in ddfuzz_fuzzer.patch.

Ablation Study

The configurations used in the ablation study are included as individual fuzzers in FuzzBench.

Reduction Functions

The three alternative reduction functions tested in the ablation study, could be implemented in C as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

uint8_t value_reduction_4bits(uint64_t v) {
	uint8_t temp = ((val >> 8) ^ val);
	return ((temp >> 4) ^ temp) & 0xF;
}

uint16_t value_reduction_12bits(uint64_t v) {
	return  (((val & 0xFF00) >> 4) ^ (val & 0xFF)) & 0xFFF;
}

uint16_t value_reduction_16bits(uint64_t v) {
	return  val & 0xFFFF;
}

Additional Data

The following additional tables can be found in tables.pdf:

• Corpus sizes of the seed corpus vs. the diversified corpora. Assessing the Diversity
• Coverage at different times for median trial starting from the saturated corpus. Transferring the Diversity: LibAFL
• Coverage at different times for median trial starting from the saturated corpus. Transferring the Diversity: WingFuzz
• Bugs discovered by StorFuzz.
• Edges covered by different coverage guided fuzzers that consider dataflow.

Additional Coverage Over Time Plots

Here are code coverage over time plots for all FuzzBench benchmarks:

• 4.3 – Getting to the Plateau
• 4.4 – Breaking out of the Plateau
• 4.6 – Transferring the Diversity: LibAFL
• 4.7 – Transferring the Diversity: WingFuzz
• 4.7 – Transferring the Diversity: WingFuzz (plotted with saturation period)
• 4.9 – Ablation Study
• Starting “from Scratch” (from a Non-Saturated Corpus)

Seed Corpora & Long-Term Archive

The OSS-Fuzz corpora and the saturated corpora can be found on Zenodo. This package also includes this repository as a long-term archive.

Acknowledgements

This work is based on research supported by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy - EXC 2092 CASA - 390781972, as well as the Vienna Science and Technology Fund (WWTF) and the City of Vienna [Grant ID: 10.47379/ICT19056]. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the respective funding agencies.