Anirudh Murali
Anirudh Murali

Founder, Economize

Back to blog

Extending Envoy’s fuzzing coverage in GSoC 2018

In 2018, I spent my second Google Summer of Code working on Envoy, a C++ proxy for cloud-native applications, through the Cloud Native Computing Foundation (CNCF). I added fuzz tests for string utilities, bootstrap configuration, and an HTTP/1 path that responds without an upstream server. I also worked through failures that Google’s OSS-Fuzz service had already found. My mentors were Matt Klein, Harvey Tuch, and Constance Caramanolis.

Finding my way into Envoy

I first heard about CNCF while interning at Hasura, where the team used Kubernetes extensively. When the GSoC organizations were announced, I spent hours reading CNCF’s project ideas. I knew I wanted to apply, but the codebases felt huge and I was still trying to understand what the projects did.

I’d worked on a large C++ codebase the previous summer with Haiku. I told Amit Kumar Jaiswal, who had done GSoC with Kubernetes, that I was looking for another C++ project. He pointed me toward Envoy. I joined an Envoy community meeting and understood very little of the technical discussion, but I got to see how people in different places worked through problems together. After talking with Matt and Harvey about the fuzzing project, I sent in a proposal. On April 23, I opened the GSoC dashboard and saw that I’d been accepted. I was on cloud nine.

My first hurdle was getting a build I could work with. I ran into trouble with my Homebrew GCC setup, so I moved to an Ubuntu VM in VirtualBox. Later, when a fuzzer build on my MacBook Pro was taking 40 to 50 minutes, I switched to a Linux VM on Google Cloud and worked over SSH with Vim and tmux. Shorter builds meant I could try a change, run the fuzzer, and get back to the code while the problem was still fresh.

What I added

Fuzzing feeds a program many generated or mutated inputs and looks for failures such as crashes. A fuzz test has to give those inputs a meaningful path through the code. Harvey helped me get oriented in Envoy’s existing fuzz tests and its coverage tracker, which gave me somewhere concrete to start.

  • String utilities. My first merged contribution exercised several of Envoy’s string utility functions with fuzzed inputs. It was a small target, which made it a good place to learn how to build a test and work through review.
  • Bootstrap configuration. I added a test that feeds a fuzzed bootstrap configuration to Envoy’s server in validation mode. I expected this to be a small change to the existing server fuzzer. Getting the test to run in the right mode took me much longer than I expected.
  • HTTP/1 direct responses. The existing capture fuzzer exercised both the client-facing and upstream sides. I added a separate test that configures Envoy to respond directly to the client. With no upstream connection involved, the test can exercise the downstream path on its own.

Working through OSS-Fuzz failures

Some of my work started with a single input that already caused a failure.

  • Invalid percentages. Passing NaN to a percent conversion helper triggered an assertion. I made Envoy reject the value with an exception and kept the failing input in the test corpus.
  • Multiple health checks. I replaced an assertion for an unsupported configuration with an explicit error.
  • Endpoint priority. I capped the priority value that had exhausted the fuzzer’s memory.
  • Socket ports. I added an upper bound for port values.
  • Header formatting. I fixed START_TIME formatting when it produced multiple pieces of output.

I also changed how OSS-Fuzz locates the Envoy corpus. The old convention assumed the directory name ended in _corpus. Reading the path from the Bazel build definition let fuzz targets use a different directory name or share a corpus. My patch for an HTTP/2 codec fuzzing failure closed without merging.

What stayed with me

The string utility PR went through several rounds of review before it merged. Later, my mentors encouraged me to respond to review faster. Balancing that with classes took some practice, and I did plenty of work on weekends and late at night. I learned to budget time for the whole PR, including explaining my choices, responding to feedback, and running the tests again after a change.

Writing fuzz tests also forced me to understand what I was testing. I had to follow how Envoy handled a configuration or request before I could choose where to introduce an input and recognize what had gone wrong. That was the part of working in a large C++ codebase I couldn’t learn from the project ideas list.

Thanks to Matt, Harvey, Constance, and the Envoy community for helping me get started and for taking the time to review my work. Thanks also to Amit for pointing me toward Envoy, to Nikhita and Chris Aniszczyk at CNCF, and to the people at my university who gave me the time to do GSoC.

Project references

Project write-ups

Setup and fuzzing resources

Related documentation and issues

Screenshots from the original report

Some of the original image URLs may be unavailable.

Getting started

Project work