📊Statistics
On the coverage metrics reported by Diligence Fuzzing
Last updated
On the coverage metrics reported by Diligence Fuzzing
Last updated
When looking at a campaign report, you'll prominently find the most important statistics:
Total instruction coverage
Analysis duration
Residual risk
Testcases per second
You'll probably be good with just those metrics, but if you want more insight into what the fuzzer is doing, then you should check out "🚀 Pro Mode" which adds a ton of metrics to your report!
Branch Coverage
Path Coverage
Number of issues found
Transaction Depth
Transaction Status
The rest of this page will describe the different types of coverage, what they mean, and how you can use them to optimize your fuzzing campaigns.
Instruction coverage tells you the percentage of instructions in the target contracts that have been covered.
Instruction coverage doesn't translate directly to line coverage, as not every line is equally represented in the bytecode instructions.
Instruction coverage gives you a good overview of the amount of features/functions that have been covered by fuzzing. A low coverage usually means that large parts of the code are not explored by the fuzzer.
We display line coverage as green markers in the file-view window. While fuzzing we mark a line as covered (green) whenever we execute an instruction that maps to code on that line.
You might notice that line coverage doesn't highlight every line in a function, even though it seems like it should have covered 100% of it. This usually happens when there are empty lines, code comments, structure definitions, etc. We don't count these lines as executable, so we keep them unmarked.
The picture below shows how fuzzing covered the transfer function completely
Line coverage is super helpful to investigate why a fuzzer is getting stuck (if it's getting stuck).
One example that we frequently encounter is forgetting to mint tokens for a fuzzing account in the seed state. In such a scenario, all transfers from the fuzzing account will result in a revert, blocking the fuzzer from further exploring the function that tries to perform the transfer.
Branch coverage tells you the absolute amount of branches that were covered during the fuzzing campaign.
You can find a branch at any point in your code where execution can "jump" to one or more places; for instance, in the conditional below:
This piece of code has three branches:
1. Line 2 -> Line 3 (a == False) 2. Line 4 -> Line 5 (a == False) 3. Line 2 -> Line 5 (a == True)
We measure branch coverage at the EVM level! The number of branches in the EVM bytecode doesn't necessarily match the number of branches in your Solidity code.
A growing number of covered branchs tells us that the fuzzer is covering more and more edge-cases in the code.
A growing number of paths tells us that the fuzzer is finding more and more test-cases that each let the code follow a new unique path. Even after instruction coverage and branch coverage converge, you'll often see this metric still growing. You can feel a little bit more safe with each additional path covered!
