The problem with fuzzing

To compound the conception that these environments are becoming more difficult to test, monolithic black box fuzz testing, while frequently efficacious in its purpose, has a tendency for a exhibiting a lack of potency. The term ``monolithic'' is included as a reference to a comprehensive execution of the entire application or driver. Fuzzing is often executed in an environment where the tester does not know the internals of the binary in question. This leads to disadvantages in which a large number of tests must be executed to get an accurate estimate of binary's reliability. This investigation can be a daunting task if not implemented in a constructive manner. The test program and data selection should ensure independence from unrelated tests or groups of tests, thereby gaining the ability of complete coverage by reducing dependency on specific variables and their decision branching.

Another disadvantage of monolithic black box fuzz testing is that it is difficult to provide coverage analysis even though the testing selection may cover the entire suite of security testing models. A further complication in this nature of testing is of cyclic dependency causing cyclic arguments which in turn leads to a lessening of coverage assurance.