Babysitting an army of monkeys: fuzzing with Charlie Miller at CanSecWest
Charlie Miller, the PWN2OWN winner and security consultant with Independent Security Evaluators, guided the attendees at CanSecWest through “Babysitting an army of monkeys: An analysis of fuzzing four products with 5 lines of Python” today.
Miller, best known for pointing out numerous security flaws in Apple products, explained how he fuzzed (which consists of sending large amounts of error-filled and odd data in order to expose and exploit bugs) through “dumb fuzzing” which is taking good input, addng anomalies to it and aiming it at a program. “Smart fuzzing” anticipates what kind of data a program expects and working around it. Smart fuzzing can be more effective but is more work. A “compromise for the lazy” is to use dumb fuzzing with lots of initial files, which hopefully fuzzes all the features of a program.
Fuzzing, Miller said, isn’t about creating test cases as much as it is about filtering. Start with a ton of test cases, filter those that don’t work, and repeat until you find exploitable flaws.
Specific applications that proved vulnerable on OS X included Adobe Reader 9.2.0. Mller created over 1500 files, fuzzed them to create over 3 million fuzzed files, then pummeled Reader with those files and noted the crashes they caused.
Miller repeated the process in Preview, Apple’s graphic viewer. The bugs that show up in Preview also appear in Safari. Of all the files he threw at Preview, five percent crashed the program, something Miller termed “not good”.
But what of the iPhone? Miller said most of the bugs he’s found don’t work on the iPhone, which has a much smaller code base. Mobile Safari will show PDFs, but it’s a much smaller browser and doesn’t “have much to work with.”
In a PDF viewer showdown, Reader came out far ahead of Preview. Miller then tried fuzzing Open Office and Microsoft Office Powerpoint (2008 for Mac). He found that in this contest, Open Office was a little more exploitable than PowerPoint, but not by much.
His conclusions were that vendors should fuzz their products, and fix the bugs they find, or else someone else will find them and possibly exploit them.
