Yihan Lian & Zhibin Hu - Smarter Peach: Add Eyes to Peach Fuzzer [rooted2017]

Peach is a smart and widely used fuzzer, which has lots of advantages like cross-platform, aware of file format, extend easily and so on. But when AFL fuzzer has appeared, peach seems to be out of date, since it doesn't have coverage feedback and run slowly. Due to peach is a flexible fuzzer framework and AFL is not, I extended peach with AFL advantages, making it more smarter.Just like AFL, I use LLVM Pass to add coverage feedback, with that I can see which mutation is interesting viz. explores new paths. The resultant effect is that the modified version is more effective.

1. Smarter Peach: Add Eyes to Peach Fuzzer Yihan Lian && Zhibin Hu/ Qihoo 360 Gear Team

2. About us Yihan Lian security researcher at Gear team of Qihoo 360 focused on vulnerability discovery of Open-Source-Software. got more than a dozen CVE last year (e.g. qemu , ntp , ffmpeg ) Fan of Xavi. Zhibin Hu security researcher at Gear team of Qihoo 360 l ast several years mainly focus on vulnerability discovery and analysis on windows, and receive msrc top 19 in 2014. recent two years interested in cloud security.

3. Agenda M otivation Peach fuzzer framework Strengths and weaknesses of Peach fuzzer A dd eyes to Peach fuzzer Performance   comparison and conclusion Demo Questions

4. motivation Stop re-inventing the wheel. Peach is mature,  high stability and strong  expandability  etc. Coverage-feedback make traditional fuzzer more efficient. The mutate strategy of AFL is a little crude, not aware of file format.

5. Comparison of mainstream fuzzers FOE Peach AFL Coverage feedback √ Source Code Agnostic √ √ Aware of file format √ aware of protocol format √

6. Peach fuzzer framework

7. Peach fuzzer framework Original Peach core framework Switch Iteration Test Case output to target Monitor Log Fault Normal File Format File Database OriginalCase Mutate  

8. Strengths and weaknesses of Peach fuzzer Strengths Aware of file format, mutation strategies are more flexible Smarter when communicating with target Cross platform weaknesses Fat and complex No Coverage Feedback

9. aware of file format 1/5 Basic elements: Number, String and Blob etc “\ xAB \ xCD ” is hex Number “ABCD” is String “AB\ xCD ” is Blob Describe relationships in the data : Relation, Fixup and Hint etc Relation: 32 is the numbers of bits of “\x41\x41\x41\x41” Fixup : 0x9B0D08F1 is CRC-32 of “\x41\x41\x41\x41” Hint: Assign a more exquisite mutate strategy

10. aware of file format 2/5 If we need to fuzz a function like this: if( data_head is "PNG "){ switch( data_tag ){ case ("IHDR"): if( check_CRC ) { Parse(data); core_code ; } else return ; c ase ... } }

11. aware of file format 3/5 Example: PNG head Elem_0 = data[0-7] type = Blob mutator = Blob mutators Elem_1 = data[8 – 0xB] type = Number_hex mutator = hex mutators //Elem_1 is the length of Elem_3 ~ elem_9. Elem_2 = data[0xC – 0xF] type = String mutator = ASCII or UNICODE … mutators Elem_3 = data[0x10 – 0x13] type = Number_hex Elem_4 = data[0x14 – 0x17] type = Number_hex Elem_5 = data[0x18] type = Number_hex …... Elem_10 = data[0x1D - 0x21] type = Number_CRC_of_elem_2-9 fixup = calculate CRC //Elem_10 is the CRC of Elem_2 ~ Elem_9.

12. aware of file format 4/5 DataModel in Peach Pit < DataModel name=" PNGTest "> < Blob name="head" length=" 8“ mutable=“false”/> < Number name=" dataLength " size="32" valueType ="hex" endian="big"> < Relation type="size" of="chunk" expressionGet ="size+4" expressionSet ="size-4"/> </ Number > < Block name="chunk"> < String name=" chunkHead " length="4 "> <Hint name=“ ValidValues ” value=“ pHYs;fcTL;IDAT ;”/> </ String > < Number name="width" size="32" valueType ="hex" endian="big"/> < Number name=" height" size="32" valueType ="hex" endian="big"/> < Number name=" enum " size="8" minOccurs ="1"/> </ Block> < Number name="CRC" size="32" endian="big"> < Fixup class=" Crc "> < Param name="ref" value="chunk"/> </ Fixup > </ Number > </ DataModel >

13. aware of file format 5/5 The mutated file still triggers core code. The length of CRC of

14. Sm art er when communicating with target 1/3 Most of fuzzers send mutated-data to target application, but can not receive valuable data from it. They are hard to fuzz servers which need to communicate client.

15. Sm art er when communicating with target 2/3 Action: Send commands through Publisher. Read and write data from target. Publisher: I/O interfaces. it could be a file or a traffic data.

16. Sm art er when communicating with target 3/3 Example: … < Action type=" input " publisher =“Target" > < DataModel name=" InputModel " ref =“model_1" /> </Action > < Action type=" slurp " valueXpath ="// InputModel // TransID ” setXpath ="// OutputModel // TransID "/> < Action type=" output " publisher =" Target "> < DataModel name=" OutputModel ” ref=“model_2”/> < Data fileName =" data/request" /> </Action > … < Publisher name=“Target" class =“ Udp "> < Param name="Host" value =“192.168.11.11" /> < Param name="Port" value =“6666" /> </ Publisher >

17. Cross platform Peach supports all major Operating-Systems. Windows Install  Microsoft.NET v4 Runtime Install  Debugging Tools for Windows Unzip Peach binary distribution to a working folder OS X Install latest  Mono packages Install Crash Wrangler ( download ) Unzip Peach binary distribution to a working folder Linux Install latest Mono packages Ubuntu/ Debian : mono-complete package SUSE: See  download instructions Unzip Peach binary distribution to a working folder Remote fuzz

18. weaknesses Fat and complex Needs to parse the seed file once in every fuzz-iteration. N eed to store too many data models and actions. No Coverage Feedback Peach belongs to Black-Box-Fuzzer. Cannot distinguish which mutated file is more valuable.

19. A dd eyes to Peach fuzzer

20. Add eyes to Peach fuzzer

21. Problems How to detect code coverage How to return coverage info better How to store valuable files How to reproduce valuable files How to select a valuable file for next fuzz circle

22. How to detect code coverage 1/3 Use LLVM Pass to insert codes in Basic Block Source code: IDA output:

23. How to detect code coverage 2/3 IDA after inserting codes l lcov_pcov_block_call is inserted into the Basic-Block of target program.

24. How to detect code coverage 3/3 llvm_pcov_block_call : llvm_pcov_block_call function is used to mark whether this Basic-Block was run before, and message to Peach the amount of New-Basic-Block in this fuzz iteration.

25. How to return coverage info better We need to insert codes in a lot of basic-blocks. In order to be more efficiency, we pass info between llvm_proc_block_call and Peach through shared memory.

26. How to store valuable files 1/3 We cannot store valuable files directly, since the valuable files does not match the File-Format in most cases, and this could raise a Peach exception. Just like this: [*] Test 'Default' finished. Error, failed to crack “**data\ png \test.png " into " PNGTest ": Block ' PNGTest.chunk ' has length of 1744830496 bits but buffer only has 168 bits left .

27. How to store valuable files 2/3 I choose to store the Fuzz status.

28. How to store valuable files 3/3 What is Fuzz status Name of seed file – file used to be mutated. Random status – some numbers used to generate a random number. Action name – action used to generate this I-File. Mutator name – mutation strategy name used before. Etc.

29. How to reproduce valuable files 1/2 On every switch- teration , Peach will store lots of information about each seed file to a Original-Data-Model object. All of the mutated data is mutated on this object.

30. How to reproduce valuable files 2/2 What we should to do is modify the original-Data-Model before this iteration beginning. File Format File Database File choice and parse

31. How to select a valuable file for next fuzz circle 1/2 There will be many valuable files after fuzzing a period of time. Valuable files info: fileNameIs0::***bin/data/ mov /test_1.mov 33 lastMutatorList lastMutatorRun_1.Initial.Action.... fileNameIs1 ::***/bin/data/ mov /test_1.mov 32 lastMutatorList lastMutatorRun_1.Initial.Action.... lastMutatorList lastMutatorRun_1.Initial.Action.... fileNameIs2::***bin/data/ mov /test_2.mov ...

32. How to select a valuable file for next fuzz circle 2/2 If a valuable file gets more New_Block_Counts , it will get more weight. And if the mutated files mutated from it get New_Block_Counts else, the weight of it will increase at the same time. Whereas: the weight will decrease. This strategy decrease or increase the weight in linear.

33. performance comparison

34. Performance comparison and conclusion After fuzzing FFMPEG with mutating MOV-file 24 hours by Peach-original, Peach- cov and AFL , the difference between the amount of triggered source code related with MOV-file is very clear. For the AFL case, it will trigger other code which is not relevant with MOV-file , so we only calculate relevant part.

35. Performance comparison and conclusion Preponderance: Is more efficient than original Peach fuzzer Is more efficient than AFL when comparing the depth of triggered source code. Future work: Strategy of valuable-file chosen use Markov chain like AFL-FAST a dd weight on mutators P erformance improvement use fork-server like AFL Automatic learn the format u se machine learning like: https :// arxiv.org/pdf/1701.07232.pdf

36. DEMO

37. Questions

38. Thank you Yihan Lian && Zhibin Hu/Gear Team , Qihoo 360 Inc [email protected] [email protected]