Ben Murdoch | f05b935 | 2011-05-05 14:36:32 +0100 | [diff] [blame] | 1 | Welcome to iExploder. a highly inefficient, but fairly effective web |
| 2 | browser tester. The code still has a lot of work to be done, but it's |
| 3 | definitely usable. Here are some notable features: |
| 4 | |
| 5 | * Tests all HTML tags and CSS attributes, as parsed from various |
| 6 | open-source browsers. |
| 7 | * HTTP Header testing |
| 8 | * Basic Media format fuzzing (jpeg, png, snd, wav, etc.) |
| 9 | * Numeric, and String overflow and formatting tests |
| 10 | * Sequential and Randomized Test Case Generation |
| 11 | * Test Case Lookups |
| 12 | * Subtest generation |
| 13 | * Test harness mode that controls your browser process and testcase |
| 14 | generation. |
| 15 | |
| 16 | Requirements: |
| 17 | ------------- |
| 18 | Make sure you have Ruby installed (comes with Mac OS X, most Linux |
| 19 | distributions). See http://www.ruby-lang.org/ if you do not. |
| 20 | |
| 21 | |
| 22 | Harness mode (Mac OS X, Linux, other UNIX based operating systems) |
| 23 | ------------------------------------------------------------------- |
| 24 | In this mode, iExploder controls the stopping and starting of your |
| 25 | web browser, reproducing crashes, and writing test cases. This is |
| 26 | the recommended mode of operation for most cases. |
| 27 | |
| 28 | |
| 29 | Usage: browser_harness.rb [options] -- <browser path> <browser options> |
| 30 | -t, --test NUM Test to start at |
| 31 | -p, --port NUM Listen on TCP port NUM (random) |
| 32 | -c, --config PATH Use PATH for configuration file |
| 33 | -d, --testdir PATH Use PATH to save testcases (/tmp) |
| 34 | -l, --logdir PATH Use PATH to save logs (/tmp) |
| 35 | -w, --watchdog NUM How many seconds to wait for pages to load (45s) |
| 36 | -r, --random Generate test numbers pseudo-randomly |
| 37 | -s, --scan NUM How often to check for new log data (5s) |
| 38 | -h, --help Display this screen |
| 39 | |
| 40 | Here is an example use with Chrome starting at test number 1000: |
| 41 | |
| 42 | % ./browser_harness.rb -t 1000 -- /usr/local/chrome-linux/chrome --incognito |
| 43 | |
| 44 | For proper use, the harness mode must *ALWAYS* be used with the browser |
| 45 | configured to not restore sessions after a restart. Here are some example |
| 46 | command lines to use: |
| 47 | |
| 48 | chrome --incognito |
| 49 | opera --nosession -newprivatetab |
| 50 | firefox -private |
| 51 | |
| 52 | On Mac OS X you may call the binary directly, or use the .app directory. The latter |
| 53 | is required for Safari.app, but does not allow arguments to be passed. Here is an |
| 54 | example: |
| 55 | |
| 56 | % ./browser_harness.rb /Applications/Safari.app |
| 57 | |
| 58 | By default, all testcases and logs will be written to ../output |
| 59 | |
| 60 | |
| 61 | |
| 62 | Viewing testcases: |
| 63 | ------------------ |
| 64 | Many test-cases make use of references to external objects (ogg, jpg, etc.) |
| 65 | where we are fuzzing the HTTP header data. When the browser harness saves a |
| 66 | testcase in HTML form, it rewrites all references to these external objects |
| 67 | to refer to http://127.0.0.1:3100/ |
| 68 | |
| 69 | To properly view these saved .html testcases, please run the built-in |
| 70 | webserver in the background. |
| 71 | |
| 72 | % ruby webserver.rb |
| 73 | |
| 74 | |
| 75 | Standalone Webserver mode: |
| 76 | -------------------------- |
| 77 | If you do not already have a webserver setup, you can use the server |
| 78 | built into iexploder. Simply go into the src/ directory and type: |
| 79 | |
| 80 | % ruby webserver.rb |
| 81 | |
| 82 | A webserver will then start on port 3100 with the iexploder form. You can |
| 83 | also pass a -p or --port option to select a different location: |
| 84 | |
| 85 | % ruby webserver.rb -p 2001 |
| 86 | |
| 87 | All requests will be logged to the path specified in 'access_log_path' |
| 88 | parameter in config.yaml. |
| 89 | |
| 90 | |
| 91 | |
| 92 | Third-party webserver mode: |
| 93 | --------------------------- |
| 94 | Copy the contents of the src/ folder to any directory served |
| 95 | by your webserver. Make sure that directory can execute CGI scripts. |
| 96 | Performance is likely to be very slow unless you use something that |
| 97 | keeps the interpreter alive like mod_ruby. |
| 98 | |
| 99 | |
| 100 | FAQ: |
| 101 | ---- |
| 102 | 1) Are the tests always the same? |
| 103 | |
| 104 | The test cases should always be the same on a single installation, but not |
| 105 | necessarily on different installations of iExploder. Random generator seeds |
| 106 | may differ between operating systems and platforms. If you alter config.yaml, |
| 107 | it is likely to change the test cases as well. |
| 108 | |
| 109 | |
| 110 | 2) I found a crash - how do I stop testing for it? |
| 111 | |
| 112 | See the 'exclude' section of config.yaml. It allows you to blacklist certain |
| 113 | tag combinations that are known to result in a crash condition. |
| 114 | |
| 115 | |
| 116 | 3) How do I look up the last successful test for a client? |
| 117 | |
| 118 | Look at your access log. There is a handy tool to parse access logs and show |
| 119 | the most recent test for each host and user-agent combo. Try: |
| 120 | |
| 121 | tools/lasthit.rb /path/to/access_log |
| 122 | |
| 123 | |
| 124 | 4) How do subtests work? |
| 125 | |
| 126 | Subtests are how iexploder attempts to isolate the crashing line of code for |
| 127 | a particular HTML document. It's a multi-pass algorithm, which delivers |
| 128 | nasty tags to your browser in the following order: |
| 129 | |
| 130 | * 1 combination, single line |
| 131 | * 2 combinations, 3 lines per combination |
| 132 | * 3 combinations, 5 lines per combination |
| 133 | * 4 combinations... |
| 134 | * 5 combinations... |
| 135 | * Your original document (in case we haven't crashed by now) |
| 136 | |
| 137 | |
| 138 | 5) How come I can't seem to repeat the crash? |
| 139 | |
| 140 | Many browser crashes are race conditions that are not easy to repeat. Some |
| 141 | crashes only happen when going from test 4 -> test 5 -> test 6. If you can't |
| 142 | repeat the crash through subtests or a lookup of the failing test, try going |
| 143 | back a few tests. |
| 144 | |
| 145 | That said, some crashes are due to race conditions that are very difficult |
| 146 | to replicate. |
| 147 | |
| 148 | |
| 149 | 6) Why did you write this? |
| 150 | |
| 151 | I wanted to make sure that FireFox had as many bugs fixed in it as possible |
| 152 | before the 1.0 release. After 1.0 came out, I kept improving it. |
| 153 | |
| 154 | |
| 155 | 7) Why does Internet Explorer run the tests so slowly? |
| 156 | |
| 157 | <META> refresh tags are very fragile in Internet Explorer, and can be easily |
| 158 | be rendered useless by other tags on the page. If this happens, a javascript |
| 159 | refresh will execute after a 1 second delay. |
| 160 | |
| 161 | |
| 162 | 8) How do I change the number of tags iExploder tests per page? |
| 163 | |
| 164 | See config.yaml. |
| 165 | |
| 166 | |
| 167 | 9) What other performance enhancements can I make? |
| 168 | |
| 169 | * Use Private Browsing or Incognito mode in your browser |
| 170 | * Before using iExploder, clear your browser history |
| 171 | * Minimize your browser while iExploder is running |
| 172 | * If you are using browser_harness, try adjusting the -w and -s options. |
| 173 | |
| 174 | |