Chapter 3. Crawlers and Spiders
In this chapter, we will cover:
Downloading a page for offline analysis with Wget
Downloading a page for offline analysis with HTTrack
Using ZAP's spider
Using Burp Suite to crawl a website
Repeating requests with Burp's repeater
Using WebScarab
Identifying relevant files and directories from crawling results