The Login form

The first form we'll automate is the Login form, which is available at http://example.webscraping.com/user/login. To understand the form, we can use our browser development tools. With the full version of Firebug or Chrome Developer Tools, it is possible to simply submit the form and check what data was transmitted in the Network tab (similar to how we did in Chapter 5, Dynamic Content). However, we can also see information about the form if we use "Inspect Element" features:

The important parts regarding how to send the form are the action, enctype, and method attributes of the form tag, and the two input fields (in the above image we have expanded the "password" field). The action attribute sets the HTTP location where the form data will be submitted, in this case, #, which represents the current URL. The enctype attribute (or encoding type) sets the encoding used for the submitted data, in this case, application/x-www-form-urlencoded. The method attribute is set to post...