XPath Selectors

There are times when using CSS selectors will not work. This is especially the case with very broken HTML or improperly formatted elements. Despite the best efforts of libraries like BeautifulSoup and lxml to properly parse and clean up the code; it will not always work - and in these cases, XPath can help you build very specific selectors based on hierarchical relationships of elements on the page.

XPath is a way of describing relationships as an hierarchy in XML documents. Because HTML is formed using XML elements, we can also use XPath to navigate and select elements from an HTML document. 

Note

To read more about XPath, check out the Mozilla developer documentation: https://developer.mozilla.org/en-US/docs/Web/XPath.

XPath follows some basic syntax rules and has some similarities with CSS selectors. Take a look at the following chart for some quick references between the two.

Selector description	XPath Selector	CSS selector
Select all links	'//a'	'a'
Select div with class "main...