Scraping

PyQuery is able to load html document from urls:

>>> pq('http://duckduckgo.com/')
[<html>]

By default it use python’s urllib.

If requests is installed then it will use it. This allow you to use most of requests parameters:

>>> pq('http://duckduckgo.com/', headers={'user-agent': 'pyquery'})
[<html>]

>>> pq('https://duckduckgo.com/', {'q': 'foo'}, method='post', verify=True)
[<html>]

Previous topic

pyquery – PyQuery complete API

Next topic

pyquery.ajax – PyQuery AJAX extension

This Page