python-requests简单介绍及⽤法
python请求并解析json数据requests是⼀个很实⽤的 HTTP客户端库，编写爬⾍和测试服务器响应数据时经常会⽤到。可以说，Requests 完全满⾜如今⽹络的需求安装⽅式⼀般采⽤$ pip install requests。其它安装⽅式参考官⽅⽂档
HTTP - requests
import requests
GET请求
r  = ('/get')
传参
>>> payload = {'key1': 'value1', 'key2': 'value2', 'key3': None}
>>> r = ('/get', params=payload)
Note that any dictionary key whose value is None will not be added to the URL's query string.
参数也可以传递列表
>>> payload = {'key1': 'value1', 'key2': ['value2', 'value3']}
< 返回headers中的编码解析的结果，可以通过r.encoding = 'gbk'来变更解码⽅式
r.json()返回JSON格式，可能抛出异常
r.status_code
r.raw返回原始socket respons，需要加参数stream=True
>>> r = ('api.github/events', stream=True)
>>> r.rawphp中的代表什么
<requests.sponse.HTTPResponse object at 0x101194810>
>>> ad(10)
'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03'
将结果保存到⽂件，利⽤r.iter_content()
with open(filename, 'wb') as fd:
for chunk in r.iter_content(chunk_size):
fd.write(chunk)
传递headers
>>> headers = {'user-agent': 'my-app/0.0.1'}
>>> r = (url, headers=headers)
传递cookies
>>> r = (url, cookies=dict(cookies_are='working'))
结束键盘输入的快捷键
>>> r.text
'{"cookies": {"cookies_are": "working"}}'
传递表单
r = requests.post('/post', data = {'key':'value'})
通常，你想要发送⼀些编码为表单形式的数据—⾮常像⼀个HTML表单。要实现这个，只需简单地传递⼀个字典给data参数。你的数据字典在发出请求时会⾃动编码为表单形式:
>>> payload = {'key1': 'value1', 'key2': 'value2'}
>>> r = requests.post("/post", data=payload)
>>> )
{
...
"form": {
"key2": "value2",
"key1": "value1"
},
...
}
很多时候你想要发送的数据并⾮编码为表单形式的。如果你传递⼀个string⽽不是⼀个dict，那么数据会被直接发布出去。
>>> r = requests.post(url, data=json.dumps(payload))
或者
>>> r = requests.post(url, json=payload)
传递⽂件
>>> r = requests.post(url, files=files)
配置files，filename, content_type and headers
files = {'file': ('report.xls', open('report.xls', 'rb'), 'application/vnd.ms-excel', {'Expires': '0'})}
files = {'file': ('report.csv', 'some,data,to,send\nanother,row,to,send\n')}
响应
r.status_code
r.heards
ocaml和haskell跳转
By default Requests will perform location redirection for all verbs except HEAD.
>>> r = ('/cookies/set?k2=v2&k1=v1')
>>> r.status_code
[<Response [302]>]
织梦多城市分站源码
If you're using HEAD, you can enable redirection as well:
r=requests.head('/cookies/set?k2=v2&k1=v1',allow_redirects=True)
You can tell Requests to stop waiting for a response after a given number of seconds with the timeout parameter:
<('github', timeout=0.001)
⾼级特性
session，⾃动保存cookies，可以设置请求参数，下次请求⾃动带上请求参数
s = requests.Session()
<('/cookies/set/sessioncookie/123456789')
r = s.get('/cookies')
)
# '{"cookies": {"sessioncookie": "123456789"}}'
session可以⽤来提供默认数据，函数参数级别的数据会和session级别的数据合并，如果key重复，函数参数级别的数据将覆盖session级别的数据。如果想取消session的某个参数，可以在传递⼀个相同key，value为None的dict
s = requests.Session()
s.auth = ('user', 'pass') #权限认证
s.headers.update({'x-test': 'true'})
# both 'x-test' and 'x-test2' are sent
<('/headers', headers={'x-test2': 'true'})
函数参数中的数据只会使⽤⼀次，并不会保存到session中
如：cookies仅本次有效
r = s.get('/cookies', cookies={'from-my': 'browser'})
session也可以⾃动关闭
with requests.Session() as s:
<('/cookies/set/sessioncookie/123456789')
响应结果不仅包含响应的全部信息，也包含请求信息
r = ('/wiki/Monty_Python')
r.headers
SSL证书验证
>>> ('kennethreitz', verify=True)
在该域名上我没有设置SSL，所以失败了。但Github设置了SSL:
>>> ('github', verify=True)
<Response [200]>
对于私有证书，你也可以传递⼀个CA_BUNDLE⽂件的路径给 verify 。你也可以设置REQUEST_CA_BUNDLE 环境变量。
>>> ('github', verify='/path/to/certfile')
如果你将 verify 设置为False，Requests也能忽略对SSL证书的验证。
>>> ('kennethreitz', verify=False)
<Response [200]>
默认情况下， verify 是设置为True的。选项 verify 仅应⽤于主机证书。
你也可以指定⼀个本地证书⽤作客户端证书，可以是单个⽂件（包含密钥和证书）或⼀个包含两个⽂件路径的元组:
>>> ('kennethreitz', cert=('/', '/path/key'))
<Response [200]>
响应体内容⼯作流
默认情况下，当你进⾏⽹络请求后，响应体会⽴即被下载。你可以通过 stream 参数覆盖这个⾏为，推迟下载响应体直到访问 t 属性:
此时仅有响应头被下载下来了，连接保持打开状态，因此允许我们根据条件获取内容:
if int(r.headers['content-length']) < TOO_LONG:
content = r.content
...
如果设置stream为True，请求连接不会被关闭，除⾮读取所有数据或者调⽤Response.close。
可以使⽤contextlib.closing来⾃动关闭连接：
import requests
from contextlib
linux配置当前用户环境变量import closing
file = r'D:\Documents\WorkSpace\Python\Test\Python34Test\'
with (tarball_url, stream=True)) as r:
with open(file, 'wb') as f:
for data in r.iter_content(1024):
f.write(data)
Keep-Alive
同⼀会话内你发出的任何请求都会⾃动复⽤恰当的连接！
注意：只有所有的响应体数据被读取完毕连接才会被释放为连接池；所以确保将 stream设置为 False 或读取 Response 对象
的 content 属性。
流式上传
Requests⽀持流式上传，这允许你发送⼤的数据流或⽂件⽽⽆需先把它们读⼊内存。要使⽤流式上传，仅需为你的请求体提供⼀个类⽂件对象即可:
读取⽂件请使⽤字节的⽅式，这样Requests会⽣成正确的Content-Length
with open('massive-body', 'rb') as f:
requests.post('some.url/streamed', data=f)
分块传输编码
对于出去和进来的请求，Requests也⽀持分块传输编码。要发送⼀个块编码的请求，仅需为你的请求体提供⼀个⽣成器
注意⽣成器输出应该为bytes
def gen():
yield b'hi'
yield b'there'
requests.post('some.url/chunked', data=gen())
For chunked encoded responses, it's best to iterate over the data using Response.iter_content(). In an ideal situation you'll have set stream=True on the request, in which case you can iterate chunk-by-chunk by calling iter_content with a chunk size parameter of None. If you want to set a maximum size of the chunk, you can set a chunk size parameter to any integer.
POST Multiple Multipart-Encoded Files
<input type="file" name="images" multiple="true" required="true"/>
To do that, just set files to a list of tuples of (form_field_name, file_info):
Custom Authentication
Requests allows you to use specify your own authentication mechanism.
Any callable which is passed as the auth argument to a request method will have the opportunity to modify the request before it is dispatched.
Authentication implementations are subclasses of requests.auth.AuthBase, and are easy to define. Requests provides two common authentication scheme implementations in requests.auth:HTTPBasicAuth and HTTPDigestAuth.
Let's pretend that we have a web service that will only respond if the X-Pizza header is set to a password value. Unlikely, but just go with it.
from requests.auth import AuthBase
class PizzaAuth(AuthBase):
"""Attaches HTTP Pizza Authentication to the given Request object."""
def __init__(self, username):
# setup any auth-related data here
self.username = username

版权声明：本站内容均来自互联网，仅供演示用，请勿用于商业和其他非法用途。如果侵犯了您的权益请与我们联系QQ:729038198，我们将在24小时内删除。