pk

pickle IO utility module.

useful method:

• load()
• dump()
• safe_dump()
• obj2bytes()
• bytes2obj()
• obj2str()
• str2obj()

dataIO.pk.is_pickle_file(abspath)

Parse file extension.

• *.pickle: uncompressed, utf-8 encode pickle file
• *.gz: compressed, utf-8 encode pickle file

dataIO.pk.lower_ext(abspath)

Convert file extension to lowercase.

dataIO.pk.load(abspath, default={}, enable_verbose=True)

Load Pickle from file. If file are not exists, returns default.

Parameters:

• abspath (string) – file path. use absolute path as much as you can. extension has to be .pickle or .gz (for compressed Pickle).
• default – default dict(), if abspath not exists, return the default Python object instead.
• enable_verbose (boolean) – default True, help-message-display trigger.

Usage:

>>> from dataIO import pk
>>> pk.load("test.pickle") # if you have a pickle file
Load from `test.pickle` ...
    Complete! Elapse 0.000432 sec.
{'a': 1, 'b': 2}

中文文档

从Pickle文件中读取数据

Parameters:

• abspath (字符串) – Pickle文件绝对路径, 扩展名需为 .pickle 或 .gz, 其中 .gz 是被压缩后的Pickle文件
• default – 默认 dict(), 如果文件路径不存在, 则会返回指定的默认值
• enable_verbose (布尔值) – 默认 True, 信息提示的开关, 批处理时建议关闭

dataIO.pk.dump(data, abspath, pk_protocol=3, overwrite=False, enable_verbose=True)

Dump picklable object to file. Provides multiple choice to customize the behavior.

Parameters:

• data (dict or list) – picklable python object.
• abspath (string) – save as path, file extension has to be .pickle or .gz (for compressed Pickle)
• pk_protocol (int) – default = your python version, use 2, to make a py2.x/3.x compatible pickle file. But 3 is faster.
• overwrite (boolean) – default False, If True, when you dump to existing file, it silently overwrite it. If False, an alert message is shown. Default setting False is to prevent overwrite file by mistake.
• enable_verbose (boolean) – default True, help-message-display trigger.

Usage:

>>> from dataIO import pk
>>> data = {"a": 1, "b": 2}
>>> dump(data, "test.pickle", overwrite=True)
Dump to `test.pickle` ...
    Complete! Elapse 0.002432 sec

中文文档

将Python中可被序列化的”字典”, “列表”以及他们的组合, 按照Json的编码方式写入文件 文件

参数列表

Parameters:

• data (字典 或 列表) – 可Pickle化的Python对象
• abspath (字符串) – Pickle文件绝对路径, 扩展名需为 .pickle 或 .gz, 其中 .gz 是被压缩后的Pickle文件
• pk_protocol (int) – 默认值为你的Python大版本号, 使用2可以使得Python2/3都能 兼容你的Pickle文件。不过Python3的速度更快。
• overwrite (“布尔值”) – 默认 False, 当为``True``时, 如果写入路径已经存在, 则会 自动覆盖原文件。而为``False``时, 则会打印警告文件, 防止误操作覆盖源文件。
• enable_verbose (布尔值) – 默认 True, 信息提示的开关, 批处理时建议关闭

dataIO.pk.safe_dump(data, abspath, pk_protocol=3, enable_verbose=True)

A stable version of dump(), this method will silently overwrite existing file.

There’s a issue with dump(): If your program is interrupted while writing, you got an incomplete file, and you also lose the original file. So this method write pickle to a temporary file first, then rename to what you expect, and silently overwrite old one. This way can guarantee atomic write.

中文文档

在对文件进行写入时, 如果程序中断, 则会留下一个不完整的文件。如果使用了覆盖式 写入, 则我们即没有得到新文件, 同时也丢失了原文件。所以为了保证写操作的原子性 (要么全部完成, 要么全部都不完成), 更好的方法是: 首先将文件写入一个临时文件中, 完成后再讲文件重命名, 覆盖旧文件。这样即使中途程序被中断, 也仅仅是留下了一个 未完成的临时文件而已, 不会影响原文件。

dataIO.pk.obj2bytes(obj, pk_protocol=3)

Convert arbitrary pickable Python Object to bytes.

中文文档

将可Pickle化的Python对象转化为bytestr

dataIO.pk.bytes2obj(b)

Load Python object from bytes.

中文文档

从bytestr中恢复Python对象

dataIO.pk.obj2str(obj, pk_protocol=3)

Convert arbitrary object to base64 encoded string.

中文文档

将可Pickle化的Python对象转化为utf-8编码的 纯ASCII字符串

dataIO.pk.str2obj(s)

Load object from base64 encoded string.

中文文档

从base64编码的 纯ASCII字符串 中恢复Python对象