python的Tqdm模块的使用
|
Tqdm 是一个快速,可扩展的Python进度条,可以在 Python 长循环中添加一个进度提示信息,用户只需要封装任意的迭代器 tqdm(iterator)。 我的系统是window环境,首先安装python,接下来就是pip。 pip安装: 在python根目录下创建一个get-pip.py的文件,内容: https://bootstrap.pypa.io/get-pip.py 然后在CMD窗口进入python下面: 输出: python -m pip install -U pip 由于Tqdm要求的pip版本是9.0所以需要手动安装pip9.0 下载安装包9.0 然后解压进入,CMD窗口输入:python setup.py install 然后就可以安装Tqdm了, pip install tqdm 安装最新的开发版的话 pip install -e git+https://github.com/tqdm/tqdm.git@master#egg=tqdm 最后看看怎么用呢?https://pypi.python.org/pypi/tqdm 基本用法: from tqdm import tqdm for i in tqdm(range(10000)): sleep(0.01) 当然除了tqdm,还有trange,使用方式完全相同
for i in trange(100):
sleep(0.1)
只要传入list都可以:
pbar = tqdm(["a","b","c","d"])
for char in pbar:
pbar.set_description("Processing %s" % char)
也可以手动控制更新
with tqdm(total=100) as pbar:
for i in range(10):
pbar.update(10)
也可以这样: pbar = tqdm(total=100) for i in range(10): pbar.update(10) pbar.close() 在Shell的tqdm用法 统计所有python脚本的行数:
$ time find . -name '*.py' -exec cat {} ; | wc -l
857365
real 0m3.458s
user 0m0.274s
sys 0m3.325s
$ time find . -name '*.py' -exec cat {} ; | tqdm | wc -l
857366it [00:03,246471.31it/s]
857365
real 0m3.585s
user 0m0.862s
sys 0m3.358s
使用参数:
$ find . -name '*.py' -exec cat {} ; |
tqdm --unit loc --unit_scale --total 857366 >> /dev/null
100%|| 857K/857K [00:04<00:00,246Kloc/s]
备份一个目录: $ 7z a -bd -r backup.7z docs/ | grep Compressing | tqdm --total $(find docs/ -type f | wc -l) --unit files >> backup.log 100%|| 8014/8014 [01:37<00:00,82.29files/s] 通过看示范的代码,我们能发现使用的核心是tqdm和trange这两个函数,从代码层面分析tqdm的功能,那首先是init.py __all__ = ['tqdm','tqdm_gui','trange','tgrange','tqdm_pandas','tqdm_notebook','tnrange','main','TqdmKeyError','TqdmTypeError','__version__'] 跟踪到_tqdm.py,能看到tqdm类的声明,首先是初始化 def __init__(self,iterable=None,desc=None,total=None,leave=True,file=sys.stderr,ncols=None,mininterval=0.1,maxinterval=10.0,miniters=None,ascii=None,disable=False,unit='it',unit_scale=False,dynamic_ncols=False,smoothing=0.3,bar_format=None,initial=0,position=None,gui=False,**kwargs):
Parameters
iterable : iterable,optional
Iterable to decorate with a progressbar.
可迭代的进度条。
Leave blank to manually manage the updates.
留空手动管理更新??
desc : str,optional
Prefix for the progressbar.
进度条的描述
total : int,optional
The number of expected iterations. If unspecified,len(iterable) is used if possible. As a last resort,only basic
progress statistics are displayed (no ETA,no progressbar).
If gui is True and this parameter needs subsequent updating,specify an initial arbitrary large positive integer,e.g. int(9e9).
预期的迭代数目,默认为None,则尽可能的迭代下去,如果gui设置为True,这里则需要后续的更新,将需要指定为一个初始随意值较大的正整数,例如int(9e9)
leave : bool,optional
If [default: True],keeps all traces of the progressbar
upon termination of iteration.
保留进度条存在的痕迹,简单来说就是会把进度条的最终形态保留下来,默认为True
file : io.TextIOWrapper or io.StringIO,optional
Specifies where to output the progress messages
[default: sys.stderr]. Uses file.write(str) and file.flush()
methods.
指定消息的输出
ncols : int,optional
The width of the entire output message. If specified,dynamically resizes the progressbar to stay within this bound.
If unspecified,attempts to use environment width. The
fallback is a meter width of 10 and no limit for the counter and
statistics. If 0,will not print any meter (only stats).
整个输出消息的宽度。如果指定,动态调整的进度停留在这个边界。如果未指定,尝试使用环境的宽度。如果为0,将不打印任何东西(只统计)。
mininterval : float,optional
Minimum progress update interval,in seconds [default: 0.1].
最小进度更新间隔,以秒为单位(默认值:0.1)。
maxinterval : float,optional
Maximum progress update interval,in seconds [default: 10.0].
最大进度更新间隔,以秒为单位(默认值:10)。
miniters : int,in iterations.
If specified,will set mininterval to 0.
最小进度更新周期
ascii : bool,optional
If unspecified or False,use unicode (smooth blocks) to fill
the meter. The fallback is to use ASCII characters 1-9 #.
如果不设置,默认为unicode编码
disable : bool,optional
Whether to disable the entire progressbar wrapper
[default: False].
是否禁用整个进度条包装(如果为True,进度条不显示)
unit : str,optional
String that will be used to define the unit of each iteration
[default: it].
将被用来定义每个单元的字符串???
unit_scale : bool,optional
If set,the number of iterations will be reduced/scaled
automatically and a metric prefix following the
International System of Units standard will be added
(kilo,mega,etc.) [default: False].
如果设置,迭代的次数会自动按照十、百、千来添加前缀,默认为false
dynamic_ncols : bool,constantly alters ncols to the environment (allowing
for window resizes) [default: False].
不断改变ncols环境,允许调整窗口大小
smoothing : float,optional
Exponential moving average smoothing factor for speed estimates
(ignored in GUI mode). Ranges from 0 (average speed) to 1
(current/instantaneous speed) [default: 0.3].
bar_format : str,optional
Specify a custom bar string formatting. May impact performance.
If unspecified,will use ‘{l_bar}{bar}{r_bar}',where l_bar is
‘{desc}{percentage:3.0f}%|' and r_bar is
‘| {n_fmt}/{total_fmt} [{elapsed_str}<{remaining_str},{rate_fmt}]'
Possible vars: bar,n,n_fmt,total,total_fmt,percentage,rate,rate_fmt,elapsed,remaining,l_bar,r_bar,desc.
自定义栏字符串格式化…默认会使用{l_bar}{bar}{r_bar}的格式,格式同上
initial : int,optional
The initial counter value. Useful when restarting a progress
bar [default: 0].
初始计数器值,默认为0
position : int,optional
Specify the line offset to print this bar (starting from 0)
Automatic if unspecified.
Useful to manage multiple bars at once (eg,from threads).
指定偏移,这个功能在多个条中有用
gui : bool,optional
WARNING: internal parameter - do not use.
Use tqdm_gui(…) instead. If set,will attempt to use
matplotlib animations for a graphical output [default: False].
内部参数…
Returns
out : decorated iterator.
返回为一个迭代器
其实不用分析更多代码,多看看几个例子:(官网的例子) 7zx.py压缩进度条
# -*- coding: utf-8 -*-
"""Usage:
7zx.py [--help | options] <zipfiles>...
Options:
-h,--help Print this help and exit
-v,--version Print version and exit
-c,--compressed Use compressed (instead of uncompressed) file sizes
-s,--silent Do not print one row per zip file
-y,--yes Assume yes to all queries (for extraction)
-D=<level>,--debug=<level>
Print various types of debugging information. Choices:
CRITICAL|FATAL
ERROR
WARN(ING)
[default: INFO]
DEBUG
NOTSET
-d,--debug-trace Print lots of debugging information (-D NOTSET)
"""
from __future__ import print_function
from docopt import docopt
import logging as log
import subprocess
import re
from tqdm import tqdm
import pty
import os
import io
__author__ = "Casper da Costa-Luis <casper.dcl@physics.org>"
__licence__ = "MPLv2.0"
__version__ = "0.2.0"
__license__ = __licence__
RE_SCN = re.compile("([0-9]+)s+([0-9]+)s+(.*)$",flags=re.M)
def main():
args = docopt(__doc__,version=__version__)
if args.pop('--debug-trace',False):
args['--debug'] = "NOTSET"
log.basicConfig(level=getattr(log,args['--debug'],log.INFO),format='%(levelname)s: %(message)s')
log.debug(args)
# Get compressed sizes
zips = {}
for fn in args['<zipfiles>']:
info = subprocess.check_output(["7z","l",fn]).strip()
finfo = RE_SCN.findall(info)
# builtin test: last line should be total sizes
log.debug(finfo)
totals = map(int,finfo[-1][:2])
# log.debug(totals)
for s in range(2):
assert(sum(map(int,(inf[s] for inf in finfo[:-1]))) == totals[s])
fcomp = dict((n,int(c if args['--compressed'] else u))
for (u,c,n) in finfo[:-1])
# log.debug(fcomp)
# zips : {'zipname' : {'filename' : int(size)}}
zips[fn] = fcomp
# Extract
cmd7zx = ["7z","x","-bd"]
if args['--yes']:
cmd7zx += ["-y"]
log.info("Extracting from {:d} file(s)".format(len(zips)))
with tqdm(total=sum(sum(fcomp.values()) for fcomp in zips.values()),unit="B",unit_scale=True) as tall:
for fn,fcomp in zips.items():
md,sd = pty.openpty()
ex = subprocess.Popen(cmd7zx + [fn],bufsize=1,stdout=md,# subprocess.PIPE,stderr=subprocess.STDOUT)
os.close(sd)
with io.open(md,mode="rU",buffering=1) as m:
with tqdm(total=sum(fcomp.values()),disable=len(zips) < 2,leave=False,unit_scale=True) as t:
while True:
try:
l_raw = m.readline()
except IOError:
break
l = l_raw.strip()
if l.startswith("Extracting"):
exname = l.lstrip("Extracting").lstrip()
s = fcomp.get(exname,0) # 0 is likely folders
t.update(s)
tall.update(s)
elif l:
if not any(l.startswith(i) for i in
("7-Zip ","p7zip Version ","Everything is Ok","Folders: ","Files: ","Size: ","Compressed: ")):
if l.startswith("Processing archive: "):
if not args['--silent']:
t.write(t.format_interval(
t.start_t - tall.start_t) + ' ' +
l.lstrip("Processing archive: "))
else:
t.write(l)
ex.wait()
main.__doc__ = __doc__
if __name__ == "__main__":
main()
(编辑:安卓应用网) 【声明】本站内容均来自网络,其相关言论仅代表作者个人观点,不代表本站立场。若无意侵犯到您的权利,请及时与联系站长删除相关内容! |
