Linux pv command
On Unix-like operating systems, the pv command displays the progress of data flowing through a pipeline.
pv was written by sysadmin and open source developer Andrew Wood.
What is pv?
The pv ("pipe viewer") program watches data flow through a pipeline, and displays information about its progress. The information displayed by pv includes:
- Time elapsed since data flow began.
- Percentage complete, with progress bar.
- Current data throughput rate.
- Total data transferred.
- ETA (estimated time of arrival) of when the operation is expected to be complete.
What is a pipeline?
Using the command line, you can "pipe" the output of one command to the input of another command using the vertical bar ("|"), or pipe, symbol. For example, if you want to read the contents of a text file and send it to standard output (the terminal screen), you can use the cat command:
cat myfile.txt
The ABC meeting is on Monday, October 4. The 123 conference is on Friday, August 12. The DEF meeting is on Tuesday, November 8. The GHI meeting is on Monday, December 12. ...
You can use that output as input to another program. Maybe you want to extract only the lines that have the word "meeting" included. You can pipe the output to grep to filter it:
cat myfile.txt | grep meeting
The ABC meeting is on Monday, October 4. The DEF meeting is on Tuesday, November 8. The GHI meeting is on Monday, December 12.
This new output can be piped to sed to modify the output. For example, the following command extracts lines about meetings, then extracts only the lines about meetings happening on a Monday. It then extracts dates of those meetings using a Perl regular expression.
cat myfile.txt | grep meeting | grep Monday | \ perl -p -e 's/^.*(?:Monday, )(.*\d+).*$/$1/'
October 4 December 12
The example above shows how the output of one command can be piped to other commands in a series, called a pipeline.
In the pipeline example above, the results are shown almost instantly, so you would not need to monitor the progress. However, other commands might take a long time, and it would be helpful to know when they might finish. That is what pv is designed to do.
Syntax
The following is the general syntax of the pv command. Note that if a parameter is included with a long option name, such as --last-written NUM, the same parameter should be specified with the corresponding short option, such as -A NUM.
pv [-p | --progress] [-t | --timer] [-e | --eta] [-I | --fineta] [-r | --rate] [-a | --average-rate] [-b | --bytes] [-T | --buffer-percent] [-A | --last-written NUM] [-F | --format FORMAT] [-n | --numeric] [-W | --wait] [-D | --delay-start SEC] [-s | --size SIZE] [-l | --line-mode] [-0 | --null] [-i | --interval SEC] [-w | --width WIDTH] [-H | --height HEIGHT] [-N | --name NAME] [-f | --force] [-c | --cursor] [-L | --rate-limit RATE] [-B | --buffer-size BYTES] [-C | --no-splice] [-E | --skip-errors] [-S | --stop-at-size] [-d | --watchfd PID[:FD] [-R | --remote PID] [-P | --pidfile FILE] [file ...]
pv [ -h | -V ]
Options
Display switches
The following options control how information is output by pv.
If no display switches are specified, pv behaves as if -p, -t, -e, -r, and -b were specified (everything except average rate is switched on, by default).
| -p, --progress | Display the progress bar. If standard input is not a file, and no size is given (with the -s modifier), the progress bar cannot indicate how close to completion the transfer is, so it moves left and right to indicate that data is moving. |
| -t, --timer | Turn on the timer, displaying the total elapsed time pv has run. |
| -e, --eta | Turn on the ETA timer. This attempts to guess, based on previous transfer rates and the total data size, how long before completion. This option has no effect if the total data size cannot be determined. (If you want an ETA when piping from standard input, specify the size with -s.) |
| -I, --fineta | Turn on the ETA timer, but display the estimated local time of arrival instead of time left. When the estimated time is more than 6 hours in the future, the date is also shown. |
| -r, --rate | Turn on the rate counter, displaying the current throughput rate. |
| -a, --average-rate | Turn on the average rate counter, displaying the current rate of data transfer. |
| -b, --bytes | Turn on the total byte counter, displaying the total amount of data transferred so far. |
| -T, --buffer-percent |
Turn on the transfer buffer percentage display, showing the percentage of the transfer buffer in use. For more information, see %T in the Formatting section below. |
| -A, --last-written NUM |
Show the last NUM bytes written. For more information, see %nA in the Formatting section below. |
| -F, --format FORMAT |
Ignore the options -p, -t, -e, -r, -a, -b, -T, and -A, and instead use the format string FORMAT to determine the output format. For more information, see the Formatting section below. |
| -n, --numeric | Numeric output. Instead of giving a visual indication of progress, pv gives an integer percentage, one per line on standard error, suitable for piping. NOTE: If combined with --bytes, this option causes the number of bytes processed to be shown, instead of a percentage. If combined with --line-mode, then instead of bytes of a percentage, the number of lines so far is shown. If combined with --timer, then each output line is prefixed with the elapsed time in seconds. |
| -q, --quiet | Show no output. This option can be useful if you are using -L and all you want to do is limit the transfer rate of the pipe. |
Output modifiers
The following options affect what information is output by pv.
| -W, --wait | Wait until the first byte is transferred before showing any progress information or calculating any ETAs. Useful if the program you are piping to or from requires extra information before it starts, such as piping data to gpg which requires a passphrase before data is processed. |
| -D, --delay-start SEC |
Wait until SEC seconds have passed before showing any progress information; for example, in a script showing a progress bar if it starts taking a long time. SEC can be a decimal, such as 0.5. |
| -s, --size SIZE | Assume the total amount of data to be transferred is SIZE bytes when calculating percentages and ETAs. The suffixes k (kilobytes), m (megabytes), etc., can be used (as with -L). SIZE must be a whole number. NOTE: This option has no effect if used with -d PID to watch all file descriptors of a process, but works with -d PID:FD (when watching a single descriptor). |
| -l, --line-mode | Instead of counting bytes, count lines (newline characters). The progress bar only moves when a newline is found, and the value passed to the -s option is interpreted as a line count. Note that file sizes are not automatically calculated when this option is used, to avoid having to read all files twice. |
| -0, --null | Count lines as null-terminated. This option implies --line-mode. |
| -i, --interval SEC |
Wait SEC seconds between updates. (The default is to update every second.) SEC can be a decimal, such as 0.1. |
| -w, --width WIDTH |
Assume the terminal is WIDTH characters wide, instead of trying to detect it. (80 is assumed if the value cannot be guessed.) |
| -H, --height HEIGHT |
Assume the terminal is HEIGHT rows high, instead of trying to detect it (25 is assumed if the height cannot be guessed.) |
| -N, --name NAME | Prefix the output information with NAME. Useful in conjunction with -c if you have a complicated pipeline and you want to measure different parts. |
| -f, --force | Force output. Normally, pv outputs nothing if standard output is not a terminal. This option forces it to do so. |
| -c, --cursor | Use cursor positioning escape sequences instead of using carriage returns. Useful in conjunction with -N if you use pv multiple times in a single pipeline. |
Data transfer modifiers
The following options affect how data flows through the pipe being watched.
| -L, --rate-limit RATE |
Limit the transfer to a maximum of RATE bytes per second. A suffix of k, m, g, or t can be added to denote kilobytes, megabytes, gigabytes, or terabytes. A suffix of K, M, G, or T can be added to denote kibibytes, mebibytes, gibibytes, or tebibytes. RATE must be a whole number. |
| -B, --buffer-size BYTES |
Use a transfer buffer size of BYTES bytes. A suffix of k, K, m, M, etc., can be used as with -L. The default buffer size is the block size of the input file's filesystem multiplied by 32 (512 KiB max), or 400 KiB if the block size cannot be determined. |
| -C, --no-splice | Never use the splice system call, even if it would be possible. The splice call is a more efficient way of transferring data from or to a pipe than regular read and write, but the transfer buffer may not be used. This prevents -A and -T from working, so if you want to use those options, you need to use -C, at the cost of a small loss in transfer efficiency. This option has no effect on systems where the splice system call is not available. |
| -E, --skip-errors | Ignore read errors by attempting to skip past the offending sections. The corresponding parts of the output are null bytes. At first, bytes are skipped individually, but if there are many errors in a row, the skips move up to chunks of 512. NOTE: This option is intended to be similar to the dd command with options conv=sync,noerror, but its behavior is not guaranteed. |
| -S, --stop-at-size | If a size was specified with -s, stop transferring data once that many bytes are written, rather than continuing to the end of input. |
| -d, --watchfd PID[:FD] |
Instead of transferring data, watch file descriptor FD of process PID, and show its progress. The pv process exits when FD changes to a different file, changes read/write mode, or is closed. Other data transfer modifiers, including -R (remote control), may not be used with this option. If only a PID is specified, then the process with that PID is watched, and all regular files and block devices it opens are shown with a progress bar. The pv process exits when the process PID exits. |
| -R, --remote PID | "Remote control" option. If PID is an instance of pv that is already running, -R PID causes that instance to act as though it was given this instance's command line instead. For example, if pv -L 123K is running with process ID 9876, then running pv -R 9876 -L 321K causes it to start using a rate limit of 321 KiB instead of 123 KiB. NOTE: Some options cannot be changed while running, such as -c, -l, -f, -D, -E, and -S. |
General Options
| -P, --pidfile FILE | Save the process ID of pv in FILE. The file is overwritten if it already exists, and is removed when pv exits. While pv is running, the file contains the process ID number of pv followed by a newline. |
| -h, --help | Print a usage message on standard output and exit successfully. |
| -V, --version | Print version information on standard output and exit successfully. |
Formatting
If the -F FORMAT option is given, then the output format is determined by the FORMAT string. In that string, the following sequences can be used.
| %p | Progress bar. Expands to fill the remaining space. Should only be specified once. Equivalent to -p. |
| %t | Elapsed time. Equivalent to -t. |
| %e | ETA as time remaining. Equivalent to -e. |
| %I | ETA as local time of completion. Equivalent to -I. |
| %r | Current data transfer rate. Equivalent to -r. |
| %a | Average data transfer rate. Equivalent to -a. |
| %b | Bytes transferred so far (or lines, if -l is specified). Equivalent to -b. |
| %T | Percentage of the transfer buffer in use. Equivalent to -T. Shows "{----}" if the transfer is being done with splice, as splicing to or from pipes does not use the buffer. |
| %nA | Show the last n bytes written (e.g., %16A for the last 16 bytes). Shows only dots if the transfer is being done with splice, as splicing to or from pipes does not use the buffer. |
| %N | Name prefix given by -N, padded to nine characters with spaces, and suffixed with :. |
| %% | A literal % character. |
The format string equivalent of turning on all display switches is:
"%N %b %T %t %r %a %p %e".
Common switches
The following are suggested common switch combinations.
| pv -ptebar | Show a progress bar, elapsed time, ETA, byte count, average rate, and current rate. |
| pv -betlap | Show a progress bar, elapsed time, ETA, line count, and average rate, counting lines instead of bytes. |
| pv -t | Show only the elapsed time. Useful as a simple timer, e.g., sleep 10m | pv -t. |
| pt -pterb | The default behavior: progress bar, elapsed time, ETA, current rate, and byte count. |
Exit status
An exit status of 0 indicates successful execution with no problems.
An exit status of 1 indicates a problem with the -R or -P options. Any other exit status is a bitmask of the following:
| 2 | One or more files could not be accessed, stat-ed, or opened. |
| 4 | An input file was the same as the output file. |
| 8 | Internal error with closing a file or moving to the next file. |
| 16 | There was an error while transferring data from one or more input files. |
| 32 | A signal was caught that caused an early exit. |
| 64 | Memory allocation failed. |
For example, error status 26 would be a combination of error statuses 2, 8, and 16:
2 00000010 0x2 | 8 00001000 0x8 | 16 00010000 0x10 = 26 00011010 0x1a
Examples
The following examples show some ways that pv can be used in a pipeline.
Use pv like cat
pv myfile
Lorem Ipsum 12.0 B 0:00:00 [ 217KiB/s] [=========================================>] 100%
The command above reads the contents of myfile, writes them to standard output, and displays the progress.
When you provide pv with a file name, it works similar to cat, but it also displays a progress bar, byte count, throughput, etc.
Pipe to pv
cat myfile | pv
Lorem Ipsum 12.0 B 0:00:00 [ 316KiB/s] [ <=> ]
In the example above, cat reads the contents of the file and sends them to pv. This time, pv doesn't show percent complete, because it's reading from standard input and a size was not specified with -s. Because it doesn't know the size of the input, it can only display the bytes transferred, time elapsed, and throughput. The progress meter only moves back and forth to indicate that data is currently being transferred.
The file contents are then sent to standard output by pv.
Test pipeline "top speed"
pv < /dev/zero > /dev/null
The command above reads from the virtual device /dev/zero that streams null characters (ASCII 0), monitors throughput, and sends output to the /dev/null virtual device. (NOTE: /dev/null points to nothing, resulting in the output being discarded.) This is a simple way to test the "top speed" of data flowing through a pipeline in your shell.
Monitor the progress of dd
The pv command can monitor the progress of dd.
For example, let's say you're using the dd command to create a Debian Linux installation disk. You've already downloaded the Debian ISO image from the official mirror site, and you want to write it to an empty USB flash drive. You have used lsblk to verify that the USB flash drive is inserted on device /dev/sda, and you have used umount or findmnt to verify that none of its partitions are currently mounted.
Be very careful when using the dd command. If you specify the wrong output device, you could overwrite data on the wrong device, including your system disk. Do not copy and run the following commands, unless you're sure you understand what the command is doing.
You can run the following command as the superuser to write the ISO file to the USB flash drive:
# dd if=debian-11.2.0-amd64-DVD-1.iso of=/dev/sda bs=4M
This operation does not display any output. When you see the shell prompt again, you know it's complete.
In the example above, dd is run only once, handling the input (if=) and output (of=) in a single command. Alternatively, you can invoke dd twice, once for the input and once for the output, and insert pv between the two invocations, to monitor the data flow.
# dd if=debian-11.2.0-amd64-DVD-1.iso | pv | dd of=/dev/sda bs=4M
In the pipeline above, the data flows like the following:
- The first dd reads the ISO file and send the data to standard output, which is piped to pv.
- pv monitors that data flow, displays information about it, and sends the data to the second instance of dd.
- The second dd command reads that data as input, and writes the data to the destination device.
The benefit of this method is that the pv progress meter tells you how long until the operation is complete.
The version of dd included with GNU Coreutils version 8.24 and above supports the dd option status=progress. When specified, this option displays the throughput rate, time elapsed, and amount of data copied. However, the method above using pv works with any version of dd, and includes an ETA if the data size is specified with -s.
Related commands
ps — Report the status of a process or processes.
top — Display a sortable, continually-updated list of processes.