Back to Linux page of KG
Data recovery and data protection tool
Like dd, dd_rescue does copy data from one file or block
device to another. You can specify file positions (called seek and
Skip in dd).
There are several differences:
- dd_rescue does not provide character conversions.
- The command syntax is different. Call dd_rescue -h.
- dd_rescue does not abort on errors on the input file, unless you
specify a maximum error number. Then dd_rescue will abort when
this number is reached.
- dd_rescue does not truncate the output file, unless asked to.
- You can tell dd_rescue to start from the end of a file and move backwards.
- It uses two block sizes, a large (soft) block size and a small (hard) block
size. In case of errors, the size falls back to the small one and is
promoted again after a while without errors.
Purpose of dd_rescue
The latter three features make it suitable for rescuing data from a medium
with errors, i.e. a hard disk with some bad sectors.
More info can be found in the README file
included in the package.
- Imagine, one of your partitions is crashed, and as there are some hard
errors, you don't want to write to this hard disk any more. Just getting
all the data off it and retiring it seems to be suitable. However, you
can't access the files, as the file system is damaged.
- Now, you want to copy the whole partition into a file. You burn it on
CD-Rom, just to never lose it again.
You can setup a loop device, and repair (fsck) it and hopefully are able
to mount it.
- Copying this partition with normal Un*x tools like cat or dd will fail, as
those tools abort on error. dd_rescue instead will try to read and if it
fails, it will go on with the next sectors. The output file naturally will
have holes in it, of course. You can write a log file, to see, where all
these errors are located.
- The data rate drops very low, when errors are encountered. If you
interrupt the process of copying, you don't lose anything. You can just
continue at any position later. The output file will just be filled in
further and not truncated as with other Un*x tools.
- If you have one spot of bad sectors within the partition, it might be a
good idea, to approach this spot from both sides. Reverse direction copy
is your friend.
- The two block sizes are a performance optimization. Large block sizes
result in superior performance, but in case of errors, you want to try
to salvage every single sector. So hardbs is best be set to the hardware
sector size (most often 512 bytes) and softbs to a large value, such as
the default 16k.
- Meanwhile, the tool
has been developed and supercedes some of dd_rescue's functionality for what
it was originally designed for.
It's still useful though for some special features, such as direct IO,
sparse copies, splice copies, preallocation ...
Noteworthy are the data protection functions that have been introduced
since 1.29 to allow safe deletion of files and storage devices with a
fast userspace pseudo random number generator.
Just one note: It does work. I unfortunately did not just create this
program for fun ...
Latest version is 1.42.1 (2014-03-06)
1.42.1 contains a fix for a
sublety how we set up a handler for SIGILL and return with longjmp to
detect the supported instruction sets of the CPU -- we need to manually
reset the process' signal mask, otherwise a second failed probe would
Previous version is 1.42 (2014-03-03)
1.42 brings the possibility to load plugins to analyze or transform
data before it's written to the output file(s). A plugin to calculate
the MD5 hash is provided. posix_fadvise() is used if available (optimization)
and dd_rescue now only provides a short usage info rather than the long
help text on wrong parameters.
Previous version was 1.41 (2014-02-25)
There has been a lot of internal refactoring that improves the detection
of CPU features (at runtime) and libc/compiler features (at build time).
One result is that this version supports building against the Android NDK.
(armv7l binaries built against Android API 17 (aka 4.2) libc can be found
below in the download section.)
Another consequence is that AVX2 support is now enabled (for saving CPU
cycles on sparse block detection).
A few minor bugs have been addressed (the most serious one a harmless
off-by-one on determining the size of a block device).
Number formatting is more consistent now.
There also a new option -u/--rmvtrim that deletes the created file
again and issues a fstrim on the filesystem -- good if you filled the
empty space of a filesystem with zeros for data protection and SSD
Previous version was 1.40.1 (2014-02-15)
It just has one patch to fix the
SSE2 detection on i386 -- the old code would end in an endless loop ...
Previous version was 1.40 (2013-08-18)
It brings copying of extended attributes (with -p/--preserve). It doubles the
default soft block size for buffered IO, but brings sparse write optimization
for half-empty blocks. It also optimizes copying by using the first write
to get rid off odd file offsets. It also adds a lot more test cases to make check.
Previous version was 1.39 (2013-08-08)
It fixes an issue where a copied file could be appended zeros if hardblocksize copy
was used (e.g. b/c hardbs==softbs, bnc #833765). There's also a bit better ARM
asm optimization, yielding a ~15% performance increase.
There's also a help/manpage clarification that syncfreq actually is a size.
And we use autoconf now to determine the target system features. Default build
target now uses libdl.
Previous version was 1.38 (2013-08-02); improving SSE sparse detection performance
(by 40%), adding a testcase for the 1.35/1.36 bug and run it in make check.
There's even an AVX version, but it's not enabled by default, as it's untested.
--force/-f now allows to ignore a non-zero output position on non-seekable
output and the curr.rate and ETA calculations have improved a bit.
Version 1.37 (2013-08-01) was fixing an issue with SSE2 sparse detection,
which could spuriously detect zero-filled blocks and thus result in corrupted
copies if option -a was used. (This would happen for blocks that had no bytes with
the uppermost bit set, such as e.g. ASCII text.) Embarassing!
Also fixed issues on big-endian machines (although these were inconsequential for dd_rescue).
Older version history ...
The previous version is 1.36, released 2013-07-24.
It fixes an overflow issue with the number output for long running dd_rescue processes.
SSE2 is now also enabled in x86 (32bit, with runtime detection) and an optimized ARM
version (assembler yeah!) to find zero blocks was added.
The previous version was 1.35, released 2013-07-17.
It had some improvements on the output that it prints -- beyond internal improvements
it introduces colours to the output unless the terminal type is clearly dumb; there is
also an option to control this. Numbers are highlighted for readability. Output is rate limited (10/s).
1.35 also brings a simple rewrite logic for handling write errors. There's an
SSE2 optimized version to find zero blocks for sparse writing.
The previous version was 1.34, released 2013-07-04,
It brought better support for various *nix systems (specifically had a few fixes for FreeBSD), better compatibility with compilers (clang and g++ and clang++).
It can now also load libfallocate at runtime (libdl) and detects a few more fatl write errors as such.
Previous version was 1.33 (2013-03-30)
and brought long options, a new double overwrite mode (-2) and a man page.
Version 1.32 has a new option -x to append to the output file and you can specify -Y (multiple
times if you wish so) to write the same data to secondary output files.
Version 1.31 (2013-02-03)
brought a few tiny improvements in the output (such as displaying the
total elapsed time in the summary as opposed to ETA of 0, and the amount of data
really written with option -W). But importantly, it has the new mode of triple
overwriting of data (options -3 and -4), with random numbers, inverse random numbers,
new random numbers (only for -4) and zeros, this way allowing paranoia-safe
deletion of information.
Version 1.30 (2013-01-25)
brought a fix for outputting data to stdout and a fix for a possible double
free operation (introduced in 1.29). The message formatting has been streamlined a bit.
The PRNG can now be initialized from a file (e.g. -Z /dev/urandom). The program now
can also avoid writing to a target block if the target block already has the same
data (option -W). Think of SSDs or other devices where you want to avoid writes.
In Version 1.29 (2013-01-22) a bug was fixed, where the last bytes where not copied
corrected if hardbs == softbs.
1.29 also brings a number of new features; the ability to write the same (softbs sized)
block again and again (option -R, automatically set if infile is /dev/zero), the
ability to limit transfer size such that the outfile won't be enlarged (-M) and
the possibility to use userspace random numbers (libc/frandom) to fill files
with random data (options -z and -Z). Last not least, OBS also builds .deb binaries
for Ubu12.04 / Deb6 now.
Version is 1.28 (released 2012-05-19) uses better defaults for hard and softblock
sizes (4k/64k for buffered I/O, 512/1M for direct IO), as suggested by Jan Kara.
Also the copying of access times with the option -p was fixed.
Version 1.27 allowed to do 512b direct IO (which is possible in latest Linux kernels)
-- idea and patch from Jan Kara. Change posix_memalign() variable assignment.
It has a number of fixes from Valentin Lab; most importantly, when exiting
because of an error, it updates the variables that are output. dd_rescue
now avoid specal characters in the logfile. It handles situations gracefully,
where wrong positions resulted in the progress graph causing faults. Some
come from illegal input (negative offset ...), which is nw detected.
Version 1.25 contains a fix for spurious "Success" messages that resulted
from overwritten (cleared) errno. Bad blocks are formatted in a way that
they are not overwritten on screen and block numbers are output as unsigned.
Version 1.24 contains a compile fix for Linux versions that contain the splice syscall
but not the other definitions. I also allows for specifying a directory (such as ".")
as output filename in which case dd_rescue just appends the input file basename to
it, just like cp does. Maybe most importantly, the RPM now contains the latest
version of dd_rhelp (0.1.2).
In version 1.23, the default to not sync every 512 soft blocks is now really
changed to only sync at the end (documented for 1.18, but mostly ineffective,
ouch). More importantly, a short read because of EOF is now not considered
an error any more and thus neither triggers messages (with quiet) nor does
it cause a confusing exit code any more.
Version 1.22 contains a number of little improvements: Display of compiler,
compile time, enabled options, don't repeat copy attempts if hardbs == softbs,
fix FPE in progress bar for non-seekable input reverse copy, and make the code
more digestable for non-GCC compilers, optional fallbacks for strsignal,
pread, pwrite. Last not least, the documentation has a few notes on the
Version 1.21 issues warnings when writing in a sparse mode into existing files
or to block devices.
In version 1.20, there's also a progress estimation if input file size is unknown,
but transfer size is limited with -m.
Version 1.19 brings the support for fallocate.
In version 1.18, we have changed the default to -y0 (no fsync), the avg.
speed is calculated in a meaningful way nevertheless now.
In version 1.17, a progress bar, completion percentage and ETA has been added.
In version 1.16, the work was done to determine and display the total amount of data that will be copied.
Version 1.15 brings support for the Linux splice() mechanism to avoid copying data to a userspace bugffer.
In version 1.14, a bug is fixed that could lead to a target file not
having the correct size when copied in sparse (-a) mode.
In version 1.13, a bug with read-errors and non-seekable output has been
In version 1.12, the ability to log bad blocks to a ASCII file has been
added. The option -y can be used to set the fsync frequency which
can speed up the copy significantly.
Since 1.10, it has support for non-seekable input and output (so you can write
to pipes, e.g. stdout). Since 1.04, dd_rescue uses a larger default soft block
size (64k instead of 16k), has support for O_DIRECT, for copying the permissions,
and some minor issues fixed.
RPMs since 1.04 also include dd_rhelp.
Since version 1.03, it includes a patch from LAB Valentin which is
needed to properly support his dd_rhelp script
from his dd_rescue page.
See the RPM change log for a more detailed list
Please use the RPM and DEB repository public key to validate the
RPMs/DEBs. Import it if you trust the packages from my OBS repositories.
You can download
Installation and compilation follows the standard ways. If you use Linux,
using rpm --recompile on the source RPM may be your best option.
- Version 1.42.1
- Version 1.42
- Version 1.41
- Version 1.40.1
- Version 1.40
- Version 1.39
- Version 1.38
- Version 1.37
- Version 1.36
- Version 1.35
- Version 1.34
- Version 1.33
- Version 1.32
- Version 1.31
- Version 1.30
- Version 1.29
- Version 1.28
- Version 1.27
- Version 1.25
- Version 1.24
- Version 1.23
- Version 1.22
- Version 1.21
- Version 1.20
- Version 1.18
- Version 1.15
- Version 1.14
- Version 1.13
- Version 1.12
- Version 1.11
- Older versions
The program is protected by the GNU
GPL (v2 or v3).
(w) by KG, last changed 2013-08-01