You are on page 1of 5

The following changes have been made between John 1.7.8 and 1.7.

9: * Added optional parallelization of the MD5-based crypt(3) code with OpenMP. * Added optional parallelization of the bitslice DES code with OpenMP. * Replaced the bitslice DES key setup algorithm with a faster one, which significantly improves performance at LM hashes, as well as at DES-based crypt(3) hashes when there's just one salt (or very few salts). * Optimized the DES S-box x86-64 (16-register SSE2) assembly code. * Added support for 10-character DES-based tripcodes (not optimized yet). * Added support for the "$2y$" prefix of bcrypt hashes. * Added two more hash table sizes (16M and 128M entries) for faster processing of very large numbers of hashes per salt (over 1M). * Added two pre-defined external mode variables: "abort" and "status", which let an external mode request the current cracking session to be aborted or the status line to be displayed, respectively. * Made some minor optimizations to external mode function calls and virtual machine implementation. * The "--make-charset" option now uses floating-point rather than 64-bit integer operations, which allows for larger CHARSET_* settings in params.h. * Added runtime detection of Intel AVX and AMD XOP instruction set extensions, with optional fallback to an alternate program binary. * In OpenMP-enabled builds, added support for fallback to a non-OpenMP build when the requested thread count is 1. * Added relbench, a Perl script to compare two "john --test" benchmark runs, such as for different machines, "make" targets, C compilers, optimization options, or/and versions of John the Ripper. * Additional public lists of "top N passwords" have been merged into the bundled common passwords list, and some insufficiently common passwords were removed from the list. * Many minor enhancements and a few bug fixes were made. The following changes have been made between John 1.7.7 and 1.7.8: * The bitslice DES S-box expressions have been replaced with those generated by Roman Rusakov specifically for John the Ripper. The corresponding assembly code for x86 with MMX, SSE2, and for x86-64 with SSE2 has been re-generated. For other CPUs and for AVX/XOP, C compilers do a reasonably good job of generating the code from the supplied C source files (with intrinsics where relevant). The S-box expressions that we were using before had a 21% larger gate count, so theoretically this could provide a 21% speedup. In practice, though, a 12% to 14% speedup at DES-based crypt(3) hashes is typical. This effort has been sponsored by Rapid7: * Corrected support for bcrypt (OpenBSD Blowfish) hashes of passwords containing non-ASCII characters (that is, characters with the 8th bit set). Added support for such hashes produced by crypt_blowfish up to 1.0.4, which contained a sign extension bug (inherited from older versions of John). The old buggy behavior may be enabled per-hash, using the "$2x$" prefix. * The external mode virtual machine's performance has been improved through additional multi-op instructions matching common instruction sequences (assign-pop and some triple- and quad-push VM instructions were added). * A few minor bug fixes and enhancements were made. The following changes have been made between John and 1.7.7: * Added Intel AVX and AMD XOP instruction sets support for bitslice DES (with C compiler intrinsics). New make targets: linux-x86-64-avx, linux-x86-64-xop, linux-x86-avx, and linux-x86-xop (these require recent versions of GCC and GNU binutils). * A "dummy" "format" is now supported (plaintext passwords encoded in hexadecimal and prefixed with "$dummy$") - for faster testing and tuning of

custom wordlists, rule sets, .chr files, and external modes on already known or artificial passwords, as well as for testing of future and modified versions of John itself. * Apache "$apr1$" MD5-based password hashes are now supported along with the FreeBSD-style MD5-based crypt(3) hashes that were supported previously. Hashes of both of these types may be loaded for cracking simultaneously. * The "--salts" option threshold is now applied before removal of previously cracked hashes for consistent behavior with interrupted and continued sessions. * The "Idle = Y" setting (which is the default) is now ignored for OpenMP-enabled hash types when the actual number of threads is greater than 1. (Unfortunately, it did not work right at least with GNU libgomp on Linux.) * When a cracking session terminates or is interrupted, John will now warn the user if the cracked passwords printed to the terminal while cracking are potentially incomplete. It will advise the user to use the "--show" option to see the complete set of cracked passwords with proper post-processing. * When loading hashes specified on a line on their own (feature introduced in 1.7.6), the loader will now ignore leading and trailing whitespace. * Unless a hash type is forced from the command line, the loader will now print warnings about additional hash types seen in the input files (beyond the hash type autodetected initially). * For use primarily by the jumbo patch (and later by future enhancements to the official versions as well), the loader now includes logic to warn the user of ambiguous hash encodings (e.g. LM vs. NTLM vs. raw-MD5, all of which may be represented as 32 hexadecimal characters) and of excessive partial hash collisions, which it works around (these are typically caused by an incomplete implementation of a new hash type). * The "unique" and "unshadow" programs have been made significantly faster. * "DateTime", "Repeats", "Subsets", "AtLeast1-Simple", "AtLeast1-Generic", and "Policy" external mode samples have been added to the default john.conf. * The self-tests have been enhanced to detect more kinds of program bugs. * A few minor bug fixes and enhancements were made. The following changes have been made between John 1.7.6 and * Corrected a logic error introduced in JtR in "single crack" mode, we need a salt's key buffer even when we have no words corresponding to that salt's hashes to base candidate passwords on. We need this buffer to hold other salts' successful guesses for testing against this salt's hashes. The following changes have been made between John and 1.7.6: * Generic crypt(3) support (enabled with "--format=crypt") has been added for auditing password hash types supported by the system but not yet supported by John's own optimized cryptographic routines (such as SHA-crypt and SunMD5). * Optional parallelization of the above has been implemented by means of OpenMP along with glibc's crypt_r(3) or Solaris' MT-safe crypt(3C). * Optional parallelization of John's own optimized code for the OpenBSD-style Blowfish-based crypt(3) (bcrypt) hashes with OpenMP has been added. * A more suitable version of 32-bit x86 assembly code for Blowfish is now chosen on Core i7 and similar CPUs (when they happen to run a 32-bit build). * More optimal DES S-box expressions for PowerPC with AltiVec (making use of the conditional select operation) contributed by Dumplinger Boy (Dango-Chu) have been integrated. * The bitslice DES C source code has been reworked to allow for the use of arbitrary SIMD intrinsics, which was previously only implemented for AltiVec as a special case. * Support for SSE2 and MMX intrinsics with bitslice DES (as an alternative to the supplied assembly code) has been added (currently only enabled for SSE2 on x86-64 when compiling with GCC 4.4+). * Support for mixed-type longer virtual vectors (such as SSE2+MMX, SSE2+ALU,

AltiVec+ALU, and other combinations) with bitslice DES has been added (not enabled by default yet, primarily intended for easy benchmarks on future CPUs, with future compiler versions, with even more SIMD instruction sets, and with different DES S-box expressions that might be available in the future). * The obsolete 32-bit SPARC assembly implementation of DES has been dropped. * The loader will now detect password hashes specified on a line on their own, not only as part of an /etc/passwd or PWDUMP format file. * When run in "--stdin" mode and reading candidate passwords from a terminal (to be typed by the user), John will no longer mess with the terminal settings. * John will now restore terminal settings not only on normal termination or interrupt, but also when forcibly interrupted with two Ctrl-C keypresses. The following changes have been made between John 1.7.5 and * A new numeric variable has been added to the word mangling rules engine: "p" for position of the character last found with the "/" or "%" commands. The following changes have been made between John and 1.7.5: * Support for the use of "--format" along with "--show" or "--make-charset" has been added. * The choice of .rec and .log filenames for custom session names has been made more intuitive. * Support for "\r" (character lists with repeats) and "\p0" (reference to the immediately preceding character list/range) has been added to the word mangling rules preprocessor. * The undefined and undocumented behavior of some subtle word mangling rules preprocessor constructs has been changed to arguably be more sensible. * Some bugs were fixed, most notably JtR crashing on no password hashes loaded (bug introduced in The following changes have been made between John 1.7.4 and * Major performance improvements for processing of very large password files or sets of files, especially with salt-less or same-salt hashes, achieved primarily through introduction of two additional hash table sizes (64K and 1M entries), changes to the loader, and smarter processing of successful guesses (to accommodate getting thousands of hashes successfully cracked per second). * Many default buffer and hash table sizes have been increased and thresholds for the use of hash tables lowered, meaning that John will now tend to use more memory to achieve better speed (unless it is told not to with the "--save-memory" option). * Some previously missed common website passwords found on public lists of "top N passwords" have been added to the bundled common passwords list. * Some bugs introduced in 1.7.4 and affecting wordlist mode's elimination of consecutive duplicate candidate passwords have been fixed. The following changes have been made between John and 1.7.4: * Support for back-references and "parallel" ranges has been added to the word mangling rules preprocessor. * The notion of numeric variables (to be used for character positions and substring lengths along with numeric constants supported previously) has been introduced into the rules engine. Two pre-defined variables ("l" for initial or updated word's length and "m" for initial or memorized word's last character position) and 11 user-defined variables ("a" through "k") have been added. Additionally, there's a new numeric constant: "z" for "infinite" position or length. * New rule commands have been added: "A" (append, insert, or prefix with a string), "X" (extract a substring from memory and insert), "v" (subtract

and assign to a numeric variable). * New rule reject flags have been added: ":" (no-op, for use along with the "parallel" ranges feature of the preprocessor) and "p" (reject unless word pair commands are allowed, for sharing of the same ruleset between "single crack" and wordlist modes). * Processing of word mangling rules has been made significantly faster in multiple ways (caching of the current length, less copying of data, code and data placement changes for better branch prediction and L1 cache usage, compiler-friendly use of local variables, code micro-optimizations, removal of no-op rule commands in an initial pass). * The default rulesets for "single crack" and wordlist modes have been revised to make use of the new features, for speed, to produce fewer duplicates, and to attempt additional kinds of candidate passwords (such as for years 2010 through 2019 with "year-based" rules). * The idle priority emulation code has been optimized for lower overhead when there appears to be no other demand for CPU time. * The default for the Idle setting has been changed from N to Y. The following changes have been made between John and * "make check" has been implemented (for Unix-like systems only). * The "--test" option will now take an optional argument - the duration of each benchmark in seconds. * Section .note.GNU-stack has been added to all assembly files to avoid the stack area unnecessarily being made executable on Linux systems that use this mechanism. * Some very minor bugs that did not affect normal operation have been fixed. * Some unimportant compiler warnings have been fixed, a source code comment has been made more verbose and more complete. The following changes have been made between John 1.7.3 and * Corrected the x86 assembly files for building on Mac OS X. * Merged in some generic changes from JtR Pro. The following changes have been made between John 1.7.2 and 1.7.3: * Two Blowfish-based crypt(3) hashes may now be computed in parallel for much better performance on modern multi-issue CPUs with a sufficient number of registers (e.g., x86-64). * Bitslice DES assembly code for x86-64 has been converted to use instruction pointer relative addressing (needed for Mac OS X support). * New make targets: macosx-universal, macosx-x86-64, solaris-x86-64-cc, solaris-x86-64-gcc, solaris-x86-sse2-cc, solaris-x86-sse2-gcc, solaris-x86-mmx-cc, solaris-x86-mmx-gcc, solaris-x86-any-cc, linux-ia64; other changes to the Makefile. * Minor bug fixes. * "DumbForce" and "KnownForce" external mode samples have been added to the default john.conf. The following changes have been made between John 1.7.1 and 1.7.2: * Bitslice DES assembly code for x86-64 making use of the 64-bit mode extended SSE2 with 16 XMM registers has been added for better performance at DES-based crypt(3) hashes with x86-64 builds on AMD processors. * New make target for FreeBSD/x86-64. The following changes have been made between John and 1.7.1: * Bitslice DES code for x86 with SSE2 has been added for better performance

at DES-based crypt(3) hashes on Pentium 4 and SSE2-capable AMD processors. * Assorted high-level changes have been applied to improve performance on current x86-64 processors. * New make target for NetBSD/SPARC64. * Minor source code cleanups. The following changes have been made between John 1.7 and * Minor bug and portability fixes. * Better handling of certain uncommon scenarios and improper uses of John. * Bonus: "Keyboard" cracker included in the default john.conf (john.ini) that will try sequences of adjacent keys on a keyboard as passwords. The following changes have been made between John 1.6 and 1.7: * Bitslice DES code for x86 with MMX: more than twice faster than older non-bitslice MMX code. * Bitsliced the LM hash code as well: now several times faster. * Significant improvements to the generic bitslice DES code: +20% on RISC. * PowerPC G4+ AltiVec support (Mac OS X and Linux): effective 128-bitness for bitslice DES, resulting in huge speedups. * First attempt at generic vectorization support for bitslice DES. * Two MD5 hashes at a time for extra ILP on RISC: up to +80% on Alpha EV5+. * Generic Blowfish x86 assembly code in addition to the original Pentium version: +15% on the Pentium Pro family (up to and including Pentium III), +20% on AMD K6 (Pentium 4 and newer AMD CPUs are more happy running the original Pentium code for Blowfish). * Verbose logging of events to the global or a session-specific log file. * Better idle priority emulation with POSIX.1b (POSIX.4) scheduling calls. * System-wide installation support for *BSD ports and Linux distributions. * AIX, DU/Tru64 C2, HP-UX tcb files support in unshadow. * New make targets for Linux/x86-64, Linux/PowerPC, FreeBSD/Alpha, OpenBSD/x86-64, OpenBSD/Alpha, OpenBSD/SPARC, OpenBSD/SPARC64, OpenBSD/PowerPC, OpenBSD/PA-RISC, OpenBSD/VAX, NetBSD/VAX, Solaris/SPARC64, Mac OS X (PowerPC and x86), SCO, BeOS. * Bug and portability fixes, and new bugs. * Bonus: "Strip" cracker included in the default john.conf (john.ini). $Owl: Owl/packages/john/john/doc/CHANGES,v 1.74 2011/11/23 01:53:36 solar Exp $