#######################################################################

QuickBMS
by Luigi Auriemma
e-mail: aluigi@autistici.org
web: aluigi.org
home: http://aluigi.org/papers.htm#quickbms
Post : PKdoR
Private Server : www.gcexpert.net
#######################################################################
1) Introduction
2) Usage
3) How to create scripts
4) Notes
5) Support
6) Additional credits
#######################################################################
===============
1) Introduction
===============
QuickBMS is a multiplatform extractor engine that can be programmed
through some simple textual scripts for extracting the files from the
archives of any program and moreover games.
The script language from which QuickBMS is derived is MexScript
documented here: http://wiki.xentax.com/index.php/BMS
So QuickBMS is FULLY compatible with that original syntax and all the
scripts that were created here:
http://forum.xentax.com/viewtopic.php?t=1086
From version 0.4.1 QuickBMS supports also the WCX plugins of Total
Commander:
http://www.totalcmd.net/directory/packer.html
http://www.ghisler.com/plugins.htm
Anyway I improved that language for removing some implied fields (like
the file number in some commands) and adding new commands (like
Encryption) and behaviours (like negative GoTo) that allow QuickBMS to
work with tons of simple and complex formats.
The tool is open source under the GPL license and works on both Windows
and *nix and on both little and big endian architectures like Intel and
PPC.
For *nix users there is a Makefile in the src folder, the only
requirements are openssl, zlib, bzip2, lzo, ucl, mcrypt libraries.
lzo, ucl and mcrypt can be disabled adding -DDISABLE_LZO, -DDISABLE_UCL
and -DDISABLE_MCRYPT at compiling or modifying the Makefile.
#######################################################################
========
2) Usage
========
Simple and quick:
- double-click on quickbms.exe
- select the script or the plugin for the type of archive you want to
extract (for example zip.bms if it's a zip file)
- select the archive or multiple files
you can also select a whole folder by entering in it and then
typing "" in the "File name:" field and pressing Open
- select the output folder where extracting the files (you can specify
any file, it will be ignored because only the directory is taken)
- watch the progress of the extraction
That one was the simple "GUI" usage but QuickBMS can do various other
things when launched from the console because it supports various
command-line options.
You can view all the available options simply launching QuickBMS
without arguments, example:
Usage: quickbms [options] <script.BMS> <input_archive/folder> <output_folder>
Options:
-l list the files without extracting them, you can use . as output folder
-f W filter the files to extract using the W wildcard, example -f "*.mp3"
-F W as above but works only with the files in the input folder (if used)
-o if the output files already exist this option will overwrite them
automatically without asking the user's confirmation
-v verbose debug informations, useful for verifying possible errors
-c quick list of BMS commands and some notes about them and this tool
-L F dump the offset/size/name of the files inside the file F
-x use the hexadecimal notation in myitoa (debug)
-w enable the write mode required to write phisical input files with Put*
-d automatically creates an additional output folder with the name of the
input file processed without extension
-E experimental option for automatically reversing the endianess of any
memory file simply reading it field by field
Remember that the script, the input archive and the output folder are
ever REQUIRED and must be specified at the end of the command-line so
if you want to list all the mp3 files in a particular archive you must
use:
quickbms -l -f "*.mp3" zip.bms myfile.zip .
the last dot is the output folder (the current one) that is ignored
when is used the -l option.
From version 0.3 QuickBMS supports also a folder as input which means
that with a single command is possible to unpack all the archives of a
game without using the "find" tool
(http://gnuwin32.sourceforge.net/packages/findutils.htm).
Imagine to use the zip.bms script with all the zip files located in the
c:\windows\temp folder:
quickbms -F "*.zip" zip.bms "c:\Program Files" c:\output_folder
Except for -l, -f, -F and maybe -o options the other are intended for
debugging and so should be ignored by the normal users.
---
Advanced users could find useful also the other options:
-d automatically creates a folder with the name of the input file where
placing all the files, sometimes it could be useful in some rare
cases where users needed it.
-E if you have a bms script that simply reads a file format you can
change the endianess of all its numeric fields on the fly simply
using this option.
for example if you have a "get SIZE long" a 32bit number will be
read as usual and additionally it will be reversed (0x11223344 to
0x44332211) and placed at the same location.
remember that you need to specify also the -w option with physical
files, alternatively you can save the whole file in a memory file
and then dumping it so that -w is not needed.
#######################################################################
========================
3) How to create scripts
========================
Originally the tool was created for myself for making me able to write
quick extractors for simple archives immediately without writing a line
of C but it revealed to be a so powerful tool that now I use it really
for anything included the parsing of some protocols and much more.
So, how to write these scripts?
Well I guess that giving a look at http://wiki.xentax.com/index.php/BMS
is a good first step to understand at least the basis of this language
originally written by Mike Zuurman (alias Mr.Mouse of XeNTaX) in the
far 1997.
Then it's good to take a look at the various examples provided on
http://aluigi.org/papers.htm#quickbms and
http://forum.xentax.com/viewforum.php?f=10 and obviously to the
following list of available commands and their description.
A programming knowledge and background is not required but it's very
useful for entering in the "logic" of the scripts, while it's required
the full knowledge of the format to implement: reversing is ever useful
for figuring the needed fields.
Luckily in the extraction process is not needed to know all the fields
of an archive so a field like a CRC just doesn't matter while the
important fields are ever the filename, the offset and the size of the
files to extract.
You can notice all these useless fields in my scripts because I assign
them the name DUMMY.
Note that I will try to keep the following documentation updated as
much as I can, in any case refer also to the source code of quickbms.c
for any other technical doubt or possible additions or particular
behaviour of the tool in particular circumstances.
The fields between [] are optionals.
---
A quick list of available commands is available also when QuickBMS is
launched with the -c option, and REMEMBER to read the notes specified
there like the fact that:
- everything is considered a variable except if it starts with a number
in which case it's considered a numeric constant so when in this
text I talk about VAR, STRING and other types of data I refer EVER to
both variables and constants because they are EXACTLY the SAME thing
- all the commands and the names of the variables are case INsensitive
so "get OFFSET long" is the same as "GeT oFfSeT lOnG"
- everything works with signed 32 bit numbers (-2147483648 to
2147483647) so QuickBMS at the moment could not work well with files
over 2 gigabytes but it can seek on files of 4 gigabytes
so consider the following limits:
- max 4gb for archives
- max 2gb for the contained files
- the so called constant strings (depends by the context of the command)
are handled as strings in C notation like "\x12\x34\\hello\"bye\0"
and so on, in this case you must know a bit how this representation
works.
the keyword is "C language escape characters" or escape sequences,
they are very simple, take a look here:
http://msdn.microsoft.com/en-us/library/h21280bw%28VS.80%29.aspx
http://www.acm.uiuc.edu/webmonkeys/book/c_guide/1.1.html
- are supported also hexadecimal numbers if they start with 0x so
1234 and 0x4d2 are the same thing
- any operation made on fields bigger than 8 bits is controlled by the
global endianess, which means that any number and unicode field is
read in little endian by default otherwise it's valid the endianess
specified with the Endian command
File numbers:
every file opened in QuickBMS has a number assigned to it and in all
the commands that access to files this number is 0 (main file) if not
specified.
the first opened file is the input archive to which is assigned the
0 number, the others must be opened with the Open command
MEMORY_FILEs:
this is a particular type of temporary file which resides in memory
and works exactly like a normal file.
it's extremely useful for doing tons of things and are supported
various memory files: MEMORY_FILE, MEMORY_FILE2, MEMORY_FILE3, ...
TEMPORARY_FILE:
exists also another type of file called TEMPORARY_FILE which instead
resides phisically on the hard-disk with that exact name.
although its "temporary" name it's not deleted by the output folder
(anyway at the end of the process QuickBMS will ask if the user wants
to remove it or not) and is created even when it's used the -l option
for listing the files which makes it ideal in certains situations
like when it's used a chunks based file system.
.
indeed in this case using a MEMORY_FILE is a bad idea because the
continuous reallocation of the memory makes it terribly slow except
if you have pre-allocated the space of the MEMORY_FILE with the
PutVarChr command but in that case remains the problem of using too
much memory.
.
for using the temporary file remember to use it like in the following
example:
log TEMPORARY_FILE 0 0 # reset it in case it already exists (optional)
append # enables the append mode
...
log TEMPORARY_FILE OFFSET SIZE
...
append # disable the append mode
open "." TEMPORARY_FILE 1 # open the temporary file on the file number 1
Types of variables supported, also know as datatypes or types:
BYTE 8 bit, 0 to 0xff
SHORT 16 bit (aka INT), 0 to 0xffff
THREEBYTE 24 bit, 0 to 0xffffff
LONG 32 bit, 0 to 0xffffffff
LONGLONG fake 64 bit, so only 0 to 0xffffffff but takes 8 bytes in Get
STRING null delimited string
ASIZE special type used to return the size of the opened file,
used only with the GET command
FILENAME special type used to return the name of the opened file
like "myfile.zip", used only with the GET command
BASENAME special type used to return the base name of the opened
file like "myfile", used only with the GET command
EXTENSION special type used to return the extension of the opened
file like "zip", used only with the GET command
UNICODE special type used for unicode utf16 strings, the
endianess of the utf16 is the same used globally in the
script (watch the Endian command), it's used also for
converting an unicode string to an ascii one:
Set ASCII_STRING UNICODE UNICODE_STRING
BINARY special type used for binary strings in C notation like
"\xff\x00\x12\x34", used mainly as a constant
COMPRESSED a special type used for setting big strings and memory
files using a small amount of text, for using this type
you must take the original text/file, compress it with
zlib (you can use my packzip tool) and then encoding the
outputfile with base64 (you can use my bde64 tool) and
placing the result like the following:
set MEMORY_FILE compressed eNrtwbEJACAMBMBecIfvnMUxPuEJAe0UHN8
1LLzrbYKwDOjI96IN1cLveRfAGqYu
LINE special type used for carriage return/line feed delimited
string (so any string ending with a 0x00, 0x0a or 0x0d)
FULLNAME full path of the file, in reality at the moment it returns
the same path used in the input filename
CURRENT_FOLDER
the path from which has been launched QuickBMS
BMS_FOLDER the folder where is located the bms script
OUTPUT_FOLDER
the extraction folder (the last argument of QuickBMS)
INPUT_FOLDER
same as above
ALLOC a type used only in the Set command for a particular thing
And now the list of commands in the same order in which they are listed
in the QuickBMS source code in the function parse_bms():
.......................................................................
QuickBMSver VERSION
checks if the current version of QuickBMS is enough recent to
support the script... used rarely, mainly for scripts created after
I have added or fixed a functionality of the tool.
arguments:
VERSION oldest version of QuickBMS for which was created the script
examples:
QuickBMSver 0.2.4
.......................................................................
Clog NAME OFFSET ZSIZE SIZE [FILENUM]
extracts the file decompressing it in real-time, this operation
doesn't affect the current position of the input file.
the decompression algorithm used in the operation is decided by the
ComType command.
the content of the extracted file can be decrypted automatically
using the Encryption command.
arguments:
NAME name of the output file
OFFSET position of the archive where is located the file
ZSIZE size of the compressed data in the archive
SIZE size of the uncompressed file
FILENUM number of the file associated to the archive (0)
examples:
Clog NAME OFFSET ZSIZE SIZE
Clog "dump.dat" 0 ZSIZE 10000000 # the file will have the real size and no
t 10000000
.......................................................................
FindLoc VAR TYPE STRING [FILENUM] [ERR_VALUE]
finds the first occurrency of a given string or number from the
current offset of the archive.
it's usually used when the format of the archive is not known or it
is a particular text file... not much used.
arguments:
VAR the variable which will receive the offset of the occurrency
TYPE can be STRING, UNICODE or a number
STRING string in C notation or number, depending by the TYPE
FILENUM number of the file associated to the archive (0)
ERR_VALUE by default FindLoc terminates the script if no string
is found but if ERR_VALUE is specified this value will
be assigned to VAR without terminating when there are no
other occurrencies
examples:
For
FindLoc OFFSET string "filename="
...
FindLoc OFFSET string "filename=" 0 ""
if OFFSET == ""
cleanexit
endif
Next
.......................................................................
For [VAR] [OP] [VALUE] [COND] [VAR]
...
Next [VAR]
a classical "for" loop with initializers, conditions and
incrementers.
there is also the Break instruction available to break the loop at
any moment (note that the Break instruction is not suggested in
presence of multiple For because in some rare cases it could give
problems).
Next is the command which delimits the loop and at the same time
increments the given variable if specified.
practically it means:
- give VALUE to VAR (or performs a mathematical operation on it)
- perform a loop till VAR is "condition" than the other VAR
- perform all the operations between For and Next
- increment VAR
all the parameters are optionals.
arguments:
VAR variable to which assign a value
OP from version 0.3.10 this value can be any of the
available Math operators (so not only the classical '=')
VALUE value to assign to the variable or part of the math operation
COND condition, the same of the IF command
VAR second part of the condition
examples:
For i = 0 < FILES
...
next i
For
# do what you want here, this is an endless loop
Next
For VAR1 = VAR1 != VAR2
# this is exactly the same of using while(VAR1 != VAR2) {...} in C
Next
.......................................................................
Get VAR TYPE [FILENUM]
reads strings and numbers from the open files, it's also the most
used command.
arguments:
VAR variable which will receive the read data
TYPE watch the description of the types explained before
FILENUM number of the file associated to the archive (0)
examples:
Get OFFSET long
Get NAME string
.......................................................................
GetDString VAR LENGTH [FILENUM]
reads a defined amount of data from the file and stores it in the
given variable.
useful with filenames and other strings that have a lenght specified
in a previous 8, 16 or 32 bit field.
arguments:
VAR variable which will receive the read data
LENGTH amount of bytes to read
FILENUM number of the file associated to the archive (0)
examples:
GetDString NAME NAME_LENGTH
GetDString NAME 0x100
.......................................................................
GoTo OFFSET [FILENUM] [TYPE]
go to the given absolute position of the file, the classical fseek
arguments:
OFFSET position to reach, if it's negative it will be considered
an "OFFSET bytes from the end of the file" position
note that from QuickBMS 0.3.5a only the constants can be
negative with GoTo (even keeping the 4gb limit in any
case), all the variables will be handled as unsigned (0
to 0xffffffff)
FILENUM number of the file associated to the archive (0)
TYPE SEEK_SET (default), SEEK_CUR, SEEK_END
examples:
GoTo OFFSET
GoTo 0x100
GoTo -4 # 4 bytes before the end of the file
.......................................................................
IDString [FILENUM] STRING
terminates if the the signature at the current position of the file
differs than the provided string.
from version 0.4.5 if the string doesn't match and is 4 bytes long
QuickBMS will swap and recompare it, if it matches the endianess
will be changed.
arguments
FILENUM number of the file associated to the archive (0)
STRING string in C notation
examples:
IDString "PK\x03\x04"
IDString " KAP"
.......................................................................
Log NAME OFFSET SIZE [FILENUM]
extracts the file, this operation doesn't affect the current
position of the input file.
the content of the extracted file can be decrypted automatically
using the Encryption command.
arguments:
NAME name of the output file
OFFSET position in the archive where is located the file
SIZE amount of the data to extract
FILENUM number of the file associated to the archive (0)
examples:
Log NAME OFFSET SIZE
Log "dump.dat" 0 SIZE
.......................................................................
Math VAR OP VAR
performs a mathematical operation between the two variables and
places the result in the first one.
note that for compatibility all the operations are performed using
signed 32 bit numbers by default.
this makes the difference with some operation like the shift ones,
so pay attention!
for unsigned operations I have implemented the 'u' additional OP
from QuickBMS 0.3.14
arguments
VAR variable which acts as input and output
OP + sum
* multiplication
/ division
- substraction
^ xor
& and
| or
% modulus
! negation of var2 (0 becomes 1 and any other value becomes 0)
~ complement of var2 (like "xor 0xffffffff")
< shift left
> shift right
l rotate left
r rotate right
s byte swapping
w bit swapping
= assign var2 to var1
n negative value of var2 (so -var2)
a absolute value of var2
v radix
p power
x alignment, examples:
var1=1, var2=16, result=16
var1=16, var2=16, result=16
var1=17, var2=16, result=32
add a 'u' before or after OP for forcing the usage of unsigned
operations useful with shift, divisions and possibly other
operations
VAR other input variable
examples:
Math SIZE *= 0x100
Math OFFSET <<= 2
Math TMP = SIZE
Math TMP ~ TMP
Math TMP n TMP
Math TMP2 a TMP
Math SIZE u/= 5
Math RADIX v= 2
.......................................................................
Open FOLDER NAME [FILENUM] [EXISTS]
opens a file for reading, practically assigns a file number/id to
an existent file that you want to use
arguments:
FOLDER FDDE, means that you want to open the file in the same
location of the input one which has the extension
provided with NAME
FDSE, it will consider NAME as a file located in the
same folder of the input file (very useful)
any other value is considered the folder where is located
the file to load so use "." for the current output
folder
NAME read above, NAME can be also a ? in which case QuickBMS
will ask the user to insert the name of the file to open
manually
if NAME is "" then will be performed a flush operation
that could be useful (or not?) only in write mode
FILENUM number of the file associated to the archive (0)
EXISTS if the file doesn't exist this variable will be set to
0 otherwise 1 (exists). by default QuickBMS terminates
with an error if the file doesn't exist.
examples:
Open FDDE DAT 0
Open FDDE IDX 1
Open FDSE "myfile.zip"
Open "." TEMPORARY_FILE 1
.......................................................................
SavePos VAR [FILENUM]
returns the current position of the file, like ftell
arguments:
VAR variable which will contain the offset
FILENUM number of the file associated to the archive (0)
examples:
SavePos OFFSET
.......................................................................
Set VAR [TYPE] VAR
command for assigning a constant or a variable to another variable
with the possibility of changing its type (like with unicode)
arguments:
VAR output variable or memory file
TYPE useless type, indeed it can be also not specified in
which case will be used the String value.
it's useless because in QuickBMS doesn't exist a real
difference between numbers and strings except some
cases (like unicode, filename, basename, extension
and possibly others).
remember that here TYPE can have also the special
BINARY type explained before or STRLEN used as
alternative at the Strlen command
VAR variable or constant to assign
examples:
Set i long 0
Set TMP long SIZE
Set TMPNAME NAME
Set MEMORY_FILE binary "\x12\x34\x56\x78"
Set ASCII_VAR unicode UNICODE_VAR
.......................................................................
Do
...
While VAR COND VAR
another less used type of cycle where is performed the check of the
condition at the end of the cycle... really rarely used.
if you need a C-like "while(...) {...}" use the For command.
arguments:
VAR first part of the condition
COND condition, check the If command below for additional info
VAR second part of the condition
examples:
Do
...
While OFFSET < MAX_OFFSET
.......................................................................
String VAR OP VAR
the equivalent of the Math command for the strings
arguments:
VAR input and output variable
OP = just a copy or if var2 is a number it will consider it
a raw strubg:
var2="0x44434241", result="ABCD"
+ append the second string to the first one
- if the second variable is a positive number the
string will be truncated at that amount of bytes
from the end
if the second variable is a negative number the
string will be truncated at that amount of bytes
from the beginning
otherwise will be removed all the occurrencies of
the second string in the variable
^ xoring of the string with the second one (looped if shorter)
< var1="thisisastring", var2="4", result="isastring"
% truncate the variable at the position obtained by
the modulus of its length and the number in the second
variable
& var1="thisisastring", var2="isa", result="isastring"
| var1="thisisastring", var2="isa", result="string"
> if the second variable is a number:
var1="thisisastring", var2="4", result="thisisast"
otherwise:
var1="thisisastring", var2="isa", result="this"
b byte2hex of var2: var2="abc", result="616263"
B as above but uses the var2 as a null delimited string (strle
n)
h hex2byte of var2: var2="616263", result="abc"
e experimental encryption based on the Encryption command
E as above but uses the var2 as a null delimited string (strle
n)
c experimental compression based on the ComType command
C as above but uses the var2 as a null delimited string (strle
n)
u var2="hello", result="HELLO"
l var2="HELLO", result="hello"
p a printf-like experimental work-around
r reversed string
VAR the second variable or string
examples:
string FULLPATH += NAME
string FULLPATH += \
string NAME -= ".zip"
string NAME -= 4
string FULLPATH p= "c:\folder\%04x%04x.dat" VAR1 VAR2
.......................................................................
CleanExit
terminates the script, it's possible also to use just Exit
.......................................................................
If VAR COND VAR
...
[Elif VAR COND VAR]
...
[Else]
...
EndIf
checks various conditions and performes the needed operation when
the condition is verified, translated:
- If is ever the first condition
- Elif is another condition and can be used endless times
- Else is the operation to do when no conditions are met, the last
- EndIf delimits the If command
arguments:
VAR first part of the condition
COND valid for both strings and numbers:
< minor
> major
!= different
== equal
>= major/equal
<= minor/equal
& string: var2 is included in var1
number: logical AND
^ string: equal
number: logical XOR
| number: logical OR
% number: modulus
/ number: division
<< number: shift left
>> number: shift right
! number: negation
!! number: true, use it to know if VAR is non-zero
~ number: complement
ext compares the string after the last dot
basename compares the string before the last dot
any other operation supported by the Math command (valid
only for the numeric variables)
add a 'u' before COND for forcing the usage of unsigned
operations useful with shift, divisions and possibly other
operations
VAR second part of the condition
Examples:
If NAME != ""
...
Endif
If MASK & 1
Elif MASK & 2
Elif MASK & 4
Elif MASK & 8
Else
Endif
.......................................................................
GetCT VAR TYPE CHAR [FILENUM]
reads a string till the reaching of the CHAR delimiter.
arguments
VAR output variable
TYPE ignored because doesn't exist a type in this operation
CHAR the delimiter character as 8bit number
FILENUM number of the file associated to the archive (0)
examples:
GetCT NAME string 0x0a
GetCT NAME string 0x3b
set DELIMITER_BYTE long 0x0a
GetCT NAME string DELIMITER_BYTE
.......................................................................
ComType ALGO [DICT]
selects the specified compression algorithm to use with the Clog
command.
from version 0.3.8 of QuickBMS I have also added the possibility of
choosing a number as ALGO for an "utopic" idea about a scanner for
being able to guess the possible compression algorithm in an unknown
compressed data block:
http://aluigi.org/papers/bms/comtype_scan.bms
or the simpler one to use (because doesn't need to be modified):
http://aluigi.org/papers/bms/comtype_scan2.bat
http://aluigi.org/papers/bms/comtype_scan2.bms
obviously this feature is only for advanced people who knows exactly
what they are doing... and works really perfectly.
note that some algorithms work only on Windows
arguments:
ALGO copy, simple copy that could be useful in some rare cases with
data encrypted with block ciphers like AES and blowfish
zlib, RFC 1590 (aka windowbit 15, the data starts with a 'x')
DICT supported
deflate, RFC 1591 (aka windowbit -15) used for example in the ZI
P files
DICT supported
lzo1a till lzo2a, LZO (remember that the most used is lzo1x)
DICT supported
lzss, with default configuration (dictionary of 4096 bytes)
this particular algorithm can be fully configured setting the
EI, EJ and P fields plus another number rarely used.
for setting them it's enough to use a DICT equal to something
like "12 4 2" which means EI:12 (N:4096), EJ:4 (F:18), P:2
lzx, used by the old (aka jurassic) unlzx tool and on Amiga
gzip, automatic handling of the gzip data
remember that in this case the uncompressed size is
ignored and calculated automatically so in CLog use
ZSIZE ZSIZE
pkware, the algorithm also known as blast/explode/implode/DCL
lzma, 5 bytes + lzma
lzma86head, 5 bytes + 8 bytes (size) + lzma
lzma86dec, 1 byte + 5 bytes + lzma
lzma86dechead, 1 byte + 5 bytes + 8 bytes (size) + lzma
lzmaefs, the format implemented in ZIP
bzip2
XMemDecompress, Xbox 360 LZX algorithm of xcompress.lib
use DICT to specify a custom WindowSize and CompressionPartiti
onSize
like "131072 524288"
hex, from "01234567" to 4 bytes: 0x01 0x23 0x45 0x67
base64, from "aGVsbG8=" to "hello", supports also the Gamespy
and URL chars
uuencode
ascii85
yenc
COM_LZW_Decompress, used in Vietcong
milestone_lzw, the lzw algorithm used in the Milestone games
lzxcab, the lzx algorithm used in the handling of the cab files
(libmspack 21 0)
lzxchm, the lzx algorithm used in the handling of the chm files
(libmspack 16 2)
rlew, 16 bit RLE algorithm used in AIM Racing
lzjb, a compression used in a file system for *nix
sfl_block, expand_block from iMatix Standard Function Library
sfl_rle, expand_rle from iMatix Standard Function Library
sfl_nulls, expand_nulls from iMatix Standard Function Library
sfl_bits, expand_bits from iMatix Standard Function Library
lzma2, 1 bytes + lzma2
lzma2_86head, 1 bytes + 8 bytes (size) + lzma2
lzma2_86dec, 1 byte + 1 bytes + lzma2
lzma2_86dechead, 1 byte + 1 bytes + 8 bytes (size) + lzma2
nrv2b, UCL
nrv2d, UCL
nrv2e, UCL
huffboh, an unrecognized compression used in the Asura engine
uncompress, the lzw algorithm used in the compress utility
(the lzw data starts from offset 3 of the .Z files)
dmc, Dynamic Markov Compression (DMC)
lzhuf, aka LZH/LHA
lzari
rle7
rle0
rle
rlea, another generic rle decompressor
use DICT to choose the escape char
bpe, byte pair encoding
quicklz
q3huff, Adaptive Huffman algorithm used in the Quake 3 engine
unmeng, algorithm used in DreamKiller
lz2k, algorithm used in various games developed by Traveller's T
ales
darksector, a very basic algorithm used in the game Dark Sector
mszh, used in the LossLess Codec Library
un49g, used in the games of 49Games
unthandor, used in the old game Thandor
doomhuff, huffman used in doom, hexen, skulltag and other doom p
orts
the DICT field can be used to specify a custom HuffFreq table
(256 float elements)
aplib
tzar_lzss, used in Tzar of HaemimontGames
DICT must contain the name of the variable with the algorithm
number to use, example: ComType tzar_lzss MYVAR
lzf, aka fastlz
clz77, the lz77 algorithm available on http://compressions.sourc
eforge.net/about.html
lzrw1
dhuff, Huffman Decompression in LDS ("lossless datacompression s
ources" kit 1.1)
fin, from LDS
lzah (not tested)
lzh12, aka -lh4-
lzh13, aka -lh5-
grzip, aka GRZipII
ckrle, Chilkat RLE
quad, note that in this version I have removed the handling
of the first 32bit number containing the size of the
uncompressed data
balz, note that in this version I have removed the handling of
the first 9 bytes from the files that contains an ID byte and
a 64bit field containing the size of the uncompressed data
(the working of this algorithm is not guarantee)
deflate64
shrink (not tested)
z-base-32
base32hex
base32crockford
base32nintendo
base???, if ALGO starts with "base" then will be taken its
subsequent number (for example 32 if it's "base32") and
used for the conversion. the function supports ANY base
conversion from 2 to 256.
for bases major than 64 will be used a char table starting
from byte 0x00 so base128 will have a charset from 0 to 0x7f
brieflz
paq6, raw data block
shcodec
hstest_hs_unpack (never tested, could be removed in future)
hstest_unpackc (never tested, could be removed in future)
sixpack (never tested)
ashford (never tested, could be removed in future)
jcalg
jam
lzhlib
srank
zzip
scpack
DICT supported (for the SCPACK_TABLE field)
rle3
bpe2
bcl_huf, Basic Compression Library
bcl_lz, Basic Compression Library
bcl_rice, Basic Compression Library
you must use DICT to specify the format (1 to 8)
bcl_rle, Basic Compression Library
bcl_sf, Basic Compression Library
scz
szip
ppmd, ppmd var.i rev.1 with ZIP specifics so 2 bytes of info fol
lowed by the compressed data
ppmdi_raw
ppmdg (could not work!)
ppmdg_raw, requires DICT "par1 par2" (could not work!)
ppmdh (could not work!)
ppmdh_raw, requires DICT "par1 par2" (could not work!)
ppmdj (could not work!)
ppmdj_raw, requires DICT "par1 par2 par3" (could not work!)
sr3c
huffmanlib
sfastpacker, smart+simple mode
sfastpacker2, smart-mode only
dk2, used in Dungeon Keeper 2 of Bullfrog/EA
lz77wii, (use input size as output size in clog)
lz77wii_raw10, tag 0x10 lz77
darkstone, lz77 compression used in the game DarkStone
sfl_block_chunked, as sfl_block with automatic handling of
the chunks if used
yuke_bpe, used in the PS2 games developed by Yuke
stalker_lza, used in STALKER, use the output size equal to
the compressed one (it's handled internally)
prs_8ing
puyo_cnx, raw compressed data from offset 0x10
puyo_cxlz, raw compressed data from offset 0x8
puyo_lz00, raw compressed data from offset 0x32
puyo_lz01, raw compressed data from offset 0x10
puyo_lzss, raw compressed data from offset 0x4
puyo_onz, raw compressed data from offset 0x4
puyo_prs
falcom
cpk, used by the CRI developers
bzip2_file, exactly like bzip2 but it automatically calculates
the output size, so use ZSIZE ZSIZE in clog
lz77wii_raw11, tag 0x11 lzss
lz77wii_raw30, tag 0x30 rle
lz77wii_raw20, tag 0x20 huffman (not supported at the moment)
pglz, postgresql compression (headerless)
UnPackSLZ
slz_01, used in tri-ace slz type 1
slz_02, used in tri-ace slz type 2
lzhl
d3101
squeeze
lzrw3
ahuff
arith
arith1
arith1e
arithn
compand
huff
lzss
lzw12
lzw15v
silence
rdc
ilzr
dmc2
diffcomp
lzr
lzs (aka mppc)
lzs_big (aka mppc_big)
mohlzss
mohrle
yaz0 (aka szs)
byte2hex
un434a
unzip_dynamic (automatic zlib/deflate and output size, cool)
xxencode
pack, the one supported in gzip
zlib_noerror, as zlib but doesn't quit in case of errors
deflate_noerror, as zlib but doesn't quit in case of errors
--- recompression algorithms ---
zlib_compress
deflate_compress
lzo_compress
xmem_compress
bzip2_compress
rnc
rnc_raw
pak_explode, alone in the dark
*note: if I have forgot to list an algorithm you can find
it in the COMP_ enumeration inside quickbms.c
DICT an optional C string containing the bytes of the dictionary
or particular parameters depending by the chosen algorithm
.......................................................................
ReverseLong VAR
inverts a 32bit variable from 0x44332211 to 0x11223344 and viceversa.
arguments:
VAR variable to flip
.......................................................................
ReverseShort VAR
inverts a 16bit variable from 0x2211 to 0x1122 and viceversa.
arguments:
VAR variable to flip
.......................................................................
Endian TYPE
changes the current global endianess of the read/written data, the
default one is little endian.
arguments:
TYPE little/intel endianess where 0x11223344 is stored as 44 33 22 11
big/network endianess where 0x11223344 is stored as 11 22 33 44
Examples:
print "little endian"
endian big
print "big endian"
endian little
print "little endian"
.......................................................................
FileXOR SEQ [OFFSET]
any read operation (Get, *Log and so on) on any file will perform
also the xoring of the read data with the numbers contained in the
given string or in the given variable.
the OFFSET field by default is zero which means that if the data
must be xored with more than one byte (a "xor key") the first byte
of the xor key is the first byte at OFFSET which is 0 by default
(beginning of the file).
resuming: the FileXOR command works with ANY file access
arguments:
SEQ sequence of space-separated 8bit numbers, it can be a:
- sequence of bytes separated by space like 0x12 or
"0x12 0x34 0x56"
- a numeric variable
at the moment is not possible to use a key in string
mode (use the Encryption command for doing it) so if you
have a string convert it to a numeric sequence first.
set it to 0 or "" for disabling the xor
OFFSET needed only for the xor key.
if the archive is xored with a xor key from its beginning
(so first byte of the archive xored with the first one
of the key) this argument is usually not necessary
if instead it's the file to extract that is xored, this
argument must have the same offset of the file (so
just reuse the same OFFSET used in Log)
examples:
filexor 0xff
filexor "0x12 0x34 123 255"
filexor MYXORBYTE
filexor "0x12 0x34 123 255" OFFSET
Log NAME OFFSET SIZE
.......................................................................
FileRot SEQ [OFFSET]
exactly as for FileXOR but it performs a sum operation.
for example if SEQ is 0x01 and the file contains "hello" it will
become "ifmmp" while if SEQ is -1 or 0xff it will become "gdkkn".
-1 and 0xff are the same because it's a 8 bit number.
resuming: the FileRot command works with ANY file access
watch the previous arguments and examples.
.......................................................................
Strlen VAR VAR
calculates the length of the second variable and stores it in the
first one.
the length is the amount of bytes till the reaching of a 0 delimiter.
note that for practical reasons this command can be emulated also
using "set VAR strlen VAR"
arguments
VAR destination variable which will contain the length
VAR variable of which calculating the length
examples
strlen NAME_LENGTH NAME
.......................................................................
GetVarChr VAR VAR OFFSET [TYPE]
a particular and sometimes very useful command which works exactly
like accessing to an array of elements contained in the second
variable (so a string or a memory file).
this simple but effective method allows the customization of strings
and variables for example for creating a custom header (like a DDS)
and moreover for performing operations on a piece of the memory (like
a custom encryption algorithm).
some real examples are my Deer Hunter 2004/2005 scripts.
arguments
VAR destination variable which will contain the read element
VAR variable or memory file from which you want to get the
element
OFFSET position of the second variable where taking the element
TYPE if not specified it's a BYTE so a 8bit number, you can
choose any of the available data types and it will go
in the destination variable
examples:
For i = 0 < SIZE
GetVarChr TMP MEMORY_FILE i
GetVarChr TMP MEMORY_FILE i long
# GetVarChr TMP MEMORY_FILE i string
Next i
.......................................................................
PutVarChr VAR OFFSET VAR [TYPE]
the "write-mode" alternative of the previous command which allows
to perform various complex operations with custom algorithms (like
in my Deer Hunter 2004/2005 scripts).
note that from version 0.3.3 PutVarChr can be also used as an
allocator of memory that could be useful for example in the
implementation of custom decompression algorithms or, moreover, for
pre-allocating a MEMORY_FILE avoiding to waste time and memory with
the incremental allocation, remember only to use Log MEMORY_FILE 0 0
after it for resetting the position of the MEMORY_FILE.
arguments
VAR variable or memory file to which you want to put the
element
OFFSET position of the first variable where placing the element
VAR source variable which will contain the element to write
TYPE if not specified it's a BYTE so a 8bit number, you can
choose any of the available data types and it will go
in the destination variable
examples:
For i = 0 < SIZE
GetVarChr TMP MEMORY_FILE i
Math TMP ^= 0xff
PutVarChr MEMORY_FILE i TMP
Next i
.......................................................................
Debug
switch command which enables the -v option in real-time for a
specific portion of the script, used only for debugging.
.......................................................................
Padding VAR [FILENUM]
when called it performs an automatic GoTo to the next position of
the file skipping the padded data.
imagine to have a file where it's used an alignment of 4 bytes and
your current file offset is 0x39, if you use Padding 4 the offset
will be automatically changed to 0x3c.
arguments:
VAR size of the alignment, for example 4 or 16 and so on
FILENUM number of the file associated to the archive (0)
examples:
Get NAME string
Padding 4
get OFFSET long
.......................................................................
Append
switch command which enables the append mode in the *Log commands,
so if the output filename is still the same it will be not
overwritten while it will be concatenated.
note that the user must ever confirm the appending for security
reasons.
examples:
append
Log "dump.dat" 0 0x10
Log "dump.dat" 0x10 0x100
the following is a particular example for allocating a MEMORY_FILE
(from version 0.3.3) and using it instead of TEMPORARY_FILE saving
space on the disk and performances:
math TMP = CHUNKS
math TMP *= 0x8000
log MEMORY_FILE 0 0
putvarchr MEMORY_FILE TMP 0 # improves the speed with pre-allocation
log MEMORY_FILE 0 0 # reset the position and size of the file
append
for i = 0 < CHUNKS
...
clog MEMORY_FILE OFFSET ZSIZE 0x8000
next i
get SIZE asize MEMORY_FILE
.......................................................................
Encryption ALGO KEY [IVEC] [MODE] [KEYLEN]
one of the most interesting commands which allow to set a decryption
algorithm used for any read operation on the files.
resuming: the Encryption command works ONLY with Log and CLog.
from version 0.3.15 QuickBMS supports also the hashing algorithms of
OpenSSL, the hash will be placed in binary mode in the variable
QUICKBMS_HASH and in hex mode in QUICKBMS_HEXHASH.
arguments:
ALGO aes, Rijndael
blowfish
des
3des-112
3des-168
rc4
tea
xtea
xxtea
idea
xor
rot
charset, the substitution algorithm which uses a charset of 256
chars
charset2, as above but the substituition is inverted (confortabl
e in some rare cases)
twofish
cast5
seed
serpent
ice
icecrypt, ICE algorithm with key implemented as in the homonym
program, the difference with "ice" is ONLY in the key
rotor, added as experiment, ivec contains the number of rotors (
12)
ssc, Leverage SSC
wincrypt, aka cryptdecrypt or cryptencrypt
use the ivec field for specifying:
- the hashing algorithm (CryptCreateHash)
- the encryption algorithm (CryptDeriveKey)
- the provider type and the provider name (CryptAcquireContext
)
example:
encryption CryptDecrypt "mykey" "CALG_MD5 CALG_RC4 PROV_RSA_
FULL"
cryptunprotect, key is used to specify the entropy so the defaul
t is ""
zipcrypto, remember that the first 12 bytes are the encryption h
eader
set the ivec to 1 for automatically cutting the first 12 bytes
md_null, from OpenSSL (does nothing)
md2, from OpenSSL
md4, from OpenSSL
md5, from OpenSSL
sha, from OpenSSL
sha1, from OpenSSL
dss, from OpenSSL
dss1, from OpenSSL
ecdsa, from OpenSSL
sha224, from OpenSSL
sha256, from OpenSSL
sha384, from OpenSSL
sha512, from OpenSSL
mdc2, from OpenSSL
ripemd160, from OpenSSL
whirlpool, from OpenSSL
enc_null, from OpenSSL (does nothing)
des_ecb, from OpenSSL
des_ede, from OpenSSL
des_ede3, from OpenSSL
des_ede_ecb, from OpenSSL
des_ede3_ecb, from OpenSSL
des_cfb64, from OpenSSL
des_cfb1, from OpenSSL
des_cfb8, from OpenSSL
des_ede_cfb64, from OpenSSL
des_ede_cfb1, from OpenSSL
des_ede_cfb8, from OpenSSL
des_ede3_cfb64, from OpenSSL
des_ede3_cfb1, from OpenSSL
des_ede3_cfb8, from OpenSSL
des_ofb, from OpenSSL
des_ede_ofb, from OpenSSL
des_ede3_ofb, from OpenSSL
des_cbc, from OpenSSL
des_ede_cbc, from OpenSSL
des_ede3_cbc, from OpenSSL
desx_cbc, from OpenSSL
dev_crypto_des_ede3_cbc, from OpenSSL
dev_crypto_rc4, from OpenSSL
dev_crypto_md5, from OpenSSL
rc4, from OpenSSL
rc4_40, from OpenSSL
idea_ecb, from OpenSSL
idea_cfb64, from OpenSSL
idea_ofb, from OpenSSL
idea_cbc, from OpenSSL
rc2_ecb, from OpenSSL
rc2_cbc, from OpenSSL
rc2_40_cbc, from OpenSSL
rc2_64_cbc, from OpenSSL
rc2_cfb64, from OpenSSL
rc2_ofb, from OpenSSL
bf_ecb, from OpenSSL (bf stands for blowfish)
bf_cbc, from OpenSSL
bf_cfb64, from OpenSSL
bf_ofb, from OpenSSL
cast5_ecb, from OpenSSL
cast5_cbc, from OpenSSL
cast5_cfb64, from OpenSSL
cast5_ofb, from OpenSSL
rc5_32_12_16_cbc, from OpenSSL
rc5_32_12_16_ecb, from OpenSSL
rc5_32_12_16_cfb64, from OpenSSL
rc5_32_12_16_ofb, from OpenSSL
aes_128_ecb, from OpenSSL
aes_128_cbc, from OpenSSL
aes_128_cfb1, from OpenSSL
aes_128_cfb8, from OpenSSL
aes_128_cfb128, from OpenSSL
aes_128_ofb, from OpenSSL
aes_128_ctr, from OpenSSL
aes_192_ecb, from OpenSSL
aes_192_cbc, from OpenSSL
aes_192_cfb1, from OpenSSL
aes_192_cfb8, from OpenSSL
aes_192_cfb128, from OpenSSL
aes_192_ofb, from OpenSSL
aes_192_ctr, from OpenSSL
aes_256_ecb, from OpenSSL
aes_256_cbc, from OpenSSL
aes_256_cfb1, from OpenSSL
aes_256_cfb8, from OpenSSL
aes_256_cfb128, from OpenSSL
aes_256_ofb, from OpenSSL
aes_256_ctr, from OpenSSL
camellia_128_ecb, from OpenSSL
camellia_128_cbc, from OpenSSL
camellia_128_cfb1, from OpenSSL
camellia_128_cfb8, from OpenSSL
camellia_128_cfb128, from OpenSSL
camellia_128_ofb, from OpenSSL
camellia_192_ecb, from OpenSSL
camellia_192_cbc, from OpenSSL
camellia_192_cfb1, from OpenSSL
camellia_192_cfb8, from OpenSSL
camellia_192_cfb128, from OpenSSL
camellia_192_ofb, from OpenSSL
camellia_256_ecb, from OpenSSL
camellia_256_cbc, from OpenSSL
camellia_256_cfb1, from OpenSSL
camellia_256_cfb8, from OpenSSL
camellia_256_cfb128, from OpenSSL
camellia_256_ofb, from OpenSSL
seed_ecb, from OpenSSL
seed_cbc, from OpenSSL
seed_cfb128, from OpenSSL
seed_ofb, from OpenSSL
blowfish, from mcrypt
des, from mcrypt
tripledes, from mcrypt
threeway, from mcrypt
gost, from mcrypt
safer-sk64, from mcrypt
safer-sk128, from mcrypt
cast-128, from mcrypt
xtea, from mcrypt
rc2, from mcrypt
twofish, from mcrypt
cast-256, from mcrypt
saferplus, from mcrypt
loki97, from mcrypt
serpent, from mcrypt
rijndael-128, from mcrypt
rijndael-192, from mcrypt
rijndael-256, from mcrypt
enigma, from mcrypt
arcfour, from mcrypt
wake, from mcrypt
3way,
skipjack
anubis
aria
crypton
frog
gost
lucifer
mars
misty1
noekeon
seal
safer
tomcrypt algorithms (not listed here because too much so
check quickbms.c or contact me)
crc, a complete and powerful checksum function that can
be fully configured in any aspect:
key is the polynomial (use "" for the default crc32)
ivec contains the size of the crc (8/16/32), the initial
value (like -1), the final xor value (-1, the complement),
the type (various supported) and the reverse mode during
the generation of the table. if you need the classical crc16
(0xc0c1) use: encryption crc "0x8005" "16 0 0 0 1"
the result is placed in the variable QUICKBMS_CRC
"", disable the encryption
note that for the algorithms supported by mcrypt you
can force their loading by preceeding ALGO with "mcrypt"
like "mcrypt_enigma" and you can decide also their mode
like "mcrypt_enigma_ecb" or "mcrypt_enigma_cbc", list:
cbc, ecb, cfb, ofb and nofb
KEY the key to use in C notation like "\x11\x22\x33\x44" or
"this is my key"
from version 0.3.9 this value can be also a variable or a
memory file
IVEC the ivec to use in C notation, an ivec is an additional
key used for increasing the security of encryption
algorithms that are usually defined as ECB without ivec
and CBC (and other names) with ivec
MODE 0 for decryption (default), 1 for forcing the encryption mode
if no ivec is used remember to place a "" at its place
KEYLEN forces the usage of a certain length of the key, this one
has been introduced only for avoiding the problem of using
a variable as KEY containing zeroes in it
examples:
Encryption aes "0123456789abcdef" "" 1 # encryption without ivec
Log MEMORY_FILE 0 SIZE
Encryption aes "0123456789abcdef" # decryption without ivec
Log "redecrypted_file.dat" 0 SIZE MEMORY_FILE
Encryption aes "\x12\x34\x56\x78"
set MEMORY_FILE binary "\x12\x34\x56\x78"
Encryption aes MEMORY_FILE
Encryption aes MY_VARIABLE
Encryption md5 ""
.......................................................................
Print MESSAGE
prints a string in C notation with the values of the variables if
they are specified between two % chars.
from version 0.4.5 is possible to specify the maximum amount of bytes
to visualize and if they must be showed in hex mode specifying some
flags after a '|' like in the examples.
arguments:
MESSAGE C notation string, each %VAR% word is converted to its
value
examples:
print "the variable OFFSET of the file %FILENAME% has the value %OFFSET%"
print "this is the first line\nthis is the second line\n"
print "variable %VAR% and %VAR2%"
print "variable %VAR|h% and %VAR2|hex%"
print "variable %VAR|3% and %VAR2|4%"
print "variable %VAR|3h% and %VAR2|h4%"
.......................................................................
GetArray VAR ARRAY VAR
and
PutArray ARRAY VAR VAR
experimental commands implemented for a possible future usage but
that at the moment have been used enough rarely although very useful.
they work on a dynamic array where is possible to store data,
something like a temporary place or a stack.
examples:
PutArray 0 0 FIRST_VAR
PutArray 0 1 SECOND_VAR
PutArray FIRST_VAR 0 0
PutArray SECOND_VAR 0 1
.......................................................................
CallFunction NAME [KEEP_VAR]
StartFunction NAME
...
EndFunction
calling and declaration of a function identified by NAME where the
values of the variables are saved till the termination of the
function when they are restored.
it works very well for recursive archives like those used by "The
Void" and "Another Day" although it could result not much immediate
in its usage to understand but it's very powerful.
so remember the rule: the content of the variables is restored when
the function terminates, except if you set KEEP_VAR to 1.
it's a good idea to place all the functions (StartFunction till
EndFunction) at the end of the scripts.
arguments:
NAME name assigned to the function
KEEP_VAR set to 1 if you want to keep the content of the variables
without resetting them, in short words:
0 = for recursive functions (default)
1 = for normal functions that change variables
examples:
watch thevoid.bms and another_day.bms
.......................................................................
ScanDir PATH NAME SIZE [FILTER]
function without a real usage, it simply scans the PATH folder and
fills the NAME and SIZE variables with the name and the size of
each file found.
at the moment this function doesn't have a purpose so ignore it.
use -F in quickbms for filtering the scanned files if you used a
folder as input (I tell this because some users could think to use
this command for that purpose which is not needed).
arguments:
PATH must be ".", the current folder
NAME output variable which receives the name of the file, it
will be "" when there are no other files
SIZE output variable which receives the size of the file, it
will be -1 when there are no other files
FILTER same job as -F, this filter is valid only if -F wasn't
specified
examples:
For
ScanDir "." NAME SIZE
if NAME == ""
cleanexit
endif
Next
...
For
ScanDir "." NAME SIZE "*.jpg"
if NAME == ""
cleanexit
endif
Next
.......................................................................
CallDLL DLLNAME FUNC/OFF CONV RET [ARG1] [ARG2] ... [ARGn]
this is the command which allows to use plugins inside QuickBMS.
the idea came from the possibility of using the custom
decompression/decryption functions (both exported and internals)
located in executables and dlls avoiding the boring reversing of
all the functions.
works with both real files and MEMORY_FILEs (even if they contain
dll data!)
unfortunately this is not much possible with the functions got from
executables where are used static variables due to some technical
reasons (memory addresses that don't match due to the different
relocation of the executable inside the memory), for example if the
function uses the memory between 006c0000 and 006d0000 it's highly
possible that such range of memory is not allocated or is already
in use because the executable has not been loaded (LoadLibrary) in
its original address because already occupied.
obviously there are no problems with the dlls.
note that you can even use a dll inside a MEMORY_FILE but be sure
it's not packed because couldn't work.
arguments:
DLLNAME name of the dll or executable where is located the
function, example "mylib.dll"
QuickBMS can even load raw binary files that contain
only the dumped function... very useful
FUNC/OFF it can be the name of the function to import in which
case it must be exported by the dll/exe
or the relative offset where is located the function,
remember that the relative offset is NOT the absolute
one but it's the offset related to the image base of
the exe/dll (so if normally the dll loads at offset
10000000 and the function is at 10012345 then the
offset is 0x12345)
CONV calling convention:
stdcall: aka winapi, used by default in Visual C
cdecl: used by almost any other C/C++ compiler
fastcall: Microsoft/gcc fastcall
borland: the fastcall convension used by the Borland
compilers like Delphi
watcom
pascal
safecall
syscall
optlink
carion
RET the variable which will contain the value returned by
the function, use "" if there is no return value
[ARGS] all the arguments of the function
from version 0.3.5 QuickBMS implements also the pointer
to arguments when they are preceded by a & or a * like
&SIZE which means that the dll/code receives the
pointer to that variable and can modify its content.
it works only with numeric variables
examples:
idstring LZ2K
get SIZE long
get ZSIZE long
log MEMORY_FILE 0xc ZSIZE
putvarchr MEMORY_FILE2 SIZE 0 # like malloc
#calldll "TransformersDemo.exe" 0x263c50 cdecl "" MEMORY_FILE MEMORY_FILE2
ZSIZE SIZE # 00663C50
calldll "unlzk.dll" "unlz2k" cdecl SIZE MEMORY_FILE MEMORY_FILE2 ZSIZE SIZ
E
log "dump.dat" 0 SIZE MEMORY_FILE2
.......................................................................
Put VAR TYPE [FILENUM]
...
PutDString VAR LENGTH [FILENUM]
...
PutCT VAR TYPE CHAR [FILENUM]
...
these are EXACTLY like the Get* functions except for the fact that
they perform write operations.
for using these commands on a phisical file (so MEMORY_FILEs
excluded) MUST be used the -w option at runtime needed for both
technical and security reasons.
.......................................................................
GetBits VAR BITS [FILENUM]
this is an experimental function for reading bits from the files.
when you use a GoTo function or change the current offset of the
file with a Get* command the variable containing the bit position
(practically the amount of bits read from the previously byte taken
from the file) will be set to 0.
note that the function is 100% endian compatible so the result
changes if you choose the little or big endian mode.
arguments:
VAR destination variable, can be a number if the bits are
from 0 to 32 or a string for major sizes
BITS number of bits to read
FILENUM number of the file associated to the archive (0)
#######################################################################
========
4) Notes
========
The following are some exceptions in the usage of QuickBMS, not real
bugs but things that can't work (at least at the moment) due to the
very flexible nature of the tool or things that it's useful or
interesting to know:
x Log "123.txt" OFFSET SIZE (SOLVED!)
creates the file 123 and not 123.txt, this happens because "123.txt"
is considered a constant number due to the rule that anything
starting with a number (or a '-') is handled as a constant number.
this behaviour didn't happen with the previous versions of the tool
because wasn't used the number optimization which saves tons of cpu
cycles in some scripts.
from version 0.3.12 I have decided to implement the full verification
of the string to know if it's a number or a string, luckily there is
almost no loss of performances
- number and strings, due to the usage of the optimizations the following
script will NOT result in "mytest46600x12349999999999", the result will
be "mytest4660-1717986919":
set NAME string "mytest"
set NUM long 0x1234
string NAME += NUM
print "%NAME%"
set NUM string "0x12349999999999"
string NAME += NUM
print "%NAME%"
this is a good compromise because the previous situation is very very
"rare" and in any case can be bypassed using multiple "string NAME += chr"
and the gain in performance is incredible for the multiple in-script
operations, so this is the best solution
- set NAME string MEMORY_FILE
log NAME 0 0
produces no phisical file because it's considered a MEMORY_FILE, it
happens because the dumping function receives "MEMORY_FILE" as output
file name.
at the moment there is no fix anyway it's a very very rare event
(never happened to find an archive containing a file with that name)
and so not a priority
- any Clog operation with a compressed or uncompressed size minor/equal
than zero produces a file with a zero size, but this is not a problem
of the tool because it's the perfectly logical behaviour in these
situations.
if it's necessary to use a compression which gets the SIZE value
automatically (like base64 or stalker_lza) is enough to specify the
compressed size as uncompressed size:
clog NAME OFFSET ZSIZE ZSIZE
or
clog NAME OFFSET ZSIZE 1
- all the extracted files are loaded completely in memory before being
dumped for various technical reasons so if the file to dump has a
size of 800 megabytes this is the same size which will be allocated
in memory or even double or more if the file must be decompressed, so
it's good to have a good amount of free RAM when handling big archives
or at least a good virtual memory/swap space
- the tool has been created to be 100% compatible with the original
MexScript language and its syntax/logic, so I have tried to add less
new commands as I could and if possible providing an alternative using
the original set of commands (for example the Strlen command and
"Set VAR strlen VAR") and in any case I have tried to maintain the
logic of the program (for example encryptions and compressions applied
in the file operations only).
so if something looks complex it has been made for matching the
original structure and logic of the scripting language
- QuickBMS uses many third party code (mainly compression) and even if
I have tried to adjust them a bit unfortunately many of these external
functions were a disaster or missed any security requirement.
that's the reason why the comtype scanning feature causes so many
crashes with invalid data
Other things to know or strange behaviours will be listed when I will
figure (or remember) them.
A curiosity for who is crazy for the optimizations of the compilers:
the PPMD (ppmd var.i rev.1) algorithm compiled with -O3 (gcc) is a lot
slower than if compiled with -O2 and not much different is the
situation of the other algorithms.
While with -Os the code is smaller (about 300kb the Windows exe) but
there is a small loss of performances of max 15/20% with some
algorithms (like PPMD) and scripts with many get/putvarchr and math
operations.
#######################################################################
==========
5) Support
==========
QuickBMS, like any other of my projects, is fully supported by me and
is ever in continous development for adding new encryption and
compression algorithms and sometimes for fixing bugs because I'm the
first user of this tool and so I have a direct interest in it.
So remember to contact me for any doubt or new idea via:
- mail to aluigi@autistici.org
- my forum on http://forum.aluigi.org
- xentax forum on http://forum.xentax.com
#######################################################################
=====================
6) Additional credits
=====================
QuickBMS uses also various public-domain code and code released under
GPL/LGPL or other open source and free licences.
Compression:
- zlib, inflateback9 (for deflate64) and blast of Jean-loup Gailly and
Mark Adler http://www.zlib.net
- LZO of Markus F.X.J. Oberhumer http://www.oberhumer.com/opensource/lzo/
- LZSS, LZARI, LZHUF of Haruhiko Okumura
- unlzx.c of Erik Meusel
- LZMA and LZMA2 of Igor Pavlov http://www.7-zip.org
- bzip2 of Julian Seward http://www.bzip.org
- ascii85 partially derived from http://www.stillhq.com/svn/trunk/ascii85/decode
85.c
- libmspack of Stuart Caie http://www.cabextract.org.uk/libmspack/
- lzjb from http://cvs.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/co
mmon/fs/zfs/lzjb.c
- iMatix SFL compression http://download.imatix.com/pub/
- UCL of Markus F.X.J. Oberhumer http://www.oberhumer.com/opensource/lzo/
- code from the uncompress utility of "The Regents of the University of Californ
ia"
- Dynamic Markov Compression implementation of Gordon V. Cormack
http://plg1.cs.uwaterloo.ca/~ftp/dmc/dmc.c
- 3 small rle algorithms from ScummVM
https://scummvm.svn.sourceforge.net/svnroot/scummvm/scummvm/tags/release-1-0-0
rc1/engines/sword1/screen.cpp
- bpe of Philip Gage http://www.csse.monash.edu.au/cluster/RJK/Compress/bpd.c
- QuickLZ of Lasse Mikkel Reinhold http://www.quicklz.com
- Quake 3 Huffman code of Id Software http://www.idsoftware.com
- mszh from the LossLess Codec Library
- Doom Huffman code from the Doom/Hexen source code
- aPLib of Jrgen Ibsen http://www.ibsensoftware.com/products_aPLib.html
- LZF of Marc Alexander Lehmann http://home.schmorp.de/marc/liblzf.html
- LZ77 of Arkadi Kagan http://compressions.sourceforge.net/about.html
- LZRW1 of Ross Williams http://www.ross.net/compression/lzrw1.html
- an Huffman implementation of Bill Demas on LDS
- the FIN algorithm (useless and very close to LZSS) on LDS
- LZAH/LZH12/13 of Dik T. Winter http://homepages.cwi.nl/~dik/english/ftp.html
- GRZipII/libGRZip of Grebnov Ilya (only the win32 code is linked to it
because it's composed by too much files and it's not a priority)
- rle of Chilkat Software http://www.chilkatsoft.com/chilkatdx/ck_rle.htm#source
- Quad of Ilia Muraviev http://quad.sourceforge.net
- Balz of Ilia Muraviev http://balz.sourceforge.net
- unshrink of Info-Zip http://www.info-zip.org/UnZip.html
- PPMd algorithms of Dmitry Shkarin http://compression.ru/ds/
- BriefLZ of Jrgen Ibsen http://www.ibsensoftware.com/products_aPLib.html
- PAQ6 of Matt Mahoney http://cs.fit.edu/~mmahoney/compression/paq.html#paq6
- shcodec of Simakov Alexander http://webcenter.ru/~xander/
- hstest of tom ehlert
- SixPack of Philip G. Gage
- ashford of Charles Ashford
- JCALG1 of Jeremy Collake http://www.bitsum.com/jcalg1.htm
- jam/unjam of W. Jiang
- lzhlib of Haruhiko Okumura and Kerwin F. Medina for the adaptation of the code
- Srank P M Fenwick http://www.cs.auckland.ac.nz/~peter-f/FTPfiles/srank.c
- Zziplib/Zzlib of Damien Debin http://damiendebin.net/archives/zzip/download.ph
p#zzlib
- scpack of Philip Gage
- rle3 and bpe2:
http://musyozoku211.blog118.fc2.com/blog-entry-13.html
http://blog-imgs-17.fc2.com/m/u/s/musyozoku211/bpe2.txt
http://blog-imgs-17.fc2.com/m/u/s/musyozoku211/rle3.txt
- Basic Compression Library of Marcus Geelnard http://bcl.comli.eu
- SCZ of Carl Kindman http://scz-compress.sourceforge.net
- szip of HDF Group http://www.hdfgroup.org/doc_resource/SZIP/
- sr3c of Kenneth Oksanen http://cessu.blogspot.com
- Huffman library of Douglas Ryan Richardson http://huffman.sourceforge.net
- SFastPacker of Aleksey Kuznetsov http://www.utilmind.com/delphi3.html
- lz77wii of Hector Martin http://wiibrew.org/wiki/Wii.py
- prs 8ing code posted by tpu http://forum.xentax.com/viewtopic.php?p=30387#p303
87
- puyo compressions of not.nmn and nickwor http://code.google.com/p/puyotools/
- falcom decompression of http://www.geocities.jp/pokan_chan/
- cpk of hcs http://hcs64.com/files/utf_tab04.zip
- DSDecmp of Barubary http://code.google.com/p/dsdecmp/
- pglz_decompress PostgreSQL Global Development Group http://www.postgresql.org/
- SLZ of Adisak Pochanayon
- LZH-Light of Sergey Ignatchenko ftp://66.77.27.238/sourcecode/cuj/1998/cujoct9
8.zip
- d3101 of Advanced Hardware Architectures/HP
- squeeze (R. Greenlaw, Amiga port by Rick Schaeffer ???)
- some algorithms of Mark Nelson & Jean-loup Gailly from The Data Compression Bo
ok
- Ed Ross Data Compression
- ilzr of Jose Renau Ardevol
- some code from the C User's Journal
- dmc from T.L. Yu
- 'Uses libLZR by BenHur' http://www.psp-programming.com/benhur/
- lzs of Matthew Chapman http://www.rdesktop.org
- yaz0 of thakis (http://www.amnoid.de/gc/)
- RNC from Simon Tatham (original author has not been specified)
- PAK_explode of Cyril VOILA
Encryption:
- all the algorithms provided by OpenSSL http://www.openssl.org
- xtea from PolarSSL http://www.polarssl.org
- some encryption algorithms from GnuPG and libgcrypt http://www.gnupg.org
- ICE of Matthew Kwan http://www.darkside.com.au/ice/index.html
- Rotor module from the Python source code
- http://mcrypt.sourceforge.net
- all the various public algorithms implemented in version 0.4.1 like
3way, anubis, gost, skipjack and so on
Others:
- MemoryModule of Joachim Bauch
- various signatures from http://mark0.net/soft-trid-e.html
- various signatures from http://toorcon.techpathways.com/uploads/headersig.txt
Notes:
- some (many?) of the original codes have been modified a bit to make
them usable in QuickBMS for the memory2memory (aka in-memory)
decompression and for other possible fixes or for reducing the amount
of code (for example removing the compression routine leaving only the
decompression one)
note that I have avoided to make this third-party code more secure
because it's not the job of QuickBMS, so almost all the code (except
some rare cases) has been used "as-is"
- almost all the algorithms implemented here have been selected because
have or could have been used or it has been claimed to have been used
in real software/games or they are enough known to deserve their
implementation in QuickBMS
- if the files have been modified or don't have the original license
informations (could happen only with small functions that didn't
contain a license header in origin) please follow the provided links
for more details
- tell me if I have forgiven someone/something in this section, my
memory and level of attention are quite ridiculous.
and tell me also if is necessary to include other files or comments
inside these third-party files or about them.
I have included the list to the original websitest as additional
reference also for having major informations about their license in
case the included files don't have it in their comments (/* */)
#######################################################################