# http://www.horizonchess.com/FAQ/Winboard/egtb.

endgame tablebase files of course, though naturally the more pieces on the board, the bigger the Endgame tablebase file. Each endgame tablebase file allows the user to check what the result is with best play. For example, the KNNKP tablebase,can allow you to input a specific position with such a material configuration, and find out what the result should be with best play. Nalimov endgame tablebases (one of several types of tablebases),for example could tell you if that position is a forced win and if so in how many moves, or a draw with best play. One thing to note is that,the raw endgame tablebases files are highly compressed,and require that you use some kind of graphic browser software, before you can view the files. .However, many Chess playing programs [See here for a list] can use endgame tablebases automatically,which will allow them to play that endgame perfectly. Most chess engines don't even need to reach the position covered in an endgame tablebases to use the tablebases. For example, a few moves before such a position, the engine calculates (but does not play yet) a series of exchanges that leads directly to a position in the tablebase. The engine will then look up (or probe) the tablebase and get the results for that hypothetical position. This should help improve play of course. But how are tablebases constructed in the first place?Basically endgame tablebases are database files of stored endgame positions calculated using retrograde analysis.Here's a basic layman explanation on how tablebases are constructed Here's another explanation by Bruce Moreland, author of Ferret (who has generated endgame tablebases of his own design). The algorithm works via induction. How these things works is that all possible positions are examined, and positions where one side is mated *now* are marked (apparently legal piece configuration, with one side to move, in check, and unable to get out of check). These are called "mated in 0" positions. Once this is done, all positions with each side to move are examined, and if any of them can reach a mated in 0 position, these are marked as "mate in 1". Now, we look at everything again and search for positions where all possible moves lead to "mate in 1" positions. These are marked as mated in 2. Now, we try to reach "mated in 2", these are "mate in 2". Next is a bit different, because we look for cases where it is impossible to avoid either "mate in 2" or "mate in 1". These are "mated in 3". And so on, until no progress is made. The rest of the positions are proven to be draws. It's a little trickier if it's possible to convert to a won ending, if you are trying to make a "distance to mate" database. You have to be careful of this idea of "no progress", because there are gaps. There might be a mate in 105 in KBP vs KN, but there might not be a mate in 95. This is because the mate in 105 might involve immediate conversion to a KBN vs KN, and yes, such positions do exist. How much stronger will it make my chess program play?

Endgame tablebases allow chess programs to play better in 2 situations. Firstly, when the endgame position covered by the tablebase occurs on the board. This is most always a gain for the chess engine (assuming you don't have a incomplete tablebase set). Unfortunately, the current set of endgame tablebases (5 pieces or less) cover mostly simple endgame positions that most chess engines can handle either by a combination of search or knowledge programmed in. So most engines will not gain much from using tablebases this way (except in rare positions). A much greater gain is achieved by the fact that chess engines can consult tablebases as a kind of oracle to decide whether to exchange down into a endgame covered by the tablebase.This allows the range of positions to be covered to be much larger. A chess engine can for example avoid exchanging down to losing positions (even if they are very deep) or win by converting into a absolutely won endgame. And yet, people have estimated that most engines gain at most 25 elo from using them, and many have found no significant difference in playing strength. Experiments results by Robert Allgeuer and Les Fernandez using Crafty and Yace show no significant difference in playing strength. In this article I listed 5 possible weaknesses of endgame tablebases that might cause this. Of the 5, some are quite rare (problems #2,#3) or can be avoided (problems #1) , except for the last problem which is the resulting slow down from accessing the hard-disk. Robert Allgeuer did a interesting experiment with Yace. And basically he found that while endgame tablebases allowed engines to reach and win such positions, in games where such positions were not reached they played weaker. By considering games where positions covered by endgame tablebases was actually reached on board ,Yace with tablebases played significantly better (about 115 Elo) On the other hand, in games where such positions was not reached, Yace with endgame tablebases played significantly weaker (about 94 elo). The implication was that in such games, using endgame tablebases was a liability. Robert writes "Apparently statistically for each game they successfully maneuver into a won 5 piece position, they also lose another one during the preceding endgame, so that overall this effect more or less cancels out the advantages of tablebases." The above results can be still extended by using different time controls, egtb caches, engines (some like SOS claim to be heavily dependent on tablebases because the author has stripped out all endgame related knowledge covered in tablebases) , but it seems that using the current set of tablebases (3-4-5 pieces), you can't expect more then a minimal strength increase if any. Still you might want to use tablebases for learning and analysis purposes. [A.2] What are the different endgame tablebases formats out there? What are the differences? There are various kinds of Tablebase formats widely in use, including Ken Thompson,Steven J. Edwards,Eugene Nalimov Tablebases and Chessmaster/De Koning EDGB (in the order in which they first appeared). While these are popular, some programmers have chosen to use endgame tablebases of their own design, for example Bruce Moreland's Ferret In general though, they fall into 2 types of tablebases. Distance to mate (DTM) and Distance to Conversion (DTC)tablebases. Roughly in layman terms, the difference is as follows. * Distance to mate (DTM) - DTM tablebase stores the shortest possible mate (in

ply) for each position. Eg Nalimov tablebases * Distance to conversion (DTC) - DTC tablebases stores the shortest number of ply between each position and a "conversion". A conversion is either a pawn promotion or a piece being captured or checkmate. Eg Thompson tablebases. * In addition, Distance to zeroing (DTZ) and the superior Distance to rule (DTR)tablebases have being proposed to overcome the fact that DTM and DTC do not recognise the fifty move rule. Only in theory so far. The tablebases also various in other ways Dr Robert Hyatt [Author of Crafty] explains the differences between them as follows [in a posting to rec.games.chess.comp 26/10/2000] "Edwards (Tablebase): Distance to mate values stored. The main problem with these is that they are larger than the others. Nalimov (Tablebase): Distance to mate values just like Edwards, but Eugene's files are compressed, and they may be used in the compressed form, with no penalty of any kind. Rather than way over 30 gigabytes for all of the 3-4-5 piece files, you end up with about 7.5 gigs. Thompson (Tablebase): Distance to conversion (capture which takes you to a smaller ending class). These are difficult to use in their compressed form, from inside a chess engine. They also provide different info than Eugene's database... i.e. it tells you something, but it doesn't differentiate between losing and drawing as the Nalimov files do. Best choice: Nalimov. Nearly every chess engine supports those... " This was of course written before the release of the De Koning EDGB format used by Chessmaster 9000 only in August 2002. Nalimov still remains the most popular tablebase format since it's supported by the largest number of engines. About Chessmaster endgame tablebase / De Koning EDGB De Koning EDGB - Propertry format used by only Chessmaster 9000. Like the Nalimov tablebase it is a Distance to mate tablebase. Unlike the Nalimov tablebase though, it stores only wins from the white point of view (similar to Thomson)."(It) searches only for wins for White. All positions that are not proven to be won by White are stored as unknown. These unknowns might be draws, or lost by White, or (in large numbers) illegal." [From FEG or Final endgame generator for Chessmaster 9000 ] Note the original version of the endgame generator had a minor flaw, and the EDGBs that shipped with the CDs also had this flaw. See here for more Other differences between the De Koning and Nalimov formats are * Nalimov's format has KxyKz and KzKxy merged. Eg in KRBKQ the wins for white (with RB) are stored as well as the losses for white. In Thompson's and De Koning's white can only win or not win. [From FEG or Final endgame generator for Chessmaster 9000 ] This change means that besides the usual pawn promotion case problems there are other "dependencies" problems between EGTB files (my comments) * Chessmaster 9000 currently does not probe the Tablebase in search but according to email correspondence this is a decision design and there is nothing in principle that stops this from occurring.

* De Koning EDGB are slightly smaller than compressed Nalimov endgame files in general.The full De Koning EGDBs came up to about 5.7 GB , the EGDB's can function with only either the WTM or BTM files.[Note during generation, you need both files!] This can reduce the space required to as little as 2.7 GB which is extremely light on hard-disk space compared to the 7.5 GB (as quoted by most people) required for Nalimov tablebaes. * The Chessmaster EDGB format generation process is also faster compared to Nalimov. (Full set of 5 piece tablebase take 5 days on 1.2 GHZ Athlon) It also requires much less free RAM (less than 13MB to be precise). The generator also makes generating 6-man with KWWWWB possible on a 32 bit computer possible , but that requires about 300MB (which isn't a lot by todays standards) of free RAM * 4 piece KxyK + KxKy 23 MB 5 piece KxyzK 1.25 GB 5 piece KxyKz 3.47 GB 5 piece KxKyz 1.04 GB (The numbers designate the full set. For simple lookup the half set will do (either wtm or btm). The half set is somewhat smaller than half of the full set.) * Otherwise De Koning EDGB's are similar to Nalimov in terms of castling and 50 moves rule (see below). More about Nalimov and Compressed Nalimov tablebases Nalimov tablebases are nearly "perfect" since they take into account en Passant. However, they don't take into account castling, but this flaw is probably of interest only to Chess problem Hobbyists. In general, though almost all modern Chess programs [including most Winboard programs] use Nalimov Tablebases partly because they are non-propriety and partly because they are more efficient. Also some of the 6 pieces Tablebases are now available in the Nalimov format. The Nalimov tablebases comes in 2 forms, uncompressed and compressed. The compressed ones end with the extension "emd". Most of the modern Chess programs( e.g. Crafty supports to the use of the compressed Nalimov format from versions 16.5 onwards) can use the tablebases in compressed form by uncompressing and using them on the fly as needed. One exception I'm aware of is a old version of Esc , a Winboard program released on 4 Feb 2001 which uses only the uncompressed form. If you generated the Nalimov tablebases yourself (see Section [A.8] ), they will already in the uncompressed form before applying datacomp.exe. datacomp.exe can also be used in reverse to get uncompressed Nalimov tablebases from compressed ones. Other uncommon types of Tablebases Bitbases There are other types of less common tablebases that work quite differently from those mentioned above. They generally go by the term "bitbases". There is a short articleby Roland Piffer, author of Patzer explaining what bitbases are. Unfortunately, it's available only in German. Here's a layman explanation attempted by yours truly.Unlike normal tablebases that store the full distance to mate or conversion, bitbases only stores information about whether a position is a forced win or loss. Sometimes, this alone is enough to help a chess engine to discard lines quickly. However bitbases (unlike typical

Nalimov , Thompson Chess Genius 6.5 Thompson Patzer**,Yace Paderborn Nalimov , Bitbases Knightdreamer,Delfi Bitbases Crafty*** Nalimov Gromit 2.2,Dragon [Pre-4.0],Bionic Edwards Most free Winboard engines**** Nalimov SCID Nalimov Madeleine Nalimov and GAFS * The latest update 02/04/2001.Refer to Rebel Web site . Prior versions did not have any endgame tablebase support. ** Available as part of the GambitSoft Winboard Edition II package. *** Old versions of Crafty used Edwards Tablebases.Crafty 15.21 onwards supports only Nalimov tablebases. Crafty 16.5 onwards also supports the use of compressed Eugene Nalimov tablebases. **** For a full list of Winboard engines that support tablebases , refer to Engine Overview page by Leo Dijksman Effects of GUI like Chessbase/Chessmaster Additional note, when Chess engines are used in Chessbase GUI or Chessmaster 9000, the GUI will use any tablebases available once the position on the board matches

Of those that came with the Chessmaster CD, the following sets: KNNK,KBKB,KBKN,KBKP,KNKR,KNKB,KNKN,KNNKP,KNKBB falls prey to this bug. I have generated these file sets and put them into one file for your download. You can them unzip them over the EDGB files installed from the CD. The chessmaster EDGB file sets are 12mb zipped and contain the Jix and Joo files necessary to be used in chessmaster. For the 2 five tablebase set KNKBB and KNNKP, I have made available only the btm files because only those are installed by the chessmaster CD. Others that are flawed include KNNKQ,KNNKR,KNNKB,KNNKN,KNPKQ,KNPKN,KNPKP,KBKQB,KBKQN,KBKQP,KBKRB,KBKRN, KBKRP,KBKBB,KBKBN,KBKBP,KBKNN,KBKNP,KBKPP,KNKQR,KNKQB,KNKQN,KNKRR ,KNKRB,KNKRN,KNKRP, KNKBN,KNKBP,KNKNN,KNKNP,KPKQR,KPKRR,KPKRB For more details see this post at CCC. Download the new FEG generator here To change the directory where chessmaster looks for tablebases edit cm.ini (found in the chessmaster 9000 directory) with a text editor. You need to change the path ofEGDBPath=whatevernewapth where your tablebases are. A second bug was found on Dec files are the ones where both sliding piece. As well as the capture (6-man only) or both. KPPKP." 26, dubbed "the transparent pawn bug". "Affected sides have at least a Pawn and White has at least a ones relying on them via promotion (KPPKP) or Regarding 3,4,5-man these are KQPKP, KRPKP, KBPKP,

Note, that even though there are some 5 piece tablebases on the CD to save space the 5 piece tablebases on the CD do not come with the WTM files.This means you have to regenerate all the 5 piece tablebases if you want the complete set due to the depedencies. [A.9] How large are the tablebase files? Can I put then all into one directory? Do I have to use a complete set of 5 men tablebases? Note the following refers to Nalimov tablebases. Users using Chessmaster format should ignore this section. Chessmaster 9000 itself is not vulnerable to the incomplete tablebase problem. The more tablebase files you install, the stronger the program will be. However, a full set of 3,4 and 5 Tablebase files takes about 7.05 GB of Hard disk space! BTW many of the "complete" endgame tablebase sets you can buy on CD are not really complete and may exclude some of the less important tablebases for example the so called 4+1 men tablebase sets [KXXXK] on the assumption that those are trivial enough to solve. Most people download a full set of 4 Piece databases and select only a few of the 5 pieces. Note you can mix all the different endgame tablebase into one directory if you wish. However if you do decide to split up the files among various directories (there might be some advantage in leaving a small subset of tablebases (clear wins like KXXXK in a chess engine game as opposed to for use in analysis.See here), you might need to check whether the Chess engine/interface you use allows you to point to multiple paths. NEW! Dieter Buerssner has informed me that in general all Chess engines/interfaces allow the use of separate paths for nalimov tablebases as long as they are separated by " commas, semicolons, or (at non- Windows systems) by colons." I have tried testing with a few engines, most seem to work. For example in Pepito , I add EGTB_PATH C:\fourpiece;g:\fivepiece.

noproblem On the other hand, some positions are not problematic, if there is a quick mate nearby after the pawn promotion that the Chess engine can see.

Crafty V19.4+ (to be safe use versions above 20) , Wilhelm are confirmed to work with them. Also confirmed working (ability to use in search) according to http://kd.lab.nig.ac.jp/chess/tablebases-online/ are Chess Tiger 15.0, Crafty 19.19, Fruit 2.2.1, Gandalf 6.0, Rybka 1.1, Shredder 9. This refers to the larger multiple split set, normal chess engines can probably handle the older and smaller 6 piece tablebase sets. Where to get them. Historically the main source for 6 men Nalimov tablebases has being at Bob Hyatt's ftp . However, as of April 2006, they are still recovering from a hard-disk crash, and there are no 6 piece tablebase there. They intend to recover though, so continue checking. http://olympuschess.com/tb6/index.html has a very limited number of 6 piece tablebase sets for download, however, the best, most complete and most reliable source for downloading 6 pieces Nalimov tablebases is still via emule here!. What are the most useful 6 piece tablebases? It's hard to say, but take a look at the following recommendations based on an analysis of positions reached by humans and computers.. Data about difficulty of winning (probably of draw) and other stats can also be found here in excel format. Historical information (outdated mostly) NEW! [Dec 14 2002] - More 6 men tablebases have being uploaded on Hyatt's site. However because of the size of the files (greater than 2 gb) they are broken into 2 parts (For example krnkbn.2.nbw.emd) and it requires a different probing code and currently [Dec 14 2002] no Chess engine can access it yet except Crafty 19.2 which will be released soon.From the user point of view, usage remains the same. As of March 2003, the new version of Wilhem can access them too. NEW![September 2003] Yet more endgame tablebases produced with more changes to the probing code. This current update includes some of the 4+2 with pawns tablebases, but very few programs can access it. One is wilhelm, the other is Crafty 19.4. Any programs released before them (around Oct 03) are unlikely to be able to use such tablebases. Here's an explanation by Eugene Nalimov. New access code would shortly be released in Crafty 19.2. Changes were necessary because (1) you should use 64-bit index to index some of 6-men TBs, (2) even after compression some files were greater than 2Gb, and some OSes cannot handle such files (or handle them in absolutely unportable way), so best solution is to split those files into several smaller ones. From chess program's point of view the only difference is that INDEX type now should be unsigned 64-bit integer, not unsigned 32-bit integer. Something like 5 minutes of work. Some unrelated problems are fixed in the code -- reported memory leaks, SMP problems, Mac OS X locks problem that manifested itself in access violation, very slow startup when TBPATH pointed to the CD-ROM drive and there was no disk in that drive, better integrity check for the TBs, etc. [Eugene Nalimov, CCC posting Dec 13,2002] The pawnless six men endgame tablebases are almost complete (except for the trivial ones). This is necessary of course to avoid the incomplete endgame tablebase problem (although you can use them to see the results without the best line of play), but it also makes them pretty useless and may actually slow down

the Chess engine if it probes useless positions. Of some interest is the KQQKQQ tablebase. It was one of the first 6 piece tablebase generated back in 1999 especially, to aid in the Kasparov versus World game. It is estimated a whole set of 6 piece Nalimov tablebases (when ready) will take up at least 1 terrabye (1000 GB). Unlike the nalimov tablebases it's possible to generate 6 piece Chessmaster tablebases using a normal 32 bit desktop and 512 ram.It has being reported that Marc Bourzutschky, has managed to use Chessmaster 9000's FEG to generate 6 piece tablebases including the KRPPKR in about a month using 3 desktop computers (1.9GHZ and 2.4GHZ P4, and an an old 800MHZ P3). He has completed the above plus ,KQPKQP,KRPKBP,KRPKNP,KPPPKR,KPPPKB,KQPPKQ,KRPPKQ and supporting tablebases, and the size is about 0.5 Terabytes. In addition certain 7 piece tablebases were completed.See here. [A.11] How can I find out which tablebase is corrupted? How do I know the endgame tablebases are working? If you download or generate your own tablebases, there is a possibility that some of them become corrupted. Some chess playing programs do a check and warn you if any of the tablebases are corrupted, but many don't. /p> In practice though, checking if endgame tablebases are working in general for a chess program is usually a easy affair. Just setup a position, cover by the tablebases, and set the Chess engine to analyze or move. If everything is done correctly, the engine will move instantly, or come up with a instant evaluation score that will not change. Usually, the thinking lines will indicate that the score comes from the endgame tablebase. E.g Crafty will display a <EGTB> in the principle variation. The above method is fine, except that it will take a long time to check all the tablebase files one by one. Here is what I recommend. In general, when setting up tablebases, there are 2 possible errors. 1. The interface is configured incorrectly. Generally this occurs when the path to the tablebase is in error. 2. The tablebase files are corrupted. The procedure above checks for the first problem. If you manage to get one tablebase working chances are the path is set correctly or the other tablebase files in the same directly should work, if they are not corrupted. Here are 3 ways to check for corrupted files. * The first solution is to make use of datacomp.exe to check the integrity of the file But you will need to do this using a batch file for each file. * The second method involves checking the MD5 sum of each file with the ones posted on the ftp site. See post by William Penn on CCC on "How to use MD5 Sums to verify tablebases".Note you can download Wilhelm a free Chess program with tablebase functions that will do this automatically * Lastly, run the endgame tablebases in the original Winboard version of Crafty or Comet. Both will inform you which specific tablebases are damaged. [A.12] What is datacomp.exe? Where can I get it?