Mikrotik Router OS "Gotchas"

http://www.gpsinformation.org/hotspot/mikrotik_router_os_gotchas.html

Mikrotik Router Operating System "GOTCHAs" by Joe Mehaffey (updated 6/1/2004)

I have received comments from users (and experienced myself) a number of problems in setting up a Mikrotik OS HotSpot router that cause a lot of wasted time. I have documented the worst here so as to (hopefully) help keep others from falling into the same traps. Recently, Mikrotik has advised me that "We are not targeting customers that need basic IP training. We can't do this with our business model." I personally had a great deal of computer experience but none in router programming and since Mikrotik has no overall application notes to get a new user started, I did require extra help at first which Mikrotik was not able to provide. Eventually, to get my application configured the way I needed it, I got (paid) assistance from Eje Gustafsson at http://www.wisp-router.com/ for a reasonable fee. Since that startup phase in about May 2003, the major problems I have had were with a continuous list of bugs found in new versions, and in trying to upgrade to get rid of the bugs and finding more bugs. At one point, I was directed to go to the Mikrotik BETA software to get around a problem I was having in a version of 2.7.3, I tried 2.8beta6 and used it successfully for many months. However, my advice is to avoid at all costs Mikrotik BETA software as it is often unstable. After not being able to find a stable beta version from 2.8beta7 through 2.8beta12, and later to 2.8rc5 (release candidate), it was suggested I try the prior edition version 2.7.18. This version has proved stable and I have found it free of "crash" or other problems as far as operation in my documented Mikrotik Hotspot configuration is concerned. As of June 2004 I am still using it in several systems. The following are things to be aware of as you consider using the Mikrotik Router OS. Generally, these are not "killer" problems but they can cause a lot of wasted time when you are a new user and not sure exactly how things are supposed to work. A good place to visit is the Mikrotik Router OS Forum. There you can read the archives and learn problems and answers to problems that others may be having with a particular version of Mikrotik OS. I am no longer regularly updating this material so some items may not be up to date with Mikrotik's latest versions. One Mikrotik manager has objected to my posting this list of potential problems. I think that Mikrotik OS users have a need to know about potential problems so as to be able to avoid them where possible. I have offered to delete/correct any item(s) as soon as Mikrotik produces an update eliminating any listed problem. I have been listing my "gotcha" list for some months and indeed, about twenty of the original problem items have been corrected and the listing eliminated. If anyone finds that ANY of the "gotcha" items listed below have been corrected in RELEASED software, PLEASE email the author so I may make the necessary corrections. It is my desire for this information to be entirely factual and correct.. Thanks for your help. Please send changes/corrections to: Joe Mehaffey. 1) The Mikrotik OS is only modestly well documented and Mikrotik has less technical support than many users expect for a production software product as complex as is the Mikrotik OS. The 400+ page manual is detailed, but much more a reference guide for the knowledgeable router programmer than a tutorial on how to initially set up a system. In the past months, the documentation has been getting better with more and more applications guides, installation wizards and examples in the manual, but getting detailed answers to most any technical question directly from Mikrotik Technical Support can be difficult. Yet, the Mikrotik software is generally well designed, robust, serviceable, generally reliable (but not well tested at the factory), and inexpensive. For these reasons I have written my HOTSPOT SETUP PROCEDURE to assist users in getting an initial system up and running.. I am hopeful that Mikrotik will soon realize that a superbly featured product like the Mikrotik OS will sell A LOT better (and be worth a higher price) if it is accompanied by quality applications guide information, good support AND if the system is tested throughly prior to release. (For example, over about six months, 2.8 software has gone through eleven betas, six release condidates and eleven full releases. Real stability began for

1 of 5

31/7/2006 2:04

Mikrotik Router OS "Gotchas"

http://www.gpsinformation.org/hotspot/mikrotik_router_os_gotchas.html

most users at around 2.8.10. Microtik appears to depend on customer debug of their software, even production versions.) Several users say they would not mind paying double the price Mikrotik charges IF they would improve software stability and quality and improve their technical support. I concur. 2) Related to #1, Mikrotik software appears to be mostly "user tested" without the use and benefits of the usual structured and detailed software test plan for software as complex as the Mikrotik OS. This leads to both beta and production software that works in many "NORMAL" configurations but with a lot of software operating sequences that remain untested until some user tries to use a particular feature in some heretofore untested scenario. Since these paths may not have not been explored before, it is not too unusual for a particular user to encounter problems in areas of the program where other users report "no problems".. For this reason alone, be SURE to maintain frequent backups so you can reset or reload the OS and get back up and running without a lengthy time delay. Always make a new backup just BEFORE you make any but the most trivial database change. 3)Mikrotik's "ChangeLog" is cryptic and apparently includes only a fraction of the changes actually made from version to version. Mikrotik has moved to correct this and as of 2.8.11 (June 2004) has promised to include more detail in the changelog so users can better determine if a problem they have has been corrected in a later version. 4) MT has been improving documentation lately and the problem of incomplete command details and command syntax information in either the manual or in the OS command listing is not so bad as it was. Still, commands sometimes have "unpublished" features and options that turn out to be very useful but that are (seemingly) documented nowhere but in the Mikrotik programmer's memory. The moral of this story is: If you cannot figure out how to do something, ASK. You likely have a 75% chance that there is a feature (or workaround) to do what you want even if it is not addressed in the manual. 5) My experience when reporting problems to Mikrotik and asking for assistance has been variable. On "failure of router to operate properly" problems, I have found that unless I have been able to give Mikrotik a detailed test sequence so they can reproduce the problem, they assume I am "just another inexperienced user who has fouled himself up" with some programming error. (Early on, this was indeed true from time to time!) In any case, Mikrotik can be excused somewhat as complex failure scenarios are about impossible to diagnose and reproduce without a detailed test sequence as a guide. The difficult part to understand is why Mikrotik does not employ a structured test procedure in their QC operations. Then when they encountered one of these strange failures and fixed it, a test sequence would be added to the test procedure. It can be a slow process, but eventually, the structured test procedure gets good enough and detailed enough to catch most problems that creep into software developments. Such test procedures are often automated. As of today, the MT OS has to get my vote as the software system with the poorest quality control of any I have ever used. 7) If you use the Mikrotik UNIVERSAL CLIENT so as to be able to allow visitors with "ANY" IP address/gateway/dns setup to log into your Hotspot without making networking IP changes (a very nice feature!), then simple 802.11 REPEATERS such as in the Dlink DWL-900AP+ will not work. You get to pick which operational feature you like best. Note: These same 900AP+ units will work fine as Access Points with Universal Client mode. 8) Mikrotik software versions below 2.7.11 and 2.8beta1 through 2.8beta4 do not support the Mikrotik HotSpot unless you are a PAID license holder. Versions 2.7.8 and up DO support the Mikrotik Hotspot (one simultaneous HotSpot user) with a DEMONSTRATION license. You will have to buy a license to be able to actually use the MT Hotspot, but the price of about $75 is reasonable.

2 of 5

31/7/2006 2:04

Mikrotik Router OS "Gotchas"

http://www.gpsinformation.org/hotspot/mikrotik_router_os_gotchas.html

9) Fixed: Router versions 2.7.20 and 2.8.X can have users with IP addresses OUTSIDE a limited DHCP range (say 10.5.50.2-10.5.50.200) of the hotspot pool and these users can now login to the hotspot. That is, with the above DHCP range, a user with ip address of 10.5.50.225 can now log into this hotspot network. This repair was not noted in the changelog so I do not know when it occurred. 10) Problem Fixed: 11) It is not a good idea to have a router, bridge or client with a fixed IP address inside the DHCP range of another router or a hotspot port. If the device with the fixed IP address is offline then it can happen that the MT hotspot can pass out this IP address to another user and then one or both users will have problems when the fixed IP address device comes back online. Often, for administrative reasons, it is desirable to have APs, bridges, repeaters, and such have fixed IP addresses. 12) Problem fixed. 13) Problem fixed. 14) Problem fixed. 2.8.11 appears to be a stable platform for most applications. 15) Problem fixed. Now, you can routinely upgrade and/or downgrade without losing your software key and having to go back to Mikrotik for a new key. 16) Problem fixed in 2.8.6 and up (maybe earlier). As you move from one software edition to another or between versions, you MAY find that the particular NIC cards identified as ether1, ether2, ether3 and etc. have "changed places" in your computer. This can lead to a fair amount of confusion if you find that the NIC card that WAS ether1 (say the connection to the Internet) is ether2 (the hotspot interface to your external AP) after you upgrade your software. 17) Mikrotik makes "running user interfaces changes" between versions without any notice to users. The changelog generally will not mention that "so and so" command(s) have been changed/moved and the old command(s) will no longer function. This also applies to configuration backup files from an older configuration which, because of changes, can cause the backup file not to be workable when an update version is loaded. Such changes are generally not mentioned in either the version changelog and corresponding changes in the manual may not be mentioned until the next major software or manual edition. This process can make a configuration file that worked fine in one software edition fail to operate at all in when you try and use the same configuration in the next software update in the same software edition. A routing system may be extremely complex and you may not have anything to work with except the backup file which cannot be used in ANY other machine except the one it was generated on! Couple this with the fact that when you encounter this problem, you may have just had a hardware failure. It may be quite awhile before Mikrotik gets back to you with a fresh software key so you can back up to your prior software and you can have quite an emergency situation on your hands. 18) So you decide that to avoid urgent problems, you are going to make up a "clone" system IDENTICAL to your existing Mikrotik system so as to be able to maintain an identical backup system at all times. This is a good idea, but you had better be SURE the hardware is absolutely identical all the way down to the MAC addresses of plug in NIC cards, wireless cards and etc. If the computer is not identical or (for instance) the NIC cards are not identical or from different manufacturers, then likely the systems cannot be made to work with each other's configuration files. I believe Mikrotik must key the configuration files off the hardware MAC addresses and such

3 of 5

31/7/2006 2:04

Mikrotik Router OS "Gotchas"

http://www.gpsinformation.org/hotspot/mikrotik_router_os_gotchas.html

instead of using the logical name (such as ether1). And so, I am not sure but do not believe that two systems can be made to be 100% compatible as to the backup configuration file as MAC addresses will differ between units. I have not been able to discover any scheme to allow me to overcome this problem. One way to handle backup files to allow a user to keep a "hot standby" would be to have two configuration backup elements. The first would backup details of hardware configuration. This part would be "hardware specific" to the particular computer platform used. The second element would backup "non hardware specific" routing tables, hotspot data, and etc. By these means, a user could first set up his hardware configuration, save this and then be able to load his "router specific" and "hotspot specific" and "user specific" features via a backup file which would function on any identically equipped hardware platform but with the hardware specific backup file from the second hardware system. Mikrotik has no such capability so you must manually build and maintain any second system configuration. There are EXPORT/IMPORT features to assist but you must use care as it is not necessarily obvious (or documented) which of these include some hardware specific items. Perhaps Mikrotik could simply provide a script file to export/import and thereby generate a method of "cloning" a configuration to another computer system. If yours is a "business" environment, continuous and reliable service is essential. To insure continuous service, I advise maintaining a duplicate "cold standby" Mikrotik system in critical situations and make any large changes and system upgrades on this system and test them BEFORE swapping out the online system for the system with your changes. THEN, wait a week or so before you put the changes into your formerly online system. I predict you will only implement this recommendation AFTER you have put in some feature that causes problems and your system is offline for half a day or more! Please note that it is NOT possible to simply a) make up a second computer system with identical NIC ports and/or other wireless gear and then b) copy the configuration backup file from the first computer and run it on the second. You have to MANUALLY input and keep up to date the two configurations. It is possible to use the EXPORT feature to assist, but it is still quite labor intensive. You must use GREAT care to be sure you export all of the right items from the first computer (and none of the hardware specific data files) and again use care that all such files are IMPORTED properly into the second (backup) machine. Keeping your two machines up to date is actually easier to do in many cases by just manually inputing changes into both computers. 19) If a user tries to login on (even) a (simple) 2 port hotspot system, there is about zero troubleshooting aid if a failure occurs. If the MT loses the internet link, it will not even put up the login screen but will simply give you a windows error message such as: "could not connect to http://www.yahoo.com" or similar. . (This is supposed to be changing soon in v 2.8 so that the login screen will be capable of display even if the internet connectivity beyond the router is lost.) Needed is a simple connectivity and troubleshooting display screen in winbox to show the existing logical connections and at least give a hint of link defects without having to resort to detailed external testing which can be tedious and time consuming. Many times even detecting that problem exists where multiple incoming/outgoing links are involved can be a problem according to reports from users. 20) Fixed: Version 2.8.8 now does not fail if you disable a hotspot user while he is logged in. I do not know when this was fixed as it is not listed in the changellog. 21) Mikrotik offers a Disk-On-Memory (DOM) Flash Memory module for use as a solid state hard drive. They also use CF memory cards in some RouterBoard units as the "Hard Drive". Long term users have noted that the number of "disk writes" has been in the range of 500,000+ in a moderately busy system after perhaps six months of use. All DOM and CF type memories (that I am aware of) are rated for a maximum number of writes (per sector) of from 300,000 to 1,000,000 before errors occur. In the DOM units (I now have three), this manifests as "shrinking HD space". MT denies that there is a wearout mechanism in Flash Memory, but we know there is.

4 of 5

31/7/2006 2:04

Mikrotik Router OS "Gotchas"

http://www.gpsinformation.org/hotspot/mikrotik_router_os_gotchas.html

Unless MT corrects their software to prevent frequent writes to their Flash Memory devices, I would recommend using a standard hard drive where possible. Note: As of 6/15/04, MT has said that (despite there not being a problem) version 2.8.11 has eliminated non-essential writes to flash memory. The key of course is that they should distribute memory writes over a large number of Flash memory sectors and not just write repeatedly to a small number of sectors and so wear them out. See: http://www.sandisk.com/pdf/oem/WPaperWearLevelv1.0.pdf. This shows that if you properly distribute your writes over a large area and they are quite infrequent, a Flash Memory can last a long time. But if you write at a high rate into just a few locations (as apparently MT is (or was) doing, degradation can occur in months instead of years. I <hope> MT's announced fix in 2.8.11 takes care of this one. 22) Mikrotik keeps telling me that their product is not suited for other than those already expert in ip table setup and router design. They will tell you straightaway that they offer no tutorial assistance in the application areas in which their routers are used.. This does make it difficult to impossible for lots of people to "come up to speed" in a reasonable time with this software system even if you are able to easily get the Hotspot system up and operating with my "cookbook" Hotspot application. "Some Study Required" is an understatement. If you are already a Linux IP Table design guru, you will likely have an easy time of it. If you complain much about software or documentation problems MT has been known to threaten to cut off technical support and then to follow through even for paid up customers with multiple licenses. If you have a software problem with your router, expect that the fix may take months and that is just the way it is. 23) If you order the inexpensive MMCX to N-Bulkhead pigtails from Mikrotik, check the crimps on the N-Bulkhead connector. It has been found that some of the crimps are not secure and allow the shield to twist in the crimp connection and break the center conductor. 24) Someone asked me if I thought the Mikrotik HotSpot system was worth wading through all these problems.. Well.. I have learned a lot and I think that the Mikrotik OS is an extremely well thought out and capable software system. It has practically every routing and bandwidth control feature that I can imagine wishing to have on a Wireless Hotspot system-- and more. Basically the Mikrotik system DOES a lot of things well and is generally reliable once you identify a relatively bug free version and get it running. The out-of-pocket cost is very low. Maybe TOO low. The learning curve for me has been steep mostly because of the lack of overall configuration examples and application tutorials. There are a lot of "Tiny Tutorials" for small parts of the setup, but nowhere did Mikrotik give an overall setup example to allow someone to quickly get a system up and running. (So I made my own.) Considering hours expended, if I knew 12 months ago what I know now, a few thousand dollars for a "solid turnkey system to do the hotspot job" would have been a cheap price to pay. But.. You cannot beat the Mikrotik price and feature set if you are already a router expert and/or have the ability and time to devote to the learning process! If you are planning on using multiple systems, the cost of replication is very low. A final bit of advice: When you get a Mikrotik software version that has the features you need and you have it working, DON'T be tempted to upgrade to the next version unless you absolutely must do so to get a new feature or fix a problem. If you DO decide to upgrade, do the initial upgrade on an offline system and get it up and running and then swap it for your online system. This way, if something goes wrong, you have a quick way to recover. Return to Mikrotik Tutorial

5 of 5

31/7/2006 2:04