Vision Targets

This document describes the visual properties of the material used to mark the field scoring targets and techniques for processing images to make measurements that may be useful for playing the game.

Retro-reflective Tape:
This material should be relatively familiar as it is often used to enhance nighttime visibility of road signs, traffic personnel, and pedestrians. The way it behaves is often confused with reflective materials, so a brief comparison is provided. Reflective materials bounce the majority of light back at a supplementary angle. That is, unce the blue and red angles shown below-left sum to 180 degrees. This is equivalent to saying that the light reflects about the reflective surfaces normal the green line.

Retro-reflective materials, however, return the majority of light back at the entry , angle. Initially, this doesnt seem like a useful safety property, but when the light and eye are near one another,, as shown below the reflected light enters the eye, and the below, material shines brightly. Note that t shine is largely independent of the the he orientation of the material, and apparent color is a combination of light source and surface coating on the material.

Demonstration:
To further study the effect, place the retro retro-reflective material on a wall or vertical surface, stand 10-20 feet away, and shine a small flashlight towards the material. Start with the light held at your belly button, and raise it until it is even with your nose. As the light gets near your eyes, the intensity of the returned light will increase rapidly. Repeat rapidly at various locations in the room. The tape shine should work over a wide range of viewing angles. Experiment with different light sources. For example, a red bicycle light light is sufficiently bright, and will demonstrate that the color of the light determines the ght, color of the shine. Place several people with different light at different locations in the lights room. Note that the light returned to an individual viewer is determined by the light coming from the individuals source Flash the light source. If the light source is not source. t overly bright, the primary difference will be the appearance and disappearance of the , shine. For fun, identify retro-reflective materials already present around you, on you clothing, backpacks, shoes, etc.

Application:
The retro-reflective tape will be applied to several surfaces on the field in order to mark e them and aid with robot navigation Each robot may use a light source and any navigation. combination of color and flashing to help identify marked areas. Note that most flashlights act as spotlights, using lenses or shaped mirrors to project bright light onto a region or spot much smaller than what the camera views. If this is the type of light source used, only the objects that are visible to the camera and also illuminated by the light source will shine. It may be possible to move the flashlight around to locate objects that will shine. An alternate approach would be to locate or design the light source so it illuminates the cameras entire viewing area, instead of just a small spot. This floodlight approach will allow all marked areas visible to the camera to shine at the same time. A very useful type of light source to research is the ring flash, or ring light. It places the light source very close to the ca camera lens/sensor and provides very even lighting. Because of their size, LEDs are particularly useful for constructing this type of device. The low-tech ring light shown to the tech right was constructed using battery battery-powered LED Christmas lights mounted at various angles by poking holes into a rigid foam ring. The foam is fragile, but otherwise, the light functions well, flooding the entire scene and shining all markers simultaneously. As a bonus, Christmas lights also come in a wide variety of colors.

Image Processing:
Calibration: If the color of the light shine is being used to identify the marker, it is important to control the camera settings that can affect the colors within an image. The most important factor is the camera white balance. This controls how the camera blends the component colors of the sensor in order to produce an image that matches the color processing of the human brain. The camera has five or six named presets, an auto setting that constantly adapts the output colors, and a hold setting -- for custom calibration. The easiest approach is to select from the named presets, looking for a setting that minimizes the saturation of other objects that might be confused for the light shine. To custom-calibrate the white balance, place a known subject in front of the camera, set to auto white balance, wait for the camera to update its filters (ten seconds or so), and switch the white balance to hold. The brightness or exposure of the image also has an impact on the colors being reported. The issue is that as overall brightness increases, color saturation will start to drop. Lets look at an example to see how this occurs. A saturated red object placed in front of the camera will return an RGB measurement high in red and low in the other two e.g. (220, 20, 30). As overall white lighting increases, the RGB value increases to (240, 40, 50), then (255, 80, 90), then (255, 120, 130), and then (255, 160, 170). Once the red component is maximized, additional light can only increase the blue and green, and acts to dilute the measured color and lower the saturation. If the point is to identify the red object, it is useful to adjust the exposure to avoid diluting your principle color. The desired image will often look somewhat dark except for the colored shine. There are two approaches to correct for bright lighting. One is to allow the camera to compute the exposure settings automatically, based on its sensors, and then adjust the cameras brightness setting to a small number to lower the exposure time. The brightness setting acts similar to the exposure compensation setting on typical cameras. The other approach is to calibrate the camera to use a custom exposure setting change the exposure setting to auto, expose the camera to bright lights so that it computes a short exposure, and then change the exposure setting to hold. When determining the approach to use, be sure to consider the effect of the black curtain often found on the scoring side of the typical FIRST field. It will cause a camera using auto to overexpose when it faces in that direction. It may be useful to include a dark wall and brightly lit foreground objects in your vision testing scenarios. Note that the light shine is almost entirely based on the light source carried by the robots. Changes to ambient lighting such as moving from a room with fluorescent lights to one with incandescent lights should have very little impact on the colors visible in the shine. Once light source and white balance are selected, exposure adjustments to maintain saturation should be sufficient when changing locations.

Algorithms:
The image below resembles how an ideal row of markers would look after you use color thresholding or image subtraction to remove all background information. The dark gray columns were added to assist with visualizing where the markers are located. Notice that the circles and rectangles do not line up when the robot is not directly in front of the column. This parallax effect occurs because the circles are closer to the camera than the rectangles.

The next image shows this effect as the camera is lowered below the row of circular markers.

Initially, this parallax effect may seem unfortunate. However with some processing, and using information we already know about the markers, it can be used to your advantage. It can be used to reconstruct 3D information from a 2D image. Here is how. Remember that the camera flattens all objects in the scene into a single plane. Looking at the same portion of the field from above, the actual column markers are shown in black, and in red, we see how the circular markers on the end of the pegs will project away from the camera and onto the backplane. Notice that the areas highlighted in yellow are similar triangles. This will allow us to use trigonometry and the known measurements of the field to compute additional values.

As an example, lets move the camera to an unknown location on the field. The 320x240 image below is taken on a practice setup using a red ring light. It is an unknown distance and location from the target. The green lines mark a few important locations such as the center of the circle and the vertical center of the rectangular strips. The horizontal distance between these lines is labeled as d. Also, the width and height of the top strip are labeled w and h. In field units, we know that w is 1 and h is 4. In pixels, w is about 11 or 12 pixels and h is about 47 pixels. This coincides well with the expected ratio and can be used to calculate the distance from the camera to the strip.

Distance Calculations:
Knowing the physical dimensions of a known object in a photo, and knowing the lens and sensor information, you can calculate the distance from camera to the object. In this case, a 1 object is about 12 pixels. The width of the image is 320 pixels, meaning that in the plane of the vertical strip, the image is 320/12 or 26.6 inches wide. From the Axis 206 datasheet, the horizontal angle of view is given as 54 degrees this is measured from edge to edge, so 27 degrees gives us the right triangle shown below. The value of distance can be found from 13.33/tan(27deg), or 26.2 inches. Note that the other Axis camera, the M1011 datasheet lists the angle of view as 47 degrees, and obviously a different lens will change the angle of view. Note that you can compute the distance in many other ways, and it may be useful to photograph a ruler from various known distances and practice a bit using different methods.

To continue, the distance labeled d measures 39 pixels, giving 39/12 or 3.25 inches. From the symmetric triangles shown below, we can determine that the camera is 26.2 minus 14 or 12.2 from the circular marker, and that 3.25/14 = x/12.2, which means that x is equal to 2.8.

So, from this image, and knowing a few field measurements, we know that the robot would need to move 2.8 to the left and 12.2 forward in order for the camera to touch the circular marker. To extend this method, you can measure from the center of the circular marker to the center point between the two vertical markers and calculate the amount you would need to raise or lower the camera in order to touch the marker. Note that this approach assumes that the camera is perfectly aligned on the field perpendicular to the markers, which is not quite true for this photo, but the results are pretty accurate anyway. Also, note that optical distortion was ignored. This will have a more noticeable impact near the corners of the image, but its effect will generally be small. If you decide to develop a more general solution, remember that all objects in the image are projected into a single plane -- a plane that is parallel to the camera sensor. You can then use known dimensions of the markers to compute the camera orientation and location relative to those markers and calculate the remaining 3D measurements. Also notice that it is possible to use the circle and vertical markers purely for qualitative feedback. An algorithm that only outputs simple (left-a-little, straight-ahead, right-alittle commands) is quite easy to build and may be sufficient for driving the robot to a specific marker.

Marker Clusters:
Viewing the markers from a greater distance introduces a new challenge. The 640x480 image shown below was produced using a ring light and color threshold. Notice that Not individual shapes are too small and JPEG compression noise is too strong to be able to make accurate measurements on individual particles. Also note how the particles can blend together at certain viewing angles. All is not lost, however. The particles are still ther very distinct and relatively easy to identify as belonging to a particular scoring grid element such as (left, top) or (center, center). Even if JPEG noise causes an individual particle to break up or if they merge, particles still help to identify the column and row identify locations in a statistical fashion. There are many approaches to this problem, and a large set of tradeoffs to consider. To T keep things simple, this paper will review a few approaches that may be effective. his

Statistical Method: Given the centers of each particle above, notice how the histogram of the X togram positions does a good job of identifying the marker columns. Absolute Y positions are more challenging, but within each column, distances are pretty well grouped.

Clustering algorithms such as K means may also be effective at identifying the grid K-means locations as shown below.

Finally, additional information such as estimated distance to target, starting location, and position information from previous images can be used to enhance the algorithms success by speeding up where processing begins, filtering results, and validating assumptions.