Objectives

At the end of this section Participants will be able to:

Color Tools
Section 4
- Recognize the Color Spaces in RGB and HSI
- Outline the Color Hardware available
- Identify the Color Tools available to solve an application
- Demonstrate the use of Color Tools to solve an application

Section 4 | Slide 2

The fourth section of the In-Sight Spreadsheets Advanced training will focus on Color Tools. At the end of this section Participants will be able to:

The purpose of this section is to help you better understand and use the In-Sight vision system’s Color - Recognize the Color Spaces in RGB and HSI
tools. We will clarify and demonstrate the Color tool’s function and some of the settings to solve an - Outline the Color Hardware available
application. - Identify the Color Tools available solve an application
- Demonstrate the use of Color Tools to solve an application

Section 4 | Slide 1 Section 4 | Slide 2

 Color Applications Color Applications

Viewing in monochrome can lose useful information

• Sorting Parts
• Color Recognition and Matching
• Assembly Verification
• Inspection

Section 4 | Slide 3 Section 4 | Slide 4

Color Applications

Sorting Parts
Often, color is the only feature that differentiates parts, such as caps, containers, or
pharmaceutical tablets.

Color Recognition and Matching

In the auto industry, suppliers found that human inspectors have a difficult time in verifying that
side-view mirrors are assembled on the correct car body. Humans often mismatch colors, while
color vision tools can reliably distinguish between them.

Assembly Verification
Identifying that the right part has been used during assembly can be challenging when the parts is
very small, or when identifying marks such as character strings or barcodes are not easily visible.
In these cases, color offers an option for accurately identifying parts.

Inspection
When color is the most unique aspect of the part, color vision can be used to determine if an item
is good or bad. For example, verifying that a red LED is present and bright enough or was properly
assembled.

Section 4 | Slide 3 Section 4 | Slide 4

 Pixels – Color Cameras Pixels – Comparative RGB / HSI Values
In a Color Camera each color is composed of three separate
color components:

0
255

RGB: HSI:

These components may be represented as combinations of: Red 255, 0, 0 Red 0, 255, 85
- Red, Green, and Blue (RGB)
- Hue, Saturation, and Intensity (HSI) Green 0, 255, 0 Green 85, 255, 85

Each component of a pixel is converted to a value from 0 to 255 Blue 0, 0, 255 Blue 170, 255, 85
Section 4 | Slide 5 Section 4 | Slide 6

Let’s start with the image chip on the camera. Cameras use imagers comprised of pixels to determine
pixel intensity with values between 0-255.
For In-Sight cameras, we use a RGB Bayer filter. This is a single chip with alternating Red, Green and
Blue pixels. The color image is therefore an approximation of the actual color.
As you will note, both color spaces contain the values 0 - 255 in their perspective spaces. In the HSI color
space, it is only the hue that determines the actual color as it lies along the color wheel. Saturation
denotes how vivid the color is, and Intensity is how dark or light the color is.
Determining which color space is best for your application may be dictated by the customer or may
require testing in both color spaces to see which one will result in determining the correct outcome. This is
often the case when inspecting objects that are very close in color.

Section 4 | Slide 5 Section 4 | Slide 6

 Color and Pixels: Bayer Pattern In-Sight Color Models

CCD “bins”: 7000 Family (7010C, 7200C, 7400C, and 7402C)

• Half detect green (G) - Great entry level product
• A quarter detect red (R) - Lighting and optics from 7000 family
• A quarter detect blue (B) - 800 x 600 Resolution (or 1280 x 1024)

5000 Family (5100C and 5400C)

- 640 x 480 Resolution
These values are interpolated into RGB values for pixels - Choice of processing speeds

Implication: Measurements on a color camera have a lower accuracy Micro Family (1100C, 1400C and 1403C)
than on a greyscale camera with the same number of pixels
- Small package
- Offers higher resolution (1600x1200)

Section 4 | Slide 7 Section 4 | Slide 8

A common technique to give color sensitivity to a black & white image sensor is the application of a color Each of the In-Sight families offer a color model. Depending on your size and resolution requirements,
mosaic filter on top of the sensor. In most cases a three-color red-green-blue (RGB) pattern is used. one may satisfy your application better than the others. All but the 7010C model can be purchased with
The In-Sight color sensors use a Bayer pattern as their RGB color mosaic filter pattern. With the Bayer PatMax.
pattern, each pixel is covered by a specific color filter, in a specific pattern. Half of the total number of
pixels are green (G), while a quarter of the total number is assigned to both red (R) and blue (B). Legacy 5000 units (with the old memory) were slower in color applications because they converted the
color image at the time that the tool would run as opposed to acquisition. The newer units (only the newer
Each color pixel is composed of three separate color components: red, green and blue. The missing units are supported as of 4.8 software) and beyond performs the color conversion at the time of
colors, for each pixel, are interpolated using the surrounding pixels at each location. For example, if a acquisition.
pixel is filtered for green, the value for the green component is known, but the values of the red and the
blue components must be calculated from the average value of surrounding red-filtered and blue-filtered
pixels. Through software interpolation, each pixel is assigned a value from 0 to 255 for the two unknown
color components.

See In-Sight Help for details and examples on how interpolation is done.

Section 4 | Slide 7 Section 4 | Slide 8

 Color Tools Color Tools
• All greyscale tools are available in a color camera.
- Color image is converted to greyscale for these tools.
• In addition, color cameras have color tools: MatchColor

- 24 bit resolution
Will cover in this section - Computes the average RGB or HSI color within
the specified region
- Compares it to the previously trained colors and
computes the difference to each
- Selects the one with the lowest difference as the
identified color

Section 4 | Slide 9 Section 4 | Slide 10

A color camera includes all the greyscale tools. If a greyscale tool is used in a job, the color image is first The Color tool will average all the pixels in the selected Region of Interest and compares this color to the
converted to a greyscale image, and the greyscale image is used by the tool. trained colors in the library and computes the difference between them and selects the one with the
lowest difference to match with. The tool utilizes 24 bit resolution to compare colors.
A color camera has additional tools that operate on the color image itself:

The tools listed in the Color category identify or verify a part based on its color or collection of colors. We
will cover these tools starting in the next slide.

ExtractColorHistogram is similar to ExtractHistogram, but instead of using a single grayscale histogram

it determines three histograms, one for R, one for G, and one for B (or HSI).

ColorToGreyscaleFilter and GreyscaleDistance convert the color image to greyscale, but with a choice
of an number of ways different from the default conversion.

ScaleColorImage resamples the pixels in an input image or region of interest (ROI) to produce a
reduced, enlarged, unrotated or unbent version of a color image.

WhiteBalance compensates for lighting that is not full-spectrum white.

Section 4 | Slide 9 Section 4 | Slide 10

 Color Tool – TrainMatchColor Library TrainMatchColor Library Training
Train colors by selecting:
Area on an image
– Multiple shapes available:
• Point, Rectangle, Circle,
Annulus, Polygon, Freehand
– Color Swatches (square box)
Train Colors to be resemble trained color
stored in a library
with the job

– Color Tolerance adjustment

Section 4 | Slide 11 Section 4 | Slide 12

Train colors that could be used for matching or identification within the library. The RGB and HSI values Multiple shapes are available to train a color. The selection of the shape should be chosen to allow
listed will be the average color value of the enclosed area. The image tab allows user to get a live image training of just the color. This should be a single color. Keep in mind that training additional colors in the
or retrieve and image from file. same region will be part of the average color and may reduce the accuracy of finding the correct part.

Edit the Name field with the correct color name. This name will be used in the MatchColor Tool.
Color Tolerance allows for a certain percent darker or lighter of the same color to be identified correctly.

Color Tolerance is for intensity (I).

Section 4 | Slide 11 Section 4 | Slide 12

 External Training Tab Color Tool – MatchColor

Allows for dynamic training of colors

- An Edit region of some type

should be referenced MatchColor tool references the trained
- The event to perform the training colors
should be referenced. This could
be a button on the spreadsheet
- The Color Tolerance limits can
reference any values from the
application Selected colors from library used with
this tool for matching / identification

Section 4 | Slide 13 Section 4 | Slide 14

Make sure your region is placed correctly before the event to train the color is initiated.
The Darker Limit and/or Lighter Limit can be dragged onto the spreadsheet to allow for dynamic changes
from the Custom View. These trained colors will be added to the library. When you use an RGB space,
the limits are based on proportions.
When you used an HSI space, the limits are for intensity (I).
The library with the trained colors is referenced from within the corresponding tool. In the Colors Tab, user
selectable check boxes next to the trained color enables the end user to perform color matching or color
identification. Using a subset of the colors will increase speed and potentially decrease confusion.
Any colors trained after the Match Color Library is added to the MatchColor tool will need to be manually
selected in the Colors tab.
NOTE: The Color tools Region of Interest should be drawn in the area to be inspected and not include any
other colors within the region that would then average those pixel intensity for the final result for a color
Match or Identify.

Section 4 | Slide 13 Section 4 | Slide 14

 Color Tool – MatchColor Settings MatchColor Applications

Reference image Advantages

Fixture for specific placement of - 24 bit resolution allows very
Region Of Interest (ROI) precise color definitions
Reference a composite region tool - Can be configured to allow for
Reference cell with color Library minor differences in the color
Color Space used (RGB or HSI) intensity
Sensor Graphics

Section 4 | Slide 15 Section 4 | Slide 16

Results will include Score, Color Distance, and Confidence Score. The tool uses 24 bit resolution to define the color space for a precise match to the trained color in the
library.
Score is 1 – (color distance/(SQRT(3)*255)). The closer the runtime color is to the trained color, the higher Though you are training on a single color, the Color Tolerance slider can allow for variations in the
the score. intensity (hue and saturation must remain the same).

Color distance for RGB and HSI is the Euclidean distance between the trained colors and the runtime
color. The smaller the number, the more similar the color.

Confidence Score represents how far the best score is from the second best score. The higher the
Confidence Score, the lower the chance of confusion with the next closest color.

Additional getters such as GetHue(), GetIntensity(),GetRed(), GetBlue() etc. are available through the
Vision Data Access folder under Color.

Section 4 | Slide 15 Section 4 | Slide 16

 Color Tools Color Tool – TrainExtractColor Library

ExtractColor
– Trains groups of colors in a Model to be stored in a library with
the job
- Works on individual pixels within the specified region
– Allows for great degree of tolerance control through Hue and
- Compares each pixel to the previously trained color groups.
Highlights
- Selects and counts the matched pixels for each color group
- Selects the one color group with the highest number of
matched pixels as the identified color
- 16 bit resolution

Section 4 | Slide 17 Section 4 | Slide 18

The Color model tool will consider each pixel in the selected Region of Interest instead of averaging all The RGB color noted next to the color name is the peak color as opposed to the average color that the
the found pixels as in the Color tool. The Color Model tool will select the group of pixels with the highest MatchColor tool uses. Peak color means the most prevalent color (highest pixel count) in the region that
number of matched pixels to the learned pixels in the Model library. is being trained.

This tool uses 16 bit resolution – 5 bits Red and Blue, 6 bits for Green. Green is more representative of
the intensity as compared to both Red and Blue.

Section 4 | Slide 17 Section 4 | Slide 18

 TrainExtractColor Library Training External Training Tab
Build Color Models Allows for dynamic training of color models
- Series of color groups
- Train Region of Interest - An Edit region of some type
for should be referenced
- Dominant Color - The event to perform the training
- Multiple Colors should be referenced. This could
- Add and subtract colors be a button on the spreadsheet
- Marks image with - Each of the settings within the
inverted color to show Training tab is available
inclusion

Section 4 | Slide 19 Section 4 | Slide 20

Training a Model to be used for Identification or Matching consists of adding all the colors of the product. Make sure your region is placed correctly before the event to train the color is initiated. All the settings on
Subtracting color(s) can assist with identification of a different product or the back ground that might be this tab can be dragged onto the spreadsheet to allow for dynamic changes from the Custom View. These
incorrectly included in this Model. trained colors will be added to the library.

When using a model with more than one color, each of the separate colors can have different settings for
the Color Tolerance.

Section 4 | Slide 19 Section 4 | Slide 20

 Dominant vs. Multiple Colors Color Tolerance – Darker and Lighter

Dominant Color Ability to include lighter and darker shades of the color
beyond its peak intensity

Multiple Color

Section 4 | Slide 21 Section 4 | Slide 22

Dominant color will choose the peak color that exists in the region. Multiple color will break the colors into
multiple ones that combine together to create the final color model.
When using Multiple Color, choose a training region to incorporate your whole part. Using Multiple Color
allows you to quickly train a complex color object.
By adjusting the values for Darker and Lighter, you are increasing the intensity tolerance allowed on the
model. The hue and saturation would stay the same. It is only looking for varying intensities of color much
like the MatchColor Tool.
For example above, Highlights is set to 1.0. The Hue is set to the default 2.0.

Section 4 | Slide 21 Section 4 | Slide 22

 Color Tolerance – Highlights Color Tolerance – Hues

Includes the pixels that contain the surface Adds more tolerance to changes in color
reflection of the part

Section 4 | Slide 23 Section 4 | Slide 24

By increasing Highlights, you are increasing the saturation allowed on the model. The hue and the
intensity would stay the same. It is including colors that are near the saturation of the trained color.
For example above, Intensity (Darker/Lighter) is set to .4 – 1.7. The Hue is set to 2.3.
When Hue is set to 0, it will only show the pixels that are at the peak value. As the Hue value in
increased, colors that are near that peak value with regards to hue will now be incorporated into the Color
Model. It shows the pixels that are receiving the same amount of light.
For example above, Highlights and Intensity (Darker/Lighter) is set to 1.0.

Section 4 | Slide 23 Section 4 | Slide 24

 Image Marking – Reviewing the Trained Colors Color Tool – ExtractColor

Mark Results for the Entire Model – ExtractColor tool

shows all colors trained in the model at once.
references the trained colors

Mark Results up to Selected Color –

shows all colors trained on list up to and Selected colors from library used with
including the selected one. this tool for matching / identification

Mark Selected Colors Only – display only

the markings of the selected color.

Section 4 | Slide 25 Section 4 | Slide 26

The Image Marking selections allows the designer to review the colors that are contained within a model. The library with the trained Extract colors is referenced from within the corresponding tool, in this case
Keep in mind that the colors do not need to be continuous in the sense that it is the same color. You can the Color Extract tool. In the Color Tab, user selectable check boxes next to the trained color enables end
save multiple colors as part of the same model. user to perform color matching or color identification.

The Image Marking selections will show markings on all the colors contained in the model, all the colors
on the list up to the selected color, or just the selected color in the list. This can be used to illustrate
common colors between parts or how much one color contributes to the model.
Any colors trained after the Extract Color Library is added to the ExtractColor tool will need to be manually
selected in the Color Models tab.
NOTE: The Color tools Region of Interest should be drawn in the area to be inspected and include other
colors which will be ignored if they have not been Trained in the Extract Color Library.

Section 4 | Slide 25 Section 4 | Slide 26

 Color Tool – ExtractColor Settings Color Tool – ExtractColor Output Image

Reference image
Fixture for specific placement of
Region Of Interest (ROI)
Reference a composite region
tool
Reference a cell with
Marked Image
ExtractColor Count pixels that
match a Model
Output image to display

Binary Image

Color Image
Section 4 | Slide 27 Section 4 | Slide 28

Count Pixels is selected by default and will return the number of pixels that the runtime image has in Marked Image
common with each of the trained models. The model with the highest pixel count is ranked first. In this output image, any pixel that matches an active Color Model is set to the marked color, which
Because this is a color model and can be a composite of multiple colors, there are not getters to hue, is the color opposite the peak of the first color present in the Color Model. Requires the largest
saturation or intensity available. amount of processing time.

Binary Image
Specifies that a binary, greyscale image will be output. In this output image, any pixel that matches
an active Color Model is set to White (255), while all other pixels are set to Black (0). Fastest of the
image outputs and least impact to memory.

Color Image
Specifies that a color image will be output. In this output image, any pixel that matches an active
Color Model will retain its original value, while all other pixels are set to 0,0,0 (RGB). Output image
is 4 times size of Binary Image as it contains both RGB and Intensity data.

Section 4 | Slide 27 Section 4 | Slide 28

 ExtractColor Applications So what’s the difference?

MatchColor ExtractColor
Products made of multiple, individual colors
Products with a broad range of color - Looks for one color in the - Looks for multiple color
RGB or HSI space groups, add/subtract colors

- Region of Interest must be on - Region of Interest may be on

a single color since it multiple colors
computes the average color
of all pixels - Returns:
- Name of trained color
- Returns the name of trained group with highest pixel
color with lowest difference count
between colors and highest - Filtered image of selected
score type

Section 4 | Slide 29 Section 4 | Slide 30

Here are some examples of applications that have several different colors to be learned and another that The ExtractColor tool can be used as the input Image to a tool like Blob for further processing. The type of
has a broad range of a single color such as the Orange plate that might be considered two colors with the image being generated is dependent on the Output Image selection of the ExtractColor tool.
color orange approaching the color yellow.

Section 4 | Slide 29 Section 4 | Slide 30

 Summary Lab Exercise

• A color camera includes the standard greyscale tools,

but has additional color tools

• Colors can be represented in both RGB and HSI space

• MatchColor tool identifies (or verifies) a single color

• ExtractColor tool can identify (or verify) models that

may contain multiple colors

• Both tools return the matching trained color

• ExtractColorModel will return a Pixel Count as well as a

filtered image of the matching color group
Section 4 | Slide 31 Section 4 | Slide 32

In this section we covered the following topics: Complete:

Lab Exercise 4.1 – Color
- A color camera includes the standard greyscale tools, but has additional color tools Lab Exercise 4.2 – Complex Color Identification
- Colors can be represented in both RGB and HSI space
- MatchColor tool identifies (or verifies) a single color
- ExtractColor tool can identify (or verify) models that may contain multiple colors
- Both tools return the matching trained color
- ExtractColorModel will return a Pixel Count as well as a filtered image of the matching color
group

Section 4 | Slide 31 Section 4 | Slide 32