Image Component Library (ICL)
|
Class for fast LUT-based color segmentation. More...
#include <ColorSegmentationOp.h>
Public Member Functions | |
ColorSegmentationOp (icl8u c0shift=2, icl8u c1shift=2, icl8u c2shift=2, core::format fmt=core::formatYUV) | |
Creates a new instance of this class with given segmentation parameters. More... | |
~ColorSegmentationOp () | |
Destructor. More... | |
virtual void | apply (const core::ImgBase *src, core::ImgBase **dst) |
main apply function More... | |
icl8u | classifyPixel (icl8u r, icl8u g, icl8u b) |
classifies a pixel in given rgb format More... | |
void | clearLUT (icl8u value=0) |
clears the whole segmentation LUT More... | |
void | setSegmentationShifts (icl8u c0shift, icl8u c1shift, icl8u c2shift) |
sets the segmentation shifts More... | |
void | setSegmentationFormat (core::format fmt) |
sets the internally used segmentation format More... | |
const icl8u * | getSegmentationShifts () const |
returns the pointer to the 3 internally used segmentation shifts More... | |
core::format | getSegmentationFormat () const |
returns the current internally used segmentation format More... | |
const core::Img8u & | getSegmentationPreview () |
returns in internally managed image of the segmentation result More... | |
void | lutEntry (icl8u a, icl8u b, icl8u c, icl8u rA, icl8u rB, icl8u rC, icl8u value) |
given a,b and c in segFormat, this function fills the LUT within a sub-volume with given radii More... | |
void | lutEntry (core::format fmt, int a, int b, int c, int rA, int rB, int rC, icl8u value) |
given a,b and c in format fmt, this function fills the LUT within a sub-volume with given radii More... | |
void | load (const std::string &filename) |
loads the segmentation LUT only (no other parameters) More... | |
void | save (const std::string &filename) |
saves the segmentation LUT only (no other parameters) More... | |
const core::Img8u & | getLUTPreview (int xDim, int yDim, icl8u zValue) |
this also creates a preview for the current segmentation LUT More... | |
const core::Img8u & | getColoredLUTPreview (int xDim, int yDim, icl8u zValue) |
this also creates a colored preview for the current segmentation LUT More... | |
const std::vector< core::Color > & | getClassMeanColors () |
returns a mean color for all used class labels (in rgb format) More... | |
const icl8u * | getLUT () const |
returs the internal lut data More... | |
icl8u * | getLUT () |
returns the internal lut data More... | |
void | getLUTDims (int &w, int &h, int &t) const |
returns the lut-sizes More... | |
virtual void | apply (const core::ImgBase *operand1, core::ImgBase **dst)=0 |
Imported apply from parent UnaryOp class. More... | |
virtual const core::ImgBase * | apply (const core::ImgBase *src) |
Imported apply from parent UnaryOp class. More... | |
Public Member Functions inherited from icl::filter::UnaryOp | |
UnaryOp () | |
Explicit empty constructor. More... | |
UnaryOp (const UnaryOp &other) | |
Explicit copy constructor. More... | |
UnaryOp & | operator= (const UnaryOp &other) |
Explicit declaration of the assignment operator. More... | |
virtual | ~UnaryOp () |
Destructor. More... | |
virtual ICL_DEPRECATED void | applyMT (const core::ImgBase *operand1, core::ImgBase **dst, unsigned int nThreads) |
apply function for multithreaded filtering (currently even slower than using one thread) More... | |
virtual const core::ImgBase * | apply (const core::ImgBase *src) |
applys the filter usign an internal buffer as output image More... | |
void | operator() (const core::ImgBase *src, core::ImgBase **dst) |
function operator (alternative for apply(src,dst) More... | |
const core::ImgBase * | operator() (const core::ImgBase *src) |
function operator for the implicit destination apply(src) call More... | |
const core::ImgBase & | operator() (const core::ImgBase &src) |
reference based function operator More... | |
void | setClipToROI (bool bClipToROI) |
sets if the image should be clip to ROI or not More... | |
void | setCheckOnly (bool bCheckOnly) |
sets if the destination image should be adapted to the source, or if it is only checked if it can be adapted. More... | |
bool | getClipToROI () const |
returns the ClipToROI status More... | |
bool | getCheckOnly () const |
returns the CheckOnly status More... | |
virtual void | setPropertyValue (const std::string &propertyName, const utils::Any &value) |
sets value of a property (always call call_callbacks(propertyName) or Configurable::setPropertyValue) More... | |
Public Member Functions inherited from icl::utils::Configurable | |
virtual | ~Configurable () |
virtual destructor More... | |
Configurable (const Configurable &other) | |
Copy constructor. More... | |
Configurable & | operator= (const Configurable &other) |
Assignment operator. More... | |
void | setConfigurableID (const std::string &ID) |
sets the ID of this configurable More... | |
const std::string & | getConfigurableID () const |
returns the configurables static ID More... | |
bool | isOrderedFlagSet () const |
returns whether the ordered flag is set More... | |
void | deactivateProperty (const std::string &pattern) |
adds an additional deativation pattern More... | |
void | deleteDeactivationPattern (const std::string &pattern) |
removed a formerly added deactivation pattern More... | |
std::vector< std::string > | getPropertyListWithoutDeactivated () const |
this returns a filtered list of properties (using all filters added by deactivateProperty) More... | |
virtual void | adaptProperty (const std::string &name, const std::string &newType, const std::string &newInfo, const std::string &newToolTip) |
this function can be used to adapt a specific property afterwards More... | |
void | registerCallback (const Callback &cb) |
add a callback for changed properties More... | |
void | removedCallback (const Callback &cb) |
removes a callback that was registered before More... | |
void | syncChangesTo (Configurable *others, int num=1) |
this can be used to let this instance also apply property changes to others More... | |
virtual std::vector< std::string > | getPropertyList () const |
returns a list of All properties, that can be set using setProperty More... | |
virtual bool | supportsProperty (const std::string &propertyName) const |
base implementation for property check (seaches in the property list) More... | |
virtual void | saveProperties (const std::string &filename, const std::vector< std::string > &propertiesToSkip=EMPTY_VEC) const |
writes all available properties into a file More... | |
virtual void | loadProperties (const std::string &filename, const std::vector< std::string > &propertiesToSkip=EMPTY_VEC) |
reads a camera config file from disc More... | |
virtual std::string | getPropertyType (const std::string &propertyName) const |
get type of property More... | |
virtual std::string | getPropertyInfo (const std::string &propertyName) const |
get information of a properties valid values More... | |
virtual Any | getPropertyValue (const std::string &propertyName) const |
returns the current value of a property or a parameter More... | |
virtual std::string | getPropertyToolTip (const std::string &propertyName) const |
returns the tooltip description for a given property More... | |
virtual int | getPropertyVolatileness (const std::string &propertyName) const |
Returns whether this property may be changed internally. More... | |
Private Attributes | |
core::format | m_segFormat |
format, that is used for internal segmentation More... | |
core::Img8u | m_inputBuffer |
internal image in depth8u and segmentation format More... | |
core::Img8u | m_outputBuffer |
internal buffer holding the output image More... | |
core::Img8u | m_segPreview |
internal buffer for providing a preview of the current segmentation More... | |
core::Img8u | m_lastDst |
last used destination image More... | |
icl8u | m_bitShifts [3] |
bit shifts for all 8-Bit channels More... | |
LUT3D * | m_lut |
color classification lookup table More... | |
Additional Inherited Members | |
Public Types inherited from icl::utils::Configurable | |
typedef Function< void, const Property & > | Callback |
Function type for changed properties. More... | |
Static Public Member Functions inherited from icl::filter::UnaryOp | |
static UnaryOp * | fromString (const std::string &definition) |
Creates a UnaryOp instance from given string definition. More... | |
static std::string | getFromStringSyntax (const std::string &opSpecifier) |
gives a string syntax description for given opSpecifier More... | |
static std::vector< std::string > | listFromStringOps () |
returns a list of all supported OP_SPEC values for the fromString function More... | |
static void | applyFromString (const std::string &definition, const core::ImgBase *src, core::ImgBase **dst) |
creates, applies and releases a UnaryOp defined by given definition string More... | |
Static Public Member Functions inherited from icl::utils::Configurable | |
static std::string | create_default_ID (const std::string &prefix) |
this function can be used in subclasses to create a default ID More... | |
static Configurable * | get (const std::string &id) |
returns configurable by given ID More... | |
static void | register_configurable_type (const std::string &classname, Function< Configurable * > creator) |
registers a configurable type More... | |
static std::vector< std::string > | get_registered_configurables () |
returns a list of all registered configurable classnames More... | |
static Configurable * | create_configurable (const std::string &classname) |
creates a configurable by given name More... | |
Static Public Attributes inherited from icl::utils::Configurable | |
static const std::vector< std::string > | EMPTY_VEC |
used as shortcut – just an empty vector of std::strings More... | |
Protected Member Functions inherited from icl::filter::UnaryOp | |
bool | prepare (core::ImgBase **ppoDst, core::depth eDepth, const utils::Size &imgSize, core::format eFormat, int nChannels, const utils::Rect &roi, utils::Time timestamp=utils::Time::null) |
virtual bool | prepare (core::ImgBase **ppoDst, const core::ImgBase *poSrc) |
check+adapt destination image to properties of given source image More... | |
virtual bool | prepare (core::ImgBase **ppoDst, const core::ImgBase *poSrc, core::depth eDepth) |
Protected Member Functions inherited from icl::utils::Configurable | |
void | addProperty (const std::string &name, const std::string &type, const std::string &info, const Any &value=Any(), const int volatileness=0, const std::string &tooltip=std::string()) |
This can be used by derived classes to store supported properties in the internal list. More... | |
void | addChildConfigurable (Configurable *configurable, const std::string &childPrefix="") |
This adds another configurable as child. More... | |
void | removeChildConfigurable (Configurable *configurable) |
removes the given child configurable More... | |
Property & | prop (const std::string &propertyName) |
this CAN be used e.g. to store a property value in internal property-list More... | |
const Property & | prop (const std::string &propertyName) const |
this CAN be used e.g. to store a property value in internal property-list More... | |
Configurable (const std::string &ID="", bool ordered=true) | |
create this configurable with given ID More... | |
void | call_callbacks (const std::string &propertyName, const Configurable *caller) const |
calls all registered callbacks More... | |
Protected Attributes inherited from icl::filter::UnaryOp | |
utils::MultiThreader * | m_poMT |
Protected Attributes inherited from icl::utils::Configurable | |
std::vector< Callback > | callbacks |
internally managed list of callbacks More... | |
Class for fast LUT-based color segmentation.
Color segmentation is a very common issue in computer vision applications. The ColorSegmentationOp class implements a common and very efficient LUT based segmentation algorithm. It can be used to apply any possible segmentation where the classification of one pixel is only influenced by that pixels color value
Please consider an input color image I. In a first step, I is converted into depth8u and the specified segmentation-color-format (this can be set in the constructor or with the corresponding setter-function ColorSegmentationOp::setSegmentationFormat). This conversion is skipped if the given image parameters are already correct. Now, every pixel pI = (A,B,C) (e.g. if the segmentation format is formatRGB, pI=(R,G,B)) is classified by applying a lookup transformation on it. Basically, we use a lookup table of the whole segmentation color space for this. This implies that size of the lookup table becomes 256*256*256. In order to reduce the amount of cache misses when picking elements of this 16MB LUT, it is possible to adjust the number of bits that are used for each color channel. In the example above, 8 bits (256 values) are used. If the number of bits are shifted (using the constructors channel-shift parameters or the setter function ColorSegmentationOp::setSegmentationShifts), the size of the lookup table becomes smaller and therefore, the lookup operation becomes faster (see Benchmarks:). Another advantage of using the channel-shifts in order to reduce the bits used to represent a certain image channel is that is also provides a better generalization. Consider the following example (see Example (Image Convolution))
Segmentation format: YUV, Shifts = (8,0,0).
In this case, the Y (brightness) information is not used for segmentation at all. Therefore the pixel classification does not depend on the pixels brightness, which is very useful for color-segmentation. The size of the used lookup table becomes 256*256 which is 65KB. As the noise level of common cameras is still quite high, it might also be possible to use only the 7 (or even only the 6) most significant bits for the two color channels U and V without a significant reduction of the segmentation quality.
Another important question that has not been answered yet is, the lookup table can be set in order to achieve a certain segmentation result. Consider you want to segment a yellow ball in front of a black background. Now, you simply have to set the lookup table values for all possible colors of this ball to a value that is different from 0 (so maybe 200). Then the segmentation result will be white where the ball was, and black (value 0) otherwise. But how can you set up the lookup table values? Of course, it might be quite hard to find all possible colors of the ball, but it might be easy to pick some of its color e.g. by mouse. If you once have these color prototype pixels, you can now use one of the the ColorSegmentationOp::lutEntry functions in order to set up all corresponding lookup table entries to your class label (which was 200 in the example above). The lutEntry methods also provide the functionality to not only set up the prototype-pixel's class-labels, but also the class-lables of all pixels within the vicinity of the prototype pixels. So sometimes, if your 'to-be-segmented'-yellow ball is quite homogeneously yellow, it can be sufficient to add only a single average yellow prototype pixel with high radii for the single color components.
If you don't want to implement mouse handling in order to be able to click at your objects to get color-prototypes, you can also use the ICL-example tool 'icl-color-picker'.
By adding color prototypes for different objects, but with different class labels, you can also use this class to segment several objects at once. In the next step, you will possibly use an instance of ICLCV/RegionDetector to extract the segemented image regions. Here, you can then obtain the corresponding class label by using the ImageRegion's getVal method. Please note that always the last lut-entry is used if your prototype-entries overlap.
Maybe it is now, if not before necessary to mention, that the ColorSegmentationOp can only be set up to classify pixels into 255 valid classes. But actually, this should not become a problem at all as the classification quality usually restricts the number of classes to a maximum of about 10.
icl::filter::ColorSegmentationOp::ColorSegmentationOp | ( | icl8u | c0shift = 2 , |
icl8u | c1shift = 2 , |
||
icl8u | c2shift = 2 , |
||
core::format | fmt = core::formatYUV |
||
) |
Creates a new instance of this class with given segmentation parameters.
c0shift | number of least significant bits that are removed from channel0 |
c1shift | number of least significant bits that are removed from channel1 |
c2shift | number of least significant bits that are removed from channel2 |
fmt | internally used segmentation format (needs to have 3 channels) |
icl::filter::ColorSegmentationOp::~ColorSegmentationOp | ( | ) |
Destructor.
|
virtual |
main apply function
Implements icl::filter::UnaryOp.
virtual void icl::filter::UnaryOp::apply |
Imported apply from parent UnaryOp class.
virtual const core::ImgBase* icl::filter::UnaryOp::apply |
Imported apply from parent UnaryOp class.
classifies a pixel in given rgb format
void icl::filter::ColorSegmentationOp::clearLUT | ( | icl8u | value = 0 | ) |
clears the whole segmentation LUT
const std::vector<core::Color>& icl::filter::ColorSegmentationOp::getClassMeanColors | ( | ) |
returns a mean color for all used class labels (in rgb format)
const core::Img8u& icl::filter::ColorSegmentationOp::getColoredLUTPreview | ( | int | xDim, |
int | yDim, | ||
icl8u | zValue | ||
) |
this also creates a colored preview for the current segmentation LUT
the method behaves like getLUTPreview, except, it tints the resulting lut-slice with mean colors of that class
const icl8u* icl::filter::ColorSegmentationOp::getLUT | ( | ) | const |
returs the internal lut data
The data order is depth-major row-major i.e. an index(x,y,z) is estimated by (x + w*y + w*h * z)
icl8u* icl::filter::ColorSegmentationOp::getLUT | ( | ) |
returns the internal lut data
Please be careful with this method :-)
void icl::filter::ColorSegmentationOp::getLUTDims | ( | int & | w, |
int & | h, | ||
int & | t | ||
) | const |
returns the lut-sizes
w = 1+(0xff >> bitShift[0]) etc.
const core::Img8u& icl::filter::ColorSegmentationOp::getLUTPreview | ( | int | xDim, |
int | yDim, | ||
icl8u | zValue | ||
) |
this also creates a preview for the current segmentation LUT
Here, you can choose, which dimension shall become the resulting images with and height. The remaining 3rd dimension is slices with given zValue
|
inline |
returns the current internally used segmentation format
const core::Img8u& icl::filter::ColorSegmentationOp::getSegmentationPreview | ( | ) |
returns in internally managed image of the segmentation result
The image has the dimensions 2^(8-shift0) x 2^(8-shift1) and it has 2^(8-shift3) channels
|
inline |
returns the pointer to the 3 internally used segmentation shifts
void icl::filter::ColorSegmentationOp::load | ( | const std::string & | filename | ) |
loads the segmentation LUT only (no other parameters)
void icl::filter::ColorSegmentationOp::lutEntry | ( | icl8u | a, |
icl8u | b, | ||
icl8u | c, | ||
icl8u | rA, | ||
icl8u | rB, | ||
icl8u | rC, | ||
icl8u | value | ||
) |
given a,b and c in segFormat, this function fills the LUT within a sub-volume with given radii
void icl::filter::ColorSegmentationOp::lutEntry | ( | core::format | fmt, |
int | a, | ||
int | b, | ||
int | c, | ||
int | rA, | ||
int | rB, | ||
int | rC, | ||
icl8u | value | ||
) |
given a,b and c in format fmt, this function fills the LUT within a sub-volume with given radii
void icl::filter::ColorSegmentationOp::save | ( | const std::string & | filename | ) |
saves the segmentation LUT only (no other parameters)
void icl::filter::ColorSegmentationOp::setSegmentationFormat | ( | core::format | fmt | ) |
sets the internally used segmentation format
void icl::filter::ColorSegmentationOp::setSegmentationShifts | ( | icl8u | c0shift, |
icl8u | c1shift, | ||
icl8u | c2shift | ||
) |
sets the segmentation shifts
|
private |
bit shifts for all 8-Bit channels
|
private |
internal image in depth8u and segmentation format
|
private |
last used destination image
|
private |
color classification lookup table
|
private |
internal buffer holding the output image
|
private |
format, that is used for internal segmentation
|
private |
internal buffer for providing a preview of the current segmentation