Introduction to the XIL Imaging Library

The Solaris^(TM) XIL^(TM) Imaging Library provides a basic set of functions for imaging and video applications. The XIL library is the imaging component of the Solaris Graphics Architecture, a strategy for providing low-level software interfaces known as foundation libraries. Application and API developers can port their code to such foundation libraries. The XIL library is complemented by the XGL^(TM) Graphics Library, which addresses application and API requirements for geometry-based graphics.

Solaris Graphics Architecture

The XIL foundation library is an integral part of the Solaris Graphics Architecture. The Solaris software, using loadable drivers, enables display devices using the Solaris Graphics Architecture to be easily installed and used, without requiring kernel modifications. The Solaris Graphics Architecture, through the XIL, XGL, and OpenWindows^(TM) software, provides a means for third-party hardware and software vendors to develop applications with the knowledge that their investment will see long-term benefits, including access to a range of computing platforms and complete integration into the Solaris environment.

---

Note - Currently, the GPI interfaces to the library discussed in this book are uncommitted interfaces; therefore, they may change in future releases in ways that could require you to change your code. In a release in the near future, the interfaces will be deemed committed, and much stricter rules will then apply to interface changes. The API interfaces are committed.

---

Division of Function in the XIL Library

The XIL architecture consists of a high-level application programming interface (API), device-independent core code (including the XIL API and GPI layers), which manages the loading and calling of specific device-dependent functions, a graphic porting interface (GPI), which separates device-independent and device-dependent code, and the device-dependent (DD) algorithm implementation. Figure 1-1 illustrates this division of function and shows how these sections relate:

Figure 1-1

This document describes the XIL core (including the XIL API and GPI layers), the graphic porting interface (GPI), and the method needed to supply alternative DD code. In general, porting new hardware to the XIL environment

involves providing new implementations of DD modules. The GPI is the interface through which the DD modules are called and is responsible for allowing the creation of new DD implementations without requiring exposure of XIL library source code.

XIL API Layer

The API layer in the XIL library contains the C wrappers on the C++ device-independent classes. It consists of functions that can be categorized in the following way:

• Create and destroy objects
• Set and get object attributes
• Modify image data
• Extract information from an image
• Modify data in non-image objects
• Synchronize operations

The semantics of the functions exposed in the API are described in the XIL Programmer's Guide and the XIL Reference Manual.

The C++ XIL API level classes are used to implement the API functions described above. These classes provide a device-independent interface to the XIL library imaging functionality and are primarily used to pass information through the GPI to the DD modules. They are listed in Table 1-1 and individually described in the following sections.

Table 1-1

Class Name	Definition
XilDebugObject	The base class from which all other classes derive
XilGlobalState	Contains a list of system states and the tree of atomic/molecular operations and their corresponding function pointers
XilSystemState	Contains the creation methods for all API classes
XilObject	The parent class for all of the XIL API classes
XilAttribute	Contains multiple device attributes
XilInterpolationTable	Contains an array of 1 x n kernels that represent the interpolation filter in either the horizontal or vertical direction

Table 1-1 (Continued)

Class Name	Definition
XilImageType	Contains an image description without data
XilImage	The basic data element for XIL functions
XilCis	Contains the compressed image data and compression functions
XilKernel	Contains kernel data used in convolution and error diffusion
XilSel	A structuring element used in erosion and dilation
XilDitherMask	Contains dither matrices for ordered dithering
XilRoi	Contains region of interest information for an image
XilLookup	Contains data for image conversion and colormap use
XilColorspace	Contains information to specify a colorspace
XilHistogram	Contains image histogram information
XilError	Contains information for reporting errors

Figure 1-2 shows the base part of the XIL library object hierarchy.

Figure 1-2

XIL Base Classes

The XilDebugObject Class

The XilDebugObject class is the base class from which all XIL classes are derived. Its purpose is to allow members to be added to all classes at the same time for debugging purposes. It is currently empty.

Part of the definition of the XilDebugObject class is shown below:

Code Example 1-1 Definition of XilDebugObject Class

class XilDebugObject { };

The XilGlobalState Class

This class contains the tree of atomic/molecular operations and their corresponding function pointers. This tree is created initially by the atoms and molecules provided with the XIL library. Each node in the tree contains a list of function pointers. The base of the tree is stored in this class. A new compute handler uses describeMembers() to add all of the member functions from the compute handler into the list of function pointers at each node. If the compute handler has a new molecule, then a node is created for that sequence of atoms. See Chapter 4, "Compute Devices," for more detail about compute devices.

A single instance of this class is created when the first XilSystemState class is created (during the call to xil_open()). Only one XilGlobalState is created; it is pointed to by the single global variable xil_global_state.

Part of the definition of the XilGlobalState class is shown below:

Code Example 1-2 Definition of XilGlobalState Class

class XilGlobalState : public XilDebugObject { public: XilOpTreeNode* getOpTreeBase(); // Get the base of the tree that holds the // atomic/molecular operations and their // corresponding function pointers. }; extern XilGlobalState* xil_global_state;// The single global variable

The actual calls to dynamically load the various modules are done through the dlopen() system interface. A private member function of the XilGlobalState class implements this and additionally provides code to allow for loading alternate versions of the modules using the XIL_DEBUG interface. For information regarding XIL_DEBUG, refer to the section "The Development Environment" on page 60 of Chapter 2.

Multiple system states may be created by multiple calls to xil_open(). The various system states are created and destroyed through the global state. A singly linked list is used to store the system state objects. The global state destructor uses the list to make sure all the system state objects get destroyed.

The global state also contains the descriptions of atomic and combined image functions, or molecules, in a tree structure. See the description of the XilOpTreeNode object in the section "The XilOpTreeNode Class" on page 53 and the discussion on deferred execution and molecules in the section "Deferred Execution" on page 43.

The XilSystemState Class

The XilSystemState class contains all the information for an individual XIL session. The constructor for this class is called by xil_open(), which returns the XilSystemState handle. Applications must save this handle, since it is a required argument for all object creation routines. At the C++ level, all constructors of the API level XIL objects are members of this class. All of the API objects for an XIL session belong to that session's system state.

Part of the definition of the XilSystemState class is shown below:

Code Example 1-3 Definition of XilSystemState Class (1 of 3)

class XilSystemState : public XilDebugObject { public: // notifyError is the main error reporting function. It is called // by the handler code, normally via the XIL_ERROR macros defined // in xil/XilError.h void notifyError(XilErrorCategory category, char id[], int primary, int line, char* file, XilObject* object); // API object creation - image creation is overloaded to support // creation from user data and creation from image type. // The following GPI functions have the same parameters as their // corresponding API functions (except the void* parameter of the // createImage function--in the API the parameter is XilAttribute). XilImageType* createImageType(unsigned int width, unsigned int height, unsigned int nbands, XilDataType datatype); XilImage* createImage (XilImageType* image_type); XilImage* createImage (unsigned int width, unsigned int height, unsigned int nbands, XilDataType datatype); XilImage* createImage (char device[], void* value); XilImage* createImageWindow (Display* display, Window window); XilKernel* createKernel (unsigned int width, unsigned int height, unsigned int keyx, unsigned int keyy, float *data); XilDitherMask* createDitherMask(unsigned int width, unsigned int height, unsigned int bands, float* data); XilSel* createSel (unsigned int width, unsigned int height, unsigned int keyx, unsigned int keyy, unsigned int *data);

Code Example 1-3 Definition of XilSystemState Class (2 of 3)

XilLookup* createLookup (XilDataType input_type, XilDataType output_type, unsigned int nbands, unsigned int num_entries, short offset, void* data); XilLookup* createColorCube(XilDataType input_type, XilDataType output_type, unsigned int nbands, short offset, int multipliers[], unsigned int dimensions[]); XilLookup* createCombinedLookup (XilLookup lookup_list[], unsigned int num_lookups); XilHistogram* createHistogram(unsigned int nbands, unsigned int nbins[], float low_value[], float high_value[]); XilColorspace* createColorspace(char* name, unsigned int opcode, unsigned short nbands); XilRoi* createRoi (); XilCis* createCis(char* compression_name); XilAttribute* createAttribute (char* device_name);// the API // xil_device_create invokes this function XilInterpolationTable*createInterpolationTable (unsigned int kernel_size, unsigned int subsamples, float* data); // Attribute function to control whether the functions for this system // state are synchronized or allowed to be deferred. Xil_boolean getSynchronize(); void setSynchronize(Xil_boolean onoff); // Attribute function to control whether image operations are reported // -1 means check environment variable XIL_DEBUG for "show_action" // 0 means no show action // 1 means show action int getShowAction(); void setShowAction(int env_off_on);

Code Example 1-3 Definition of XilSystemState Class (3 of 3)

// Entry point to the named object database XilObject* getObjectByName(char* name,XilObjectType type); };

The XilObject Class

XilObject is the base class from which all of the API level XIL objects are derived. It contains all the attributes and functions that are generic to those exposed objects. It is an abstract class; no instance of this class is ever created.

Each API level object has a version number, which is updated using the member function newVersion() each time that the object is modified. This version number is a 64-bit quantity, and can be returned for any XilObject derived class by use of the member function getVersion(). A copy of an object returns the same version number. Use of object versions allows intelligent caching of API objects within the implementation.

Part of the definition of the XilObject class is shown below:

Code Example 1-4 Definition of XilObject Class

class XilObject : public XilDebugObject { public: char* getName (); // get a copy of the object's name void setName (char *name);// set the object's name to the one supplied void destroy (); // destroy objects. Use this rather than "delete" XilSystemState* getSystemState();// get a pointer to the system state that // this object was created under // All objects have unique version numbers, // which change every time the contents or // attributes of the object change. This allows implementations to // perform intelligent caching of objects. XilVersionNumber getVersion(); // get the version number of this object

Code Example 1-4 Definition of XilObject Class (Continued)

// set the version number of this object to the one supplied. This is // used when making identical copies of objects void setVersion(XilVersionNumber new_version);// set the version number // of this object to the one supplied. // This is used when making identical // copies XilObjectType getObjectType(); // return the XilObjectType of the object // each object has a character pointer associated with it for use in // error reporting. getErrorString copies "storage_size" characters // from the buffer pointed to by this pointer to the "error_storage" // buffer. setErrorString sets this pointer to "str". virtual void getErrorString(char* error_storage, int storage_size); virtual void setErrorString(char* str); };

The XilDeviceType Class

The XIL library has the concept of devices--software and hardware--which are represented by the device handler modules. More than one instance of a device may be created. In this case, information that is common to all instances of a device should be held in the device type class. XilDeviceType is an abstract class, containing the general information needed at this level. The XIL library subclasses XilDeviceType for each of the kinds of devices that are supported.

---

Note - Each class derived from the XilDeviceType class must have its own destructor (even if it takes no action). The base class has a virtual destructor. If the derived class does not supply the destructor, the compiler attempts to in-line the virtual destructor. This results in a warning message and creates out-of-line copies of the destructor, which could result in wasted space.

---

Part of the definition of the XilDeviceType class is shown next:

Code Example 1-5 Definition of XilDeviceType Class

#include "XilDebugObject.h" // This class contains all information common to I/O, storage, and compute // device types. XilDeviceType is an abstract class. The library subclasses one // instance of an XilDeviceType for each of the kinds of devices it supports. class XilDeviceType : public XilDebugObject { public: virtual void printStatus(); // print any information that the // device writer thinks might be // useful for debugging virtual void optimizeMemoryUsage(); // routine to request that the // device free any "optional" system // memory that it may be using virtual ~XilDeviceType(); // destructor XilDeviceType(char *dname); // constructor with name param };

The XilDevice Class

XilDevice is an abstract class that is subclassed for each of the kinds of devices supported by the XIL library (described below). It is similar to XilDeviceType, but contains the general information needed for each instance of the device.

Part of the definition of the XilDevice class is shown below:

Code Example 1-6 Definition of XilDevice Class

#include "XilDebugObject.h" // XilDevice // // This is the base class for all the various XIL devices // printStatus() is potentially useful for debugging. class XilDevice : public XilDebugObject { virtual void printStatus(); virtual ~XilDevice(); };

XIL API Level Classes

As described earlier in this chapter, XilObject is the base class for all the API level classes. You cannot instantiate an object from these classes; instead, you must get a copy of or a reference (pointer) to the object. If you get a copy, you are responsible for freeing the allocated data.

Figure 1-3 describes the relationship among the API level objects. The classes are described in the following sections.

Figure 1-3

The XilImageType Class

This class carries information about an image, independent of its associated pixel values. It is used at the API level to return information about the kind of image the application should create to act as a destination from a decompression or device capture, or as a source to a compression or device display. XilImageType is subclassed by the XIL library to create the XilImage class. There is an API function that creates an image directly from an XilImageType object.

Part of the definition of XilImageType class is shown next:

Code Example 1-7 Definition of XilImageType Class

class XilImageType : public XilObject { public: unsigned short getWidth(); // the width (extent in x) of the image unsigned short getHeight(); // the height (extent in y) of the image // overloaded functions to return the x and y image size void getSize(unsigned short* width, unsigned short* height); void getSize(unsigned int* width, unsigned int* height); unsigned short getBands(); // the number of bands in the image XilDataType getDataType(); // the data type of the image // overloaded functions to return all of the image parameters void getInfo(unsigned int* width, unsigned int* height, unsigned int* nbands, XilDataType* datatype); void getInfo(unsigned short* width, unsigned short* height, unsigned short* nbands, XilDataType* datatype); // functions to set or get a copy of the XilImageType's colorspace void setColorspace(XilColorspace* colorspace); XilColorspace* getColorspace(); };

The XilImage Class

Derived from the XilImageType class, XilImage represents an image along with its associated data. The XilImage class contains member functions that make up the XIL image functions. It also contains member functions for storage and retrieval of image attributes.

Three of the member functions getStorage(), getMemoryStorage(), and requestStorage() have similar names and deserve an explanation noting their differences.

getStorage() returns a pointer to a structure that describes the data storage for the specified storage type. If the current device is not the specified type and cannot emulate the type; then the routine returns NULL. NULL indicates you

must try another storage type. If the routine is successful, it returns a pointer. The structure does not take child image offsets into account, so you must calculate the offsets by using the offsetX, offsetY, and offsetBand fields returned by the getChildOffsets() member function. For example, for a byte image that is a child image, you would perform the calculation below to have an accurate pointer to the start of the image data:

new_storage.byte.data = storage->byte.data+ storage->byte.scanline_stride*offsetY+ storage->byte.pixel_stride*offsetX+ offsetBand;

getMemoryStorage() does not allow you to specify the storage type. It uses only the memory storage handler structures defined in

XilMemoryDefines.h. This routine takes child offsets into account, so there are not extra calculations that you must perform.

requestStorage() returns a pointer to a structure that describes the data storage for the specified storage type, but does not force a propagation as getStorage does. If the storage has not already been allocated for this image, the routine returns NULL. If the requested device does not have the storage or cannot efficiently emulate the requested device, the routine returns NULL.

Note that an exported image is required to stay in memory and will not be propagated to another storage type, which may prevent acceleration.

Two member functions enable you to indicate which pixels in the destination image are touched by an operation and to indicate that the XIL core is responsible for freeing the ROI--setPixelsTouchedRoi() and setPixelsTouchedRoi_flag().

Operations must intersect the ROI for the source image and destination image to determine the ROI that is written by the operation. This determination is standard procedure and is usually performed by using XiliGetRoiList(). For example,

(set up for operation) . . // get the intersection of source and destination image's ROIs

intersected_list = XiliGetRoiList(&intersected_roi,src1,src2,dest) . . (loop over the intersected list of rectangles to perform operation) . . // clean up from operation intersected_roi->destroy(); intersected_list->destroy(); return(XIL_SUCCESS);

For device images, XIL may need the intersected ROI after the operation has completed and the display operation has begun (see Chapter 3, "I/O Devices," for information about device images). Therefore, the intersected ROI should not be destroyed as shown in the code above. Instead, the intersected ROI should be stored (as shown above) and assigned as the "PixelsTouchedRoi" (the pixels that were touched) for the image. Then, a flag should be set that indicates that the XIL core is responsible for freeing the intersected ROI. For example,

(set up for operation) . . // get the intersection of source and destination image's ROIs intersected_list = XiliGetRoiList(&intersected_roi,dest,src1,src2) // set up the PixelsTouchedRoi src1->setPixelsTouchedRoi(intersected_roi); src1->setPixelsTouchedRoi_flag(TRUE); . . (loop over the intersected list of rectangles to perform operation) . .

// clean up from operation intersected_list->destroy(); return(XIL_SUCCESS);

---

Note - If PixelsTouchedRoi is assigned, the routine must not destroy the intersected ROI.

---

If a routine does not assign the intersected ROI as the PixelsTouchedRoi for the image, the ROI on the display image determines which pixels are touched.

Part of the definition of the XilImage class is shown below:

Code Example 1-8 Definition of XilImage Class (1 of 5)

class XilImage : public XilImageType { public: // get only attributes (set at create time) XilImage *getParent(); // return the pointer to the image's // parent, NULL if there is no parent. void* getMemoryStorage(); // get a pointer to a structure // that describes the memory // layout. This function takes child // image offsets into account. void setMemoryStorage(XilMemoryStorage* storage);// set the memory // storage of an image void* getStorage(char storage_type[]); // get a pointer to a structure // that describes the data storage. // This function returns the // storage information on the parent // only, but allows the calling process // to request a particular kind of // storage. XilDeviceInputOutput* getDeviceInputOutput(); // access the input/output // device, if the image is a device // image. This will be NULL if it is // not a device image.

Code Example 1-8 Definition of XilImage Class (2 of 5)

void getDimensions (unsigned int *x_size,// get the image size parameters unsigned int *y_size, unsigned int *nbands); void getChildOffsets(unsigned int *offsetX,// get the child image offsets unsigned int *offsetY, // if this is a child image. unsigned int *offsetBand);// If this is not a child // image, the offsets will be 0 // functions to return whether this image can be read (if it is an // output device image, it might not be), or written (if it is an // input device image, it probably cannot be). Xil_boolean isReadable (); Xil_boolean isWriteable (); // get/set attributes float getOriginX(); // get the x image origin float getOriginY(); // get the y image origin void getOrigin(float* x, float* y); // get the image origin in floats void getOrigin(long* x, long* y); // get the image origin in longs void setOrigin(float x, float y); // set the image origin int getAttribute (char attribute_name[], void**); // get (return a pointer // to) a user-assigned attribute int setAttribute (char attribute_name[], void*); // set a user- // assigned attribute int getDeviceAttribute(char name[], void**); // get (return a // pointer to) a device-specific // attribute. This only applies // to images that are device // images. int setDeviceAttribute(char name[], void* value); // set a device- // specific attribute. // This only applies to images // that are device images.

Code Example 1-8 Definition of XilImage Class (3 of 5)

// functions used by the blackGeneration() function void setUndercolor (float color);// set the image undercolor value float getUndercolor (); // get the image undercolor value void setBlack (float color); // set the image black color value float getBlack (); // get the image black color value // ROI functions XilRoi* getRoi(); // return a copy of the ROI // assigned to this image. If no ROI is // assigned, NULL is returned. XilRoi* getImageSpaceRoi(); // return a pointer to the // image-origin adjusted, clipped ROI // assigned to this image. If no ROI was // initially assigned, a pointer to an // ROI that encompasses the entire image // is returned. The calling routine must // not delete this ROI. void setRoi(XilRoi* roi); // set the image's ROI. An // internal copy is made of the ROI that // is passed to this function. // PIXELS TOUCHED functions // Functions to set the region of pixels touched by the last routine // that writes to the image. The flag indicates whether the routine used // the PTRoi mechanism (This means only new routines will know to set // the flag. Old routines will default to the old behavior) void setPixelsTouchedRoi (XilRoi* roi); // assign the roi as the // images's PixelsTouchedRoi void setPixelsTouchedRoi_flag (Xil_boolean value);// set a flag to // indicate the image has a valid // pixelsTouchedRoi

Code Example 1-8 Definition of XilImage Class (4 of 5)

// functions pertaining to deferred execution void sync (); // evaluate the contents of // this image immediately void toss(); // indicate to the deferred- // execution mechanism that we are no // longer interested in the contents of // this image. void setSynchronize (Xil_boolean onoff);// set the synchronization // of the image. If set to TRUE, // operations are always executed // immediately. Setting it to FALSE // enables deferral of operations // (and possible optimization) Xil_boolean getSynchronize (); // return the current synchronization // value of the image // overloaded functions for child image creation XilImage* createChild (unsigned int xstart, unsigned int ystart, unsigned int width, unsigned int height, unsigned int startband, unsigned int numbands); XilImage* createChild (unsigned short xstart, unsigned short ystart, unsigned short width, unsigned short height, unsigned short startband, unsigned short numbands); // overloaded functions for making image copies XilImage* createCopy (unsigned int xstart, unsigned int ystart, unsigned int width, unsigned int height, unsigned int startband, unsigned int numbands); XilImage* createCopy (unsigned short xstart, unsigned short ystart, unsigned short width, unsigned short height, unsigned short startband, unsigned short numbands); // overloaded functions for setting and getting single pixel values void setPixel(unsigned short x, unsigned short y, float* pixel_value); void setPixel(unsigned int x, unsigned int y, float* pixel_value); void getPixel(unsigned short x, unsigned short y, float* pixel_value); void getPixel(unsigned int x, unsigned int y, float* pixel_value);

Code Example 1-8 Definition of XilImage Class (5 of 5)

// import export stuff int export(); // cause the image data to be moved // from the library space to // application space (memory). // Applications cannot access image // data that has not been exported. // The function returns XIL_SUCCESS or // XIL_FAILURE. void import(Xil_boolean change_flag);// cause the image data // to be moved from application space // to the XIL library space int getExported(); // return one of three possible values: // 0 if the image is not exported // 1 if the image is exported // -1 if the image is not exportable // (for example, a device image) void* getExportedMemoryStorage();// get a pointer to a structure // that describes the memory // layout once the image has been // exported. This function takes child // image offsets into account. void setDimensions (unsigned int x_size,// set the image size unsigned int y_size, // parameters (resize the image) unsigned int nbands); void* requestStorage (char storage_type[]);// get a pointer to a // to a that describes the data // storage. This function returns the // storage information on the parent only // but allows the calling process to // to request a particular kind of // storage. If the request cannot be // satisfied, NULL is returned. };

The XilKernel Class

The XilKernel class represents a two-dimensional array of floating point values. XilKernel objects are used as parameters in the image convolution and blend functions.

Part of the definition of the XilKernel class is shown below:

Code Example 1-9 Definition of XilKernel Class

class XilKernel : public XilObject { public: unsigned short getWidth (); // return the width (x size) of the kernel unsigned short getHeight (); // return the height (y size) of the kernel unsigned short getKeyX (); // return the x key pixel value of the kernel unsigned short getKeyY (); // return the y key pixel value of the kernel float *getValue (); // return a pointer to the actual kernel data XilKernel* createCopy(); // return a copy of the kernel };

The XilRoi Class

XilRoi describes an arbitrary region of interest (ROI). Member functions exist to manipulate and logically combine XIL ROIs. The functions of the XilRoi class are all pure virtual but are implemented in derived classes within the XIL library.

Each ROI is made up of a list of rectangles. If the ROI is empty, then the number of rectangles in the list is 0 (zero). An empty ROI may be returned when the intersection of specified ROIs have no overlapping pixels. In the following case, the intersect routine does not return a NULL pointer; instead, it returns an empty ROI.

roi_intersected = roi_intersect(roi2); if (roi_intersected->getNumRects()==0) { // code here for empty ROI } else { // process list of rectangles }

To process a list of rectangles, take the following steps:

1. Get the list of rectangles from a specific ROI by using the getRectList() function of the XilRoi class.

2. Move through the list of rectangles by using the next() function of the XilRoiList class.

class XilRoiList : public XilObject { public: virtual Xil_boolean next ( long *x, // rectangle left-most x long *y, // rectangle topmost y unsigned int *x_size, // rectangle size in x unsigned int *y_size = 0);// rectangle size in y

The next() function expects pointers to variables. The function writes information about the rectangle (starting x, starting y, width in x, and height in y) to these variables. The next() function returns a status of TRUE when there is a rectangle whose information has been loaded into the given parameters. Each time TRUE is returned, XilRoiList updates its internal state, tracking the current "next" rectangle. The next() function returns a status of FALSE when no more rectangles are in the list. If the next() function encounters an empty ROI, it returns FALSE on the first call.

Typically, routines operate on an intersected ROI, the area intersected by the source(s) and destination ROIs. Source and destination ROIs are intersected in image space, taking into account the original offsets and size of the image to which they are attached. For example,

roi = (dst->getImageSpaceRoi())-> intersect(src->getImageSpaceRoi());

If no ROI is active on an image, then the image has a ROI that is the same size as the image. This semantic is handled internally in XIL by the getImageSpaceRoi() function. Once the routine has the intersected ROI, the routine loops over the rectangle list (roi_list).

roi_list = roi->getRectList(); while(roi_list->next(&over_x,&over_y, &over_x_size,&over_y_size)) { // process the rectangles }

When the routine completes the operation, it must destroy the roi_list.

roi_list->destroy;

Normally, the ROI is stored via the setPixelsTouchedRoi() function of the XilImage class. See the section "The XilImage Class" on page 14 for more information.

You can use the XiliGetRoiList() routine as a shortcut for forming a roi_list. This routine intersects up to four images using image space ROIs. It stores in outroi the copy of the final intersected ROI and returns a copy of the rectangle list of that ROI. The prototype is shown next:

XilRoiList* XiliGetRoiList ( XilRoi** outroi, XilImage* image1, XilImage* image2, XilImage* image3=NULL, XilImage* image4=NULL);

Only image pointers that are non-NULL are valid for the calculations of a rectangle list, and the image pointers are searched up to a NULL. You must provide at least the image1 and image2 pointers (normally destination/source1). If there are more sources, then use image3 and image4. For example, if you have a routine that needs an ROI intersection for two source images and a destination image, XiliGetRoiList() would look like:

roi_list=XiliGetRoiList(&roi,dest,src1,src2);

Part of the definition of the XilRoi class is shown next. Notice that the member functions in this class are all pure virtual. Subclasses that contain the functionality for each of these virtual functions are provided in the library. The caller of any function that returns a new ROI is responsible for destroying it.

Code Example 1-10 Definition of XilRoi Class (1 of 4)

class XilRoi : public XilObject { public: virtual int addImage (XilImage* image) = 0;// add values that are nonzero // in the passed image to the ROI. // The image is expected to be of type // XIL_BIT. Images are converted to // rectangles and added to the ROI. virtual int addRect (long x, long y, // add the specified rectangle long width, long height) = 0; // to the ROI

Code Example 1-10 Definition of XilRoi Class (2 of 4)

virtual int addRegion (Region region) = 0;// add the specified X region // to the ROI virtual XilRoi* affine(float* matrix) = 0;// return a new ROI that is the // result of the affine transformation // of this ROI with the specified matrix virtual XilRoi* createCopy () = 0; // return a new ROI which is a // copy of this ROI virtual void dump () = 0; // print out debugging information // describing the ROI virtual XilImage* getAsImage () = 0; // return an XIL_BIT image which // represents the ROI. The image // needs to encompass the entire // extent of the ROI, with pixels // within the ROI set to 1, // pixels outside set to 0. virtual Region getAsRegion () = 0; // return an X Region which // the ROI virtual XilRoi* intersect (XilRoi* roi) = 0;// return a new ROI that // is the result of an // intersection between "this" // ROI and the passed ROI virtual XilRoi* intersect (short* cliplist, int orgx, int orgy) = 0; // return a new ROI that is the // result of an intersection between // "this" ROI and the passed cliplist // (primarily used for DGA cliplists) // return a new ROI which is the result of an intersection between // the current ("this") ROI and all the passed ROIs virtual XilRoi* intersect (XilRoi* roi1, XilRoi* roi2) = 0; virtual XilRoi* intersect (XilRoi* roi1, XilRoi* roi2, XilRoi* roi3) =0;

Code Example 1-10 Definition of XilRoi Class (3 of 4)

virtual XilRoi* rotate (float angle, float xorigin, float yorigin) = 0; // return a new ROI that is the result // of a rotation of "this" ROI about // the specified origin virtual XilRoi* scale (float xscale, float yscale, float xorigin, float yorigin) = 0; // return a new ROI that // is the result of scaling of "this" // ROI about the specified origin virtual int subtractRect(long x, long y, long width, long height)=0; // remove the specified rectangle // from the ROI virtual XilRoi* translate(int x, int y)=0;// return a new ROI that is the // result of a translation of "this" // ROI by the specified x and y // amounts virtual XilRoi* transpose (XilFlipType fliptype, float xorigin, float yorigin) = 0; // return a new ROI that // is the result of transposing // "this" ROI about the specified // origin virtual XilRoi* unite (XilRoi* roi) = 0;// return a new ROI that is the // union of "this" ROI with the // specified ROI virtual Xil_boolean pointInRegion (long x, long y) = 0; // return TRUE or FALSE, depending on // whether the specified point is // within or outside the ROI virtual int numRects () = 0; // return the number of rectangles // it would take to fully specify // the ROI virtual void boundingBox (long* x1, long* x2, long* y1, long* y2) = 0; // return a rectangle that bounds // the whole ROI

Code Example 1-10 Definition of XilRoi Class (4 of 4)

virtual XilRoiList* getRectList () = 0; // return a list-of-rectangles // object for this ROI. This allows // application code to step through // the rectangles in an ROI using // using XilRoiList::next() // XilRoi object constructor XilRoi(XilSystemState* system_state) : XilObject(system_state,XIL_ROI) {}; };

The XilLookup Class

The XilLookup class describes a table of data that is used to interpret image data. It can be used to modify the data, creating an output image by treating each input image pixel as an array index. The lookup table can have multiple output data for each input value; that is, it can convert a single band image into a multiple band image. In the special case of three bands output, it can be thought of as a colormap. The data elements may be extracted from the lookup and placed into an X colormap.

To support the common occurrence of performing a lookup on a color image with different response curves for each band, XilLookup also supports multiband lookups. The XilLookup object can contain a separate data array for each band in the input image. The number of bands in the output image must match the number of bands in the input.

Part of the definition of the XilLookup class is shown below:

Code Example 1-11 Definition of XilLookup Class

class XilLookup : public XilObject { public: XilDataType getInputDataType (); // return the datatype of the input XilDataType getOutputDataType ();// return the datatype of the output unsigned int getNumEntries (); // return the total number of entries // in the table or returns 0 for a // combined (multiband) lookup.

Code Example 1-11 Definition of XilLookup Class (Continued)

unsigned short getNBands (); // return the number of bands in the // output void* getData (); // return a pointer to the actual data short getOffset (); // return the offset that describes // the input value corresponding to // the first table value or returns 0 // for a combined (multiband) lookup. void setOffset (short); // set the offset that describes // the input value corresponding to // the first table value. Returns 0 if it // is a multiband lookup. void getValues (short start, unsigned int count, void* data); // copy 'count' data values from the // LUT starting at the table entry // position 'start' into buffer 'data' f void setValues (short start, unsigned int count, void* data); // copy 'count' data values from the // buffer 'data' into the LUT starting // at the table entry position 'start'. // Generates an error if it is a // multiband lookup. Xil_boolean getIsColorCube (); // return TRUE if the LUT is formatted // as a colorcube, FALSE otherwise Xil_boolean getColorCubeInfo(int multipliers[], unsigned int dimensions[], short* origin); // return information on the colorcube // formatting if this LUT is a colorcube XilLookup* convert (XilLookup* dst);// calculate and return a copy of // the LUT that converts // between the two LUTs "this" // and dst. The resulting LUT's input // datatype will be that of the input // datatype of "this", and its output // datatype will be that of the input

Code Example 1-11 Definition of XilLookup Class (Continued)

// datatype of dst. The LUT's offset // and number of entries are the same // as those for "this". Index N of the // resulting LUT contains the index of // the nearest color in dst to the color // at index N in "this". Nearest color // is determined by Euclidean distance. // Source and destination LUTs must have // the same input datatypes, output // datatypes, and number of bands. XilLookup* createCopy (); // return a copy of the LUT unsigned short getInputNBands ();// return the number of bands in // the input // Each of the following functions returns a list where each index // in the list corresponds to a band in the combined (multiband) lookup. unsigned int* getEntriesList (); // return the list of the number of // entries for each lookup in a combined // table short* getOffsetsList (); // return the list of the offset for each // lookup in a combined table void** getDataList (); // return the list of the data for each // lookup in a combined table XilLookup* getBandLookup (unsigned int band_num);// return an XilLookup // from a combined lookup };

The XilCis Class

The XilCis (for compressed image sequence) class is the primary object for compression in the XIL library. It contains member functions to allow access to and movement through compressed data. The XilCis is created by loading a specified compressor.

Part of the definition of the XilCis class is shown below:

Code Example 1-12 Definition of XilCis Class (1 of 4)

class XilCis : public XilObject { public: // get only attributes (set at create time or by compressor) char* getCompressor(); // return the name of a compressor char* getCompressionType(); // return the name of the type of // compressor Xil_boolean getRandomAccess(); // return TRUE if the compressor // supports random accessing of // individual frames of the sequence; // otherwise, returns FALSE int getStartFrame(); // return the index to the first // compressed image in the CIS int getReadFrame(); // return the index to the current // read frame int getWriteFrame(); // return the index to the next // frame that will be written XilImageType* getInputType(); // return the XilImageType that the // CIS will accept for compression XilImageType* getOutputType(); // return the XilImageType // produced by a compressor

Code Example 1-12 Definition of XilCis Class (2 of 4)

Xil_boolean getReadInvalid(); // return TRUE if a bitstream error // occurs during decompression. // Otherwise, returns FALSE // indicating that the CIS is valid // and able to be decompressed. Xil_boolean getWriteInvalid(); // return TRUE if a bitstream error // occurs during compression. // Otherwise, returns FALSE // indicating that the CIS is valid // and compression can continue. // get functions: these return something about the state of // the cis (probably attributes) int hasData(); // return the number of bytes of // compressed data the CIS contains int numberOfFrames(); // return the number of complete // frames the CIS contains Xil_boolean hasFrame(); // return TRUE if a complete frame // exists at the read frame position. // Otherwise, return FALSE. // get/set attributes int getKeepFrames(); // return the value that has been set // as the maximum number of frames // that the CIS should keep in a // compressor buffer void setKeepFrames(int k); // set the number of frames before // the read frame that the CIS should // keep in a compressor buffer int getMaxFrames(); // return the value that has been set // as the maximum number of // compressed frames that the CIS // will buffer at one time

Code Example 1-12 Definition of XilCis Class (3 of 4)

void setMaxFrames(int m); // set the upper limit on the number // of compressed frames that the CIS // should buffer int getFramesToCompress(); // return the value that has been set // as the number of frames that are // compressed if the source image is // SEQUENTIAL void setFramesToCompress(int number_of_frames); // set the number of frames to // compress if the image is // SEQUENTIAL Xil_boolean getAutorecover(); // return TRUE if autorecovery // has been turned ON. Otherwise, // return FALSE. void setAutorecover(Xil_boolean on_off);// set autorecovery ON // or OFF. The default is OFF (FALSE). // If autorecovery is ON, recovery is // attempted after a bitstream error // occurs. int getAttribute(char attribute_name[], void** value); // return the value of the specified // compressor attribute int setAttribute(char attribute_name[], void* value); // set a compressor attribute // compression/decompression functions // these functions use this image as a destination so they just insert // an operation // void flush(); // instruct the compressor to // complete any pending write // operations void sync(); void reset(); void seek(int framenumber, int relative_to);

Code Example 1-12 Definition of XilCis Class (4 of 4)

void attemptRecovery(unsigned int nframes, unsigned int nbytes); // attempt recovery after an error // occurs in a CIS // this function cannot be deferred because it interacts with // non-library data void getErrorString(char* error_storage, int storage_size); // get the XilDeviceCompression pointer for this CIS XilDeviceCompression* getDeviceCompression(void); // This function is used by device compressions that do not have a // reliable way to get back to deferred frames. They MUST get back // to the current frame, but can use this function to avoid having any // frames older than read_frame - 1. It is in the public part instead // of the private in order to avoid having to install XilCisPrivate.h // in the install point. void flushPriorDecompressOps(int frame_no); }; #define XIL_CIS_ERROR(category,id,primary,dc, read_invalid, write_invalid) \ do { \ if (!(dc)->inMolecule()) \ (dc)->generateError((category), (id), (primary), \ (read_invalid), (write_invalid), __LINE__, __FILE__); \ } while (0); #define XIL_CIS_UNCOND_ERROR(category,id,primary,dc,read_invalid, write_invalid) \ (dc)->generateError((category), (id), (primary), (read_invalid), \ (write_invalid), __LINE__, __FILE__);

The XilError Class

This class describes errors in the XIL library. Its member functions allow programs to get information about the error, to retrieve the object that is associated with the error, and to control the error handling routines.

Part of the definition of the XilError class is shown below:

Code Example 1-13 Definition of XilError Class

class XilError { public: char* getString(); // get a string associated with the error. // This function uses the localization functions // bindtextdomain() and dgettext() to parse the // error id string into a localized message char* getId(); // get the error id string void setId(char error_string[]);// set the error id string int getLine(); // get the line number where the error occurred void setLine(int line); // set the line number where the error occurred char* getFile(); // get the file in which the error occurred void setFile(char* file); // set the file in which the error occurred char* getLocation(); // primarily an internal routine to indicate // where and in which file an error occurred XilErrorCategory getCategory();// get the error category define; one of: // XIL_ERROR_SYSTEM // XIL_ERROR_RESOURCE // XIL_ERROR_ARITHMETIC // XIL_ERROR_CIS_DATA // XIL_ERROR_USER // XIL_ERROR_CONFIGURATION // XIL_ERROR_OTHER char* getCategoryString();// get the error category as a string void setCategory(XilErrorCategory category); // set the error category int getPrimary(); // get the type of error (primary or secondary) void setPrimary(int primary);// set the type of error XilObject* getObject(); // get the object associated with the error

Code Example 1-13 Definition of XilError Class (Continued)

void setObject(XilObject* object);// set the object associated with error XilSystemState* getSystemState(); // get the system state associated with // the error void setSystemState (XilSystemState* sysSt);// set the system state // associated with the error }; // defines for different ways of reporting errors #define XIL_ERROR(sysSt,category,id,primary) \ { \ XilSystemState* _state=sysSt; \ _state->notifyError(category,id,primary,__LINE__,__FILE__, (XilObject*)NULL); \ } #define XIL_OBJ_ERROR(sysSt,category,id,primary,object) \ { \ XilSystemState* _state=sysSt; \ _state->notifyError(category,id,primary,__LINE__,__FILE__,object); \ } #define XIL_OBJ_STR_ERROR(sysSt,category,id,primary,object,str) \ { \ XilSystemState* _state=sysSt; \ object->setErrorString(str); \ _state->notifyError(category,id,primary,__LINE__,__FILE__,object); \ object->setErrorString(NULL); \ }

The XilHistogram Class

The XilHistogram class describes a multidimensional histogram. This object can be used to gather statistical information on images.

Part of the definition of the XilHistogram class is shown below:

Code Example 1-14 Definition of XilHistogram Class

class XilHistogram : public XilObject { public: unsigned short getNBands (); // the number of bands in the // histogram (this is a // multi-dimensional object) void getNBins (unsigned int *nbins); // the number of bins for each band unsigned int getNElements (); // total number of elements in the // array void getLowValue(float *low_value); // copy the array of floats that // define the value of the first bin // for each band to the user-supplied // array "low_value" void getHighValue(float *high_value);// copy the array of floats that // define the value of the last bin // for each band to the user-supplied // array "high_value" void getData (unsigned int *data); // copy the histogram data into the // user-supplied buffer "data" unsigned int *getDataPtr(); // return a pointer to the actual data };

The XilColorspace Class

XilColorspace describes a color space of an image in such a way that images may be transformed from one color space to another. The XIL Imaging Library supports ten color spaces, which are created at the time of a call to xil_open(). Each of the supported color spaces is assigned an opcode. This opcode is referenced by the color conversion routines. The correlation between

opcode and color space is defined in the cs.h file and is shown in Table 1-2. In the table, the string that follows the "CS" prefix is the color space name. For more information regarding supported color spaces, see XIL Programmer's Guide or the man page for xil_colorspace_get_by_name().

Table 1-2 Opcodes

Color Space	Opcode
CSrgblinear	1
CSrgb709	2
CSphotoycc	3
CSycc601	4
CSycc709	5
CSylinear	6
CSy601	7
CSy709	8
CScmy	9
CScmyk	10

In the current release of the XIL library, there is no way to create a color space; all supported color spaces are created as standard objects at start-up time and can be retrieved using the xil_colorspace_get_by_name() functions in the core. Future releases of the library will enhance this scheme to allow the creation of device-dependent color spaces, to enable device color management.

Part of the definition of the XilColorspace class is shown below:

Code Example 1-15 Definition of XilColorspace Class

class XilColorspace : public XilObject { public: unsigned int getOpcode(); // return the opcode associated with // the XIL supported color space unsigned short getNBands(); // return the number of bands in the // colorspace XilColorspace* createCopy(); // return a copy of the colorspace };

The XilSel Class

The XilSel class describes a structuring element, which is a two-dimensional description of a pixel neighborhood. In the XIL library, the structuring element is described with a two-dimensional boolean (integer) array, with pixels in a neighborhood having true values in the array, and pixels excluded from the neighborhood having false values. Structuring elements are currently used as parameters to the xil_erode() and xil_dilate() functions.

Part of the definition of the XilSel class is shown below:

Code Example 1-16 Definition of XilSel Class

class XilSel : public XilObject { public: unsigned short getWidth (); // return the width (x size) of the sel unsigned short getHeight (); // return the height (y size) of the sel unsigned short getKeyX (); // return the x key pixel value of the sel unsigned short getKeyY (); // return the y key pixel value of the sel unsigned int *getValue (); // return a pointer to the actual sel data XilSel* createCopy(); // return a copy of the sel };

The XilDitherMask Class

In the simplest case, the dither mask is a two-dimensional array of values that determines how the noise added during the dither process is spread across the image. In the XIL library, the dither mask can have multiple bands, each band with its own matrix. This allows noise to be spread differently for each channel of a true-color image, which can enhance the result of the dither operation. For dithering of multiband images, the number of bands in the dither mask must match the number of bands in the source image.

Part of the definition of the XilDitherMask class is shown below:

Code Example 1-17 Definition of XilDitherMask Class

class XilDitherMask : public XilObject { public: unsigned short getWidth (); // return the width (x size) of the dither // mask unsigned short getHeight (); // return the height (y size) of the dither // mask unsigned short getBands (); // return the number of bands of the dither // mask float *getValue (); // return a pointer to the actual dither // mask data XilDitherMask* createCopy(); // return a copy of the dither mask };

The XilAttribute Class

The XilAttribute class describes the attribute/value pairs of a device. The member functions of this class enable you to access and set a device's attributes.

An XilAttribute object can be used to set multiple device attributes simultaneously. This is important when device images are created and when the setting of device attributes incurs substantial overhead.

When you use this object to create a device, you should set only attributes the device understands. If the device does not recognize an attribute that you have set through the XilAttribute object, an error is generated. You should set default values for a device's attributes based on the list of attribute/value pairs returned by the XilAttribute object.

The definition of the XilAttribute class is shown next:

Code Example 1-18 Definition of XilAttribute Class

// Data structure for the attribute-value pairs typedef struct __XilKeyValue Pairs { char* key; // store the attribute name void* value; // store the attribute value } XilKeyValuePairs; class XilAttribute : public XilObject { public: // return a pointer to a list of attribute-value pairs that exist for // this XilAttribute object. The list is of length "list_length" XilKeyValuePairs** getAttributes (unsigned int* list_length); // assign an attribute-value pair of this XilAttribute object // if the attribute name has already been set, then the specified // attribute_value will replace the previous value void setValue (char* attribute_name, void* attribute_value); };

The XilInterpolationTable Class

The XilInterpolationTable class supports general interpolation. See XIL Programmer's Guide for a discussion about general interpolation. This class describes an array of 1 x n kernels. The array represents the interpolation filter in either the horizontal or vertical direction. The member functions of this class enable you to access the data in an XilInterpolationTable object.

The definition of the XilInterpolationTable class is shown next.

Code Example 1-19 Definition of XilInterpolationTable Class

class XilInterpolationTable : public XilObject { public: unsigned int getKernelSize (); // return the size of the // interpolation filter unsigned int getSubsamples (); // return the number of subsamples // kernel entry float* getData (); // return pointer to the table data XilInterpolationTable* createCopy ();// return a copy of the table };

XIL Core Layer

The Core layer in the XIL library manages the dynamic loading of device handlers, deferred execution, and operation scheduling.

Deferred Execution

The primary problem in achieving adequate performance in an imaging library comes from the way in which the units (or atoms) of functionality are arbitrarily combined to perform useful work. The typical imaging case is much more general than, for example, XGL with its well-defined processing pipeline. This has tended to limit the usefulness of general imaging libraries, since any reasonable division of imaging functionality into atoms renders the performance of many applications substandard. The result is that useful libraries tend to be closely tailored to applications and vertical markets.

The use of multiple passes of atoms impedes performance in at least three ways:

• Multiple passes through an image cause the entire image to be paged into memory multiple times. Since in many cases the images are large compared with available physical memory, and the application is often working with multiple images simultaneously, this significantly impairs performance.

Often a pixel operation can be performed in a single CPU clock cycle, so the time spent getting to the data far outweighs the time needed for the operation. Imaging is often a worst case of I/O bound processing.

• Many combinations of atoms can be performed in a single logical step with little penalty. For example, in the case of a rotation followed by a zoom, the backward-mapped algorithms often used to perform the rotation can perform both operations in nearly the same time as the rotation alone.
• The application must often create temporary images to hold intermediate results. Such intermediate images are not needed in customized code and may be avoided if the operations can be combined.

The XIL Library Method

There are several methods that can address these problems. In the XIL library, we have chosen to implement deferred execution and multiple atomic operation replacement. In this approach, the core-layer code keeps track of image dependencies and causes the operations to occur as late as possible. This enables significant performance improvements as described below.

In the library, atomic functions are, by default, deferred as long as possible. To implement this, the API level function creates an instance of the XilOp class, adds the API parameters to the XilOp, and then places the operation on a tree-like structure that holds deferred operation information. void is then returned to the calling routine (the C binding in this case).

The deferred execution data is stored as a directed acyclic graph (DAG), where the nodes are the instances of the XilOp class described above. The fact that a destination function depends on its sources is stored, along with the operation and parameters necessary to produce the destination image once the sources are produced. As image results are needed, the parts of the graph that hold that information are evaluated. Their dependent images are generated by performing the operations that have been stored.

Several actions can cause the evaluation of a subgraph:

• The reuse of an image on which other images depend
• The use of a destination image that has the member synchronized set
• A call to xil_set_synchronize() that turns on synchronization for an image, or for another image that depends on that image.

The DAG is disassembled upon a call to xil_close().

Graph Evaluation and Molecules

When the graph is evaluated, each node's op (the operation used to produce the node's destination image) is available and could be used to index into a list of function pointers. In fact, the library does something a little more general than this, and thus gains the ability to accelerate combined operations.

The XIL library stores its function table as an array of trees, each tree having one of the atomic functions as its base. Branches exist from the base node describing each composite operation (molecule) that exists. This structure is built from the description of the contents of each compute device handler (described below). As the core code looks at the DAG, it attempts to match the longest sequence of atoms in the DAG to the function table. If the needed molecule is available, it is called; otherwise, the sequence of functions is checked again, leaving off the last function, which is performed atomically.

Each node on the function tree is a list of possible functions, usually using different compute devices. The core code calls the first function, which is assumed to be the optimal (accelerated) one. The accelerated function is allowed to fail gracefully, in which case the second function in the list is called. Typically, the last function in the list is the unaccelerated memory port, which is guaranteed to work for all cases. This construction allows an IHV to accelerate a function for only a subset of the input parameters. For example, the code supporting an accelerator that only scales images up can fail gracefully (and cause the memory function to be called) if the scale factors it pulls off the DAG are less than unity. The mechanism for inserting a function into the table is described later in this document.

The core code does not require porting.

Device porters can accelerate either atomic functions or molecules.

Some Considerations

The time needed to determine the sequence of operations from the DAG and choose the appropriate function from the table appears to be trivial compared to typical image operations.

Not all operations can be deferred. An example of this is the xil_extrema() function, which supplies the maximum and minimum image values. The library makes no effort to hide the values returned in an opaque structure, the contents of which could be deferred. Thus, the use of xil_extrema() causes an evaluation of the source image. In general, only the functions that have as their destination an XilImage or XilCis object (or create those objects) can be deferred. The complete list of the rules for deferred execution is as follows:

1. Functions that return information based on values in the current image cannot be deferred. These functions are:

  xil_choose_colormap()
  xil_compress_colormap()
  xil_extrema()
  xil_generate_colormap()
  xil_histogram()
  xil_squeeze_range()

2. Functions that have nonstandard ROI, origin, or size behavior cannot be deferred. These functions are:

xil_affine()
xil_paint()
xil_rotate()
xil_scale()*
xil_subsample_adaptive()
xil_subsample_binary_to_gray()
xil_tablewarp()
xil_tablewarp_horizontal()
xil_tablewarp_vertical()
xil_translate()
xil_transpose()*

* These operations may be deferred under special circumstances. See XIL Programmer's Guide.

3. General rules that apply to the other XIL functions are as follows:

   a. The source and destination images must have the same ROI.

   b. The source and destination images must have the same origins.**

   c. The source and destination images must have the same width (xsize) and height (ysize).**

d. The source images cannot have the same parent as the destination image.

** The xil_copy_pattern() function is exempt from this rule.

We do not envision a large number of molecules in a typical release. In particular, display(zoom(decompress())) molecules have proven to be advantageous. Other display pipelines (display(zoom()), display(dither(zoom())), etc.) will prove useful. It is possible, however, for a third party to add molecules that particularly benefit its vertical market, without requiring that other software running on the XIL library be modified.

One goal of deferred execution is that the application need never know when functions are actually performed. Asynchronous error reporting allows this to be the case in general. However, some cases are impossible to hide. Consider the case of a frame grabber used as a source to an operation that is done in response to an external signal. In normal operation, the actual grab would be postponed until the dependency tree was evaluated, possibly several steps further in the program. A possible resolution to this is to make the destination of the grab operation synchronized. This causes the grab to occur when the function call is made, but precludes any optimization of the grab function. In the end, the application must choose whether the operation should be deferred or not, and when the synchronization should occur. With the general rule "no optimization through synchronization," the application writer can judge an appropriate place to synchronize.

Molecules must behave semantically like the sequence of atomic operations, and produce the same (or nearly the same) results as calling the individual atomic functions. A molecule cannot have a greater precision than the atomic functions that the molecule contains.

Unusual Effects of Deferred Execution

One effect of deferred execution is that in some cases source code may not accurately reflect the actual operations done. Consider the following case, where im2 is not set to be synchronous, but display_image is set to be synchronous:

for (i=0; i<N; i++) { a[0] = i; xil_add_const(im1, a, im2); } xil_copy(im2, display_image);

In the XIL library, only one add (the last one) is done as a result of this code, since the earlier results are obscured by the later ones. If the final copy were not called, no evaluation of the add would take place at all. In normal code, such cases rarely arise, but one must be careful in benchmarking the library. This is not unlike the situation that occurs with optimizing compilers.

Consider another case where only the final decompress is executed.

while (xil_cis_has_frame(cis)) { xil_decompress(cis,im2); } xil_copy(im2,display_image);

Each call to xil_decompress() schedules a frame from cis to be decompressed into image im2. This destination image is not used until the decompress loop is exited. The last decompressed frame is copied to a display image; this is the only operation that is evaluated.

Core Layer Classes

There are a few classes that are defined as part of the core layer. The class XilOp is created, and its members are set, in the API layer. It is used, but not modified, in the DD code.

Table 1-3

Class Name	Definition
XilOp	The class that holds the information required to store the operation on the DAG
XilOpTreeNode	Defines the tree node for deferred execution

The XilOp Class

The class XilOp contains the information that is stored in the DAG representing a specific XIL operation. The source and destination images of atomic functions must be accessible to the operation. These images are stored in the XilOp object. Atomic operations may have up to three source images, depending on the function that performs the operation. For example,

xil_copy (source1, dest);
xil_multiply (source1, source2, dest);
xil_blend (source1, source2, dest, source3);

You can extract the source and destination images of an operation using the following XilOp member functions:

• getSrc1(), get pointer to source image 1
• getSrc2(), get pointer to source image 2
• getSrc3(), get pointer to source image 3
• getDst(), get pointer to destination image

In the case of xil_compress, the source is an image and the destination is a CIS. Therefore, you would use getDstCis() instead of getDst() to get the pointer to the destination CIS. Likewise for xil_decompress, the destination is an image and the source is a CIS. Therefore, you would use getSrcCis() instead of getSrc1()to get the pointer to the source CIS.

The parameters of XIL functions also are stored in the XilOp object. You can extract them by using the following XilOp member functions:

• getLongParam()
• getPtrParam()
• getFloatParam()
• getObjParam()

The number of image sources supported by an XIL operation and the XilOp member functions that you must use to extract the image sources and to extract an XIL function's parameters from the XilOp object are listed in Appendix C, "XilOp Object."

The arguments to each element (atomic or molecular) in a compute device handler are op and op_count. For example,

int XilDeviceComputeTypeMemory::Add8( XilOp* op, // a pointer into the DAG int op_count); // the number of combined operations to be // performed

For an atomic operation, op_count is always 1.

Let's look at an example of extracting from the XilOp object the source image, the destination image, and the parameters for the function xil_fill(). This function has one source image, a destination image, and four parameters.

source = op->getSrc1(); dest = op->getDst(); xseed = op->getFloatParam(1); // get float value xseed yseed = op->getFloatParam(2); // get float value yseed boundary = (float *)(op->getPtrParam(3));// get &boundary[0] fill_color = (float *)(op->getPtrParam(4));// get &fill_color[0]

A molecule is a chain of atomic operations. For a molecule, the chain must be followed properly to extract in a logical order the parameters and images from the XilOp object. For a molecule, the op_count parameter specifies how many atomic operations exist along the chain for the molecule. This is necessary because each molecule might contain a different number of operations.

The op passed to the routine is the op associated with the last operation in the chain (the operation that writes its output to a destination image). The molecule must extract the destination image and the function's parameters from the XilOp object. Then, to move up the chain of operations, you can use the following functions:

• getOp1(), get pointer to op that has source1 as its destination
• getOp2(), get pointer to op that has source2 as its destination
• getOp3(), get pointer to op that has source3 as its destination

For more information about molecules and a chain of operations, see the section "Adding a New Molecule" on page 130 of Chapter 4.

Part of the definition of the XilOp class is shown below:

Code Example 1-20 Definition of the XilOp Class (1 of 3)

union Xil_param { // union structure for op parameters long l; float f; void *p; XilObject* o; }; class XilOp { public: XilImage *getSrc1 (); // get the first input image XilImage *getSrc2 (); // get the second input image XilImage *getSrc3 (); // get the third input image XilImage *getDst (); // get the destination image XilCis *getSrcCis (); // get a pointer to the source cis XilCis *getDstCis (); // get a pointer to the destination cis XilOp *getOp1(); // get the first operation XilOp *getOp2(); // get the second operation XilOp *getOp3(); // get the third operation unsigned int getOp (); // get the op number

Code Example 1-20 Definition of the XilOp Class (2 of 3)

Code Example 1-20 Definition of the XilOp Class (3 of 3)

// NOTE: the parameters must be fetched in the format that they were stored long getLongParam (int n); // get the nth parameter as a long float getFloatParam (int n); // get the nth parameter as a float void *getPtrParam (int n); // get the nth parameter as a pointer XilObject* getObjParam(int n); // get the nth parameter as an XilObject // function to cause the op to be executed void flush(); };

The XilOpTreeNode Class

The XilOpTreeNode class contains the data structure and member functions that store the descriptions of compute device capabilities. The use of the member functions of this class is handled automatically by the creation of the describeMembers.cc routine generated by the xilcompdesc program. See page 128 (step 3) for a discussion of xilcompdesc. The core code adds new functions to the tree using the addFunction() member when it loads various computation modules. In general, the device-dependent code should not need to access this class explicitly.

Each XilOpTreeNode contains both a list of function pointers that implement the (possibly combined) operation that the node represents and a pointer to other XilOpTreeNode objects that perform more combined operations. So, for example, the XilOpTreeNode for "multiply" will point to the list of function pointers that perform the atomic "multiply" operation, plus potentially a set of pointer to other XilOpTreeNode objects that can perform a multiply followed by an add, a multiply followed by another multiply, and so on.

Part of the definition of the XilOpTreeNode class is shown below:

Code Example 1-21 Definition of the XilOpTreeNode Class

class XilOpTreeNode { public: // Each of the following three functions returns a pointer to the // XilOpTreeNode that contains the list of function pointers that // perform the operation described by the 'operator_name' character // string. Combined operation nodes are accessed by successive branches. XilOpTreeNode* branch(char operator_name[]); XilOpTreeNode* branch2(char operator_name[]); XilOpTreeNode* branch3(char operator_name[]); int addFunction(XilDeviceComputeType* compute_type,// add a new function MemberFuncPtr member_func); // to the current XilOpTreeNode // that will implement the // operation that this node // describes int addMarker(void); // for multibranch molecules, the end points of // each branch other than the right-most branch // terminates in a marker instead of a function Xil_boolean removeFunctions(XilDeviceComputeType* compute_type); // remove all the functions that have been // inserted in the tree for a particular compute // device. This provided so that the partial // loading of a compute device does not happen. };

XIL GPI Layer

The GPI (Graphics Porting Interface) layer is the interface for device-dependent code. In general, porting a device to the XIL library requires subclassing one or more of the base device classes defined below, and then configuring the resulting object files so that they can be loaded at run-time by the library. In addition to enabling third parties to port hardware, the functions and device access in the standard XIL release are provided through this interface as well.

In the XIL library, there are classes that define four types of devices:

• I/O Devices
• Compute Devices
• Storage Devices
• Compression Devices

These devices are represented by the GPI layer classes, which are discussed individually in Chapters 3 through 6:

Table 1-4

Class Name	Definition
XilDeviceType	The base class for the device type object
XilDevice	The base class for the device object
XilDeviceComputeType	The abstract class for compute devices
XilDeviceInputOutputType	The abstract class for I/O devices
XilDeviceInputOutput	The device-specific base class for I/O
XilDeviceStorageType	The abstract class for storage devices
XilDeviceStorage	The device-specific base class for storage
XilDeviceCompressionType	The abstract class for compression devices
XilDeviceCompression	The device-specific base class for compression

I/O Devices

I/O devices include any devices that can produce or display images, such as frame grabbers, image files, and displays. Configured I/O devices appear as "device images" to XIL applications, and may be used as sources and destinations for all XIL imaging operations. These devices are described in Chapter 3, "I/O Devices."

Compute Devices

Compute handlers contain the device-dependent implementation of one or more atoms or molecules. For example, a compute device might implement the geometric operators accelerated by an add-on card, or might provide a combination of frequently used functions in the form of a molecule. A compute

device may be hardware specific, or may be a software-only implementation of a superior algorithm. Compute handlers are loaded during the first call to xil_open(). These handlers are described in e."

Storage Devices

Storage devices allow images to reside in other places besides host CPU memory. Such a device is typically associated with a compute device, allowing an accelerator to take advantage of image data remaining local to the accelerator during sequential function calls.

The handlers for storage devices are responsible for allocating, deallocating, and describing the data format of the storage on their devices. They are also responsible for data conversion between storage devices. In addition, it is useful to have the storage handler perform single-pixel access for xil_get_pixel() and xil_set_pixel() to avoid having to convert image data in those cases.

Typically, a compute device handler will cause the storage device handler for the device to be loaded when it first tries to create an image on the device. The CPU memory storage handler is loaded at the time of the first image creation. Storage devices are discussed in detail in Chapter 5, "Storage Devices."

Compression Devices

Compression devices contain most of the utility functions for implementing a method of compression and decompression, even though the actual compress and decompress functions are provided in an associated compute device handler. The compression device performs buffer management and implements the semantics of the XilCis object. A compression device for a specified compressor is loaded when xil_cis_create() is called. Compression devices are discussed in Chapter 6,

"Compression/Decompression."