About Compute Devices

Compute devices implement XIL image processing operations. The computation can take place on the CPU or on an auxiliary image processing board.

About Compute Devices	page 93
Implementing an XIL Function	page 93
Loading Compute Handlers	page 94
Compute Device Handler- Basic Structure Variations	page 100

Implementing an XIL Function

---

Note - The values produced by the implementation of an XIL function should match as closely as possible the values produced by the memory port. This has several implications. Molecules must behave semantically like the sequence of atomic operations, and produce the same (or nearly the same) results as calling the individual atomic functions. A molecule cannot have a greater precision than the atomic functions that the molecule contains. For many of the simple functions, any difference from the default version should not be tolerated. More complicated operations, where there are many floating-point or fixed-point calculations done for each pixel, do not always allow pixel-for-pixel accuracy with any sort of reasonable code. Often, new algorithms provide

slightly different values. It is up to the implementor of the algorithm to make sure that there are no systematic differences between the new implementation and the old one.

---

The XIL Test Suite can aid you in verifying new implementations of XIL functions. The XIL Test Suite enables you to perform regression tests of new code against verified reference signatures and includes the capability of specifying a tolerance for the comparison. This test suite is described in a separate document, XIL Test Suite User's Guide, which is part of this release.

Error handling in an implementation is performed by calling the macro XIL_ERROR. Both compute handler examples use this interface. The method used to add error messages for device-dependent errors is discussed in Chapter 2, "More on Writing Device Handlers."

Loading Compute Handlers

To provide for maximum flexibility in adding software device handlers, your compute device handler is dynamically loaded as a shared object at run time. The list of loadable objects (device handlers) is maintained in a configuration database file called config. You must edit the config file using scripts to add entries for compute device handlers that will be used by XIL.

XIL merges the two files from /etc/openwin/lib/xil/config and /usr/openwin/lib/xil/config. If duplicate information between the two files exists, the information in /etc/openwin/lib/xil/config takes precedence. You should edit the /etc/openwin/lib/xil/config file only.

This section explains how to create entries for compute device handlers, as well as how to add and delete these entries from the config file.

config Entry

This example shows a config text file entry for a dynamically loadable compute device handler.

#Start STOCKTICKERdevicename handler class="XIL-COMPUTE" name="SOCKTICKERdevicename" priority="500" dependencies="STOCKTICKERiodevname"; #End STOCKTICKERdevicename handler

Formatting Guidelines

The config file has some basic formatting guidelines.

• Any characters following the pound character (#) through the end of a line are treated as a comment and are disregarded when the file is read.
• Quotation marks around value strings are required only if the string contains delimiters such as white space or a semicolon (;).
• The back slash character (\) can be used as an escape character. For example, \" is used to include the double quotation mark character as part of a string value.
• Parsing routines strip the quotation marks surrounding string values and pass the string only to the underlying software. The parsing software treats all values as strings; interpretation of the string value is up to the device handler.

By convention, the strings 'Start' and 'End' indicate the beginning and end of the database definitions for a set of loadable device handlers. In the example above (see "config Entry" on page 94) the string 'STOCKTICKERdevicename handler' identifies the STOCKTICKER device handler. The contents of the string you use are up to you. You use this string in a script file to append or delete your entry from the config file.

Between the 'Start' and 'End' strings are one or more loadable compute device handler entries for XIL. Each compute device handler description consists of up to four "attribute=value" pairs separated by white space (including tabs, spaces, and new line characters) and terminated by a semicolon.

Table describes each attribute=value pair.

Table 4-1

Attribute=	Meaning=
class=	This value is always XIL-COMPUTE, which uniquely identifies the class of compute device handlers for XIL.
name=	This value is the name of the compute device. It uniquely identifies the instance of the class object in the config file and is different for each device handler. XIL derives the actual filename from this string as xilcompute_name.so.2.
priority=	This field is used by XIL to determine which compute device function to call when there are overlaps. Most compute devices will implement an atomic function already implemented by the default XIL memory pipelines. For example, a medical imaging pipeline may be accelerated single band "lookup 16->8" and no other lookup routines. Since it is a specialized routine taking advantage of hardware or a clever algorithm, you will want it called before the memory device routine. The priority for the memory pipelines are set at 100. We recommend device handler providers specify a priority of 500 unless they're aware of other devices. A priority of "-1" will turn off the named pipeline such that it will not be loaded by XIL.
dependencies=	These are the names of I/O or compression devices which must be loaded before the described compute device can be loaded.

Using Script Files

Before your compute device handler can be loaded, you need to add an entry in config. To add (or delete) config entries, you should create an executable script file.

Appending An Entry

You add entry to config by appending the entry to the file. The following procedure illustrates this process.

1. Create an entry.

   Use the example below as a template to create your entry, filling in values for your device handler. Replace the string associated with 'Start' and 'End' with a string of your choice.

#Start STOCKTICKERdevicename class="XIL-COMPUTE" name="STOCKTICKERdevicename" priority="500" dependencies="STOCKTICKERiodevname"; #End STOCKTICKERdevicename

2. Save your entry.

   Save locally as config.

3. Create a script file ins.config to append your entry. Use the script file shown below as a template.

#! /bin/ksh # # Installation script for the config class # If a config file existed, remove any entry belonging to # this package, and append a new entry # echo $1 echo $2 chmod 644 $2 if [ -r $2 ] then #It is editable by this script. Edit it. cp $2 /tmp/$$config || exit 2 sed -e "/# Start STOCKTICKERdevicename/,/# End STOCKTICKERdevicename/d" \ /tmp/$$config > $2 || exit 2 cat $1 >> $2 || exit 2 rm -f /tmp/$$config else #A config file was not present cat $1 >> $2 ||exit 2 fi chmod 444 $2 exit 0

4. Execute the script file.

   First you need to become superuser. The executable script file ins.config takes two arguments: the relative path to the local text file entry (config) and the full path to the system config file

(/etc/openwin/lib/xil/config).

As the script file executes, it displays the values (config and /usr/openwin/lib/xil/config) of the two arguments passed to it.

% su Password: # ./ins.config /etc/openwin/lib/xil/config config /etc/openwin/lib/xil/config # ^D

Removing An Entry

To remove a config entry, use the following procedure.

1. Use the script rm.config below as a template. Replace the string 'STOCKTICKERdevicename' with the string for the entry you want to remove. Save the script with a name such as rm.config, and make the file executable.

#! /bin/ksh # # Removal script for the config class # Remove entries that belong to this device handler # Delete the file if it is empty # echo $1 chmod 644 $1 sed -e "/# Start STOCKTICKERdevicename/,/# End STOCKTICKERdevicename/d" $1 > \ /tmp/$$config || exit 2 if [ -s /tmp/$$config ] then mv /tmp/$$config $1 || exit 2 else rm $1 || exit 2 fi chmod 444 $1 exit 0

2. Execute the script.

   First become superuser. In this case, the script file rm.config takes one argument: the path to the system config file. As the script executes, it displays the value (/etc/openwin/lib/xil/config) of the single argument passed to it, as shown here.

% su Password: # ./rm.config /etc/openwin/lib/xil/config /etc/openwin/lib/xil/config #^D

Compute Device Handler- Basic Structure Variations

Compute device handlers follow the same basic structure as all operations in the XIL library. (See "Implementing an XIL Operation" on page 52 for a description of this basic structure.) Compute handlers also include special case operations such as convolution and geometric operations that involve variations on that basic structure.

The following sections take you through basic structure variations that are required for special case compute operations. Operations are presented in the following categories:

• Data collection (for example, xil_histogram(), xil_extrema())
• Area-based (for example, xil_convolve(), xil_erode())
• Geometric (for example, xil_affine(), xil_scale())

Data Collection Operations

Data collection operations such as xil_histogram() take an image and generate data from that image to create a histogram. These operations operate in a multi-threaded fashion by having each compute routine produce data for the box list that they are handed and then reporting that data to the XilOp object which is responsible for collecting the data into a single entity and reporting the entity to the API user. The XIL core is responsible for coordinating the multiple calls to the compute routine and reporting the collected results back to the API user.

The compute routines report the data they have collected to the XilOp object using the XilOp::reportResults() function. XilOp::reportResults() takes a variable number of void* arguments, the types of which vary based on the compute routine. See Appendix A, "XilOp Object," for details on the parameters passed back for reportResults(). The example below illustrates how the histogram compute routine uses this function.

// // Allocate and init histogram data array // unsigned int nElements = histogram->getNElements(); unsigned int* data = new unsigned int[nElements];

// // Code deleted for brevity XilStatus status = op->reportResults((void**)&data);

Area-Based Operations

This section covers area-based operations such as xil_convolve(), xil_erode(), xil_dilate(), xil_fill(), and xil_error_diffusion(). Unlike point-based operations such as xil_add()which generate a single destination pixel from a single source pixel, area-based operations generate a single destination pixel from multiple source pixels.

To do this, the compute routine needs to access source data outside of the pixel area defined by the operation.

XIL accomplishes this by modifying the source boxes passed in to the compute routine in the XilBoxList to ensure that enough storage exists to access all needed source pixels.

In a point-based operation, the source box represents the source pixels that correspond to the destination pixels being touched. In an area-based operation, the source box represents the area that corresponds to the destination pixels being touched--with an additional border region to provide the source pixels for processing the edge pixels of the destination area when there is enough data in the source image to do so.

The various area-based operations have slightly different requirements, each of which is detailed in this section.

Convolution, Erode, and Dilate

Convolution, erode, and dilate operations use an XilKernel or an XilSel object to describe how to combine source pixels in the destination pixel. For these operations, the call to XilOp::splitOnTileBoundaries() produces source boxes labeled as one of the following types:

• XIL_AREA_TOP_LEFT_CORNER
• XIL_AREA_TOP_EDGE
• XIL_AREA_TOP_RIGHT_CORNER
• XIL_AREA_LEFT_EDGE

• XIL_AREA_CENTER
• XIL_AREA_RIGHT_EDGE
• XIL_AREA_BOTTOM_LEFT_CORNER
• XIL_AREA_BOTTOM_EDGE
• XIL_AREA_BOTTOM_RIGHT_CORNER

Generation of these types is based on the position of the box within the source image, the dimensions of the XilKernel or XilSel object, and its key value.

As an example, Figure 4-1 shows a 5x5 kernel with a key value of 1,1. Edges represent the distance in each direction from the key value to the edge of the kernel. The edges determine how much extra storage is needed outside a given box and are referred to by their position: left, right, top, and bottom.

Figure 4-1

Figure 4-2 illustrates a source image and shows all nine box types and their positions within the image. This layout is for the kernel shown in Figure 4-1.

Figure 4-2

---

Note - You cannot go (nor should ever need to go) outside the destination box.

---

Center Boxes A center box (that is, XIL_AREA_CENTER) represents the simplest case for processing. A center box represents a source box which provides the required storage on all sides of the box. (See Figure 4-3.)

Figure 4-3

Convolution and Edge Conditions With convolution, the user may select one of the following edge condition options:

• XIL_EDGE_EXTEND
• XIL_EDGE_NO_WRITE
• XIL_EDGE_ZERO_FILL

See the XIL Programmer's Guide for more information.

When the API user selects the XIL_EDGE_NO_WRITE option, the XIL core only provides boxes of the XIL_AREA_CENTER type. The other two options produce edge and corner boxes for processing.

---

Note - As a protection against future versions of XIL that may add other edge conditions, the compute routine should test for and fail if it encounters an edge condition other than XIL_EDGE_EXTEND, XIL_EDGE_NO_WRITE, or XIL_EDGE_ZERO_FILL.

---

In Figure 4-3, the solid line box shows the pixel area in the source, and the dashed line shows the storage available outside the box. When requesting the storage information for the box, the XIL core sets the data pointer at the top left hand pixel coordinate. The compute routine can then be guaranteed that it can place the kernel key in the top left hand corner of the box to start computing the destination pixel, knowing also that when it reaches the bottom edge of the box, source storage is available to correctly compute the destination area. No edge handling is needed for this case.

Edge Boxes An edge box (XIL_AREA_TOP, XIL_AREA_BOTTOM, XIL_AREA_LEFT, XIL_AREA_RIGHT) differs from a center box in that it only has source storage available on three sides. Figure 4-4 represents an XIL_AREA_LEFT_EDGE box

Figure 4-4 XIL_AREA_LEFT_EDGE

Again, the solid line represents the pixel area and the dashed line, the provided storage. The box follows the edge of the image in one dimension (unless limited by a ROI) and is the width of the appropriate kernel edge in the other. In the case of the XIL_AREA_LEFT_EDGE box, the height of the box (assuming no ROI) is the height of the image minus the TOP_LEFT_CORNER and BOTTOM_LEFT_CORNER boxes. The width of the box is 1.

The data pointer again is returned at the top left pixel coordinate but, in this case, no storage is available for the left hand side of the kernel.

The compute routine must process the pixels represented by this box according to the rules of the operation or, in the case of convolution, according to the edge condition selected by the API user.

Corner Boxes A corner box (XIL_AREA_TOP_LEFT_CORNER,

XIL_AREA_TOP_RIGHT_CORNER, XIL_AREA_LOWER_LEFT_CORNER,
XIL_AREA_LOWER_RIGHT_CORNER) is a degenerate case of an edge box. Two
sides of a corner box cannot provide enough source data for the pixel area to
be processed. Figure 4-5 illustrates the case of XIL_AREA_TOP_LEFT_CORNER.

Figure 4-5

A corner box is guaranteed to be the size of the kernel edges for the given corner. In the case of Figure 4-6 with the kernel defined in Figure 4-1, the size of the XIL_AREA_TOP_LEFT_CORNER is 1x1.

As with edge boxes, the compute routine is responsible for processing the box according to the rules of the operation.

Getting The Box Type To get the type of the box, use the getBoxTag() function. This function returns a void* which, when cast to an XilAreaBoxType, provides the box type. The same tag is applied to both the source and destination box.

XilBoxAreaType tag = (XilBoxAreaType) src_box->getTag(); switch (tag) { case XIL_AREA_TOP_LEFT_CORNER: case XIL_AREA_TOP_EDGE: case XIL_AREA_TOP_RIGHT_CORNER: case XIL_AREA_RIGHT_EDGE: case XIL_AREA_CENTER: case XIL_AREA_LEFT_EDGE: case XIL_AREA_BOTTOM_LEFT_CORNER:

XilBoxAreaType tag = (XilBoxAreaType) src_box->getTag(); case XIL_AREA_BOTTOM_EDGE: case XIL_AREA_BOTTOM_RIGHT_CORNER: break;

Although destination boxes have tags, the tag only has meaning with regard to the source box.

Performance Considerations It is optional to call XilOp::splitOnTileBoundaries() at the beginning of a compute routine. However, this has additional ramifications in area operations.

In the general case, splitOnTileBoundaries() ensures that all source boxes in the XilBoxList do not cross tile boundaries. To do so, it may have to split the passed-in boxes into smaller regions. This means, when getStorage() is subsequently called for each set of boxes in the box list, the core never has to cobble tiled regions of data for access by the compute routine.

In the area-operation case, splitOnTileBoundaries()not only splits the boxes along source tiles, it secondarily splits those boxes as needed to indicate edge or corner conditions.

To process pixels in the destination corresponding to source pixels that lie along source tile boundaries, the compute routine must access pixels from the neighboring tile. To minimize cobbling, splitOnTileBoundaries()generates boxes just large enough to handle these tile edges. As long as these boxes do not lie on source image edges, the boxes are tagged as XIL_AREA_CENTER and are processed by the compute routine as any other center box.

If the compute routine chooses not to call splitOnTileBoundaries(), the box passed in represents the entire destination region to be processed and the corresponding source region, regardless of source tiles. None of the boxes are tagged, and it is up to the compute routine to identify those destination pixels for which there may not be enough source data. In such cases it should not try to access outside of the source image.

Kernel Inversion In the case of convolution, the kernel that the user generates is inverted before being passed to the compute routine. This is a difference from previous XIL releases in which the compute routine was responsible for inverting the kernel.

---

Note - This is convenient for the compute routine since the kernel is stored on the op in the form needed for convolution processing.

---

Fill and Error Diffusion

Fill and error diffusion operations are handled in a similar manner. Currently these operations are called from a single thread. The destination box supplied is the same as for the basic case (that is, it corresponds to the destination area to be written). The source box pixel area corresponds to the basic case with the exception that the storage is available for the entire source image. This makes it possible for the compute routine to always have access to the entire source image data. The data pointer returned from

XilStorage::getStorageInfo() points to the box pixel location. The box location within the image can be retrieved using the XilBox::getAsRect() function. Figure 4-6 illustrates the source box setup for fill and error diffusion.

Figure 4-6

Geometric Operations

Geometric operations such as affine, transpose, rotate, scale, translate, and tablewarp manipulate the image in some geometric fashion. This means that the mapping of a destination pixel to a pixel in the source is dependent on the parameters of the operation. Implementation of each of these functions varies from the implementation of the basic case described in "Basic Structure: Atomic Function" on page 54.

This section first discusses the transpose operation as it introduces the concept common to all geometric compute routines of backward mapping from a destination area to get the corresponding area in the source. The affine operation builds on this concept and introduces two new objects, the XilConvexRegionList and the XilScanLineList.

The tablewarp operation is the most complex. The compute routine is required to handle almost all the computation, because the XIL core has no knowledge of how a destination pixel maps back to the source.

Transpose

The transpose operation is a point operator in the sense that it requires only one source pixel to generate each destination pixel. However, which source pixel corresponds to which destination pixel is dependent on the flip-type provided to the operation.

The source box passed in to the transpose routine maps exactly to the destination box. The compute routine then needs to calculate the mapping of the pixels within one box to the pixels in the other. The compute routine may choose to map destination to source pixels for each flip-type explicitly or to use the XilOp::backwardMap() routine provided by XIL for convenience, as shown here.

The full interface can be found in _XilOp.hh

// // Backward map a single point in destination box space to the // corresponding point in source box space. The last (optional) // argument indicates which source to backward map into. // XilStatus backwardMap(XilBox* dst_box, float dx, float dy, XilBox* src_box, float* sx, float* sy, unsigned int src_number = 1);

As in the base case (see the subsection entitled"Step 5: Processing the Data" on page 59" in "Basic Structure: Atomic Function"), the transpose routine generates an XilRectList from the passed-in ROI and destination box. It is then responsible for correctly copying the equivalent source rectangles into the destination based on the flip type. As before, the data pointers returned in the storage object from XilImage::getStorage() are relative to the upper left corner of each of the boxes.

The following example shows how the backwardMap() function could be used in the memory transpose compute routine to get the correct source area.

// // Create a list of rectangles to loop over. The resulting list // of rectangles is the area created by intersecting the ROI with // the destination box. // XilRectList rl(roi, dst_box); // // loop over the list of rectangles // while (rl.getNext(&dstR_x, &dstR_y, &dstR_xsize, &dstR_ysize)) { // // The rectangle in the list applies to the dst, so we have // to find appropriate rectangle in the src according to // the fliptype. // // The op does the backward map for us. // { float srcx; float srcy; op->backwardMap(dst_box, dstR_x, dstR_y, src_box, &srcx, &srcy); srcR_x = (int)srcx; srcR_y = (int)srcy; } src_scanline = src_data + (srcR_y * src_scanline_stride) + (srcR_x * src_pixel_stride); dst_scanline = dst_data + (dstR_y * dst_scanline_stride) + (dstR_x * dst_pixel_stride); // // Note that the compute routine will still have to take into // account incrementing the source in the appropriate // direction for the flip-type. //

Affine

In an affine operation, backward mapping a rectangle from the destination to the source may generate a region that is not a rectangle. In order to support this, the XIL GPI represents an ROI as an XilConvexRegionList.

A convex region is defined as a convex polygon whose points are stored as floating-point values. The convex region is represented as two arrays of floating-point values: one for the x-points and the other, forthe y-points. A point_count indicates the length of the arrays, as shown here. The full interface can be found in _XilConvexRegionList.hh.

// // Construction using a roi and a box. The clipped regions in // the list are all relative to (0, 0) in the box // XilConvexRegionList(XilRoi* roi, XilBox* dest_box); // // Get the next convex region on the list // Xil_boolean getNext(const float** x_array,const float** y_array, unsigned int* point_count); // // Allows the original full ROI convex region list to be // clipped by a different box // XilStatus reinit(XilBox* dest_box);

As with the XilRectList, we recommend creating the XilConvexRegionList on the stack to minimize the use of new() and delete() within the compute routine. The compute routine then loops through all convex regions in the list processing each one, until XilConvexRegionList::getNext() returns NULL.

To process a given region, the compute routine must backward map the destination region to the equivalent source region. This can be done by the compute routine with the affine matrix on the op, or it can be done with XilOp::backwardMap().

As with area operators, the source boxes provided to the compute routine have enough storage associated with them to provide all the data for the given interpolation type. The destination box is small enough to guarantee enough source storage is available for interpolating along all destination edges.

Once the compute routine has obtained a destination region and mapped it to the corresponding source region, it is responsible for moving the data pointers to the correct positions within the boxes and for generating the destination pixels using the appropriate interpolation kernel.

For convenience, XIL provides the XilScanLineList object, which can be used to turn a convex region into a list of scanlines, as shown here. The full interface can be found in _XilScanlineList.hh.

XilScanlineList(const float* x_array,const float* y_array,unsigned int num_points); // // Get the next scanline. // Xil_boolean getNext(int* y, float* x1, float* x2); Xil_boolean getNext(unsigned int* y, unsigned int* x1, unsigned int* x2); // // Return the number of scanlines in the list // unsigned int getNumScanlines();

The following example show the use of of the XilConvexRegionList and the XilScanLineList.

// // Loop over convex regions in the destination. // XilConvexRegionList crl(roi, dst_box); unsigned int num_pts; const float* dst_xarray; const float* dst_yarray; while(crl.getNext(&dst_xarray, &dst_yarray, &num_pts)) { // // Create scanline list // XilScanlineList dst_scanlines(dst_xarray, dst_yarray, num_pts); // // Loop over scanlines. // unsigned int y; unsigned int dst_scan_start; unsigned int dst_scan_end; while(dst_scanlines.getNext(&y, &dst_scan_start, &dst_scan_end)) { // // Backward map the two endpoints of the line // op->backwardMap(dst_box, (float)dst_scan_start, (float)y, src_box, &start_x, &start_y); op->backwardMap(dst_box, (float)dst_scan_end, (float)y, src_box, &end_x, &end_y); // // process destination line, walking the source line //

Kernel Definitions The edge areas available outside the source pixel retion are defined as follows:

1. Nearest neighbor: Left = Right = Top = Bottom = 0

2. Bilinear: Left = Top =0

3. Bicubic: Left = Top = 1

4. General: Values are calculated based on the size of the horizontal interpolation table width and size of the vertical interpolation table height using these equations.

unsigned int keyX = (width - 1)/2; unsigned int keyY = (height - 1)/2; Left = keyX; Right = width - (keyX + 1); Top = keyY; Bottom = height - (keyY + 1);

Performance Considerations The destination box does not cross-tile boudnaries. The corresponding source box may cross tile boundaries, and the compute routine may choose to call XilOp::splitOnTileBoundaries() to split the boxes in the original box list along source tile boundaries. Because of the area-based nature of affine operations, some source boxes must cross source tile boundaries. These boxes are made as small as possible to minimize source data cobbling.

---

Note - XilOpAffine::splitOnTileBoundaries() currently does nothing.

---

Rotate

Rotate is very similar to affine. In fact, the XIL core treats the two operations in the same manner. (See "Affine" on page 112 for details.)

Scale and Translate

Scale and translate operations are special cases of affine that allow you to use rectangles rather than convex regions. Although the compute routine can use XilConvexRegionList, performance can be enhanced if it instead uses XilRectList to generate the destination regions. The compute routine can use XilOp::backwardMap() to generate the equivalent source rectangle.

Tablewarp

In the case of the the xil_tablewarp(), xil_tablewarp_vertical(), and xil_tablewarp_horizontal() functions, the compute routine is responsible for handling all the backward mappings from the destination box through the tablewarp image to the source image.

For tablewarp, the ROI passed in to the compute routine is just the ROI of the destination image. The source ROI is passed in as a separate argument. It is the responsibility of the compute routine to position the warp image correctly on the destination ROI and to take the source ROI into account when calculating the backward mapping position. The source box passed in to the compute routine represents the whole source image, since mapping of destination to source pixels is known.