Skip to ContentSun and Oracle Channel Sun How to Buy Log In

XIL Test Suite User's Guide

Contidos dentro

Localizar Mais Documentação

Destaques de Recursos de Suporte

Fazer download desta apostila em PDF

Running Test Programs in the XIL Test Suite

2

This chapter describes how to run the XIL Test Suite. It explains how the XIL Test Suite works, explains what options you can set when running the test suite, and provides examples of invocations.

Getting Started

Before you begin using the XIL Test Suite, make sure that you have the correct packages of the Solaris Driver Developer's Kit (DDK) installed, that you have the correct resources, and that you set the needed environment variables.

Packaging

You must have the following Driver Developer's Kit packages installed to run the XIL Test Suite and write your own test programs.

SUNWxildk
SUNWxiltg
SUNWxilts, if you are running on SPARC
SUNWxiltx, if you are running on x86

Resources You Need

The XIL Test Suite is supported on SPARC^{^(R)} and x86 platforms with the following minimum configurations:

SPARC: A SPARCstation^{^(TM)} 2 (with 100MB of swap space) running the current version of Solaris system software
x86: A 486 (with 100MB of swap space) running the current version of Solaris system software

Before you can run the XIL Test Suite, you also need the following:

The XIL Imaging Library
SPARC: The SPARCompiler^{^(TM)} C 2.0.1 and SPARCWorks^{^(TM)} 2.0.1 software (if you plan to write your own test programs)

SPARC - Although the XIL Programmer's Guide recommends the SPARCompiler C 3.0 or later compiler for building XIL applications, the SPARCompiler C++ 3.0 compiler cannot be used for writing your own test programs.

x86: The ProCompiler^{^(TM)} C 2.0.1 and ProWorks^{^(TM)} 2.0.1 software (if you plan to write your own test programs)

Environment Variables

Before running the XIL Test Suite, you must set some environment variables. If you used the default installation directory when you installed the XIL library, you probably have already set the XILHOME environment variable as follows:

% setenv XILHOME /opt/SUNWits/Graphics-sw/xil

Assuming that you've installed the Solaris XIL Imaging Library Driver Developer's Kit (of which the XIL Test Suite is a part) according to the defaults, you must set the other needed environment variables as follows:

Set the XILCHHOME environment variable to the location of the current Xilch directory:

% setenv XILCHHOME /opt/SUNWddk/xil/ddk_2.4/Xilch/arch

where arch is either sparc or i386 depending on which platform you are running

Update your LD_LIBRARY_PATH to include $XILCHHOME/lib:

% setenv LD_LIBRARY_PATH \
$XILHOME/lib:$XILCHHOME/lib:$LD_LIBRARY_PATH

Note - The path to the directory that contains the example test programs begins at the same level as the current Xilch directory. Therefore, you can use the XILCHHOME environment variable to locate the examples directory. For example:

% cd $XILCHHOME/../src/tests/examples

Environment Notes

To display decompression tests, you must be running the OpenWindows^(TM) environment. Also, a user cannot use a system to run tests if he or she is not the user logged into the console.

iv>

How the XIL Test Suite Works

As discussed in Chapter 1, "Overview of the XIL Test Suite," the XIL Test Suite has three parts:

Xilch, the master control program, or harness, that runs all of the tests in the test suite.
libts, the test suite library that provides utility functions and functions to generate and verify data. Also, you can write your own test program using these functions.
A set of test programs, which are dynamically linked with functions in libts, that generate test data.

You can run Xilch to:

Verify XIL functions against reference signatures
Verify XIL functions against reference data (for example, images and color maps)
Create reference signatures for XIL functions and place the signatures in a reference directory
Create reference data for XIL functions and place the data in a reference directory

When you create test data, Xilch determines which platform you are running on and generates the test data for that platform. Reference signatures and data may be platform-specific, or they may be generic (same result on multiple

platforms). The SPARC platform is considered the "generic" platform. By default, the reference directory for the SPARC platform is named generic and contains all reference signatures for all test programs.

When you invoke a test program through Xilch, the test program dynamically links with the XIL Imaging Library. If you are verifying an XIL function, test signatures or data are created for the function and compared to the stored reference signatures or data for that function.

Figure 2-1 is an example of how Xilch works. In the figure, the testlist file, a file that contains a list of test programs, begins by starting test1. In this example, test1 calls:

ts_get_src1_image(), which retrieves input data as specified in the text matrix file via the function in libts.
xil_add(), which makes a call to the XIL Imaging Library.
ts_verify(), which verifies that the test data is identical to the stored reference data. The comparison is made through libts. Results of the comparison are displayed on your terminal (stdout). Or, when you invoke Xilch, you can specify that results print to a log file of a specified name.

Note - If you invoked Xilch to create reference data, the test data generated by each test program is placed in a reference directory; no comparison is made.

Figure 2-1 Xilch

Verifying Against References

As discussed in Chapter 1, "Overview of the XIL Test Suite," you can use reference signatures and data to verify the accuracy of XIL Imaging Library functionality. Reference signatures are used primarily for regression testing, and a set of reference signatures for the XIL Imaging Library functions are provided with the XIL Test Suite. Reference data is used primarily for verifying the accuracy of new development code.

When you want to verify against reference signatures, you need not specify an option when running Xilch because the -test_signatures option is the default. When you want to verify against reference data, you specify the -test_data option when running Xilch.

Creating References

Reference data for the XIL Imaging Library functions are not provided with the XIL Test Suite; only reference signatures are provided. However, you can create reference data by running all of the XIL Test Suite test programs with the -create_data option when running Xilch.

Also, if you run the test programs using different test matrix files (input data) than the test matrix files provided with the XIL Test Suite, you will need to create your own set of references (signature and/or data) for the new input data. Test matrix files are discussed in the section "The Test Matrix File" on page 16.

Files Used or Created by `Xilch`

Xilch uses two files and can create one file when it runs. When you invoke Xilch, you can specify which testlist and test matrix file to use. Default versions of both of these files are called automatically if you don't specify them on the Xilch command line.

When you invoke Xilch, you can specify that a log file be created that will contain information reported by Xilch as it runs. By default, this information is displayed at your terminal and not printed to a log file.

The Testlist File

The testlist file is a list of test programs, one per line, and any options for those tests. If you do not specify a testlist file, Xilch first looks for a file called testlist in the current directory; then, it looks in $XILCHHOME/config. The default testlist file contains many of the available test programs. If you want a testlist file that contains all the test programs, you must create your own testlist file.

If you want to use a testlist file other than the default file when running Xilch, you specify the testlist file to Xilch using the -f option. For example,

% Xilch -f my_testlist

If one of the test programs in the testlist file fails, Xilch continues to run through the rest of the programs in the list.

When you create a testlist file, you can specify conditions for a particular test by using Xilch options. For example, suppose you want to use a different test matrix file for a test program named my_test in your testlist file. In this case, after my_test in your testlist file, you would use the -m option to specify the name of the test matrix file you want to use. For example,

% $XILCHHOME/bin/my_test -m my_test_matrix

Then, when you invoke Xilch, you can either use the default test matrix file or specify another test matrix file. When Xilch reaches my_test in your testlist file, it will substitute the test matrix file specified for my_test.

Note - Testlist file options override command-line options.

An easy way to create your own customized testlist file is to start with the testlist file in $XILCHHOME/config. Copy it to a file with a name you choose. Then edit the file, removing the test programs you don't want to run and inserting additional programs. The following table lists the test programs available at the time of the printing of this manual. For the most up-to-date list of test programs, look in the $XILCHHOME/bin directory.

Table 2-1 $XILCHHOME/bin (1 of 4)

XIL Function Type	Test Program
Arithmetic	`arith_test` `logical_test`
Compression/CIS	`cbm_test` `cell_compress_test` `cell_decompress_test` `cellb_compressor_test` `cellb_decompress_test` `fax_test`

Table 2-1 $XILCHHOME/bin (2 of 4)

XIL Function Type

Test Program

jpeg_compressor_test
jpeg_decompress_test
jpegll_test
mpeg1_decompress_test
px64_decompress_test

Geometry

affine_test

rotate_test

scale_test

subsample_adaptive_test

subsample_binary_to_gray_test

tablewarp_test

translate_test

transpose_test

Miscellaneous

absolute_test
band_combine_test
black_generation_test
color_convert_test
combined_lookup_test
convolve_test
copy_pattern_test
copy_test
copy_with_planemask_test
dilate_test
edge_detection_test
erode_test
extrema_test
fb_rw_test

Table 2-1 $XILCHHOME/bin (3 of 4)

XIL Function Type	Test Program
	`histogram_test` `lookup_convert_test` `minmax_test` `photocd_test` `rescale_test` `set_value_test` `threshold_test` `xil_lookup_test`
Molecules	`gx_test` `memory_test`
Object	`dag_test` `dithermask_test` `image_test` `kernel_test` `lookup_test` `roi_test` `roi_get_as_image_test` `state_test` `sel_test`
Presentation	`blend_test` `choose_colormap_test` `error_diffusion_test` `fill_test` `nearest_color_test` `ordered_dither_test` `paint_test`

Table 2-1 $XILCHHOME/bin (4 of 4)

XIL Function Type	Test Program
	`soft_fill_test` `squeeze_range_test`
Utility	`cast_test`

Most of the names of the test programs include the name of the XIL function that's tested. For example, the Xilch test program affine_test verifies the correct functioning of the XIL function xil_affine(). However, the names listed below are not obvious.

`cbm_test`	Tests CIS Buffer Manager functionality
`fb_rw_test`	Tests frame buffer read-write functionality

The Test Matrix File

A test matrix file provides descriptions of input data (images) to Xilch. A test program loops through the test matrix file, retrieving images via libts functions, until it runs out of images. If you do not specify a test matrix file, the default file, xilch_tests (in $XILCHHOME/config), is used. Chapter 3, "Writing Test Programs in the XIL Test Suite Environment," provides an example of a test matrix file.

The test programs for some XIL functions are designed to work with specialized matrix files, which are also in $XILCHHOME/config. For example, the matrix file called fax_tests is designed to be used with the test program that verifies the fax decompression portion of the XIL library.

Note - The images referenced in the test matrix files are located in the directory $XILCHHOME/data/images/std. These images are all in .vff format. This is an unsupported, Sun-internal format. The format of these images may change in future releases.

You specify the fax_tests matrix file by using the -m option in your testlist file. In this case when you invoke Xilch, it uses the default matrix file for every test program in your testlist file except for the fax test program, where it

uses the fax_tests matrix file that you specified in your testlist file. If you decide to add a test program to the XIL Test Suite, you might want to create new test matrix files as discussed next.

Writing a Test Matrix File

Images can be generated from parameters or loaded from a specified file. The functions ts_get_src[123]_image() and ts_get_dst[123]_image() return source and destination images based on the tables defined in the test matrix file (explained in the next section). See Code Example 3-2 for an example of how this is done. A test matrix file contains tags, which specify the images used by a test program.

Note - The images referenced in the test matrix files in $XILCHHOME/config are located in the following directory: $XILCHHOME/data/images/std. These images are all in .vff format. This is an unsupported, Sun-internal format. The format of these images might change in future releases.

Tags of a Test Matrix File

Note - Tags and labels are case insensitive; parameters are case sensitive. All parameters must be specified in the order shown, separated by whitespace; there are no default values. Also, the values of the parameters must be of the correct data type.

Table [SRC1|SRC2|SRC3|DST1|DST2|DST3] xsize ysize nbands datatype imagecontent value xorigin yorigin filename

The Table tag denotes the beginning of a table of image data. The Table tag must be followed by one of the following table labels: SRC1, SRC2, SRC3, DST1, DST2, or DST3. SRC and DST labels correspond to the parameter names in the libts functions ts_get_src[123]_image() and ts_get_dst[123]_image(). Parameters of the Table tag are:

*xsize*	Width of the image.
*ysize*	Height of the image.
*nbands*	Number of bands in the image.

*datatype*	XIL_BIT (unsigned), XIL_BYTE (unsigned), or XIL_SHORT (signed).
*imagecontent*	TS_RANDOM: image filled with numbers from a random number generator (user-specified seed number), TS_CONSTANT: image filled with a user-specified constant, TS_RAMP: image filled with increasing numbers, or TS_NOFILL: image filled with whatever is in memory.
*value*	Value associated with the *imagecontent*. It can be a seed or a constant. This number must be less than 256 for XIL_BYTE TS_CONSTANT images. For XIL_SHORT TS_CONSTANT images, this number must be between -32,768 and 32,767.
*xorigin*	x coordinate of the origin.
*yorigin*	y coordinate of the origin.
*filename*	File name of a stored image file in .vff format. All other parameters in the line, except *xorigin* and *yorigin, are ignored but must appear as placeholders. If you do not specify a prestored image, then you must enter "NULL" for this parameter (see the example). If a file name is specified, its parameters supersede other specified parameters, such as xsize* and *ysize*.

Child <Label> ch_x ch_y ch_xsize ch_ysize ch_startband ch_num_of_bands

The Child tag specifies the characteristics of a child image. A Label name is not required, but can be used as an optional identifier. A Child tag always immediately follows the parameters of its parent image (see example test matrix files). Parameters of the Child tag are:

*ch_x*	x coordinate offset from the parent image.
*ch_y*	y coordinate offset from the parent image.
*ch_xsize*	Width of the child image.

*ch_ysize*	Height of the child image.
*ch_startband*	Specifies which band in the parent image the child image starts in (0 based).

ch_num_of_bands Number of bands in the child image.

ROI roi1_x roi1_y roi1_xsize roi1_ysize roi2_x roi2_y etc...

The ROI tag is used to specify the characteristics of a region of interest. The Label parameter option is not available with this tag. Parameters of the ROI tag are:

*roi1_x*	x coordinate for the first region.
*roi1_y*	y coordinate for the first region.
*roi1_xsize*	Width of the first region.
*roi1_ysize*	Height of the first region.

End <Label>

The End tag indicates the end of a Table. A Label name, corresponding to the Table label, is optional (see test matrix file examples).

Table REF1 ref_name

The REF1 label lists the reference names for the ts_*_verify() function. For ts_*_verify() to look in the test matrix file for reference names, you must specify the -ref_matrix option when running Xilch.

ref_name....Name of a reference that corresponds to the sequential calls to ts_*_verify() in the test program.

Requirements and Limitations

Test programs that use two source images should call ts_get_src1() and ts_get_src2(). A test program can create its own destination image, or it can call ts_get_dst1(). The XIL Test Suite library makes no consistency checks between tables. For example, it would not notice if the number of bands in the third image of the SRC2 table was the same as the number for the third image in the SRC1 table.

Test Matrix File Example

The following example will give you an idea of the kind of information contained in a typical test matrix file. Note that each file can contain as many parent and child images as you want. Child images must always follow the parent image with which they are associated; ROI tags must always follow the image, parent, or child with which they are associated.

This example specifies one source table (SRC1) that has two parent images and a child image associated with the second parent image, and one destination table (DST1) that has two parent images.

The Source Table The first parent image is a 256 x 256, single-banded, short image initially filled with random data, using 12989 as the seed; its origin is (0.0, 0.0). The second parent image is a 257 x 193, 3-banded, unfilled bit image with origin coordinates of (10.0, 10.0). The child image is offset from its parent (the image specified directly above it) in x by 7 and y by 13. It's a 23 x 37, 2-banded image that begins in the second band of its parent.

The Destination Table The first parent image is a 256 x 256, single-banded, short image filled with constant data; its origin is (10.0, 10.0). The second parent image is a 257 x 193, 3-banded, bit image initially filled with random data, using 73142 as the seed; its origin is (0.0, 0.0).


  TABLE SRC1  
       256 256 1 XIL_SHORT TS_RANDOM 12989 0.0 0.0 NULL  
       257 193 3 XIL_BIT TS_NOFILL 0 10.0 10.0 NULL  
  CHILD 7 13 23 37 1 2  
  END SRC1  
  
  TABLE DST1  
       256 256 1 XIL_SHORT TS_CONSTANT 0 10.0 10.0 NULL  
       257 193 XIL_BIT TS_RANDOM 73142 0.0 0.0 NULL  
  END DST1

The Log File

By default, the information reported by Xilch is displayed on your terminal (stdout). You can specify that this information be printed to a log file by using the -l option when you invoke Xilch.

The information reported by Xilch consists of a header and the results of the test programs as they iterate through the test matrix file. The header contains the following information:

`Xilch` Started	Time the `Xilch` run began
User	User name of the person running `Xilch`
Host	Name of machine running `Xilch`
Invocation	What the user typed to invoke this run of `Xilch`
`XILHOME`	The value of this environment variable
`XILCHHOME`	The value of this environment variable

XILCHREFDATA        The value of this environment variable

At the end of the log file is information stating the number of programs executed, the number of failures, and the percentage of program success. You can look at this information to see quickly the results of the test program.

The following is an excerpt from a Xilch run that has no failures.

Code Example 2-1 Log File Example (1 of 2)


  ***************************************************************  
  Xilch Started: May 20 94 01:03:13  
  User: xil  
  Host: emulsion  
  Invocation: bin/Xilch -l Xilch_run.log  
  XILHOME: /home/xil/devel/sparc/sparc  
  XILCHHOME: /home/xil/xilch/devel/sparc/sparc  
  XILCHREFHOME: /home/xil/xilch/devel/sparc/data/references  
  ---------------------------------------------------------------  
  > /home/xil/xilch/devel/sparc/sparc/bin/arith_test: Started  
  01:03:14  
  > /home/xil/xilch/devel/sparc/sparc/bin/arith_test: Passed  
  01:05:30  
  ---------------------------------------------------------------

Code Example 2-1 Log File Example (2 of 2)


  ***************************************************************  
  > /home/xil/xilch/devel/sparc/sparc/bin/black_generation_test:  
  Started 01:05:30  
  > /home/xil/xilch/devel/sparc/sparc/bin/black_generation_test:  
  Passed 01:06:50  
  ---------------------------------------------------------------  
  > /home/xil/xilch/devel/sparc/sparc/bin/blend_test -m  
  blend_tests: Started 01:06:50  
  > /home/xil/xilch/devel/sparc/sparc/bin/blend_test -m  
  blend_tests: Passed 01:12:24  
  ---------------------------------------------------------------  
  > /home/xil/xilch/devel/sparc/sparc/bin/cast_test -m  
  quick_tests: Started 01:12:25  
  > /home/xil/xilch/devel/sparc/sparc/bin/cast_test -m  
  quick_tests: Passed 01:13:13  
  .  
  .  
  .  
  ---------------------------------------------------------------  
  > /home/xil/xilch/devel/sparc/sparc/bin/xil_lookup_test: Started  
  01:37:30  
  > /home/xil/xilch/devel/sparc/sparc/bin/xil_lookup_test: Passed  
  01:38:59  
  ---------------------------------------------------------------  
  
  >>>>> Number of programs executed: 27  
  >>>>> Number of program failures: 0  
  >>>>> Program success %: 100.00%  
  
  Xilch Done: May 20 994 01:38:59

`Xilch` Command Line Options

The Xilch harness runs the test programs in the test suite. Use its options to vary testing conditions. You have already learned about some of the options you can use with Xilch: -f to specify a testlist file, -m to specify a test matrix file, and -l to specify a log file. Following is a complete list of the command line options.

Table 2-2 Xilch (1 of 4)

Option	Description
-`create_data`	Creates reference data in the reference directory. This option cannot be used with -`create_signatures`, `-test_signatures`, or -`test_data`. See also `-ref_directory`.
`-create_signatures`	Creates reference signatures in the reference directory. This option cannot be used with -`create_data`, `-test_signatures`, or -`test_data`. See also `-ref_directory`.
`-D`	Turns on display mode. This option displays both the test image and the reference image, if any.
`-f` *file*	Use *file* as the testlist file. The default file is `testlist` in the current directory, and if not found there, the file `testlist` in `$XILCHHOME/config`. Each line in the testlist file specifies the name of a test program to run and its options, if any.
`-l` *logfile*	Uses *logfile* to log test information reported by `Xilch` and the test suite library. The default log file is `stdout`.
`-m` *file*	Uses *file* as the test matrix file for all test programs in this `Xilch` run. If file cannot be found in the current directory, then `$XILCHHOME/config` is searched. If *file* is still not found, then `$XILCHHOME/config/xilch_tests` is used. This option does not really affect the operation of `Xilch` itself. Instead, it is passed to the test program and used by the test suite library to find and/or generate its test images.
`-no_ref_backup`	Does not back up the references before creating new references. By default a backup reference is created.

Table 2-2 Xilch (2 of 4)

Option	Description
-`percent` *value*	Uses *value*, a floating point number between 0 and 100, as the acceptable percentage difference for comparisons. A value of `0` means that the test data and the reference data must match perfectly. A value of `10` means up to 10% of the pixels can be different.
-`platform` *string*	Species the name of the platform *(string)* for which you want to create references or verify data (for example, x86). The default value is the CPU on which the test is run. For example, if you run the tests on a SPARC platform, the default platform name is `generic`; if you run the tests on an `x86` platform, the default platform name is `x86`. This can be used to create accelerator-specific references. When creating references, `Xilch` uses the specified platform to create the references and stores them in a directory for that platform. When testing, `Xilch` uses the references stored for the specified platform.
-`ref_directory` *dir*	Uses the directory *dir* to obtain references when testing or to store references when creating references. When this option is specified, the value overrides the default directory name or the directory name specified in the environment variable `XILCHREFDATA`. When testing, the reference directory you specify must have subdirectories: signatures for signature testing and `data` for data testing. When creating reference signatures or data, `Xilch` creates subdirectories if they do not exist. This option is useful to create references in a temporary location during development.
-`ref_matrix`	Uses the reference names listed in the test matrix file instead of the names specified in the calls to `ts__verify()`. When you use this option, the last parameter of `ts__verify()`, *ref_name*, is ignored. This option can be used only with `-test_data` and is intended for use during test development, not regression testing.

Table 2-2 Xilch (3 of 4)

Option	Description
`-t` *prog args*	Runs only the test program *prog. `Xilch` runs in single-test mode when this option is specified and ignores any testlist file. When you specify only one test program to run, you can also specify any of the other `Xilch` command line options, args*, other than `-f`.
`-test_data`	Tests images against data. This option cannot be used with `-`test_signatures, `-create_data`, or `-create_signatures`.
`-test_signatures`	Tests images against signatures. This is the default. This option cannot be used with `-test_data`, -`create_data`, or -`create_signatures`.
-`tol` *value*	Uses *value* as the floating point tolerance for comparisons. For `ts__verify()`, this option is meaningful when used with the `-test_data` option. For XIL_SHORT images, value* can be any number between 0 and 65535. For XIL_BYTE images, *value* can be any number between 0 and 255. In both these cases, a *value* of 0 means that the test data perfectly matches the reference data. If you specify the maximum *value*, the comparison always succeeds. For `XIL_BIT` images, the tolerance value is ignored. For most XIL functions, you should specify a tolerance of 0, because you should be able to achieve a perfect pixel-to-pixel comparison. However, this perfect correspondence is not achievable for the geometric operators. It also may not be possible for dithering, error diffusion, and compression operations. In these cases, your tolerance value probably needs to be greater than 0, keeping in mind that you want to achieve as close a match as possible.

Table 2-2 Xilch (4 of 4)

Option	Description
-`use_ref_backup`	Uses the backup references for each test case. By default, when you create references, `Xilch` backs up the existing references before creating the new references. When you specify this option on the `Xilch` command line, `Xilch` uses the backup references. See also `-no_ref_backup`. This option is valid only when either the -`test_data` or `-test_signatures` option is specified.
`-v`	Runs in verbose mode.

Note - All of the Xilch options can be used inside a testlist file.

Example Invocations

In this section, several example invocations of Xilch, using different options, are provided to give you a feel for ways to use the test suite environment.

% Xilch

Xilch is invoked with the following default conditions:

The file testlist in the current directory is used as the testlist file. If this file is not found, then $XILCHHOME/config/testlist is used.
Test information is logged to stdout.
The file xilch_tests in $XILCHHOME/config is used as the default test matrix file.
Terse test information is provided.
Images are tested against reference signatures.

% Xilch -f mytests -l mytests.log -v