Documentation

Support

Asset Transformer SDK

User Manual

Python API

C# API

Changelog

Discussions

Asset Transformer SDK

Image Management

Images are a crucial type of objects in the SDK, especially concerning scene appearance, since they are used to define textures for model materials.
Read time 9 minutesLast updated 10 hours ago

Images are a crucial type of objects in the SDK, especially concerning scene appearance, since they are used to define textures for model materials. Additionally to the pixel data themselves, each image comes with a description of how these data are encoded and interpreted as well as a toolset of useful functions for processing and manipulation.

Image formats

The pixel format of an image describes how its binary data must be interpreted in order to produce a color. A format is defined by:
  • a layout, saying how many channels the image is made of and which component is stored in each of them,
  • a component type, describing how values of image components are encoded.

Layout

The layout describes which component is stored in each image channel. Each component can be either R (red), G (green), B (blue), A (alpha) or Lum (luminance, for gray scale images), and the layout is a combination of these values. It can be recovered by the functions material.getImageLayout and material.getImageDefinition. Note that the layout corresponds to the internal arrangement of the image data, but accessing image pixels via the material.getImagePixelColor function is agnostic to the internal representation and always returns a consistent color. The set of supported image layouts is listed in the following table:

Layout
material.ImageLayout.
...

Number of
channels

Color returned by
material.getImagePixelColor

R
1
(R, 0, 0, 1)
R
1
(R, 0, 0, 1)
G
1
(0, G, 0, 1)
B
1
(0, 0, B, 1)
A
1
(0, 0, 0, A)
RG
2
(R, G, 0, 1)
RGB
3
(R, G, B, 1)
RGBA
4
(R, G, B, A)
BGR
3
(R, G, B, 1)
BGRA
4
(R, G, B, A)
ABGR
4
(R, G, B, A)
Lum
1
(Lum, Lum, Lum, 1)
LumA
2
(Lum, Lum, Lum, A)

Component type

A component type is defined by four attributes:
  • a size (number of bits), which implicitely defines the numerical precision of the type,
  • an internal representation (floating point or integer),
  • a signedness flag, defining whether negative values are allowed or not,
  • a normalization flag, indicating whether integers must be interpreted as reals or not.
The list of supported component types is defined by combinations of these attributes:

Type
material.ImageComponentType.
...

Size
(in bytes)

Internal
representation

Signed

Normalized

UInt8_Norm
1integer✔
UInt8_Int
1integer
SInt8_Norm
1integer✔✔
SInt8_Int
1integer✔
UInt16_Norm
2integer✔
UInt16_Int
2integer
SInt16_Norm
2integer✔✔
SInt16_Int
2integer✔
Float16
2floating point✔
UInt32_Norm
4integer✔
UInt32_Int
4integer
SInt32_Norm
4integer✔✔
SInt32_Int
4integer✔
Float32
4floating point✔
UInt64_Norm
8integer✔
UInt64_Int
8integer
SInt64_Norm
8integer✔✔
SInt64_Int
8integer✔
Float64
8floating point✔
The component type of a given image can be recovered by the functions material.getImageComponentType and material.getImageDefinition. The values of the aforementionned attributes for a given type can be queried via functions material.getImagePixelInfo, material.getImagePixelInfoFromDefinition and material.getImagePixelInfoFromLayoutAndType, which provide useful details about image formats. The internal floating point representation follows the IEEE754 norm, meaning that Float16, Float32 and Float64 respectively correspond to half, single and double precision floats, and their ranges of valid values are as defined by the norm. Non normalized integers are interpreted as they are, with a range of possible values depending on the type size 
N
in bits: 
[0, 2^N-1]
for unsigned integers, and 
[-2^{N-1}, 2^{N-1}-1]
for signed integers. Normalized integers are interpreted as reals in the range 
[0, 1]
for unsigned integers and 
[-1, 1]
for signed integers. The corresponding real value 
x_r
is obtained by dividing the integer 
x_i
by the maximum value allowed by the type size 
N
: 
x_r = \frac{x_i}{2^N - 1}
for unsigned integers, and 
x_r = \frac{x_i}{2^{N-1} - 1}
for signed integers. Note that, according to these conversion rules, the valid signed integer 
x_i = -2^{N-1}
is expected to be lower than 
-1
. To avoid issues, this particular value is simply clamped to 
-1
so as to not exceed the normalized range boundaries.

Format conversion rules

The API provides functions for image conversions between different formats (
material.convertImage
, 
material.convertImageToDefinition
 and 
material.convertFloat32To8BitsImage
). Since each format has its own constraints in terms of supported components, value range and precision, some rules apply when switching from one to another:
  • a component that exists in both source and destination layouts is transferred by converting the data type:
    • out-of-bound values are clamped to the validity range of the destination type, if needed,
    • reals are converted to integers by rounding them to the closest one.
  • if a component in the destination does not exist in the source, a default value is assigned to it:
    • default R, G, B or Lum = 0.0,
    • default A = 1.0.
  • the Lum component in the source is transferred to any of the destination component of type R, G or B.
  • the R, G and B components in the source are converted to the Lum component in the destination by: 
    Lum = (R + G + B) / 3
    .
  • if only the layout is modified and not the component type, channels are rearranged so as to match the destination format but their contents are left unchanged.
According to these rules, converting from layout 
material.ImageLayout.R
to layout 
material.ImageLayout.B
, for instance, does not result to converting image colors from 
(R, 0, 0, 1)
to 
(0, 0, B = R, 1
), but to 
(0, 0, 0, 1)
. If you want to change the way image channels are interpreted, use the function 
material.overrideImageFormat
 instead.

Image operations

Import / export

The list of image file formats and extensions available for 
material.importImage
 and 
material.exportImage
 can be recovered thanks to the following functions: During export, image data are converted to be compliant with the selected export format requirements (eg. HDR images are converted to 8 bits when exported to PNG).

Creation

Accessors

The two main structures that contain information about images are: 
  • material.ImageDefinition
    , which allows to access raw image pixel data, as well as image dimension and format,
  • material.PixelInfo
    , which gives additional details about the pixel format, like component count and size, and type attributes (internal representation, normalization, signedness).
Access to image format details (see above for more details): Access to image attributes and content:

Modifiers

Conversion and component manipulation: Transformation: Processing:

Edge filters

Some of the image modifiers listed above accept an edge filter as argument (
material.blurImage
 for instance). The edge filter describes how to manage coordinates that exceed the image boundaries. There are four possible ways to assign colors to these out-of-bound locations: 

material.EdgeFilter.Clamp

material.EdgeFilter.Extend

material.EdgeFilter.Repeat

material.EdgeFilter.Mirror

clamp
extend
repeat
mirror
The default color 
(0,0,0,1)
is returned		The color of the closest valid boundary pixel is returnedThe image is repeated by taking coordinates modulusThe image is repeated in a mirrored way

Regions of interest

Sometimes, it may be desired to apply a modifier to only a sub-part of an image. This can be achieved by the mean of regions of interest (RoI). The function 
material.setImageRoI
 enables to specify a rectangular window inside the image frame which defines the pixels that will be affected by the subsequent processes. The following example illustrates the result of the function 
material.rotateImage
 applied to the whole image, or to only a part of it specified thanks to a RoI:

original

rotated

roi_rotated

Original imageRotation without RoIRotation with RoI