Image Management

Images are a crucial type of objects in the SDK, especially concerning scene appearance, since they are used to define textures for model materials.

読み終わるまでの所要時間 9 分

最終更新 2日前

material.getImagePixelInfoFromDefinition

material.getImagePixelInfoFromLayoutAndType

Images are a crucial type of objects in the SDK, especially concerning scene appearance, since they are used to define textures for model materials. Additionally to the pixel data themselves, each image comes with a description of how these data are encoded and interpreted as well as a toolset of useful functions for processing and manipulation.

Image formats

The pixel format of an image describes how its binary data must be interpreted in order to produce a color. A format is defined by:

a layout, saying how many channels the image is made of and which component is stored in each of them,
a component type, describing how values of image components are encoded.

Layout

The layout describes which component is stored in each image channel. Each component can be either R (red), G (green), B (blue), A (alpha) or Lum (luminance, for gray scale images), and the layout is a combination of these values. It can be recovered by the functions material.getImageLayout and material.getImageDefinition. Note that the layout corresponds to the internal arrangement of the image data, but accessing image pixels via the material.getImagePixelColor function is agnostic to the internal representation and always returns a consistent color. The set of supported image layouts is listed in the following table:

Layout `material.ImageLayout.` ...	Number of channels	Color returned by material.getImagePixelColor
`R`	1	`(R, 0, 0, 1)`	-------------------------------------	:--------------------:	-------------------------------------------------------------------------------------
`R`	1	`(R, 0, 0, 1)`
`G`	1	`(0, G, 0, 1)`
`B`	1	`(0, 0, B, 1)`
`A`	1	`(0, 0, 0, A)`
`RG`	2	`(R, G, 0, 1)`
`RGB`	3	`(R, G, B, 1)`
`RGBA`	4	`(R, G, B, A)`
`BGR`	3	`(R, G, B, 1)`
`BGRA`	4	`(R, G, B, A)`
`ABGR`	4	`(R, G, B, A)`
`Lum`	1	`(Lum, Lum, Lum, 1)`
`LumA`	2	`(Lum, Lum, Lum, A)`

Component type

A component type is defined by four attributes:

a size (number of bits), which implicitely defines the numerical precision of the type,
an internal representation (floating point or integer),
a signedness flag, defining whether negative values are allowed or not,
a normalization flag, indicating whether integers must be interpreted as reals or not.

The list of supported component types is defined by combinations of these attributes:

Type `material.ImageComponentType.` ...	Size (in bytes)	Internal representation	Signed	Normalized
`UInt8_Norm`	1	integer		✔
`UInt8_Int`	1	integer
`SInt8_Norm`	1	integer	✔	✔
`SInt8_Int`	1	integer	✔
`UInt16_Norm`	2	integer		✔
`UInt16_Int`	2	integer
`SInt16_Norm`	2	integer	✔	✔
`SInt16_Int`	2	integer	✔
`Float16`	2	floating point	✔
`UInt32_Norm`	4	integer		✔
`UInt32_Int`	4	integer
`SInt32_Norm`	4	integer	✔	✔
`SInt32_Int`	4	integer	✔
`Float32`	4	floating point	✔
`UInt64_Norm`	8	integer		✔
`UInt64_Int`	8	integer
`SInt64_Norm`	8	integer	✔	✔
`SInt64_Int`	8	integer	✔
`Float64`	8	floating point	✔

The component type of a given image can be recovered by the functions material.getImageComponentType and material.getImageDefinition. The values of the aforementionned attributes for a given type can be queried via functions material.getImagePixelInfo, material.getImagePixelInfoFromDefinition and material.getImagePixelInfoFromLayoutAndType, which provide useful details about image formats. The internal floating point representation follows the IEEE754 norm, meaning that Float16, Float32 and Float64 respectively correspond to half, single and double precision floats, and their ranges of valid values are as defined by the norm. Non normalized integers are interpreted as they are, with a range of possible values depending on the type size

in bits:

[0, 2^N-1]

for unsigned integers, and

[-2^{N-1}, 2^{N-1}-1]

for signed integers. Normalized integers are interpreted as reals in the range

[0, 1]

for unsigned integers and

[-1, 1]

for signed integers. The corresponding real value

x_r

is obtained by dividing the integer

x_i

by the maximum value allowed by the type size

x_r = \frac{x_i}{2^N - 1}

for unsigned integers, and

x_r = \frac{x_i}{2^{N-1} - 1}

for signed integers. Note that, according to these conversion rules, the valid signed integer

x_i = -2^{N-1}

is expected to be lower than

-1

. To avoid issues, this particular value is simply clamped to

-1

so as to not exceed the normalized range boundaries.

Format conversion rules

The API provides functions for image conversions between different formats (

material.convertImage

material.convertImageToDefinition

and

material.convertFloat32To8BitsImage

). Since each format has its own constraints in terms of supported components, value range and precision, some rules apply when switching from one to another:

a component that exists in both source and destination layouts is transferred by converting the data type:
- out-of-bound values are clamped to the validity range of the destination type, if needed,
- reals are converted to integers by rounding them to the closest one.
if a component in the destination does not exist in the source, a default value is assigned to it:
- default R, G, B or Lum = 0.0,
- default A = 1.0.
the Lum component in the source is transferred to any of the destination component of type R, G or B.
the R, G and B components in the source are converted to the Lum component in the destination by:
```
Lum = (R + G + B) / 3
```
.
if only the layout is modified and not the component type, channels are rearranged so as to match the destination format but their contents are left unchanged.

According to these rules, converting from layout

material.ImageLayout.R

to layout

material.ImageLayout.B

, for instance, does not result to converting image colors from

(**R**, 0, 0, 1)

(0, 0, **B = R**, 1

), but to

(0, 0, 0, 1)

. If you want to change the way image channels are interpreted, use the function

material.overrideImageFormat

instead.

Image operations

Import / export

The list of image file formats and extensions available for

material.importImage

and

material.exportImage

can be recovered thanks to the following functions:

During export, image data are converted to be compliant with the selected export format requirements (eg. HDR images are converted to 8 bits when exported to PNG).

Creation

```
material.createImageFromData
```
can be used to create an image from the raw data corresponding to a valid image file format.

material.createImageFromDefinition

material.createImagesFromDefinitions

```
material.updateImageFromDefinition
```
/
```
material.updateImagesFromDefinitions
```
have the same behaviour as
```
createImageFromDefinition
```
, except that they replace existing images instead of creating a new ones.
```
material.createCheckerboardImage
```
creates a bicolor checkerboard. Mainly useful for tests or debugging purposes.

Accessors

The two main structures that contain information about images are:

```
material.ImageDefinition
```
, which allows to access raw image pixel data, as well as image dimension and format,
```
material.PixelInfo
```
, which gives additional details about the pixel format, like component count and size, and type attributes (internal representation, normalization, signedness).

Access to image format details (see above for more details):

material.getImagePixelInfo

material.getImagePixelInfoFromDefinition

material.getImagePixelInfoFromLayoutAndType

Access to image attributes and content:

```
material.getAllImages
```
returns the list of all images available in the current session.
```
material.getImageDefinition
```
/
```
material.getImageDefinitions
```
returns the definition (as described above) of one or multiple images.
```
material.getImageSize
```
/
```
material.getImagesSizes
```
return the dimensions (width and height) of one or multiple images.
```
material.getImagePixelColor
```
returns the color quadruplet (R,G,B,A) of a single image pixel, converted from the raw data according to the image format.
```
material.getImageColorBilinear
```
returns a bi-linearly interpolated color from image coordinates expressed as real numbers.
```
material.getImageColorRange
```
returns, for each color component, the min and max values of the whole image. This can be provided, for instance, to the function
```
convertFloat32To8BitsImage
```
in order to remap from an unbound range to a valid 8bits range.

Modifiers

Conversion and component manipulation:

```
material.overrideImageFormat
```
enables to change the image format of an image without modifying its data, in order to change the way data are interpreted.

material.convertImage

material.convertImageToDefinition

material.convertFloat32To8BitsImage

provide a way to convert the image from one format to another.

```
material.extractImageChannels
```
/
```
material.extractImageComponents
```
allow to decompose the different components the image is made of.

Transformation:

```
material.resizeImage
```
changes the image dimensions and rescales its content accordingly.
```
material.rotateImage
```
applies a rotation to the image content.
```
material.stretchImage
```
applies a non-uniform scaling to the image content.
```
material.translateImage
```
applies a translation to the image content.
```
material.transformImage
```
applies a generic 3x3 matrix to the image content.

Processing:

```
material.fillImageWithColor
```
fills a sub-part or a whole image with a single color.
```
material.invertImageColor
```
replaces, for each pixel, the value V by 1 - V. This function is not expected to produce relevant results for images with non-normalized component types.
```
material.setSubImage
```
/
```
material.getSubImage
```
specifies or extracts the sub-part of an image.
```
material.blurImage
```
applies a box or a Gaussian filter to the image.
```
material.fillUnusedPixels
```
allows to fill pixels in areas considered as "invalid" by dilation of valid areas. The pixel validity state can be specified either by a background color or by a validity mask.
```
material.flipImageY
```
reverts the order of scanlines along the image Y axis.

Edge filters

Some of the image modifiers listed above accept an edge filter as argument (

material.blurImage

for instance). The edge filter describes how to manage coordinates that exceed the image boundaries. There are four possible ways to assign colors to these out-of-bound locations:

`material.EdgeFilter.Clamp`	`material.EdgeFilter.Extend`	`material.EdgeFilter.Repeat`	`material.EdgeFilter.Mirror`

The default color `(0,0,0,1)` is returned	The color of the closest valid boundary pixel is returned	The image is repeated by taking coordinates modulus	The image is repeated in a mirrored way

Regions of interest

Sometimes, it may be desired to apply a modifier to only a sub-part of an image. This can be achieved by the mean of regions of interest (RoI). The function

material.setImageRoI

enables to specify a rectangular window inside the image frame which defines the pixels that will be affected by the subsequent processes. The following example illustrates the result of the function

material.rotateImage

applied to the whole image, or to only a part of it specified thanks to a RoI:


Original image	Rotation without RoI	Rotation with RoI

ドキュメント

Development

User Acquisition

Monetization

産業

ドキュメント

Development

User Acquisition

Monetization

産業

Asset Transformer SDK

User Manual

Python API

C# API

Changelog

Asset Transformer SDK

Image Management

Image formats

Layout

Layout
`material.ImageLayout.`
...

Number of
channels

Color returned by
material.getImagePixelColor

Component type

Type
`material.ImageComponentType.`
...

Size
(in bytes)

Internal
representation

Signed

Normalized

Format conversion rules

Image operations

Import / export

Creation

Accessors

Modifiers

Edge filters

`material.EdgeFilter.Clamp`

`material.EdgeFilter.Extend`

`material.EdgeFilter.Repeat`

`material.EdgeFilter.Mirror`

Regions of interest

Image Management

API functions

Image formats

Layout

Layoutmaterial.ImageLayout....

Number ofchannels

Color returned bymaterial.getImagePixelColor

Component type

Typematerial.ImageComponentType....

Size(in bytes)

Internalrepresentation

Signed

Normalized

Format conversion rules

Image operations

Import / export

Creation

Accessors

Modifiers

Edge filters

material.EdgeFilter.Clamp

material.EdgeFilter.Extend

material.EdgeFilter.Repeat

material.EdgeFilter.Mirror

Regions of interest

Layout
`material.ImageLayout.`
...

Number of
channels

Color returned by
material.getImagePixelColor

Type
`material.ImageComponentType.`
...

Size
(in bytes)

Internal
representation

`material.EdgeFilter.Clamp`

`material.EdgeFilter.Extend`

`material.EdgeFilter.Repeat`

`material.EdgeFilter.Mirror`