文档

Prerequisites

Ensure that you have the required configuration, tooling, and permissions.
阅读时间15 分钟最后更新于 13 天前

Access to the container registry

Ensure that you have credentials for the Unity private container registry
uccmpprivatecloud.azurecr.io
. Unity provides these credentials. You need this information to download shdctl, pull release packages, and sync container images and ORAS artifacts to your private registry.

Tooling

Ensure that you have this tooling:
  • Access to a terminal, and basic knowledge of the command line
  • One of these tools if you use Microsoft Windows:
    • Windows Subsystem for Linux (WSL)
    • Git for Windows
  • ORAS CLI, for the shdctl install script, which downloads shdctl from the registry
  • Helm (version 3.9 or later), to install the Helm charts
  • kubectl, to interact with your Kubernetes cluster
  • Docker, to sync the container images to your private registry
  • ArgoCD (recommended), for continuous delivery via GitOps. If you choose the ArgoCD deployment method, install it in the target cluster before you deploy: shdctl applies a single bootstrap
    Application
    and ArgoCD takes over from there. Every application is created in the deployment namespace, so install ArgoCD in that namespace, or configure it to watch it: ArgoCD's "Applications in any namespace" needs the namespace in both its
    application.namespaces
    setting and the
    default
    project's
    sourceNamespaces
    . An ArgoCD that watches only its own namespace leaves the applications unsynced, and reports no error. If you deploy with Helm, you don't need ArgoCD at all.
  • shdctl is the Unity CLI tool that manages the entire deployment lifecycle. It replaces vpctl, which can't pull Self-Hosted Deployment 2.0.0 or later. Install shdctl 1.1.0 or later. For its install steps, requirements, and registry credential setup, refer to Install shdctl.
注意
Unity recommends ArgoCD as the deployment method. ArgoCD handles CRD installation ordering and detects configuration drift automatically, which makes ongoing operations more reliable. Helm is also fully supported if you prefer direct deployments without a GitOps workflow.

Kubernetes

The deployment requires a Kubernetes cluster that you manage.
Use Kubernetes version 1.34 or later.
Your cluster must include at least three schedulable nodes in the general workloads pool. By default, the following components run several replicas, and place each replica on a different node:
  • MongoDB (Percona Server for MongoDB): three replicas.
  • PostgreSQL: three replicas.
  • RabbitMQ: three replicas.
  • Garage object storage: three replicas. If you raise
    configuration.infrastructure.components.garage.replicas
    , the pool needs at least that many nodes.
  • Elasticsearch: three master nodes and two data nodes, each in its own pod. A master pod and a data pod can share a Kubernetes node.
The components spread their replicas with
podAntiAffinity
on
kubernetes.io/hostname
. If your cluster has only two nodes, the scheduler can't meet this requirement, and some pods stay in the
Pending
state.

Storage classes

The cluster must provide two storage classes:
  • A block storage class for ReadWriteOnce volumes. For example, a local-path provisioner, a SAN-backed CSI driver, or any block storage provisioner.
  • A ReadWriteMany (RWX) storage class for shared volumes that multiple pods can mount simultaneously. For example, an NFS provisioner, a distributed filesystem, or any CSI driver that supports the
    ReadWriteMany
    access mode.
In the manifest file, name both storage classes:
  • Set
    configuration.kubernetes.storage.readWriteOnceStorageClass
    to the block storage class. Every ReadWriteOnce volume uses it, so the cluster doesn't need a default storage class.
  • Set
    configuration.kubernetes.storage.readWriteManyStorageClass
    to the RWX storage class.
For an example, refer to the annotated manifest example.
If your deployment was created with Self-Hosted Deployment 2.0.0 or earlier, its manifest names
configuration.kubernetes.storage.defaultStorageClass
instead of
readWriteOnceStorageClass
. Keep that field, and don't replace it with
readWriteOnceStorageClass
: Kubernetes can't change the storage class of an existing volume. In that deployment:
  • Garage and the Unity Licensing Server use the class that
    defaultStorageClass
    names.
  • Every other ReadWriteOnce volume uses the cluster's default storage class.
The cluster must therefore have a default storage class. To make a storage class the default, give it the
storageclass.kubernetes.io/is-default-class: "true"
annotation. For more information, refer to Change the default StorageClass in the Kubernetes documentation.
To confirm that your manifest's storage field matches the volumes the deployment already has, run
shdctl cluster check
. The check fails if they don't match. For more information, refer to shdctl cluster command.

Metrics API

The cluster must serve the
metrics.k8s.io
API, which the platform's horizontal pod autoscalers require. Most distributions provide it through
metrics-server
. Without it, autoscaled workloads can't read their resource usage and don't scale.

Namespaces

Services come preconfigured for use within a single Kubernetes namespace. You configure the target namespace in the manifest file.
Create the namespace before you deploy. The deployment doesn't create it for you.

Node pools

The platform separates general application workloads from asset transformation jobs. Configure your cluster with three node pools (or equivalent node groups):

Node pool

Node label

Taint

Recommended node size

Purpose

General workloadsNone requiredNone8–32 vCPUs; compute, general-purpose, or memory-optimized instancesApplication services, databases, messaging
Transformations
aks-node-pool=argocpu
None8–32 vCPUs and at least 64 GiB of memory; general-purpose or memory-optimized instancesTransformation workflow pods
Transformations (large)
aks-node-pool=argocpu-large
aks-node-pool=argocpu-large:NoSchedule
32–64 vCPUs and at least 256 GiB of memory; general-purpose or memory-optimized instancesEscalation pool for memory-intensive transformations; only pods that explicitly tolerate the taint are scheduled here
Transformation steps request up to 32 GiB of memory on the
argocpu
pool, and their retries up to 220 GiB on the
argocpu-large
pool. A step that no node can fit stays
Pending
until its workflow times out.
Monitoring adds load to the general pool. When Prometheus, log collection, and database monitoring are all enabled (
configuration.monitoring
), their components request about 3.5 vCPUs and 12 GiB of memory across the pool. This leaves 8-vCPU nodes little headroom. With all three enabled, use general nodes of at least 16 vCPUs, or add a fourth 8-vCPU node.
The exact mechanism to create node pools depends on your Kubernetes distribution (for example, static node labels, a node autoscaler, or a cluster API provider). If an autoscaler other than Karpenter scales the transformation pool from zero,
shdctl cluster check
reports the pool as missing while it is empty; that failure is expected.
All nodes in every pool must use the x86-64 (
amd64
) architecture. Unity publishes the platform's container images for
linux/amd64
only. Pods scheduled on Arm64 nodes, such as AWS Graviton, fail to pull their images and return an error that includes the following:
no match for platform in manifest
GPU nodes are optional. To run transformations on GPUs, add a fourth pool as described in GPU nodes.

GPU nodes

By default, transformations don't use GPUs:
  • Transformation actions that can use a GPU run on the
    argocpu
    pool, with a software renderer.
  • Pipelines don't offer the GPU configuration profiles, and a step that uses one runs on CPU.
To run transformations on NVIDIA GPUs:
  1. Add a GPU node pool to your cluster. Its nodes must meet the GPU node requirements and have the NVIDIA driver and device plugin installed.
  2. In the manifest, set
    configuration.transformations.gpu.enabled: true
    .
  3. Verify the GPU nodes.
  4. Deploy the release. For more information, refer to Deploy the solution.
After the deployment:
  • Pipelines offer the GPU configuration profiles, including in app versions that are already installed. A step that uses a GPU profile, such as Optimize and Convert on its GPU profile, requests a GPU.
  • Other actions that can use a GPU request one in app versions that you install after the deployment. App versions that support it also switch to GPU from the next step that starts, without reinstalling the app.
To stop using GPUs, set the field back to
false
, and deploy again. After the deployment:
  • Pipelines don't offer the GPU configuration profiles, and steps on a GPU profile run on CPU.
  • Actions in app versions that support the change switch to CPU from the next step that starts, without reinstalling the app.
  • Actions in other app versions keep the choice they were installed with. An action installed while GPUs were on still requests a GPU until you install a newer app version. If no GPU node is left, the action stays in the
    Pending
    state until then.

GPU node requirements

Each node in the GPU node pool must meet the following requirements:

Requirement

Value

GPUOne NVIDIA GPU for each node, such as the NVIDIA T4. Each step uses a whole GPU.
CPU and memoryAt least 8 vCPUs and 32 GiB of memory for each node, for example, a
g4dn.2xlarge
instance on AWS. The smallest GPU configuration profile requests one GPU, 16 GiB of memory, and up to 4 vCPUs, so a node with 16 GiB of memory can't run it.
NVIDIA driverInstalled on each node, with Vulkan support, because steps render through Vulkan on the GPU.
Device pluginRunning on each GPU node, so that the node advertises the
nvidia.com/gpu
resource.
Taint
nvidia.com/gpu:NoSchedule
, which keeps other workloads off the GPU nodes. GPU steps tolerate this taint.
Node labelNot required. GPU steps run on any node that advertises
nvidia.com/gpu
. To restrict them to one node pool, refer to Restrict GPU steps to one node pool.
Architecturex86-64 (
amd64
), like every other pool.

Install the NVIDIA driver and device plugin

Whether you need to install the NVIDIA driver and device plugin yourself depends on your platform:
  • Amazon EKS Auto Mode: You don't need to install anything. Nodes in a NodePool of NVIDIA GPU instances, such as the
    g4dn
    family, come with the NVIDIA driver and device plugin. The
    aws-baseline-example
    in the release package includes such a NodePool,
    gpu-nodepool.yaml
    . Its
    just cluster-bootstrap
    recipe applies the NodePool when you run it with
    gpu=true
    .
  • Amazon EKS with managed node groups: You don't need to install anything. Use the
    BOTTLEROCKET_x86_64_NVIDIA
    AMI type for the node group, which includes the driver and the device plugin.
  • Other platforms: Install the NVIDIA driver and a device plugin that advertises
    nvidia.com/gpu
    . Follow your platform's documentation.
On every platform, verify the GPU nodes afterward.

Scale the GPU node pool from zero

GPU nodes cost more than nodes in the other pools. To reduce costs, you can let an autoscaler scale the GPU node pool down to zero nodes when no GPU step is running. The trade-off is a delay: after an idle period, the first GPU step stays in the
Pending
state while a GPU node starts and pulls the step's image. This wait counts toward the workflow's timeout. If your pipelines can't absorb the delay, keep at least one GPU node running.
shdctl cluster check
passes its GPU check when it finds either of the following:
  • A node in the
    Ready
    state that advertises
    nvidia.com/gpu
    .
  • A Karpenter NodePool that declares the
    nvidia.com/gpu
    taint.
If another autoscaler scales the GPU node pool from zero, the pool has no node while it's empty, so the check shows a warning instead of passing. A warning doesn't fail the command.

Verify the GPU nodes

To check that the GPU nodes are ready for GPU steps:
  1. List the nodes and the number of GPUs that each one advertises:
    kubectl get nodes -o custom-columns='NAME:.metadata.name,GPU:.status.allocatable.nvidia\.com/gpu'
    Each GPU node shows
    1
    in the
    GPU
    column. Nodes in the other pools show
    <none>
    . If a GPU node shows
    <none>
    , check its NVIDIA driver and device plugin. For more information, refer to Install the NVIDIA driver and device plugin.
  2. Run
    shdctl cluster check
    , and check that the
    gpu nodes
    check passes. For more information, refer to Validate your cluster.

Restrict GPU steps to one node pool

By default, a step on a GPU configuration profile runs on any node that advertises
nvidia.com/gpu
. To restrict these steps to one node pool:
  1. In the manifest, add the following settings under
    configuration.overrides.automation.values.common-chart.configMaps.automation-config.data
    :
    • UCAUTOMATION_ArgoTemplateOptions__NodeSelectorLabels__UsesGPU__Label
      : the key of the node pool's label.
    • UCAUTOMATION_ArgoTemplateOptions__NodeSelectorLabels__UsesGPU__Value
      : the value of the node pool's label.
    For example:
    configuration: overrides: automation: values: common-chart: configMaps: automation-config: data: UCAUTOMATION_ArgoTemplateOptions__NodeSelectorLabels__UsesGPU__Label: "<label-key>" UCAUTOMATION_ArgoTemplateOptions__NodeSelectorLabels__UsesGPU__Value: "<label-value>"
  2. Generate and deploy the charts again. For more information, refer to Deploy the solution.
This restriction doesn't apply to 3D Data Streaming. Its GPU steps still run on any node that advertises
nvidia.com/gpu
.

IPv6 clusters

The platform runs on IPv4 clusters by default. To deploy on a single-stack IPv6 cluster:
  1. Make sure that the cluster's nodes, pods, and Kubernetes Services use IPv6 networking. Each node must report an IPv6
    InternalIP
    address. To check, run
    kubectl get nodes -o wide
    , and look at the
    INTERNAL-IP
    column.
  2. In the manifest, set
    configuration.networking.ipFamily: ipv6
    .
shdctl cluster check
warns if no node reports an IPv6
InternalIP
address.

Network policies

During deployment to an existing cluster, you may need to control the flow of network traffic by using network policies. You can deploy most Kubernetes resources in a single namespace, which you can use to scope network isolation from other services that run in your cluster.

Clusters behind an HTTP proxy

If your AKS cluster sends its outbound traffic through an HTTP proxy (AKS HTTP proxy), AKS injects
HTTP_PROXY
,
HTTPS_PROXY
and
NO_PROXY
into every pod. Platform services call each other by their short Service names, which no
NO_PROXY
entry matches, so those calls go to the proxy and fail.
To keep the platform pods off the proxy, add the AKS opt-out annotation to every pod in the manifest file. Nodes still pull images through the proxy, but the pods get no internet access:
configuration: kubernetes: podAnnotations: kubernetes.azure.com/no-http-proxy-vars: "true"
The automation-manager job pulls the release manifest from your registry itself, so it still needs the proxy. Unless your cluster reaches the registry without the proxy, pass it through the chart's overrides:
configuration: overrides: automation-manager: values: env: httpsProxy: http://<proxy>:<port> noProxy: .svc,.cluster.local,localhost,127.0.0.1
Alertmanager carries these annotations too, so a receiver outside the cluster needs
httpConfig.proxyURL
in its
AlertmanagerConfig
.
configuration.kubernetes.podAnnotations
applies to every pod the release creates. To change or remove an annotation for a single chart, refer to Per-chart Helm value overrides.
The
azure-baseline-example
in the release package can create such a cluster:
aks_http_proxy
sets the proxy,
aks_private_cluster_enabled
makes the API server private, and
aks_egress_via_proxy_only
denies all other outbound traffic from the cluster's subnet. Its README describes each variable.

What the proxy must allow

  • AKS's own outbound traffic. Nodes bootstrap and keep running through the proxy, so it must allow the destinations in AKS outbound network and FQDN rules, for example
    mcr.microsoft.com
    and
    packages.aks.azure.com
    .
  • Your container registry. Nodes pull every image through the proxy, and the automation-manager job reaches the registry through it too: the registry in
    configuration.kubernetes.docker.repository
    , or
    uccmpprivatecloud.azurecr.io
    if you pull from it directly. Allow the endpoints the registry serves image layers from as well; for Azure Container Registry, refer to Configure rules to access an Azure container registry behind a firewall.
  • Requests from the nodes. Pod traffic reaches the proxy from its node's address, so allow the node subnet as a source.
  • HTTP and HTTPS only. HTTPS goes through the proxy as
    CONNECT
    on port 443, and plain HTTP on port 80. Outbound traffic that isn't HTTP, such as SMTP or SSH, has no path through the proxy.
  • TLS inspection. If the proxy decrypts HTTPS, give the cluster its CA certificate (
    trustedCa
    in the AKS HTTP proxy configuration). The certificates must use subject alternative names.

Container registry

By default, the platform pulls container images directly from the Unity source registry (
uccmpprivatecloud.azurecr.io
). This is the simplest setup and doesn't require a separate registry.
If your environment is air-gapped or if you need full control over artifact distribution, you can mirror artifacts to your own private container registry. Any OCI-compliant registry is supported, for example Harbor, JFrog Artifactory, or a cloud-managed registry such as Amazon ECR or Azure Container Registry. Use these commands to mirror artifacts from the Unity source registry to your private registry:
  • shdctl artifact sync images
    : this command syncs the Docker container images.
  • shdctl artifact sync oras
    : this command syncs the ORAS artifacts, for example, the Pixyz workflow templates.
  • shdctl artifact sync verify
    : this command checks that your registry holds every image your manifest needs.

System requirements

Ensure that you have these elements:
  • A valid hostname that can be updated to point to the IP of the load balancer
  • A valid Unity Version Control (UVCS) license to run the UVCS server
  • A valid Unity Asset Transformer SDK license to run transformations on assets
    A valid Unity Asset Transformer license is required for all users who perform asset transformations. The platform uses static licenses, so a floating license server is not required.

Validate your cluster

Once your
manifest.yaml
exists, check the prerequisites on this page automatically. The command reads the namespace, the storage classes, and the ArgoCD settings from the manifest, and step 3 of the deployment runs it:
shdctl cluster check
The command verifies:
  • The Kubernetes version.
  • The number of schedulable nodes.
  • The transformation node pools.
  • The storage classes.
  • The target namespace.
  • The metrics API.
  • The in-cluster ArgoCD installation if your manifest configures ArgoCD. On the Helm deployment path, these checks are skipped.
The command reads from the cluster and doesn't change it. It prints a remediation hint for anything that doesn't pass.
Add
--probe-storage
to verify that each storage class can provision a volume, including
ReadWriteMany
support.
For the full list of checks and options, refer to shdctl cluster command.

Next steps