Prerequisites
Ensure that you have the required configuration, tooling, and permissions.
阅读时间15 分钟最后更新于 13 天前
Access to the container registry
Ensure that you have credentials for the Unity private container registry . Unity provides these credentials. You need this information to download shdctl, pull release packages, and sync container images and ORAS artifacts to your private registry.
uccmpprivatecloud.azurecr.ioTooling
Ensure that you have this tooling:
-
Access to a terminal, and basic knowledge of the command line
-
One of these tools if you use Microsoft Windows:
- Windows Subsystem for Linux (WSL)
- Git for Windows
-
ORAS CLI, for the shdctl install script, which downloads shdctl from the registry
-
Helm (version 3.9 or later), to install the Helm charts
-
kubectl, to interact with your Kubernetes cluster
-
Docker, to sync the container images to your private registry
-
ArgoCD (recommended), for continuous delivery via GitOps. If you choose the ArgoCD deployment method, install it in the target cluster before you deploy: shdctl applies a single bootstrapand ArgoCD takes over from there. Every application is created in the deployment namespace, so install ArgoCD in that namespace, or configure it to watch it: ArgoCD's "Applications in any namespace" needs the namespace in both its
Applicationsetting and theapplication.namespacesproject'sdefault. An ArgoCD that watches only its own namespace leaves the applications unsynced, and reports no error. If you deploy with Helm, you don't need ArgoCD at all.sourceNamespaces -
shdctl is the Unity CLI tool that manages the entire deployment lifecycle. It replaces vpctl, which can't pull Self-Hosted Deployment 2.0.0 or later. Install shdctl 1.1.0 or later. For its install steps, requirements, and registry credential setup, refer to Install shdctl.
Kubernetes
The deployment requires a Kubernetes cluster that you manage.
Use Kubernetes version 1.34 or later.
Your cluster must include at least three schedulable nodes in the general workloads pool. By default, the following components run several replicas, and place each replica on a different node:
- MongoDB (Percona Server for MongoDB): three replicas.
- PostgreSQL: three replicas.
- RabbitMQ: three replicas.
- Garage object storage: three replicas. If you raise , the pool needs at least that many nodes.
configuration.infrastructure.components.garage.replicas - Elasticsearch: three master nodes and two data nodes, each in its own pod. A master pod and a data pod can share a Kubernetes node.
The components spread their replicas with on . If your cluster has only two nodes, the scheduler can't meet this requirement, and some pods stay in the state.
podAntiAffinitykubernetes.io/hostnamePendingStorage classes
The cluster must provide two storage classes:
- A block storage class for ReadWriteOnce volumes. For example, a local-path provisioner, a SAN-backed CSI driver, or any block storage provisioner.
- A ReadWriteMany (RWX) storage class for shared volumes that multiple pods can mount simultaneously. For example, an NFS provisioner, a distributed filesystem, or any CSI driver that supports the access mode.
ReadWriteMany
In the manifest file, name both storage classes:
- Set to the block storage class. Every ReadWriteOnce volume uses it, so the cluster doesn't need a default storage class.
configuration.kubernetes.storage.readWriteOnceStorageClass - Set to the RWX storage class.
configuration.kubernetes.storage.readWriteManyStorageClass
For an example, refer to the annotated manifest example.
If your deployment was created with Self-Hosted Deployment 2.0.0 or earlier, its manifest names instead of . Keep that field, and don't replace it with : Kubernetes can't change the storage class of an existing volume. In that deployment:
configuration.kubernetes.storage.defaultStorageClassreadWriteOnceStorageClassreadWriteOnceStorageClass- Garage and the Unity Licensing Server use the class that names.
defaultStorageClass - Every other ReadWriteOnce volume uses the cluster's default storage class.
The cluster must therefore have a default storage class. To make a storage class the default, give it the annotation. For more information, refer to Change the default StorageClass in the Kubernetes documentation.
storageclass.kubernetes.io/is-default-class: "true"To confirm that your manifest's storage field matches the volumes the deployment already has, run . The check fails if they don't match. For more information, refer to shdctl cluster command.
shdctl cluster checkMetrics API
The cluster must serve the API, which the platform's horizontal pod autoscalers require. Most distributions provide it through . Without it, autoscaled workloads can't read their resource usage and don't scale.
metrics.k8s.iometrics-serverNamespaces
Services come preconfigured for use within a single Kubernetes namespace. You configure the target namespace in the manifest file.
Create the namespace before you deploy. The deployment doesn't create it for you.
Node pools
The platform separates general application workloads from asset transformation jobs. Configure your cluster with three node pools (or equivalent node groups):
Node pool | Node label | Taint | Recommended node size | Purpose |
|---|---|---|---|---|
| General workloads | None required | None | 8–32 vCPUs; compute, general-purpose, or memory-optimized instances | Application services, databases, messaging |
| Transformations | | None | 8–32 vCPUs and at least 64 GiB of memory; general-purpose or memory-optimized instances | Transformation workflow pods |
| Transformations (large) | | | 32–64 vCPUs and at least 256 GiB of memory; general-purpose or memory-optimized instances | Escalation pool for memory-intensive transformations; only pods that explicitly tolerate the taint are scheduled here |
Transformation steps request up to 32 GiB of memory on the pool, and their retries up to 220 GiB on the pool. A step that no node can fit stays until its workflow times out.
argocpuargocpu-largePendingMonitoring adds load to the general pool. When Prometheus, log collection, and database monitoring are all enabled (), their components request about 3.5 vCPUs and 12 GiB of memory across the pool. This leaves 8-vCPU nodes little headroom. With all three enabled, use general nodes of at least 16 vCPUs, or add a fourth 8-vCPU node.
configuration.monitoringThe exact mechanism to create node pools depends on your Kubernetes distribution (for example, static node labels, a node autoscaler, or a cluster API provider). If an autoscaler other than Karpenter scales the transformation pool from zero, reports the pool as missing while it is empty; that failure is expected.
shdctl cluster checkAll nodes in every pool must use the x86-64 () architecture. Unity publishes the platform's container images for only. Pods scheduled on Arm64 nodes, such as AWS Graviton, fail to pull their images and return an error that includes the following:
amd64linux/amd64no match for platform in manifest
GPU nodes are optional. To run transformations on GPUs, add a fourth pool as described in GPU nodes.
GPU nodes
By default, transformations don't use GPUs:
- Transformation actions that can use a GPU run on the pool, with a software renderer.
argocpu - Pipelines don't offer the GPU configuration profiles, and a step that uses one runs on CPU.
To run transformations on NVIDIA GPUs:
- Add a GPU node pool to your cluster. Its nodes must meet the GPU node requirements and have the NVIDIA driver and device plugin installed.
- In the manifest, set .
configuration.transformations.gpu.enabled: true - Verify the GPU nodes.
- Deploy the release. For more information, refer to Deploy the solution.
After the deployment:
- Pipelines offer the GPU configuration profiles, including in app versions that are already installed. A step that uses a GPU profile, such as Optimize and Convert on its GPU profile, requests a GPU.
- Other actions that can use a GPU request one in app versions that you install after the deployment. App versions that support it also switch to GPU from the next step that starts, without reinstalling the app.
To stop using GPUs, set the field back to , and deploy again. After the deployment:
false- Pipelines don't offer the GPU configuration profiles, and steps on a GPU profile run on CPU.
- Actions in app versions that support the change switch to CPU from the next step that starts, without reinstalling the app.
- Actions in other app versions keep the choice they were installed with. An action installed while GPUs were on still requests a GPU until you install a newer app version. If no GPU node is left, the action stays in the state until then.
Pending
GPU node requirements
Each node in the GPU node pool must meet the following requirements:
Requirement | Value |
|---|---|
| GPU | One NVIDIA GPU for each node, such as the NVIDIA T4. Each step uses a whole GPU. |
| CPU and memory | At least 8 vCPUs and 32 GiB of memory for each node, for example, a |
| NVIDIA driver | Installed on each node, with Vulkan support, because steps render through Vulkan on the GPU. |
| Device plugin | Running on each GPU node, so that the node advertises the |
| Taint | |
| Node label | Not required. GPU steps run on any node that advertises |
| Architecture | x86-64 ( |
Install the NVIDIA driver and device plugin
Whether you need to install the NVIDIA driver and device plugin yourself depends on your platform:
- Amazon EKS Auto Mode: You don't need to install anything. Nodes in a NodePool of NVIDIA GPU instances, such as the family, come with the NVIDIA driver and device plugin. The
g4dnin the release package includes such a NodePool,aws-baseline-example. Itsgpu-nodepool.yamlrecipe applies the NodePool when you run it withjust cluster-bootstrap.gpu=true - Amazon EKS with managed node groups: You don't need to install anything. Use the AMI type for the node group, which includes the driver and the device plugin.
BOTTLEROCKET_x86_64_NVIDIA - Other platforms: Install the NVIDIA driver and a device plugin that advertises . Follow your platform's documentation.
nvidia.com/gpu
On every platform, verify the GPU nodes afterward.
Scale the GPU node pool from zero
GPU nodes cost more than nodes in the other pools. To reduce costs, you can let an autoscaler scale the GPU node pool down to zero nodes when no GPU step is running. The trade-off is a delay: after an idle period, the first GPU step stays in the state while a GPU node starts and pulls the step's image. This wait counts toward the workflow's timeout. If your pipelines can't absorb the delay, keep at least one GPU node running.
Pendingshdctl cluster check- A node in the state that advertises
Ready.nvidia.com/gpu - A Karpenter NodePool that declares the taint.
nvidia.com/gpu
If another autoscaler scales the GPU node pool from zero, the pool has no node while it's empty, so the check shows a warning instead of passing. A warning doesn't fail the command.
Verify the GPU nodes
To check that the GPU nodes are ready for GPU steps:
-
List the nodes and the number of GPUs that each one advertises:kubectl get nodes -o custom-columns='NAME:.metadata.name,GPU:.status.allocatable.nvidia\.com/gpu'Each GPU node showsin the
1column. Nodes in the other pools showGPU. If a GPU node shows<none>, check its NVIDIA driver and device plugin. For more information, refer to Install the NVIDIA driver and device plugin.<none> -
Run, and check that the
shdctl cluster checkcheck passes. For more information, refer to Validate your cluster.gpu nodes
Restrict GPU steps to one node pool
By default, a step on a GPU configuration profile runs on any node that advertises . To restrict these steps to one node pool:
nvidia.com/gpu-
In the manifest, add the following settings under:
configuration.overrides.automation.values.common-chart.configMaps.automation-config.data- : the key of the node pool's label.
UCAUTOMATION_ArgoTemplateOptions__NodeSelectorLabels__UsesGPU__Label - : the value of the node pool's label.
UCAUTOMATION_ArgoTemplateOptions__NodeSelectorLabels__UsesGPU__Value
For example:configuration: overrides: automation: values: common-chart: configMaps: automation-config: data: UCAUTOMATION_ArgoTemplateOptions__NodeSelectorLabels__UsesGPU__Label: "<label-key>" UCAUTOMATION_ArgoTemplateOptions__NodeSelectorLabels__UsesGPU__Value: "<label-value>" -
Generate and deploy the charts again. For more information, refer to Deploy the solution.
This restriction doesn't apply to 3D Data Streaming. Its GPU steps still run on any node that advertises .
nvidia.com/gpuIPv6 clusters
The platform runs on IPv4 clusters by default. To deploy on a single-stack IPv6 cluster:
- Make sure that the cluster's nodes, pods, and Kubernetes Services use IPv6 networking. Each node must report an IPv6 address. To check, run
InternalIP, and look at thekubectl get nodes -o widecolumn.INTERNAL-IP - In the manifest, set .
configuration.networking.ipFamily: ipv6
shdctl cluster checkInternalIPNetwork policies
During deployment to an existing cluster, you may need to control the flow of network traffic by using network policies. You can deploy most Kubernetes resources in a single namespace, which you can use to scope network isolation from other services that run in your cluster.
Clusters behind an HTTP proxy
If your AKS cluster sends its outbound traffic through an HTTP proxy (AKS HTTP proxy), AKS injects , and into every pod. Platform services call each other by their short Service names, which no entry matches, so those calls go to the proxy and fail.
HTTP_PROXYHTTPS_PROXYNO_PROXYNO_PROXYTo keep the platform pods off the proxy, add the AKS opt-out annotation to every pod in the manifest file. Nodes still pull images through the proxy, but the pods get no internet access:
configuration: kubernetes: podAnnotations: kubernetes.azure.com/no-http-proxy-vars: "true"
The automation-manager job pulls the release manifest from your registry itself, so it still needs the proxy. Unless your cluster reaches the registry without the proxy, pass it through the chart's overrides:
configuration: overrides: automation-manager: values: env: httpsProxy: http://<proxy>:<port> noProxy: .svc,.cluster.local,localhost,127.0.0.1
Alertmanager carries these annotations too, so a receiver outside the cluster needs in its .
httpConfig.proxyURLAlertmanagerConfigconfiguration.kubernetes.podAnnotationsThe in the release package can create such a cluster: sets the proxy, makes the API server private, and denies all other outbound traffic from the cluster's subnet. Its README describes each variable.
azure-baseline-exampleaks_http_proxyaks_private_cluster_enabledaks_egress_via_proxy_onlyWhat the proxy must allow
- AKS's own outbound traffic. Nodes bootstrap and keep running through the proxy, so it must allow the destinations in AKS outbound network and FQDN rules, for example and
mcr.microsoft.com.packages.aks.azure.com - Your container registry. Nodes pull every image through the proxy, and the automation-manager job reaches the registry through it too: the registry in , or
configuration.kubernetes.docker.repositoryif you pull from it directly. Allow the endpoints the registry serves image layers from as well; for Azure Container Registry, refer to Configure rules to access an Azure container registry behind a firewall.uccmpprivatecloud.azurecr.io - Requests from the nodes. Pod traffic reaches the proxy from its node's address, so allow the node subnet as a source.
- HTTP and HTTPS only. HTTPS goes through the proxy as on port 443, and plain HTTP on port 80. Outbound traffic that isn't HTTP, such as SMTP or SSH, has no path through the proxy.
CONNECT - TLS inspection. If the proxy decrypts HTTPS, give the cluster its CA certificate (in the AKS HTTP proxy configuration). The certificates must use subject alternative names.
trustedCa
Container registry
By default, the platform pulls container images directly from the Unity source registry (). This is the simplest setup and doesn't require a separate registry.
uccmpprivatecloud.azurecr.ioIf your environment is air-gapped or if you need full control over artifact distribution, you can mirror artifacts to your own private container registry. Any OCI-compliant registry is supported, for example Harbor, JFrog Artifactory, or a cloud-managed registry such as Amazon ECR or Azure Container Registry. Use these commands to mirror artifacts from the Unity source registry to your private registry:
- : this command syncs the Docker container images.
shdctl artifact sync images - : this command syncs the ORAS artifacts, for example, the Pixyz workflow templates.
shdctl artifact sync oras - : this command checks that your registry holds every image your manifest needs.
shdctl artifact sync verify
System requirements
Ensure that you have these elements:
-
A valid hostname that can be updated to point to the IP of the load balancer
-
A valid Unity Version Control (UVCS) license to run the UVCS server
-
A valid Unity Asset Transformer SDK license to run transformations on assetsA valid Unity Asset Transformer license is required for all users who perform asset transformations. The platform uses static licenses, so a floating license server is not required.
Validate your cluster
Once your exists, check the prerequisites on this page automatically. The command reads the namespace, the storage classes, and the ArgoCD settings from the manifest, and step 3 of the deployment runs it:
manifest.yamlshdctl cluster check
The command verifies:
- The Kubernetes version.
- The number of schedulable nodes.
- The transformation node pools.
- The storage classes.
- The target namespace.
- The metrics API.
- The in-cluster ArgoCD installation if your manifest configures ArgoCD. On the Helm deployment path, these checks are skipped.
The command reads from the cluster and doesn't change it. It prints a remediation hint for anything that doesn't pass.
Add to verify that each storage class can provision a volume, including support.
--probe-storageReadWriteManyFor the full list of checks and options, refer to shdctl cluster command.