TECH NEWS -Interest in Nvidia can only be described in superlatives. The same cannot be said for AMD, however.
AMD‘s MI infrastructure has not been the top choice for hyperscalers lately, especially after Nvidia‘s Blackwell series debuted. SoftBank‘s name comes up several times when talking about AMD‘s most important customers, and this time their technology division has created something very interesting. According to SoftBank‘s latest blog post, Orchestrator has been paired with AMD Instinct AI chips. The goal is to distribute computing resources based on workload intensity and availability.
“In collaboration with AMD, SoftBank has developed an enhanced Orchestrator feature that leverages the GPU partitioning capabilities of AMD Instinct GPUs. These capabilities allow a single GPU to function as multiple logical devices. This feature enables flexible and optimal GPU resource allocation based on AI application requirements, such as model size and concurrency,” wrote SoftBank.
SoftBank Orchestrator focuses on distributing the computational load between AMD Instinct GPUs by segregating loads based on multiple GPU instances running on each Accelerator Complex Die (XCD) unit. There can be a single instance (SPX mode) or up to eight (CPX mode), with each configuration increasing the level of detail. In addition to XCD sharing, Orchestrator leverages AMD‘s high-capacity memory and divides it into unique HBM regions associated with each GPU instance.
SoftBank wanted to use Orchestrator to achieve lower-level computing resource control while ensuring strict hardware isolation to avoid unpredictable latency spikes. The company has not yet released performance data but mentions optimal resource allocation, which is more efficient in SLM and MLM workloads.
SoftBank also plans to try such orchestrators with other AI accelerators; however, for now, implementation is limited to AMD.
