TECH NEWS – Following Nvidia’s Chat With RTX, here’s AMD’s own localized chatbot, powered by a GPT-based large language model (LLM) and requiring relatively new hardware.

Back in February, we reported on Chat With RTX, which can run locally on Nvidia’s GeForce RTX 4000 and 3000 cards using TensorRT-LLM, producing a local data packet. Now we have a similar one from AMD, but they don’t just have it running on their graphics cards. If you have an AMD AI-enabled CPU, you don’t need to get one of Radeon’s latest 7000 series graphics cards to get the technology working.

XDNA NPU (MI accelerator) cores can be used in AMD Ryzen 7000 and 8000 APUs, and MI accelerator cores can be used in Radeon RX 7000 cards. The company has published a blog post on how to launch the chatbot on your PC. Download and run the appropriate version of LM Studio (for AMD Ryzen processors, for AMD Radeon RX 7000 series graphics cards). In the Search tab, copy and paste the following search term depending on what you want to run. In the case of Mistral 7b it should be “TheBloke/OpenHermes-2.5-Mistral-7B-GGUF” and select it from the results on the left, chances are it will be the first in the list. To run LLAMA v2 7b, search for “TheBloke/Llama-2-7B-Chat-GGUF”. You can also experiment with other models: in the right pane, scroll down to the Q4 KM model file and download it. AMD recommends this for most Ryzen AI models. Switch to the chat box and wait for it to start.

It’s a bit different if you don’t have this processor and are using an AMD Radeon 7000 graphics card instead. In this case, you will need to select the GPU Offload option in the right panel and turn it all the way up. Then check that the identified GPU type is AMD ROCm. Then we can start using the chatbot.

Artificial intelligence is becoming part of our lives in all sorts of ways, and that’s why we’re seeing expansion from Nvidia and AMD.

Source: WCCFTech, AMD, LM Studio, LM Studio ROCm