TECH NEWS – Nvidia has strengthened its AI ecosystem, and anyone with a relatively new graphics card can now try out the chatbot.

Chat With RTX is available for Windows PCs and is powered by TensorRT-LLM. It doesn’t require an internet connection, so you can always access the personalized GPT chatbot on your PC if you have a GeForce RTX 3000 or 4000 graphics card. (Presumably the RTX A-series and RTX Ada-series workstation GPUs are supported; the RTX 2000 that appeared in yesterday’s news is certainly supported, as it also mentioned TensorRT-LLM!)

Nvidia is using TensorRT-LLM and the Retrieval Augmented Generation (RAG) software. These were announced by the company last year and can be attached by the user to a local dataset (.txt, .pdf, .doc, .docx, .xml file format) to be linked to e.g. the Mistral or Llama 2 Large Language Model (LLM). Some links (e.g. YouTube videos or playlists) can also be added to improve the database search results. Then we can use Chat With RTX in the same way as ChatGPT: we can make queries, and since the results are based on the provided datasets, we can get better answers.

All data and projects are stored locally (but this can take up a lot of space, so be careful!), not in the cloud. This saves time and leads to more accurate results. RAG is one of the techniques that makes AI hits faster with the local directory, and we can take advantage of LLM’s language interpretation capabilities to make our hits more accurate. Nvidia says the performance gains are fivefold with TensorRT-LLM v0.6.0 coming later this month, which will bring support for Mistral 7B and Nemotron 3 8B LLMs.

So, the system requirements are that you are running Windows 10 or Windows 11 and have an Ampere or Ada Lovelace architecture Nvidia card in your PC. Once you have that, you can start updating the driver (the newer the better, it also optimizes the running of TensorRT-LLM), then you can download the Chat With RTX application in no time.

Source: WCCFTech, Nvidia