NVIDIA has unveiled Chat with RTX, a client that runs Generative AI chatbots on our own personal computers, eliminating the need for external servers.
Chat with RTX is a platform that utilizes open-source language models (currently supporting Llama 2 and Mistral) to run on the Tensor Cores of NVIDIA GeForce RTX Series 30 and above (powered by TensorRT-LLM and RAG on Windows). Currently, this platform is only supported on Windows PCs.
The standout feature of Chat with RTX is that both the LLM model and the data that the model needs to read are stored on the local PC. It can read from files such as txt, pdf, doc/docx, xml, within specified folders, thus preventing data leakage. However, it can also reference internet-based data, such as having the AI watch YouTube videos with specified parameters.
NVIDIA states that Chat with RTX is currently a tech demo and has not yet indicated whether it will be further developed in the long term. Nevertheless, it serves as an example to demonstrate that running AI chatbots locally is now possible.
TLDR: NVIDIA has introduced Chat with RTX, a client that enables the running of Generative AI chatbots on personal computers without relying on external servers. This platform supports open-source language models and runs on NVIDIA GeForce RTX Series 30 and above. It can read local files and prevent data leakage, while also being able to reference internet-based data. Although categorized as a tech demo, it showcases the feasibility of running AI chatbots locally.
Leave a Comment