forster Posted January 7 Share Posted January 7 So I'm a bit of a computer/networking freak, and I've some spare machines kicking around of varying levels of computing goodness. I wondered what our members are using with relation to AI - specifically, if you are running AI workloads: What hardware are you using What software are you using Which AI model/LLM are you using What kind of benefit are you getting out of it As someone who is very AI-curious it would be super-duper if you could enlighten me and give me (and everyone else) a starting point as to what you're running and how you're doing it. I appreciate this may be a mega involved question, so feel free to just give me (us) the basics, but if you could point towards any resources that helped you get started it would be of great educational help. Thank you! Link to comment https://www.neowin.net/forum/topic/1450569-neowins-ai-hardwaresoftwareworkloads/ Share on other sites More sharing options...
Nick H. Supervisor Posted January 7 Supervisor Share Posted January 7 Right now? Nothing. I don't like the AI situation of the moment, I don't like the scraping of my information in order to improve a company's AI, plus the possibility that the information that is scraped could be provided to another party without my permission. There are too many faults at the moment for me to think about using it. A local LLM machine though? I'm looking into that. It only plays off my data, and my data is with me on my network. That sounds fun. acetken 1 Share Link to comment https://www.neowin.net/forum/topic/1450569-neowins-ai-hardwaresoftwareworkloads/#findComment-598957131 Share on other sites More sharing options...
forster Posted January 7 Author Share Posted January 7 On 07/01/2025 at 19:37, Nick H. said: Right now? Nothing. I don't like the AI situation of the moment, I don't like the scraping of my information in order to improve a company's AI, plus the possibility that the information that is scraped could be provided to another party without my permission. There are too many faults at the moment for me to think about using it. A local LLM machine though? I'm looking into that. It only plays off my data, and my data is with me on my network. That sounds fun. Expand Local is where it's at - ChatGPT is great but if I had a LLM in a VM, with tailscale allowing me to access it wherever and whenever I want, even if it doesnt have an elegant UI, I'd certainly go for it. Link to comment https://www.neowin.net/forum/topic/1450569-neowins-ai-hardwaresoftwareworkloads/#findComment-598957133 Share on other sites More sharing options...
Mindovermaster Global Moderator Posted January 7 Global Moderator Share Posted January 7 I'm with Nick, I'd rather stay away from it. Remember what happened in the Matrix? Link to comment https://www.neowin.net/forum/topic/1450569-neowins-ai-hardwaresoftwareworkloads/#findComment-598957135 Share on other sites More sharing options...
forster Posted January 7 Author Share Posted January 7 On 07/01/2025 at 19:43, Mindovermaster said: I'm with Nick, I'd rather stay away from it. Remember what happened in the Matrix? Expand Neo got real-world super powers? Damn! Im into that! Link to comment https://www.neowin.net/forum/topic/1450569-neowins-ai-hardwaresoftwareworkloads/#findComment-598957138 Share on other sites More sharing options...
Mindovermaster Global Moderator Posted January 7 Global Moderator Share Posted January 7 On 07/01/2025 at 19:46, forster said: Neo got real-world super powers? Damn! Im into that! Expand Well, Neo was technically Jesus, so.. Talking more about the machines sent to kill the human race. forster 1 Share Link to comment https://www.neowin.net/forum/topic/1450569-neowins-ai-hardwaresoftwareworkloads/#findComment-598957141 Share on other sites More sharing options...
forster Posted January 7 Author Share Posted January 7 On 07/01/2025 at 19:48, Mindovermaster said: Well, Neo was technically Jesus, so.. Talking more about the machines sent to kill the human race. Expand "Do you see the beauty of it? The inevitability? You rise, only to fall. You, Avengers, you are my meteor. My swift and terrible sword and the Earth will crack with the weight of your failure. Purge me from your computers; turn my own flesh against me. It means nothing! When the dust settles, the only thing living in this world... will be metal!" Mindovermaster 1 Share Link to comment https://www.neowin.net/forum/topic/1450569-neowins-ai-hardwaresoftwareworkloads/#findComment-598957145 Share on other sites More sharing options...
Nick H. Supervisor Posted January 7 Supervisor Share Posted January 7 On 07/01/2025 at 19:46, forster said: Neo got real-world super powers? Damn! Im into that! Expand On 07/01/2025 at 19:54, forster said: "Do you see the beauty of it? The inevitability? You rise, only to fall. You, Avengers, you are my meteor. My swift and terrible sword and the Earth will crack with the weight of your failure. Purge me from your computers; turn my own flesh against me. It means nothing! When the dust settles, the only thing living in this world... will be metal!" Expand Pick a lane. Matrix, Avengers... Jokes aside, we appear to be aligned in the idea of AI for the moment. Local > off-site for data privacy. I think I've got my new project for my Pi. My only thought is, "how much will I really use it?" Link to comment https://www.neowin.net/forum/topic/1450569-neowins-ai-hardwaresoftwareworkloads/#findComment-598957153 Share on other sites More sharing options...
forster Posted January 7 Author Share Posted January 7 On 07/01/2025 at 20:28, Nick H. said: Pick a lane. Matrix, Avengers... Jokes aside, we appear to be aligned in the idea of AI for the moment. Local > off-site for data privacy. I think I've got my new project for my Pi. My only thought is, "how much will I really use it?" Expand Avengers for sure. The dream is the general AI Jarvis, the natural UI speech recognition helper that automates all the things. That’s all I want! I use ChatGPT now and again, it’s a utility, like water, or electricity. I don’t use it all the time, but when I need it, it’s there. So especially if I can run it locally and it’s there when I need it, that has to be worth its weight in gold… or megabytes, or mibibytyes. Or whatever. Link to comment https://www.neowin.net/forum/topic/1450569-neowins-ai-hardwaresoftwareworkloads/#findComment-598957155 Share on other sites More sharing options...
Nick H. Supervisor Posted January 7 Supervisor Share Posted January 7 On 07/01/2025 at 20:41, forster said: Avengers for sure. The dream is the general AI Jarvis, the natural UI speech recognition helper that automates all the things. That’s all I want! I use ChatGPT now and again, it’s a utility, like water, or electricity. I don’t use it all the time, but when I need it, it’s there. So especially if I can run it locally and it’s there when I need it, that has to be worth its weight in gold… or megabytes, or mibibytyes. Or whatever. Expand J.A.R.V.I.S. is local. No way would Tony let his AI's memory get accessed from outside of his network...didn't that happen at some point and he needed to reel it in? I'm spinning off on a tangent with this. Let's get back to the point of the discussion. Link to comment https://www.neowin.net/forum/topic/1450569-neowins-ai-hardwaresoftwareworkloads/#findComment-598957158 Share on other sites More sharing options...
Third Envoqation Posted January 7 Share Posted January 7 Currently using a Asus ProArt X670 with a 7900X, 64Gb RAM and a 7900XT GPU and a 10Gb/e link to the NAS for back up and model archives The system has around 10Tb of storage over 3 NVMe drives. I use AI models for illustration, design, and also for D&D type lore building with various models, not settled on any one model yet. The LLM tools I use are SillyTavern for conversational stuff, msty and LLM studio. Clients I use for AI gen are Invoke AI and ComfyUI. For image gen I using WSL ROCm whereas LLMs are native on Win 11. Also using it to try and convert about twenty odd years of computer manuals into machine readable formats, such as printer, workstation and server maintenance manuals which is going about as you'd think due to the abysmal PDF standards of the 2000s resulting in a lot of OCR work. I am about to build a pure Linux server for AI work, still undecided for the actual spec, but it won't be a nVidia GPU in it, just too damned rich for what is essentially a back up hobby machine. forster 1 Share Link to comment https://www.neowin.net/forum/topic/1450569-neowins-ai-hardwaresoftwareworkloads/#findComment-598957160 Share on other sites More sharing options...
BlaineMontgomery Posted March 5 Share Posted March 5 Thank you so much for the info. Link to comment https://www.neowin.net/forum/topic/1450569-neowins-ai-hardwaresoftwareworkloads/#findComment-598971144 Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now