Off-and-on trying out an account over at @tal@oleo.cafe due to scraping bots bogging down lemmy.today to the point of near-unusability.

  • 23 Posts
  • 1.65K Comments
Joined 3 years ago
cake
Cake day: October 4th, 2023

help-circle






  • I was once told “I write like someone who thinks they are good at writing.” That was met with rounds of applause by the people who can’t (or won’t) spell, capitalize, or punctuate anything.

    https://en.wikipedia.org/wiki/Tall_poppy_syndrome

    Tall poppy syndrome is a term that originated in Australia and New Zealand in the 1980s that refers to people with notable public success or achievements that are ostracised by others as a form of egalitarianism.[1][2] Intense scrutiny and criticism of such a person is termed as “cutting down the tall poppy”.[3]

    “That one sounds uppity!”









  • Most commercial establishments in the US are air-conditioned. Taiwan has — as you point out — a pretty warm, humid climate. Are Taiwanese bookstores not not typically air-conditioned? I’d think that they’d be a prime candidate.

    I was reading about public library humidity and temperature control a month or two back, and it sounded like it was also pretty much the norm to air condition those, for the preservation reasons that you mention.


  • Chief among them was Seewa AI, which offered its users the “perfect AI girlfriend”, drew them in with intimate conversations and promised that all chats were strictly confidential. (They weren’t.)

    End of the day, if it’s in the cloud instead of local — and this applies for pretty much any service, not just LLM models — you’re relying on trust in the service provider (as well as anyone who might buy them, and trust that they are able to secure their network against people who might break into it).

    Only real exception to that is situations where you use client software from another source and the service provider only gets to see any data in encrypted form — think of someone providing storage for a restic backup, say.

    In 2026, we don’t have enough memory to do local LLM computation everywhere, so there’s a real limitation versus most other forms of SaaS — we can’t choose to run LLMs locally for more than a fraction of society. Cloud-based LLMs require less memory to provision one machine in the cloud and share that hardware, since any one user is only using it a fraction of the time. But as the memory shortage ends, I expect you’ll have more people running LLMs locally. Still be more-expensive from a hardware standpoint than cloud computation, though.


  • Steam can’t really “take the game back” from you (again, at least the offline functionality).

    I mean…sort of. Steam will only run in offline mode if you first run it in online mode and ask Steam to allow the machine to enter offline mode. I imagine that this is to keep people from simply installing games on many PCs and then taking them all offline.

    checks

    Yeah, sounds like a maximum of one host can be offline for an account, and I imagine that they fingerprint hosts.

    Some games don’t require Steam to be running. Like, Steam just downloads them and launches them, and they still work even if you launch them without Steam running, don’t use Steamlib. For those, yes.

    There are other PC game vendors who do explicitly support this mode of operation, like GOG.




  • Like, two RX 7900 XTXs as an alternative to one RTX 5090?

    For some applications.

    For games, in general, I believe no, not in 2026. GPUs did have an era where there was some real deployment of games that could leverage multiple GPUs, but I understand that this hasn’t been supported for quite a while.

    https://en.wikipedia.org/wiki/Scalable_Link_Interface

    Scalable Link Interface (SLI) is the brand name for a now discontinued multi-GPU technology developed by Nvidia for linking two or more video cards together to produce a single output. The technology was invented and developed by 3dfx and later purchased by Nvidia during the acquisition of the company. SLI is a parallel processing algorithm for computer graphics, meant to increase the available processing power.[1]

    For AI compute, it depends a lot on what exactly you’re doing.

    If you want to do two different things that both require no more than one card, sure. Like, Card A is running Stable Diffusion to generate images and Card B is running Gemma to generate text.

    If you want to run a model designed for a larger system that doesn’t fit on a single card’s VRAM…maybe.

    If you can fit some of the model’s neural net layers on one card, then you can do the compute on one card, then (slowly) move outputs from one card to another card, then compute more layers there. I know, off the cuff, that KoboldAI, which is for use with text LLMs, can make use of multiple cards (as well as the CPU) in this way. I think that it may (or, at least, did) suffer from the same limitation that I saw in a video that Jeff Gearling did last year where he was experimenting with clustering Framework Desktop systems (each of which can have 128GB of VRAM) to build larger clusters that could run very large LLMs.

    https://www.jeffgeerling.com/blog/2025/i-clustered-four-framework-mainboards-test-huge-llms/

    He could run ollama on multiple systems, but at least at the time, it wasn’t capable of running batched work on cards concurrently. So you’d run some layers on Framework Desktop A, then ollama would move data over to Framework Desktop B and do more work there while Framework Desktop A was idle. That’s not a fundamental hardware limitation, just a software limitation. It meant that while you could run larger models that would otherwise be practical (e.g. four 128GB Framework Desktop systems could run a neural network that’d normally take half a terabyte of memory to work with), you wouldn’t be making use of all of their compute capability concurrently. What he was getting was more-or-less analogous to the compute power of a single Framework Desktop with 512GB of memory. But…there shouldn’t be a hard limitation. As long as you have a queue of requests to feed hardware that use the same model, I imagine that it should be possible to keep all your cards hot. And that limitation might have already been addressed. I think I remember reading about another LLM engine (vllm?) that can do this…but I’m going out of fuzzy memory; I wasn’t actually aiming to build a cluster myself.

    It’s also possible, if you’re really going to commit to a particular hardware layout that uses multiple independent cards, to build a neural network designed for this particular card layout. I’d guess that that probably makes more sense if you’re…oh, say, a Chinese AI company with a lot of funds to build out training infrastructure that isn’t sure that it’s going to be able to get its hands on US hardware that’s really dedicated to AI. Then you train a model that runs a neural network on consumer GPU card A. That neural network has only sparse connections, only a few of them, to a neural network on card B, so it’s cheap in terms of bandwidth to move outputs from card A to Card B. In terms of the quality of what the neural network generates, that won’t perform as well as a system that doesn’t have that requirement to be sparsely-connected, but it’s very possible that that performance limitation may not be very significant — I think that the question of how significant it is probably a very interesting and presently-unresolved question for many countries and businesses involved in the industry, since if you can use consumer hardware in parallel to effectively compete with models running on dedicated AI hardware, that eliminates a lot of the “moat” that might let, say, Nvidia price-discriminate between gaming and AI customers, the US ensure that it has a military or industrial advantage relative to China, or a given AI company limit competition in a given field based on access to hardware.

    For non-AI parallel compute tasks, like…I dunno, say, cracking passwords via trying to brute-force reverse a cryptographic hash or something, in most cases a task can probably be parallelized and probably doesn’t care much about VRAM anyway (at least compared to AI compute), probably yes, as long as your software can handle it. I had a buddy who used clusters of machines with parallel compute hardware to do multi-body simulations for astronomy stuff, and I’m sure it would work there, since he was, well, doing it. I wouldn’t expect any fundamental limitation to be coming from the hardware there.

    EDIT: A more-prominent concern if we’re talking what someone might immediately care about between the two cards if they’re doing some forms of AI compute is probably whether they’re willing to pay a premium for Nvidia hardware to use Nvidia’s software stack, which is more mature than AMD’s. Like, ROCm is definitely less mature than CUDA, and while I’ve been pretty comfortable using Vulkan on AMD hardware to do AI compute with llama.cpp, some software needs to go through ROCm (e.g. ComfyUI), and my experience there has definitely been less positive (though to be fair, I’m also determined to use it on Debian, whereas AMD presently only officially supports Ubuntu, Fedora, and Arch kernels with the newer versions of ROCm, so…shrugs).