You are correct in your understanding. However the last part of your comment needs a big asterisk. Its important to consider quantization.

The full f16 deepseek r1 gguf from unsloth requires 1.34tb of ram. Good luck getting the ram sticks and channels for that.

The q4_km mid range quant is 404gb which would theoretically fit inside 512gb of ram with leftover room for context.

512gb of ram is still a lot, theoretical you could run a lower quant of r1 with 256gb of ram. Not super desirable but totally doable.

I have been using deephermes daily. I think CoT reasoning is so awesome and such a game changer! It really helps the model give better answers especially for hard logical problems. But I don't want it all the time especially on an already slow model. Being able to turn it on and off wirhout switching models is awesome. Mistral 24b deephermes is relatively uncensored, powerful and not painfully slow on my hardware. a high quant of llama 3.1 8b deephermes is able to fit entirely on my 8gb vram.

Very interesting stuff! Thanks for sharing.

To me it's more like the numbers mean something different. While they still mean mostly nothing, the numbers that display on your profile tell what kind of user you are and how much seniority you have here. Been here close to two years? Maybe you came from the API fiasco. Got 1000+ comments and 50+ post? Somewhat active user you probably seen once or twice before. 0 post and below 50 comments but seemingly active? Potential lurker. Still mostly meaningless because it could be alt and who really cares but NGL when I look at the stats of the really active users I'm like damn they really contributed to the Lemmy content mill.

You are free to pick an instance that aligns with your values and preferences for moderation. Its a double edged sword because it enables echo chambers which I think isnt great but it seems many people like those actually.

The sub's, filters, and block list serves as a manual replacement for the algorithm. Its hard building it up but once you do Lemmy becomes mostly enjoyable as long as you keep to what you like.

What is it? Oh I see the sticker now :-) yes quite the beastly graphics card so much vram!

Its all about ram and vram. You can buy some cheap ram sticks get your system to like 128gb ram and run a low quant of the full deepseek. It wont be fast but it will work. Now if you want fast you need to be able to get the model on some graphics card vram ideally all of it. Thats where the high end Nvidia stuff comes in, getting 24gb of vram all on the same card at maximum band with speeds. Some people prefer macs or data center cards. You can use amd cards too its just not as well supported.

Localllama users tend use smaller models than the full deepseek r1 that fit on older cards. 32b partially offloaded between a older graphics card and ram sticks is around the limit of what a non dedicated hobbiest can achieve with ther already existing home hardware. Most are really happy with the performance of mistral small and qwen qwq and the deepseek distills. those that want more have the money to burn on multiple nvidia gpus and a server rack.

LLM wise Your phone can run 1-4b models, Your laptop 4-8b, your older gaming desktop with a 4-8gb vram card can run around 8-32b. Beyond that needs the big expensive 24gb cards and further beyond needs multiples of them.

Stable diffusion models in my experience is very compute intensive. Quantization degredation is much more apparent so You should have vram, a high quant model, and should limit canvas size as low as tolerable.

Hopefully we will get cheaper devices meant for AI hosting like cheaper versions of strix and digits.

Thank you! I like to spread the word about things I feel passionate about. Theres so much crap that promises to improve your life and only a few good things that actually do. Dry herb vapes rocked my world and its my privilege to potentially be the internet comment ear worm that eventually convinces some to try the journey to see if it changes their world too.

Unfortunately just a lot of close minded individuals who are happy with what they got going on and dont understand the point or had a bad experience 10 years ago or just confuse it with cartridge vaping. Some people dont like the look of something so they refuse to ever try it. Just because it didn't pass their vibe check. Its fustrating but thats people and stubborn tradition for you. I believe that if its meant for you, then eventually it will find a way into your life when you need it.

Been treating my insomnia with the good stuff pretty much daily over a decade. Dry herb microdose vaping and getting down the timing of the high cycle is key to maintaining tolerance in the long term.

Let's touch on the later point quick first. Everyones different but for me being high is something like vape -> get high -> crashout(sleepy) -> (caffeine or more vaping). For me its like 30min-180mins of high followed by the crash depending on bud quality and its strains terp composition. If you can establish a time based sleep schedule to align your circadian rythm while timing the final crashout of the day at the same general time youre golden. If bedtime were 9pm I would smoke up two hours before maybe one moe hit 30 mins before bed. Usually if I'm crashing extra hit not needed.

Now for tolerance. Im going to be blunt with you, most cannabis smokers/vapers are doing it.... unscientifically. The first thought is just to smoke more to combat tolerance which works OK until you run out of bud or your lungs are covered in black tar. They've never heard of a dry herb vape or if they did its a decade old dinosaur like the fucking pax, never considered microdosing. For som reason most never wanted to understand what the bare minimum and healthiest vaping methods is. If your burning your bud, your doing more harm to your lungs than good to your brain. Also your wasting your herb big time. Sorry, thats how it is.

The journey that these questions lead me on changed my life and truly turned the herb into dosable medicine. You want to stop building tolerance forever? You need to work your way down to 0.05g-0.10g dry herb hits green. Its effectively the smallest unit of bud for an appreciable hit. You can microdose all day and never build appreciable T. it will basically fully reset tommorow or the day after.

And guess what? All the black and brown leftover After Vaped Bud is still good for use. Its fully decarbed and chock full of CBD,CBN, and some leftover THC. Save it up in a jar and process into nighttime cannaoil sleeping pills

Which ones are not actively spending an amount of money that scales directly with the number of users?

Most of these companies offer direct web/api access to their own cloud supercomputer datacenter, and All cloud services have some scaling with operation cost. The more users connect and use computer, the better hardware, processing power, and data connection needed to process all the users. Probably the smaller fine tuners like Nous Research that take a pre-cooked and open-licensed model, tweak it with their own dataset, then sell the cloud access at a profit with minimal operating cost, will do best with the scaling. They are also way way cheaper than big model access cost probably for similar reasons. Mistral and deepseek do things to optimize their models for better compute power efficency so they can afford to be cheaper on access.

OpenAI, claude, and google, are very expensive compared to competition and probably still operate at a loss considering compute cost to train the model + cost to maintain web/api hosting cloud datacenters. Its important to note that immediate profit is only one factor here. Many big well financed companies will happily eat the L on operating cost and electrical usage as long as they feel they can solidify their presence in the growing market early on to be a potential monopoly in the coming decades. Control, (social) power, lasting influence, data collection. These are some of the other valuable currencies corporations and governments recognize that they will exchange monetary currency for.

but its treated as the equivalent of electricity and its not

I assume you mean in a tech progression kind of way. A better comparison might be is that its being treated closer to the invention of transistors and computers. Before we could only do information processing with the cold hard certainty of logical bit calculations. We got by quite a while just cooking fancy logical programs to process inputs and outputs. Data communication, vector graphics and digital audio, cryptography, the internet, just about everything today is thanks to the humble transistor and logical gate, and the clever brains that assemble them into functioning tools.

Machine learning models are based on neuron brain structures and biological activation trigger pattern encoding layers. We have found both a way to train trillions of transtistors simulate the basic information pattern organizing systems living beings use, and a point in time which its technialy possible to have the compute available needed to do so. The perceptron was discovered in the 1940s. It took almost a century for computers and ML to catch up to the point of putting theory to practice. We couldn't create artificial computer brain structures and integrate them into consumer hardware 10 years ago, the only player then was google with their billion dollar datacenter and alphago/deepmind.

Its exciting new toy that people think can either improve their daily life or make them money, so people get carried away and over promise with hype and cram it into everything especially the stuff it makes no sense being in. Thats human nature for you. Only the future will tell whether this new way of precessing information will live up to the expectations of techbros and academics.

Theres more than just chatgpt and American data center/llm companies. Theres openAI, google and meta (american), mistral (French), alibaba and deepseek (china). Many more smaller companies that either make their own models or further finetune specialized models from the big ones. Its global competition, all of them occasionally releasing open weights models of different sizes for you to run your own on home consumer computer hardware. Dont like big models from American megacorps that were trained on stolen copyright infringed information? Use ones trained completely on open public domain information.

Your phone can run a 1-4b model, your laptop 4-8b, your desktop with a GPU 12-32b. No data is sent to servers when you self-host. This is also relevant for companies that data kept in house.

Like it or not machine learning models are here to stay. Two big points. One, you can self host open weights models trained on completely public domain knowledge or your own private datasets already. Two, It actually does provide useful functions to home users beyond being a chatbot. People have used machine learning models to make music, generate images/video, integrate home automation like lighting control with tool calling, see images for details including document scanning, boilerplate basic code logic, check for semantic mistakes that regular spell check wont pick up on. In business 'agenic tool calling' to integrate models as secretaries is popular. Nft and crypto are truly worthless in practice for anything but grifting with pump n dump and baseless speculative asset gambling. AI can at least make an attempt at a task you give it and either generally succeed or fail at it.

Models around 24-32b range in high quant are reasonably capable of basic information processing task and generally accurate domain knowledge. You can't treat it like a fact source because theres always a small statistical chance of it being wrong but its OK starting point for researching like Wikipedia.

My local colleges are researching multimodal llms recognizing the subtle patterns in billions of cancer cell photos to possibly help doctors better screen patients. I would love a vision model trained on public domain botany pictures that helps recognize poisonous or invasive plants.

The problem is that theres too much energy being spent training them. It takes a lot of energy in compute power to cook a model and further refine it. Its important for researchers to find more efficent ways to make them. Deepseek did this, they found a way to cook their models with way less energy and compute which is part of why that was exciting. Hopefully this energy can also come more from renewable instead of burning fuel.

Theoretically you may be able to store the core seed information that encodes the starting constants that lead to the beginning of the universe. Its not really the same thing like the difference between a cake and the recipe used to make it. Information systems can be distilled to core seed equations and regenerated by iterating that equation many times. This is Barnsley's collage theorem.

Any tips on sexing cannabis early?

Diy success: building my own solar system that works over a year later.

Diy failure: upgrading a computer with new GPU as a young teenager, not understanding different graphics card sizes and case limits, getting a three fan to replace a two fan, forcing it into the case (I forget how I modified things to fit) and having the whole thing blow up a few months later.

Diy success: wiring up a cheap induction heater board when I couldn't afford a nice one.

Diy failure: not giving a shit about proper project boxes. Also using electrical tape, heat shrink, splicing screw caps, and quick disconnects instead of soldering (I fucking hate soldering and welding hate working with molten metal liquids man). I'll never be able to flex my project online without fellow electrical engineers rightfully calling me out on my lack of code following, could-go-wrongisms, and general poor layout.

Diy success: turning a old gaming computer into a local model engine server.

Diy failure: when I was just learning how to use a voltmeter I acidentally put the probes into the house outlet while testing amperage. It got fried.

Lowtechmagazine wrote an excellent article about decadent mist showers that use many small nozzles spraying very fine mist particle sizes water on you as a much more efficient way to use water for showers. I would love to install and try out something like that one day.

Assuming its empty, i would take the grog oggah boogah solution of smash the blue plastic bowl down the edge of your countertop. Something will give sometime.

Otherwise, did you try twisting the bowl one direction and the plate the other? Torque is typically a more effective force than pulling for friction.

the owner of the picture themselves possibly put on the tie on their cat used to thst kind of thing and lied about it for an internet caption meme. The facial expression of cat looks blurry but relaxed tbh its obviously well fed and groomed.

And also a tad bit of folly from making said creation a mr-potato-head ass motherfucker stitched together from corpse parts and a half rotten brain. Professionals have standards, could have sourced some fresher parts for his whack ass meat baby.

Exactly its a great tool but gotta use it responsibly in a way that your information isn't being collected

If you are asking questions try out deephermes finetune of llama 3.1 8b and turn on CoT reasoning with the special system prompt.

It really helps the smaller models come up with nicer answers but takes them a little more time to bake an answer with the thinking part. Its unreal how good models have come in a year thanks to leveraging reasoning in context space.