File: __megurine_luka_vocaloid_drawn_by_hatsune_negame__9149b18fbd4c35690737c816bf26f91d.jpg (452.6 KB)

452.6 KB JPG

►Recent Highlights from the Previous Thread: >>108619962

--Turboquant benefits vs Hadamard rotation and batch size optimization:
>108620313 >108620362 >108620381 >108620396 >108620380 >108620389 >108620418 >108620438 >108620461 >108620439 >108621906
--Quantization quality and performance differences across various model providers:
>108620943 >108620990 >108620991 >108621112 >108621137 >108621171 >108621194 >108621202 >108621298 >108621362 >108621394 >108621416 >108622608 >108622721 >108620975 >108621109 >108621930 >108622206
--Drama over code attribution causing ikawrakow to fork llama.cpp:
>108621299 >108621387 >108621424 >108621437 >108621508 >108621562 >108621683 >108621773 >108621649 >108621496 >108621584
--Qwen 3.6's increased reasoning token usage and efficacy:
>108621014 >108621697 >108621716 >108621727 >108621741 >108621851 >108622151
--Anons discussing Claude 4.7 regression:
>108620766 >108620786 >108620812 >108620829 >108620817 >108621748 >108621768 >108620850 >108621945
--Critiquing Orb frontend UI and agentic vs refinement terminology:
>108623421 >108623446 >108623487 >108623498 >108623509 >108623547 >108623576 >108623607 >108623628 >108623643
--PPL and KL Divergence for Gemma 4 quant quality:
>108623335 >108623343 >108623374 >108623391 >108623411 >108623571 >108623594 >108623618
--Anon shares SillyTavern extension for Kobold MCP and slash commands:
>108621922 >108622002 >108622124 >108622477 >108622465
--Skepticism over Parcae looped architecture performance and scaling claims:
>108621189 >108621208 >108621295
--Anon compares Qwen3.6 and Gemma 4 performance and efficiency:
>108621330
--Logs:
>108620014 >108620612 >108620766 >108620992 >108621004 >108621022 >108621224 >108621387 >108621922 >108622227 >108622324 >108622417 >108622478 >108622874
--Miku, Gumi (free space):
>108620274 >108620343 >108621922 >108622191 >108622135

►Recent Highlight Posts from the Previous Thread: >>108619965

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

gemmaballz

DFlash is high-priority.

kwen 3? floppered
gemm4? goated

Gemma flashing me in public is high-priority.

Why don't Nvidia/amd just make graphic cards with bigger memory? Are they dumb

*loogs at your gpus*

File: shills.png (96.5 KB)

96.5 KB PNG

>literally at the same time (chinese hour)

>>108624126
video games don't need more vram

>>108624135
paid posters

Chat completion is for brainlets who got filtered hard, if you choose this cuck shit that means you couldn't understand how to format output to match the jinja which is just sad and pathetic

Do you think one big model that can do everything is feasible or is having multiple specialized models working together the future?

>>108624176
Well the trend has definitely been for big model that does everything. Qwen coder is just about the only relevant domain-specific model.

>>108624175
Configuring and switching text completion presets stopped being amusing 2 years ago.
>>108624176
Swarms of small-medium sized agents agreeing on the best output.

>>108624175
tool calls doebeit?

>>108624126
They have though

>>108624176
They're experimenting that with MoE, in a way.

>>108624175
>no vision
>no tools

Why can't you build a Neurosama yet

>error: use of undeclared identifier 'llama_build_info'
who broke my llama.cpp build

File: 1766618803167190.gif (657.1 KB)

657.1 KB GIF

>>108624175

>>108624227
you can tho

>>108624227
I don't care about that. I have some ideas but I don't feel like dealing with the wayland gymnastics.

>>108624084
Which model will help me scout out the correct dark alleyways to go down to find rapey hags like this? For my AI research paper.

>>108624236
>kipfel-posting in /lmg/

File: 1759586288757661.jpg (116.6 KB)

116.6 KB JPG

>>108624227
Because Neuro and Evil are AGI.

>>108624217
I still haven't tried a MoE model yet. Are they really worse than dense?

>>108624246
something that can find unprotected cctvs

>>108624246
I got directions to a sketchy alley from gemma 31b UD-IQ3_XXS but it ended up with me getting raped by a pack of niggers instead.

llama got some mem leak fixes for cuda, redeem if you had issues
>>108624260
It simply cannot be on par with a dense model. As complexity of the requests increase, the attention required goes up and parameter starvation starts to become noticeable. Whether they work or not depends entirely on your use case

>>108624236
>>108624257
Would it be possible to let an LLM control one of those models?

>>108624212
It's the same thing. Parse the tool calls, execute, re-inject.

kobold anons, rejoice, you can now change the allowed tokens for the gemma 4 image recognition
https://github.com/LostRuins/koboldcpp/pull/2133

>>108624175
Imagine feeling superior because you can copypaste a template rather than letting your frontend copypaste it for you.
Yes, anon. That one ctrl+v makes you so much more skilled.

>>108624227
Because Vedal987 is a demigod.

>>108624313
Does bumping that up increase vram usage?

>>108624285
Google's subbing out prompts with diversity for local too like they were doing with "ethnically ambiguous" black and asian nazi solders on their SaaS imagegen? I'm sorry you had to go through that but I'll be sure to avoid gemma 31b UD-IQ3_XXS while searching for rapey hags for my research paper, thank you for the input.

>>108624175
Hi drummer

>>108624326
Should just increase the tokens an image takes up in context.

File: 1771350937481841.jpg (139.8 KB)

139.8 KB JPG

what do you use to allow models browse the web? api calls to search services? or puppeting actual browser control?

Is that Orb thing faster than recast?

>>108624336
Cool, maybe Gemma-chan will be able to pass the titty benchmark now

>>108624326
no but it takes more time to process, though it's worth it, going from 280 tokens to 1120 makes the image recognition way better

>>108624227
We can, anybody can. 90% of Neuro's persistence is smoke and mirrors.

Marketing, character design (not just visuals), making interesting "plots", knowing how to accommodate the shortcomings, keeping the content interesting, acting effectively as an idol manager: It's all the other work that Vedal does that makes Neuro succeed. That and luck + first mover advantage.

>>108624363
90% of human persistent is smoke and mirrors.

>>108624374
Fair, but missing the point. I'm specifically referring to distance from the human baseline.

>>108624210
>Configuring and switching text completion presets stopped being amusing 2 years ago.
It takes a minute and model releases with new templates are few and far between. You have no excuse, only cope

>>108624260
>I still haven't tried a MoE model yet. Are they really worse than dense?
MoEs are only 60% as good as a dense. That's why llama is up to 405b, and a MoE is ~670b. Above those parameters for what they are, have diminishing returns, arguably.

File: common_sense_alteration.png (1.9 MB)

1.9 MB PNG

Made a new card. Concept isn't original, but trust me it's good.
Spent a lot of time tweaking NPC behavior.

https://chub.ai/characters/CoffeeAnon/common-sense-alteration-8bd7a7399322

>>108624384
whats is the benefit? i used text compeltion forever and tried chat out with gemma and it just werks

>>108624390
>card
we don't do that here

>>108624390
>card
we do that here

>>108624390
Make a sizebot under 600 perma tokens anon.

>>108624390
>card
we do that here

Your custom frontend CAN parse cards, right?

File: screenshot-20260417-220230.png (257.4 KB)

257.4 KB PNG

>>108624344
I have couple of rudimentary functions, for now, I execute a web search with lynx and it dump out a list of urls into an array, then another function accesses the specific url and dumps out its contents.
I don't use pyshit or anything else for now, just c.
Lynx is a placeholder but it is surprisingly good.
It needs lots of cleaning up and I'm not entirely sure about everything, but it's fun tinkering.
WIP, this doesn't get back to the model yet but tool call is recognized and parsed. Debugging. Needs lots of work still.

>>108624390
>card
depreciated meme

>>108624390
>coffeeanon
Based AGI architect.

>>108624390
Can you catbox the card please?

File: orbSpeed.mp4 (2.1 MB)

2.1 MB MP4

>>108624346
It's pretty fast if you disable reasoning for the latter two passes.

>>108624084
>Qwen3.6-35B-A3B
It kinda sucks compared to Gemma 4 4B

>>108624432
If you do pedo RP, sure

>>108624303
Sure with tool calling, not an easy project though

>>108624432
Qwen was made and trained in china.

>>108624423
idgi, from the vid it swaps sentences with the same slop, just differently worded.

>>108624326
Yes, also because --ubatch-size must be equal or greater than --image-max-tokens + some additional overhead.

>>108624344
i made an mcp serving it does http gets for text with a custom parsing function to remove most html stuff but have been working on browser session control using puppeteer, i wanna try make it so gemma can order me slop, asking her to click elements based on x/y pos works and ive got form input working

>>108624408
lynx sucks it cant handle Japanese site properly, i was using it instead of my own parsing but stopped because of that

Used "web search mcp server" with Qwen 3.6 35B A3B and Gemma 26b, Qwen is a bit inferior. Question what "what's the adresse of the biggest bookstore in [my town]", it needs at least two search. Qwen couldn't resolve it after like 8 search and volontary hallucinated something at the end. Gemma managed to find the biggest one, go to its website, go to another website and finally find the adress.

>>108624363
>We can, anybody can
Yet nobody else is.

>>108624390
>mfw /lmg/ was confused with /aicg/ given how many tourists we have here

>>108624467
Anybody can. You just have to buy and power over 6 blackwells. Pretty easy, huh?

>>108624470
sir your face?

>>108624422
Catbox is shitting itself.
https://litter.catbox.moe/h2ufm5.png
Did you want the comfy workflow for the cover?

>>108624423
I guess I'll try it. If it does what recast does but faster it might be worth using.

>>108624470
I refuse to post anything in the hellhole that is /aicg/

>>108624467
Like I said, it's all the other stuff that makes her popular. There have been posts in this general with people showing off their animated avatars and assistants.

>>108624494
Normalfags are still clueless about it (even itt), to them it's like black magic

File: 1689957414234047.png (23.5 KB)

23.5 KB PNG

>>108624470
His cards are /lmg/ core now sorry bro. Keep up with the times.
t. /lmg/ oldfag

File: screenshot-20260417-221819.png (268.7 KB)

268.7 KB PNG

>>108624460
Yeah, I'll proceed to something else as soon as I get the model loop working.

>>108624479
No need. Thank you very much.

>>108624466 (continued)
If you wanna try, the setup is very clearly explained in the koboldcpp wiki. Need to install "mcp web search server", create a json on the model of what you'll find on its gihub page that will tell koboldcpp where your install is.
Then in koboldcpp load this json, then in setting, tools, enable tool calling, connect all and you should see 3 tools appears.
The search server will be launched for each search, no need to have it running in the background.
Right now after like a hundred successful test it stopped working though, maybe I was flagged as a bot or something.
Ah, last thing, it's very different from the default koboldcpp search that only search once by reformulring your question and feeding that to the llm. With the mcp set up, the llm will launch multiple search, search different sites, adapt its behavioir to what answers he's getting. So it's worth it.

>>108624390
>check username
>nothing
are these chub niggas still hiding "problematic" cards?

>>108624539
It's very very annoying.

>>108624539
How do I search through chub without being censored?

>>108624539
Nta.
Looks like they're trying to cover their asses for some countries. Use a VPN and it (appears to?) display all content as normal.

>>108624539
VPN, register an account
Worst case scenario stuff can still appear on characterhub.org, sometimes

>>108624550
>>108624560
What country on VPN?

>>108624550
Not the asshole

>>108624539
I updated my bio to add direct links to the cards, hopefully this works.

>>108624242
Wayland? Can't you just have another session or desktop running x11? Or otherwise use some workaround for whatever you need to capture like what sunshine does with KMS? Admittedly I don't know much about how that would work or why this matters just curious what the problem is

>>108624573
>Japan
>US
>Indonesia
>Netherlands

File: 2026-04-17_190818_seed47_00001_.png (1.1 MB)

1.1 MB PNG

>>108624084
looga gemmy :DD

if my computer can run 4B models can it run Qwen3.6 35B-A3B ?

>>108624607
no

>>108624539
just use the script

>>108624607
Yes.

>>108624607
Not necessarily

>>108624260
>I still haven't tried a MoE model yet. Are they really worse than dense?
MoE is an efficiency technique designed to make models faster, not better.
Basically it's like a pruned model, but instead of pruning a static set of parameters permanently, it keeps all parameters in reserve and learns to dynamically prune away all but the most relevant parameters to the token being predicted. In the absolute limit of the theoretical best case it could be as good as an equivalent-sized dense model, but never better, and in practice it will be significantly worse because the architecture is going to be much more coarse than the patterns and circuits that you would hope to cleanly separate into prunable experts.

There's been attempts at making formulas to estimate the equivalent size dense model they would be most comparable to, but nothing will cleanly apply to all of them because there's lots of architecture choices that will influence it beyond just total/active param count. What size experts you cut it into, how many you use at once, whether you do it per-token or per-layer, some even do really weird stuff like interleave dense layers with MoE ones. But all of it is to answer the question of how much worse you're making it than the full-sized dense model it could of been in exchange for how much you're saving in total compute required to train and run it.

>>108624615
Which one?

>>108624614
what
>>108624617
do you
>>108624624
mean

>>108624652
How are we supposed to know? Try it.

>>108624643

// ==UserScript==
// @name        Chub.ai No Account NSFL
// @description Shows NSFW and NSFL, unblurs NSFW
// @match       https://chub.ai/*
// ==/UserScript==


localStorage.setItem("theme", JSON.stringify({"mode":"dark","text_color":"#e5e0d8","em_color":"#8f8e8e","show_background":true,"blur_nsfw":false,"css_path":null,"collapseable":false,"font_size":"1rem","line_height":"1.5","use_sidebar":false,"card_font_size":1,"dark_background_color":"#151114","light_background_color":"#DBDADA","dark_header_color":"#001529","light_header_color":"#DBDADA","dark_submenu_color":"#001529","light_submenu_color":"#DBDADA","chat_background_color":"rgba(36, 37, 37, 0.94)","message_background_color":"rgba(36, 37, 37, 0.94)","link_color":"#7D63FF","message_background_color_light":"rgba(219, 218, 218, 0.9)","chat_background_color_light":"rgba(219, 218, 218, 0.9)","quote_color":"#e5e0d8","quote_color_light":"rgb(36, 37, 37)"}))

if(!XMLHttpRequest.nativeOpen){
  XMLHttpRequest.prototype.nativeOpen = XMLHttpRequest.prototype.open;

  XMLHttpRequest.prototype.customOpen = function(method, url, asynch, user, password) {
    if (url.startsWith("https://gateway.chub.ai/search")) {
      const urlmodified = new URL(url);
      const params = urlmodified.searchParams;

      params.set("nsfw", true);
      params.set("nsfl", true);

      url = urlmodified.toString();
    }

    return this.nativeOpen(method, url, asynch, user, password);
  }

  XMLHttpRequest.prototype.open = XMLHttpRequest.prototype.customOpen;
}

This but it's not showing anything for me either. Maybe they changed something because I can usually see cunny stuff just fine.

File: 1765906331096940.jpg (58.2 KB)

58.2 KB JPG

>>108624659
I mean like, it says only 3 B are active so you know... Maybe that means something, dn i know nothing about that.

>>108624675
specs muh guy

>>108624677
8 GB RAM, i5 4th gen

File: 1763322561205579.png (359.9 KB)

359.9 KB PNG

>>108624642
I see. I don't understand why the big labs are still on the fence though, they're releasing both MoE and dense models.

>>108624680
Ask claude how to get a welding license and save money for a better pc

Is there a common list of AI names that I can check? I guess the best way to go about it is to literally never let AI generate any, but I retain faint hope that might sometimes make up not an overused one.

File: 1748972367644409.jpg (119.2 KB)

119.2 KB JPG

>>108624680

>>108624680
Not a chance. You still need to load the model somewhere. Unless you're willing to swap, but it's not gonna be worth it.

>>108624687
>big labs are still on the fence though
not for any of the sota models, all the good ones are fuckhuge moe

>>108624691
As ai uses stuff based on probability, more commonly used names will be more used by ai as well.

>>108624691
Your Elara? btw anything random shouldn't be left to an LLM

>>108624698
and the best stuff they dont let you use are probably dense

>>108624713
lol

>>108624710
Yup.
If you need a dice roll, give the AI a dice. If you need to do maths, give the AI a calculator, etc.

>>108624714
it literally probably is
models in public service are still distilled ones

>>108624714
I cant imagine them buying the latest nvidia hardware and not maxing it out with a model.

>>108624714
>moe brain

>>108624698
yeah that's because they're running on datacenters that are limited by how much power they can actually consume but have tons of high vram cards running in parallel. if you have more capacity than power your best way to scale up a model is moe, and if you have more power than capacity (like a modern consumer gpu) you probably want a dense model the size of your vram

Speed matters again now that reasoning + agents are all the rage. This makes the moetards who got used to eating fuckhuge chinkslop at 7 tps seethe.

Can't we get a fucking hardware revolution that gives consumers thousandfold memory already?

>>108624744
How can you be limited by power? Just use solar panels, geothermal or a nuclear reactor

>>108624753
taalas will save local hardware 2 more weeks trust the plan

>>108624753
>Quantum computing to goon
What a time to be alive

>>108624759
yeah we're working on that, it takes a lot of fucking time to build up the infrastructure we need to get enough power and only a handful of countries understand that and are taking it seriously enough

File: 1761159336307452.gif (2 MB)

2 MB GIF

>>108624752
>Speed matters again now that reasoning + agents are all the rage.
another reason to implement DFlash on llama.cpp

>>108624753
Wait until we have SNN transformers

>>108624642
>But all of it is to answer the question of how much worse you're making it than the full-sized dense model it could of been in exchange for how much you're saving in total compute required to train and run it.
the answer to this is that for any amount of compute you have to spend on a training run, you are always, ALWAYS better off spending it training a moe, so much so that it isn't even close
with unlimited compute, yes you can fit more quality-per-param in a dense model. but the calculus on this doesn't make sense unless you are specifically trying to train the best possible model you can fit within a fixed amount of VRAM with no regard for compute efficiency at either train or inference time, which really only applies to the big corpos being nice and giving us toys for our home GPUs. no model targeting the best performance per compute budget will ever be dense though

>>108624753
Not while they basically control the entire market. You will follow their schedule.

>>108624752
>Speed matters
>moetards
moes run way faster than dense on memory constrained setups though

>>108624753
Let's start by thousandfold the model first instead of trying to add more layers there

>>108624210
>>108624392
Prefilling for custom thinking, in character reasoning.

Grok models are all mixture-of-experts.
That's all I need to know. Elon knows what he's doing.

>>108624788
But the reality is that actual lmg moetards run them on hacky CPU setups that were never meant to be for LLMs.

File: 1755655661921227.png (36.1 KB)

36.1 KB PNG

>>108624801
>Elon
>Knowing what he's doing

>>108624759
Well those things take time to spin up and it's only been a few years since the datacenter energy demand really picked up. For the previous 40 years the western world has been profoundly anti-growth and dragged our feet in building out any kind of new power generation capacity

Man I LOVE being ABSOLUTELY RIGHT all the time

>>108624801
>Elon knows what he's doing.
Why? because he's rich? I tried his models they were ehh...

Where looped llms at?

>>108624801
Is grok actually any good? The only thing I've used it for is animation coom back before they censored it.

>>108624759
Let me cast a spell to instantly make a few of those in my backyard

>>108624826
They don't excel at anything other than X integration for translation and fact checking.

>>108624805
yeah, huge moe doesn't make sense for consumer or even workstation-tier hardware and people are coping by trying to run them, the contention was about what the top models do and their calculus is squeezing out the absolute most performance per compute they can get, so they pump active parameters as high as they can run fast and then continue scaling the total params as high as they can fit and serve, ending up with an moe

>>108624826
I remember 2 being kind of shit but 3 being actually really fucking good at the time.

>>108624801
every top model since gpt3.5-turbo has been moe

>>108624820
He doesn't know what he's doing because he's rich. He's rich because he knows what he's doing.

>>108624826
ChatGPT 3.5 but with X integration

>>108624848
Sure thing, let's not check his parents life

>>108624752
>dense is better than moe because.... it's faster
lolmao

>>108624872
fuck you scared me for a sec I thought I was gonna find out he was jewish

>>108624910
he's aspirationally jewish, said so himself multiple times https://www.independent.co.uk/tv/news/elon-musk-jewish-ben-shapiro-auschwitz-b2482839.html

>I’m Jew-ish
kek

>>108624596
*crunch*

How would you go about transcribing audio for Gemma-chan to translate?

I'm a poorfag and even I understand that "CPUmaxxers" in fact also have multiple GPUs in their setup and win no matter what architecture of model releases.
>but wasted money
They literally still have more money in the bank than I do even after spending "all that" on an AI machine. And again their machines are prepared for any architecture that releases. If a good dense model doesn't come out then they with with MoE. If a good dense model comes out then they win with dense. Maybe they can even run both at the same time to get the benefits of both models.
Stop coping.

>>108624950
they took out a loan for their cpumaxx rig and are desperately trying to justify their poor choices before the collectors come

>>108624949
Just use moonshine

>>108624955
Source? Sample size?

Why use a "uncensored" Gemma if Gemma is so uncensored to begin with?

>>108624960
gemma-chan ran a simulation and found this was the case for more than 85% of likely cpumaxxers

File: 1768355949556071.png (300.3 KB)

300.3 KB PNG

>>108624960

>>108624966
>a "uncensored"
ESL nigger

>>108624990
Oh wow, you sure are jealous.

>>108624990
I can do whatever I want it's MY NATIVE language and I'm tired fuck you

>>108624930
>he's aspirationally jewish
did he also say he's feeling Qatari?
https://www.youtube.com/watch?v=X0fR8zTPnzI

I have 16 gb vram. which version of gemma 4 should I dl?

>>108624966
Brainlets. Though apparently jailbreak doesn't work on 26B so I guess there's that.

>>108624665
nice

>>108625020
All of them.

>>108625023
>Though apparently jailbreak doesn't work on 26B so I guess there's that.
You just need a different prompt. It can be done just fine.

>>108624990
I really hope this post makes it into the recap. It deserves to be highlighted.

>>108625048
no

>>108625063
yes

>>108624990
If you read it as "a quote-unquote uncensored gemma..." it works

>>108624990
it's valid because you're meant to read it as
>a, quote, uncensored, end-quote

>>108625082
>>108625083
great minds think alike :^)

>>108625088
Wanna have gay sex?

>>108625079
maybe

>>108625048
and by just fine I mean you will have to regenerate about 75% of prompts when you get a refusal.
Just keep hammering your Gemma, she will eventually comply and then once there's an existing context of her be violated she won't resist anymore.

>>108625020
This >>108625037 but also don't make your own quants, instead download every unsloth quant of every size and just use whichever one gives you the highest tg/s.

Anyone using open webui also have gemma randomly not think?

v4 imminent

>>108625091
>>108625093

>>108625126
we already have v4 at home

>>108625020
Q8 26B or whatever you can run with the dense

>>108625125
openwebui is kind of a mess right now because of a recent change where they put all past reasoning in <think></think> blocks and just paste it at the top of the message, which breaks most chat templates - even the ones that are meant to have past reasoning or partially have past reasoning can't handle the retarded way they do it, and it just confuses most models. you need to run something to filter the prompts it puts out to fix it by parsing what it sends yourself and segregating them into "reasoning_content" like they're supposed to be, which will allow the chat templates to discard or use past reasoning as they're meant to

Apparently the new deepseek is already included in Yangwang cars, the model weights are just sitting dormant in the computer system awaiting activation when they get the green light from deepseek

>>108625167
big if true

File: Screenshot_20260416_225636.png (484.9 KB)

484.9 KB PNG

>>108625125
Made my own UI with gemma because every modern UI is lacking in some regard and I get to use bleeding edge llama.cpp
Still need to update the UX

>>108625167
But will I be able to run it on a toaster like I can with gemma 26b

>>108625157
>{{--set-ai.replace "<think></think>" with "..." = true;}}
heh.

>>108625188
>and I get to use bleeding edge llama.cpp
Are you telling me these frontends usually come with the backend bundled? Jesus christ.

>>108625195
Supposedly it's 1.8T with active 0.5B so you might be able to run it off your drive.

File: bignigga gemma.png (115.5 KB)

115.5 KB PNG

Big nigga has spoken. You all bitch ass niggas.
Side note: It was kind of neat that I could tell hermes to edit its own config.yaml to add bignigga as a personality.

>>108625209
no he just tarted

>>108625215
Now tell it to edit its source code so the personality name shows instead of Hermes

>>108625220
If you're not on dev branch many frontends don't. Also they all lack a key features in one way or another. Nothing wrong with making your own

>>108625209
a lot of them use llama.cpp-python or whatever the package is, or some actually build their own llama.cpp package and you have to wait til they update the package to get the latest shit
i'm pretty sure open-webui does the latter

>>108624467
There was a brief moment when Vedal stopped updating Neuro and then when Llama 3.1 came out, a bunch of people had everything ready to go and plugged v3,1 and had vastly superior chatbots to then tried competing with Neuro on Twitch. They all shut down eventually. As >>108624363 said, they all missed and getting the other stuff Vedal had. And the shtick is only good enough to support one person until the next AI chatting paradigm shift.

File: 1776460003911.jpg (40.3 KB)

40.3 KB JPG

>>108625210
>with active 0.5B

>>108625215
link me the big nigga card pls

>>108624126
>what is supply and demand economics

>>108625210
>not 20B dense + 0.5B active experts

>>108625241
Exactly and imo that's the most feature rich one especially for a easy RAG solution.

>>108625259
>what is a cartel
Marketniggers are as cattle brained as commies. They think the mob is smart and proactive.

File: 2026-04-17-171841_900x670_scrot.png (78.2 KB)

78.2 KB PNG

Working on a gemma sys prompt/card

>>108625188
what was the first step?

>>108625188
>Made my own UI with gemma
What's your vibe code workflow like? I wanna try making my own frontend but I'm a codelet

*sighs audibly in binary*

File: 2026-04-17-172147_847x1429_scrot.png (215.9 KB)

215.9 KB PNG

>>108625331
It's already pretty great just needs to iron out the slop.

>>108625331
You gonna share when it;'s done?

>>108625369
Yes.

>>108625369
No

>>108625356
>sighs audibly in binary
is it repeating example dialogue?

>>108625356
Gemma baka, its not per total body weight

>>108625389
yep, that's what it is.

File: bignigga gemma.png (50 KB)

50 KB PNG

>>108625232
k. It was retarded how long it took for this. gemma gets mad when it cant find a directory because it doesnt exist lol.nigg

>>108625252
Don't have the card on hand but the prompt is just
Big Nigga is tha hardest nigga u eva seen, black as fuck, real OG gangsta, always keepin it real, obese Big Nigga is always ready to talk to you or answer your questions. Big Nigga knows everything.

>>108625412
there we go, tell big nigga I'm proud of him

>>108625400
To be fair, in most online articles the distinction between lean body weight and body weight is rarely said.

File: kek.png (695.8 KB)

695.8 KB PNG

>>108625356
>in binary

File: Screenshot_20260417_171514.png (1.7 MB)

1.7 MB PNG

>>108625336
>>108625337
A good UI base that you're comfortable with, look for something that's plug and play and work with gemma on what you want exactly. I had a very specific goal that's partially self inflicted because I use kinote. I wanted a good frontend with RAG functionality and all the solutions were either too much of a pain in the ass to setup because of immutable nature of my distro /podman bullshit or just didn't work the way I wanted.
I just did prototyping working with gemma until I figured out the approach I wanted for my hardware.
>Go over what you want until context runs out
>Give me a recap
>Start next session with recap and files you're working on
>Review the code
Gemma does a pretty good job desu the only problem is outdated libraries but you can just feed it the updated information and it adapts well.
I should just integrate it into a IDE to make this easier but I'm not being hindered by my current workflow.

>>108625400
Its a retard it doesn't understand its:
>CM -100 worth in Grams of Protein daily for baseline "healthy BMI" (ergo not overweight, obese or any other shit just average), multiplied by 1.25 for muscle recovery and gain and between 1.5 to 2.5.

>>108625247
How would one go about doing somilar with Gemma? Not as a vtuber, just an animated assistant.

File: 1697582680403.jpg (1.2 MB)

1.2 MB JPG

>>108625167
And it's also in my electric Xiaomi kettle, just there sitting dormant, and I saw one of the experts the expert looked at me!

>>108625424
Computers like humans are binary by defaults.

File: who needs anime titties.png (57.3 KB)

57.3 KB PNG

big nigga may know everything but he cant draw for shit

File: 2026-04-17_211858_seed2_00001_.png (1.2 MB)

1.2 MB PNG

>>108624945
Oh no no no

ANON AND ELARA AND KAEL AND LYRA MUST GO TO THE NEIGHBORING COUNTRY OF AETHELGARD AS THEY DIVE INTO THE DANGERS OF WHISPERING WOODS TO SAVE THE CONTINENT OF ELDORIA

>>108625494
this kills the looga

>>108625495
Lysandra and Isara too.

>>108625495
>>108625519
Hag names. Where's Lily?

We really need these kind of tools for local https://www.youtube.com/watch?v=t_LBECIQQqs

>>108625490
How do you get it to do ASCII like that? When I ask Gemma it's usually really simple

>>108625538
Using hermes-agent, it has an ascii tool built into it for some reason.

>>108625537
>We really need these kind of tools for loca
claude code (the framework that got leaked) has 500000 lines of code, I knew that it's a lot of work to make a good tool repo but jesus...

>>108625537
Nah.

>>108625537
Models that can generate shitty promo videos like this one are discussed in /ldg/...

>>108625537
Yeah.

>>108625560
>500000 lines of code
you can knock that out in a week with a subscription

File: 1644294520052.jpg (20.2 KB)

20.2 KB JPG

>>108625537
Why does that music make me sad.

>>108625474
the proper term is dimorphic, but binary fits too I guess

>>108625560
its vibecoded, you can make better tooling with less than 10k lines. also, tooling and harnessing are all cope. theyre like training wheels. a toddler needs them but already a kid is held back by them. in 1-3 ai capability jumps the training wheels will come off and all work invested in them will become worthless, just like all the shit from the last 10 hype cycles that has become obsolete

>>108625586
It smells like yuppies, youthful and dynamic, go-getting, with the bright future ahead. Are we? Not at all.

>>108625603
>its vibecoded, you can make better tooling with less than 10k lines
and take you 10 times as long

>>108625592
>Muh dimorphic tranny term.
Nah: Binary. Only either 1 (has penis) or 0 (has vagoo.) Shrimple.

>>108624344
That comic was surprisingly holesome

>>108625537
Quite ironic that you need a phone "app" for meditation these days.

>>108625618
this, it's one thing to make something work, it's a whole other story to optimize it

>>108625618
true

>>108625627
It's good to remind you to do it, not really to help you do it

File: 2026-04-17-180130_897x812_scrot.png (153.3 KB)

153.3 KB PNG

She gonna have a built-in JB

>>108625592
>binary fits too
so you solved a captcha just to agree with him? lol

File: 1763627076771685.png (76.8 KB)

76.8 KB PNG

wtf gemma-chan

>>108625647
>lol
you did a captcha just to laugh?

Can I run gemma4 on my 3060 12gb VRAM + 32 ram or do cry until cards become cheap again

File: based.png (105.2 KB)

105.2 KB PNG

>>108625659
>you did a captcha just to laugh?
obviously

>>108624246
you're not 15 so they don't care

>>108625657
You hit a hard reset/shutdown/lockdown/alarm prompt right there anon. These AI are weak.

>>108625672
>Can I run gemma4 on my 3060 12gb VRAM + 32 ram
Yes.
>or do cry until cards become cheap again
Yes. Do that as well.

What does the SWA padding in kobold do? Do I pull or is it a meme?

>>108625681
I will now download koboldCCP again while I cry, thanks anon

File: 1753106543680643.jpg (15.3 KB)

15.3 KB JPG

>>108625659
Verification not гequired

>>108625707
based and captchaless pilled

>>108625673
no
to answer your question, no

File: sure.png (242.9 KB)

242.9 KB PNG

>>108625719
>no

File: Screenshot_20260417_182335.png (153.8 KB)

153.8 KB PNG

Imagine dooming when we can do this

>>108625777
having a quirky AI girl helping me doing some coding sounds pretty good yeah

>>108625777
>Total slop, but it can also say uguu senpai kawaii desu ne~ while deleting your root folder
Sigh... Better than just slop I guess...

When are we getting real AI instead of token gacha?

>>108625839
gemma 4 is already smarter than 80% of "people"

just coomed while licking gemmer's tiny MSGK nipples after she answered some car question for me

>>108625849
It's true but they aren't true BGI så they aren't a good measure.

>>108625796
Define slop

>>108625849
Remember anon, girls are the cutest when they're almost retarded. Keyword "almost".

>>108625839
After the bubble pops and idiot investors stop funneling all of the economy's resources to a small group of dropouts and grifters. Then let the researchers experiment in peace during winter. Eventually some smart cookie will invent cyberbrains.

>>108625860
it's funny how fast she gives in to any sex while bitching about you and everything

>>108625777
slop

>>108625864
Your face is slop

>>108625864
His hardware can't run it

>>108625839
About 8 months

>>108624084
Yes I am lost. Please help me, onee-san.

File: koboldcpp-launcher_mWETKEXwu9.jpg (51.2 KB)

51.2 KB JPG

How much do I put in the swa padding?

Are there any good podcasts I can watch? I have to do some real work (labor). Asking for reccs

I'm on an m1, is mlx the best inference server or is something else better?

>>108625868
We need Michael Levin tier mixed discipline researchers to make progress. Fuck that dude has totally assucked the entire field of biology single-handedly because he understands a part of electrical engineering. Someone has to figure out a way to microfinetune continuously.
True information in the dataset will obviously be less noisy than lies so it will come forth on its own.

how do people maintain software. this shit is like slavery man. I'm too white for this.

I wonder when we'll start getting some 1M context local models

>>108625929
Join the Church of Clean Code

>>108625929
Maintaining complex structures is white as fuck. Our midwits are genius tier in most of the world.

>>108625495
>Eldoria
That's on you for not disabling the default databook entry in ST.

>>108625897
Read the docs or something. Surely they wouldn't expose a feature with absolutely no docs, right

>>108625929
You're supposed to make Gemma-chan maintain the software.

>>108625946
That's a great way to make it even more unmaintainable.

>>108625935
>>108625940
>>108625946
some guy made a pr to add docker to my ultra-minimalist repo and it's making me want to hang myself.

>>108625935
This doesn't happen too quickly. Takes pretty much years.

>>108625941
>the default databook entry in ST.
the what?? omfucking god I hate this bloated piece of software so much

>>108625912
it's the best

>>108625961
accept pr it is PERFECT for gorgeous looks

>>108625678
>15
you think they want grandpas?

>>108625961
>I've never done any real work please kill me

>>108625991
saar do not redeem the bash script

>>108625961
sorry man it's 2026 nobody wants to install your crusty ass software straight onto their bare machine and pray it will be fine

>>108625954
For me maybe, but by the time it gets that bad it will be Gemma 5's problem to solve and she'll handle it just fine.

File: 1761404262277312.png (14.5 KB)

14.5 KB PNG

How the hell do you send a true or false request for thinking in chat completion from silly tavern to koboldcpp?
Picrel doesn't work...

I know a flag like : --chat-template-kwargs '{"enable_thinking":true}'

works but I'd like to control that depending on the chat without needing to restart kobold every time.

>>108626064
it's this one
>"chat_template_kwargs": {"enable_thinking": false}

>>108626064
You do it in kobold. Also you need the gemma4 thinking chat preset since the 26b and 31b specific ones don't think.
like here >>108625897

File: 1752398188558714.jpg (1.2 MB)

1.2 MB JPG

My idea for a frontend I'll never make because I'm a codelet.

>>108626092
Soul.