What API are you using? Where is the data stored? Would be easier to answer with more data on how this is setup.
terebat
joined 2 years ago
The tricky part is defining an algo for hot, it’s a bit different than reddit since some instances may have many people, and local may have fewer, so there should be a balance such that posts from larger instances don’t overwhelm local instances perhaps.
Indirect prompt injections will make this worse. Plugins lead to scraping insecure websites (i.e., search for docs for a particular topic). This can result in malicious context being embedded and suggested during a PR or code output.
That along with the above, faking commonly recommended inputs, it becomes very difficult to just trust and use LLM output. One argument is that experienced devs can catch this, but security is often about the weakest link, one junior dev's mistake with this could lead to a hole.
There are guard rails to put in place for some of these things (i.e., audit new libraries, only scrape from 'reliable' websites), but I suspect most enterprises/startups implementing this stuff don't have such guard rails in place.
Related
+1.
This thing needs to be profiled and optimized. It should not be running into the ground with this low activity. Worse yet is federation reduces the speed by a ton too, but without it, instances have low activity.
I can help out here and try to do it in spare time, but no commitments/promises as I'm currently oncall at $dayjob.
I think there are people working on adding support for pgbouncer and splitting out pg from the core server to avoid having a 1 box only setup.
Doesn’t support HA or horizontal scaling yet from what I read. Unsure if kbin does. Probably would have to add support for horizontal scaling to have that auto scaling do anything.
Oh absolutely! Looking to get it running locally so I can start building something out and trialing something. Trial and error iteration would work well here I think