shaserlark

I’m doing a lot of coding and what I would ideally like to have is a long context model (128k tokens) that I can use to throw in my whole codebase.

I’ve been experimenting e.g. with Claude and what usually works well is to attach e.g. the whole architecture of a CRUD app along with the most recent docs of the framework I’m using and it’s okay for menial tasks. But I am very uncomfortable sending any kind of data to these providers.

Unfortunately I don’t have a lot of space so I can’t build a proper desktop. My options are either renting out a VPS or going for something small like a MacStudio. I know speeds aren’t great, but I was wondering if using e.g. RAG for documentation could help me get decent speeds.

I’ve read that especially on larger contexts Macs become very slow. I’m not very convinced but I could get a new one probably at 50% off as a business expense, so the Apple tax isn’t as much an issue as the concern about speed.

Any ideas? Are there other mini pcs available that could have better architecture? Tried researching but couldn’t find a lot

Edit: I found some stats on GitHub on different models: https://github.com/ggerganov/llama.cpp/issues/10444

Based on that I also conclude that you’re gonna wait forever if you work with a large codebase.

Basically I’m looking for the picture in here: https://redlib.kylrth.com//r/Unexpected/comments/2aosv2/hey_guys_check_out_my_new_watch/

It’s just a guy showing his new watch but the watch was really beautiful, so if anyone has a screenshot of this still somewhere that’d be amazing.

I‘m heavily involved in the self-hosting community and I already run a bunch of services at home but I was wondering if anyone has experience with self-hosting jail? 

Apologies for the wall of text incoming but there’s a lot to consider!

Pros:

1. It’s a lot of fun to self-host and you learn a lot
2. I see an opportunity here because there’s always crime happening which means business generates itself. 
3. I could probably run some targeted ads on social media platforms to promote crime and attract more potential customers
4. The customers can be incentivized to work which is rewarding for them since they can give back to the community.
   • They could produce merch helping me cover the cost of self-hosting, win-win! 
   • I looked into the economics and actually they don’t really ask for money, they are very generous and just want to give back! 

There are also some cons though:

1. The starting costs are pretty high, I would have to expand my house significantly 
2. NIMBYs, I can imagine they would cause trouble if they learn that I self-host a jail even though it’s a lot of fun and potentially generates jobs for the community. There can be many sticklers in the better neighborhoods.
3. Death penalty. Some states have it and I find it very inefficient to kill your clients. They could do work instead.
   • I would have to be careful to avoid such places
4. DEI. I looked into the system and it’s not very diverse. We would have to educate the recruiters from law enforcement to do better.
   • They seem very biased in their sourcing process. I would like this to be a place for everyone and I don’t think they are sensitive at all regrading this topic.
   • Plus they seem to be randomly shooting potential clients which is super unprofessional and inefficient?!
5. Vibes. I watched some movies and there’s a lot of issues with jail culture. I would have to invest a lot into educating the clients to battle e.g. homophobia which seems common or also fascism.
   • Apparently there’s some labor union called “Aryan Brotherhood“? And labor unions are mostly separated by race? This is obviously not okay.
   • Maybe it’s not good to allow unions in the first place as they struggle to foster an inclusive environment and there seems to be a lot of in-fighting

Any ideas? Anyone done this before? As a European I’m still new to the US system but was fascinated that it’s possible to run private jails, it’s much more inclusive than the socialist place where I’m from. There, the government has a monopoly on incarcerating people, totally disregarding the fact that it’s a lot of fun to self-host and incentivize them to work at almost no cost! It’s hard to try out new stuff in a place without freedom :(

I’ve started building a small decentralized, non commercial app with a Rust backend + Node.js frontend running on k8s. I would have my own dedicated server for this. Just mentioning the setup because it might grow and for git there seem to be only GitHub and GitLab around and I prefer GitLab.

I care a lot about security and was wondering if it makes sense to self-host GitLab. I‘m not afraid of doing it, but after setup it shouldn’t take more than 1-2 hours per week for me to maintain it in the long run and I’m wondering if that’s realistic.

Would love to hear about the experience of people who did what I’m planning to do.

EDIT: Thanks for all the answers, trying my best to reply. I want CI/CD, container registry and secrets management that's what I was hoping to get out of GitLab.

As far as I’m concerned, we’re done

The worst part? It seems to be true