filister

joined 2 years ago
[–] [email protected] 42 points 2 days ago (1 children)

That's a very toxic attitude.

Inference is in principle the process of generation of the AI response. So when you run locally and LLM you are using your GPU only for inference.

[–] [email protected] 13 points 2 days ago

The whole startup industry rely on investors to cover for their costs for years, while they work on a loss, in order to obtain a bigger market share. Look at Netflix, Facebook, WhatsApp, etc.

So buying an account you are increasing their market share.

But feel free to use Mistral, Deepseek, etc. that would be better

[–] [email protected] 1 points 2 days ago

I mean you can exchange the network card with at least 2.5Gbps of your Intel computer, install Linux and create a share and use the Ryzen mini PC for managing and transcoding media files, but it will complicate your setup and won't be very energy efficient.

[–] [email protected] 2 points 2 days ago (2 children)

No, by CPU is an odd choice, I meant for the i7-6700K. The Ryzen CPU is quite recent and very powerful, and energy efficient. Again, for a multimedia system you need a big case like the Define one and lots of SATA ports and bays. The Intel one checks the boxes, but you need a better processor or alternatively an external GPU, like the Intel Battlemage to have hardware encoding acceleration.

The mini PC is nice but not suitable for multimedia machine, as it lacks the SATA ports and bays. You can use it as a router, like OPNSense.

[–] [email protected] 2 points 2 days ago

With friends like the US who needs enemies?

[–] [email protected] 2 points 2 days ago (4 children)

Why are you buying a rack when neither of your machines are rack mountable?

What is your use case, what are you using the big PC and the small one? Why Unraid and not TrueNAS Scale for example.

If you are planning on using the big machine for multimedia, be aware that the i7-6700K doesn't support hardware encoding/decoding of HEVC and the CPU is not powerful enough for live 4K software transcoding.

This CPU is quite an odd choice though, it is 10 years old and in my opinion extremely outdated. Get at least 8-9 gen, that at least have this running.

[–] [email protected] 1 points 2 days ago (1 children)

Please show me an LLM model that is really open source. My understanding is that most of the open models are open weights. For the record Mistral is also releasing Open weights models.

[–] [email protected] 3 points 2 days ago

Let the witch hunt begin.

[–] [email protected] 58 points 2 days ago (10 children)

What is amazing in this case is that they achieved spending a fraction of the inference cost that OpenAI is paying.

Plus they are a lot cheaper too. But I am pretty sure that the American government will ban them in no time, citing national security concerns, etc.

Nevertheless, I think we need more open source models.

Not to mention that NVIDIA also needs to be brought to earth.

[–] [email protected] 4 points 6 days ago

That would be mine, but I am afraid that all stock will be snatched even at inflated prices

[–] [email protected] 2 points 1 week ago

Great choice mate. Enjoy it.

 

I am building a Proxmox server running on an SFF PC. Right now I have:

  • 1 x 250 GB Kingston A400 Sata SSD
  • 1 x 512 Gb Samsung NVMe 970 Evo Plus
  • 1 x 512 Gb Kingston NVMe KC3000
  • 1 x 12 Tb Seagate Ironwolf Re-certified disk

I plan to install Proxmox on the 250Gb Kingston disk using ext4 and use it only for Proxmox and nothing else.

I am thinking of configuring ZFS mirrored raid on the two NVMe disks. Here one disk is on my mobo, and the other is connected to the PCIe slot with an adapter, as I have only one M2 slot on the mobo. I plan to use this zpool for VMs and containers.

Finally, the re-certified 12 Tb disk is currently going through a long smarctl test to confirm that it is usable and it is supposed to be used primarily for storing media and non-critical data and VM snapshots, which I don't care much about it. I will in parallel most likely adopt the critical data to a cloud location as an additional way to protect my most important data.

My question is should I be really concerned about the lack of DRAM in the Kingston A400 SSD and its relatively low TBW endurance (85 TB) in case I would run it only to boot Proxmox from it and I think the wear out of the drive would be negligible.

  • I have the option to exchange the Proxmox boot drive with a proper SSD, like a Samsung 870 Evo (SATA SSD, using MLC NAND and having DRAM cache). I would of course need to pay around 60% more but I am just thinking that this might be an overkill.
  • Do you think that using ZFS pool for the two NVMe drives will wear them out very quickly? I will have 3-4 VMs and a bunch of containers.
  • Is the use of a slow Proxmox boot drive (SATA SSD) going to slow down the VMs and containers as they will run on much quicker NVMe SSDs, or it won't matter?
  • Shall I format the Seagate HDD in xfs to speed up the transfer of large files or shall I stick to ext4?
  • What other tests shall I run to confirm that the HDD is indeed fine and I can use it?
 
view more: next ›