cyd

joined 2 years ago
[–] [email protected] 11 points 13 hours ago* (last edited 13 hours ago)

Aww come on. There's plenty to be mad at Zuckerberg about, but releasing Llama under a semi-permissive license was a massive gift to the world. It gave independent researchers access to a working LLM for the first time. For example, Deepseek got their start messing around with Llama derivatives back in the day (though, to be clear, their MIT-licensed V3 and R1 models are not Llama derivatives).

As for open training data, its a good ideal but I don't think it's a realistic possibility for any organization that wants to build a workable LLM. These things use trillions of documents in training, and no matter how hard you try to clean the data, there's definitely going to be something lawyers can find to sue you over. No organization is going to open themselves up to the liability. And if you gimp your data set, you get a dumb AI that nobody wants to use.

[–] [email protected] 2 points 13 hours ago* (last edited 13 hours ago)

The underlying research story is interesting, but the way it's written up actively makes it worse.

The researchers based s1 on Qwen2.5, an open-source model from Alibaba Cloud.

Watch me create a racing car for less than $50. Step 1: start with a Mercedes F1 racer...

[–] [email protected] 23 points 2 days ago (3 children)

It's definitely a trend. More and more top Chinese students are also opting to stay in China for university, rather than going to the US or Europe to study. It's in part due to a good thing, i.e. the improving quality of China's universities and top companies. But I think it's a troubling development for China overall. One of China's strengths over the past few decades has been their people's eagerness to engage with the outside world, and turning inward will not be beneficial for them in the long run.

[–] [email protected] 1 points 2 days ago

But Mistral could do all that, with a far lower chance of pissing away the money...

[–] [email protected] 1 points 3 days ago* (last edited 3 days ago) (2 children)

Why not just put the money into Mistral? Mistral seems to be pretty cash-strapped and they're just about the only EU entity doing anything interesting with LLMs, and they've released open models before. Commissioning a bunch of models from them would be a better use of money than spreading it among a bunch of randos.

[–] [email protected] 5 points 3 days ago

Chinese or not, it's MIT licensed. A world where any company can spend ~$10k to locally deploy a frontier reasoning model is very different from one where you can only get AI via API access to a handful of US tech giants.

[–] [email protected] 12 points 3 days ago (5 children)

Rare earths to begin with. There will be more demands.

[–] [email protected] 20 points 3 days ago* (last edited 3 days ago) (6 children)

Base models are general purpose language models, mainly useful for AI researchers and people who want to build on top of them.

Instruct or chat models are chatbots. They are made by fine-tuning base models.

The V3 models linked by OP are Deepseek's non-reasoning models, similar to Claude or ChatGPT4o. These are the "normal" chatbots that reply with whatever comes to their mind. Deepseek also has a reasoning model, R1. Such models take time to "think" before supplying their final answer; they tend to give better performance for stuff like math problems, at the cost of being slower to get the answer.

It should be mentioned that you probably won't be able to run these models yourself unless you have a data center style rig with 4-5 GPUs. The Deepseek V3 and R1 models are chonky beasts. There are smaller "distilled" forms of R1 that are possible to run locally, though.

[–] [email protected] 7 points 5 days ago (1 children)

Going after US tech is an obvious move. Digital services taxes, etc.

[–] [email protected] 4 points 5 days ago (8 children)

"Via Greenland" makes no sense. The trouble with Canada-Europe trade is that Canada unfortunately lacks a good port on its east coast (certainly nothing comparable to Vancouver in the west). For the foreseeable future, if the trade dispute with the US drags on, Canada's best bet is to expand its trade with Asia.

[–] [email protected] 6 points 5 days ago

Intriguingly, there's reason to believe the R1 distills are nowhere close to their peak performance. In the R1 paper they say that the models are released as proofs of concept of the power of distillation, and the performance can probably be improved by doing an additional reinforcement learning step (like what was done to turn V3 into R1). But they said they basically couldn't be bothered to do it and are leaving it for the community to try.

2025 is going to be very interesting in this space.

[–] [email protected] 34 points 5 days ago (2 children)

They have no armed forces. Panama always assumed that because of the importance of the canal, in case of external aggression the US will step in to defend them. LOL.

 

He claims Trump would act immediately upon winning the election, before taking office. Which sounds legally dubious, but not that that's ever stopped Trump....

 

Archive link: https://archive.is/vGKin

 

Always weird to me how France is so insistent on clinging to its colonial empire, two decades into the 21st century, despite the headaches that causes.

 

Guess which country is doing the alleged interference...

"Mr Chan, the managing director of several real estate investment firms, was invited to attend China’s annual Two Sessions parliamentary meetings in March 2023 as an “overseas Chinese representative”."

 

I'm somewhat surprised that Singapore chose to stick its neck out with a statement, since you-know-who won't like this...

view more: next ›