![]() |
#1
|
||||||
|
||||||
Running DeepSeek R1 locally
DeepSeek has its flagship V3 model equivalent to GPT4 and it's reasoning model R1 freely accessible:
Quote:
If you have a GPU recommend OLlama which works on Windows, Linux and Mac (can also rin Facebook/Meta's Llama models): Quote:
Quote:
For a frontend chat interface, I recommend Chatbot AI: Quote:
R1: Quote:
Quote:
|
The Following 4 Users Say Thank You to chants For This Useful Post: | ||
blue_devil (01-29-2025), DARKER (01-29-2025), Doit (01-30-2025), wx69wx2023 (01-31-2025) |
#2
|
||||
|
||||
Are there any reverse engineering (especially for decompilation), specific models?
|
#3
|
||||
|
||||
This makes DIGITS even more interesting to me, considering buying one. (But it will only be out in may). https://www.nvidia.com/en-eu/project-digits/
What I dont get though is how 1PFLOG in FP4 is a selling point when the regular GTX5090 has like 600 TFLOP in FP32? Am I missing something or is the 128GB integrated RAM the selling factor? |
#4
|
|||
|
|||
There is even more powerfull ML model then DeepSeek - Alibaba’s Qwen2.5-Max
|
#5
|
|||
|
|||
DeepSeek censorship:
Code:
https://www.theguardian.com/technology/2025/jan/28/we-tried-out-deepseek-it-works-well-until-we-asked-it-about-tiananmen-square-and-taiwan |
The Following 2 Users Say Thank You to DARKER For This Useful Post: | ||
blue_devil (01-30-2025), Gyrus (01-30-2025) |
#6
|
|||
|
|||
It would be nice to train an RE model. The good news now is that training is being shown to be feasible possibly on an academic grant level budget. Someone should train a proper open source RE model at some point.
1PFLOP FP4 is a marketing gimmick maybe, that amount of RAM is a big plus tho. The new DeepSeek models use FP8 and have shown it's reliable for training, a good breakthrough. Sounds good enough to run good size models at moderate load. Alibaba sounds interesting haven't heard much about it. By the way DeepSeek censorship from demos I saw is on the website but at least running R1 locally, it seems to not be censoring those things much or at all. |
#7
|
|||
|
|||
Censorship update: it appears if you download models and run locally there is no censorship.
Censoring is definitely done on their public website if you run your queries on their hardware. Here is how it works: when you send your query it goes into the real model avatars thinking or generating a response. At the same time, it is sent to a classifier model that is far cheaper, faster and specific. This classifier is trained with a prompt similar to "is the following '<prompt>' related to the following list of sensitive topics". If it returns yes, the main query is immediately aborted and a message displayed. Sometimes you can see it start to think and cut off a few sentences into it's deepthink. Other time it cuts off so fast that it appears nothing yet was emitted. No problem, run it locally, you won't deal with the censorship classifier. It would be interesting though to have a list of the sensitive topics but that is probably kept secretly and securely. Update: According to this article I am mistaken and the censorship concerns mentioned are legit: Quote:
Last edited by chants; 02-04-2025 at 07:04. |
#8
|
||||
|
||||
I only can prompt twice, then deepseek says,
Quote:
|
![]() |
Tags |
deepseek, ollama |
Thread Tools | |
Display Modes | |
|
|