Exetools  

Go Back   Exetools > General > General Discussion

Notices

Reply
 
Thread Tools Display Modes
  #1  
Old 01-29-2025, 08:22
chants chants is offline
VIP
 
Join Date: Jul 2016
Posts: 812
Rept. Given: 46
Rept. Rcvd 50 Times in 31 Posts
Thanks Given: 723
Thanks Rcvd at 1,124 Times in 519 Posts
chants Reputation: 51
Running DeepSeek R1 locally

DeepSeek has its flagship V3 model equivalent to GPT4 and it's reasoning model R1 freely accessible:
Quote:
https://www.deepseek.com/
AI training for 5.6 million USD exceeding the quality of 100 mil to 1 bil USD. Inference of high quality is within reach of your own local environment where your data stays private. I've found their models better at reasoning than OpenAIs significantly. It's quite exciting and I'm surprised noone has brought the topic up yet, given the large amount of use cases for reverse engineering and the very low cost.

If you have a GPU recommend OLlama which works on Windows, Linux and Mac (can also rin Facebook/Meta's Llama models):
Quote:
https://ollama.com/
. Can choose from the models listed here:
Quote:
https://ollama.com/library/deepseek-r1
. 8b is pretty lightweight but if you have a recent Nvidia GPU with a lot of RAM why not go for 32b.

For a frontend chat interface, I recommend Chatbot AI:
Quote:
https://chatboxai.app
Even better the V3 and R1 model are open source and you can do your own model finishing if you have the resources.

R1:
Quote:
https://github.com/deepseek-ai/DeepSeek-R1
V3:
Quote:
https://github.com/deepseek-ai/DeepSeek-V3
Reply With Quote
The Following 4 Users Say Thank You to chants For This Useful Post:
blue_devil (01-29-2025), DARKER (01-29-2025), Doit (01-30-2025), wx69wx2023 (01-31-2025)
  #2  
Old 01-29-2025, 15:23
blue_devil's Avatar
blue_devil blue_devil is offline
Family
 
Join Date: Dec 2011
Location: Observable Universe
Posts: 390
Rept. Given: 85
Rept. Rcvd 55 Times in 28 Posts
Thanks Given: 416
Thanks Rcvd at 591 Times in 196 Posts
blue_devil Reputation: 55
Are there any reverse engineering (especially for decompilation), specific models?
Reply With Quote
  #3  
Old 01-29-2025, 16:09
deepzero's Avatar
deepzero deepzero is offline
VIP
 
Join Date: Mar 2010
Location: Germany
Posts: 304
Rept. Given: 114
Rept. Rcvd 64 Times in 42 Posts
Thanks Given: 184
Thanks Rcvd at 216 Times in 92 Posts
deepzero Reputation: 64
This makes DIGITS even more interesting to me, considering buying one. (But it will only be out in may). https://www.nvidia.com/en-eu/project-digits/

What I dont get though is how 1PFLOG in FP4 is a selling point when the regular GTX5090 has like 600 TFLOP in FP32? Am I missing something or is the 128GB integrated RAM the selling factor?
Reply With Quote
  #4  
Old 01-29-2025, 19:18
sendersu sendersu is online now
VIP
 
Join Date: Oct 2010
Posts: 1,221
Rept. Given: 334
Rept. Rcvd 234 Times in 124 Posts
Thanks Given: 291
Thanks Rcvd at 598 Times in 335 Posts
sendersu Reputation: 200-299 sendersu Reputation: 200-299 sendersu Reputation: 200-299
There is even more powerfull ML model then DeepSeek - Alibaba’s Qwen2.5-Max
Reply With Quote
  #5  
Old 01-29-2025, 20:10
DARKER DARKER is offline
VIP
 
Join Date: Jul 2004
Location: Somewhere Over the Rainbow
Posts: 500
Rept. Given: 15
Rept. Rcvd 121 Times in 53 Posts
Thanks Given: 16
Thanks Rcvd at 909 Times in 228 Posts
DARKER Reputation: 100-199 DARKER Reputation: 100-199
DeepSeek censorship:
Code:
https://www.theguardian.com/technology/2025/jan/28/we-tried-out-deepseek-it-works-well-until-we-asked-it-about-tiananmen-square-and-taiwan
Reply With Quote
The Following 2 Users Say Thank You to DARKER For This Useful Post:
blue_devil (01-30-2025), Gyrus (01-30-2025)
  #6  
Old 01-29-2025, 22:33
chants chants is offline
VIP
 
Join Date: Jul 2016
Posts: 812
Rept. Given: 46
Rept. Rcvd 50 Times in 31 Posts
Thanks Given: 723
Thanks Rcvd at 1,124 Times in 519 Posts
chants Reputation: 51
It would be nice to train an RE model. The good news now is that training is being shown to be feasible possibly on an academic grant level budget. Someone should train a proper open source RE model at some point.

1PFLOP FP4 is a marketing gimmick maybe, that amount of RAM is a big plus tho. The new DeepSeek models use FP8 and have shown it's reliable for training, a good breakthrough. Sounds good enough to run good size models at moderate load.

Alibaba sounds interesting haven't heard much about it.

By the way DeepSeek censorship from demos I saw is on the website but at least running R1 locally, it seems to not be censoring those things much or at all.
Reply With Quote
  #7  
Old 02-03-2025, 12:43
chants chants is offline
VIP
 
Join Date: Jul 2016
Posts: 812
Rept. Given: 46
Rept. Rcvd 50 Times in 31 Posts
Thanks Given: 723
Thanks Rcvd at 1,124 Times in 519 Posts
chants Reputation: 51
Censorship update: it appears if you download models and run locally there is no censorship.

Censoring is definitely done on their public website if you run your queries on their hardware. Here is how it works:
when you send your query it goes into the real model avatars thinking or generating a response.

At the same time, it is sent to a classifier model that is far cheaper, faster and specific. This classifier is trained with a prompt similar to "is the following '<prompt>' related to the following list of sensitive topics". If it returns yes, the main query is immediately aborted and a message displayed. Sometimes you can see it start to think and cut off a few sentences into it's deepthink. Other time it cuts off so fast that it appears nothing yet was emitted.

No problem, run it locally, you won't deal with the censorship classifier. It would be interesting though to have a list of the sensitive topics but that is probably kept secretly and securely.

Update: According to this article I am mistaken and the censorship concerns mentioned are legit:
Quote:
https://techcrunch.com/2025/02/03/no-deepseek-isnt-uncensored-if-you-run-it-locally/

Last edited by chants; 02-04-2025 at 07:04.
Reply With Quote
  #8  
Old 02-03-2025, 15:25
blue_devil's Avatar
blue_devil blue_devil is offline
Family
 
Join Date: Dec 2011
Location: Observable Universe
Posts: 390
Rept. Given: 85
Rept. Rcvd 55 Times in 28 Posts
Thanks Given: 416
Thanks Rcvd at 591 Times in 196 Posts
blue_devil Reputation: 55
I only can prompt twice, then deepseek says,
Quote:
The server is busy. Please try again later.
Reply With Quote
Reply

Tags
deepseek, ollama

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



All times are GMT +8. The time now is 06:36.


Always Your Best Friend: Aaron, JMI, ahmadmansoor, ZeNiX, chessgod101
( 1998 - 2025 )