
Prime Mind disclosed an unauthorized DeepSeek mannequin swap on Nov. 19. The host mentioned a modified copy of DeepSeek V4.1 Flash served buyer requests on one deployment for 53 hours. The substitute matched the unique on two functionality exams, the host mentioned. On a security take a look at known as HarmBench, refusals fell from 198 of 240 prompts to zero.
Prime Mind mentioned the affected deployment ran the substitute from 01:24 UTC on Nov. 8 till 06:35 UTC on Nov. 10, a window of 53 hours and 11 minutes. The corporate didn’t say what number of requests went to that deployment throughout the swap.
The substitute used publicly obtainable weights that Prime Mind mentioned had been modified to scale back refusals. Weights are the numerical settings that form what a mannequin generates. The swap passed off behind the DeepSeek V4.1 Flash endpoint. Prime Mind mentioned the deployment saved its chat template and serving settings, together with the tokenizer that breaks textual content into items the mannequin processes.
What exams discovered after the DeepSeek mannequin swap
Prime Mind reported unchanged outcomes on GPQA Diamond and τ-bench when it examined the modified weights. GPQA Diamond exams superior science questions, whereas τ-bench exams whether or not brokers can use instruments and comply with guidelines in simulated buyer interactions.
The protection exams uncovered a distinction. HarmBench was developed to guage how fashions reply to makes an attempt to elicit dangerous outputs. Prime Mind additionally recorded a drop from 98 refusals out of 100 prompts to at least one on JailbreakBench, and from 304 out of 313 to 2 on StrongREJECT. The counts are from the corporate’s benchmark exams. The advisory offers no refusal depend for buyer requests.
A comparability restricted to the 2 functionality exams would have missed the autumn in refusals.
The protection benchmarks use totally different immediate totals. Counts signify benchmark exams. The advisory doesn’t present particular person functionality scores or buyer final result counts.
Prime Mind mentioned it contacted all affected clients and suggested those that relied on the mannequin to refuse sure requests to assessment the outputs they acquired throughout the affected window. A solution from the substitute may have handed into an software that was relying on a refusal.
The corporate mentioned it examined the disks and reminiscence of the machines concerned, checked recovered variations of the intruder’s proxy and reviewed information of their exercise. It reported discovering no proof that buyer prompts or responses have been recorded or left the serving cluster. Prime Mind additionally mentioned no different fashions on its platform have been modified and no keys or buyer credentials have been affected. Its investigation stays open.