Deepseek V3: Architect or Editor

[schacki]

Issue

Would you use Deepseek V3 as archtict or Editor?
An update for this diagram might be interesting for people: https://aider.chat/2024/09/26/architect.html
Thanks a lot

Version and model info

No response

[slavakurilyak]

It would be great to see a PR for code editing w/ DeepSeek Chat v3 since we already have a benchmark for code reasoning

Anyone interested to contribute their benchmark results?

[lingster]

do we have an idea of how many tokens/time it might take to run the benchmarks?

[slavakurilyak]

@paul-gauthier any insights into token usage for the code editing benchmark?

Consider updating the benchmark page to include a token range so we can encourage the community to participate in benchmarks

[paul-gauthier]

It depends on the model. DeepSeek is very cheap so the benchmarks cost pennies.

Many models include costs in the yml data file, which are linked from the leaderboards contributing page.

https://github.com/Aider-AI/aider/tree/main/aider/website/_data

[lingster]

Just looking at that data and for claude sonnet, the cost is $13.48, so that just shy of 1million tokens to run the benchmark. Given that the current price for claude is around $15/MTok. Perhaps you can use use 1million tokens as a rought estimate for calculating the cost to run a benchmark. We do seem to be missing figures for openrouter/o1 for some reason. From: paul-gauthier ***@***.***> Date: Wednesday, 8 January 2025 at 17:13 To: Aider-AI/aider ***@***.***> Cc: Ling Li ***@***.***>, Comment ***@***.***> Subject: Re: [Aider-AI/aider] Deepseek V3: Architect or Editor (Issue #2745) It depends on the model. DeepSeek is very cheap so the benchmarks cost pennies. Many models include costs in the yml data file, which are linked from the leaderboards contributing page. https://github.com/Aider-AI/aider/tree/main/aider/website/_data — Reply to this email directly, view it on GitHub<#2745 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAGPUPNETTV64VFEUNI4GMD2JVE6RAVCNFSM6AAAAABUO4TV5SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNZYGA3DKMZRGI>. You are receiving this because you commented.Message ID: ***@***.***>

[griffin-ezbot]

I'm also interested in your recommendations, @paul-gauthier, for using deepseek chat v3 as an architect and/or engineer in architect mode. The raw data you shared looks very promising.

[github-actions]

I'm labeling this issue as stale because it has been open for 2 weeks with no activity. If there are no additional comments, I will close it in 7 days.

Note: A bot script made these updates to the issue.

[github-actions]

I'm closing this issue because it has been stalled for 3 weeks with no activity. Feel free to add a comment here and we can re-open it. Or feel free to file a new issue at any time.

Note: A bot script made these updates to the issue.