feat: add support for Claude & Llama 3.x 70b with Groq

[borisyankov]

Add support and experiment how well these two models perform:

• Claude 3.5 Sonnet - often cited as much better than 4 (and even 4o) at coding
• Llama 3.x 70b + Groq - not as good as the Claude or GPT 4o but inference is much faster and for many tasks it might be good enough

[grabbou]

Yup, ideally, we should allow configurability of the model, together with (if required) a model-specific prompt. The goal (and I think required next step) is to allow this as well as some self-hosted models.

I would also love to self-hosted something for everyone, so no API key is required. But it won't bring much to the table, unless I go bankrupt paying for it!

[adeleke5140]

Hi @grabbou, my team at mastra has been building an AI framework for a while and we have support for these models out of the box.

I see another issue regarding a logger and we have that built in too.

We are also built on top of AI sdk so I believe it should be plug and play. We can onboard you if you're interested.

[grabbou]

Thanks for reaching out @adeleke5140 but we're not looking to adopt any higher-level framework at this point. At this point, we want to stay close to the metal and focus on experimenting with more tools instead.

I am not against this in general, especially as we grow, so let me keep this in the back of my head. Thanks!

[adeleke5140]

@grabbou You are welcome. I understand and I'm excited to see where Cali goes.