Truefoundry

Engineering and Product

Scaling Up serving of Fine-tuned LoRA Models

In the realm of AI application deployment, scaling the serving of finetuned models has emerged as a pivotal challenge. Consider a scenario: you're a SaaS startup catering to 500 customers with diverse needs, requiring fine-tuned Language Model (LLM) adaptations for internal support and personalized marketing content. This entails

LLMs & GenAI

Benchmarking Popular Opensource LLMs: Llama2, Falcon, and Mistral

In this blog, we will show the summary of various open-source LLMs that we have benchmarked. We benchmarked these models from a latency, cost, and requests per second perspective. This will help you evaluate if it can be a good choice based on the business requirements. Please note that we

LLMs & GenAI

Benchmarking Mistral-7B

We benchmark the performance of Mistral-7B in this article from latency, cost, and requests per second perspective. This will help us evaluate if it can be a good choice based on the business requirements. Please note that we don't cover the qualitative performance in this article - there

LLMs & GenAI

Benchmarking Falcon-40B

We benchmark the performance of Falcon-40B-Instruct in this article from latency, cost, and requests per second perspective. This will help us evaluate if it can be a good choice based on the business requirements. Please note that we don't cover the qualitative performance in this article - there

LLMs & GenAI

Benchmarking Llama-2-70B

We benchmark the performance of LLama2-70B in this article from latency, cost, and requests per second perspective. This will help us evaluate if it can be a good choice based on the business requirements. Please note that we don't cover the qualitative performance in this article - there

LLMs & GenAI

Benchmarking Llama-2-13B

We benchmark the performance of LLama2-13B in this article from latency, cost, and requests per second perspective. This will help us evaluate if it can be a good choice based on the business requirements. Please note that we don't cover the qualitative performance in this article - there

True ML Talks

True ML Talks #21 - Machine Learning Platform @ Loblaw Digital

We are back with another episode of True ML Talks. In this, we dive deep into Loblaw Digital ML Platform and we are speaking with Adhithya Ravichandran Adhithya is a Senior Software Engineer at Loblaw Digital. He is part of the machine learning platform team, where he builds and maintains

LLMs & GenAI

Benchmarking Llama-2-7B

This blog captures Llama 2 7B benchmarks - where a model excels and the areas where it struggles. Make informed decisions about its practical deployment. In this blog, we have benchmarked the Llama-2-7B model from NousResearch on huggingface.

AI Center of Excellence: Future-proof your business

In this blog we cover what is an Artificial Intelligence (AI) center of excellence and how it is important for companies to meaningfully explore AI/ML use cases which can impact both their bottom-line and top-line effectively. Artificial Intelligence (AI) is no longer a futuristic concept that belongs in  science

Engineering and Product

Training Machine Learning Models using TrueFoundry's Jobs

Introduction In this blog, we will introduce you to training machine learning models on the TrueFoundry Platform. We will discuss how we can run training jobs on TrueFoundry. We will also see how you can perform hyperparameter tuning easily for your machine-learning models, and run your jobs on GPUs. Problem

LLMs & GenAI

How to Fine-Tune a Llama-2 (7B) to Beat ChatGPT

Large language models (LLMs) are becoming increasingly powerful, and they are being used for a variety of tasks, including chatbots, text generation, and question-answering. However, LLMs can be expensive and resource-intensive to train. In this blog post, we will show you how to fine-tune a smaller LLM (7B) to perform

True ML Talks

True ML Talks #17 - ML Platforms @ Slack, LLMs and SlackGPT

We are back with another episode of True ML Talks. In this, we dive deep into Slack's Recommend API, SlackGPT and LLM's and we are speaking with Katrina Ni. Katrina Ni is a senior ML engineer as well as a lead member of the technical staff

LLMs & GenAI

LLMOps CoE: The next frontier in the MLOps Landscape

In this blog, we will explore the importance of LLMOps and how it tackles the challenges associated with LLMs, such as iteration, prompt management and testing complexities. We also go a step further and suggest how you can get started on your LLMOps journey. Large language models (LLMs) have caused

Engineering and Product

Hosted Jupyter Notebooks and VS Code on Kubernetes

Jupyter notebooks are a powerful and popular tool that provides an interactive computing environment, combining code, data visualization, and explanatory text, making it easier to work with data and share insights. Data scientists use Jupyter notebooks for various tasks throughout the data analysis and machine learning lifecycle like exploratory data

Culture

Unleashing Innovation: Highlights from TrueFoundry's Internal Hackathon

At TrueFoundry, we are dedicated to developing an integrated platform solution that empowers data scientists to experiment, version, fine-tune and deploy their ML/LLM models with ease. Alongside our customer-focused endeavours, we believe in fostering a culture of innovation and providing our team with the opportunity to explore their ideas

True ML Talks

True ML Talks #14 - LLMs and Reinforcement Learning Co-founder @ CX Score

We are back with another episode of  True ML Talks. In this, we again dive deep into LLMs, Reinforcement Learning, and CX Score and we are speaking with Ashwin Rao. Ashwin Rao is a distinguished professional with a diverse background in academia, industry leadership, and entrepreneurship. He is currently a

Engineering and Product

Machine Learning Tool Integrations #2 DVC for your Data Versioning

Data version control (DVC) is a powerful tool for managing and versioning large datasets and models in machine learning projects. DVC is an open-source tool that works with a wide range of storage options, including local disk, network file systems, Amazon S3, Google Cloud Storage, and many others. DVC can

True ML Talks

True ML Talks #13 - ML Platform @ Cookpad

We are back with another episode of  True ML Talks. In this, we'll delve deep into the ML architecture at Cookpad, one of the world's largest recipe service platforms.  We will also cover the challenges of building a successful machine learning platform, how they use Nvidia

Engineering and Product

TrueFoundry Now Deploys and Fine-tunes Open Source LLMs With a Few Clicks!

TrueFoundry offers an intuitive solution for LLM deployment and fine-tuning. With our Model Catalogue, companies can self-host LLMs on Kubernetes, reducing inference costs by 10x in just one click. Discover how to deploy a Dolly-v2-3b model and fine-tune a Pythia-70m using TrueFoundry in our blog.

True ML Talks

True ML Talks #12 - Cofounder @ Llama-Index

We are back with another episode of  True ML Talks. In this, we dive deep into Llama-Index, and we are speaking with Jerry Liu. Jerry Liu is the creator and co-founder of LlamaIndex. He brings his expertise in ML research and engineering from esteemed companies like Uber, Quora, and Robust

True ML Talks

True ML Talks #11 - LLMs, LLMops and GenAI CTO @ Greenhouse

We are back with another episode of  True ML Talks. In this, we again dive deep into LLMs, LLMops and Generative AI, and we are speaking with Michael Boufford. Michael is the CTO of Greenhouse who joined as the first employee about 11 years ago, and so wrote the first

Engineering and Product

Open Source LLMs: Embrace or Perish

We speak with many enterprises and business leaders who are trying to figuring out their strategy for using LLMs in this pedestal of AI - should we go OpenAI or OpenSource LLMs? There are many good blogs that showcase the pros and cons of different approaches with neutral sentiment. We

True ML Talks

True ML Talks #10 - LLMs and GenAI with engineering director @ Meta

We are back with another episode of  True ML Talks. In this, we dive deep into LLM's and Generative AI's, and we are speaking with Anant. Anant is an engineering leader. Currently, an engineering director at Meta, working on some of the smart glasses product experiences.

True ML Talks

True ML Talks #9 - Machine Learning Platform @ DoorDash

We are back with another episode of  True ML Talks. In this, we dive deep into DoorDash's ML Platform, and we are speaking with Hien Luu. Hien Luu is the Senior Engineering Manager at DoorDash, building the build out of DoorDash's ML platform. DoorDash, as everyone

True ML Talks

True ML Talks #8 - Machine Learning Platform @ Intuit

We are back with another episode of  True ML Talks. In this, we dive deep into Intuit's ML Platform NumaFlow, and we are speaking with Vigith Maurice. Vigith is the principal engineer at Intuit for the AI ops platform. As anyone who has used TurboTax, Credit Karma, Mint,