TrueFoundry Blog (Page 2)

TrueFoundry Blog

Efficiently Serving LoRA fine-tuned models

Efficiently Serving LoRA fine-tuned models

In the domain of language models, the practice of fine-tuning is widely adopted to tailor an existing language model for specific tasks and datasets. There are a lot of good resources online that describe what is fine tuning and what are the different parameter efficient techniques for it. This blog

Exploring Transformers, Embeddings and LLMs

True ML Talks #20 - Transformers, Embeddings and LLMs - ML Scientist @ Turnitin

We are back with another episode of True ML Talks. In this, we'll delve deep into the main ideas of the fascinating paper titled Analyzing Transformer Dynamics as Movement through Embedding Space. This paper introduces a novel perspective on how Transformers operate, emphasizing that they learn an embedding

Benchmarking Llama-2-7B

Benchmarking Llama-2-7B

This blog captures Llama 2 7B benchmarks - where a model excels and the areas where it struggles. Make informed decisions about its practical deployment. In this blog, we have benchmarked the Llama-2-7B model from NousResearch on huggingface.

AI Center of Excellence

AI Center of Excellence: Future-proof your business

In this blog we cover what is an Artificial Intelligence (AI) center of excellence and how it is important for companies to meaningfully explore AI/ML use cases which can impact both their bottom-line and top-line effectively. Artificial Intelligence (AI) is no longer a futuristic concept that belongs in science

Machine Learning Tool Integrations #3 Label Studio for All your Labelling Needs

Engineering and Product

Machine Learning Tool Integrations #3 Label Studio for All your Labelling Needs

In the following blog we are going to talk about Label Studio and how you can easily utilize Label Studio for Labelling via deploying it on TrueFoundry Quality of data is much more important than quantity. This is a well-established principle in the machine learning community, and it is becoming

Is AI Workload Bloating Up Your Cloud Bill?

Thought Leadership

Is AI Workload Bloating Up Your Cloud Bill?

We firmly believe that every company will be a Machine Learning (ML) company in the coming few years. As organizations embrace ML, one of the significant challenges they face is managing the associated cloud costs. Running AI/ML workloads in the cloud can quickly become expensive, but with careful planning

Training Machine Learning Models using TrueFoundry's Jobs

Engineering and Product

Training Machine Learning Models using TrueFoundry's Jobs

Introduction In this blog, we will introduce you to training machine learning models on the TrueFoundry Platform. We will discuss how we can run training jobs on TrueFoundry. We will also see how you can perform hyperparameter tuning easily for your machine-learning models, and run your jobs on GPUs. Problem

Machine Learning Deployments in 2023

Machine Learning Deployments in 2023

Deployment of applications and infrastructure is a complex process and requires coordination from Infra and application developer teams. An internal developer platform helps both teams move fast with fewer mistakes — resulting in increased productivity with a higher amount of stability. Key things to take care of in Deployment and Software

Helm Charts on ArtifactHub via hosting on Github pages

Engineering and Product

Helm Charts on ArtifactHub via hosting on Github pages

Let's discuss how we can host our helm charts on GitHub pages and allow other people to use our charts on ArtifactHub. Before understanding the process we will try to understand these terms in the simplest form What is a Helm Chart Helm charts is the way you

LLM-powered QA Chatbot on your data in your Cloud

Engineering and Product

LLM-powered QA Chatbot on your data in your Cloud

OpenAI and Langchain have made it really easy to build a demo for question-answering over your documents. There are plenty of articles on the internet outlining how to do this. We also have a working notebook in case you want to play with a system end-to-end: truefoundry-examples/docs-qa-langchain/simple at

How to Fine-Tune a Llama-2 (7B) to Beat ChatGPT

How to Fine-Tune a Llama-2 (7B) to Beat ChatGPT

Large language models (LLMs) are becoming increasingly powerful, and they are being used for a variety of tasks, including chatbots, text generation, and question-answering. However, LLMs can be expensive and resource-intensive to train. In this blog post, we will show you how to fine-tune a smaller LLM (7B) to perform

ML Machine Learning Platform Slack LLM SlackGPT MLOps

True ML Talks #17 - ML Platforms @ Slack, LLMs and SlackGPT

We are back with another episode of True ML Talks. In this, we dive deep into Slack's Recommend API, SlackGPT and LLM's and we are speaking with Katrina Ni. Katrina Ni is a senior ML engineer as well as a lead member of the technical staff

LLMOps: Mastering the Art of Managing Large Language Models - Challenges, Components, and Top Platforms

LLMOps: Mastering the Art of Managing Large Language Models - Challenges, Components, and Top Platforms

Table of Contents 1. Introduction to LLMOps 2. Essential Components of LLMOps 3. Benefits and Advantages of LLMOps 4. LLMOps Platforms and Tools 5. Conclusion Introduction to LLMOps LLMOps and its role in managing large language models LLMOps, also known as Large Language Model Operations, encompass the specialized practices and

Mastering MLOps: A Comprehensive Guide to MLOps Platforms and Best Practices for Successful ML Operations

Engineering and Product

Mastering MLOps: A Comprehensive Guide to MLOps Platforms and Best Practices for Successful ML Operations

What is MLOps? MLOps, short for Machine Learning Operations, is a discipline that merges machine learning (ML) development and operations to streamline the deployment of ML models in real-world applications. Its primary goal is to standardize and automate the continuous delivery of high-performing ML systems, ensuring their reliability and scalability.

Deploy and Fine Tune Llama 2 on your cloud

Engineering and Product

Llama 2 LLM: Deploy & Fine Tune on your cloud

The ChatGPT moment of the open source world is here- Meta released its latest set of open-source large language models, called Llama 2 - a collection of pre-trained and fine-tuned generative text models ranging in scale from 7 billion to 70 billion parameters. Fun Facts & Mnemonics about Llama 2

LLMOps CoE: The next frontier in the MLOps Landscape

LLMOps CoE: The next frontier in the MLOps Landscape

In this blog, we will explore the importance of LLMOps and how it tackles the challenges associated with LLMs, such as iteration, prompt management and testing complexities. We also go a step further and suggest how you can get started on your LLMOps journey. Large language models (LLMs) have caused

Hosted Jupyter Notebooks and VS Code on Kubernetes

Engineering and Product

Hosted Jupyter Notebooks and VS Code on Kubernetes

Jupyter notebooks are a powerful and popular tool that provides an interactive computing environment, combining code, data visualization, and explanatory text, making it easier to work with data and share insights. Data scientists use Jupyter notebooks for various tasks throughout the data analysis and machine learning lifecycle like exploratory data

TrueFoundry's team offsite and hackathon

Unleashing Innovation: Highlights from TrueFoundry's Internal Hackathon

At TrueFoundry, we are dedicated to developing an integrated platform solution that empowers data scientists to experiment, version, fine-tune and deploy their ML/LLM models with ease. Alongside our customer-focused endeavours, we believe in fostering a culture of innovation and providing our team with the opportunity to explore their ideas

Deploy Falcon-40B

Deploy Falcon-40B on AWS: 40% Cheaper than Sagemaker

In this article, we discuss about deploying Falcon model on your own cloud. The Technology Innovation Institute in Abu Dhabi has developed Falcon, an innovative series of language models. These models, released under the Apache 2.0 license, represent a significant advancement in the field. Notably, Falcon-40B stands out as

Thought Leadership

The $360K question about Large Language Models Economics

The purpose of this article is to educate the reader about how Large Language Models (LLM) pricing works. This is motivated by our conversations with multiple companies using LLMs commercially. We realized in these conversations that LLM economics is often misunderstood, leaving a huge scope for optimization. Do you realize

True ML Talks #14 - LLMs and Reinforcement Learning Co-founder @ CX Score

True ML Talks #14 - LLMs and Reinforcement Learning Co-founder @ CX Score

We are back with another episode of True ML Talks. In this, we again dive deep into LLMs, Reinforcement Learning, and CX Score and we are speaking with Ashwin Rao. Ashwin Rao is a distinguished professional with a diverse background in academia, industry leadership, and entrepreneurship. He is currently a

TrueFoundry Architecture - Machine Learning on Kubernetes

TrueFoundry Architecture - Machine Learning on Kubernetes

TrueFoundry makes it really easy to deploy applications on Kubernetes clusters in your own cloud provider account. It does so by abstracting out the infrastructure components for data scientists and developers while enforcing the best practices from security, infrastructure, and cost optimization perspectives. The key motivations behind the current architecture

Machine Learning Tool Integrations #2 DVC for your Data Versioning

Engineering and Product

Machine Learning Tool Integrations #2 DVC for your Data Versioning

Data version control (DVC) is a powerful tool for managing and versioning large datasets and models in machine learning projects. DVC is an open-source tool that works with a wide range of storage options, including local disk, network file systems, Amazon S3, Google Cloud Storage, and many others. DVC can

TrueFoundry with ML Platform Lead of Cookpad

True ML Talks #13 - ML Platform @ Cookpad

We are back with another episode of True ML Talks. In this, we'll delve deep into the ML architecture at Cookpad, one of the world's largest recipe service platforms. We will also cover the challenges of building a successful machine learning platform, how they use Nvidia

TrueFoundry for ML deployment

Is TrueFoundry ML Platform Right for you?

The ML infrastructure landscape is filled with some of the most impressive solutions out there to simplify the ML pipeline. TrueFoundry can be a solution if you relate to some of the problems mentioned below: Its taking us quite long to get our ML models into production, and there is