Fine-tune and deploy a model

Fine-tuning allows you to customize pre-trained language models for your specific use cases and datasets, improving their performance on domain-specific tasks. The Dell Enterprise Hub provides you with guided steps, pre-configured settings and optimized training containers for fine-tuning models from the Dell Model Catalog and deploying them on your Dell infrastructure.

To start training one of the models available in the Dell Model Catalog, please follow the following steps:

1. Select base model: Start by choosing a trainable model in the Model Catalog. Currently the following models are available for training:
   • Meta-Llama-3.1-8B
   • Meta-Llama-3.1-70B
2. Configure training settings: From the Model Card, click Train, then select the Dell Platform you want to use. Next, set the local path of the CSV training dataset file, and the path to store the fine-tuned model. Learn below how to format and prepare your dataset at how should my dataset look. Finally, adjust the training configuration default settings to match your requirements.
3. Deploy training container: With Dell Enterprise Hub, model training jobs are configured within ready-to-use, optimized training containers. You can run your training job by deploying the container using the provided command, executed within your Dell environment.
4. Monitor training job: Track the progress of your training job to ensure optimal performance and results.

Training containers leverage Hugging Face autotrain, a powerful tool that simplifies the process of model training. Hugging Face autotrain supports a variety of configurations to customize training jobs, including:

• lr: Initial learning rate for the training.
• epochs: The number of training epochs.
• batch_size: Size of the batches used during training.

More details on these configurations can be found in the Autotrain CLI documentation.

Formatting your dataset

To finetune LLMs your dataset should have a column with the formatted training samples. The column used for training is defined through the text-column argument when starting your training, below it would be text.

Example format:

text
human: hello \n bot: hi nice to meet you
human: how are you \n bot: I am fine
human: What is your name? \n bot: My name is Mary
human: Which is the best programming language? \n bot: Python

You can use both CSV and JSONL files. For more details, refer to the original documentation.

Deploying a fine-tuned model

To deploy a fine-tuned model on your Dell Platform, you can use the special "Bring Your Own Model" (BYOM) Dell inference container available in the Dell Enterprise Hub. This makes it easy to integrate fine-tuned models seamlessly into your Dell environment.

1. Select base model: In the Model Catalog, open the Model Card for the base model used for fine-tuning, then click "Deploy Fine-Tuned Model" to access the BYOM feature.
2. Configure inference settings: Select the Dell Platform you want to use, and the configuration options. Make sure to correctly set the Path to the local directory where your fine-tuned model is stored.
3. Run deployment command: Copy the generated command, and run it inside your Dell environment.
4. Test your model: Once the BYOM container is set up and endpoints are up and running, test your model with the provided sample code snippets.

Unlike direct deployment of models provided in the Dell Model Catalog, when you deploy a fine-tuned model, the model is mounted to the BYOM Dell inference container. It's important to make sure that the mounted directory contains the fine-tuned model and the provided path is correct.

Hardware requirements

Gemma

For models fine-tuned from the Gemma base model, the following hardware configurations are recommended for deployment:

Llama 3.1 8B

For models fine-tuned from the Llama 3.1 8B base model, the following SKUs are suitable:

Llama 3.1 70B

For models fine-tuned from the Llama 3.1 70B base model, use these configurations for deployment:

Mistral 7B

Hardware configurations for models fine-tuned from the Mistral 7B are as follows:

Mixtral 8x7B

For models fine-tuned from the Mixtral base model, the deployment configurations are: