What is the Dell Enterprise Hub?

The Dell Enterprise Hub is an online portal making it easy to train and deploy the latest open AI models on-premise using Dell platforms, and securely build Generative AI applications. The Dell Enterprise Hub is the result of a deep engineering collaboration between Dell Technologies and Hugging Face, it includes:

1. Optimized Containers: Ready-to-use containers and scripts for model deployment and fine-tuning, optimized for each base model and Dell hardware configuration.
2. Dell Platforms native support: Dell customers easily filter models by the Dell Platforms they have access to, and access tested and maintained configurations for training and inference.
3. Bring Your Own Model: Dell Enterprise Hub supports importing compatible fine-tuned models into the optimized containers.
4. Enterprise Security: Leverage Hugging Face Enterprise Hub advanced security features including Single Sign-On, fine-grained access controls, audit logs and malware scanning, along with the enterprise-curated models offered in Dell Enterprise Hub.

The Dell Enterprise hub provides a secure, streamlined experience for Dell customers to build Generative AI applications in confidence, taking full advantage of the computing power of the Dell Platforms at their disposal.

How do I deploy a Model?

Deploying a model on a Dell Platform is a simple 4 steps process:

1. Choose Your Model: Select a model from the Dell Model Catalog. Currently, supported models for direct deployment are:
   • Meta-Llama-3.1-8B-Instruct
   • Meta-Llama-3.1-70B-Instruct
   • Meta-Llama-3.1-405B-Instruct
   • Meta-Llama-3.1-405B-Instruct-FP8
   • Zephyr-7B Beta
   • Mixtral-8x22B-Instruct v0.1
2. Configure Inference Settings: In the Model Card, click Deploy, then select a compatible Dell Platform and the desired configuration.
3. Run Deployment Command: Copy the generated command, and run it inside the Dell infrastructure.
4. Test Your Model: Once the container is set up and endpoints are up and running, test your model with the provided sample code snippets.

If you want to deploy a fine-tuned model instead of the curated models above, refer to How can I deploy a fine-tuned model?

The Dell Enterprise Hub inference containers leverage Hugging Face ML production technologies, including Text Generation Inference for Large Language Models. The predefined configurations provided can be easily adjusted to fit your needs, by changing the default values for:

• NUM_SHARD: The number of shards, or tensor parallelism, used for the model.
• (optional) MAX_INPUT_LENGTH: The maximum input length that the model can handle.
• (optional) MAX_TOTAL_TOKENS: The maximum total tokens the model can generate.
• (optional) MAX_BATCH_PREFILL_TOKENS: The maximum number of tokens to prefill the batch used for continuous batching.

More information can be found in the Hugging Face Text Generation Inference documentation, and also in the Text Generation Inference v3 release notes.

How can I fine-tune a Model?

To start training one of the models available in the Dell Model Catalog, please follow the following steps:

1. Select Base Model: Start by choosing a trainable model in the Model Catalog. Currently the following models are available for training:
   • Meta-Llama-3.1-8B
   • Meta-Llama-3.1-70B
   • Google Gemma-7B
   • Mistral 7B v0.1
   • Mixtral 8x7B v0.1
2. Configure Training Settings: From the Model Card, click Train, then select the Dell Platform you want to use. Next, set the local path of the CSV training dataset file, and the path to store the fine-tuned model. Learn how to format and prepare your dataset at how should my dataset look. Finally, adjust the training configuration default settings to match your requirements.
3. Deploy Training Container: With Dell Enterprise Hub, model training jobs are configured within ready-to-use, optimized training containers. You can run your training job by deploying the container using the provided command, executed within your Dell environment.
4. Monitor Training Job: Track the progress of your training job to ensure optimal performance and results.

Training containers leverage Hugging Face autotrain, a powerful tool that simplifies the process of model training. Hugging Face autotrain supports a variety of configurations to customize training jobs, including:

• lr: Initial learning rate for the training.
• epochs: The number of training epochs.
• batch_size: Size of the batches used during training.

More details on these configurations can be found in the Autotrain CLI documentation.

How should my dataset look?

To finetune LLMs your dataset should have a column with the formatted training samples. The column used for training is defined through the text-column argument when starting your training, below it would be text.

Example Format:

text
human: hello \n bot: hi nice to meet you
human: how are you \n bot: I am fine
human: What is your name? \n bot: My name is Mary
human: Which is the best programming language? \n bot: Python

You can use both CSV and JSONL files. For more details, refer to the original documentation.

How can I deploy a fine-tuned model?

To deploy a fine-tuned model on your Dell Platform, you can use the special "Bring Your Own Model" (BYOM) Dell inference container available in the Dell Enterprise Hub. This makes it easy to integrate fine-tuned models seamlessly into your Dell environment.

1. Select Base Model: In the Model Catalog, open the Model Card for the base model used for fine-tuning, then click "Deploy Fine-Tuned Model" to access the BYOM feature.
2. Configure Inference Settings: Select the Dell Platform you want to use, and the configuration options. Make sure to correctly set the Path to the local directory where your fine-tuned model is stored.
3. Run Deployment Command: Copy the generated command, and run it inside your Dell environment.
4. Test Your Model: Once the BYOM container is set up and endpoints are up and running, test your model with the provided sample code snippets.

Unlike direct deployment of models provided in the Dell Model Catalog, when you deploy a fine-tuned model, the model is mounted to the BYOM Dell inference container. It's important to make sure that the mounted directory contains the fine-tuned model and the provided path is correct.

Hardware Requirements

Gemma

For models fine-tuned from the Gemma base model, the following hardware configurations are recommended for deployment:

Llama 3.1 8B

For models fine-tuned from the Llama 3.1 8B base model, the following SKUs are suitable:

Llama 3.1 70B

For models fine-tuned from the Llama 3.1 70B base model, use these configurations for deployment:

Mistral 7B

Hardware configurations for models fine-tuned from the Mistral 7B are as follows:

Mixtral 8x7B

For models fine-tuned from the Mixtral base model, the deployment configurations are:

What does deprecated mean?

A deprecated model status indicates the model is no longer actively maintained, but remains fully functional for inference and fine-tuning (if applicable), whilst it will no longer receive updates, or regular maintenance.

Deprecation typically occurs due to low usage metrics, customers can always continue using deprecated models, but we recommend migrating to actively maintained alternatives e.g. Meta Llama 3.1 may be deprecated, but Meta Llama 3.3 is available.

Resources

• Hugging Face Enterprise Hub documentation
• Hugging Face Text Generation Inference documentation
• Hugging Face AutoTrain documentation