What is the Dell Enterprise Hub?

The Dell Enterprise Hub is an online portal making it easy to train and deploy the latest open AI models on-premise using Dell platforms, and securely build Generative AI applications. The Dell Enterprise Hub is the result of a deep engineering collaboration between Dell Technologies and Hugging Face, it includes:

1. Optimized Containers: Ready-to-use containers and scripts for model deployment and fine-tuning, optimized for each base model and tested for Dell hardware configurations.
2. Dell Platforms native support: Dell customers easily filter models by the Dell Platforms they have access to, and access tested and maintained configurations for training and inference.
3. Bring Your Own Model: Dell Enterprise Hub supports importing compatible fine-tuned models into the optimized containers.
4. Enterprise Security: Leverage Hugging Face Enterprise Hub advanced security features including Single Sign-On, fine-grained access controls, audit logs and malware scanning, along with the enterprise-curated models offered in Dell Enterprise Hub.

The Dell Enterprise hub provides a secure, streamlined experience for Dell customers to build Generative AI applications in confidence, taking full advantage of the computing power of the Dell Platforms at their disposal.

How do I deploy a Model?

Deploying a model on a Dell Platform is a simple 4 steps process:

1. Choose Your Model: Select a model from the Dell Model Catalog.
2. Configure Inference Settings: In the Model Card, click Deploy, then select a compatible Dell Platform and the desired configuration.
3. Run Deployment Command: Copy the generated command, and run it inside the Dell infrastructure.
4. Test Your Model: Once the container is set up and endpoints are up and running, test your model with the provided sample code snippets.

If you want to deploy a fine-tuned model instead of the curated models above, refer to How can I deploy a fine-tuned model?

The Dell Enterprise Hub inference containers leverage Hugging Face ML production technologies, including Text Generation Inference for Large Language Models. The predefined configurations provided can be easily adjusted to fit your needs, by changing the default values for:

• NUM_SHARD: The number of shards, or tensor parallelism, used for the model.
• (optional) MAX_INPUT_LENGTH: The maximum input length that the model can handle.
• (optional) MAX_TOTAL_TOKENS: The maximum total tokens the model can generate.
• (optional) MAX_BATCH_PREFILL_TOKENS: The maximum number of tokens to prefill the batch used for continuous batching.

More information can be found in the Hugging Face Text Generation Inference documentation, and also in the Text Generation Inference v3 release notes.

How can I fine-tune a Model?

To start training one of the models available in the Dell Model Catalog, please follow the following steps:

1. Select Base Model: Start by choosing a trainable model in the Model Catalog. Currently the following models are available for training:
   • Meta-Llama-3.1-8B
   • Meta-Llama-3.1-70B
2. Configure Training Settings: From the Model Card, click Train, then select the Dell Platform you want to use. Next, set the local path of the CSV training dataset file, and the path to store the fine-tuned model. Learn how to format and prepare your dataset at how should my dataset look. Finally, adjust the training configuration default settings to match your requirements.
3. Deploy Training Container: With Dell Enterprise Hub, model training jobs are configured within ready-to-use, optimized training containers. You can run your training job by deploying the container using the provided command, executed within your Dell environment.
4. Monitor Training Job: Track the progress of your training job to ensure optimal performance and results.

Training containers leverage Hugging Face autotrain, a powerful tool that simplifies the process of model training. Hugging Face autotrain supports a variety of configurations to customize training jobs, including:

• lr: Initial learning rate for the training.
• epochs: The number of training epochs.
• batch_size: Size of the batches used during training.

More details on these configurations can be found in the Autotrain CLI documentation.

How should my dataset look?

To finetune LLMs your dataset should have a column with the formatted training samples. The column used for training is defined through the text-column argument when starting your training, below it would be text.

Example Format:

text
human: hello \n bot: hi nice to meet you
human: how are you \n bot: I am fine
human: What is your name? \n bot: My name is Mary
human: Which is the best programming language? \n bot: Python

You can use both CSV and JSONL files. For more details, refer to the original documentation.

How can I deploy a fine-tuned model?

To deploy a fine-tuned model on your Dell Platform, you can use the special "Bring Your Own Model" (BYOM) Dell inference container available in the Dell Enterprise Hub. This makes it easy to integrate fine-tuned models seamlessly into your Dell environment.

1. Select Base Model: In the Model Catalog, open the Model Card for the base model used for fine-tuning, then click "Deploy Fine-Tuned Model" to access the BYOM feature.
2. Configure Inference Settings: Select the Dell Platform you want to use, and the configuration options. Make sure to correctly set the Path to the local directory where your fine-tuned model is stored.
3. Run Deployment Command: Copy the generated command, and run it inside your Dell environment.
4. Test Your Model: Once the BYOM container is set up and endpoints are up and running, test your model with the provided sample code snippets.

Unlike direct deployment of models provided in the Dell Model Catalog, when you deploy a fine-tuned model, the model is mounted to the BYOM Dell inference container. It's important to make sure that the mounted directory contains the fine-tuned model and the provided path is correct.

How does Dell Enterprise Hub strengthen supply chain security?

In order to enable users to easily run popular models from the Hugging Face Hub on a variety of devices on the Dell Enterprise Hub, we provide custom docker images. Security being a top concern of our customers, these docker images are not only tested for functionality and performance, but also for security. When it comes to these artifacts, there are two major components of security testing.

1. Docker image security: This refers to the security posture of packages within the container image itself. While docker images are static artifacts, the open source community finds and publishes new package vulnerabilities regularly via the National Vulnerability Database. For docker image security, we currently use Amazon Inspector to scan for vulnerabilities in our model containers. We follow the CVSS v3 ratings associated with a particular vulnerability. Images are scanned daily, and we are notified immediately when a critical vulnerability emerges.
2. Model security: This refers to the safety of the model itself, including its files and potential malicious functional behavior. Hugging Face performs malware and pickle scans on all model repositories with in-house scanners and third-party scanners, to assess model security. Reports can be found by navigating to the Files and versions tab on every model page. These scans protect enterprises against the risk of malicious content being introduced via models. For more information on the scanners used, please visit the Hugging Face Hub documentation.

DEH containers provide a complete, self-contained environment enabling on-premises model deployment. Once a container and model files are downloaded, no external internet connection is required. This ensures secure execution fully within your infrastructure, without needing to transmit data externally.

Hardware Requirements

Gemma

For models fine-tuned from the Gemma base model, the following hardware configurations are recommended for deployment:

Llama 3.1 8B

For models fine-tuned from the Llama 3.1 8B base model, the following SKUs are suitable:

Llama 3.1 70B

For models fine-tuned from the Llama 3.1 70B base model, use these configurations for deployment:

Mistral 7B

Hardware configurations for models fine-tuned from the Mistral 7B are as follows:

Mixtral 8x7B

For models fine-tuned from the Mixtral base model, the deployment configurations are:

What does deprecated mean?

A deprecated model status indicates the model is no longer actively maintained, but remains fully functional for inference and fine-tuning (if applicable), whilst it will no longer receive updates, or regular maintenance.

Deprecation typically occurs due to low usage metrics, customers can always continue using deprecated models, but we recommend migrating to actively maintained alternatives e.g. Meta Llama 3.1 may be deprecated, but Meta Llama 3.3 is available.

If we stop seeing usage from deprecated models for multiple months, we then remove them. For reference, here is the list of models we have removed from the catalog so far:

Resources

• Hugging Face Enterprise Hub documentation
• Hugging Face Text Generation Inference documentation
• Hugging Face AutoTrain documentation