Skip to main content

Using XetHub as a MLflow artifact store

MLflow is an open source platform for managing end-to-end machine learning workflows. The MLflow-XetHub plugin allows you to turn your XetHub repository into a MLflow artifact store.

Install​

Install the latest version of the plugin from PyPI

pip install mlflow-xethub

Or clone the plugin repository and install locally for the latest code:

git clone https://github.com/xetdata/MLflow-XetHub.git
cd MLflow-XetHub
pip install .

Create a XetHub repository​

If you haven't already, create an XetHub account, then install and login.

Create a new repository from the XetHub UI or via the command line with:

xet repo make xet://<username>/<repo>

Start your MLflow server​

MLflow allows users to specify where runs and artifacts are stored. Start your MLflow server with your XetHub repository set as the artifacts destination and the plugin will automatically detect when to save to XetHub.

mlflow server --backend-store-uri ./mlruns --artifacts-destination xet://<username>/<repo>/<branch> --default-artifact-root xet://<username>/<repo>/<branch>

The command above uses the mlruns directory on your machine as file store backend and XetHub as the artifact store backend.

Run an MLflow experiment​

note

Before running this experiment, ensure that the MLflow code and server are in the same directory. The MLflow-XetHub plugin and MLflow must also be running in the same Python environment.

Follow MLflow's quickstart to create a Random Forest model on a simple dataset.

View artifacts in XetHub and the MLflow UI​

The artifacts will be automatically stored on XetHub under the specified repo and branch. artifact_on_xethub

The MLflow server will show the artifacts with UI on the default http://127.0.0.1:5000 or your own host. artifact_on_mlflow_ui