Warning: This blog post references deprecated XetHub links and functionality. Please use as a reference only and follow our current work on Hugging Face.

August 14, 2023

August Edition: Capsules, Llama 2, and PyXet

Ann Huang

Ann Huang

Ann Huang

Happy August, Xetters!  

We're excited to kickoff our first monthly XetHub newsletter, where we showcase new product features, fun projects built on XetHub, and upcoming events. The team has been hard at work these last few months and, in this edition, we're excited to talk about XetHub Capsules, mounting and running LLAMA 2 in seconds, and new XetHub access patterns with PyXet and Xet CLI.

🚀 Deploy Streamlit or Gradio Apps Using Capsules

To eliminate the burden of sharing Streamlit and Gradio apps, we’re excited to launch Capsules in early access! With Capsules, you can deploy interactive Python apps from your XetHub repo in one click. These apps can even write files back into your repo and are easily shareable via URL. Experience the power of Capsules using our Stable Diffusion Text-to-Image Generator.

🦙 Mount & Run Llama 2 in Minutes

When Meta announced Llama 2, we couldn’t wait to try it out … only to realize that it takes a long time to move ~660GB worth of files between servers. Thanks to XetHub’s ability to stream files stored in a repo, you can mount and load our Llama 2 repo in minutes and run inference on your Macbook in seconds. Here's a link to the tutorial.

🐍 PyXet and Xet CLI: More ways to use XetHub

We started our adventure at XetHub using our content de-duplication technology to scale Git for big data. We iterated on our underlying technologies to support 10+ TB repos and then tested our original Git-Xet commands on huge repositories like RedPajama and Falcon.

Boy did we learn a lot! Git is hard for big files. Your early feedback agreed — data and ML teams were much more interested in a S3-like interface with automatic versioning. We’ve made this a reality with Xet CLI and PyXet.

We’re working on writing some blog posts to share all our lessons learned, but in the meantime, check out our new Getting Started flow to try XetHub for yourself.

Share on