Creating a Onepanel dataset from a Kaggle dataset using the Kaggle API is easy and can accomplished in a few steps
1. Create a workspace that is at least double the size of the dataset.
Hint: For example if the Kaggle dataset is 90GB then you should create a Onepanel Workspace that is at least 200GB. This is due to duplication during the git commit process. If you do not do this it will fail.
Open Terminal from the Workspace and paste the following commands into the terminal:
2. Create the Onepanel dataset:
onepanel datasets create <dataset-name>
3. Download Kaggle API
pip install --user kaggle-cli
kg download -u <username> -p <password> -c <competition>
Hint: The competition name can be found on the kaggle competitions "data" tab. It looks something like this:
4. Push your files to the Onepanel code repo: