Creating datasets with CLI

Note: See Getting Started with Onepanel CLI for installation instructions

If you are creating a new dataset, use the create  command as follows:

onepanel datasets create <dataset-name>

This will create a new directory called <dataset-name>  and will create the respective dataset in Onepanel.

If you already have a directory with files, cd  into that directory and run the following command:

onepanel datasets init

This will create a Onepanel Dataset using the name of the directory.

You can then upload the contents of the directory to Onepanel by running:

onepanel datasets push [-m <message>]


Once you are done uploading files and want to lock the dataset from future changes, run:

onepanel datasets push -u [-m <message>]

This will upgrade the version and push any new files. Any future runs of onepanel push  will push the files to the new version. 

Example

onepanel datasets create cifar-10
cd cifar-10
wget https://www.cs.toronto.edu/~kriz/cifar-100-python.tar.gz
tar -xvzf cifar-100-python.tar.gz
onepanel datasets push -u -m 'pushing first version'

Watching dataset changes

If you want to monitor your dataset for file changes and automatically upload them, run:

onepanel datasets push -w 

This will output file changes as they happen. 

If you want to run this in the background add a -b flag.

onepanel datasets push -wb

Pulling down dataset changes

If you have made changes to a dataset on one computer, and want to get the changes on another computer, run the following command in the directory that contains your dataset:

onepanel datasets pull
Did this answer your question?