Creating datasets with CLI
Note: See Getting Started with Onepanel CLI for installation instructions
If you are creating a new dataset, use the
create command as follows:
onepanel datasets create <dataset-name>
This will create a new directory called
<dataset-name> and will create the respective dataset in Onepanel.
If you already have a directory with files,
cd into that directory and run the following command:
onepanel datasets init
This will create a Onepanel Dataset using the name of the directory.
You can then upload the contents of the directory to Onepanel by running:
onepanel datasets push [-m <message>]
Once you are done uploading files and want to lock the dataset from future changes, run:
onepanel datasets push -u [-m <message>]
This will upgrade the version and push any new files. Any future runs of
onepanel push will push the files to the new version.
onepanel datasets create cifar-10
tar -xvzf cifar-100-python.tar.gz
onepanel datasets push -u -m 'pushing first version'
Watching dataset changes
If you want to monitor your dataset for file changes and automatically upload them, run:
onepanel datasets push -w
This will output file changes as they happen.
If you want to run this in the background add a -b flag.
onepanel datasets push -wb
Pulling down dataset changes
If you have made changes to a dataset on one computer, and want to get the changes on another computer, run the following command in the directory that contains your dataset:
onepanel datasets pull