To create a dataset from Google Cloud Storage (GCS) buckets, you will need to first launch a Workspace.  Make sure to select a storage size that is at least as big as your dataset.

Once the Workspace is launched, install gsutil :

curl https://sdk.cloud.google.com | bash

Once installation is complete, restart shell:

exec -l $SHELL


Then, you need to either login with your personal account:

gcloud auth login

or you can alternatively login using a service account.

Once logged in, you can download the bucket into a local folder: 

gsutil cp gs://<gs-bucket-name>/ /onepanel/input/datasets/<your-dataset-name>

Then, cd  into that folder and create the Onepanel dataset as follows:

cd /onepanel/input/datasets/<your-dataset-name>
onepanel datasets init
onepanel datasets push -b

Note that by setting the -b option on onepanel datasets push , your upload will run in the background.  You will receive an email when the upload is complete.

Did this answer your question?