Google Cloud Storage Connect your data

       We outline the process to set up GCS to connect your data to the Tenyks platform.


Before you start

📣 Please make sure you read the following points before continuing.

1. Expected folder structure

Here's the expected folder structure based on the option you choose:

read-only

Container name: {your_tenyks_data_bucket}

Images Directory: {your_tenyks_data_bucket}/{your_dataset_name}/{images_directory_name}/img_n.png

Predictions File (in COCO Format): {your_tenyks_data_bucket}/{your_dataset_name}/predictions.json

Annotations File (in COCO Format): {your_tenyks_data_bucket}/{your_dataset_name}/annotations.json

Metadata Directory: We will set this up for you!

For the read-write option, follow the same structure as in read-only but since your metadata remains in your own cloud, please add a Metadata Directory:

{your_tenyks_data_bucket}/{your_dataset_name}/{metadata_directory_name} (see Section 3.2)

2. Setting up GCS buckets

We assume you have set up the necessary bucket(s) based on the structure described above. If you haven't set up your buckets, this short tutorial shows you how to create GCS buckets.

3. Access Policies

🚧

Coming soon!

We're currently in the process of developing the read-only option.

Please, use the read-writeoption 😃.

Add the appropriate permissions depending on the approach you chose:

read-write

storage.objects.get

storage.objects.list

storage.objects.create

storage.objects.delete

storage.objects.update


read-only

storage.objects.get

storage.objects.list


JSON

(

    resource.type == 'storage.googleapis.com/Bucket' &&

    resource.name.startsWith("projects/_/buckets/{your_tenyks_data_bucket}")

) || (

    resource.type == 'storage.googleapis.com/Object' &&

    resource.name.startsWith("projects/_/buckets/{your_tenyks_data_bucket}/objects/{your_dataset_name}/")

)


⚠️ Note that you should replace the terms in brackets (e.g. {your_tenyks_data_bucket}) with your own bucket and folder names. Leave the rest unchanged.

Your JSON key should have a format similar to the following:

JSON

{

  "type": "service_account",

  "project_id": "gcs-testing-...",

  "private_key_id": "...",

  "private_key": "",  

  "client_email": "...",

  "client_id": "...",

  "auth_uri": "https://accounts.google.com/o/oauth2/auth",

  "token_uri": "https://oauth2.googleapis.com/token",

  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",

  "client_x509_cert_url": "",

  "universe_domain": "googleapis.com"

}


🚨 Note that the full JSON object should be copied and pasted as the value of credentials in the request body of some of the endpoints.

4. Metadata bucket CORS settings

If you are storing metadata on your own cloud, to use functionalities like the Embedding Viewer, please configure the CORS settings for your bucket as follows:


5. Summary

We successfully created the following: