Skip to content

chore: Migrate gsutil usage to gcloud storage #65

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
bhandarivijay-png wants to merge 1 commit into
base: master
Choose a base branch
from
Open

chore: Migrate gsutil usage to gcloud storage #65

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions tensorflow-planespotting/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,11 +76,11 @@ This also allows better shuffling, which is important for the final accuracy.
Both formats are available in a public Cloud Storage bucket. You can list them using `gsutil`:
```bash
# large aerial photographs and ROI files:
gsutil ls -l gs://planespotting-data-public/USGS_public_domain_photos
gsutil ls -l gs://planespotting-data-public/USGS_public_domain_photos_eval
gcloud storage ls --long gs://planespotting-data-public/USGS_public_domain_photos
gcloud storage ls --long gs://planespotting-data-public/USGS_public_domain_photos_eval
# 256x256 pixel tiles and adjusted ROI files:
gsutil ls -l gs://planespotting-data-public/tiles_from_USGS_photos
gsutil ls -l gs://planespotting-data-public/tiles_from_USGS_photos_eval
gcloud storage ls --long gs://planespotting-data-public/tiles_from_USGS_photos
gcloud storage ls --long gs://planespotting-data-public/tiles_from_USGS_photos_eval
```

You can run the data generation alone using this script. Make sure you provide your
Expand Down Expand Up @@ -158,8 +158,8 @@ run it from a local web browser:
cd webui-mark
# copy the existing data, add yours if you have any
mkdir USGS_public_domain_airports
gsutil -m cp gs://planespotting-data-public/USGS_public_domain_photos/* USGS_public_domain_airports
gsutil -m cp gs://planespotting-data-public/USGS_public_domain_photos_eval/* USGS_public_domain_airports
gcloud storage cp gs://planespotting-data-public/USGS_public_domain_photos/* USGS_public_domain_airports
gcloud storage cp gs://planespotting-data-public/USGS_public_domain_photos_eval/* USGS_public_domain_airports
# run from local server (any port will do)
python -m SimpleHTTPServer 8000
```
Expand All @@ -184,4 +184,4 @@ in the `serving_input_fn` function so that it happens on the fly when the
model is deployed on AI Platform. It is a good example of how deployed models
can run preprocessing on their input data directly on AI Platform while serving.
Code here: [trainer/train.py](trainer/train.py) function `serving_input_fn`.