Hey folks, I’m in a bit of a pickle here. I use GCP VMs for my research stuff and they spit out these huge files. Problem is, I’m always on the move between the US and India. My VM is in us-central, which is great when I’m stateside, but a total nightmare for downloads when I’m in India. Even with blazing fast internet, I’m crawling at snail speeds.
I figured I could work around this by:
Pushing my files to my Google Drive
Grabbing them from there since Drive downloads are usually super quick
Anyone know how to get those files from my VM to Drive? I’m totally stumped on the first part. Help a researcher out!
I’ve encountered similar challenges with data transfers across regions. One effective solution I’ve implemented is using the ‘gsutil’ command-line tool, which is part of the Google Cloud SDK. It’s designed for efficient transfers between GCP services and Google Drive.
First, ensure you have the necessary permissions set up. Then, you can use a command like:
The ‘-m’ flag enables multi-threading for faster transfers. Once your files are in Cloud Storage, you can easily access them from Google Drive using the ‘Google Drive for desktop’ application. This method has significantly improved my workflow, reducing transfer times and simplifying access across different locations.
yo, been there done that! i use rclone, its super easy. just install it on ur VM, configure it for gdrive (google how), then run ‘rclone copy /path/to/files gdrive:destination’. works like a charm for me. give it a shot!
Hey there, fellow researcher! I’ve been in your shoes, juggling data between different continents. Here’s what worked for me:
I used the ‘gcloud’ command-line tool to transfer files from my VM to Google Cloud Storage first. Then, I set up a Google Cloud Function that automatically moves files from GCS to my Drive. It’s a bit of a workaround, but it’s been reliable.
For the GCS step, something like ‘gsutil cp /path/to/your/file gs://your-bucket/’ does the trick. Then the Cloud Function handles the rest. It took some initial setup, but now it’s hands-off.
This method’s been a lifesaver for me, especially when I’m globe-trotting. Hope it helps you too! Let me know if you need more specifics on the setup.