Authentication challenges with Google Drive API and service accounts

I’m struggling with Google Drive API authentication and I need some help figuring this out.

My situation: I have a large collection of files (around 35GB) stored in my personal Google Drive. I built a web application where users can log in and access specific folders and files from my Drive account. The idea is that only my server should directly connect to my Drive - users interact through the web app.

The problem: I looked into using service accounts for server-to-server authentication, which seemed perfect at first. However, I discovered that service accounts get their own separate Drive storage space that can’t be expanded. Since my files are bigger than the service account limit, I would need to share everything manually, which doesn’t work for my use case.

Alternative approach: I considered using regular OAuth to get access tokens for API calls, but these tokens expire and I can’t manually refresh them every time.

My questions:

  1. Can service account storage quotas be increased somehow? Or is there a way to make my regular account work like a service account?
  2. When setting up OAuth credentials, you get a client secret JSON file. I tested the sample code but still had to log in manually. What exactly does this JSON file do?
  3. If OAuth is my only option, can I automatically refresh access tokens without manual intervention each time?

I know OAuth typically requires user interaction, so I’m not sure if automated token refresh is even possible.

You’re hitting a common architectural problem. Service account quotas can’t be increased past standard limits, and they’re meant for app-specific data, not accessing personal content. You’re right that OAuth is the way to go. About that client secret JSON file - it’s your app’s credentials with Google, but users still need to give permission first. Think of it as your app’s ID card. Google uses it to verify requests, but you still need explicit consent to touch their Drive data. You can definitely automate token refresh without any manual work. After the initial OAuth flow, store the refresh token securely on your server. Access tokens expire after about an hour, but your app can automatically grab new ones using the refresh token. I’ve got this running in production where systems go months without any human intervention. One thing that bit me: refresh tokens can go stale if your app sits unused too long or if users change their passwords. Build error handling to catch failed refreshes and have a clean way to re-authorize when needed. Also refresh tokens before they expire to avoid any downtime.

Yeah, OAuth refresh tokens are defnnitely your best bet. Just watch out for Google revoking your refresh token - I’ve seen this with suspicious activity or long inactive periods. Keep your auth flow ready so users can reauthorize if needed.

The client secret JSON you’ve mentioned grants access to your OAuth credentials, which indeed necessitates a manual authorization step initially. However, once you’ve completed this setup, you can automate the refresh of your access tokens. First, configure OAuth with your personal account and perform the manual login one time to retrieve your refresh token, which should be stored securely on your server. This refresh token remains valid unless revoked, enabling you to programmatically obtain new access tokens as needed. As for service accounts, you’re correct that there’s a storage limit, and they are not designed for managing extensive personal Drive content directly. In your case, relying on OAuth and securely managing the refresh tokens is advisable. Be mindful of handling token refresh failures, as these tokens can be revoked and may require you to reauthorize later. My setup using this approach has been running smoothly for over two years with minimal intervention.