Monitoring file transfer progress with Google Drive API in Java/Android applications

Other Questions Google drive
1

I’m working on a Java Android app that uploads and downloads files using Google Drive API. I need help with tracking the progress of these operations.

For uploads: When I use driveService.files().create(fileMetadata, content).execute(), it only returns the final file object after completion. I want to show real-time progress updates to users, preferably every second or two. Is there a way to monitor upload progress during the operation?

For downloads: My current approach is to manually track bytes read from the InputStream compared to total file size to calculate percentage. Is this the recommended method or are there better alternatives?

Here’s my current upload implementation:

private void uploadDocumentToDrive() {
    Thread uploadThread = new Thread(new Runnable() {
        @Override
        public void run() {
            try {
                // Document content
                java.io.File documentFile = new java.io.File(documentPath.getPath());
                FileContent docContent = new FileContent("application/pdf", documentFile);

                // Document metadata
                File metadata = new File();
                metadata.setName(documentFile.getName());
                metadata.setMimeType("application/pdf");

                File uploadedDoc = driveService.files().create(metadata, docContent).execute();
                if (uploadedDoc != null) {
                    displayMessage("Document uploaded: " + uploadedDoc.getName());
                }
            } catch (UserRecoverableAuthIOException authException) {
                startActivityForResult(authException.getIntent(), AUTH_REQUEST_CODE);
            } catch (IOException ioException) {
                ioException.printStackTrace();
            }
        }
    });
    uploadThread.start();
}

Update: I tried resumable uploads with progress tracking:

Drive.Files.Create createRequest = driveService.files().create(metadata, docContent);
MediaHttpUploader mediaUploader = createRequest.getMediaHttpUploader();
mediaUploader.setDirectUploadEnabled(false);
mediaUploader.setChunkSize(5*1024*1024);
mediaUploader.setProgressListener(new DocumentUploadProgressListener());
com.google.api.services.drive.model.File result = createRequest.execute();

However, I’m facing two issues:

  1. Resumable uploads are significantly slower than direct uploads due to chunking
  2. With smaller chunk sizes (1MB), uploads fail around 70% completion with “unexpected end of stream” errors

Any suggestions for reliable progress tracking without sacrificing upload speed?

2

Yeah, this chunk size issue is super common with resumable uploads. I’ve found 8MB chunks work best - big enough for decent speed but won’t timeout on you. Those “unexpected end of stream” errors happen when your connection drops mid-transfer. Smaller chunks actually make this worse since you’re doing more round trips. For download progress tracking, manually tracking bytes from InputStream is pretty much the standard way. I usually wrap the InputStream in a custom implementation that fires progress callbacks as data flows through. Just watch out for the content-length header - some files don’t have it set right. One thing that really helped my upload reliability was adding proper retry logic. When resumable uploads fail, you can check the upload status and pick up where you left off instead of starting over. MediaHttpUploader has built-in retries but sometimes you need custom handling for network drops.

3

Google Drive API progress tracking is a nightmare. I wasted hours on those same chunking issues.

Skip the resumable upload mess entirely - I switched to Latenode for file operations. You get clean progress callbacks without fighting chunk sizes and stream errors.

Set up a Latenode workflow to handle Drive uploads and push progress updates to your Android app via webhooks or polling. No more MediaHttpUploader timeouts or hunting for that magical chunk size.

Same for downloads. Stop manually tracking InputStream bytes (breaks when content-length is wrong). Let Latenode do the work, you just grab the progress data.

I use this for all file sync now. Way more reliable than babysitting Google’s API, and progress tracking actually works without killing performance.

4

I’ve been fighting this same issue for months in production. Chunking that everyone suggests sucks on mobile - connections drop constantly. Here’s what actually worked for me: ditch Google’s MediaHttpUploader and use OkHttp interceptors instead. Build an interceptor that wraps your request body and tracks bytes as they’re written during the HTTP call. You get real progress callbacks without chunked uploads, so transfers stay fast. The trick is monitoring at the HTTP layer, not the Drive API layer. For stream errors, use exponential backoff with jitter - completely fixed my issues. Your manual byte tracking for downloads is spot on, just buffer your reads properly so you don’t block the UI thread. Pro tip: progress updates can kill performance if you fire them too often. Most devs miss this.

5

use InputStreamContent instead of FileContent - you’ll get way more control. Just wrap your file stream with a custom one that tracks bytes and fires progress events. Much easier than dealing with mediaHttpUploader’s chunking mess. For timeouts, bump up your read timeouts on the http transport - mobile connections suck. I go with 60 seconds minimum.

6

Had this exact issue in my enterprise Android app last year. Your MediaHttpUploader approach is solid - those stream errors are usually network problems, not chunk size issues. Skip messing with chunk sizes. Build a custom FileContent wrapper extending AbstractInputStreamContent. Override writeTo() to intercept the stream and track progress yourself. You get fast direct uploads plus progress tracking. For stream errors, add connection pooling and exponential backoff retries. These errors hit way more on mobile vs WiFi. Custom HttpRequestInitializer with longer timeouts fixed most of mine. Your manual byte tracking for downloads is actually standard practice. Just handle cases where servers don’t send content-length headers. I use a ProgressInputStream class that fires callbacks every few KB instead of per byte - stops the UI thread from getting hammered.

7

The Problem: You’re experiencing unreliable upload progress tracking and slow speeds with Google Drive API resumable uploads in your Java Android app. Resumable uploads, while offering robustness, are causing “unexpected end of stream” errors around 70% completion with smaller chunk sizes, and are slower than direct uploads. You want reliable progress tracking without sacrificing upload speed.

:thinking: Understanding the “Why” (The Root Cause):

The challenges you’re facing stem from a combination of factors related to resumable uploads and network conditions on mobile devices. Resumable uploads rely on breaking the file into chunks, uploading each, and resuming from the point of failure if a connection interruption occurs. Smaller chunks increase the overhead of managing the upload process, leading to slower speeds and increased susceptibility to network issues. The “unexpected end of stream” error typically points to a connection problem during a chunk upload—the connection drops before the entire chunk is transmitted. On mobile networks, intermittent connectivity is common, exacerbating this problem. Furthermore, aggressively frequent progress updates can consume significant resources, potentially impacting overall upload speed and reliability.

:gear: Step-by-Step Guide:

  1. Migrate to a More Robust Upload Approach: Instead of wrestling with the complexities and limitations of Google Drive’s MediaHttpUploader and its inherent issues with chunking and network interruptions on mobile, consider a different strategy. A more reliable approach uses a higher-level HTTP client library like OkHttp that provides built-in features to handle network issues gracefully. This library offers more fine-grained control over the upload process, allowing you to manage retries, timeouts, and progress updates effectively. This approach involves directly handling the HTTP requests instead of relying on Google Drive’s upload mechanism, giving you a more deterministic process that’s better equipped to deal with mobile network conditions.

  2. Implement Custom Progress Tracking with OkHttp: Use OkHttp’s request body capabilities to create a custom RequestBody implementation that intercepts bytes written during the upload. This allows you to track the number of bytes uploaded and generate progress updates at intervals appropriate for your application (e.g., every few kilobytes instead of every byte to avoid performance issues). This allows for more precise progress monitoring without the drawbacks of MediaHttpUploader.

  3. Robust Error Handling and Retry Mechanism: Implement a retry mechanism in your OkHttp upload code to automatically handle transient network issues. Exponential backoff with jitter is recommended—this approach gradually increases the retry delay after each failure, providing sufficient time to recover from temporary interruptions. Connection pooling in OkHttp can also improve performance and resilience. Ensure that you handle timeouts appropriately, potentially by using exponential backoff as well. Avoid repeatedly starting the upload process from the beginning when errors occur; instead, pick up where the upload left off.

  4. Optimized Progress Update Frequency: Don’t flood your UI with excessive progress updates. While real-time updates are desirable, sending frequent updates (e.g., every second or two) can negatively impact the application’s performance and responsiveness, especially on mobile devices. Determine a reasonable update frequency (e.g., every few kilobytes or at specific intervals) that balances real-time feedback with performance considerations.

  5. Download Progress Tracking: Your current method of manually tracking bytes read from the InputStream is a standard and effective approach for download progress tracking. Ensure you handle cases where the server doesn’t provide the Content-Length header appropriately. Buffer your reads to prevent blocking the UI thread.

:mag: Common Pitfalls & What to Check Next:

  • Network Connectivity: Mobile networks are unreliable. Ensure that your app handles network errors robustly.
  • Timeout Configuration: Configure appropriate timeouts for your HTTP requests in OkHttp to prevent deadlocks. Use exponential backoff for retries.
  • Resource Management: Overly frequent progress updates can impact performance. Carefully consider the update rate.
  • Content-Length Header: The Content-Length header may be missing or inaccurate. Handle this gracefully in your download progress calculation.

:speech_balloon: Still running into issues? Share your (sanitized) config files, the exact command you ran, and any other relevant details. The community is here to help!

8

This topic was automatically closed 24 hours after the last reply. New replies are no longer allowed.