Socket Error 98 in Django Scheduled Task with Spotify Web API Integration

Other Questions Spotify
1

I’m having trouble with a Django scheduled task that connects to Spotify’s Web API. The task runs every few minutes using django-crontab to manage music playlists, but I keep getting socket errors.

The Issue:
My scheduled task fails with this error message:

ERROR 2024-09-06 01:11:02,768 cron Failed to update playlist: [Errno 98] Address already in use

What I’ve Done So Far:

  • Tested the Spotify API calls manually and they work fine
  • Added retry logic for common API errors like rate limits
  • Checked running processes with lsof command
  • Made sure only one cron instance runs at a time

My Code:

def update_playlist_tracks(task, playlist_id, song_uris, spotify_client, account):
    max_attempts = 3
    wait_time = 5
    USER_NAME = account.spotify_credentials.user_id

    logger.info(f"Updating playlist {playlist_id} for user {USER_NAME}")

    for try_count in range(max_attempts):
        try:
            logger.debug(f"Adding {len(song_uris)} songs to playlist, try {try_count + 1}")
            spotify_client.user_playlist_add_tracks(USER_NAME, playlist_id, song_uris)
            logger.info(f"Successfully updated playlist {playlist_id} with {len(song_uris)} tracks")

            mark_task_complete(task)
            return

        except SpotifyException as spotify_error:
            logger.error(f"Spotify error: {spotify_error.msg}")
            
            if spotify_error.http_status == 403:
                logger.error(f"Permission denied for user {USER_NAME}")
                return

            if spotify_error.http_status == 429:
                wait_seconds = int(spotify_error.headers.get('Retry-After', wait_time))
                logger.info(f"Rate limited, waiting {wait_seconds} seconds")
                time.sleep(wait_seconds)

        except OSError as os_error:
            if os_error.errno == 98:
                logger.error(f"Failed to update playlist: Address already in use. Waiting {wait_time} seconds.")
                time.sleep(wait_time)
            else:
                logger.error(f"System Error: {os_error}")
                time.sleep(wait_time)

        except Exception as general_error:
            logger.error(f"Failed to update playlist: {general_error}")
            time.sleep(wait_time)

    logger.error(f"Could not update playlist {playlist_id} after {max_attempts} tries")

Cron Configuration:

CRONJOBS = [
    ('*/5 * * * *', 'myapp.tasks.cron.run_playlist_update'),
]
CRONTAB_LOCK_JOBS = True

System Info:

  • Ubuntu 20.04
  • Django (latest version)
  • Python 3.12
  • Using spotipy library for Spotify API

The error happens right when trying to call the Spotify API function. Everything works fine until that point. I can see from the logs that the authentication and setup work correctly.

Questions:

  • Why am I getting this socket error when calling the Spotify API from a cron job?
  • How can I fix this network issue in my scheduled task?
  • Are there better ways to handle API calls in Django cron jobs?
2

Socket error 98 in cron jobs usually happens because Django doesn’t close database connections properly in scheduled tasks. Your cron job leaves connections open from previous runs, which messes with HTTP connections to Spotify’s API. Add django.db.close_old_connections() at the start of your cron function before any API calls. I’ve seen this fix the exact same problem - code works fine in Django views but breaks in cron jobs. Cron doesn’t have the cleanup mechanisms that web servers do, so you’ve got to handle connection cleanup yourself. Also wrap your whole cron function in a try-finally block so connections get closed even when things go wrong.

3

I’ve hit the same issues building automated music tools with Spotify’s API. Your cron environment probably handles network connections differently than when you test manually. Set an explicit session timeout in your spotipy client - try spotipy.Spotify(requests_timeout=30, retries=1). Also check if you’re running multiple Django processes that might be conflicting. I’ve found adding a small random delay before the API call prevents race conditions when multiple scheduled tasks kick off at once. That socket error usually fixes itself if you give the system time to release the connection.

4

errno 98 means you’ve got a connection already open to that endpoint. Add connection pooling or make sure you’re closing HTTP connections after each Spotify call. Also check if you’re reusing your spotipy client properly - you might be creating multiple instances that aren’t cleaning up in the cron environment.