I’m working on a Discord bot using discord.py and I’m stuck with an async problem.
My bot needs to check a website every hour using selenium to gather data. I put this in a separate thread so it doesn’t freeze the bot while scraping.
@tasks.loop(hours=1)
async def check_site_data(self):
worker = threading.Thread(target=self.scrape_data_function)
worker.start()
Inside my scraping function:
def scrape_data_function(self):
# selenium stuff here
target_channel.send(f"```{gathered_data}```")
The issue is that target_channel.send() requires await but I can’t use await in a regular function. However, I can’t make scrape_data_function async either because threads don’t work with async functions.
I looked into asyncio but couldn’t figure it out. What’s the right way to handle this?
Had this exact issue last year. The problem is you’re mixing sync selenium with discord.py’s async stuff. I kept the threading but used bot.loop.call_soon_threadsafe() to push the message sending back to the main thread. In your scraping function, don’t call send directly - do bot.loop.call_soon_threadsafe(asyncio.create_task, target_channel.send(f"```{gathered_data}```")). This queues the async operation on the bot’s event loop from your worker thread. You’ll need to pass the bot instance to your scraping function for loop access. Keeps selenium in its own thread while handling Discord API calls properly.
use asyncio.run_coroutine_threadsafe() for this. in your scrape function, add asyncio.run_coroutine_threadsafe(target_channel.send(f"```{gathered_data}```"), bot.loop) where bot.loop is your bot’s event loop. this lets you safely call async functions from regular threads.
You don’t need threading for this. @tasks.loop already runs in the background without blocking your bot. Just make your scraping function async and call it directly from the loop. Change it to async def scrape_data_function(self): then use await self.scrape_data_function() in your task loop. Ditch the threading completely. I made this same mistake when starting out - tasks.loop handles background execution automatically. If selenium’s still too slow, try aiohttp for web requests, but for hourly checks selenium won’t cause any noticeable lag.
This topic was automatically closed 4 days after the last reply. New replies are no longer allowed.