I’m working with the python-telegram-bot library and running into a threading issue. The framework uses asyncio but everything runs on the main thread.
My problem is that I need to execute an async function on a different thread. I have some heavy data processing and API calls that take several minutes to complete. When these operations run, my entire bot becomes unresponsive and won’t handle any other user requests.
I found a potential solution using asyncio.run_coroutine_threadsafe(coro, loop) but I’m not sure how to implement it properly. Can anyone help me figure out the correct approach?
Here’s what my current code looks like:
async def DataProcessor(context: ContextTypes.DEFAULT_TYPE) -> None:
# Heavy computations and API requests here
# Takes several minutes to complete
await context.bot.send_message(
chat_id=context.job.chat_id,
text=processed_result
)
async def start_processing(update: Update, context: ContextTypes.DEFAULT_TYPE):
task = context.job_queue.run_repeating(DataProcessor, 600, first=10, chat_id=user_id, name="data_processor")
I’ve been fighting this same problem for months with my trading bot. Here’s the thing - run_repeating doesn’t spawn separate threads. It just schedules everything on the same event loop.
What actually worked for me: use asyncio.create_task() with asyncio.gather() for the heavy stuff. You stay in the asyncio framework but get concurrent execution. Throw in a semaphore if you need to cap how many heavy tasks run at once.
Another trick that saved my ass - chop up your data processing into smaller async chunks. Drop in await asyncio.sleep(0.1) calls periodically. This hands control back to the event loop so other handlers can process incoming messages. Not as clean as real threading, but way more reliable.
try using concurrent.futures.ThreadPoolExecutor with asyncio. just put your heavy tasks into a normal func and then use loop.run_in_executor(executor, func) to keep your bot responsive while the work goes on in the bg.
Had the exact same problem building a bot for large datasets. job_queue doesn’t actually help here - it still blocks the main event loop despite looking like it runs separately.
I fixed it by spinning up a dedicated thread with its own asyncio event loop for the heavy stuff. Use threading.Thread to create the thread, then asyncio.new_event_loop() inside it. Just make sure you’re using thread-safe queues or callbacks to talk back to the main bot thread.
Alternatively, chunk your heavy processing and throw in await asyncio.sleep(0) between operations. Lets the event loop handle other requests while working. Not as clean as proper threading but way easier to implement and debug.