I’m working on a Python Discord bot that needs to collect channel statistics. Specifically, I want to track how many messages each user has posted in a specific channel. Here’s my current implementation:
if message.content.startswith('!count'):
status_msg = await client.send_message(message.channel, 'Processing...')
message_count = 0
async for log_entry in client.logs_from(message.channel, limit=999999):
if log_entry.author == message.author:
message_count += 1
await client.edit_message(status_msg, '{} has posted {} messages in {}.'.format(message.author, str(message_count), message.channel))
While this approach works correctly, it takes way too long to process all the messages. The bot becomes unresponsive during calculation. Are there any optimization techniques or alternative approaches that could speed up this message counting process significantly?
Your bottleneck is that massive limit parameter and synchronous processing. I hit this same issue building moderation tools for my server. Quick fix: drop the limit and use pagination - process messages in chunks of 1000-5000 with small delays between batches (asyncio.sleep(0.1)) so you don’t block everything. You can also cache results with dictionaries or Redis that expire after a set time, so you’re not rescanning for repeat requests. For the heavy processing, throw it in a separate thread with asyncio.run_in_executor() to keep your bot responsive while it counts. This combo gave me around 70% better performance without rebuilding everything.
Yes, the synchronous message iteration is causing significant slowdowns. I faced similar issues while developing analytics for a large server and shifted to a database approach, reducing processing time by 95%. Instead of scanning history each time, implement a message listener that updates counters in real-time using on_message events. You could use SQLite or another database for storage. For existing messages, consider running a one-time batch job during off-peak hours to establish initial counts. This will make your !count command instantaneous, as it will only perform a quick database query rather than processing thousands of messages.
you could also use the newer discord.py version with message intents and track counts as they happen. just store everything in a json file or dict that survives restarts. way faster than scanning through chat history every time someone runs the command.