Icon for JAPI.tools project

JAPI.tools

A collection of tools using JAPI.rest.

WebsiteStatisticsAnalyticsTrackerScraperToolsDiscord Bot

Like most of my projects, this one started with a problem I wanted to solve. I wanted a way to track how many Discord servers were using my bot over time, rather than just knowing the current count. What started as a simple idea eventually grew into something much bigger.

History

Back in 2021, there was a website called OverWatch that tracked Discord users and the servers they were in using Discord self-bots.

In May of 2022, the site pivoted to tracking Discord bots instead of users. Instead of relying on self-bots connected to Discord's (websocket) gateway, it used the Discord API to retrieve estimated server counts for bots.

Archived screenshot of the website from the Wayback Machine

Later on September 9th, 2022, it was renamed to Discord.Watch, and it was a great tool to track your bot's statistics over time and compare against other bots. I used it for a while, and liked it.

But then the site was abruptly shutdown on August 8th, 2024. As I never tracked these stats myself and nor was anyone else (as far as I know), that meant that now all that historical data and statistics were no longer publicly accessible. I knew the owner of the site, and I asked them if I could buy it and take over and continue the project, but they wanted $1,000 for it, as they said it was getting around 30k visits per month. At the time I didn't have the money to buy it, so I kinda sat on the idea for a while.

On October 12th 2025, discord.watch came back online under the same domain but now owned and operated by Snazzah. I was happy to see the site back online, but unfortunately, Snazzah didn't have access to the historical data from 2021-2024. I reached out to the previous owner if they still had a copy and they said they did but forgot the password.

Snazzah also had a different approach to collecting bots statistics, as they didn't want to rely on Discord's undocumented APIs. He initially only tracked bots that were listed on the Discord App Directory.

At the time, that meant they were only tracking approximately 17,000 bots, which is a tiny fraction of the bots that are actually on Discord.

The Project

On October 30th 2025, I decided to make my own version of the site, but with a key difference, I would track as many publicly discoverable bots, rather than limiting myself to those on the Discord App Directory.

I started by building it as a upgrade of my JAPI Rest project, as I had already been retrieving bot data through Discord's API, but I wasn't storing any of it. This gave me an opportunity to build something that could collect that data and preserve historical bot server statistics over time.

The current version stores the generic bot data (like name, description, prefix, etc.), along with the latest approximate bot server count in a PostgreSQL database, and the historical approximate bot server statistics are stored separately in Clickhouse.

The data collection system relies on undocumented Discord API endpoints to retrieve information that isn't otherwise publicly available through the documented API.

Initially, the project was just a Hono API that returned JSON responses. I later built a Next.js frontend to make the data easier to explore.

My original frontend was heavily based on the new discord.watch design. I liked how their site looked and wanted something similar, although I also added a few features that weren't available there at the time, such as searching for bots by app or bot ID, and the ability to request a bot to be added to the database if it wasn't already there.

Many of those features have since been added to discord.watch as well, which has been great to see.

Working with discord.watch

On December 12, 2025, I shared the project in a few Discord servers for bot developers, where it received some positive feedback.

One of the people who took an interest was Snazzah, the new owner of discord.watch. He asked whether he could import my project's data into discord.watch, and I was happy to help.

The import added approximately 50,000 bots to discord.watch's database, a substantial increase from the roughly 17,000 bots it had previously been tracking.

I also shared how I was retrieving bot data through Discord's undocumented API endpoints. Snazzah was interested in that approach and ended up implementing it into discord.watch as well.

As a result, both projects can now collect statistics for a much wider range of Discord bots, rather than being limited to those listed in the App Directory.

Not long after that, I showed the project to Ash. He wasn't a fan of my original frontend design and offered to redesign the site for me. The new design was a huge improvement over what I had originally built.

Challenges

Rate Limits

Everything was going smoothly and I was able to track around 100,000 bots in just eight hours every day.

However, sometime on August 13th 2026, Discord tightened the rate limits, and that made it where it was impossible to scrape and track a hundred thousand bots in a single day.

I needed a different approach. Otherwise, the reduced request capacity would mean that some bots might only receive updated statistics once a month, which was not ideal. I ended up implementing a sorted queue system, where after every batch of 25 bots, it would sort the remaining bots by when they were last fetched (and prioritizing app directory bots), and then continue scraping the next batch of 25 bots. This way, I could still scrape all known bots, but it would take longer to get updated data for bots that were not on the Discord App Directory.

For a brief period, I experimented with a different approach. Instead of prioritizing bots by their last update time, I sorted them by their most recently recorded server count.

My reasoning was that bots with larger server counts were more likely to be viewed by users, so it made sense to update them more frequently.

However, I quickly realized this approach had a significant flaw. The same popular bots would be fetched repeatedly, while smaller bots would rarely, if ever, receive updates. This also meant that a smaller bot experiencing rapid growth might go unnoticed because its previous server count was too low to prioritize.

I eventually switched back to prioritizing bots by their last update time. While this means some bots might take a month or two to receive fresh statistics, it ensures that every bot in the database eventually gets updated.

Finding Bots

One of the project's biggest ongoing challenges is discovering new bots. Discord doesn't provide a public API endpoint for retrieving a complete list of bots, so I have to discover them through other sources.

My initial approach was to collect bot IDs from Discord.Watch, which indexed bots from the Discord App Directory, as well as several bot listing websites. These included top.gg, discord.bots.gg, and my own bot list, Void Bots, alongside a few other sources.

Another way the project discovers new bots is through users visiting the site.

Whenever someone tries to open a bot's page on the site, the API checks if the bot is already in the database, and if it is not, the API will query Discord for the bot's information. If Discord returns valid bot data, it will be added to the database, otherwise it just returns a not-found error. This makes it possible to discover and add bots that users are actively looking for, without having to manually add each one myself.

Expansion

Eventually, I wanted the site to offer more than just bot statistics.

I had several small Discord-related tools that I wanted to use myself, so rather than building separate websites for each one, I decided to add them to the existing project.

Some of the tools I've added include a Discord Permissions Bitmask Calculator, a Discord Snowflake Decoder, a Discord Webhook URL Checker, and a few image generation tools for creating snowmen, terminal-style images, and text using a Thonk emoji font.

This expansion is what turned the original bot statistics tracker into Japi Tools, a collection of Discord-related utilities alongside the bot tracking functionality.

I plan to continue adding more tools and features as I come up with ideas for things I'd find useful.

Technical Details

For anyone interested in how the project works behind the scenes, I designed it around a microservice architecture, with each service handling a specific responsibility independently.

There are five main services:

  1. The API, which is responsible for serving bot data to the frontend and other api users. Made using Hono, Typescript, and Node.js.
  2. The frontend, which is Responsible for displaying bot statistics and providing the user interface for the API. Made using Next.js, Typescript, Tailwind CSS, and Shadcn UI.
  3. The "fetcher" service, which is responsible for retrieving bot data from Discord API. It processes entries from the paired_ids table, adds new bots to the bots table, and updates existing bot information, including estimated server counts. It also stores historical server statistics in ClickHouse.
  4. The "radar" service, which is responsible for discovering new bots from external sources, such as bot listing websites, and adding their IDs to the pending_ids table for further processing.
  5. The "resolver" service, which is a responsible for processing entries from the pending_ids table and matching Discord bot IDs with their corresponding application IDs, and vice versa. This service exists because, before 2017, Discord used separate IDs for bots and their applications. Once resolved, the ID pairs are stored in the paired_ids table, where they can be picked up by the Fetcher service.

Together, these services form a pipeline: Radar discovers potential bots, Resolver matches their IDs, Fetcher retrieves and stores their information, and the API makes that data available to the frontend and other clients.

PostgreSQL handles the main bot database and the tables used to coordinate these services, while Clickhouse stores the historical server counts used to display statistics over time.

There is also a small Redis instance used for caching and rate limiting, and Meilisearch is used to provide search functionality for the frontend.

Future Plans

There are still a few things I'd like to add to the project:

  • Tracking user app installations. Discord supports installing apps directly to user accounts rather than just servers, but I haven't found a publicly accessible API endpoint that exposes these installation counts. I have a few ideas for how I might be able to collect this data, but nothing concrete yet.
  • Add more tools to the site (like the ones I already have, and more).

What started as a way to track my own bot's server count has grown into a much larger project. Along the way, it has also helped expand the number of bots tracked by discord.watch, which wasn't something I expected when I first started building it.

From the feedback I've received, most people familiar with both sites seem to prefer Japi.Tools, mainly because it provides more up-to-date statistics and bot information. It's been great to see people finding the project useful beyond what I originally built it for.

If you have any suggestions for new tools or features, or just want to share your thoughts on the project, feel free to reach out to me on Discord.

Similar projects

View all projects ↗