A common tool for determining the cause of lag and slowdowns of Minecraft servers at the moment is Timings, built into Spigot. However, these reports can be deceiving unless you know what to look for, and they’re not always able to actually tell you the source of lag.
View Modes
One crucial thing to firstly note is that Timings has two different view modes, seen in the below image. By default, Timings shows the "Lag" view, which is only useful and accurate under extreme amounts of server lag. It's recommended to switch the mode to "All", to get a more accurate representation of what is slowest on your server. Even under extreme lag, the “All” view can be a good idea.
Common Problems
Many people look at a Timings report and blindly blame it on whichever shows up first. This is very incorrect for multiple reasons. Firstly, Timings cannot accurately determine which plugin causes slowdowns (a point I'll come back to later). Secondly, a plugin may be slow in response to another problem with the server. If you're experiencing multiple plugins being listed as a cause of lag in Timings, you likely have another issue with the server, causing plugins to perform slowly.
This Timings report is a perfect example of another issue with the server showing up as a plugin problem. At first glance, many would assume that WorldGuard is causing the server to behave slowly here. This is not the case at all. If you look closely, you'll notice that a massive number of physics events are being passed to WorldGuard per tick. This indicates that the server is being held back by block updates, generally caused by giant redstone contraptions or malicious lag machines created by players. Recently hoppers and the “InventoryMoveItemEvent” have been a frequent culprit of this too, where giant hopper contraptions or malicious lag machines are throwing millions of very costly events at plugins, making it look like plugins are the source of the issue.
Removing WorldGuard here is not the solution, as it will not fix the problem and will show up in Timings for any plugin that listens to physics events, or even be mysteriously absent if nothing does. The real solution is to find and remove the lag machine.
How does Timings work?
To talk about the next common problem people encounter when using Timings, I need to briefly explain how Timings works.
Timings is what is known as a sampler. This means throughout the server code, multiple "sample" points have been added. Timings can then extrapolate that everything occurring inside a specific sample section is caused by a particular task. This means everything that occurs in the code section where chunk loading occurs is marked as chunk loading, and everything that happens in the event handler for CraftBook's redstone event listener is marked as CraftBook.
This sounds fine at first glance, however, what if a plugin toggles a lever that then triggers a large redstone contraption? This is where inaccuracies start to occur within Timings. Due to plugins being complicated, and Minecraft having so many "side effects" of every action, it would be impossible for something that works like Timings to provide perfect accuracy. Timings can blame a vast majority of the servers tick processing on a single plugin in some cases. For example, if a CraftBook Integrated Circuit triggers a large redstone contraption that loads chunks and moves around items in hoppers, CraftBook is blamed for all of that. This also isn't something that happens rarely, this is one of the most common reasons for some plugins to show up on Timings.
The Solution
Like most things, there isn't one single perfect solution. Understanding what is happening on your server and knowing the available tools will always be the best way to fix performance problems.
WarmRoast
A tool similar to Timings that provides significantly higher accuracy is WarmRoast. This is a CPU Sampler explicitly built for Minecraft, by the original creator of WorldEdit, WorldGuard, and CraftBook. Running WarmRoast itself can appear to be significantly more complex than running Timings, so WorldGuard bundles it in an easy command. Another developer, Luck, has also made a standalone WarmRoast plugin called Spark. This does work the same way as WorldGuard's bundled WarmRoast so will not obtain different results, either can be used the same way. If you’re using Paper, Spark is included by default and doesn’t need to be installed separately.
To use WorldGuard's bundled WarmRoast, run /wg profile -p when you're noticing the lag. Unlike Timings, this runs for a short period (5 minutes by default). You can pass the command the number of minutes to run for if you want to change this, by running /wg profile -p 10 for 10 minutes. It's essential to have this running while lag occurs, as it can't determine the cause of lag that's not currently happening. As this is a more intense process, it can't efficiently run in the background like Timings, meaning it can only measure performance from when the command is used.
This produces an output similar to Timings, but with much finer granularity. From here, you can determine what function in the code running on your server is causing slowdowns. This profile is more complicated to read than Timings but provides significantly further detail and accuracy. You can also use this to find other problems easier, such as finding out that lag is being caused by zombies targetting villagers. This probably means a mob farm of some description is the issue.
Conclusion
Overall, the correct way to solve performance problems is to understand what a Minecraft server is doing and the tools available to you to solve the problem.
If you're encountering lag, a good first step is to generate a Timings report. As Timings is always running, this is a natural step to rule out the "low hanging fruit". If Timings shows a considerable number of specific events or that chunk generation is the cause, you probably don't need to investigate further. You can find the problem contraption or pre-generate your world. If however, it's inconclusive, WarmRoast can provide further information to identify what the problem is.
It's good to make a well-informed judgment regarding performance, rather than just deleting plugins that Timings points to. Removing plugins is almost always a red herring, and in some cases only masks the issue for it to come back worse in the future.
If all else fails, plugin developers will be able to do a lot more with a WarmRoast profile than they will with a Timings report. If it is a plugin issue, the developer will have a better idea of the exact problem, allowing for a much simpler fix. If in doubt, always ask the plugin developer.
This article is part of a series on how to run a Minecraft server. Click here to check out more!
About the Author
Hi, I'm Maddy Miller, a Senior Software Engineer at Clipchamp at Microsoft. In my spare time I love writing articles, and I also develop the Minecraft mods WorldEdit, WorldGuard, and CraftBook. My opinions are my own and do not represent those of my employer in any capacity. Find out more.