"Manage Site Content and Structure" in SharePoint Online had gone. So, we cannot manually delete the items easily, anymore.
If there is less than 5000 items in the list, things are much easier: we can save the list template, then delete the list, and rebuild it through the list template. But, if there are more than 10,000 items, it's time consuming.
Here is a simple solution: Using PowerShell to load all items into an array variable, then delete the items one by one, and then commit the requests in batch.
During my test, to delete 48622 items from one list, it took around 6300 seconds. So it's around 7.7 items per second.
If you know any way to improve the performance, please let me know.