First I tried to use normal PHP arrays (using integer as the key and amount of entries as the value), but I was not happy about performance, so I decided to look for more specialized data types. Though using them improved memory usage a little bit, but did not improve timings.

Also I decided to use pagination because the amount of duplicates is huge and browser was not happy about it

The better solution would be to use queries and chunk by chunk process the information on background

---

Kind of surprisingly, but on linux server the performance became much better