I find that hardly surprising. There aren't many lemmy instances out there yet, it is relatively hard to discover groups hosted elsewhere, and the whole UX lends itself towards a "many groups on one server, why venture elsewhere?" approach. Naturally users cluster to the(ir) "main" instance. From a resilience/federation perspective that is a Bad Thing^TM^, just like it is with Mastodon (that monopolized the microblogging federation space so much, it has become synonymous with it in many peoples minds).
Here's to hoping other lemmy-compatible^1^ servers will be written.
^1^ Let's not get into a discussion of how all ActivityPub implementations are compatible, shall we?