I don't think there was the feature to run only specific GEMs.

For your reference, the feature to disable testing of specific GEMs was added in mruby-3.3.0.
https://github.com/mruby/mruby/blob/3.3.0/doc/guides/mrbgems.md#gem-testing
It's tedious, but you should be able to get close to what you want by putting all your dependencies GEMs one by one in your build configuration file.
Still, it is inevitable that the mruby core tests will always be run.
If that is really necessary, mrbgems/mruby-test/driver.c needs to be extended.