I’m not aware of any other issues with the power switch or USB connectivity. What operating system are you using? And did you buy the Starter Kit that comes with it’s own microUSB cable, or are you using your own cable? I’ve had connection issues with microUSB jack/cable combinations on other devices, so maybe try a different cable and see if that helps. Do you have an FTDI cable or adapter board that you could use to power and program the Mayfly to see if touching the power switch has the same effect?
Another thing to troubleshoot whether it’s communication or power related would be to put a sketch on the board that lights both LEDs constantly, then disconnect the USB or FTDI cable, and plug in a battery to the LiPo jack. Then you can turn the switch on and keep an eye on the LEDs while touching the switch to see if perhaps there’s a loose connection in the switch.