User registration and signup using private and public key

I am trying to create a react native app where users can log in without using passwords or emails. The idea is that at registration, we capture users cellphone number, verify it using OTP and once verified we create a RSA public/private key pair. The private key stays on the users phone while the public key is sent to the backend (using HTTPS API). All content going forward, is encrypted using private/public keys.

To log in and identify the user in future, we basically use the cellphone number + public key.

My question is purely from a security perspective - is the above method safe in absence of a regular username/password type scenario? Are there any drawbacks to using this approach?

Are there any frameworks/apps that already use this approach?