You are right - the security roles are only applicable on the 'Activity' super type.
If you want to hide for different roles, it will be a significant customisation effort since the add activity buttons appear in many places.
You can customise the Ribbon Commands where there are individual buttons that can be hidden based on security roles (ribbonworkbench.uservoice.com/.../76681-hide-a-standard-out-of-the-box-button-based-on-a)
The drop down ribbon button will have to be completely hidden and a custom dropdown created with buttons similar to the standard ones.
There are some buttons in the outlook add-in that will not be able to be hidden.
The exact steps are too complex to include here - and I would recommend not performing this customisation since the total cost of ownership is likely to out weigh the benefits.
hth