Another cause of your issue could well be worn contacts inside the starter motor. These would exhibit the same fault as you describe. One tell tale way to know if it is them is when it is playing up to carefully listen when you turn the key for a relay clicking. If you hear the click, this would suggest that the key part is working and energising the starter correctly, hence an issue with the starter. Do a search on google for "worn toyota starter contacts". There's plenty of info about - even videos on youtube. A loose/sloppy key should not cause the issue you are experiencing. If you are able to turn the key, then that should be enough to activate the switch behind the cylinder. The switch could be faulty, there are stories on here of that. However, I would expect it to exhibit other symptoms such as radio not coming on when at ACC and dash not lighting up when turned to ON. It's quite easy to remove the ignition barrel on these cars. You have to remove the lower dash trim to expose the barrel from underneath. Insert the key and turn to ACC. You then depress a small button at the bottom of the ignition barrel housing and pull the cylinder out with the key. To replace, simply reinsert all the way, turn the key to "OFF" and remove the key. Doing this, you can inspect the back of the cylinder and switch for any signs of wear. Again, plenty of videos on youtube on how to do this.