I also had similar concerns but the point is one has to be very careful while handling the CC.
I use HSBC CC extensively and they have introduced a concept of entering a password which is generated in the security device sent to the card holder. It gives a 6 digit number which refreshes every 15 seconds. Now if one loses that also, then I think that person should not keep a CC in the first place