It requires context though for people to know you're referring to time. The same with other SI units. They're used as units of measurement but once the context has been established. If you write 5m that could refer to 5 meters (as the SI unit) but could also in another context be 5 minutes or 5 million. 4K is 4 Kelvins as an SI unit, but could also mean 4 thousand in another context. cd is candela as an SI unit but could also mean compact disc.