Occasional use of Epsom salts (especially on the feet) does not seem to be harmful. Many people swear by Epsom salts, though there doesn't seem to be many scientific references to its use. There are a few cautions of using Epsom salts, however, and they are mostly for people with dry skin.
The FDA doesn't seem very impressed with Epsom salts, even for foot care. They claim that Epsom salt foot baths show no benefit over soaking in a regular table salt bath in terms of softening the foot. They further caution that too much soaking in Epsom salt can cause excessive drying of the feet. People with diabetes and others with fragile skin are advised to soak in a foot bath with liquid dishwashing soap (the kind with skin softeners) instead.
The National Institute of Arthritis and Musculoskeletal and Skin Diseases (part of the NIH) does mention that an Epsom salt bath can help remove or soften any scaling that occurs with psoriasis.
In short, when using Epsom salts, be very careful about drying out the skin. Start with just a little (1/4 cup) in the bath, and gradually increase as needed. Monitor you skin closely for dryness. Try an Epsom salt bath once a week at first to see how your skin reacts. It may be that you need to avoid Epsom salt baths in the wintertime (when both the air and your skin tend to be drier), but can take them more frequently in the summer if you live in a humid location.