Pattaya = 75% Go-go and 25% beach
Phuket = 75% beach and 25% go-go
Depends on what you seek.
I've visited Pattaya but been living in Phuket for 8 years now. It is a great place for families. I agree to steer clear of Patong. Karon Beach is a pretty good option because there are good family-friendly hotels and activities available there... and if you do need to get a 'Patong fix', it is only 10-15 minutes taxi ride away.
Patong is not ALL sleaze. There's a good shopping centre there with plenty of restaurants, shops, cinema etc... but it depends what you want out of your holiday. Karon Beach is the second longest stretch of beach in Phuket and is a much quieter place to be - a lot less girly bars!
From Karon you can go elephant trekking close-by, take a snorkel tour... and SUB AQUA Dive Center on Karon Beach road offer free scuba trials (using the pool at Hilton Phuket Arcadia Resort and Spa) for the whole family. It is a clean beach resort/area without being too commercialised, but also not too quiet.
Phuket and Pattaya same style but my vote for Phuket cause ;
-Phuket is Ireland
-Far from Bangkok
-Pattaya more cosmopolit than Phuket
-Pattaya more crowded than Phuket