Update: I recently found this data at GeoNames.org, among many related data sets. The file includes city, county, state, latitude and longitude. It would be a separate exercise to roll it up to MSA or CSA.
Direct link to US Postal codes ZIP file
For many people, the best way to get this data is to buy it. There's a cottage industry of data services that have been providing this to small businesses for years, the costs are pretty low, and the length of the rest of this answer verifies that it's involved, although certainly something you can do if you have some coding and data skills.
If you'd rather create this dataset than buy it, here are things to know:
The Census Bureau states:
The USPS ZIP Codes identify the individual post office or metropolitan area delivery station associated with mailing addresses. USPS ZIP Codes are not areal features but a collection of mail delivery routes.
If the distinction matters, much more information can be found on this page The key things to know: ZCTAs overlap city boundaries and certainly at least a few exist outside of any city boundary. Also, if you're starting from mailing addresses, you will probably have some ZIP codes which aren't in the ZCTA dataset. And, the USPS may have changed ZIP codes since the last time the Census Bureau produced ZCTA definitions. (Some commercial data vendors promise to keep up with those changes as another value-add.)
So anyway, this means that naming is not straightforward. The Census Bureau just uses the 5-digit reference ZIP Code as a label. If you want to assign them names based on the nearest town, etc, you have some work to do.
ZCTAs are areas, not points, but when you ask for the latitude and longitude, you're probably interested in the centroid. You can find this in the Census Bureau's ZIP code tabulation area (ZCTA) gazetteer file, which can be downloaded here. (You want the INTPTLAT and INTPTLONG columns.)
Once you have a ZCTA centroid, you can locate it in a CSA or MSA. GIS software makes this fairly straightforward, with shapefiles provided by the Census's TIGER program.
But your question is further complicated by this: to the Census Bureau, CSA and MSA are different (and overlapping) things.
The bureau doesn't use the term "MSA" but rather combine "metropolitan statistical areas" and "micropolitan statistical areas" into a set called "core based statistical areas" or CBSAs.
To the Census, CSAs are "Combined Statistical Areas" and they are composed of some groups of CBSAs that are economically related. Not all CBSAs are part of any CSA. On Wikipedia, there's a page of "primary statistical areas" that articulates which CBSAs aren't in any CSA, but I haven't seen a GIS shapefile that matches that list. But maybe you can get by with just the CBSAs.
(which searched for 'cats' when I was looking for ZCTAs, which I assume will have zip code data).
