Below is the link to city level datasets in data.gov
You will see there is only 2600 datasets, so there is still alot of city datasets not accessible from data.gov. In these cases, you will need to go to the city's own open data portal.
You can find a fairly extensive listing (links) to open data portals at the [US] city level the link below. This is a crowd sourced project for cataloging government data portals throughout the world.
You can use the
organization_type filter on the CKAN API, e.g.
But as others have said, Data.gov is only including city datasets that have given Data.gov permission to do so and who are providing their metadata using the Project Open Data (DCAT JSON) metadata standard. And even many of the cities that are providing metadata to data.gov this way are not currently able to include all of their metadata because of a bug in the service they're using to publish their metadata. You can see stats on how many records are being left out because of this bug at http://dropbox.ashlock.us/temp/local/