I'm looking for a database of water features (rivers, streams, lakes and ponds) that groups the water feature by state/province.
There are variety of sources both for the US and Worldwide. Here are a few that I am familiar with:
http://nhd.usgs.gov/ (vector data)
The National Hydrography Dataset (NHD) and Watershed Boundary Dataset (WBD) are used to portray surface water on The National Map. The NHD represents the drainage network with features such as rivers, streams, canals, lakes, ponds, coastline, dams, and streamgages. The WBD represents drainage basins as enclosed areas in eight different size categories. Both datasets represent the real world at a nominal scale of 1:24,000-scale, which means that one inch of The National Map data equals 2,000 feet on the ground. To maintain mapping clarity not all water features are represented and those that are use a moderate level of detail.
The USGS geographic name dataset contains the names of geographic features in the United States. This includes both natural and manmade water features (e.g., canal, dam, beach, channel, lake)
The GNIS contains information about physical and cultural geographic features of all types in the United States, associated areas, and Antarctica, current and historical, but not including roads and highways. The database holds the Federally recognized name of each feature and defines the feature location by state, county, USGS topographic map, and geographic coordinates. Other attributes include names or spellings other than the official name, feature designations, feature classification, historical and descriptive information, and for some categories the geometric boundaries.
The National Geospatial Intelligence Agency (NGA) provides the same data for the countries of the world.
The database is the official repository of foreign place-name decisions approved by the BGN. Geographic Area of Coverage: Worldwide excluding the United States and Antarctica. For names in the U.S. and Antarctica, please visit the United States Geological Survey (USGS) Geographic Names Information System (GNIS) web site.
At my website, I have both the USGS and NGA datasets converted to Linked CSV format: