I'm looking for height & weight data points, with associated age & sex.
This is the kind of raw data from which Web percentile calculators derive their calculations.
The link below is from the CDC. In addition to the charts, look for the entries on Data Tables. This is the raw data that is used to derive the charts. Its still in a PDF format, so you will have to extract the data from the tables by hand.
World Health Organization (WHO) has similar raw data. It is in a tab delimited format. I've never used it. I suggest reading the instructions that accompany the data.