1

I am looking for a large dataset (greater than 100MB), preferably CSV, which is raw and unprocessed, with missing values. It is for a data preprocessing assignment as part of my college course on Big Data Analytics. I have to demonstrate data preprocessing in R or Python which I will work on. However, I am having a hard time finding a good dataset. On Kaggle, the datasets I looked at did not have missing values.

The dataset I have been considering is one from the New Zealand 2018 Census from https://www.stats.govt.nz/large-datasets/csv-files-for-download/. The one I have selected is the Age and sex by ethnic group (grouped total responses), for census usually resident population counts, 2006, 2013, and 2018 Censuses (RC, TA, SA2, DHB). It seems to be a good dataset for this purpose. It has around 34 million rows: Dataset Dimension

However, the problem here is 18M out of those 34M records have a non-numeric value called "..C" for the count column which I'm not exactly sure what it means. I thought about considering them as "missing values" but that is more than 50% of the entire dataset: Dataset NA values

Any help regarding finding a good dataset or further information on the above dataset would help!

Update: The above dataset seems to have a lot of outliers. So I would prefer a recommendation for a new dataset instead.

1
  • 1
    It seems odd that your instructor would make finding an appropriate dataset part of the assignment. It seems like a waste of the students' time without any educational benefit. Just sayin' ...
    – Tom Morris
    Oct 4, 2022 at 18:33

1 Answer 1

2
+50

Japanese Trade Stats This seems to satisfy your request. There are other data sets on that site if your find this particular one doesn’t.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.