Julian McAuley of UCSD compiles a list of datasets for recommender systems. These are a diverse set of labelled text data fortified with additional measures available for various datasets (e.g. network relationships, geospatial data, categorical information, quantitative measures, etc.).
One dataset that has been frequently used in topic modelling and ...
In addition to the answer by @Erwan, I came across
Both are curated lists of text datasets for NLP tasks. A few datasets that may fit your requirement of temporal variation:
Cheng-Caverlee-Lee September 2009 - January 2010 Twitter Scrape: ...
The NPS Chat Corpus is a sample of messages from various online chat platforms tagged by part of speech at the word and passage level. Excerpts from the dataset description:
Description of the NPS Chat Corpus
The NPS Chat Corpus, Release 1.0 consists of 10,567 posts out of
approximately 500,000 posts we have gathered from various online chat
It depends what kind of text you're looking for. I can think of the following three sources for text data with years of publication:
Project Gutenberg is a repository containing 60,000 books in the public domain. It includes a lot of literature classics. I don't think the original publication date is included but for many books it's not to hard to retrieve ...