The English GigaWord dataset by the Linguistics Dataset Consortium contains some 10 million articles alongside an equally large vocabulary set. It's a lot of organised data. If I had to say it less formally, I'd call it every data scientist's wet dream.
However, it costs a non-member $3000 to obtain.
You heard me. Three grands.
The CNN and DailyMail data that DeepMind uses here sizes up to some 300,000 articles which is a mere 30% which would reflect by a similar proportion in terms of learning.
Might be a long-shot but are there any large open datasets which could be used for the purpose of text summarization?