I am writing some neural network training code that I will later use to solve a specific problem. First, though, I want to validate that the code is working and is able to train an RNN as expected.

I'm looking for a simple problem and dataset that I know a simple network can do very well on. Here are some guidelines/criteria:

- The problem should not be too difficult. I want to be able to train networks quickly so that I can fix bugs in the code quickly, without waiting several says for the training to complete. Thus, small networks should be able to solve the problem well.
- Ideally, reasonably small networks would be able to achieve near-perfect accuracy. This gives me a metric I know I should be able to aim for.
- The problem should not be trivial. Several resources online suggest generating simple number sequences, but I worry that my training code might break down when I switch to my real dataset.
- A computer vision based task would be preferable, since my real problem is one of action recognition. I hesitate to use datasets as complex as Sports1M and friends, though, since work on datasets like these is still the cutting edge--hardly a toy problem.

If I were implementing pure CNNs, I would simply use MNIST or CIFAR, but I'm unsure what the analog is for convolutional RNNs. What are some datasets that I might use?