I believe that Avery makes a NON PERMANENT/REUSABLE 8.5 x 11 sheet, I don't know what they call it, full page label material, or something like that.
The key is to find the non-permanant, re-usable kind of adhesive backed, not your regular Avery label paper.
Then you can print many things on it, and just cut them out and use as the time comes.
The other idea is to print on regular paper, cut them out, and then use the little strips of Post-It Note Adhesive things with the non-permanent, reusable stuff. They're probably more like post-it note stuff than the Avery thing is, but I'd try them both and see what works for you and your little one!