What would be a good non-permanent adhesive?

Try museum adhesive...you can get it at craft stores and on QVC (qvc.com or 1-800-345-1515). It comes in a jar and is the consistency of putty, but is clear. It's used in museums and gift shops to secure gl collectibles to tables so they can be dusted without handling or breaking. It's clear and comes right off the surface when you want to remove it