Athelstan Museum has a wide ranging collection about Malmesbury history.