Interior chinking

I recently inherited a log cabin. Someone in the past used something (possibly regular caulk) in the checks and between logs. it is tacky in some places and just falling out in others. It looks like chewing gum! I have been prying it out with a screwdriver but it is tacky enough that it is still sticking to the wood grain. In some places the spacing is too small to get behind the gum to pull it out. I assume I will need to sand this off before applying regular chinking material. I am afraid it won't stick otherwise. Most of the logs are so close together there isn't space between them and this tacky material is just sitting on the surface but is still tacky enough to penetrate the wood grain. Am I on the right track? Any suggestions? Thank you!